2023-10-26T05:33:32.082 INFO:root:teuthology version: 0.0.1.dev201+g202b180 2023-10-26T05:33:32.086 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --owner scheduled_pdonnell@teuthology --name pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/orig.config.yaml 2023-10-26T05:33:32.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-26T05:33:32.207 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445 branch: wip-batrick-testing-20231024.144545 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: pdonnell@redhat.com first_in_suite: false job_id: '7438445' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20231024.144545 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 384993681abceb8cb8affea10538d607052db4b4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 384993681abceb8cb8affea10538d607052db4b4 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231024.144545 sha1: 384993681abceb8cb8affea10538d607052db4b4 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7595 sha1: 384993681abceb8cb8affea10538d607052db4b4 sleep_before_teardown: 0 subset: 66/256 suite: fs suite_branch: wip-batrick-testing-20231024.144545 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 384993681abceb8cb8affea10538d607052db4b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtb4fp23k+VlYx1uYJYTCE44p699TI4MG/A7y/cOvNDnxwF8mkBhrRrS9zGYanOJNq36Q0kdVZ9kdcE65mBQbo= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISjMIFjx/GgWg+LUaxrD5urfkmsGBl6ukeyfx7q03DIKhzwI112qkd5pvKzNU+5gyaKcd+pSCE5dcZ++k7lSQQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 202b180cb047e798fb131047314a862593f45403 timestamp: 2023-10-26_05:21:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.770726 2023-10-26T05:33:32.207 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa; will attempt to use it 2023-10-26T05:33:32.208 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa/tasks 2023-10-26T05:33:32.208 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-26T05:33:32.210 INFO:teuthology.task.internal:Checking packages... 2023-10-26T05:33:32.234 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '384993681abceb8cb8affea10538d607052db4b4' 2023-10-26T05:33:32.235 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-26T05:33:32.235 INFO:teuthology.packaging:ref: None 2023-10-26T05:33:32.235 INFO:teuthology.packaging:tag: None 2023-10-26T05:33:32.236 INFO:teuthology.packaging:branch: wip-batrick-testing-20231024.144545 2023-10-26T05:33:32.236 INFO:teuthology.packaging:sha1: 384993681abceb8cb8affea10538d607052db4b4 2023-10-26T05:33:32.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20231024.144545 2023-10-26T05:33:32.466 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6863-g38499368-1jammy 2023-10-26T05:33:32.466 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-26T05:33:32.473 INFO:teuthology.task.internal:no buildpackages task found 2023-10-26T05:33:32.473 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-26T05:33:32.514 INFO:teuthology.task.internal:Saving configuration 2023-10-26T05:33:32.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-26T05:33:32.538 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T05:33:32.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 05:27:47.988476', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGVAdUDinB+RNibIyOMuKJUpp6WYZFaW10gNQzxNYVwA'} 2023-10-26T05:33:32.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 05:27:47.990281', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICFtoZ7+H1Crk7D/QSYzQ0PXtQg/mJxUSShwGysZqEHJ'} 2023-10-26T05:33:32.588 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-26T05:33:32.594 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-26T05:33:32.594 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-26T05:33:32.595 INFO:teuthology.run_tasks:Running task console_log... 2023-10-26T05:33:32.689 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb9d6975b80>, signals=[15]) 2023-10-26T05:33:32.690 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-26T05:33:32.698 INFO:teuthology.task.internal:Opening connections... 2023-10-26T05:33:32.699 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:33:32.701 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:33:32.775 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:33:32.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:33:32.853 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-26T05:33:32.861 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2023-10-26T05:33:32.866 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2023-10-26T05:33:32.867 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2023-10-26T05:33:32.913 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-26T05:33:32.913 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Ubuntu" 2023-10-26T05:33:32.913 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="22.04" 2023-10-26T05:33:32.914 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-26T05:33:32.914 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_CODENAME=jammy 2023-10-26T05:33:32.914 INFO:teuthology.orchestra.run.smithi071.stdout:ID=ubuntu 2023-10-26T05:33:32.914 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE=debian 2023-10-26T05:33:32.914 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-26T05:33:32.915 INFO:teuthology.orchestra.run.smithi071.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-26T05:33:32.915 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-26T05:33:32.915 INFO:teuthology.orchestra.run.smithi071.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-26T05:33:32.915 INFO:teuthology.orchestra.run.smithi071.stdout:UBUNTU_CODENAME=jammy 2023-10-26T05:33:32.916 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2023-10-26T05:33:32.948 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2023-10-26T05:33:32.954 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2023-10-26T05:33:32.954 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2023-10-26T05:33:33.000 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-26T05:33:33.001 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2023-10-26T05:33:33.001 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="22.04" 2023-10-26T05:33:33.001 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-26T05:33:33.001 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=jammy 2023-10-26T05:33:33.002 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2023-10-26T05:33:33.002 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2023-10-26T05:33:33.002 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-26T05:33:33.002 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-26T05:33:33.002 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-26T05:33:33.003 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-26T05:33:33.003 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=jammy 2023-10-26T05:33:33.003 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2023-10-26T05:33:33.031 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-26T05:33:33.041 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-26T05:33:33.063 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-26T05:33:33.064 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2023-10-26T05:33:33.067 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2023-10-26T05:33:33.071 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-26T05:33:33.089 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-26T05:33:33.089 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2023-10-26T05:33:33.115 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2023-10-26T05:33:33.121 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-26T05:33:33.177 INFO:teuthology.run_tasks:Running task kernel... 2023-10-26T05:33:33.192 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-26T05:33:33.193 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-26T05:33:33.193 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T05:33:33.194 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-26T05:33:33.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:33:33.199 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2023-10-26T05:33:33.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:33:33.201 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2023-10-26T05:33:33.246 INFO:teuthology.orchestra.run.smithi071.stdout:5.15.0-56-generic 2023-10-26T05:33:33.246 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.15.0-56-generic 2023-10-26T05:33:33.247 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get clean 2023-10-26T05:33:33.248 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-56-generic 2023-10-26T05:33:33.248 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.15.0-56-generic 2023-10-26T05:33:33.249 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get clean 2023-10-26T05:33:33.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2023-10-26T05:33:33.367 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2023-10-26T05:33:33.508 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-26T05:33:33.514 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-26T05:33:33.532 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-26T05:33:33.532 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-10-26T05:33:33.534 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-26T05:33:33.591 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-10-26T05:33:33.610 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-10-26T05:33:33.641 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-10-26T05:33:33.749 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [896 kB] 2023-10-26T05:33:33.751 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,103 kB] 2023-10-26T05:33:33.825 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [515 kB] 2023-10-26T05:33:33.840 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [353 kB] 2023-10-26T05:33:33.845 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [180 kB] 2023-10-26T05:33:33.848 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-10-26T05:33:33.862 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [515 kB] 2023-10-26T05:33:33.865 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,016 kB] 2023-10-26T05:33:33.868 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [239 kB] 2023-10-26T05:33:33.871 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.0 kB] 2023-10-26T05:33:33.872 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,036 kB] 2023-10-26T05:33:33.879 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.0 kB] 2023-10-26T05:33:33.880 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [164 kB] 2023-10-26T05:33:33.880 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [536 B] 2023-10-26T05:33:33.880 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [562 kB] 2023-10-26T05:33:33.884 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,103 kB] 2023-10-26T05:33:33.886 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [793 kB] 2023-10-26T05:33:33.892 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.4 kB] 2023-10-26T05:33:33.893 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [167 kB] 2023-10-26T05:33:33.894 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [145 kB] 2023-10-26T05:33:33.896 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-10-26T05:33:33.897 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-10-26T05:33:33.898 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-10-26T05:33:33.899 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [536 B] 2023-10-26T05:33:33.899 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [995 kB] 2023-10-26T05:33:33.902 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-10-26T05:33:33.902 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-10-26T05:33:33.911 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [239 kB] 2023-10-26T05:33:33.915 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.0 kB] 2023-10-26T05:33:33.915 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.4 kB] 2023-10-26T05:33:33.916 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,036 kB] 2023-10-26T05:33:33.926 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [167 kB] 2023-10-26T05:33:33.929 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [536 B] 2023-10-26T05:33:33.930 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [995 kB] 2023-10-26T05:33:33.935 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [660 kB] 2023-10-26T05:33:33.936 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [660 kB] 2023-10-26T05:33:33.941 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [218 kB] 2023-10-26T05:33:33.942 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.0 kB] 2023-10-26T05:33:33.943 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [218 kB] 2023-10-26T05:33:33.946 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-10-26T05:33:33.946 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-10-26T05:33:33.948 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.0 kB] 2023-10-26T05:33:33.948 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-10-26T05:33:33.948 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-10-26T05:33:33.948 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-10-26T05:33:33.949 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-10-26T05:33:33.949 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-10-26T05:33:33.949 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-10-26T05:33:33.950 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-10-26T05:33:33.950 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-10-26T05:33:33.950 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [64.2 kB] 2023-10-26T05:33:33.951 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [56.3 kB] 2023-10-26T05:33:33.957 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-10-26T05:33:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-10-26T05:33:33.963 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-10-26T05:33:33.964 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-10-26T05:33:33.964 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.9 kB] 2023-10-26T05:33:33.977 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-10-26T05:33:33.977 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.9 kB] 2023-10-26T05:33:33.978 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.8 kB] 2023-10-26T05:33:33.978 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-10-26T05:33:33.979 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-10-26T05:33:33.980 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-10-26T05:33:33.980 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-10-26T05:33:34.082 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [353 kB] 2023-10-26T05:33:34.345 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [896 kB] 2023-10-26T05:33:34.491 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [180 kB] 2023-10-26T05:33:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-10-26T05:33:34.511 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.0 kB] 2023-10-26T05:33:34.513 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,016 kB] 2023-10-26T05:33:34.579 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [164 kB] 2023-10-26T05:33:34.588 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [536 B] 2023-10-26T05:33:34.588 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [562 kB] 2023-10-26T05:33:34.613 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [793 kB] 2023-10-26T05:33:34.803 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [145 kB] 2023-10-26T05:33:35.186 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-10-26T05:33:35.193 INFO:teuthology.orchestra.run.smithi071.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-10-26T05:33:35.209 INFO:teuthology.orchestra.run.smithi071.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-10-26T05:33:35.209 INFO:teuthology.orchestra.run.smithi071.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-10-26T05:33:35.212 INFO:teuthology.orchestra.run.smithi071.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-10-26T05:33:39.146 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 9,806 kB in 1s (6,644 kB/s) 2023-10-26T05:33:39.564 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 9,806 kB in 2s (5,514 kB/s) 2023-10-26T05:33:40.319 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:33:40.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-26T05:33:40.396 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:33:40.582 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-26T05:33:40.582 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-26T05:33:40.743 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:33:40.743 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:33:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:33:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:33:40.744 INFO:teuthology.orchestra.run.smithi086.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-26T05:33:40.745 INFO:teuthology.orchestra.run.smithi086.stdout: python2.7-minimal 2023-10-26T05:33:40.745 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:33:40.745 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2023-10-26T05:33:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: linux-generic linux-headers-5.15.0-87 linux-headers-5.15.0-87-generic 2023-10-26T05:33:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: linux-headers-generic linux-image-5.15.0-87-generic 2023-10-26T05:33:40.746 INFO:teuthology.orchestra.run.smithi086.stdout: linux-modules-5.15.0-87-generic linux-modules-extra-5.15.0-87-generic 2023-10-26T05:33:40.747 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2023-10-26T05:33:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-10-26T05:33:40.747 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2023-10-26T05:33:40.747 INFO:teuthology.orchestra.run.smithi086.stdout: thermald 2023-10-26T05:33:40.790 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2023-10-26T05:33:40.790 INFO:teuthology.orchestra.run.smithi086.stdout: linux-headers-5.15.0-87 linux-headers-5.15.0-87-generic 2023-10-26T05:33:40.790 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.15.0-87-generic linux-modules-5.15.0-87-generic 2023-10-26T05:33:40.791 INFO:teuthology.orchestra.run.smithi086.stdout: linux-modules-extra-5.15.0-87-generic 2023-10-26T05:33:40.791 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2023-10-26T05:33:40.792 INFO:teuthology.orchestra.run.smithi086.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-26T05:33:40.962 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:33:40.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-26T05:33:41.030 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:33:41.193 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2023-10-26T05:33:41.194 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2023-10-26T05:33:41.335 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:33:41.335 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:33:41.335 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:33:41.336 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:33:41.336 INFO:teuthology.orchestra.run.smithi071.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-26T05:33:41.336 INFO:teuthology.orchestra.run.smithi071.stdout: python2.7-minimal 2023-10-26T05:33:41.337 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:33:41.337 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2023-10-26T05:33:41.337 INFO:teuthology.orchestra.run.smithi071.stdout: linux-generic linux-headers-5.15.0-87 linux-headers-5.15.0-87-generic 2023-10-26T05:33:41.338 INFO:teuthology.orchestra.run.smithi071.stdout: linux-headers-generic linux-image-5.15.0-87-generic 2023-10-26T05:33:41.338 INFO:teuthology.orchestra.run.smithi071.stdout: linux-modules-5.15.0-87-generic linux-modules-extra-5.15.0-87-generic 2023-10-26T05:33:41.338 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2023-10-26T05:33:41.339 INFO:teuthology.orchestra.run.smithi071.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-10-26T05:33:41.339 INFO:teuthology.orchestra.run.smithi071.stdout:Recommended packages: 2023-10-26T05:33:41.339 INFO:teuthology.orchestra.run.smithi071.stdout: thermald 2023-10-26T05:33:41.379 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2023-10-26T05:33:41.379 INFO:teuthology.orchestra.run.smithi071.stdout: linux-headers-5.15.0-87 linux-headers-5.15.0-87-generic 2023-10-26T05:33:41.380 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.15.0-87-generic linux-modules-5.15.0-87-generic 2023-10-26T05:33:41.380 INFO:teuthology.orchestra.run.smithi071.stdout: linux-modules-extra-5.15.0-87-generic 2023-10-26T05:33:41.380 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages will be upgraded: 2023-10-26T05:33:41.381 INFO:teuthology.orchestra.run.smithi071.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-26T05:33:41.435 INFO:teuthology.orchestra.run.smithi071.stdout:3 upgraded, 5 newly installed, 0 to remove and 269 not upgraded. 2023-10-26T05:33:41.435 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 113 MB of archives. 2023-10-26T05:33:41.435 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 582 MB of additional disk space will be used. 2023-10-26T05:33:41.436 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-87-generic amd64 5.15.0-87.97 [22.4 MB] 2023-10-26T05:33:41.891 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-87-generic amd64 5.15.0-87.97 [11.5 MB] 2023-10-26T05:33:41.992 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-87-generic amd64 5.15.0-87.97 [64.0 MB] 2023-10-26T05:33:42.213 INFO:teuthology.orchestra.run.smithi086.stdout:3 upgraded, 5 newly installed, 0 to remove and 269 not upgraded. 2023-10-26T05:33:42.213 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 113 MB of archives. 2023-10-26T05:33:42.213 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 582 MB of additional disk space will be used. 2023-10-26T05:33:42.214 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-87-generic amd64 5.15.0-87.97 [22.4 MB] 2023-10-26T05:33:42.889 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-87-generic amd64 5.15.0-87.97 [11.5 MB] 2023-10-26T05:33:43.014 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.87.84 [1,698 B] 2023-10-26T05:33:43.014 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.87.84 [2,482 B] 2023-10-26T05:33:43.015 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-87-generic amd64 5.15.0-87.97 [64.0 MB] 2023-10-26T05:33:43.015 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-87 all 5.15.0-87.97 [12.3 MB] 2023-10-26T05:33:43.914 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-87-generic amd64 5.15.0-87.97 [2,888 kB] 2023-10-26T05:33:43.942 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.87.84 [1,698 B] 2023-10-26T05:33:43.943 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.87.84 [2,482 B] 2023-10-26T05:33:43.943 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-87 all 5.15.0-87.97 [12.3 MB] 2023-10-26T05:33:44.015 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.87.84 [2,322 B] 2023-10-26T05:33:44.074 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-87-generic amd64 5.15.0-87.97 [2,888 kB] 2023-10-26T05:33:44.096 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.87.84 [2,322 B] 2023-10-26T05:33:44.416 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 113 MB in 3s (43.0 MB/s) 2023-10-26T05:33:44.489 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 113 MB in 3s (34.3 MB/s) 2023-10-26T05:33:44.603 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-modules-5.15.0-87-generic. 2023-10-26T05:33:44.648 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-modules-5.15.0-87-generic. 2023-10-26T05:33:46.640 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-10-26T05:33:46.644 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../0-linux-modules-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:33:46.648 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-10-26T05:33:46.652 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../0-linux-modules-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:33:46.736 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-modules-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:33:46.742 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-modules-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:33:48.979 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-image-5.15.0-87-generic. 2023-10-26T05:33:49.000 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../1-linux-image-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:33:49.059 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-image-5.15.0-87-generic. 2023-10-26T05:33:49.081 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../1-linux-image-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:33:49.090 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:33:49.167 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-image-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:33:49.549 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-87-generic. 2023-10-26T05:33:49.569 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:33:49.607 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-modules-extra-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:33:49.620 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-87-generic. 2023-10-26T05:33:49.641 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:33:49.679 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-modules-extra-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:33:56.257 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../3-linux-generic_5.15.0.87.84_amd64.deb ... 2023-10-26T05:33:56.358 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-generic (5.15.0.87.84) over (5.15.0.56.54) ... 2023-10-26T05:33:56.633 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.87.84_amd64.deb ... 2023-10-26T05:33:56.677 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../3-linux-generic_5.15.0.87.84_amd64.deb ... 2023-10-26T05:33:56.729 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-generic (5.15.0.87.84) over (5.15.0.56.54) ... 2023-10-26T05:33:56.805 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-generic (5.15.0.87.84) over (5.15.0.56.54) ... 2023-10-26T05:33:56.998 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-headers-5.15.0-87. 2023-10-26T05:33:57.009 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../5-linux-headers-5.15.0-87_5.15.0-87.97_all.deb ... 2023-10-26T05:33:57.048 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-5.15.0-87 (5.15.0-87.97) ... 2023-10-26T05:33:57.154 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.87.84_amd64.deb ... 2023-10-26T05:33:57.275 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-image-generic (5.15.0.87.84) over (5.15.0.56.54) ... 2023-10-26T05:33:57.570 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-headers-5.15.0-87. 2023-10-26T05:33:57.586 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../5-linux-headers-5.15.0-87_5.15.0-87.97_all.deb ... 2023-10-26T05:33:57.619 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-5.15.0-87 (5.15.0-87.97) ... 2023-10-26T05:34:02.467 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-headers-5.15.0-87-generic. 2023-10-26T05:34:02.504 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../6-linux-headers-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:34:02.551 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:03.925 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package linux-headers-5.15.0-87-generic. 2023-10-26T05:34:03.963 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../6-linux-headers-5.15.0-87-generic_5.15.0-87.97_amd64.deb ... 2023-10-26T05:34:04.000 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:05.614 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.87.84_amd64.deb ... 2023-10-26T05:34:05.736 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-generic (5.15.0.87.84) over (5.15.0.56.54) ... 2023-10-26T05:34:06.076 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-5.15.0-87 (5.15.0-87.97) ... 2023-10-26T05:34:06.198 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:06.325 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-generic (5.15.0.87.84) ... 2023-10-26T05:34:06.452 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-modules-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:06.785 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.87.84_amd64.deb ... 2023-10-26T05:34:06.893 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking linux-headers-generic (5.15.0.87.84) over (5.15.0.56.54) ... 2023-10-26T05:34:07.226 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-5.15.0-87 (5.15.0-87.97) ... 2023-10-26T05:34:07.348 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:07.490 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-headers-generic (5.15.0.87.84) ... 2023-10-26T05:34:07.618 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-modules-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:07.888 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:09.369 INFO:teuthology.orchestra.run.smithi086.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-10-26T05:34:09.369 INFO:teuthology.orchestra.run.smithi086.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-10-26T05:34:09.370 INFO:teuthology.orchestra.run.smithi086.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-87-generic 2023-10-26T05:34:09.370 INFO:teuthology.orchestra.run.smithi086.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-image-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:09.640 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-modules-extra-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:10.973 INFO:teuthology.orchestra.run.smithi071.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-10-26T05:34:10.973 INFO:teuthology.orchestra.run.smithi071.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-10-26T05:34:10.973 INFO:teuthology.orchestra.run.smithi071.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-87-generic 2023-10-26T05:34:10.973 INFO:teuthology.orchestra.run.smithi071.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:11.032 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-generic (5.15.0.87.84) ... 2023-10-26T05:34:11.130 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-modules-extra-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:11.158 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-generic (5.15.0.87.84) ... 2023-10-26T05:34:11.285 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for linux-image-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:11.340 INFO:teuthology.orchestra.run.smithi086.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-26T05:34:11.341 INFO:teuthology.orchestra.run.smithi086.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:12.523 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-image-generic (5.15.0.87.84) ... 2023-10-26T05:34:12.649 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up linux-generic (5.15.0.87.84) ... 2023-10-26T05:34:12.798 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for linux-image-5.15.0-87-generic (5.15.0-87.97) ... 2023-10-26T05:34:12.858 INFO:teuthology.orchestra.run.smithi071.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-26T05:34:12.858 INFO:teuthology.orchestra.run.smithi071.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:25.709 INFO:teuthology.orchestra.run.smithi086.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-26T05:34:25.710 INFO:teuthology.orchestra.run.smithi086.stdout:Sourcing file `/etc/default/grub' 2023-10-26T05:34:25.727 INFO:teuthology.orchestra.run.smithi086.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-26T05:34:25.755 INFO:teuthology.orchestra.run.smithi086.stdout:Generating grub configuration file ... 2023-10-26T05:34:26.003 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.15.0-87-generic 2023-10-26T05:34:26.023 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:26.319 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-26T05:34:26.324 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-26T05:34:26.396 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-26T05:34:26.401 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-26T05:34:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-26T05:34:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-10-26T05:34:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-26T05:34:26.615 INFO:teuthology.orchestra.run.smithi086.stdout:done 2023-10-26T05:34:27.102 DEBUG:teuthology.orchestra.run.smithi086:> dpkg -s linux-image-generic 2023-10-26T05:34:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:Package: linux-image-generic 2023-10-26T05:34:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:Status: install ok installed 2023-10-26T05:34:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:Priority: optional 2023-10-26T05:34:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:Section: kernel 2023-10-26T05:34:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:Installed-Size: 20 2023-10-26T05:34:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:Maintainer: Ubuntu Kernel Team 2023-10-26T05:34:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: amd64 2023-10-26T05:34:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:Source: linux-meta 2023-10-26T05:34:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:Version: 5.15.0.87.84 2023-10-26T05:34:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-87), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-26T05:34:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:Depends: linux-image-5.15.0-87-generic, linux-modules-extra-5.15.0-87-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-26T05:34:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:Recommends: thermald 2023-10-26T05:34:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:Description: Generic Linux kernel image 2023-10-26T05:34:27.128 INFO:teuthology.orchestra.run.smithi086.stdout: This package will always depend on the latest generic kernel image 2023-10-26T05:34:27.128 INFO:teuthology.orchestra.run.smithi086.stdout: available. 2023-10-26T05:34:27.128 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-87-generic 2023-10-26T05:34:27.129 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-26T05:34:27.129 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-26T05:34:27.129 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-26T05:34:27.129 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=None, version=distro) 2023-10-26T05:34:27.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get clean 2023-10-26T05:34:27.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2023-10-26T05:34:27.424 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-26T05:34:27.460 INFO:teuthology.orchestra.run.smithi071.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-26T05:34:27.461 INFO:teuthology.orchestra.run.smithi071.stdout:Sourcing file `/etc/default/grub' 2023-10-26T05:34:27.480 INFO:teuthology.orchestra.run.smithi071.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-26T05:34:27.506 INFO:teuthology.orchestra.run.smithi071.stdout:Generating grub configuration file ... 2023-10-26T05:34:27.569 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-26T05:34:27.652 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-26T05:34:27.814 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.15.0-87-generic 2023-10-26T05:34:27.835 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:28.003 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-26T05:34:28.155 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-26T05:34:28.160 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-26T05:34:28.194 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 119 kB in 1s (163 kB/s) 2023-10-26T05:34:28.231 INFO:teuthology.orchestra.run.smithi071.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-26T05:34:28.237 INFO:teuthology.orchestra.run.smithi071.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-26T05:34:28.432 INFO:teuthology.orchestra.run.smithi071.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-26T05:34:28.432 INFO:teuthology.orchestra.run.smithi071.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-10-26T05:34:28.433 INFO:teuthology.orchestra.run.smithi071.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-26T05:34:28.468 INFO:teuthology.orchestra.run.smithi071.stdout:done 2023-10-26T05:34:28.948 DEBUG:teuthology.orchestra.run.smithi071:> dpkg -s linux-image-generic 2023-10-26T05:34:28.977 INFO:teuthology.orchestra.run.smithi071.stdout:Package: linux-image-generic 2023-10-26T05:34:28.977 INFO:teuthology.orchestra.run.smithi071.stdout:Status: install ok installed 2023-10-26T05:34:28.977 INFO:teuthology.orchestra.run.smithi071.stdout:Priority: optional 2023-10-26T05:34:28.978 INFO:teuthology.orchestra.run.smithi071.stdout:Section: kernel 2023-10-26T05:34:28.978 INFO:teuthology.orchestra.run.smithi071.stdout:Installed-Size: 20 2023-10-26T05:34:28.978 INFO:teuthology.orchestra.run.smithi071.stdout:Maintainer: Ubuntu Kernel Team 2023-10-26T05:34:28.978 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: amd64 2023-10-26T05:34:28.978 INFO:teuthology.orchestra.run.smithi071.stdout:Source: linux-meta 2023-10-26T05:34:28.979 INFO:teuthology.orchestra.run.smithi071.stdout:Version: 5.15.0.87.84 2023-10-26T05:34:28.979 INFO:teuthology.orchestra.run.smithi071.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-87), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-26T05:34:28.979 INFO:teuthology.orchestra.run.smithi071.stdout:Depends: linux-image-5.15.0-87-generic, linux-modules-extra-5.15.0-87-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-26T05:34:28.979 INFO:teuthology.orchestra.run.smithi071.stdout:Recommends: thermald 2023-10-26T05:34:28.980 INFO:teuthology.orchestra.run.smithi071.stdout:Description: Generic Linux kernel image 2023-10-26T05:34:28.980 INFO:teuthology.orchestra.run.smithi071.stdout: This package will always depend on the latest generic kernel image 2023-10-26T05:34:28.980 INFO:teuthology.orchestra.run.smithi071.stdout: available. 2023-10-26T05:34:28.980 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-87-generic 2023-10-26T05:34:28.980 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-26T05:34:28.981 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-26T05:34:28.981 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-26T05:34:28.981 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2023-10-26T05:34:28.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get clean 2023-10-26T05:34:29.078 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:34:29.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-26T05:34:29.147 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:34:29.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2023-10-26T05:34:29.312 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-26T05:34:29.314 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-26T05:34:29.409 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-26T05:34:29.456 INFO:teuthology.orchestra.run.smithi086.stdout:linux-image-generic is already the newest version (5.15.0.87.84). 2023-10-26T05:34:29.456 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:34:29.456 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:34:29.457 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:34:29.457 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:34:29.457 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-26T05:34:29.458 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-26T05:34:29.458 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:34:29.482 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-26T05:34:29.492 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 0 newly installed, 0 to remove and 269 not upgraded. 2023-10-26T05:34:29.495 DEBUG:teuthology.orchestra.run.smithi086:> dpkg -s linux-image-generic 2023-10-26T05:34:29.550 INFO:teuthology.orchestra.run.smithi086.stdout:Package: linux-image-generic 2023-10-26T05:34:29.551 INFO:teuthology.orchestra.run.smithi086.stdout:Status: install ok installed 2023-10-26T05:34:29.551 INFO:teuthology.orchestra.run.smithi086.stdout:Priority: optional 2023-10-26T05:34:29.551 INFO:teuthology.orchestra.run.smithi086.stdout:Section: kernel 2023-10-26T05:34:29.551 INFO:teuthology.orchestra.run.smithi086.stdout:Installed-Size: 20 2023-10-26T05:34:29.551 INFO:teuthology.orchestra.run.smithi086.stdout:Maintainer: Ubuntu Kernel Team 2023-10-26T05:34:29.551 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: amd64 2023-10-26T05:34:29.552 INFO:teuthology.orchestra.run.smithi086.stdout:Source: linux-meta 2023-10-26T05:34:29.552 INFO:teuthology.orchestra.run.smithi086.stdout:Version: 5.15.0.87.84 2023-10-26T05:34:29.552 INFO:teuthology.orchestra.run.smithi086.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-87), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-26T05:34:29.552 INFO:teuthology.orchestra.run.smithi086.stdout:Depends: linux-image-5.15.0-87-generic, linux-modules-extra-5.15.0-87-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-26T05:34:29.552 INFO:teuthology.orchestra.run.smithi086.stdout:Recommends: thermald 2023-10-26T05:34:29.553 INFO:teuthology.orchestra.run.smithi086.stdout:Description: Generic Linux kernel image 2023-10-26T05:34:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: This package will always depend on the latest generic kernel image 2023-10-26T05:34:29.553 INFO:teuthology.orchestra.run.smithi086.stdout: available. 2023-10-26T05:34:29.553 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2023-10-26T05:34:29.595 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.nDUnar0u4T 2023-10-26T05:34:29.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub/grub.cfg /tmp/tmp.nDUnar0u4T 2023-10-26T05:34:29.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.nDUnar0u4T 2023-10-26T05:34:29.764 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.nDUnar0u4T is 10KB 2023-10-26T05:34:29.776 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.nDUnar0u4T 2023-10-26T05:34:29.782 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-26T05:34:29.835 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:34:29.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-26T05:34:29.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-26T05:34:29.840 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-26T05:34:29.903 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-87-generic 2023-10-26T05:34:29.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo update-grub 2023-10-26T05:34:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2023-10-26T05:34:31.051 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-26T05:34:31.053 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2023-10-26T05:34:31.244 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-87-generic 2023-10-26T05:34:31.250 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-26T05:34:31.485 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-26T05:34:31.525 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-26T05:34:31.529 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-26T05:34:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-26T05:34:31.644 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-26T05:34:31.645 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-26T05:34:31.658 INFO:teuthology.orchestra.run.smithi086.stderr:done 2023-10-26T05:34:31.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2023-10-26T05:34:32.119 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-26T05:34:32.319 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 119 kB in 3s (39.8 kB/s) 2023-10-26T05:34:33.191 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:34:33.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-26T05:34:33.269 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:34:33.457 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2023-10-26T05:34:33.458 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2023-10-26T05:34:33.620 INFO:teuthology.orchestra.run.smithi071.stdout:linux-image-generic is already the newest version (5.15.0.87.84). 2023-10-26T05:34:33.620 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:34:33.620 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:34:33.621 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:34:33.621 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:34:33.621 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-26T05:34:33.621 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-26T05:34:33.622 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:34:33.662 INFO:teuthology.orchestra.run.smithi071.stdout:0 upgraded, 0 newly installed, 0 to remove and 269 not upgraded. 2023-10-26T05:34:33.663 DEBUG:teuthology.orchestra.run.smithi071:> dpkg -s linux-image-generic 2023-10-26T05:34:33.679 INFO:teuthology.orchestra.run.smithi071.stdout:Package: linux-image-generic 2023-10-26T05:34:33.679 INFO:teuthology.orchestra.run.smithi071.stdout:Status: install ok installed 2023-10-26T05:34:33.679 INFO:teuthology.orchestra.run.smithi071.stdout:Priority: optional 2023-10-26T05:34:33.679 INFO:teuthology.orchestra.run.smithi071.stdout:Section: kernel 2023-10-26T05:34:33.680 INFO:teuthology.orchestra.run.smithi071.stdout:Installed-Size: 20 2023-10-26T05:34:33.680 INFO:teuthology.orchestra.run.smithi071.stdout:Maintainer: Ubuntu Kernel Team 2023-10-26T05:34:33.680 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: amd64 2023-10-26T05:34:33.680 INFO:teuthology.orchestra.run.smithi071.stdout:Source: linux-meta 2023-10-26T05:34:33.680 INFO:teuthology.orchestra.run.smithi071.stdout:Version: 5.15.0.87.84 2023-10-26T05:34:33.681 INFO:teuthology.orchestra.run.smithi071.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-87), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-26T05:34:33.681 INFO:teuthology.orchestra.run.smithi071.stdout:Depends: linux-image-5.15.0-87-generic, linux-modules-extra-5.15.0-87-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-26T05:34:33.681 INFO:teuthology.orchestra.run.smithi071.stdout:Recommends: thermald 2023-10-26T05:34:33.681 INFO:teuthology.orchestra.run.smithi071.stdout:Description: Generic Linux kernel image 2023-10-26T05:34:33.681 INFO:teuthology.orchestra.run.smithi071.stdout: This package will always depend on the latest generic kernel image 2023-10-26T05:34:33.682 INFO:teuthology.orchestra.run.smithi071.stdout: available. 2023-10-26T05:34:33.682 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2023-10-26T05:34:33.724 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.KfUTBl5DxE 2023-10-26T05:34:33.725 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub/grub.cfg /tmp/tmp.KfUTBl5DxE 2023-10-26T05:34:33.781 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.KfUTBl5DxE 2023-10-26T05:34:33.906 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.KfUTBl5DxE is 10KB 2023-10-26T05:34:33.919 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.KfUTBl5DxE 2023-10-26T05:34:33.924 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-26T05:34:33.980 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:34:33.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-26T05:34:33.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-26T05:34:34.088 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-87-generic 2023-10-26T05:34:34.088 DEBUG:teuthology.orchestra.run.smithi071:> sudo update-grub 2023-10-26T05:34:35.524 INFO:teuthology.orchestra.run.smithi071.stderr:Sourcing file `/etc/default/grub' 2023-10-26T05:34:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-26T05:34:35.528 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2023-10-26T05:34:35.735 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.15.0-87-generic 2023-10-26T05:34:35.741 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.15.0-87-generic 2023-10-26T05:34:36.000 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-26T05:34:36.005 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-26T05:34:36.046 INFO:teuthology.orchestra.run.smithi071.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-26T05:34:36.052 INFO:teuthology.orchestra.run.smithi071.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-26T05:34:36.170 INFO:teuthology.orchestra.run.smithi071.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-26T05:34:36.171 INFO:teuthology.orchestra.run.smithi071.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-26T05:34:36.171 INFO:teuthology.orchestra.run.smithi071.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-26T05:34:36.189 INFO:teuthology.orchestra.run.smithi071.stderr:done 2023-10-26T05:34:36.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2023-10-26T05:35:01.694 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-26T05:35:01.695 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:35:01.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:35:06.247 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-26T05:35:06.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:35:06.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:35:20.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-10-26T05:35:24.798 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-26T05:35:29.194 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:35:29.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:35:32.250 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-10-26T05:35:33.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:35:33.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:35:36.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-26T05:35:44.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:35:44.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:35:48.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:35:48.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:36:02.682 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-10-26T05:36:07.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2023-10-26T05:36:17.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:36:17.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:36:17.891 DEBUG:teuthology.orchestra.run.smithi086:> true 2023-10-26T05:36:18.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:36:18.467 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-87-generic"... 2023-10-26T05:36:18.468 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2023-10-26T05:36:18.513 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-87-generic 2023-10-26T05:36:18.514 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-87-generic vs 5.15.0-87-generic 2023-10-26T05:36:18.514 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-26T05:36:18.514 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-26T05:36:19.515 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-26T05:36:19.515 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-26T05:36:19.605 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2023-10-26T05:36:19.627 DEBUG:teuthology.parallel:result is None 2023-10-26T05:36:22.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:36:22.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:36:22.641 DEBUG:teuthology.orchestra.run.smithi071:> true 2023-10-26T05:36:23.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:36:23.898 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-87-generic"... 2023-10-26T05:36:23.899 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2023-10-26T05:36:23.949 INFO:teuthology.orchestra.run.smithi071.stdout:5.15.0-87-generic 2023-10-26T05:36:23.950 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-87-generic vs 5.15.0-87-generic 2023-10-26T05:36:23.950 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-26T05:36:23.950 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-26T05:36:24.952 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-26T05:36:24.952 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-26T05:36:25.067 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2023-10-26T05:36:25.088 DEBUG:teuthology.parallel:result is None 2023-10-26T05:36:25.089 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-26T05:36:25.099 INFO:teuthology.task.internal:Creating test directory... 2023-10-26T05:36:25.100 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T05:36:25.103 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-26T05:36:25.109 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-26T05:36:25.132 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-26T05:36:25.210 INFO:teuthology.task.internal:Creating archive directory... 2023-10-26T05:36:25.213 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-26T05:36:25.243 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-26T05:36:25.249 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-26T05:36:25.250 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T05:36:25.271 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-26T05:36:25.288 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T05:36:25.296 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T05:36:25.298 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T05:36:25.305 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-26T05:36:25.306 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-26T05:36:25.313 INFO:teuthology.task.internal:Configuring sudo... 2023-10-26T05:36:25.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T05:36:25.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-26T05:36:25.361 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-26T05:36:25.370 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-26T05:36:25.371 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T05:36:25.403 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-26T05:36:25.409 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T05:36:25.455 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T05:36:25.503 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:36:25.503 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T05:36:25.573 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-26T05:36:25.580 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-26T05:36:25.626 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:36:25.627 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-26T05:36:25.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2023-10-26T05:36:25.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2023-10-26T05:36:25.812 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-26T05:36:25.820 INFO:teuthology.task.internal:Starting timer... 2023-10-26T05:36:25.820 INFO:teuthology.run_tasks:Running task pcp... 2023-10-26T05:36:25.860 INFO:teuthology.run_tasks:Running task selinux... 2023-10-26T05:36:25.892 DEBUG:teuthology.task.selinux:Excluding smithi071: OS 'ubuntu' does not support SELinux 2023-10-26T05:36:25.892 DEBUG:teuthology.task.selinux:Excluding smithi086: OS 'ubuntu' does not support SELinux 2023-10-26T05:36:25.893 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-26T05:36:25.893 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-26T05:36:25.893 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-26T05:36:25.894 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-26T05:36:25.926 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-26T05:36:25.926 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-26T05:36:25.942 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-26T05:36:25.943 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-26T05:44:00.744 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2023-10-26T05:44:00.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:44:00.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:44:00.822 DEBUG:teuthology.orchestra.run.smithi071:> true 2023-10-26T05:44:00.930 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2023-10-26T05:44:00.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:44:00.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-26T05:44:01.007 DEBUG:teuthology.orchestra.run.smithi086:> true 2023-10-26T05:44:01.118 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-26T05:44:01.118 INFO:teuthology.run_tasks:Running task clock... 2023-10-26T05:44:01.130 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-26T05:44:01.131 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T05:44:01.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T05:44:01.133 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-26T05:44:01.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T05:44:01.158 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-26T05:44:01.159 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Command line: ntpd -gq 2023-10-26T05:44:01.159 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: ---------------------------------------------------- 2023-10-26T05:44:01.159 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: ntp-4 is maintained by Network Time Foundation, 2023-10-26T05:44:01.159 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-26T05:44:01.159 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: corporation. Support and training for ntp-4 are 2023-10-26T05:44:01.160 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: available at https://www.nwtime.org/support 2023-10-26T05:44:01.160 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: ---------------------------------------------------- 2023-10-26T05:44:01.161 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: proto: precision = 0.063 usec (-24) 2023-10-26T05:44:01.161 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: basedate set to 2022-02-04 2023-10-26T05:44:01.161 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: gps base set to 2022-02-06 (week 2196) 2023-10-26T05:44:01.161 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-26T05:44:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-26T05:44:01.162 INFO:teuthology.orchestra.run.smithi071.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-26T05:44:01.163 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: restrict ::: KOD does nothing without LIMITED. 2023-10-26T05:44:01.163 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listen and drop on 0 v6wildcard [::]:123 2023-10-26T05:44:01.163 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-26T05:44:01.164 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listen normally on 2 lo 127.0.0.1:123 2023-10-26T05:44:01.164 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listen normally on 3 enp3s0f1 172.21.15.71:123 2023-10-26T05:44:01.164 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listen normally on 4 lo [::1]:123 2023-10-26T05:44:01.164 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d765%5]:123 2023-10-26T05:44:01.164 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:01 ntpd[27361]: Listening on routing socket on fd #22 for interface updates 2023-10-26T05:44:01.192 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-26T05:44:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Command line: ntpd -gq 2023-10-26T05:44:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: ---------------------------------------------------- 2023-10-26T05:44:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: ntp-4 is maintained by Network Time Foundation, 2023-10-26T05:44:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-26T05:44:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: corporation. Support and training for ntp-4 are 2023-10-26T05:44:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: available at https://www.nwtime.org/support 2023-10-26T05:44:01.194 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: ---------------------------------------------------- 2023-10-26T05:44:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: proto: precision = 0.046 usec (-24) 2023-10-26T05:44:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: basedate set to 2022-02-04 2023-10-26T05:44:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: gps base set to 2022-02-06 (week 2196) 2023-10-26T05:44:01.195 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-26T05:44:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-26T05:44:01.196 INFO:teuthology.orchestra.run.smithi086.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-26T05:44:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: restrict ::: KOD does nothing without LIMITED. 2023-10-26T05:44:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listen and drop on 0 v6wildcard [::]:123 2023-10-26T05:44:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-26T05:44:01.197 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listen normally on 2 lo 127.0.0.1:123 2023-10-26T05:44:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listen normally on 3 enp3s0f1 172.21.15.86:123 2023-10-26T05:44:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listen normally on 4 lo [::1]:123 2023-10-26T05:44:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d809%5]:123 2023-10-26T05:44:01.198 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:01 ntpd[27377]: Listening on routing socket on fd #22 for interface updates 2023-10-26T05:44:02.159 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:02.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:02.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:02.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:02.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:02.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:02.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:02.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:02 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:02.193 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:02 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:02.193 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:02 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:02.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:02 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:02.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:02 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:02.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:02 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:02.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:02 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:03.193 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:03.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:03.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:03.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:03.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:03 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:04.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:04.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:04.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:04.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:04 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:04 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:05.193 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:05 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:05 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:05 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:05 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:05 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:05.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:05 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:06.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:06.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:06.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:06.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:06.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.164 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:06.164 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:06 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:06.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:06 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:07 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:07 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:07 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:07 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:07 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:07 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:08.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.160 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.161 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:08.162 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:08.162 INFO:teuthology.orchestra.run.smithi071.stdout:26 Oct 05:44:08 ntpd[27361]: ntpd: time slew +0.000967 s 2023-10-26T05:44:08.162 INFO:teuthology.orchestra.run.smithi071.stdout:ntpd: time slew +0.000967s 2023-10-26T05:44:08.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-26T05:44:08.163 INFO:teuthology.orchestra.run.smithi071.stderr:26 Oct 05:44:08 ntpd[27361]: can't open /var/log/ntpstats/loopstats.20231026: Permission denied 2023-10-26T05:44:08.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.194 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:08.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:08.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-26T05:44:08.195 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/rawstats.20231026: Permission denied 2023-10-26T05:44:08.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-26T05:44:08.196 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/peerstats.20231026: Permission denied 2023-10-26T05:44:08.196 INFO:teuthology.orchestra.run.smithi086.stdout:26 Oct 05:44:08 ntpd[27377]: ntpd: time slew -0.000206 s 2023-10-26T05:44:08.196 INFO:teuthology.orchestra.run.smithi086.stdout:ntpd: time slew -0.000206s 2023-10-26T05:44:08.197 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-26T05:44:08.197 INFO:teuthology.orchestra.run.smithi086.stderr:26 Oct 05:44:08 ntpd[27377]: can't open /var/log/ntpstats/loopstats.20231026: Permission denied 2023-10-26T05:44:08.289 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2023-10-26T05:44:08.289 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2023-10-26T05:44:08.289 INFO:teuthology.orchestra.run.smithi086.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.290 INFO:teuthology.orchestra.run.smithi086.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.290 INFO:teuthology.orchestra.run.smithi086.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.290 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.331 INFO:teuthology.orchestra.run.smithi071.stdout: remote refid st t when poll reach delay offset jitter 2023-10-26T05:44:08.331 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================== 2023-10-26T05:44:08.331 INFO:teuthology.orchestra.run.smithi071.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.332 INFO:teuthology.orchestra.run.smithi071.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.332 INFO:teuthology.orchestra.run.smithi071.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.332 INFO:teuthology.orchestra.run.smithi071.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-26T05:44:08.332 INFO:teuthology.run_tasks:Running task install... 2023-10-26T05:44:08.343 DEBUG:teuthology.task.install:project ceph 2023-10-26T05:44:08.343 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}} 2023-10-26T05:44:08.344 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'} 2023-10-26T05:44:08.344 INFO:teuthology.task.install:Using flavor: default 2023-10-26T05:44:08.350 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-26T05:44:08.351 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-26T05:44:08.351 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-key list | grep Ceph 2023-10-26T05:44:08.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-key list | grep Ceph 2023-10-26T05:44:08.413 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-26T05:44:08.429 INFO:teuthology.orchestra.run.smithi071.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-26T05:44:08.443 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-26T05:44:08.443 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph.com (release key) 2023-10-26T05:44:08.444 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-26T05:44:08.444 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-26T05:44:08.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T05:44:08.457 INFO:teuthology.orchestra.run.smithi071.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-26T05:44:08.457 INFO:teuthology.orchestra.run.smithi071.stdout:uid [ unknown] Ceph.com (release key) 2023-10-26T05:44:08.458 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-26T05:44:08.458 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-26T05:44:08.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T05:44:08.680 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default/ 2023-10-26T05:44:08.681 INFO:teuthology.task.install.deb:Package version is 18.0.0-6863-g38499368-1jammy 2023-10-26T05:44:08.687 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default/ 2023-10-26T05:44:08.688 INFO:teuthology.task.install.deb:Package version is 18.0.0-6863-g38499368-1jammy 2023-10-26T05:44:08.809 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:44:08.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-26T05:44:08.824 DEBUG:teuthology.orchestra.run.smithi071:> sudo apt-get update 2023-10-26T05:44:08.872 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:44:08.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-26T05:44:08.887 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2023-10-26T05:44:08.993 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-26T05:44:09.006 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-26T05:44:09.026 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-26T05:44:09.075 INFO:teuthology.orchestra.run.smithi071.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy InRelease 2023-10-26T05:44:09.099 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-26T05:44:09.115 INFO:teuthology.orchestra.run.smithi071.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-26T05:44:09.121 INFO:teuthology.orchestra.run.smithi071.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-26T05:44:09.144 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-10-26T05:44:09.168 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy InRelease 2023-10-26T05:44:09.174 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2023-10-26T05:44:09.181 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-26T05:44:09.204 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-26T05:44:09.242 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-26T05:44:09.279 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2023-10-26T05:44:09.323 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 28.5 kB in 0s (88.3 kB/s) 2023-10-26T05:44:09.324 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-10-26T05:44:10.197 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:44:10.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6863-g38499368-1jammy cephadm=18.0.0-6863-g38499368-1jammy ceph-mds=18.0.0-6863-g38499368-1jammy ceph-mgr=18.0.0-6863-g38499368-1jammy ceph-common=18.0.0-6863-g38499368-1jammy ceph-fuse=18.0.0-6863-g38499368-1jammy ceph-test=18.0.0-6863-g38499368-1jammy ceph-volume=18.0.0-6863-g38499368-1jammy radosgw=18.0.0-6863-g38499368-1jammy python3-rados=18.0.0-6863-g38499368-1jammy python3-rgw=18.0.0-6863-g38499368-1jammy python3-cephfs=18.0.0-6863-g38499368-1jammy python3-rbd=18.0.0-6863-g38499368-1jammy libcephfs2=18.0.0-6863-g38499368-1jammy libcephfs-dev=18.0.0-6863-g38499368-1jammy librados2=18.0.0-6863-g38499368-1jammy librbd1=18.0.0-6863-g38499368-1jammy rbd-fuse=18.0.0-6863-g38499368-1jammy python3-cephfs=18.0.0-6863-g38499368-1jammy cephfs-shell=18.0.0-6863-g38499368-1jammy cephfs-top=18.0.0-6863-g38499368-1jammy cephfs-mirror=18.0.0-6863-g38499368-1jammy 2023-10-26T05:44:10.262 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:44:10.426 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2023-10-26T05:44:10.427 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2023-10-26T05:44:10.576 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:44:10.577 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:44:10.577 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:44:10.577 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:44:10.577 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-26T05:44:10.578 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-26T05:44:10.579 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:44:10.579 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2023-10-26T05:44:10.579 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-26T05:44:10.579 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-26T05:44:10.580 INFO:teuthology.orchestra.run.smithi071.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-26T05:44:10.580 INFO:teuthology.orchestra.run.smithi071.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-26T05:44:10.580 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-26T05:44:10.580 INFO:teuthology.orchestra.run.smithi071.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-26T05:44:10.581 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-26T05:44:10.581 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-26T05:44:10.581 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-26T05:44:10.581 INFO:teuthology.orchestra.run.smithi071.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-26T05:44:10.581 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-26T05:44:10.581 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-26T05:44:10.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-26T05:44:10.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-26T05:44:10.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-26T05:44:10.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-26T05:44:10.582 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-26T05:44:10.583 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-26T05:44:10.583 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-26T05:44:10.583 INFO:teuthology.orchestra.run.smithi071.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-26T05:44:10.583 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-26T05:44:10.583 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-26T05:44:10.584 INFO:teuthology.orchestra.run.smithi071.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-26T05:44:10.584 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2023-10-26T05:44:10.584 INFO:teuthology.orchestra.run.smithi071.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-26T05:44:10.585 INFO:teuthology.orchestra.run.smithi071.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-26T05:44:10.585 INFO:teuthology.orchestra.run.smithi071.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-26T05:44:10.585 INFO:teuthology.orchestra.run.smithi071.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-26T05:44:10.585 INFO:teuthology.orchestra.run.smithi071.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-26T05:44:10.585 INFO:teuthology.orchestra.run.smithi071.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-26T05:44:10.586 INFO:teuthology.orchestra.run.smithi071.stdout: python3-watchdog 2023-10-26T05:44:10.586 INFO:teuthology.orchestra.run.smithi071.stdout:Recommended packages: 2023-10-26T05:44:10.586 INFO:teuthology.orchestra.run.smithi071.stdout: btrfs-tools 2023-10-26T05:44:10.678 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2023-10-26T05:44:10.679 INFO:teuthology.orchestra.run.smithi071.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-26T05:44:10.679 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-26T05:44:10.679 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-26T05:44:10.679 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-26T05:44:10.680 INFO:teuthology.orchestra.run.smithi071.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-26T05:44:10.681 INFO:teuthology.orchestra.run.smithi071.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-26T05:44:10.681 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-26T05:44:10.681 INFO:teuthology.orchestra.run.smithi071.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-26T05:44:10.681 INFO:teuthology.orchestra.run.smithi071.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-26T05:44:10.682 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-26T05:44:10.682 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-26T05:44:10.682 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-26T05:44:10.682 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-26T05:44:10.682 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-26T05:44:10.683 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-26T05:44:10.683 INFO:teuthology.orchestra.run.smithi071.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-26T05:44:10.683 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-26T05:44:10.683 INFO:teuthology.orchestra.run.smithi071.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-26T05:44:10.683 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-26T05:44:10.683 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-26T05:44:10.684 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-26T05:44:10.684 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-26T05:44:10.684 INFO:teuthology.orchestra.run.smithi071.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-26T05:44:10.684 INFO:teuthology.orchestra.run.smithi071.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-26T05:44:10.684 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-26T05:44:10.685 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-26T05:44:10.685 INFO:teuthology.orchestra.run.smithi071.stdout: socat xmlstarlet 2023-10-26T05:44:10.744 INFO:teuthology.orchestra.run.smithi071.stdout:0 upgraded, 115 newly installed, 0 to remove and 257 not upgraded. 2023-10-26T05:44:10.744 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 184 MB of archives. 2023-10-26T05:44:10.745 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-26T05:44:10.745 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-26T05:44:10.764 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-26T05:44:10.775 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-26T05:44:10.825 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-26T05:44:10.832 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-26T05:44:10.848 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6863-g38499368-1jammy [3,437 kB] 2023-10-26T05:44:10.851 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-26T05:44:10.854 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-26T05:44:10.866 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-26T05:44:10.914 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-26T05:44:10.919 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-26T05:44:10.930 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-26T05:44:10.934 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-26T05:44:10.934 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-26T05:44:10.943 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-26T05:44:10.943 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-26T05:44:10.944 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-26T05:44:10.945 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-26T05:44:10.945 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-26T05:44:10.957 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-26T05:44:10.958 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-26T05:44:10.958 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-26T05:44:10.959 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-26T05:44:10.977 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-26T05:44:10.982 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-26T05:44:10.982 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-26T05:44:10.982 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-26T05:44:10.982 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-26T05:44:10.983 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-26T05:44:10.996 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-26T05:44:10.997 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-26T05:44:10.998 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-26T05:44:10.998 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-26T05:44:11.016 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-26T05:44:11.021 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-26T05:44:11.021 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-26T05:44:11.022 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-26T05:44:11.023 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-26T05:44:11.024 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-26T05:44:11.035 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-26T05:44:11.036 INFO:teuthology.orchestra.run.smithi071.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-26T05:44:11.037 INFO:teuthology.orchestra.run.smithi071.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-26T05:44:11.038 INFO:teuthology.orchestra.run.smithi071.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-26T05:44:11.055 INFO:teuthology.orchestra.run.smithi071.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-26T05:44:11.056 INFO:teuthology.orchestra.run.smithi071.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-26T05:44:11.058 INFO:teuthology.orchestra.run.smithi071.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-26T05:44:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-26T05:44:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6863-g38499368-1jammy [3,129 kB] 2023-10-26T05:44:11.112 INFO:teuthology.orchestra.run.smithi071.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-26T05:44:11.113 INFO:teuthology.orchestra.run.smithi071.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-26T05:44:11.114 INFO:teuthology.orchestra.run.smithi071.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-26T05:44:11.114 INFO:teuthology.orchestra.run.smithi071.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-26T05:44:11.115 INFO:teuthology.orchestra.run.smithi071.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-26T05:44:11.132 INFO:teuthology.orchestra.run.smithi071.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-26T05:44:11.134 INFO:teuthology.orchestra.run.smithi071.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-26T05:44:11.135 INFO:teuthology.orchestra.run.smithi071.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-26T05:44:11.149 INFO:teuthology.orchestra.run.smithi071.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-26T05:44:11.152 INFO:teuthology.orchestra.run.smithi071.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-26T05:44:11.153 INFO:teuthology.orchestra.run.smithi071.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-26T05:44:11.153 INFO:teuthology.orchestra.run.smithi071.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-26T05:44:11.154 INFO:teuthology.orchestra.run.smithi071.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-26T05:44:11.154 INFO:teuthology.orchestra.run.smithi071.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-26T05:44:11.172 INFO:teuthology.orchestra.run.smithi071.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-26T05:44:11.172 INFO:teuthology.orchestra.run.smithi071.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-26T05:44:11.175 INFO:teuthology.orchestra.run.smithi071.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-26T05:44:11.177 INFO:teuthology.orchestra.run.smithi071.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-26T05:44:11.178 INFO:teuthology.orchestra.run.smithi071.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-26T05:44:11.191 INFO:teuthology.orchestra.run.smithi071.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-26T05:44:11.194 INFO:teuthology.orchestra.run.smithi071.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-26T05:44:11.197 INFO:teuthology.orchestra.run.smithi071.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-26T05:44:11.197 INFO:teuthology.orchestra.run.smithi071.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-26T05:44:11.201 INFO:teuthology.orchestra.run.smithi071.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-26T05:44:11.211 INFO:teuthology.orchestra.run.smithi071.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-26T05:44:11.211 INFO:teuthology.orchestra.run.smithi071.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-26T05:44:11.212 INFO:teuthology.orchestra.run.smithi071.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-26T05:44:11.212 INFO:teuthology.orchestra.run.smithi071.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-26T05:44:11.213 INFO:teuthology.orchestra.run.smithi071.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-26T05:44:11.221 INFO:teuthology.orchestra.run.smithi071.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6863-g38499368-1jammy [689 kB] 2023-10-26T05:44:11.230 INFO:teuthology.orchestra.run.smithi071.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-26T05:44:11.237 INFO:teuthology.orchestra.run.smithi071.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-26T05:44:11.237 INFO:teuthology.orchestra.run.smithi071.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-26T05:44:11.238 INFO:teuthology.orchestra.run.smithi071.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-26T05:44:11.239 INFO:teuthology.orchestra.run.smithi071.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-26T05:44:11.241 INFO:teuthology.orchestra.run.smithi071.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6863-g38499368-1jammy [356 kB] 2023-10-26T05:44:11.248 INFO:teuthology.orchestra.run.smithi071.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6863-g38499368-1jammy [32.8 kB] 2023-10-26T05:44:11.249 INFO:teuthology.orchestra.run.smithi071.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6863-g38499368-1jammy [180 kB] 2023-10-26T05:44:11.250 INFO:teuthology.orchestra.run.smithi071.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-26T05:44:11.255 INFO:teuthology.orchestra.run.smithi071.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6863-g38499368-1jammy [62.3 kB] 2023-10-26T05:44:11.260 INFO:teuthology.orchestra.run.smithi071.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6863-g38499368-1jammy [330 kB] 2023-10-26T05:44:11.267 INFO:teuthology.orchestra.run.smithi071.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6863-g38499368-1jammy [6,222 kB] 2023-10-26T05:44:11.473 INFO:teuthology.orchestra.run.smithi071.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6863-g38499368-1jammy [112 kB] 2023-10-26T05:44:11.475 INFO:teuthology.orchestra.run.smithi071.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6863-g38499368-1jammy [439 kB] 2023-10-26T05:44:11.489 INFO:teuthology.orchestra.run.smithi071.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6863-g38499368-1jammy [22.6 MB] 2023-10-26T05:44:12.331 INFO:teuthology.orchestra.run.smithi071.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6863-g38499368-1jammy [6,859 kB] 2023-10-26T05:44:12.457 INFO:teuthology.orchestra.run.smithi071.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6863-g38499368-1jammy [238 kB] 2023-10-26T05:44:12.463 INFO:teuthology.orchestra.run.smithi071.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6863-g38499368-1jammy [129 kB] 2023-10-26T05:44:12.468 INFO:teuthology.orchestra.run.smithi071.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6863-g38499368-1jammy [1,497 kB] 2023-10-26T05:44:12.601 INFO:teuthology.orchestra.run.smithi071.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6863-g38499368-1jammy [9,892 kB] 2023-10-26T05:44:12.848 INFO:teuthology.orchestra.run.smithi071.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6863-g38499368-1jammy [31.7 MB] 2023-10-26T05:44:14.057 INFO:teuthology.orchestra.run.smithi071.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6863-g38499368-1jammy [14.0 kB] 2023-10-26T05:44:14.058 INFO:teuthology.orchestra.run.smithi071.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6863-g38499368-1jammy [840 kB] 2023-10-26T05:44:14.073 INFO:teuthology.orchestra.run.smithi071.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6863-g38499368-1jammy [2,150 kB] 2023-10-26T05:44:14.112 INFO:teuthology.orchestra.run.smithi071.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6863-g38499368-1jammy [284 kB] 2023-10-26T05:44:14.116 INFO:teuthology.orchestra.run.smithi071.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6863-g38499368-1jammy [138 kB] 2023-10-26T05:44:14.118 INFO:teuthology.orchestra.run.smithi071.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6863-g38499368-1jammy [1,907 kB] 2023-10-26T05:44:14.140 INFO:teuthology.orchestra.run.smithi071.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6863-g38499368-1jammy [8,625 kB] 2023-10-26T05:44:14.400 INFO:teuthology.orchestra.run.smithi071.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6863-g38499368-1jammy [14.2 kB] 2023-10-26T05:44:14.400 INFO:teuthology.orchestra.run.smithi071.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6863-g38499368-1jammy [47.8 MB] 2023-10-26T05:44:16.230 INFO:teuthology.orchestra.run.smithi071.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6863-g38499368-1jammy [120 kB] 2023-10-26T05:44:16.242 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-26T05:44:16.261 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-26T05:44:16.281 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-26T05:44:16.302 INFO:teuthology.orchestra.run.smithi071.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6863-g38499368-1jammy [227 kB] 2023-10-26T05:44:16.307 INFO:teuthology.orchestra.run.smithi071.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6863-g38499368-1jammy [34.0 kB] 2023-10-26T05:44:16.308 INFO:teuthology.orchestra.run.smithi071.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6863-g38499368-1jammy [23.4 kB] 2023-10-26T05:44:16.308 INFO:teuthology.orchestra.run.smithi071.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6863-g38499368-1jammy [38.3 kB] 2023-10-26T05:44:16.310 INFO:teuthology.orchestra.run.smithi071.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6863-g38499368-1jammy [12.3 MB] 2023-10-26T05:44:16.544 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 28.5 kB in 7s (3,814 B/s) 2023-10-26T05:44:16.607 INFO:teuthology.orchestra.run.smithi071.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6863-g38499368-1jammy [92.2 kB] 2023-10-26T05:44:17.161 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 184 MB in 6s (31.1 MB/s) 2023-10-26T05:44:17.306 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-26T05:44:17.402 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165984 files and directories currently installed.) 2023-10-26T05:44:17.407 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-26T05:44:17.422 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:44:17.437 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6863-g38499368-1jammy cephadm=18.0.0-6863-g38499368-1jammy ceph-mds=18.0.0-6863-g38499368-1jammy ceph-mgr=18.0.0-6863-g38499368-1jammy ceph-common=18.0.0-6863-g38499368-1jammy ceph-fuse=18.0.0-6863-g38499368-1jammy ceph-test=18.0.0-6863-g38499368-1jammy ceph-volume=18.0.0-6863-g38499368-1jammy radosgw=18.0.0-6863-g38499368-1jammy python3-rados=18.0.0-6863-g38499368-1jammy python3-rgw=18.0.0-6863-g38499368-1jammy python3-cephfs=18.0.0-6863-g38499368-1jammy python3-rbd=18.0.0-6863-g38499368-1jammy libcephfs2=18.0.0-6863-g38499368-1jammy libcephfs-dev=18.0.0-6863-g38499368-1jammy librados2=18.0.0-6863-g38499368-1jammy librbd1=18.0.0-6863-g38499368-1jammy rbd-fuse=18.0.0-6863-g38499368-1jammy python3-cephfs=18.0.0-6863-g38499368-1jammy cephfs-shell=18.0.0-6863-g38499368-1jammy cephfs-top=18.0.0-6863-g38499368-1jammy cephfs-mirror=18.0.0-6863-g38499368-1jammy 2023-10-26T05:44:17.454 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-26T05:44:17.496 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:44:17.665 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-26T05:44:17.665 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-26T05:44:17.706 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-26T05:44:17.733 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-26T05:44:17.782 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-26T05:44:17.812 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:44:17.813 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:44:17.813 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:44:17.813 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:44:17.813 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-26T05:44:17.813 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-26T05:44:17.814 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:44:17.815 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2023-10-26T05:44:17.815 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-26T05:44:17.815 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-26T05:44:17.816 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-26T05:44:17.816 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-26T05:44:17.816 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-26T05:44:17.816 INFO:teuthology.orchestra.run.smithi086.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-26T05:44:17.816 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-26T05:44:17.817 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-26T05:44:17.817 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-26T05:44:17.817 INFO:teuthology.orchestra.run.smithi086.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-26T05:44:17.817 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-26T05:44:17.817 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-26T05:44:17.817 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-26T05:44:17.818 INFO:teuthology.orchestra.run.smithi086.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-26T05:44:17.818 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-26T05:44:17.818 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-26T05:44:17.818 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-26T05:44:17.818 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-26T05:44:17.819 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-26T05:44:17.819 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-26T05:44:17.819 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-26T05:44:17.819 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-26T05:44:17.819 INFO:teuthology.orchestra.run.smithi086.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-26T05:44:17.820 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2023-10-26T05:44:17.820 INFO:teuthology.orchestra.run.smithi086.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-26T05:44:17.820 INFO:teuthology.orchestra.run.smithi086.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-26T05:44:17.820 INFO:teuthology.orchestra.run.smithi086.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-26T05:44:17.820 INFO:teuthology.orchestra.run.smithi086.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-26T05:44:17.821 INFO:teuthology.orchestra.run.smithi086.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-26T05:44:17.821 INFO:teuthology.orchestra.run.smithi086.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-26T05:44:17.821 INFO:teuthology.orchestra.run.smithi086.stdout: python3-watchdog 2023-10-26T05:44:17.821 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2023-10-26T05:44:17.821 INFO:teuthology.orchestra.run.smithi086.stdout: btrfs-tools 2023-10-26T05:44:17.916 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2023-10-26T05:44:17.916 INFO:teuthology.orchestra.run.smithi086.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-26T05:44:17.916 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-26T05:44:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-26T05:44:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-26T05:44:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-26T05:44:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-26T05:44:17.917 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-26T05:44:17.918 INFO:teuthology.orchestra.run.smithi086.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-26T05:44:17.918 INFO:teuthology.orchestra.run.smithi086.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-26T05:44:17.918 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-26T05:44:17.918 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-26T05:44:17.918 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-26T05:44:17.919 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-26T05:44:17.919 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-26T05:44:17.919 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-26T05:44:17.919 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-26T05:44:17.919 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-26T05:44:17.920 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-26T05:44:17.920 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-26T05:44:17.920 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-26T05:44:17.920 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-26T05:44:17.920 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-26T05:44:17.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-26T05:44:17.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-26T05:44:17.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-26T05:44:17.921 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-26T05:44:17.921 INFO:teuthology.orchestra.run.smithi086.stdout: socat xmlstarlet 2023-10-26T05:44:17.976 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 115 newly installed, 0 to remove and 257 not upgraded. 2023-10-26T05:44:17.976 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 184 MB of archives. 2023-10-26T05:44:17.976 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-26T05:44:17.977 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-26T05:44:17.993 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-26T05:44:18.003 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-26T05:44:18.044 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-26T05:44:18.052 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-26T05:44:18.059 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6863-g38499368-1jammy [3,437 kB] 2023-10-26T05:44:18.068 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-26T05:44:18.071 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-26T05:44:18.080 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-26T05:44:18.118 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cmd2. 2023-10-26T05:44:18.125 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-26T05:44:18.129 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-26T05:44:18.140 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-26T05:44:18.143 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-26T05:44:18.144 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-26T05:44:18.144 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-26T05:44:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-26T05:44:18.152 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-26T05:44:18.153 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-26T05:44:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-26T05:44:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-26T05:44:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-26T05:44:18.161 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-26T05:44:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-26T05:44:18.178 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-26T05:44:18.179 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-26T05:44:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-26T05:44:18.181 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-26T05:44:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-26T05:44:18.182 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-26T05:44:18.183 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-26T05:44:18.184 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-26T05:44:18.194 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-26T05:44:18.195 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-26T05:44:18.211 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-26T05:44:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-26T05:44:18.212 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-26T05:44:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-26T05:44:18.217 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-26T05:44:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-26T05:44:18.218 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-26T05:44:18.220 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-26T05:44:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-26T05:44:18.228 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-26T05:44:18.238 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-26T05:44:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-26T05:44:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-26T05:44:18.242 INFO:teuthology.orchestra.run.smithi086.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-26T05:44:18.244 INFO:teuthology.orchestra.run.smithi086.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-26T05:44:18.268 INFO:teuthology.orchestra.run.smithi086.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6863-g38499368-1jammy [3,129 kB] 2023-10-26T05:44:18.296 INFO:teuthology.orchestra.run.smithi086.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-26T05:44:18.297 INFO:teuthology.orchestra.run.smithi086.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-26T05:44:18.298 INFO:teuthology.orchestra.run.smithi086.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-26T05:44:18.299 INFO:teuthology.orchestra.run.smithi086.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-26T05:44:18.310 INFO:teuthology.orchestra.run.smithi086.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-26T05:44:18.310 INFO:teuthology.orchestra.run.smithi086.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-26T05:44:18.328 INFO:teuthology.orchestra.run.smithi086.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-26T05:44:18.330 INFO:teuthology.orchestra.run.smithi086.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-26T05:44:18.331 INFO:teuthology.orchestra.run.smithi086.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-26T05:44:18.347 INFO:teuthology.orchestra.run.smithi086.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-26T05:44:18.347 INFO:teuthology.orchestra.run.smithi086.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-26T05:44:18.348 INFO:teuthology.orchestra.run.smithi086.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-26T05:44:18.348 INFO:teuthology.orchestra.run.smithi086.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-26T05:44:18.348 INFO:teuthology.orchestra.run.smithi086.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-26T05:44:18.349 INFO:teuthology.orchestra.run.smithi086.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-26T05:44:18.349 INFO:teuthology.orchestra.run.smithi086.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-26T05:44:18.363 INFO:teuthology.orchestra.run.smithi086.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6863-g38499368-1jammy [689 kB] 2023-10-26T05:44:18.365 INFO:teuthology.orchestra.run.smithi086.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-26T05:44:18.369 INFO:teuthology.orchestra.run.smithi086.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-26T05:44:18.370 INFO:teuthology.orchestra.run.smithi086.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-26T05:44:18.371 INFO:teuthology.orchestra.run.smithi086.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-26T05:44:18.372 INFO:teuthology.orchestra.run.smithi086.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-26T05:44:18.375 INFO:teuthology.orchestra.run.smithi086.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-26T05:44:18.382 INFO:teuthology.orchestra.run.smithi086.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-26T05:44:18.382 INFO:teuthology.orchestra.run.smithi086.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-26T05:44:18.385 INFO:teuthology.orchestra.run.smithi086.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6863-g38499368-1jammy [356 kB] 2023-10-26T05:44:18.386 INFO:teuthology.orchestra.run.smithi086.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-26T05:44:18.389 INFO:teuthology.orchestra.run.smithi086.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-26T05:44:18.396 INFO:teuthology.orchestra.run.smithi086.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6863-g38499368-1jammy [32.8 kB] 2023-10-26T05:44:18.397 INFO:teuthology.orchestra.run.smithi086.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6863-g38499368-1jammy [180 kB] 2023-10-26T05:44:18.398 INFO:teuthology.orchestra.run.smithi086.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-26T05:44:18.399 INFO:teuthology.orchestra.run.smithi086.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-26T05:44:18.400 INFO:teuthology.orchestra.run.smithi086.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-26T05:44:18.401 INFO:teuthology.orchestra.run.smithi086.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-26T05:44:18.401 INFO:teuthology.orchestra.run.smithi086.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-26T05:44:18.404 INFO:teuthology.orchestra.run.smithi086.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6863-g38499368-1jammy [62.3 kB] 2023-10-26T05:44:18.405 INFO:teuthology.orchestra.run.smithi086.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6863-g38499368-1jammy [330 kB] 2023-10-26T05:44:18.408 INFO:teuthology.orchestra.run.smithi086.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-26T05:44:18.415 INFO:teuthology.orchestra.run.smithi086.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-26T05:44:18.416 INFO:teuthology.orchestra.run.smithi086.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-26T05:44:18.416 INFO:teuthology.orchestra.run.smithi086.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6863-g38499368-1jammy [6,222 kB] 2023-10-26T05:44:18.417 INFO:teuthology.orchestra.run.smithi086.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-26T05:44:18.418 INFO:teuthology.orchestra.run.smithi086.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-26T05:44:18.512 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-26T05:44:18.538 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-26T05:44:18.620 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-26T05:44:18.622 INFO:teuthology.orchestra.run.smithi086.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6863-g38499368-1jammy [112 kB] 2023-10-26T05:44:18.654 INFO:teuthology.orchestra.run.smithi086.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6863-g38499368-1jammy [439 kB] 2023-10-26T05:44:18.663 INFO:teuthology.orchestra.run.smithi086.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6863-g38499368-1jammy [22.6 MB] 2023-10-26T05:44:18.931 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-26T05:44:18.954 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-26T05:44:18.998 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-26T05:44:19.318 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-26T05:44:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-26T05:44:19.385 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-26T05:44:19.477 INFO:teuthology.orchestra.run.smithi086.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6863-g38499368-1jammy [6,859 kB] 2023-10-26T05:44:19.616 INFO:teuthology.orchestra.run.smithi086.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6863-g38499368-1jammy [238 kB] 2023-10-26T05:44:19.622 INFO:teuthology.orchestra.run.smithi086.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6863-g38499368-1jammy [129 kB] 2023-10-26T05:44:19.626 INFO:teuthology.orchestra.run.smithi086.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6863-g38499368-1jammy [1,497 kB] 2023-10-26T05:44:19.676 INFO:teuthology.orchestra.run.smithi086.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6863-g38499368-1jammy [9,892 kB] 2023-10-26T05:44:19.888 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-26T05:44:19.914 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-26T05:44:19.962 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-26T05:44:20.002 INFO:teuthology.orchestra.run.smithi086.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6863-g38499368-1jammy [31.7 MB] 2023-10-26T05:44:20.441 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-26T05:44:20.467 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-26T05:44:20.516 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:44:20.903 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-26T05:44:20.929 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-26T05:44:20.969 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:44:21.196 INFO:teuthology.orchestra.run.smithi086.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6863-g38499368-1jammy [14.0 kB] 2023-10-26T05:44:21.197 INFO:teuthology.orchestra.run.smithi086.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6863-g38499368-1jammy [840 kB] 2023-10-26T05:44:21.214 INFO:teuthology.orchestra.run.smithi086.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6863-g38499368-1jammy [2,150 kB] 2023-10-26T05:44:21.253 INFO:teuthology.orchestra.run.smithi086.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6863-g38499368-1jammy [284 kB] 2023-10-26T05:44:21.258 INFO:teuthology.orchestra.run.smithi086.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6863-g38499368-1jammy [138 kB] 2023-10-26T05:44:21.260 INFO:teuthology.orchestra.run.smithi086.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6863-g38499368-1jammy [1,907 kB] 2023-10-26T05:44:21.281 INFO:teuthology.orchestra.run.smithi086.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6863-g38499368-1jammy [8,625 kB] 2023-10-26T05:44:21.322 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-26T05:44:21.348 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-26T05:44:21.388 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:44:21.541 INFO:teuthology.orchestra.run.smithi086.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6863-g38499368-1jammy [14.2 kB] 2023-10-26T05:44:21.542 INFO:teuthology.orchestra.run.smithi086.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6863-g38499368-1jammy [47.8 MB] 2023-10-26T05:44:21.758 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-26T05:44:21.785 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-26T05:44:21.816 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-26T05:44:22.111 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librados2. 2023-10-26T05:44:22.138 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../011-librados2_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:22.177 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librados2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:22.630 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librbd1. 2023-10-26T05:44:22.657 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../012-librbd1_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:22.697 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librbd1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:23.127 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libcephfs2. 2023-10-26T05:44:23.154 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:23.185 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libcephfs2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:23.333 INFO:teuthology.orchestra.run.smithi086.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6863-g38499368-1jammy [120 kB] 2023-10-26T05:44:23.335 INFO:teuthology.orchestra.run.smithi086.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6863-g38499368-1jammy [227 kB] 2023-10-26T05:44:23.340 INFO:teuthology.orchestra.run.smithi086.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6863-g38499368-1jammy [34.0 kB] 2023-10-26T05:44:23.341 INFO:teuthology.orchestra.run.smithi086.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6863-g38499368-1jammy [23.4 kB] 2023-10-26T05:44:23.342 INFO:teuthology.orchestra.run.smithi086.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6863-g38499368-1jammy [38.3 kB] 2023-10-26T05:44:23.343 INFO:teuthology.orchestra.run.smithi086.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6863-g38499368-1jammy [12.3 MB] 2023-10-26T05:44:23.454 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rados. 2023-10-26T05:44:23.469 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:23.512 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rados (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:23.689 INFO:teuthology.orchestra.run.smithi086.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20231024.144545/384993681abceb8cb8affea10538d607052db4b4/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6863-g38499368-1jammy [92.2 kB] 2023-10-26T05:44:23.840 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-26T05:44:23.867 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:23.909 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-ceph-argparse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:24.159 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cephfs. 2023-10-26T05:44:24.186 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:24.225 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cephfs (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:24.242 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 184 MB in 6s (31.9 MB/s) 2023-10-26T05:44:24.441 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-26T05:44:24.541 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165984 files and directories currently installed.) 2023-10-26T05:44:24.546 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-26T05:44:24.597 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-26T05:44:24.820 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-26T05:44:24.846 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:24.849 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-26T05:44:24.865 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-26T05:44:24.895 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:24.899 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-26T05:44:25.194 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cmd2. 2023-10-26T05:44:25.221 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-26T05:44:25.261 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-26T05:44:25.273 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-prettytable. 2023-10-26T05:44:25.299 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-26T05:44:25.348 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-26T05:44:25.547 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-26T05:44:25.573 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-26T05:44:25.647 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-26T05:44:25.651 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rbd. 2023-10-26T05:44:25.677 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:25.717 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rbd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:25.975 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-26T05:44:26.002 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-26T05:44:26.041 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-26T05:44:26.062 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-26T05:44:26.088 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-26T05:44:26.137 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-26T05:44:26.395 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-26T05:44:26.422 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-26T05:44:26.462 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-26T05:44:26.548 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package librgw2. 2023-10-26T05:44:26.574 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../021-librgw2_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:26.622 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking librgw2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:26.815 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-26T05:44:26.841 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-26T05:44:26.881 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-26T05:44:27.192 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rgw. 2023-10-26T05:44:27.218 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:27.258 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:27.301 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-26T05:44:27.328 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-26T05:44:27.375 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:44:27.595 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-26T05:44:27.621 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-26T05:44:27.661 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-26T05:44:27.780 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-26T05:44:27.806 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-26T05:44:27.846 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:44:28.014 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libradosstriper1. 2023-10-26T05:44:28.040 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:28.089 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libradosstriper1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:28.207 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-26T05:44:28.234 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-26T05:44:28.274 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:44:28.602 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-26T05:44:28.617 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-common. 2023-10-26T05:44:28.628 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-26T05:44:28.643 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:28.669 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-26T05:44:28.692 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:28.980 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librados2. 2023-10-26T05:44:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../011-librados2_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:29.046 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librados2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:29.643 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librbd1. 2023-10-26T05:44:29.669 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../012-librbd1_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:29.741 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librbd1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:29.948 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-base. 2023-10-26T05:44:29.974 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:30.024 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-base (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:30.219 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs2. 2023-10-26T05:44:30.245 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:30.285 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:30.571 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rados. 2023-10-26T05:44:30.597 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:30.638 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rados (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:30.727 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-26T05:44:30.754 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-26T05:44:30.794 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-26T05:44:30.932 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-26T05:44:30.958 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:30.990 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-argparse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:31.096 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cheroot. 2023-10-26T05:44:31.123 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-26T05:44:31.163 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-26T05:44:31.210 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cephfs. 2023-10-26T05:44:31.235 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:31.276 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cephfs (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:31.499 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-26T05:44:31.526 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-26T05:44:31.566 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-26T05:44:31.596 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-26T05:44:31.622 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:31.662 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:31.869 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-26T05:44:31.895 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-26T05:44:31.935 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-26T05:44:31.999 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-prettytable. 2023-10-26T05:44:32.025 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-26T05:44:32.065 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-26T05:44:32.205 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-26T05:44:32.231 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-26T05:44:32.263 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-26T05:44:32.309 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rbd. 2023-10-26T05:44:32.325 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:32.359 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rbd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:32.566 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-tempora. 2023-10-26T05:44:32.592 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-26T05:44:32.632 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-26T05:44:32.671 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-26T05:44:32.697 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-26T05:44:32.737 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-26T05:44:32.935 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-portend. 2023-10-26T05:44:32.962 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-26T05:44:33.002 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-26T05:44:33.082 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librgw2. 2023-10-26T05:44:33.108 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../021-librgw2_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:33.148 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librgw2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:33.305 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-26T05:44:33.331 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-26T05:44:33.371 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-26T05:44:33.641 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-26T05:44:33.650 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rgw. 2023-10-26T05:44:33.667 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-26T05:44:33.667 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:33.707 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-26T05:44:33.708 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:34.012 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-26T05:44:34.039 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-26T05:44:34.044 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-natsort. 2023-10-26T05:44:34.070 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-26T05:44:34.071 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-26T05:44:34.111 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-26T05:44:34.373 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libradosstriper1. 2023-10-26T05:44:34.397 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-logutils. 2023-10-26T05:44:34.399 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:34.424 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-26T05:44:34.439 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libradosstriper1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:34.464 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-26T05:44:34.750 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-26T05:44:34.776 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-26T05:44:34.825 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-26T05:44:34.860 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-common. 2023-10-26T05:44:34.886 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:34.926 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:35.378 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-mako. 2023-10-26T05:44:35.404 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-26T05:44:35.444 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-26T05:44:35.756 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-26T05:44:35.782 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-26T05:44:35.823 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-26T05:44:36.083 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-26T05:44:36.109 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-26T05:44:36.158 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-26T05:44:36.374 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-base. 2023-10-26T05:44:36.400 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:36.458 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-base (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:36.461 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-webob. 2023-10-26T05:44:36.488 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-26T05:44:36.528 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-26T05:44:36.880 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-waitress. 2023-10-26T05:44:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-26T05:44:36.975 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-26T05:44:37.236 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-26T05:44:37.262 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-26T05:44:37.275 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-tempita. 2023-10-26T05:44:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-26T05:44:37.303 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-26T05:44:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-26T05:44:37.605 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cheroot. 2023-10-26T05:44:37.619 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-paste. 2023-10-26T05:44:37.632 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-26T05:44:37.635 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-26T05:44:37.669 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-26T05:44:37.672 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-26T05:44:37.980 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-26T05:44:37.996 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-26T05:44:38.000 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-26T05:44:38.027 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-26T05:44:38.039 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-26T05:44:38.067 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-26T05:44:38.309 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-26T05:44:38.335 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-26T05:44:38.367 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-26T05:44:38.411 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-26T05:44:38.438 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-26T05:44:38.486 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-26T05:44:38.670 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-webtest. 2023-10-26T05:44:38.696 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-26T05:44:38.736 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-26T05:44:38.831 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-26T05:44:38.857 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-26T05:44:38.897 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-26T05:44:39.056 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pecan. 2023-10-26T05:44:39.082 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-26T05:44:39.123 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-26T05:44:39.200 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempora. 2023-10-26T05:44:39.226 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-26T05:44:39.267 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-26T05:44:39.477 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-certifi. 2023-10-26T05:44:39.504 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-26T05:44:39.543 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-26T05:44:39.570 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-portend. 2023-10-26T05:44:39.596 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-26T05:44:39.644 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-26T05:44:39.880 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-idna. 2023-10-26T05:44:39.906 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-26T05:44:39.946 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-26T05:44:40.098 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-26T05:44:40.125 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-26T05:44:40.181 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-26T05:44:40.224 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-urllib3. 2023-10-26T05:44:40.250 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-26T05:44:40.407 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-26T05:44:40.483 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-26T05:44:40.509 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-26T05:44:40.550 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-26T05:44:40.711 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-requests. 2023-10-26T05:44:40.738 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-26T05:44:40.810 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-26T05:44:40.906 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-natsort. 2023-10-26T05:44:40.932 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-26T05:44:40.972 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-26T05:44:41.122 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-26T05:44:41.148 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-26T05:44:41.188 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-26T05:44:41.317 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-logutils. 2023-10-26T05:44:41.343 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-26T05:44:41.383 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-26T05:44:41.483 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-26T05:44:41.510 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:41.549 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:41.644 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-26T05:44:41.659 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-26T05:44:41.694 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-26T05:44:41.904 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-26T05:44:41.931 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:41.970 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:41.980 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-mako. 2023-10-26T05:44:42.006 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-26T05:44:42.047 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-26T05:44:42.257 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-openssl. 2023-10-26T05:44:42.283 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-26T05:44:42.323 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-26T05:44:42.417 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-26T05:44:42.443 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-26T05:44:42.483 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-26T05:44:42.676 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr. 2023-10-26T05:44:42.703 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:42.743 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:42.786 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-26T05:44:42.813 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-26T05:44:42.852 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-26T05:44:43.146 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webob. 2023-10-26T05:44:43.163 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mon. 2023-10-26T05:44:43.169 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-26T05:44:43.189 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:43.229 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mon (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:43.230 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-26T05:44:43.624 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-waitress. 2023-10-26T05:44:43.651 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-26T05:44:43.701 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-26T05:44:44.082 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-osd. 2023-10-26T05:44:44.093 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempita. 2023-10-26T05:44:44.111 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:44.120 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-26T05:44:44.160 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-26T05:44:44.446 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-paste. 2023-10-26T05:44:44.461 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-26T05:44:44.496 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-26T05:44:44.557 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-osd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:44.824 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-26T05:44:44.851 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-26T05:44:44.891 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-26T05:44:45.252 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-26T05:44:45.278 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-26T05:44:45.310 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-26T05:44:45.621 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webtest. 2023-10-26T05:44:45.648 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-26T05:44:45.687 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-26T05:44:45.999 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pecan. 2023-10-26T05:44:46.026 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-26T05:44:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-26T05:44:46.445 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-certifi. 2023-10-26T05:44:46.472 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-26T05:44:46.511 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-26T05:44:46.661 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph. 2023-10-26T05:44:46.676 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../061-ceph_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:46.719 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:46.814 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-idna. 2023-10-26T05:44:46.840 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-26T05:44:46.897 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-26T05:44:47.080 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-fuse. 2023-10-26T05:44:47.107 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:47.155 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:47.233 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-urllib3. 2023-10-26T05:44:47.260 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-26T05:44:47.309 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-26T05:44:47.609 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mds. 2023-10-26T05:44:47.635 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:47.670 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests. 2023-10-26T05:44:47.683 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mds (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:47.696 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-26T05:44:47.744 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-26T05:44:48.062 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package cephadm. 2023-10-26T05:44:48.088 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../064-cephadm_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:48.128 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:48.131 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-26T05:44:48.157 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-26T05:44:48.205 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-26T05:44:48.500 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-26T05:44:48.514 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-26T05:44:48.515 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:48.541 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-26T05:44:48.566 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:48.589 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-26T05:44:48.925 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-26T05:44:48.941 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-26T05:44:48.954 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-26T05:44:48.981 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:48.983 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-26T05:44:49.021 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:49.332 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-openssl. 2023-10-26T05:44:49.358 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-26T05:44:49.398 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-26T05:44:49.718 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr. 2023-10-26T05:44:49.744 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:49.776 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:50.022 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-babel. 2023-10-26T05:44:50.049 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-26T05:44:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-26T05:44:50.121 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mon. 2023-10-26T05:44:50.147 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:50.187 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mon (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:50.392 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-jinja2. 2023-10-26T05:44:50.418 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-26T05:44:50.458 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-26T05:44:50.970 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-26T05:44:50.985 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:51.027 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:51.116 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-osd. 2023-10-26T05:44:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:51.182 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-osd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:51.373 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-26T05:44:51.401 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-26T05:44:51.439 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-26T05:44:51.726 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-26T05:44:51.752 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-26T05:44:51.792 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-26T05:44:52.053 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-routes. 2023-10-26T05:44:52.080 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-26T05:44:52.111 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-26T05:44:52.432 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-26T05:44:52.458 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:52.490 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:52.918 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-26T05:44:52.945 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-26T05:44:52.985 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-26T05:44:53.435 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph. 2023-10-26T05:44:53.455 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-joblib. 2023-10-26T05:44:53.462 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../061-ceph_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:53.482 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-26T05:44:53.510 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:53.521 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-26T05:44:53.884 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-26T05:44:53.904 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-fuse. 2023-10-26T05:44:53.910 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-26T05:44:53.931 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:53.950 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-26T05:44:53.979 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:54.261 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-sklearn. 2023-10-26T05:44:54.288 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-26T05:44:54.327 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-26T05:44:54.341 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mds. 2023-10-26T05:44:54.367 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:54.407 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mds (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:54.794 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephadm. 2023-10-26T05:44:54.820 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../064-cephadm_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:44:54.860 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:55.045 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-26T05:44:55.072 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:55.120 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:55.238 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-26T05:44:55.264 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-26T05:44:55.313 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-26T05:44:55.649 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-26T05:44:55.675 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-26T05:44:55.724 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-26T05:44:56.172 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-cachetools. 2023-10-26T05:44:56.199 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-26T05:44:56.247 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-26T05:44:56.584 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-26T05:44:56.611 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-26T05:44:56.650 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-26T05:44:56.695 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-babel. 2023-10-26T05:44:56.722 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-26T05:44:56.762 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-26T05:44:56.987 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-26T05:44:57.014 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-26T05:44:57.053 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-26T05:44:57.098 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jinja2. 2023-10-26T05:44:57.125 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-26T05:44:57.165 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-26T05:44:57.356 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-rsa. 2023-10-26T05:44:57.383 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-26T05:44:57.422 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-26T05:44:57.518 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-26T05:44:57.544 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:57.584 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:57.836 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-google-auth. 2023-10-26T05:44:57.863 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-26T05:44:57.902 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-26T05:44:57.921 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-26T05:44:57.947 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-26T05:44:57.987 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-26T05:44:58.206 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-26T05:44:58.232 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-26T05:44:58.264 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-26T05:44:58.265 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-26T05:44:58.281 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-26T05:44:58.315 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-26T05:44:58.542 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-websocket. 2023-10-26T05:44:58.569 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-26T05:44:58.608 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-26T05:44:58.618 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-routes. 2023-10-26T05:44:58.644 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-26T05:44:58.684 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-26T05:44:58.928 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-26T05:44:58.955 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-26T05:44:58.994 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-26T05:44:58.996 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-26T05:44:59.023 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:59.062 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:44:59.466 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-26T05:44:59.481 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-26T05:44:59.515 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-26T05:44:59.709 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-26T05:44:59.736 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:44:59.775 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:00.036 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-joblib. 2023-10-26T05:45:00.063 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-26T05:45:00.102 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-26T05:45:00.103 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-26T05:45:00.130 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-26T05:45:00.161 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-26T05:45:00.481 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-26T05:45:00.506 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-26T05:45:00.508 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-26T05:45:00.533 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-26T05:45:00.547 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-26T05:45:00.572 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-26T05:45:00.842 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package jq. 2023-10-26T05:45:00.842 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn. 2023-10-26T05:45:00.857 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-26T05:45:00.868 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-26T05:45:00.900 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-26T05:45:00.908 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-26T05:45:01.311 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package socat. 2023-10-26T05:45:01.326 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-26T05:45:01.361 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-26T05:45:01.639 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package xmlstarlet. 2023-10-26T05:45:01.655 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-26T05:45:01.689 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-26T05:45:01.900 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-26T05:45:01.928 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:01.966 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:02.035 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-test. 2023-10-26T05:45:02.051 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:02.093 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-test (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:02.969 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cachetools. 2023-10-26T05:45:02.996 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-26T05:45:03.037 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-26T05:45:03.338 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-26T05:45:03.365 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-26T05:45:03.405 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-26T05:45:03.716 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-26T05:45:03.743 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-26T05:45:03.774 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-26T05:45:04.086 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rsa. 2023-10-26T05:45:04.112 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-26T05:45:04.145 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-26T05:45:04.397 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ceph-volume. 2023-10-26T05:45:04.424 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:04.463 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ceph-volume (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:04.549 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-google-auth. 2023-10-26T05:45:04.576 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-26T05:45:04.615 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-26T05:45:04.859 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-26T05:45:04.887 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:04.925 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking cephfs-mirror (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:04.935 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-26T05:45:04.962 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-26T05:45:05.001 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-26T05:45:05.237 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-colorama. 2023-10-26T05:45:05.265 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-26T05:45:05.304 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-26T05:45:05.304 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-websocket. 2023-10-26T05:45:05.331 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-26T05:45:05.371 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-26T05:45:05.665 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package cephfs-shell. 2023-10-26T05:45:05.692 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:05.791 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-26T05:45:05.814 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking cephfs-shell (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:05.818 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-26T05:45:05.857 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-26T05:45:06.094 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package cephfs-top. 2023-10-26T05:45:06.110 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:06.153 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking cephfs-top (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:06.431 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-26T05:45:06.447 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:06.462 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-26T05:45:06.478 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:06.497 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libcephfs-dev (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:06.512 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:06.790 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-26T05:45:06.817 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-26T05:45:06.890 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-26T05:45:06.959 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package nvme-cli. 2023-10-26T05:45:06.987 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-26T05:45:07.042 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-26T05:45:07.218 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-26T05:45:07.245 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-26T05:45:07.284 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-26T05:45:07.431 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-26T05:45:07.459 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-26T05:45:07.506 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-26T05:45:07.553 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package jq. 2023-10-26T05:45:07.569 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-26T05:45:07.611 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-26T05:45:07.855 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package socat. 2023-10-26T05:45:07.871 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-26T05:45:07.905 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-26T05:45:08.010 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-26T05:45:08.037 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-26T05:45:08.084 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-26T05:45:08.183 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package xmlstarlet. 2023-10-26T05:45:08.199 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-26T05:45:08.233 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-26T05:45:08.404 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pastescript. 2023-10-26T05:45:08.432 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-26T05:45:08.471 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-26T05:45:08.529 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-test. 2023-10-26T05:45:08.556 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:08.595 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-test (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:08.791 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pluggy. 2023-10-26T05:45:08.819 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-26T05:45:08.857 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-26T05:45:09.228 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-psutil. 2023-10-26T05:45:09.256 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-26T05:45:09.302 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-26T05:45:09.664 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-py. 2023-10-26T05:45:09.692 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-26T05:45:09.730 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-26T05:45:10.051 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pygments. 2023-10-26T05:45:10.078 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-26T05:45:10.117 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-26T05:45:10.589 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-26T05:45:10.616 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-26T05:45:10.646 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-26T05:45:10.824 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-volume. 2023-10-26T05:45:10.852 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:10.891 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-volume (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:10.949 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-toml. 2023-10-26T05:45:10.977 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-26T05:45:11.016 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-26T05:45:11.245 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-26T05:45:11.272 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:11.311 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-pytest. 2023-10-26T05:45:11.311 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-mirror (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:11.338 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-26T05:45:11.377 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-26T05:45:11.631 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-colorama. 2023-10-26T05:45:11.658 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-26T05:45:11.697 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-26T05:45:11.725 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package python3-simplejson. 2023-10-26T05:45:11.751 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-26T05:45:11.789 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-26T05:45:12.042 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-26T05:45:12.057 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-26T05:45:12.100 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-26T05:45:12.200 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-shell. 2023-10-26T05:45:12.227 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:12.266 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-shell (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:12.519 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-top. 2023-10-26T05:45:12.535 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6863-g38499368-1jammy_all.deb ... 2023-10-26T05:45:12.577 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-top (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:12.788 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package radosgw. 2023-10-26T05:45:12.804 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../113-radosgw_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:12.837 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking radosgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:12.846 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-26T05:45:12.864 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:12.905 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs-dev (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:13.258 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package nvme-cli. 2023-10-26T05:45:13.285 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-26T05:45:13.324 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-26T05:45:13.558 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package rbd-fuse. 2023-10-26T05:45:13.586 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:13.624 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking rbd-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:13.679 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-26T05:45:13.707 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-26T05:45:13.745 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-26T05:45:13.937 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-26T05:45:14.174 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-26T05:45:14.181 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-26T05:45:14.201 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-26T05:45:14.240 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-26T05:45:14.306 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-26T05:45:14.518 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastescript. 2023-10-26T05:45:14.545 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-26T05:45:14.571 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-26T05:45:14.575 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-26T05:45:14.837 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pluggy. 2023-10-26T05:45:14.865 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-26T05:45:14.895 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-26T05:45:14.927 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-26T05:45:15.206 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-psutil. 2023-10-26T05:45:15.234 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-26T05:45:15.273 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-26T05:45:15.455 INFO:teuthology.orchestra.run.smithi071.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-26T05:45:15.582 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:15.626 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-py. 2023-10-26T05:45:15.653 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-26T05:45:15.692 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-26T05:45:16.004 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pygments. 2023-10-26T05:45:16.031 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-26T05:45:16.070 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-26T05:45:16.126 INFO:teuthology.orchestra.run.smithi071.stdout:Adding system user cephadm....done 2023-10-26T05:45:16.272 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-26T05:45:16.532 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-26T05:45:16.557 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-26T05:45:16.560 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-26T05:45:16.599 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-26T05:45:16.910 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-toml. 2023-10-26T05:45:16.937 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-26T05:45:16.976 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-26T05:45:17.025 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-26T05:45:17.159 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-26T05:45:17.245 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pytest. 2023-10-26T05:45:17.261 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-26T05:45:17.295 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-26T05:45:17.411 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-26T05:45:17.623 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplejson. 2023-10-26T05:45:17.637 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-26T05:45:17.651 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-26T05:45:17.690 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-26T05:45:17.746 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-26T05:45:17.976 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-26T05:45:17.992 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-26T05:45:18.034 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-26T05:45:18.056 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-26T05:45:18.425 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-26T05:45:18.685 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-26T05:45:18.713 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package radosgw. 2023-10-26T05:45:18.740 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../113-radosgw_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:18.779 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking radosgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:18.921 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-26T05:45:19.163 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-26T05:45:19.439 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-26T05:45:19.515 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package rbd-fuse. 2023-10-26T05:45:19.542 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6863-g38499368-1jammy_amd64.deb ... 2023-10-26T05:45:19.582 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking rbd-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:19.774 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-26T05:45:19.885 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-26T05:45:20.066 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-ceph-argparse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:20.136 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-26T05:45:20.262 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-26T05:45:20.317 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-26T05:45:20.419 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-26T05:45:20.527 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-26T05:45:20.670 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-26T05:45:20.885 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-26T05:45:20.921 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-26T05:45:21.047 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-26T05:45:21.355 INFO:teuthology.orchestra.run.smithi086.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-26T05:45:21.415 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-26T05:45:21.472 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:21.541 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-26T05:45:21.809 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-26T05:45:22.007 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user cephadm....done 2023-10-26T05:45:22.135 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-26T05:45:22.153 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-26T05:45:22.429 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-26T05:45:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-26T05:45:22.680 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-26T05:45:22.798 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-26T05:45:23.051 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-26T05:45:23.114 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-26T05:45:23.310 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-26T05:45:23.373 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-26T05:45:23.436 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-26T05:45:23.500 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-26T05:45:23.626 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-26T05:45:23.738 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-26T05:45:23.919 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-26T05:45:24.106 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-26T05:45:24.295 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-26T05:45:24.341 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-26T05:45:24.538 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-26T05:45:24.584 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-26T05:45:24.860 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-26T05:45:24.906 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-26T05:45:25.111 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-26T05:45:25.216 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-26T05:45:25.378 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-26T05:45:25.526 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-26T05:45:25.638 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-argparse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:25.877 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-26T05:45:25.914 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-26T05:45:26.023 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-26T05:45:26.103 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-26T05:45:26.221 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:26.257 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-26T05:45:26.501 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-26T05:45:26.514 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-idna (3.3-1) ... 2023-10-26T05:45:26.634 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-26T05:45:26.823 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-26T05:45:26.994 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-26T05:45:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-26T05:45:27.621 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-26T05:45:27.888 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-26T05:45:28.022 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:45:28.156 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-26T05:45:28.190 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-26T05:45:28.416 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-26T05:45:28.542 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-26T05:45:28.643 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-26T05:45:28.825 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-26T05:45:28.936 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-26T05:45:29.084 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-26T05:45:29.179 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-26T05:45:29.218 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-26T05:45:29.344 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-26T05:45:29.430 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:45:29.556 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-26T05:45:29.645 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-26T05:45:29.682 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-26T05:45:30.018 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-26T05:45:30.021 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-26T05:45:30.248 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-26T05:45:30.260 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-26T05:45:30.495 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-26T05:45:30.632 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-26T05:45:30.746 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-26T05:45:30.909 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-26T05:45:30.997 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-26T05:45:31.201 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-26T05:45:31.273 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-26T05:45:31.399 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-26T05:45:31.544 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-26T05:45:31.684 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-26T05:45:31.785 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-26T05:45:31.795 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-26T05:45:31.921 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:32.053 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-26T05:45:32.214 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-idna (3.3-1) ... 2023-10-26T05:45:32.405 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-26T05:45:32.507 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-26T05:45:32.664 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-26T05:45:33.054 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-26T05:45:33.309 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-26T05:45:33.410 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-26T05:45:33.596 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:45:33.721 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-26T05:45:33.778 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-26T05:45:33.972 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-26T05:45:34.038 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-26T05:45:34.073 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-26T05:45:34.182 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-26T05:45:34.297 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:45:34.424 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-26T05:45:34.467 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-26T05:45:34.718 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-26T05:45:34.750 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-26T05:45:34.978 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:45:35.053 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-26T05:45:35.112 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-26T05:45:35.238 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-26T05:45:35.311 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-26T05:45:35.437 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-26T05:45:35.564 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-26T05:45:35.705 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-26T05:45:35.798 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-26T05:45:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-26T05:45:36.049 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-26T05:45:36.257 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-26T05:45:36.309 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-26T05:45:36.560 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-26T05:45:36.634 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-26T05:45:36.836 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-26T05:45:36.910 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-26T05:45:36.962 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-26T05:45:37.246 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-26T05:45:37.254 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-26T05:45:37.364 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-26T05:45:37.372 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librados2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:37.498 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librgw2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:37.657 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-26T05:45:37.702 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:37.811 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libcephfs2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:37.937 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libradosstriper1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:37.941 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-26T05:45:38.063 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-26T05:45:38.192 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-26T05:45:38.340 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up librbd1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:38.500 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-modules-core (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:38.587 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-26T05:45:38.609 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:38.807 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-26T05:45:38.808 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-26T05:45:38.836 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-26T05:45:38.962 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-26T05:45:39.339 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-26T05:45:39.352 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libcephfs-dev (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:39.478 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rados (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:39.564 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-26T05:45:39.604 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up cephfs-top (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:39.731 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-26T05:45:39.799 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-26T05:45:39.908 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-26T05:45:40.243 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-26T05:45:40.535 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-26T05:45:40.811 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-26T05:45:40.937 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-26T05:45:40.943 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rbd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:41.070 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up rbd-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:41.196 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-rgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:41.213 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-26T05:45:41.322 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up python3-cephfs (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:41.457 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-26T05:45:41.465 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up cephfs-shell (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:41.591 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:41.690 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-26T05:45:42.075 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-26T05:45:42.093 INFO:teuthology.orchestra.run.smithi071.stdout:Adding group ceph....done 2023-10-26T05:45:42.377 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-26T05:45:42.495 INFO:teuthology.orchestra.run.smithi071.stdout:Adding system user ceph....done 2023-10-26T05:45:42.630 INFO:teuthology.orchestra.run.smithi071.stdout:Setting system user ceph properties....done 2023-10-26T05:45:42.647 INFO:teuthology.orchestra.run.smithi071.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-26T05:45:42.720 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-26T05:45:42.779 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-26T05:45:42.846 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librados2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:42.980 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librgw2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:43.134 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-26T05:45:43.183 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:43.318 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs2 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:43.444 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libradosstriper1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:43.570 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-26T05:45:43.668 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up cephfs-mirror (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:43.846 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librbd1 (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:43.911 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-26T05:45:43.912 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-26T05:45:43.947 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-modules-core (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:44.073 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:44.270 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-26T05:45:44.270 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-26T05:45:44.587 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-test (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:44.713 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up radosgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:44.783 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs-dev (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:44.909 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rados (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:45.035 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-top (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:45.161 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-26T05:45:45.226 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-26T05:45:45.226 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-26T05:45:45.757 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-base (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.095 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-26T05:45:46.315 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rbd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.424 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up rbd-fuse (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.550 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.617 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mds (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.676 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cephfs (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.827 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-shell (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:46.830 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-26T05:45:46.830 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-26T05:45:46.953 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-common (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:47.344 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:47.436 INFO:teuthology.orchestra.run.smithi086.stdout:Adding group ceph....done 2023-10-26T05:45:47.563 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-26T05:45:47.564 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-26T05:45:47.838 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user ceph....done 2023-10-26T05:45:48.015 INFO:teuthology.orchestra.run.smithi086.stdout:Setting system user ceph properties....done 2023-10-26T05:45:48.032 INFO:teuthology.orchestra.run.smithi086.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-26T05:45:48.115 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-osd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:48.168 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-26T05:45:48.404 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-26T05:45:48.405 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-26T05:45:48.532 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-26T05:45:48.942 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:49.062 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-mirror (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:49.068 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:49.219 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mon (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:49.273 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-26T05:45:49.274 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-26T05:45:49.432 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-26T05:45:49.432 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-26T05:45:49.830 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-test (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:49.931 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up radosgw (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:49.946 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:50.088 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:50.214 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-mgr-dashboard (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:50.358 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ceph-volume (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:50.399 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-26T05:45:50.400 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-26T05:45:50.677 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-26T05:45:50.907 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-base (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:51.248 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-26T05:45:51.713 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-26T05:45:51.751 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mds (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:51.984 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-26T05:45:51.985 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-26T05:45:52.178 INFO:teuthology.orchestra.run.smithi071.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-26T05:45:52.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-26T05:45:52.283 INFO:teuthology.orchestra.run.smithi071.stdout:Reading package lists... 2023-10-26T05:45:52.468 INFO:teuthology.orchestra.run.smithi071.stdout:Building dependency tree... 2023-10-26T05:45:52.468 INFO:teuthology.orchestra.run.smithi071.stdout:Reading state information... 2023-10-26T05:45:52.503 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:52.630 INFO:teuthology.orchestra.run.smithi071.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-26T05:45:52.630 INFO:teuthology.orchestra.run.smithi071.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-26T05:45:52.630 INFO:teuthology.orchestra.run.smithi071.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-26T05:45:52.631 INFO:teuthology.orchestra.run.smithi071.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-26T05:45:52.631 INFO:teuthology.orchestra.run.smithi071.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-26T05:45:52.631 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:45:52.631 INFO:teuthology.orchestra.run.smithi071.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:45:52.631 INFO:teuthology.orchestra.run.smithi071.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:45:52.632 INFO:teuthology.orchestra.run.smithi071.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:45:52.632 INFO:teuthology.orchestra.run.smithi071.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-26T05:45:52.633 INFO:teuthology.orchestra.run.smithi071.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-26T05:45:52.633 INFO:teuthology.orchestra.run.smithi071.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:45:52.633 INFO:teuthology.orchestra.run.smithi071.stdout:The following additional packages will be installed: 2023-10-26T05:45:52.633 INFO:teuthology.orchestra.run.smithi071.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-26T05:45:52.634 INFO:teuthology.orchestra.run.smithi071.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-26T05:45:52.634 INFO:teuthology.orchestra.run.smithi071.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-26T05:45:52.634 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-26T05:45:52.634 INFO:teuthology.orchestra.run.smithi071.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-26T05:45:52.634 INFO:teuthology.orchestra.run.smithi071.stdout:Suggested packages: 2023-10-26T05:45:52.634 INFO:teuthology.orchestra.run.smithi071.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-26T05:45:52.635 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-26T05:45:52.635 INFO:teuthology.orchestra.run.smithi071.stdout: libengine-pkcs11-openssl 2023-10-26T05:45:52.668 INFO:teuthology.orchestra.run.smithi071.stdout:The following NEW packages will be installed: 2023-10-26T05:45:52.669 INFO:teuthology.orchestra.run.smithi071.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-26T05:45:52.669 INFO:teuthology.orchestra.run.smithi071.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-26T05:45:52.670 INFO:teuthology.orchestra.run.smithi071.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-26T05:45:52.670 INFO:teuthology.orchestra.run.smithi071.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-26T05:45:52.670 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-26T05:45:52.670 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-26T05:45:52.670 INFO:teuthology.orchestra.run.smithi071.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-26T05:45:52.671 INFO:teuthology.orchestra.run.smithi071.stdout:The following packages will be upgraded: 2023-10-26T05:45:52.671 INFO:teuthology.orchestra.run.smithi071.stdout: libssl3 2023-10-26T05:45:52.723 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-26T05:45:52.723 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-26T05:45:52.747 INFO:teuthology.orchestra.run.smithi071.stdout:1 upgraded, 39 newly installed, 0 to remove and 256 not upgraded. 2023-10-26T05:45:52.748 INFO:teuthology.orchestra.run.smithi071.stdout:Need to get 54.6 MB of archives. 2023-10-26T05:45:52.748 INFO:teuthology.orchestra.run.smithi071.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-26T05:45:52.748 INFO:teuthology.orchestra.run.smithi071.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-26T05:45:52.863 INFO:teuthology.orchestra.run.smithi071.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-10-26T05:45:52.934 INFO:teuthology.orchestra.run.smithi071.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-26T05:45:52.944 INFO:teuthology.orchestra.run.smithi071.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-26T05:45:52.944 INFO:teuthology.orchestra.run.smithi071.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-26T05:45:52.946 INFO:teuthology.orchestra.run.smithi071.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-26T05:45:52.948 INFO:teuthology.orchestra.run.smithi071.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-26T05:45:52.948 INFO:teuthology.orchestra.run.smithi071.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-26T05:45:52.950 INFO:teuthology.orchestra.run.smithi071.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-26T05:45:52.951 INFO:teuthology.orchestra.run.smithi071.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-26T05:45:52.952 INFO:teuthology.orchestra.run.smithi071.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-26T05:45:52.957 INFO:teuthology.orchestra.run.smithi071.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-26T05:45:52.989 INFO:teuthology.orchestra.run.smithi071.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-26T05:45:53.279 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-osd (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:53.386 INFO:teuthology.orchestra.run.smithi071.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-26T05:45:53.388 INFO:teuthology.orchestra.run.smithi071.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-26T05:45:53.389 INFO:teuthology.orchestra.run.smithi071.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-26T05:45:53.391 INFO:teuthology.orchestra.run.smithi071.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-26T05:45:53.391 INFO:teuthology.orchestra.run.smithi071.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-26T05:45:53.395 INFO:teuthology.orchestra.run.smithi071.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-26T05:45:53.397 INFO:teuthology.orchestra.run.smithi071.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-26T05:45:53.404 INFO:teuthology.orchestra.run.smithi071.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-26T05:45:53.404 INFO:teuthology.orchestra.run.smithi071.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-10-26T05:45:53.424 INFO:teuthology.orchestra.run.smithi071.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-26T05:45:53.426 INFO:teuthology.orchestra.run.smithi071.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-26T05:45:53.437 INFO:teuthology.orchestra.run.smithi071.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-26T05:45:53.460 INFO:teuthology.orchestra.run.smithi071.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-26T05:45:53.464 INFO:teuthology.orchestra.run.smithi071.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-26T05:45:53.464 INFO:teuthology.orchestra.run.smithi071.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-26T05:45:53.465 INFO:teuthology.orchestra.run.smithi071.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-26T05:45:53.465 INFO:teuthology.orchestra.run.smithi071.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-26T05:45:53.476 INFO:teuthology.orchestra.run.smithi071.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-26T05:45:53.478 INFO:teuthology.orchestra.run.smithi071.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-26T05:45:53.610 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-26T05:45:53.610 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-26T05:45:53.709 INFO:teuthology.orchestra.run.smithi071.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-26T05:45:53.710 INFO:teuthology.orchestra.run.smithi071.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-26T05:45:53.710 INFO:teuthology.orchestra.run.smithi071.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-26T05:45:53.710 INFO:teuthology.orchestra.run.smithi071.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-26T05:45:53.710 INFO:teuthology.orchestra.run.smithi071.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-26T05:45:53.711 INFO:teuthology.orchestra.run.smithi071.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-26T05:45:53.721 INFO:teuthology.orchestra.run.smithi071.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-26T05:45:53.739 INFO:teuthology.orchestra.run.smithi071.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-26T05:45:54.136 INFO:teuthology.orchestra.run.smithi071.stdout:Preconfiguring packages ... 2023-10-26T05:45:54.155 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:54.257 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:54.313 INFO:teuthology.orchestra.run.smithi071.stdout:Fetched 54.6 MB in 1s (51.4 MB/s) 2023-10-26T05:45:54.391 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mon (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:54.455 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package flex. 2023-10-26T05:45:54.548 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172903 files and directories currently installed.) 2023-10-26T05:45:54.553 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-26T05:45:54.595 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-26T05:45:54.603 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-26T05:45:54.603 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-26T05:45:55.112 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-26T05:45:55.134 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-cephadm (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:55.252 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-10-26T05:45:55.285 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:55.411 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-dashboard (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:55.562 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-volume (18.0.0-6863-g38499368-1jammy) ... 2023-10-26T05:45:55.670 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-26T05:45:55.906 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-26T05:45:56.205 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package bison. 2023-10-26T05:45:56.305 INFO:teuthology.orchestra.run.smithi071.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172976 files and directories currently installed.) 2023-10-26T05:45:56.311 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-26T05:45:56.366 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-26T05:45:56.759 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-26T05:45:56.786 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-26T05:45:56.825 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-26T05:45:57.015 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-26T05:45:57.187 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package dump. 2023-10-26T05:45:57.215 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-26T05:45:57.253 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking dump (0.4b47-3) ... 2023-10-26T05:45:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-26T05:45:57.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-26T05:45:57.501 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-26T05:45:57.557 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package indent. 2023-10-26T05:45:57.585 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:45:57.632 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-26T05:45:57.687 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-26T05:45:57.688 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-26T05:45:57.852 INFO:teuthology.orchestra.run.smithi086.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-26T05:45:57.852 INFO:teuthology.orchestra.run.smithi086.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-26T05:45:57.853 INFO:teuthology.orchestra.run.smithi086.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-26T05:45:57.853 INFO:teuthology.orchestra.run.smithi086.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-26T05:45:57.853 INFO:teuthology.orchestra.run.smithi086.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-26T05:45:57.853 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-26T05:45:57.853 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-26T05:45:57.854 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-26T05:45:57.854 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-26T05:45:57.854 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-26T05:45:57.855 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-26T05:45:57.855 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-26T05:45:57.855 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2023-10-26T05:45:57.856 INFO:teuthology.orchestra.run.smithi086.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-26T05:45:57.856 INFO:teuthology.orchestra.run.smithi086.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-26T05:45:57.856 INFO:teuthology.orchestra.run.smithi086.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-26T05:45:57.856 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-26T05:45:57.856 INFO:teuthology.orchestra.run.smithi086.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-26T05:45:57.857 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2023-10-26T05:45:57.857 INFO:teuthology.orchestra.run.smithi086.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-26T05:45:57.857 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-26T05:45:57.858 INFO:teuthology.orchestra.run.smithi086.stdout: libengine-pkcs11-openssl 2023-10-26T05:45:57.894 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2023-10-26T05:45:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-26T05:45:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-26T05:45:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-26T05:45:57.896 INFO:teuthology.orchestra.run.smithi086.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-26T05:45:57.896 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-26T05:45:57.896 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-26T05:45:57.897 INFO:teuthology.orchestra.run.smithi086.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-26T05:45:57.897 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2023-10-26T05:45:57.897 INFO:teuthology.orchestra.run.smithi086.stdout: libssl3 2023-10-26T05:45:57.901 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-26T05:45:57.917 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-26T05:45:57.950 INFO:teuthology.orchestra.run.smithi086.stdout:1 upgraded, 39 newly installed, 0 to remove and 256 not upgraded. 2023-10-26T05:45:57.951 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 54.6 MB of archives. 2023-10-26T05:45:57.951 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-26T05:45:57.951 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-26T05:45:57.952 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-26T05:45:58.050 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-10-26T05:45:58.120 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-26T05:45:58.159 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-26T05:45:58.180 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-26T05:45:58.200 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-26T05:45:58.202 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-26T05:45:58.203 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-26T05:45:58.206 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-26T05:45:58.207 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-26T05:45:58.208 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-26T05:45:58.209 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-26T05:45:58.211 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-26T05:45:58.247 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-26T05:45:58.274 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-26T05:45:58.313 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-26T05:45:58.487 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-26T05:45:58.489 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-26T05:45:58.547 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-26T05:45:58.549 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-26T05:45:58.550 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-26T05:45:58.554 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-26T05:45:58.555 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-26T05:45:58.563 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-26T05:45:58.563 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-10-26T05:45:58.583 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-26T05:45:58.587 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-26T05:45:58.588 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-26T05:45:58.598 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-26T05:45:58.599 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-26T05:45:58.616 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-26T05:45:58.619 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-26T05:45:58.620 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-26T05:45:58.620 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-26T05:45:58.621 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-26T05:45:58.631 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-26T05:45:58.632 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-26T05:45:58.641 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-26T05:45:58.830 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-26T05:45:58.831 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-26T05:45:58.831 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-26T05:45:58.831 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-26T05:45:58.831 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-26T05:45:58.832 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-26T05:45:58.842 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-26T05:45:58.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-26T05:45:58.953 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-26T05:45:58.981 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-26T05:45:59.011 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-26T05:45:59.230 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-26T05:45:59.242 INFO:teuthology.orchestra.run.smithi086.stdout:Preconfiguring packages ... 2023-10-26T05:45:59.246 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-26T05:45:59.289 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-26T05:45:59.383 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 54.6 MB in 1s (58.1 MB/s) 2023-10-26T05:45:59.522 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package flex. 2023-10-26T05:45:59.533 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libjson-perl. 2023-10-26T05:45:59.549 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-26T05:45:59.591 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-26T05:45:59.621 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172903 files and directories currently installed.) 2023-10-26T05:45:59.627 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-26T05:45:59.670 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-26T05:46:00.012 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-26T05:46:00.039 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-26T05:46:00.078 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-26T05:46:00.202 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-26T05:46:00.384 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-10-26T05:46:00.784 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-26T05:46:01.360 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package bison. 2023-10-26T05:46:01.464 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172976 files and directories currently installed.) 2023-10-26T05:46:01.470 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-26T05:46:01.511 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-26T05:46:01.698 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-26T05:46:01.725 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-26T05:46:01.764 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:01.996 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-26T05:46:02.000 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-26T05:46:02.023 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-26T05:46:02.028 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-26T05:46:02.059 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:02.070 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-26T05:46:02.395 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-26T05:46:02.415 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package dump. 2023-10-26T05:46:02.422 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-26T05:46:02.442 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-26T05:46:02.461 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:02.481 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking dump (0.4b47-3) ... 2023-10-26T05:46:02.785 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package indent. 2023-10-26T05:46:02.812 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:02.814 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-26T05:46:02.842 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-26T05:46:02.851 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-26T05:46:02.881 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-26T05:46:03.137 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-26T05:46:03.165 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-26T05:46:03.193 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-26T05:46:03.212 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-26T05:46:03.220 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-26T05:46:03.251 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-26T05:46:03.587 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-26T05:46:03.615 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:03.654 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:03.893 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-26T05:46:03.921 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-26T05:46:03.960 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-26T05:46:03.990 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-26T05:46:04.018 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:04.056 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:04.229 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-26T05:46:04.256 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-26T05:46:04.295 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-26T05:46:04.343 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-26T05:46:04.359 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:04.393 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:04.622 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-26T05:46:04.636 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-26T05:46:04.678 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-26T05:46:04.715 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-26T05:46:04.742 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-26T05:46:04.790 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-26T05:46:05.092 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-26T05:46:05.120 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-26T05:46:05.168 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-26T05:46:05.207 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-26T05:46:05.235 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-26T05:46:05.299 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-26T05:46:05.520 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjson-perl. 2023-10-26T05:46:05.537 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-26T05:46:05.578 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-26T05:46:05.636 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package modemmanager. 2023-10-26T05:46:05.663 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-26T05:46:05.702 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:05.890 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-26T05:46:05.917 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-26T05:46:05.964 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-26T05:46:06.334 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package network-manager. 2023-10-26T05:46:06.361 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-26T05:46:06.408 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-26T05:46:07.058 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package ppp. 2023-10-26T05:46:07.085 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-26T05:46:07.410 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-26T05:46:07.692 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-26T05:46:07.713 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package pptp-linux. 2023-10-26T05:46:07.719 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-26T05:46:07.740 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-26T05:46:07.758 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:07.779 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-26T05:46:08.019 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-26T05:46:08.032 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-26T05:46:08.047 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-26T05:46:08.048 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-26T05:46:08.086 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:08.090 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-26T05:46:08.394 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-26T05:46:08.414 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-26T05:46:08.422 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-26T05:46:08.441 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-26T05:46:08.460 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql-client-common (238) ... 2023-10-26T05:46:08.480 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:08.739 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-26T05:46:08.767 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:08.806 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:08.934 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-26T05:46:08.961 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-26T05:46:09.008 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-26T05:46:09.386 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-26T05:46:09.413 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-26T05:46:09.428 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql-common. 2023-10-26T05:46:09.452 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-26T05:46:09.455 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-26T05:46:09.513 INFO:teuthology.orchestra.run.smithi071.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-26T05:46:09.543 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql-common (238) ... 2023-10-26T05:46:09.805 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-26T05:46:09.832 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:09.872 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:10.011 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql-14. 2023-10-26T05:46:10.039 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:10.080 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:10.266 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-26T05:46:10.293 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:10.341 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:10.677 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-26T05:46:10.704 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:10.743 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:10.987 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-26T05:46:11.014 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-10-26T05:46:11.054 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-26T05:46:11.456 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql. 2023-10-26T05:46:11.473 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-26T05:46:11.506 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql (14+238) ... 2023-10-26T05:46:11.557 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-26T05:46:11.585 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-26T05:46:11.632 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-26T05:46:11.717 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql-client. 2023-10-26T05:46:11.733 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-26T05:46:11.775 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql-client (14+238) ... 2023-10-26T05:46:12.020 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-26T05:46:12.047 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-26T05:46:12.077 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package modemmanager. 2023-10-26T05:46:12.086 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-26T05:46:12.104 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-26T05:46:12.176 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:12.406 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package tcl8.6. 2023-10-26T05:46:12.433 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-26T05:46:12.472 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-26T05:46:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package tcl. 2023-10-26T05:46:12.724 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-26T05:46:12.763 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-26T05:46:12.832 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager. 2023-10-26T05:46:12.860 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-26T05:46:12.907 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-26T05:46:13.128 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package wpasupplicant. 2023-10-26T05:46:13.156 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-26T05:46:13.194 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-26T05:46:13.547 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ppp. 2023-10-26T05:46:13.557 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-26T05:46:13.575 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-26T05:46:13.585 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-26T05:46:13.624 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-26T05:46:13.874 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-26T05:46:13.944 INFO:teuthology.orchestra.run.smithi071.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-26T05:46:13.972 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-26T05:46:14.011 INFO:teuthology.orchestra.run.smithi071.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-26T05:46:14.286 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pptp-linux. 2023-10-26T05:46:14.305 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql-client-common (238) ... 2023-10-26T05:46:14.313 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-26T05:46:14.352 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-26T05:46:14.525 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-26T05:46:14.580 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-26T05:46:14.607 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-26T05:46:14.638 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-26T05:46:14.652 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-26T05:46:14.770 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:14.896 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-26T05:46:14.907 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-26T05:46:14.935 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-26T05:46:15.015 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-common (238) ... 2023-10-26T05:46:15.038 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-26T05:46:15.164 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:15.321 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-26T05:46:15.349 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:15.388 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:15.646 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-26T05:46:15.729 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-26T05:46:16.092 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-common. 2023-10-26T05:46:16.108 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-26T05:46:16.108 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-26T05:46:16.120 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-26T05:46:16.185 INFO:teuthology.orchestra.run.smithi086.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-26T05:46:16.215 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-common (238) ... 2023-10-26T05:46:16.615 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-26T05:46:16.716 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-14. 2023-10-26T05:46:16.733 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-26T05:46:16.744 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-26T05:46:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:16.859 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-26T05:46:16.977 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-26T05:46:17.103 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-26T05:46:17.229 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-26T05:46:17.355 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-26T05:46:17.961 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-26T05:46:18.087 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up dump (0.4b47-3) ... 2023-10-26T05:46:18.181 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-26T05:46:18.247 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:18.261 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql. 2023-10-26T05:46:18.290 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-26T05:46:18.336 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql (14+238) ... 2023-10-26T05:46:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-26T05:46:18.613 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client. 2023-10-26T05:46:18.629 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-26T05:46:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client (14+238) ... 2023-10-26T05:46:18.884 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-26T05:46:18.932 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-26T05:46:18.948 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-26T05:46:18.990 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-26T05:46:19.109 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-26T05:46:19.218 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-26T05:46:19.291 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-26T05:46:19.353 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-26T05:46:19.480 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-26T05:46:19.656 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-26T05:46:19.747 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl8.6. 2023-10-26T05:46:19.775 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-26T05:46:19.782 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:19.814 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-26T05:46:19.908 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-26T05:46:20.052 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql-client (14+238) ... 2023-10-26T05:46:20.100 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl. 2023-10-26T05:46:20.127 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-26T05:46:20.153 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-26T05:46:20.171 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-26T05:46:20.569 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package wpasupplicant. 2023-10-26T05:46:20.597 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-26T05:46:20.644 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-26T05:46:20.647 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-26T05:46:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:which means they are currently configured by ifupdown: 2023-10-26T05:46:20.647 INFO:teuthology.orchestra.run.smithi071.stdout:- enp3s0f1 2023-10-26T05:46:20.648 INFO:teuthology.orchestra.run.smithi071.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-26T05:46:20.648 INFO:teuthology.orchestra.run.smithi071.stdout:remove their configuration from /etc/network/interfaces. 2023-10-26T05:46:20.648 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:20.771 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-26T05:46:21.040 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-26T05:46:21.056 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-26T05:46:21.098 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-26T05:46:21.109 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-26T05:46:21.475 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-26T05:46:21.476 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-26T05:46:21.504 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-26T05:46:21.551 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-26T05:46:21.846 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-common (238) ... 2023-10-26T05:46:22.022 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:22.057 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-26T05:46:22.157 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-26T05:46:22.183 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-26T05:46:22.258 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-26T05:46:22.292 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:22.368 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql-common (238) ... 2023-10-26T05:46:22.402 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-26T05:46:22.528 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-26T05:46:22.655 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:23.111 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-26T05:46:23.185 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-26T05:46:23.253 INFO:teuthology.orchestra.run.smithi071.stdout:Adding user postgres to group ssl-cert 2023-10-26T05:46:23.458 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:23.458 INFO:teuthology.orchestra.run.smithi071.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-26T05:46:23.626 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-26T05:46:23.627 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-26T05:46:23.633 INFO:teuthology.orchestra.run.smithi071.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-26T05:46:23.633 INFO:teuthology.orchestra.run.smithi071.stdout:Removing obsolete dictionary files: 2023-10-26T05:46:24.222 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-26T05:46:24.266 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-26T05:46:24.356 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-10-26T05:46:24.491 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-26T05:46:24.617 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-26T05:46:24.743 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-26T05:46:24.779 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:24.885 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-26T05:46:25.020 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-26T05:46:25.102 INFO:teuthology.orchestra.run.smithi071.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-26T05:46:25.124 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-26T05:46:25.171 INFO:teuthology.orchestra.run.smithi071.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-26T05:46:25.171 INFO:teuthology.orchestra.run.smithi071.stdout:This user must also own the server process. 2023-10-26T05:46:25.171 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:25.171 INFO:teuthology.orchestra.run.smithi071.stdout:The database cluster will be initialized with locale "en_US". 2023-10-26T05:46:25.172 INFO:teuthology.orchestra.run.smithi071.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-26T05:46:25.172 INFO:teuthology.orchestra.run.smithi071.stdout:The default text search configuration will be set to "english". 2023-10-26T05:46:25.172 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:25.172 INFO:teuthology.orchestra.run.smithi071.stdout:Data page checksums are disabled. 2023-10-26T05:46:25.172 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:25.173 INFO:teuthology.orchestra.run.smithi071.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-26T05:46:25.173 INFO:teuthology.orchestra.run.smithi071.stdout:creating subdirectories ... ok 2023-10-26T05:46:25.173 INFO:teuthology.orchestra.run.smithi071.stdout:selecting dynamic shared memory implementation ... posix 2023-10-26T05:46:25.186 INFO:teuthology.orchestra.run.smithi071.stdout:selecting default max_connections ... 100 2023-10-26T05:46:25.204 INFO:teuthology.orchestra.run.smithi071.stdout:selecting default shared_buffers ... 128MB 2023-10-26T05:46:25.213 INFO:teuthology.orchestra.run.smithi071.stdout:selecting default time zone ... Etc/UTC 2023-10-26T05:46:25.214 INFO:teuthology.orchestra.run.smithi071.stdout:creating configuration files ... ok 2023-10-26T05:46:25.329 INFO:teuthology.orchestra.run.smithi071.stdout:running bootstrap script ... ok 2023-10-26T05:46:25.784 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-26T05:46:25.910 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up dump (0.4b47-3) ... 2023-10-26T05:46:25.992 INFO:teuthology.orchestra.run.smithi071.stdout:performing post-bootstrap initialization ... ok 2023-10-26T05:46:26.004 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-26T05:46:26.070 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:26.246 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-26T05:46:26.397 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-26T05:46:26.548 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-26T05:46:26.599 INFO:teuthology.orchestra.run.smithi071.stdout:syncing data to disk ... ok 2023-10-26T05:46:26.674 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-26T05:46:26.764 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-26T05:46:26.834 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-26T05:46:26.943 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-26T05:46:27.007 INFO:teuthology.orchestra.run.smithi071.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-26T05:46:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-26T05:46:27.246 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:27.372 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-26T05:46:27.540 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client (14+238) ... 2023-10-26T05:46:27.666 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-26T05:46:28.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:28.137 INFO:teuthology.orchestra.run.smithi086.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-26T05:46:28.137 INFO:teuthology.orchestra.run.smithi086.stdout:which means they are currently configured by ifupdown: 2023-10-26T05:46:28.137 INFO:teuthology.orchestra.run.smithi086.stdout:- enp3s0f1 2023-10-26T05:46:28.138 INFO:teuthology.orchestra.run.smithi086.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-26T05:46:28.138 INFO:teuthology.orchestra.run.smithi086.stdout:remove their configuration from /etc/network/interfaces. 2023-10-26T05:46:28.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:28.253 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-26T05:46:28.576 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-26T05:46:28.941 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-26T05:46:29.494 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-26T05:46:29.811 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:29.921 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-26T05:46:30.047 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up postgresql (14+238) ... 2023-10-26T05:46:30.173 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:30.299 INFO:teuthology.orchestra.run.smithi071.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:30.570 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-26T05:46:30.570 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-26T05:46:30.716 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-26T05:46:31.058 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-26T05:46:31.177 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-26T05:46:31.317 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-common (238) ... 2023-10-26T05:46:31.504 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-26T05:46:31.656 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-26T05:46:32.594 INFO:teuthology.orchestra.run.smithi086.stdout:Adding user postgres to group ssl-cert 2023-10-26T05:46:32.873 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:32.873 INFO:teuthology.orchestra.run.smithi086.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-26T05:46:32.918 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-26T05:46:33.026 INFO:teuthology.orchestra.run.smithi071.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-26T05:46:33.061 INFO:teuthology.orchestra.run.smithi086.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-26T05:46:33.061 INFO:teuthology.orchestra.run.smithi086.stdout:Removing obsolete dictionary files: 2023-10-26T05:46:33.638 INFO:teuthology.orchestra.run.smithi071.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-26T05:46:33.642 DEBUG:teuthology.parallel:result is None 2023-10-26T05:46:33.654 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-26T05:46:34.067 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-26T05:46:34.410 INFO:teuthology.orchestra.run.smithi086.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-26T05:46:34.426 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-26T05:46:34.468 INFO:teuthology.orchestra.run.smithi086.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-26T05:46:34.468 INFO:teuthology.orchestra.run.smithi086.stdout:This user must also own the server process. 2023-10-26T05:46:34.468 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:34.468 INFO:teuthology.orchestra.run.smithi086.stdout:The database cluster will be initialized with locale "en_US". 2023-10-26T05:46:34.469 INFO:teuthology.orchestra.run.smithi086.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-26T05:46:34.469 INFO:teuthology.orchestra.run.smithi086.stdout:The default text search configuration will be set to "english". 2023-10-26T05:46:34.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:34.469 INFO:teuthology.orchestra.run.smithi086.stdout:Data page checksums are disabled. 2023-10-26T05:46:34.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:34.470 INFO:teuthology.orchestra.run.smithi086.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-26T05:46:34.470 INFO:teuthology.orchestra.run.smithi086.stdout:creating subdirectories ... ok 2023-10-26T05:46:34.470 INFO:teuthology.orchestra.run.smithi086.stdout:selecting dynamic shared memory implementation ... posix 2023-10-26T05:46:34.483 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default max_connections ... 100 2023-10-26T05:46:34.502 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default shared_buffers ... 128MB 2023-10-26T05:46:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default time zone ... Etc/UTC 2023-10-26T05:46:34.509 INFO:teuthology.orchestra.run.smithi086.stdout:creating configuration files ... ok 2023-10-26T05:46:34.621 INFO:teuthology.orchestra.run.smithi086.stdout:running bootstrap script ... ok 2023-10-26T05:46:35.290 INFO:teuthology.orchestra.run.smithi086.stdout:performing post-bootstrap initialization ... ok 2023-10-26T05:46:36.155 INFO:teuthology.orchestra.run.smithi086.stdout:syncing data to disk ... ok 2023-10-26T05:46:36.581 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-26T05:46:39.408 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:39.518 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-26T05:46:39.660 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql (14+238) ... 2023-10-26T05:46:39.786 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-26T05:46:39.913 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-26T05:46:40.169 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-26T05:46:40.169 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-26T05:46:40.832 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-26T05:46:41.134 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-26T05:46:41.294 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-26T05:46:42.420 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-26T05:46:42.512 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-26T05:46:43.127 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-26T05:46:43.130 DEBUG:teuthology.parallel:result is None 2023-10-26T05:46:43.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T05:46:43.274 DEBUG:teuthology.orchestra.run.smithi071:> dpkg-query -W -f '${Version}' ceph 2023-10-26T05:46:43.294 INFO:teuthology.orchestra.run.smithi071.stdout:18.0.0-6863-g38499368-1jammy 2023-10-26T05:46:43.296 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863-g38499368-1jammy 2023-10-26T05:46:43.297 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863-g38499368-1jammy is installed. 2023-10-26T05:46:43.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=384993681abceb8cb8affea10538d607052db4b4 2023-10-26T05:46:43.440 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-query -W -f '${Version}' ceph 2023-10-26T05:46:43.460 INFO:teuthology.orchestra.run.smithi086.stdout:18.0.0-6863-g38499368-1jammy 2023-10-26T05:46:43.460 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6863-g38499368-1jammy 2023-10-26T05:46:43.460 INFO:teuthology.task.install:The correct ceph version 18.0.0-6863-g38499368-1jammy is installed. 2023-10-26T05:46:43.461 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-26T05:46:43.461 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:43.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T05:46:43.476 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:43.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-26T05:46:43.517 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-26T05:46:43.518 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:43.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T05:46:43.532 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T05:46:43.587 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:43.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2023-10-26T05:46:43.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-26T05:46:43.656 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-26T05:46:43.657 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:43.657 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T05:46:43.670 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T05:46:43.723 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:43.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-26T05:46:43.736 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-26T05:46:43.791 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-26T05:46:43.792 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:43.792 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T05:46:43.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T05:46:43.859 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:43.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2023-10-26T05:46:43.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-26T05:46:43.928 INFO:teuthology.run_tasks:Running task ceph... 2023-10-26T05:46:43.996 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-26T05:46:43.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /var/log/ceph 2023-10-26T05:46:44.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2023-10-26T05:46:44.013 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-26T05:46:44.013 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-26T05:46:44.054 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-26T05:46:44.069 INFO:tasks.ceph:Creating extra log directories... 2023-10-26T05:46:44.069 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-26T05:46:44.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-26T05:46:44.130 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-26T05:46:44.130 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-26T05:46:44.130 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-26T05:46:44.131 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:44.131 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-26T05:46:44.165 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-26T05:46:44.239 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:44.240 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-26T05:46:44.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-26T05:46:44.325 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-26T05:46:44.325 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-26T05:46:44.325 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445', 'branch': 'wip-batrick-testing-20231024.144545', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7438445', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20231024.144545'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231024.144545', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 65, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7595, 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'sleep_before_teardown': 0, 'subset': '66/256', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20231024.144545', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_384993681abceb8cb8affea10538d607052db4b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '384993681abceb8cb8affea10538d607052db4b4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi071.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtb4fp23k+VlYx1uYJYTCE44p699TI4MG/A7y/cOvNDnxwF8mkBhrRrS9zGYanOJNq36Q0kdVZ9kdcE65mBQbo=', 'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISjMIFjx/GgWg+LUaxrD5urfkmsGBl6ukeyfx7q03DIKhzwI112qkd5pvKzNU+5gyaKcd+pSCE5dcZ++k7lSQQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '384993681abceb8cb8affea10538d607052db4b4'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '384993681abceb8cb8affea10538d607052db4b4', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '202b180cb047e798fb131047314a862593f45403', 'timestamp': '2023-10-26_05:21:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.770726'} 2023-10-26T05:46:44.326 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-26T05:46:44.330 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-26T05:46:44.369 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m0777 -- /var/run/ceph 2023-10-26T05:46:44.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2023-10-26T05:46:44.424 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:44.424 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T05:46:44.437 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T05:46:44.437 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2023-10-26T05:46:44.485 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T05:46:44.485 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:44.486 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 887 Links: 1 2023-10-26T05:46:44.486 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:44.486 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2023-10-26 05:40:40.044950902 +0000 2023-10-26T05:46:44.486 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2023-10-26 05:40:39.728953709 +0000 2023-10-26T05:46:44.487 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2023-10-26 05:40:39.728953709 +0000 2023-10-26T05:46:44.487 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2023-10-26T05:46:44.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T05:46:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2023-10-26T05:46:44.539 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2023-10-26T05:46:44.540 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000433757 s, 1.2 MB/s 2023-10-26T05:46:44.541 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T05:46:44.590 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2023-10-26T05:46:44.637 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T05:46:44.637 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:44.637 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-10-26T05:46:44.638 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:44.638 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2023-10-26 05:40:40.652945499 +0000 2023-10-26T05:46:44.638 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2023-10-26 05:40:40.308948556 +0000 2023-10-26T05:46:44.638 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2023-10-26 05:40:40.308948556 +0000 2023-10-26T05:46:44.638 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2023-10-26T05:46:44.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T05:46:44.691 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2023-10-26T05:46:44.691 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2023-10-26T05:46:44.691 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000380349 s, 1.3 MB/s 2023-10-26T05:46:44.692 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T05:46:44.742 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2023-10-26T05:46:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T05:46:44.789 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:44.789 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 920 Links: 1 2023-10-26T05:46:44.790 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:44.790 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2023-10-26 05:40:41.272939991 +0000 2023-10-26T05:46:44.790 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2023-10-26 05:40:40.908943225 +0000 2023-10-26T05:46:44.790 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2023-10-26 05:40:40.908943225 +0000 2023-10-26T05:46:44.791 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2023-10-26T05:46:44.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T05:46:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2023-10-26T05:46:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2023-10-26T05:46:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000442516 s, 1.2 MB/s 2023-10-26T05:46:44.844 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T05:46:44.894 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2023-10-26T05:46:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T05:46:44.941 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:44.941 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 938 Links: 1 2023-10-26T05:46:44.942 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:44.942 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2023-10-26 05:40:41.528937717 +0000 2023-10-26T05:46:44.942 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2023-10-26 05:40:41.524937751 +0000 2023-10-26T05:46:44.942 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2023-10-26 05:40:41.524937751 +0000 2023-10-26T05:46:44.942 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2023-10-26T05:46:44.943 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T05:46:44.995 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2023-10-26T05:46:44.995 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2023-10-26T05:46:44.995 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000410456 s, 1.2 MB/s 2023-10-26T05:46:44.996 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T05:46:45.046 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:45.046 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:45.047 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2023-10-26T05:46:45.052 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-26T05:46:45.053 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2023-10-26T05:46:45.101 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-26T05:46:45.101 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:45.102 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 887 Links: 1 2023-10-26T05:46:45.102 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:45.102 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-26 05:40:44.267623404 +0000 2023-10-26T05:46:45.102 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-26 05:40:43.895626722 +0000 2023-10-26T05:46:45.103 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-26 05:40:43.895626722 +0000 2023-10-26T05:46:45.103 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-26T05:46:45.103 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-26T05:46:45.155 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-26T05:46:45.156 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-26T05:46:45.156 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000403701 s, 1.3 MB/s 2023-10-26T05:46:45.157 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-26T05:46:45.206 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2023-10-26T05:46:45.252 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-26T05:46:45.253 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:45.253 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-10-26T05:46:45.253 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:45.253 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-26 05:40:44.831618372 +0000 2023-10-26T05:46:45.253 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-26 05:40:44.487621441 +0000 2023-10-26T05:46:45.254 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-26 05:40:44.487621441 +0000 2023-10-26T05:46:45.254 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-26T05:46:45.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-26T05:46:45.307 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-26T05:46:45.307 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-26T05:46:45.307 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000390275 s, 1.3 MB/s 2023-10-26T05:46:45.308 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-26T05:46:45.358 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2023-10-26T05:46:45.405 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-26T05:46:45.405 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:45.405 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 923 Links: 1 2023-10-26T05:46:45.405 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:45.406 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-26 05:40:45.431613019 +0000 2023-10-26T05:46:45.406 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-26 05:40:45.087616087 +0000 2023-10-26T05:46:45.406 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-26 05:40:45.087616087 +0000 2023-10-26T05:46:45.406 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-26T05:46:45.407 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-26T05:46:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-26T05:46:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-26T05:46:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000409747 s, 1.2 MB/s 2023-10-26T05:46:45.460 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-26T05:46:45.510 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2023-10-26T05:46:45.556 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-26T05:46:45.557 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-26T05:46:45.557 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 938 Links: 1 2023-10-26T05:46:45.557 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-26T05:46:45.558 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-26 05:40:46.047607524 +0000 2023-10-26T05:46:45.558 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-26 05:40:45.667610915 +0000 2023-10-26T05:46:45.558 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-26 05:40:45.667610915 +0000 2023-10-26T05:46:45.558 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-26T05:46:45.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-26T05:46:45.610 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-26T05:46:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-26T05:46:45.611 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000302157 s, 1.7 MB/s 2023-10-26T05:46:45.612 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-26T05:46:45.662 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:45.662 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-26T05:46:45.663 INFO:tasks.ceph:Generating config... 2023-10-26T05:46:45.664 INFO:tasks.ceph:[client] pid file = 2023-10-26T05:46:45.664 INFO:tasks.ceph:[client.0] debug client = 20 2023-10-26T05:46:45.664 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-10-26T05:46:45.665 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-10-26T05:46:45.665 INFO:tasks.ceph:[client.1] debug client = 20 2023-10-26T05:46:45.665 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-10-26T05:46:45.666 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-10-26T05:46:45.666 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-26T05:46:45.666 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-26T05:46:45.666 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-26T05:46:45.667 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-26T05:46:45.667 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-26T05:46:45.667 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-10-26T05:46:45.667 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-10-26T05:46:45.668 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-10-26T05:46:45.668 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-26T05:46:45.668 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-26T05:46:45.668 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-26T05:46:45.669 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-26T05:46:45.669 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-26T05:46:45.669 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-26T05:46:45.670 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-26T05:46:45.670 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-10-26T05:46:45.670 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-26T05:46:45.670 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-26T05:46:45.671 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-26T05:46:45.671 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-10-26T05:46:45.671 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-10-26T05:46:45.672 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-10-26T05:46:45.672 INFO:tasks.ceph:Setting up mon.a... 2023-10-26T05:46:45.672 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-26T05:46:45.707 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.keyring 2023-10-26T05:46:45.717 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-26T05:46:45.794 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-26T05:46:45.848 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.71'), ('mon.b', '172.21.15.86'), ('mon.c', '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]')] 2023-10-26T05:46:45.849 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.71,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}} 2023-10-26T05:46:45.851 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:45.853 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-26T05:46:45.897 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.71 --add b 172.21.15.86 --addv c '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:45.982 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:45.983 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: generated fsid e2c462c9-9202-452f-a70c-a4a81693dc19 2023-10-26T05:46:45.983 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = pacific 2023-10-26T05:46:45.983 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 0 2023-10-26T05:46:45.983 INFO:teuthology.orchestra.run.smithi071.stdout:fsid e2c462c9-9202-452f-a70c-a4a81693dc19 2023-10-26T05:46:45.984 INFO:teuthology.orchestra.run.smithi071.stdout:last_changed 2023-10-26T05:46:45.980971+0000 2023-10-26T05:46:45.984 INFO:teuthology.orchestra.run.smithi071.stdout:created 2023-10-26T05:46:45.980971+0000 2023-10-26T05:46:45.984 INFO:teuthology.orchestra.run.smithi071.stdout:min_mon_release 16 (pacific) 2023-10-26T05:46:45.984 INFO:teuthology.orchestra.run.smithi071.stdout:election_strategy: 1 2023-10-26T05:46:45.984 INFO:teuthology.orchestra.run.smithi071.stdout:0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2023-10-26T05:46:45.985 INFO:teuthology.orchestra.run.smithi071.stdout:1: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2023-10-26T05:46:45.985 INFO:teuthology.orchestra.run.smithi071.stdout:2: [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] mon.c 2023-10-26T05:46:45.985 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-26T05:46:45.986 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-26T05:46:46.032 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e2c462c9-9202-452f-a70c-a4a81693dc19... 2023-10-26T05:46:46.033 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-26T05:46:46.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-26T05:46:46.106 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2023-10-26T05:46:46.106 INFO:teuthology.orchestra.run.smithi071.stdout: chdir = "" 2023-10-26T05:46:46.107 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-26T05:46:46.107 INFO:teuthology.orchestra.run.smithi071.stdout: auth supported = cephx 2023-10-26T05:46:46.107 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.107 INFO:teuthology.orchestra.run.smithi071.stdout: filestore xattr use omap = true 2023-10-26T05:46:46.107 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.108 INFO:teuthology.orchestra.run.smithi071.stdout: mon clock drift allowed = 1.000 2023-10-26T05:46:46.108 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.108 INFO:teuthology.orchestra.run.smithi071.stdout: osd crush chooseleaf type = 0 2023-10-26T05:46:46.108 INFO:teuthology.orchestra.run.smithi071.stdout: auth debug = true 2023-10-26T05:46:46.108 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.109 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on old message = true 2023-10-26T05:46:46.109 INFO:teuthology.orchestra.run.smithi071.stdout: ms die on bug = true 2023-10-26T05:46:46.109 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.109 INFO:teuthology.orchestra.run.smithi071.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-26T05:46:46.109 INFO:teuthology.orchestra.run.smithi071.stdout: mon pg warn max object skew = 0 2023-10-26T05:46:46.109 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.110 INFO:teuthology.orchestra.run.smithi071.stdout: # disable pg_autoscaler by default for new pools 2023-10-26T05:46:46.110 INFO:teuthology.orchestra.run.smithi071.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-26T05:46:46.110 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.110 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default size = 2 2023-10-26T05:46:46.110 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.111 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow primary affinity = true 2023-10-26T05:46:46.111 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd allow pg remap = true 2023-10-26T05:46:46.111 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on legacy crush tunables = false 2023-10-26T05:46:46.111 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on crush straw calc version zero = false 2023-10-26T05:46:46.111 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on no sortbitwise = false 2023-10-26T05:46:46.111 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on osd down out interval zero = false 2023-10-26T05:46:46.112 INFO:teuthology.orchestra.run.smithi071.stdout: mon warn on too few osds = false 2023-10-26T05:46:46.112 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-26T05:46:46.112 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-26T05:46:46.112 INFO:teuthology.orchestra.run.smithi071.stdout: mon_allow_pool_size_one = true 2023-10-26T05:46:46.112 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.112 INFO:teuthology.orchestra.run.smithi071.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-26T05:46:46.113 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.113 INFO:teuthology.orchestra.run.smithi071.stdout: osd default data pool replay window = 5 2023-10-26T05:46:46.113 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.114 INFO:teuthology.orchestra.run.smithi071.stdout: mon allow pool delete = true 2023-10-26T05:46:46.114 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.114 INFO:teuthology.orchestra.run.smithi071.stdout: mon cluster log file level = debug 2023-10-26T05:46:46.114 INFO:teuthology.orchestra.run.smithi071.stdout: debug asserts on shutdown = true 2023-10-26T05:46:46.114 INFO:teuthology.orchestra.run.smithi071.stdout: mon health detail to clog = false 2023-10-26T05:46:46.115 INFO:teuthology.orchestra.run.smithi071.stdout: mon host = "172.21.15.71,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2023-10-26T05:46:46.115 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = e2c462c9-9202-452f-a70c-a4a81693dc19 2023-10-26T05:46:46.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.115 INFO:teuthology.orchestra.run.smithi071.stdout:[osd] 2023-10-26T05:46:46.115 INFO:teuthology.orchestra.run.smithi071.stdout: osd journal size = 100 2023-10-26T05:46:46.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.116 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub load threshold = 5.0 2023-10-26T05:46:46.116 INFO:teuthology.orchestra.run.smithi071.stdout: osd scrub max interval = 600 2023-10-26T05:46:46.116 INFO:teuthology.orchestra.run.smithi071.stdout: osd mclock profile = high_recovery_ops 2023-10-26T05:46:46.116 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.116 INFO:teuthology.orchestra.run.smithi071.stdout: osd recover clone overlap = true 2023-10-26T05:46:46.117 INFO:teuthology.orchestra.run.smithi071.stdout: osd recovery max chunk = 1048576 2023-10-26T05:46:46.117 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.117 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug shutdown = true 2023-10-26T05:46:46.117 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug op order = true 2023-10-26T05:46:46.117 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug verify stray on activate = true 2023-10-26T05:46:46.117 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.118 INFO:teuthology.orchestra.run.smithi071.stdout: osd open classes on start = true 2023-10-26T05:46:46.118 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug pg log writeout = true 2023-10-26T05:46:46.118 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.118 INFO:teuthology.orchestra.run.smithi071.stdout: osd deep scrub update digest min age = 30 2023-10-26T05:46:46.118 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.119 INFO:teuthology.orchestra.run.smithi071.stdout: osd map max advance = 10 2023-10-26T05:46:46.119 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.119 INFO:teuthology.orchestra.run.smithi071.stdout: journal zero on create = true 2023-10-26T05:46:46.119 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.119 INFO:teuthology.orchestra.run.smithi071.stdout: filestore ondisk finisher threads = 3 2023-10-26T05:46:46.119 INFO:teuthology.orchestra.run.smithi071.stdout: filestore apply finisher threads = 3 2023-10-26T05:46:46.120 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.120 INFO:teuthology.orchestra.run.smithi071.stdout: bdev debug aio = true 2023-10-26T05:46:46.120 INFO:teuthology.orchestra.run.smithi071.stdout: osd debug misdirected ops = true 2023-10-26T05:46:46.120 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore block size = 96636764160 2023-10-26T05:46:46.120 INFO:teuthology.orchestra.run.smithi071.stdout: bluestore fsck on mount = True 2023-10-26T05:46:46.121 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluefs = 20 2023-10-26T05:46:46.121 INFO:teuthology.orchestra.run.smithi071.stdout: debug bluestore = 20 2023-10-26T05:46:46.121 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2023-10-26T05:46:46.121 INFO:teuthology.orchestra.run.smithi071.stdout: debug osd = 20 2023-10-26T05:46:46.121 INFO:teuthology.orchestra.run.smithi071.stdout: debug rocksdb = 10 2023-10-26T05:46:46.122 INFO:teuthology.orchestra.run.smithi071.stdout: memstore device bytes = 200000000 2023-10-26T05:46:46.122 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd backfillfull_ratio = 0.85 2023-10-26T05:46:46.122 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd full ratio = 0.9 2023-10-26T05:46:46.122 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd nearfull ratio = 0.8 2023-10-26T05:46:46.122 INFO:teuthology.orchestra.run.smithi071.stdout: osd failsafe full ratio = 1.0 2023-10-26T05:46:46.123 INFO:teuthology.orchestra.run.smithi071.stdout: osd mon report interval = 5 2023-10-26T05:46:46.123 INFO:teuthology.orchestra.run.smithi071.stdout: osd objectstore = memstore 2023-10-26T05:46:46.123 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.123 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr] 2023-10-26T05:46:46.124 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2023-10-26T05:46:46.124 INFO:teuthology.orchestra.run.smithi071.stdout: debug mgr = 20 2023-10-26T05:46:46.124 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2023-10-26T05:46:46.124 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2023-10-26T05:46:46.124 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2023-10-26T05:46:46.125 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2023-10-26T05:46:46.125 INFO:teuthology.orchestra.run.smithi071.stdout: mgr/telemetry/nag = false 2023-10-26T05:46:46.125 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.125 INFO:teuthology.orchestra.run.smithi071.stdout:[mon] 2023-10-26T05:46:46.125 INFO:teuthology.orchestra.run.smithi071.stdout: debug ms = 1 2023-10-26T05:46:46.126 INFO:teuthology.orchestra.run.smithi071.stdout: debug mon = 20 2023-10-26T05:46:46.126 INFO:teuthology.orchestra.run.smithi071.stdout: debug paxos = 20 2023-10-26T05:46:46.126 INFO:teuthology.orchestra.run.smithi071.stdout: debug auth = 20 2023-10-26T05:46:46.126 INFO:teuthology.orchestra.run.smithi071.stdout: mon data avail warn = 5 2023-10-26T05:46:46.126 INFO:teuthology.orchestra.run.smithi071.stdout: mon mgr mkfs grace = 240 2023-10-26T05:46:46.127 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min pgs per osd = 4 2023-10-26T05:46:46.127 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd reporter subtree level = osd 2023-10-26T05:46:46.127 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd prime pg temp = true 2023-10-26T05:46:46.127 INFO:teuthology.orchestra.run.smithi071.stdout: mon reweight min bytes per osd = 10 2023-10-26T05:46:46.127 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.127 INFO:teuthology.orchestra.run.smithi071.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-26T05:46:46.128 INFO:teuthology.orchestra.run.smithi071.stdout: auth mon ticket ttl = 660 # 11m 2023-10-26T05:46:46.128 INFO:teuthology.orchestra.run.smithi071.stdout: auth service ticket ttl = 240 # 4m 2023-10-26T05:46:46.128 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.128 INFO:teuthology.orchestra.run.smithi071.stdout: # don't complain about insecure global_id in the test suite 2023-10-26T05:46:46.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-26T05:46:46.128 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-26T05:46:46.129 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.129 INFO:teuthology.orchestra.run.smithi071.stdout: # 1m isn't quite enough 2023-10-26T05:46:46.129 INFO:teuthology.orchestra.run.smithi071.stdout: mon_down_mkfs_grace = 2m 2023-10-26T05:46:46.129 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.129 INFO:teuthology.orchestra.run.smithi071.stdout: mon_warn_on_filestore_osds = false 2023-10-26T05:46:46.130 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd backfillfull ratio = 0.6 2023-10-26T05:46:46.130 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd full ratio = 0.7 2023-10-26T05:46:46.130 INFO:teuthology.orchestra.run.smithi071.stdout: mon osd nearfull ratio = 0.6 2023-10-26T05:46:46.130 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:46:46.131 INFO:teuthology.orchestra.run.smithi071.stdout:[client] 2023-10-26T05:46:46.131 INFO:teuthology.orchestra.run.smithi071.stdout: rgw cache enabled = true 2023-10-26T05:46:46.131 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable ops log = true 2023-10-26T05:46:46.131 INFO:teuthology.orchestra.run.smithi071.stdout: rgw enable usage log = true 2023-10-26T05:46:46.131 INFO:teuthology.orchestra.run.smithi071.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-26T05:46:46.131 INFO:teuthology.orchestra.run.smithi071.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-26T05:46:46.132 INFO:teuthology.orchestra.run.smithi071.stdout: pid file = "" 2023-10-26T05:46:46.132 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.a] 2023-10-26T05:46:46.132 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.b] 2023-10-26T05:46:46.132 INFO:teuthology.orchestra.run.smithi071.stdout:[mon.c] 2023-10-26T05:46:46.132 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a] 2023-10-26T05:46:46.133 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c] 2023-10-26T05:46:46.133 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b] 2023-10-26T05:46:46.133 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d] 2023-10-26T05:46:46.133 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:46:46.133 INFO:teuthology.orchestra.run.smithi071.stdout: debug client = 20 2023-10-26T05:46:46.133 INFO:teuthology.orchestra.run.smithi071.stdout: debug objectcacher = 20 2023-10-26T05:46:46.134 INFO:teuthology.orchestra.run.smithi071.stdout: debug objecter = 20 2023-10-26T05:46:46.134 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2023-10-26T05:46:46.134 INFO:teuthology.orchestra.run.smithi071.stdout: debug client = 20 2023-10-26T05:46:46.134 INFO:teuthology.orchestra.run.smithi071.stdout: debug objectcacher = 20 2023-10-26T05:46:46.134 INFO:teuthology.orchestra.run.smithi071.stdout: debug objecter = 20 2023-10-26T05:46:46.136 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2023-10-26T05:46:46.136 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2023-10-26T05:46:46.136 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-26T05:46:46.137 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2023-10-26T05:46:46.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.137 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2023-10-26T05:46:46.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2023-10-26T05:46:46.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.138 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2023-10-26T05:46:46.138 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2023-10-26T05:46:46.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.138 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2023-10-26T05:46:46.139 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2023-10-26T05:46:46.139 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.139 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-26T05:46:46.139 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2023-10-26T05:46:46.139 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.140 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2023-10-26T05:46:46.140 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-26T05:46:46.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.140 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2023-10-26T05:46:46.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2023-10-26T05:46:46.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2023-10-26T05:46:46.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2023-10-26T05:46:46.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2023-10-26T05:46:46.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2023-10-26T05:46:46.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2023-10-26T05:46:46.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2023-10-26T05:46:46.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-26T05:46:46.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-26T05:46:46.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2023-10-26T05:46:46.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.143 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-26T05:46:46.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.143 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2023-10-26T05:46:46.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2023-10-26T05:46:46.144 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2023-10-26T05:46:46.144 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2023-10-26T05:46:46.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2023-10-26T05:46:46.145 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.71,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2023-10-26T05:46:46.145 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = e2c462c9-9202-452f-a70c-a4a81693dc19 2023-10-26T05:46:46.145 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.145 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2023-10-26T05:46:46.145 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2023-10-26T05:46:46.146 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.146 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2023-10-26T05:46:46.146 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2023-10-26T05:46:46.146 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2023-10-26T05:46:46.146 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.146 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2023-10-26T05:46:46.147 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2023-10-26T05:46:46.147 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.147 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2023-10-26T05:46:46.147 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2023-10-26T05:46:46.147 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2023-10-26T05:46:46.148 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.148 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2023-10-26T05:46:46.148 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2023-10-26T05:46:46.148 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.148 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2023-10-26T05:46:46.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.149 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2023-10-26T05:46:46.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.149 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2023-10-26T05:46:46.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.149 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2023-10-26T05:46:46.150 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2023-10-26T05:46:46.150 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.150 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2023-10-26T05:46:46.150 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2023-10-26T05:46:46.150 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2023-10-26T05:46:46.151 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2023-10-26T05:46:46.151 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 20 2023-10-26T05:46:46.151 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 20 2023-10-26T05:46:46.151 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-26T05:46:46.151 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2023-10-26T05:46:46.152 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 10 2023-10-26T05:46:46.152 INFO:teuthology.orchestra.run.smithi086.stdout: memstore device bytes = 200000000 2023-10-26T05:46:46.152 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2023-10-26T05:46:46.152 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2023-10-26T05:46:46.152 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2023-10-26T05:46:46.153 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 1.0 2023-10-26T05:46:46.153 INFO:teuthology.orchestra.run.smithi086.stdout: osd mon report interval = 5 2023-10-26T05:46:46.153 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = memstore 2023-10-26T05:46:46.153 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.153 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2023-10-26T05:46:46.153 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-26T05:46:46.154 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2023-10-26T05:46:46.154 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2023-10-26T05:46:46.154 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2023-10-26T05:46:46.154 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2023-10-26T05:46:46.154 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2023-10-26T05:46:46.155 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2023-10-26T05:46:46.155 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.155 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2023-10-26T05:46:46.155 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-26T05:46:46.155 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2023-10-26T05:46:46.156 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2023-10-26T05:46:46.156 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2023-10-26T05:46:46.156 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2023-10-26T05:46:46.156 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2023-10-26T05:46:46.156 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2023-10-26T05:46:46.156 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2023-10-26T05:46:46.157 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2023-10-26T05:46:46.157 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2023-10-26T05:46:46.157 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.157 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-26T05:46:46.157 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2023-10-26T05:46:46.158 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2023-10-26T05:46:46.158 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.158 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2023-10-26T05:46:46.158 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-26T05:46:46.158 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-26T05:46:46.158 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.159 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2023-10-26T05:46:46.159 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2023-10-26T05:46:46.159 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.159 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2023-10-26T05:46:46.159 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull ratio = 0.6 2023-10-26T05:46:46.160 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.7 2023-10-26T05:46:46.160 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.6 2023-10-26T05:46:46.160 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-26T05:46:46.160 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2023-10-26T05:46:46.160 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2023-10-26T05:46:46.160 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2023-10-26T05:46:46.161 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2023-10-26T05:46:46.161 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-26T05:46:46.161 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-26T05:46:46.161 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2023-10-26T05:46:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2023-10-26T05:46:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2023-10-26T05:46:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2023-10-26T05:46:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2023-10-26T05:46:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2023-10-26T05:46:46.162 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2023-10-26T05:46:46.163 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2023-10-26T05:46:46.163 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2023-10-26T05:46:46.163 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2023-10-26T05:46:46.163 INFO:teuthology.orchestra.run.smithi086.stdout: debug objectcacher = 20 2023-10-26T05:46:46.164 INFO:teuthology.orchestra.run.smithi086.stdout: debug objecter = 20 2023-10-26T05:46:46.164 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2023-10-26T05:46:46.164 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2023-10-26T05:46:46.165 INFO:teuthology.orchestra.run.smithi086.stdout: debug objectcacher = 20 2023-10-26T05:46:46.165 INFO:teuthology.orchestra.run.smithi086.stdout: debug objecter = 20 2023-10-26T05:46:46.165 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-26T05:46:46.165 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-26T05:46:46.229 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-26T05:46:46.230 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:46.230 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-26T05:46:46.236 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:46.236 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-26T05:46:46.285 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:46:46.285 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:46.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-26T05:46:46.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-26T05:46:46.350 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:46.350 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:46.397 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:46:46.397 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:46.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-26T05:46:46.398 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-26T05:46:46.420 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:46.421 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:46.468 INFO:tasks.ceph:Setting up mon nodes... 2023-10-26T05:46:46.469 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-26T05:46:46.469 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-26T05:46:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-26T05:46:46.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-26T05:46:46.558 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-26T05:46:46.567 INFO:tasks.ceph:Setting up mds nodes... 2023-10-26T05:46:46.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-26T05:46:46.607 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-26T05:46:46.609 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-26T05:46:46.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-26T05:46:46.751 INFO:teuthology.orchestra.run.smithi071.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-26T05:46:46.754 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-26T05:46:46.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-26T05:46:46.852 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-26T05:46:46.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-26T05:46:46.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-26T05:46:46.952 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-26T05:46:46.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-26T05:46:47.008 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-26T05:46:47.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-26T05:46:47.045 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-26T05:46:47.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-26T05:46:47.134 INFO:teuthology.orchestra.run.smithi071.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-26T05:46:47.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-26T05:46:47.179 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-26T05:46:47.190 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-26T05:46:47.264 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-26T05:46:47.275 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-26T05:46:47.276 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi071.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-26T05:46:47.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-26T05:46:47.290 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:47.290 INFO:tasks.ceph:role: osd.0 2023-10-26T05:46:47.290 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:46:47.291 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-26T05:46:47.352 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:47.353 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:47.353 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:47.353 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:47.354 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:47.354 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:47.354 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:47.354 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:47.355 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:47.355 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:47.803 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2023-10-26T05:46:47.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2023-10-26T05:46:47.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-26T05:46:47.910 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-26T05:46:47.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:47.921 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:47.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-26T05:46:47.975 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:47.975 INFO:tasks.ceph:role: osd.1 2023-10-26T05:46:47.976 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:46:47.976 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-26T05:46:48.037 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:48.038 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:48.038 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:48.038 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:48.039 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:48.498 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2023-10-26T05:46:48.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2023-10-26T05:46:48.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-26T05:46:48.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-26T05:46:48.653 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:48.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:48.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-26T05:46:48.708 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:48.708 INFO:tasks.ceph:role: osd.2 2023-10-26T05:46:48.709 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:46:48.709 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-26T05:46:48.772 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:48.773 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:48.773 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:48.774 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:48.774 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:48.774 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:48.774 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:49.252 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2023-10-26T05:46:49.254 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2023-10-26T05:46:49.254 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-26T05:46:49.395 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-26T05:46:49.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:49.407 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:49.408 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-26T05:46:49.465 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:49.465 INFO:tasks.ceph:role: osd.3 2023-10-26T05:46:49.465 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:46:49.466 DEBUG:teuthology.orchestra.run.smithi071:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-26T05:46:49.529 INFO:teuthology.orchestra.run.smithi071.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:49.529 INFO:teuthology.orchestra.run.smithi071.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:49.530 INFO:teuthology.orchestra.run.smithi071.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:49.530 INFO:teuthology.orchestra.run.smithi071.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:49.530 INFO:teuthology.orchestra.run.smithi071.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:49.530 INFO:teuthology.orchestra.run.smithi071.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:49.531 INFO:teuthology.orchestra.run.smithi071.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:49.531 INFO:teuthology.orchestra.run.smithi071.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:49.955 INFO:teuthology.orchestra.run.smithi071.stdout:Discarding blocks...Done. 2023-10-26T05:46:49.957 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi071.front.sepia.ceph.com -o noatime 2023-10-26T05:46:49.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-26T05:46:50.045 DEBUG:teuthology.orchestra.run.smithi071:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-26T05:46:50.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:50.056 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:50.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:50.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.164+0000 7f4a31677640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-26T05:46:50.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.168+0000 7f4a31677640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-26T05:46:50.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.172+0000 7f4a31677640 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-10-26T05:46:50.193 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-26T05:46:50.206 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.280+0000 7f5814649640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-26T05:46:50.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.284+0000 7f5814649640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-26T05:46:50.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.284+0000 7f5814649640 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-10-26T05:46:50.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-26T05:46:50.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:50.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.436+0000 7ff374cf1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-26T05:46:50.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.444+0000 7ff374cf1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-26T05:46:50.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.452+0000 7ff374cf1640 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-10-26T05:46:50.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-26T05:46:50.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:50.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.612+0000 7f690e4cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-26T05:46:50.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.620+0000 7f690e4cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-26T05:46:50.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:46:50.628+0000 7f690e4cb640 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-10-26T05:46:50.657 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-26T05:46:50.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-26T05:46:50.727 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:50.728 INFO:tasks.ceph:role: osd.4 2023-10-26T05:46:50.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:46:50.728 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-26T05:46:50.793 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:50.794 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:50.794 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:50.794 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:50.794 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:50.795 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:50.795 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:50.795 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:51.248 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-26T05:46:51.250 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-26T05:46:51.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-26T05:46:51.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-26T05:46:51.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:51.353 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:51.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-26T05:46:51.406 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:51.407 INFO:tasks.ceph:role: osd.5 2023-10-26T05:46:51.407 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:46:51.407 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-26T05:46:51.469 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:51.469 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:51.469 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:51.469 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:51.470 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:51.470 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:51.470 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:51.470 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:51.470 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:51.471 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:51.935 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-26T05:46:51.937 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-26T05:46:51.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-26T05:46:52.094 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-26T05:46:52.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:52.107 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:52.107 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-26T05:46:52.160 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:52.161 INFO:tasks.ceph:role: osd.6 2023-10-26T05:46:52.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:46:52.161 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-26T05:46:52.224 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:52.224 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:52.225 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:52.225 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:52.225 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:52.226 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:52.226 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:52.226 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:52.703 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-26T05:46:52.705 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-26T05:46:52.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-26T05:46:52.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-26T05:46:52.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:52.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-26T05:46:52.912 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-26T05:46:52.912 INFO:tasks.ceph:role: osd.7 2023-10-26T05:46:52.912 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:46:52.913 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-26T05:46:52.974 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-26T05:46:52.975 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-26T05:46:52.975 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-26T05:46:52.975 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-26T05:46:52.975 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-26T05:46:52.976 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-26T05:46:52.976 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-26T05:46:52.976 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-26T05:46:52.976 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-26T05:46:52.976 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-26T05:46:53.398 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-26T05:46:53.400 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-26T05:46:53.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-26T05:46:53.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-26T05:46:53.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:46:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-26T05:46:53.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:53.614 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.613+0000 7fbe3cfaa640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-26T05:46:53.620 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.621+0000 7fbe3cfaa640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-26T05:46:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.621+0000 7fbe3cfaa640 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-10-26T05:46:53.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-26T05:46:53.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:53.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.729+0000 7f97b7b25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-26T05:46:53.731 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.729+0000 7f97b7b25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-26T05:46:53.737 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.737+0000 7f97b7b25640 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-10-26T05:46:53.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-26T05:46:53.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:53.892 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.889+0000 7f87eff3e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-26T05:46:53.895 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.893+0000 7f87eff3e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-26T05:46:53.902 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:53.901+0000 7f87eff3e640 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-10-26T05:46:53.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-26T05:46:53.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:54.061+0000 7f045e3ce640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-26T05:46:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:54.065+0000 7f045e3ce640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-26T05:46:54.073 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:46:54.073+0000 7f045e3ce640 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-10-26T05:46:54.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-26T05:46:54.151 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-26T05:46:54.152 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.152 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-26T05:46:54.167 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-26T05:46:54.221 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.222 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-26T05:46:54.276 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-26T05:46:54.331 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-26T05:46:54.387 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.388 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-26T05:46:54.439 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-26T05:46:54.493 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-26T05:46:54.507 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-26T05:46:54.560 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.561 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-26T05:46:54.615 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.615 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-26T05:46:54.668 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-26T05:46:54.723 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-26T05:46:54.776 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-26T05:46:54.832 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.832 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-26T05:46:54.838 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:46:54.839 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-26T05:46:54.886 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.887 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-26T05:46:54.892 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:46:54.893 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-26T05:46:54.941 INFO:tasks.ceph:Adding keys to all mons... 2023-10-26T05:46:54.941 DEBUG:teuthology.orchestra.run.smithi071:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-26T05:46:54.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-26T05:46:54.986 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.y] 2023-10-26T05:46:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA== 2023-10-26T05:46:54.987 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a] 2023-10-26T05:46:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg== 2023-10-26T05:46:54.987 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c] 2023-10-26T05:46:54.987 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw== 2023-10-26T05:46:54.988 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.0] 2023-10-26T05:46:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg== 2023-10-26T05:46:54.988 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.1] 2023-10-26T05:46:54.988 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw== 2023-10-26T05:46:54.988 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.2] 2023-10-26T05:46:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA== 2023-10-26T05:46:54.989 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.3] 2023-10-26T05:46:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg== 2023-10-26T05:46:54.989 INFO:teuthology.orchestra.run.smithi071.stdout:[mgr.x] 2023-10-26T05:46:54.989 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw== 2023-10-26T05:46:54.990 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b] 2023-10-26T05:46:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA== 2023-10-26T05:46:54.990 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d] 2023-10-26T05:46:54.990 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w== 2023-10-26T05:46:54.990 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.4] 2023-10-26T05:46:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg== 2023-10-26T05:46:54.991 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.5] 2023-10-26T05:46:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w== 2023-10-26T05:46:54.991 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.6] 2023-10-26T05:46:54.991 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA== 2023-10-26T05:46:54.992 INFO:teuthology.orchestra.run.smithi071.stdout:[osd.7] 2023-10-26T05:46:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw== 2023-10-26T05:46:54.992 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:46:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:46:54.992 INFO:teuthology.orchestra.run.smithi071.stdout:[client.2] 2023-10-26T05:46:54.992 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA== 2023-10-26T05:46:54.993 INFO:teuthology.orchestra.run.smithi071.stdout:[client.1] 2023-10-26T05:46:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ== 2023-10-26T05:46:54.993 INFO:teuthology.orchestra.run.smithi071.stdout:[client.3] 2023-10-26T05:46:54.993 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q== 2023-10-26T05:46:54.994 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2023-10-26T05:46:54.995 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA== 2023-10-26T05:46:54.995 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2023-10-26T05:46:54.995 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg== 2023-10-26T05:46:54.995 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2023-10-26T05:46:54.995 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw== 2023-10-26T05:46:54.996 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2023-10-26T05:46:54.996 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg== 2023-10-26T05:46:54.996 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2023-10-26T05:46:54.996 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw== 2023-10-26T05:46:54.996 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2023-10-26T05:46:54.997 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA== 2023-10-26T05:46:54.997 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2023-10-26T05:46:54.997 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg== 2023-10-26T05:46:54.997 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2023-10-26T05:46:54.997 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw== 2023-10-26T05:46:54.998 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2023-10-26T05:46:54.998 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA== 2023-10-26T05:46:54.998 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2023-10-26T05:46:54.998 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w== 2023-10-26T05:46:54.998 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2023-10-26T05:46:54.998 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg== 2023-10-26T05:46:54.999 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2023-10-26T05:46:54.999 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w== 2023-10-26T05:46:54.999 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2023-10-26T05:46:54.999 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA== 2023-10-26T05:46:54.999 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2023-10-26T05:46:55.000 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw== 2023-10-26T05:46:55.000 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2023-10-26T05:46:55.000 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:46:55.000 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2023-10-26T05:46:55.000 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA== 2023-10-26T05:46:55.001 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2023-10-26T05:46:55.001 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ== 2023-10-26T05:46:55.001 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2023-10-26T05:46:55.001 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q== 2023-10-26T05:46:55.002 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T05:46:55.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T05:46:55.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.126 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.172 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.346 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.387 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.559 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T05:46:55.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-26T05:46:55.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.686 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-26T05:46:55.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.858 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.939 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:55.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:56.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:56.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-26T05:46:56.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.226 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-26T05:46:56.439 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-26T05:46:56.439 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-26T05:46:56.491 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-26T05:46:56.584 DEBUG:teuthology.orchestra.run.smithi071:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-26T05:46:56.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-26T05:46:56.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-26T05:46:56.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-26T05:46:56.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-26T05:46:56.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-26T05:46:56.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-26T05:46:56.881 DEBUG:teuthology.orchestra.run.smithi071:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:56.884 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-26T05:46:56.928 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-26T05:46:56.929 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-26T05:46:56.929 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-26T05:46:56.931 INFO:tasks.ceph.mon.a:Started 2023-10-26T05:46:56.932 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-26T05:46:56.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-26T05:46:56.973 INFO:tasks.ceph.mon.b:Started 2023-10-26T05:46:56.974 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-26T05:46:56.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-26T05:46:56.976 INFO:tasks.ceph.mon.c:Started 2023-10-26T05:46:56.976 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-26T05:46:56.977 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-26T05:46:56.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-26T05:46:56.979 INFO:tasks.ceph.mgr.y:Started 2023-10-26T05:46:56.980 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-26T05:46:56.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-26T05:46:56.982 INFO:tasks.ceph.mgr.x:Started 2023-10-26T05:46:56.983 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2023-10-26T05:46:56.983 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2023-10-26T05:46:56.985 INFO:tasks.ceph:setting config [client] debug_client = 20 2023-10-26T05:46:56.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client debug_client 20 2023-10-26T05:46:57.029 INFO:tasks.ceph:setting config [client] debug_ms = 1 2023-10-26T05:46:57.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client debug_ms 1 2023-10-26T05:46:57.077 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2023-10-26T05:46:57.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2023-10-26T05:46:57.125 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2023-10-26T05:46:57.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2023-10-26T05:46:57.173 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2023-10-26T05:46:57.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds debug_mds 20 2023-10-26T05:46:57.217 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2023-10-26T05:46:57.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2023-10-26T05:46:57.265 INFO:tasks.ceph:setting config [mds] debug_ms = 1 2023-10-26T05:46:57.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds debug_ms 1 2023-10-26T05:46:57.315 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2023-10-26T05:46:57.315 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2023-10-26T05:46:57.361 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2023-10-26T05:46:57.361 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2023-10-26T05:46:57.405 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2023-10-26T05:46:57.406 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2023-10-26T05:46:57.449 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2023-10-26T05:46:57.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2023-10-26T05:46:57.493 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2023-10-26T05:46:57.493 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2023-10-26T05:46:57.537 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2023-10-26T05:46:57.538 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2023-10-26T05:46:57.589 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2023-10-26T05:46:57.589 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2023-10-26T05:46:57.633 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2023-10-26T05:46:57.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2023-10-26T05:46:57.677 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2023-10-26T05:46:57.677 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2023-10-26T05:46:57.725 DEBUG:tasks.ceph:set 17 configs 2023-10-26T05:46:57.725 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi071') 2023-10-26T05:47:02.638 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:02.637+0000 7fef6bda1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T05:47:02.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 -- 172.21.15.71:0/3556469199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74000052b0 msgr2=0x7f74180adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:02.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 --2- 172.21.15.71:0/3556469199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74000052b0 0x7f74180adae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7410002240 tx=0x7f7400005800 comp rx=0 tx=0).stop 2023-10-26T05:47:02.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 -- 172.21.15.71:0/3556469199 shutdown_connections 2023-10-26T05:47:02.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 --2- 172.21.15.71:0/3556469199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74000052b0 0x7f74180adae0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 --2- 172.21.15.71:0/3556469199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74000047d0 0x7f7418139f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 --2- 172.21.15.71:0/3556469199 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7400003980 0x7f74180a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 -- 172.21.15.71:0/3556469199 >> 172.21.15.71:0/3556469199 conn(0x7f74180af7d0 msgr2=0x7f741815be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:02.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 -- 172.21.15.71:0/3556469199 shutdown_connections 2023-10-26T05:47:02.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7f741e554640 1 -- 172.21.15.71:0/3556469199 wait complete. 2023-10-26T05:47:02.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.724+0000 7fb4feffd640 1 -- 172.21.15.71:0/3169040871 <== mon.0 v2:172.21.15.71:3300/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fb5000246e0 con 0x7fb4f00072f0 2023-10-26T05:47:02.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.728+0000 7fefbaffd640 1 -- 172.21.15.71:0/3188247509 <== mon.1 v2:172.21.15.86:3300/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7fefb40c36c0 con 0x7fefb00047d0 2023-10-26T05:47:02.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.728+0000 7f036affd640 1 -- 172.21.15.71:0/1984874202 <== mon.1 v2:172.21.15.86:3300/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f0374016b30 con 0x7f037c14e0d0 2023-10-26T05:47:02.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.728+0000 7f675effd640 1 -- 172.21.15.71:0/1566422889 <== mon.2 v2:172.21.15.86:3301/0 10 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7f676401d400 con 0x7f6754003980 2023-10-26T05:47:02.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.728+0000 7fefbaffd640 1 -- 172.21.15.71:0/3188247509 <== mon.1 v2:172.21.15.86:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v13)=0 v13) v1 ==== 114+0+0 (secure 0 0 0) 0x7fefb4006a70 con 0x7fefb00047d0 2023-10-26T05:47:02.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.728+0000 7fefcc767640 1 -- 172.21.15.71:0/3188247509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb00047d0 msgr2=0x7fefc415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:02.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.728+0000 7fefcc767640 1 --2- 172.21.15.71:0/3188247509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb00047d0 0x7fefc415bb30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fefb40a9000 tx=0x7fefb0005800 comp rx=0 tx=0).stop 2023-10-26T05:47:02.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 -- 172.21.15.71:0/3188247509 shutdown_connections 2023-10-26T05:47:02.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 --2- 172.21.15.71:0/3188247509 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fefb0005190 0x7fefc415c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 --2- 172.21.15.71:0/3188247509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefb00047d0 0x7fefc415bb30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 --2- 172.21.15.71:0/3188247509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefb0003980 0x7fefc413c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 -- 172.21.15.71:0/3188247509 >> 172.21.15.71:0/3188247509 conn(0x7fefc40af7d0 msgr2=0x7fefc413db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:02.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 -- 172.21.15.71:0/3188247509 shutdown_connections 2023-10-26T05:47:02.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fefcc767640 1 -- 172.21.15.71:0/3188247509 wait complete. 2023-10-26T05:47:02.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.732+0000 7fb4feffd640 1 -- 172.21.15.71:0/3169040871 <== mon.0 v2:172.21.15.71:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v14)=0 v14) v1 ==== 106+0+0 (secure 0 0 0) 0x7fb5000280b0 con 0x7fb4f00072f0 2023-10-26T05:47:02.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 -- 172.21.15.71:0/3169040871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4f00072f0 msgr2=0x7fb5101ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:02.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 --2- 172.21.15.71:0/3169040871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4f00072f0 0x7fb5101ef9b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c570 tx=0x7fb50000ca30 comp rx=0 tx=0).stop 2023-10-26T05:47:02.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 -- 172.21.15.71:0/3169040871 shutdown_connections 2023-10-26T05:47:02.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 --2- 172.21.15.71:0/3169040871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4f8004d60 0x7fb51015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 --2- 172.21.15.71:0/3169040871 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb4f8003980 0x7fb51014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 --2- 172.21.15.71:0/3169040871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb4f00072f0 0x7fb5101ef9b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 -- 172.21.15.71:0/3169040871 >> 172.21.15.71:0/3169040871 conn(0x7fb51006fd20 msgr2=0x7fb5100cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:02.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 -- 172.21.15.71:0/3169040871 shutdown_connections 2023-10-26T05:47:02.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.736+0000 7fb514e2c640 1 -- 172.21.15.71:0/3169040871 wait complete. 2023-10-26T05:47:02.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f675effd640 1 -- 172.21.15.71:0/1566422889 <== mon.2 v2:172.21.15.86:3301/0 11 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v15)=0 v15) v1 ==== 121+0+0 (secure 0 0 0) 0x7f676401d570 con 0x7f6754003980 2023-10-26T05:47:02.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 -- 172.21.15.71:0/1566422889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6754003980 msgr2=0x7f6768158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:02.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 --2- 172.21.15.71:0/1566422889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6754003980 0x7f6768158c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6764013ad0 tx=0x7f6764013f90 comp rx=0 tx=0).stop 2023-10-26T05:47:02.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 -- 172.21.15.71:0/1566422889 shutdown_connections 2023-10-26T05:47:02.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 --2- 172.21.15.71:0/1566422889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6754005260 0x7f676806b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 --2- 172.21.15.71:0/1566422889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67540047d0 0x7f676806d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 --2- 172.21.15.71:0/1566422889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6754003980 0x7f6768158c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 -- 172.21.15.71:0/1566422889 >> 172.21.15.71:0/1566422889 conn(0x7f67680afe40 msgr2=0x7f67680b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:02.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 -- 172.21.15.71:0/1566422889 shutdown_connections 2023-10-26T05:47:02.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.744+0000 7f6770267640 1 -- 172.21.15.71:0/1566422889 wait complete. 2023-10-26T05:47:02.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.748+0000 7f036affd640 1 -- 172.21.15.71:0/1984874202 <== mon.1 v2:172.21.15.86:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0374016d80 con 0x7f037c14e0d0 2023-10-26T05:47:02.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 -- 172.21.15.71:0/1984874202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037c14e0d0 msgr2=0x7f037c13f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:02.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 --2- 172.21.15.71:0/1984874202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037c14e0d0 0x7f037c13f620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f035c009de0 tx=0x7f03740098d0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 -- 172.21.15.71:0/1984874202 shutdown_connections 2023-10-26T05:47:02.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 --2- 172.21.15.71:0/1984874202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f037c14e0d0 0x7f037c13f620 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 --2- 172.21.15.71:0/1984874202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f037c15bcb0 0x7f037c13cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 --2- 172.21.15.71:0/1984874202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03640047d0 0x7f037c14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:02.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 -- 172.21.15.71:0/1984874202 >> 172.21.15.71:0/1984874202 conn(0x7f037c0caa90 msgr2=0x7f037c0b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:02.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 -- 172.21.15.71:0/1984874202 shutdown_connections 2023-10-26T05:47:02.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:02.752+0000 7f0380a59640 1 -- 172.21.15.71:0/1984874202 wait complete. 2023-10-26T05:47:03.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/2918315834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30008680 msgr2=0x7eff3000d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/2918315834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30008680 0x7eff3000d100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7eff44071270 tx=0x7eff3c01c680 comp rx=0 tx=0).stop 2023-10-26T05:47:03.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/2918315834 shutdown_connections 2023-10-26T05:47:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/2918315834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30008680 0x7eff3000d100 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/2918315834 >> v2:172.21.15.86:3300/0 conn(0x7eff30007d60 0x7eff30008140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/2918315834 >> 172.21.15.71:0/2918315834 conn(0x7eff440afe40 msgr2=0x7eff440b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/2918315834 shutdown_connections 2023-10-26T05:47:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/2918315834 wait complete. 2023-10-26T05:47:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 Processor -- start 2023-10-26T05:47:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- start start 2023-10-26T05:47:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff30003b70 0x7eff4413e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 0x7eff441380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff30008680 0x7eff4413a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4413eac0 con 0x7eff30007d60 2023-10-26T05:47:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff441385e0 con 0x7eff30003b70 2023-10-26T05:47:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4413f0a0 con 0x7eff30008680 2023-10-26T05:47:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 0x7eff441380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 0x7eff441380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45286/0 (socket says 172.21.15.71:45286) 2023-10-26T05:47:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff30003b70 0x7eff4413e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 -- 172.21.15.71:0/3612834730 learned_addr learned my addr 172.21.15.71:0/3612834730 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff48c60640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff30008680 0x7eff4413a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 -- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff30008680 msgr2=0x7eff4413a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff30008680 0x7eff4413a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 -- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff30003b70 msgr2=0x7eff4413e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff30003b70 0x7eff4413e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 -- 172.21.15.71:0/3612834730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff440b2ce0 con 0x7eff30007d60 2023-10-26T05:47:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff437fe640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 0x7eff441380a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff38011ae0 tx=0x7eff38011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38010c50 con 0x7eff30007d60 2023-10-26T05:47:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(4 keys) v1 ==== 110+0+0 (secure 0 0 0) 0x7eff38018070 con 0x7eff30007d60 2023-10-26T05:47:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff441f5100 con 0x7eff30007d60 2023-10-26T05:47:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3801e650 con 0x7eff30007d60 2023-10-26T05:47:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff440c3730 con 0x7eff30007d60 2023-10-26T05:47:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7eff38005020 con 0x7eff30007d60 2023-10-26T05:47:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.104+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff3801e7b0 con 0x7eff30007d60 2023-10-26T05:47:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.108+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff44140480 con 0x7eff30007d60 2023-10-26T05:47:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.108+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7eff380047b0 con 0x7eff30007d60 2023-10-26T05:47:03.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.184+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) v1 -- 0x7eff440c6aa0 con 0x7eff30007d60 2023-10-26T05:47:03.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v17)=0 v17) v1 ==== 123+0+0 (secure 0 0 0) 0x7eff380054b0 con 0x7eff30007d60 2023-10-26T05:47:03.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff40ff9640 1 -- 172.21.15.71:0/3612834730 <== mon.0 v2:172.21.15.71:3300/0 8 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff380056d0 con 0x7eff30007d60 2023-10-26T05:47:03.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 msgr2=0x7eff441380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 0x7eff441380a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7eff38011ae0 tx=0x7eff38011fa0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 shutdown_connections 2023-10-26T05:47:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff30008680 0x7eff4413a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff30007d60 0x7eff441380a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 --2- 172.21.15.71:0/3612834730 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff30003b70 0x7eff4413e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 >> 172.21.15.71:0/3612834730 conn(0x7eff440afe40 msgr2=0x7eff440aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 shutdown_connections 2023-10-26T05:47:03.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.188+0000 7eff4a6c3640 1 -- 172.21.15.71:0/3612834730 wait complete. 2023-10-26T05:47:03.207 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:03.205+0000 7fef6bda1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T05:47:03.211 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.212 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.212 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.213 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.213 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.213 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.214 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '1'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.214 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.214 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.215 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.215 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.215 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.216 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.216 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.217 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.217 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi071') 2023-10-26T05:47:03.273 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:03.268+0000 7fbeae4ae180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-26T05:47:03.400 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:03.397+0000 7fef6bda1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T05:47:03.745 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:03.740+0000 7fbeae4ae180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-26T05:47:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 -- 172.21.15.71:0/3370572329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440003b40 msgr2=0x7f3440006600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 --2- 172.21.15.71:0/3370572329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440003b40 0x7f3440006600 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34540d1260 tx=0x7f344401d520 comp rx=0 tx=0).stop 2023-10-26T05:47:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 -- 172.21.15.71:0/3370572329 shutdown_connections 2023-10-26T05:47:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 --2- 172.21.15.71:0/3370572329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3440007460 0x7f3440007910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 --2- 172.21.15.71:0/3370572329 >> v2:172.21.15.71:3300/0 conn(0x7f3440006b40 0x7f3440006f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 --2- 172.21.15.71:0/3370572329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440003b40 0x7f3440006600 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 -- 172.21.15.71:0/3370572329 >> 172.21.15.71:0/3370572329 conn(0x7f34540cdd80 msgr2=0x7f34540ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.828+0000 7f34598fe640 1 -- 172.21.15.71:0/3370572329 shutdown_connections 2023-10-26T05:47:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.832+0000 7f34598fe640 1 -- 172.21.15.71:0/3370572329 wait complete. 2023-10-26T05:47:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.832+0000 7f34598fe640 1 Processor -- start 2023-10-26T05:47:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.832+0000 7f34598fe640 1 -- start start 2023-10-26T05:47:03.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 0x7f3454158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:03.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3440006b40 0x7f34540caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:03.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440007460 0x7f34540cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345406a650 con 0x7f3440003b40 2023-10-26T05:47:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34540c9ed0 con 0x7f3440007460 2023-10-26T05:47:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34540c9bf0 con 0x7f3440006b40 2023-10-26T05:47:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 0x7f3454158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:03.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 0x7f3454158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45302/0 (socket says 172.21.15.71:45302) 2023-10-26T05:47:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 -- 172.21.15.71:0/1172576289 learned_addr learned my addr 172.21.15.71:0/1172576289 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34537fe640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440007460 0x7f34540cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34527fc640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3440006b40 0x7f34540caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 -- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3440006b40 msgr2=0x7f34540caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3440006b40 0x7f34540caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 -- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440007460 msgr2=0x7f34540cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440007460 0x7f34540cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 -- 172.21.15.71:0/1172576289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454069d40 con 0x7f3440003b40 2023-10-26T05:47:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f3452ffd640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 0x7f3454158440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f344401ddc0 tx=0x7f34440067b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440b4450 con 0x7f3440003b40 2023-10-26T05:47:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34440b46a0 con 0x7f3440003b40 2023-10-26T05:47:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540c6020 con 0x7f3440003b40 2023-10-26T05:47:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440b4a90 con 0x7f3440003b40 2023-10-26T05:47:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34540c65e0 con 0x7f3440003b40 2023-10-26T05:47:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f34440b4bf0 con 0x7f3440003b40 2023-10-26T05:47:03.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f34440a9420 con 0x7f3440003b40 2023-10-26T05:47:03.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.836+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3454066b10 con 0x7f3440003b40 2023-10-26T05:47:03.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.840+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f34440af070 con 0x7f3440003b40 2023-10-26T05:47:03.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.892+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f34540721e0 con 0x7f3440003b40 2023-10-26T05:47:03.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.896+0000 7f343bfff640 1 -- 172.21.15.71:0/1172576289 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v18)=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f34440cd0c0 con 0x7f3440003b40 2023-10-26T05:47:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.896+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 msgr2=0x7f3454158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.896+0000 7f34598fe640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 0x7f3454158440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f344401ddc0 tx=0x7f34440067b0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 shutdown_connections 2023-10-26T05:47:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3440007460 0x7f34540cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3440006b40 0x7f34540caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 --2- 172.21.15.71:0/1172576289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3440003b40 0x7f3454158440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:03.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 >> 172.21.15.71:0/1172576289 conn(0x7f34540cdd80 msgr2=0x7f34540b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:03.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 shutdown_connections 2023-10-26T05:47:03.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:03.900+0000 7f34598fe640 1 -- 172.21.15.71:0/1172576289 wait complete. 2023-10-26T05:47:03.919 INFO:tasks.ceph:Setting crush tunables to default 2023-10-26T05:47:03.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph osd crush tunables default 2023-10-26T05:47:03.929 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:03.924+0000 7fbeae4ae180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-26T05:47:03.992 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:03.993+0000 7fef6bda1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T05:47:04.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 -- 172.21.15.71:0/33617719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 msgr2=0x7fb43c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 --2- 172.21.15.71:0/33617719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 0x7fb43c06dd30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009900 tx=0x7fb42c01c800 comp rx=0 tx=0).stop 2023-10-26T05:47:04.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 -- 172.21.15.71:0/33617719 shutdown_connections 2023-10-26T05:47:04.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 --2- 172.21.15.71:0/33617719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb43c0c9780 0x7fb43c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 --2- 172.21.15.71:0/33617719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 0x7fb43c06dd30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 -- 172.21.15.71:0/33617719 >> 172.21.15.71:0/33617719 conn(0x7fb43c06fd20 msgr2=0x7fb43c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:04.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 -- 172.21.15.71:0/33617719 shutdown_connections 2023-10-26T05:47:04.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.032+0000 7fb44094c640 1 -- 172.21.15.71:0/33617719 wait complete. 2023-10-26T05:47:04.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 Processor -- start 2023-10-26T05:47:04.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- start start 2023-10-26T05:47:04.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb43c0b10f0 0x7fb43c14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 0x7fb43c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 0x7fb43c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c14fc90 con 0x7fb43c0c9780 2023-10-26T05:47:04.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0c4b20 con 0x7fb43c1591f0 2023-10-26T05:47:04.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c0c4e00 con 0x7fb43c0b10f0 2023-10-26T05:47:04.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 0x7fb43c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 0x7fb43c1543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45314/0 (socket says 172.21.15.71:45314) 2023-10-26T05:47:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 -- 172.21.15.71:0/1328069347 learned_addr learned my addr 172.21.15.71:0/1328069347 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb43a59c640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb43c0b10f0 0x7fb43c14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb43ad9d640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 0x7fb43c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 -- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb43c0b10f0 msgr2=0x7fb43c14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb43c0b10f0 0x7fb43c14f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 -- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 msgr2=0x7fb43c154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 0x7fb43c154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 -- 172.21.15.71:0/1328069347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42c009560 con 0x7fb43c0c9780 2023-10-26T05:47:04.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb433fff640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 0x7fb43c1543f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb42400e9f0 tx=0x7fb42400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:04.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424009710 con 0x7fb43c0c9780 2023-10-26T05:47:04.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb42400ce70 con 0x7fb43c0c9780 2023-10-26T05:47:04.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43c0c50e0 con 0x7fb43c0c9780 2023-10-26T05:47:04.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424010690 con 0x7fb43c0c9780 2023-10-26T05:47:04.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43c0c3d30 con 0x7fb43c0c9780 2023-10-26T05:47:04.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb4240107f0 con 0x7fb43c0c9780 2023-10-26T05:47:04.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb424014070 con 0x7fb43c0c9780 2023-10-26T05:47:04.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.036+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43c0c58e0 con 0x7fb43c0c9780 2023-10-26T05:47:04.052 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.045+0000 7fef6bda1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T05:47:04.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.040+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb43c0c58e0 con 0x7fb43c0c9780 2023-10-26T05:47:04.098 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.097+0000 7fef6bda1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T05:47:04.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.096+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb43c066d00 con 0x7fb43c0c9780 2023-10-26T05:47:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb432ffd640 1 -- 172.21.15.71:0/1328069347 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb424026060 con 0x7fb43c0c9780 2023-10-26T05:47:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2023-10-26T05:47:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 msgr2=0x7fb43c1543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 0x7fb43c1543f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb42400e9f0 tx=0x7fb42400eeb0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 shutdown_connections 2023-10-26T05:47:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb43c1591f0 0x7fb43c154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb43c0c9780 0x7fb43c1543f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 --2- 172.21.15.71:0/1328069347 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb43c0b10f0 0x7fb43c14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 >> 172.21.15.71:0/1328069347 conn(0x7fb43c06fd20 msgr2=0x7fb43c0cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:04.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 shutdown_connections 2023-10-26T05:47:04.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:04.100+0000 7fb44094c640 1 -- 172.21.15.71:0/1328069347 wait complete. 2023-10-26T05:47:04.123 INFO:tasks.ceph:check_enable_crimson: False 2023-10-26T05:47:04.123 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-26T05:47:04.124 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:04.124 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-26T05:47:04.175 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:04.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-26T05:47:04.176 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.173+0000 7fef6bda1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T05:47:04.226 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.225+0000 7fef6bda1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T05:47:04.228 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:04.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-26T05:47:04.282 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:04.283 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-26T05:47:04.303 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.301+0000 7fef6bda1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T05:47:04.335 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:47:04.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-26T05:47:04.348 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:47:04.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-26T05:47:04.355 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.353+0000 7fef6bda1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T05:47:04.366 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:47:04.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-26T05:47:04.419 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:47:04.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-26T05:47:04.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-10-26T05:47:04.497 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.497+0000 7fef6bda1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T05:47:04.507 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.500+0000 7fbeae4ae180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-26T05:47:04.558 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:04.557+0000 7fef6bda1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T05:47:04.560 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.556+0000 7fbeae4ae180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-26T05:47:04.612 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.604+0000 7fbeae4ae180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-26T05:47:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 -- 172.21.15.86:0/1616753955 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec157eb0 msgr2=0x7f9eec1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/1616753955 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec157eb0 0x7f9eec1582f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009840 tx=0x7f9ee001c800 comp rx=0 tx=0).stop 2023-10-26T05:47:04.619 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 -- 172.21.15.86:0/1616753955 shutdown_connections 2023-10-26T05:47:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/1616753955 >> v2:172.21.15.71:3300/0 conn(0x7f9eec0b2e50 0x7f9eec0b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/1616753955 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec157eb0 0x7f9eec1582f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 -- 172.21.15.86:0/1616753955 >> 172.21.15.86:0/1616753955 conn(0x7f9eec0ca230 msgr2=0x7f9eec0ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 -- 172.21.15.86:0/1616753955 shutdown_connections 2023-10-26T05:47:04.620 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.617+0000 7f9ef1d21640 1 -- 172.21.15.86:0/1616753955 wait complete. 2023-10-26T05:47:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 Processor -- start 2023-10-26T05:47:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- start start 2023-10-26T05:47:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 0x7f9eec1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.623 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec157ad0 0x7f9eec150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec157eb0 0x7f9eec152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec153810 con 0x7f9eec157eb0 2023-10-26T05:47:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec153af0 con 0x7f9eec157ad0 2023-10-26T05:47:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec151bf0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.624 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 0x7f9eec1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.625 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 0x7f9eec1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:60430/0 (socket says 172.21.15.86:60430) 2023-10-26T05:47:04.625 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec157ad0 0x7f9eec150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.625 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 -- 172.21.15.86:0/3364373741 learned_addr learned my addr 172.21.15.86:0/3364373741 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:04.625 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eebfff640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec157eb0 0x7f9eec152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.625 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 -- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec157ad0 msgr2=0x7f9eec150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec157ad0 0x7f9eec150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 -- 172.21.15.86:0/3364373741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec157eb0 msgr2=0x7f9eec152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec157eb0 0x7f9eec152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 -- 172.21.15.86:0/3364373741 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee00094a0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.626 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeaffd640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec157ad0 0x7f9eec150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9eeb7fe640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 0x7f9eec1526d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4001570 tx=0x7f9ee400cd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4003db0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ee4005070 con 0x7f9eec0b2e50 2023-10-26T05:47:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec15c560 con 0x7f9eec0b2e50 2023-10-26T05:47:04.627 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4005480 con 0x7f9eec0b2e50 2023-10-26T05:47:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec1521b0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9ee400a640 con 0x7f9eec0b2e50 2023-10-26T05:47:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9ee401cad0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.628 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.621+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec158bf0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.629 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.625+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9ee4017070 con 0x7f9eec0b2e50 2023-10-26T05:47:04.689 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.684+0000 7fbeae4ae180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-26T05:47:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.693+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f9eec1523a0 con 0x7f9eec0b2e50 2023-10-26T05:47:04.700 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ed3fff640 1 -- 172.21.15.86:0/3364373741 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9ee400ab20 con 0x7f9eec0b2e50 2023-10-26T05:47:04.701 INFO:teuthology.orchestra.run.smithi086.stdout:0 2023-10-26T05:47:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 msgr2=0x7f9eec1526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 0x7f9eec1526d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4001570 tx=0x7f9ee400cd00 comp rx=0 tx=0).stop 2023-10-26T05:47:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 shutdown_connections 2023-10-26T05:47:04.702 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eec157eb0 0x7f9eec152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9eec157ad0 0x7f9eec150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 --2- 172.21.15.86:0/3364373741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9eec0b2e50 0x7f9eec1526d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 >> 172.21.15.86:0/3364373741 conn(0x7f9eec0ca230 msgr2=0x7f9eec0ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 shutdown_connections 2023-10-26T05:47:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.701+0000 7f9ef1d21640 1 -- 172.21.15.86:0/3364373741 wait complete. 2023-10-26T05:47:04.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-10-26T05:47:04.742 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.736+0000 7fbeae4ae180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-26T05:47:04.818 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.812+0000 7fbeae4ae180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-26T05:47:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 -- 172.21.15.86:0/3885435760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 msgr2=0x7fb3940b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 --2- 172.21.15.86:0/3885435760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb3940b2b80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3940b37c0 tx=0x7fb38801c8a0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.836 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 -- 172.21.15.86:0/3885435760 shutdown_connections 2023-10-26T05:47:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 --2- 172.21.15.86:0/3885435760 >> v2:172.21.15.71:3300/0 conn(0x7fb3940b30c0 0x7fb39414f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 --2- 172.21.15.86:0/3885435760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb3940b2b80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 --2- 172.21.15.86:0/3885435760 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 0x7fb39415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 -- 172.21.15.86:0/3885435760 >> 172.21.15.86:0/3885435760 conn(0x7fb394070d10 msgr2=0x7fb3940af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:04.837 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 -- 172.21.15.86:0/3885435760 shutdown_connections 2023-10-26T05:47:04.838 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.833+0000 7fb39940b640 1 -- 172.21.15.86:0/3885435760 wait complete. 2023-10-26T05:47:04.840 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 Processor -- start 2023-10-26T05:47:04.840 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 -- start start 2023-10-26T05:47:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3940b30c0 0x7fb39415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb39414b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 0x7fb394140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39415c140 con 0x7fb3940b30c0 2023-10-26T05:47:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39415b4c0 con 0x7fb3940cabb0 2023-10-26T05:47:04.841 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39415b7a0 con 0x7fb3940cb9e0 2023-10-26T05:47:04.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb39414b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3937fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 0x7fb394140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3937fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 0x7fb394140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:60444/0 (socket says 172.21.15.86:60444) 2023-10-26T05:47:04.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb39414b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34222/0 (socket says 172.21.15.86:34222) 2023-10-26T05:47:04.842 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3937fe640 1 -- 172.21.15.86:0/602331610 learned_addr learned my addr 172.21.15.86:0/602331610 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:04.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb392ffd640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3940b30c0 0x7fb39415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:04.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 -- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 msgr2=0x7fb394140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 0x7fb394140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 -- 172.21.15.86:0/602331610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3940b30c0 msgr2=0x7fb39415aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3940b30c0 0x7fb39415aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.843 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 -- 172.21.15.86:0/602331610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb388009560 con 0x7fb3940cabb0 2023-10-26T05:47:04.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb3927fc640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb39414b450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3880ac000 tx=0x7fb3880a57f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:04.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880042c0 con 0x7fb3940cabb0 2023-10-26T05:47:04.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb388005030 con 0x7fb3940cabb0 2023-10-26T05:47:04.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 -- 172.21.15.86:0/602331610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39415ba80 con 0x7fb3940cabb0 2023-10-26T05:47:04.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388004530 con 0x7fb3940cabb0 2023-10-26T05:47:04.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb39940b640 1 -- 172.21.15.86:0/602331610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39414b990 con 0x7fb3940cabb0 2023-10-26T05:47:04.845 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb3880b4070 con 0x7fb3940cabb0 2023-10-26T05:47:04.845 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.837+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fb3880039b0 con 0x7fb3940cabb0 2023-10-26T05:47:04.845 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.841+0000 7fb39940b640 1 -- 172.21.15.86:0/602331610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb394066d00 con 0x7fb3940cabb0 2023-10-26T05:47:04.845 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.841+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb3880b2070 con 0x7fb3940cabb0 2023-10-26T05:47:04.869 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:04.864+0000 7fbeae4ae180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-26T05:47:04.923 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.921+0000 7fb39940b640 1 -- 172.21.15.86:0/602331610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7fb3940ac630 con 0x7fb3940cabb0 2023-10-26T05:47:04.933 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb37b7fe640 1 -- 172.21.15.86:0/602331610 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb388003b70 con 0x7fb3940cabb0 2023-10-26T05:47:04.934 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:47:04.935 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb3797fa640 1 -- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 msgr2=0x7fb39414b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:04.935 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb3797fa640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb39414b450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb3880ac000 tx=0x7fb3880a57f0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.936 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb3797fa640 1 -- 172.21.15.86:0/602331610 shutdown_connections 2023-10-26T05:47:04.936 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb3797fa640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3940cb9e0 0x7fb394140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.937 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb3797fa640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3940cabb0 0x7fb39414b450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.937 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.933+0000 7fb3797fa640 1 --2- 172.21.15.86:0/602331610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3940b30c0 0x7fb39415aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:04.937 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.937+0000 7fb3797fa640 1 -- 172.21.15.86:0/602331610 >> 172.21.15.86:0/602331610 conn(0x7fb394070d10 msgr2=0x7fb3940af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:04.937 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.937+0000 7fb3797fa640 1 -- 172.21.15.86:0/602331610 shutdown_connections 2023-10-26T05:47:04.938 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:04.937+0000 7fb3797fa640 1 -- 172.21.15.86:0/602331610 wait complete. 2023-10-26T05:47:04.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-10-26T05:47:05.011 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:05.004+0000 7fbeae4ae180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-26T05:47:05.069 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:05.064+0000 7fbeae4ae180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-26T05:47:05.086 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 -- 172.21.15.86:0/115724788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1520 msgr2=0x7fb3d00b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.087 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 --2- 172.21.15.86:0/115724788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1520 0x7fb3d00b1900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0065ec0 tx=0x7fb3c801cb60 comp rx=0 tx=0).stop 2023-10-26T05:47:05.087 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 -- 172.21.15.86:0/115724788 shutdown_connections 2023-10-26T05:47:05.087 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 --2- 172.21.15.86:0/115724788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 0x7fb3d0158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.087 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 --2- 172.21.15.86:0/115724788 >> v2:172.21.15.71:3300/0 conn(0x7fb3d00b1040 0x7fb3d01589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.088 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 --2- 172.21.15.86:0/115724788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1520 0x7fb3d00b1900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.088 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 -- 172.21.15.86:0/115724788 >> 172.21.15.86:0/115724788 conn(0x7fb3d00ca230 msgr2=0x7fb3d00ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.088 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 -- 172.21.15.86:0/115724788 shutdown_connections 2023-10-26T05:47:05.088 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 -- 172.21.15.86:0/115724788 wait complete. 2023-10-26T05:47:05.090 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 Processor -- start 2023-10-26T05:47:05.091 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.085+0000 7fb3d725b640 1 -- start start 2023-10-26T05:47:05.091 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 0x7fb3d0144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.091 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 0x7fb3d014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.091 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d00b1520 0x7fb3d0150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.091 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d014e390 con 0x7fb3d00b1520 2023-10-26T05:47:05.092 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0146720 con 0x7fb3d00b1040 2023-10-26T05:47:05.092 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0146440 con 0x7fb3d0070e10 2023-10-26T05:47:05.092 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 0x7fb3d014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.092 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d4ff7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 0x7fb3d0144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.092 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 0x7fb3d014f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34248/0 (socket says 172.21.15.86:34248) 2023-10-26T05:47:05.092 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 -- 172.21.15.86:0/2016557520 learned_addr learned my addr 172.21.15.86:0/2016557520 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d57f8640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d00b1520 0x7fb3d0150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 -- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 msgr2=0x7fb3d0144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 0x7fb3d0144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 -- 172.21.15.86:0/2016557520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d00b1520 msgr2=0x7fb3d0150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.093 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d00b1520 0x7fb3d0150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.094 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 -- 172.21.15.86:0/2016557520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4009560 con 0x7fb3d00b1040 2023-10-26T05:47:05.094 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d4ff7640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 0x7fb3d0144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:05.094 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cffff640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 0x7fb3d014f0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e950 tx=0x7fb3c400ee10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:05.094 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c400cd80 con 0x7fb3d00b1040 2023-10-26T05:47:05.094 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3c4004020 con 0x7fb3d00b1040 2023-10-26T05:47:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4004430 con 0x7fb3d00b1040 2023-10-26T05:47:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c8009820 con 0x7fb3d00b1040 2023-10-26T05:47:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d014e8a0 con 0x7fb3d00b1040 2023-10-26T05:47:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb3c40047d0 con 0x7fb3d00b1040 2023-10-26T05:47:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb3c4014070 con 0x7fb3d00b1040 2023-10-26T05:47:05.095 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.089+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d0066d00 con 0x7fb3d00b1040 2023-10-26T05:47:05.096 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.093+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb3c4004cb0 con 0x7fb3d00b1040 2023-10-26T05:47:05.158 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.157+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7fb3d01468f0 con 0x7fb3d00b1040 2023-10-26T05:47:05.164 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.161+0000 7fb3cd7fa640 1 -- 172.21.15.86:0/2016557520 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb3c4005390 con 0x7fb3d00b1040 2023-10-26T05:47:05.164 INFO:teuthology.orchestra.run.smithi086.stdout:2 2023-10-26T05:47:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 msgr2=0x7fb3d014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.165 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 0x7fb3d014f0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e950 tx=0x7fb3c400ee10 comp rx=0 tx=0).stop 2023-10-26T05:47:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 shutdown_connections 2023-10-26T05:47:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3d00b1520 0x7fb3d0150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00b1040 0x7fb3d014f0a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 --2- 172.21.15.86:0/2016557520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d0070e10 0x7fb3d0144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.166 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 >> 172.21.15.86:0/2016557520 conn(0x7fb3d00ca230 msgr2=0x7fb3d0069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 shutdown_connections 2023-10-26T05:47:05.167 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.165+0000 7fb3d725b640 1 -- 172.21.15.86:0/2016557520 wait complete. 2023-10-26T05:47:05.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-10-26T05:47:05.301 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- 172.21.15.86:0/577506451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b1230 msgr2=0x7f60bc0b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 --2- 172.21.15.86:0/577506451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b1230 0x7f60bc0b1c60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009840 tx=0x7f60b001eff0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- 172.21.15.86:0/577506451 shutdown_connections 2023-10-26T05:47:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 --2- 172.21.15.86:0/577506451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b1230 0x7f60bc0b1c60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 --2- 172.21.15.86:0/577506451 >> v2:172.21.15.86:3300/0 conn(0x7f60bc141a40 0x7f60bc0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- 172.21.15.86:0/577506451 >> 172.21.15.86:0/577506451 conn(0x7f60bc0aec60 msgr2=0x7f60bc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- 172.21.15.86:0/577506451 shutdown_connections 2023-10-26T05:47:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- 172.21.15.86:0/577506451 wait complete. 2023-10-26T05:47:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 Processor -- start 2023-10-26T05:47:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- start start 2023-10-26T05:47:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60bc0b1230 0x7f60bc15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 0x7f60bc157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60bc141a40 0x7f60bc137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1e1c50 con 0x7f60bc141a40 2023-10-26T05:47:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1e1900 con 0x7f60bc0b1230 2023-10-26T05:47:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60c238e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc157e80 con 0x7f60bc0b21a0 2023-10-26T05:47:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 0x7f60bc157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60bc0b1230 0x7f60bc15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.307 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 0x7f60bc157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:59774/0 (socket says 172.21.15.86:59774) 2023-10-26T05:47:05.308 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 -- 172.21.15.86:0/2575989448 learned_addr learned my addr 172.21.15.86:0/2575989448 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:05.308 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 -- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60bc0b1230 msgr2=0x7f60bc15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.308 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60bc0b1230 0x7f60bc15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.308 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 -- 172.21.15.86:0/2575989448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60bc141a40 msgr2=0x7f60bc137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.308 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60bc141a40 0x7f60bc137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.309 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 -- 172.21.15.86:0/2575989448 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac0094a0 con 0x7f60bc0b21a0 2023-10-26T05:47:05.309 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60bb7fe640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 0x7f60bc157940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009790 tx=0x7f60b0005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:05.309 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.301+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0005220 con 0x7f60bc0b21a0 2023-10-26T05:47:05.309 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f60b0005470 con 0x7f60bc0b21a0 2023-10-26T05:47:05.309 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b00094a0 con 0x7f60bc0b21a0 2023-10-26T05:47:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b00b1b10 con 0x7f60bc0b21a0 2023-10-26T05:47:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc15a900 con 0x7f60bc0b21a0 2023-10-26T05:47:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f60b00b7040 con 0x7f60bc0b21a0 2023-10-26T05:47:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f60b0004e50 con 0x7f60bc0b21a0 2023-10-26T05:47:05.310 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc066b10 con 0x7f60bc0b21a0 2023-10-26T05:47:05.311 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.305+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f60b00c9060 con 0x7f60bc0b21a0 2023-10-26T05:47:05.374 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.373+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f60bc1e1d90 con 0x7f60bc0b21a0 2023-10-26T05:47:05.381 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60b8ff9640 1 -- 172.21.15.86:0/2575989448 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f60bc1e1d90 con 0x7f60bc0b21a0 2023-10-26T05:47:05.381 INFO:teuthology.orchestra.run.smithi086.stdout:3 2023-10-26T05:47:05.382 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 msgr2=0x7f60bc157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 0x7f60bc157940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009790 tx=0x7f60b0005dd0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 shutdown_connections 2023-10-26T05:47:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60bc141a40 0x7f60bc137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60bc0b21a0 0x7f60bc157940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.383 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 --2- 172.21.15.86:0/2575989448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60bc0b1230 0x7f60bc15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 >> 172.21.15.86:0/2575989448 conn(0x7f60bc0aec60 msgr2=0x7f60bc1343b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 shutdown_connections 2023-10-26T05:47:05.384 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.381+0000 7f60c238e640 1 -- 172.21.15.86:0/2575989448 wait complete. 2023-10-26T05:47:05.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-10-26T05:47:05.464 INFO:tasks.ceph.mgr.x.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-26T05:47:05.464 INFO:tasks.ceph.mgr.x.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-26T05:47:05.464 INFO:tasks.ceph.mgr.x.smithi086.stderr: from numpy import show_config as show_numpy_config 2023-10-26T05:47:05.513 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:05.513+0000 7fef6bda1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T05:47:05.516 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.513+0000 7f0860a93640 1 -- 172.21.15.86:0/4066255005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 msgr2=0x7f085c06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.516 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.513+0000 7f0860a93640 1 --2- 172.21.15.86:0/4066255005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c06dd30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f084c009900 tx=0x7f084c01c7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.517 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- 172.21.15.86:0/4066255005 shutdown_connections 2023-10-26T05:47:05.517 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 --2- 172.21.15.86:0/4066255005 >> v2:172.21.15.86:3300/0 conn(0x7f085c0c9780 0x7f085c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.517 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 --2- 172.21.15.86:0/4066255005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c06dd30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.517 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- 172.21.15.86:0/4066255005 >> 172.21.15.86:0/4066255005 conn(0x7f085c06fd20 msgr2=0x7f085c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.518 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- 172.21.15.86:0/4066255005 shutdown_connections 2023-10-26T05:47:05.518 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- 172.21.15.86:0/4066255005 wait complete. 2023-10-26T05:47:05.521 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 Processor -- start 2023-10-26T05:47:05.522 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- start start 2023-10-26T05:47:05.522 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f085c0b10f0 0x7f085c0c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.522 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 0x7f085c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.523 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c0c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.523 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c06bfa0 con 0x7f085c0b10f0 2023-10-26T05:47:05.523 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0c89e0 con 0x7f085c0c9780 2023-10-26T05:47:05.524 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0c8cc0 con 0x7f085c1591f0 2023-10-26T05:47:05.524 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 0x7f085c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.524 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f085ad9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c0c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.524 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 0x7f085c14c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43000/0 (socket says 172.21.15.86:43000) 2023-10-26T05:47:05.525 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f085ad9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c0c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:59786/0 (socket says 172.21.15.86:59786) 2023-10-26T05:47:05.525 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 -- 172.21.15.86:0/2341844429 learned_addr learned my addr 172.21.15.86:0/2341844429 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:05.525 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f085a59c640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f085c0b10f0 0x7f085c0c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.526 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 -- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 msgr2=0x7f085c0c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.526 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c0c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.526 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 -- 172.21.15.86:0/2341844429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f085c0b10f0 msgr2=0x7f085c0c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.527 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f085c0b10f0 0x7f085c0c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.527 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 -- 172.21.15.86:0/2341844429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c009560 con 0x7f085c0c9780 2023-10-26T05:47:05.527 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0859d9b640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 0x7f085c14c6b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f085000ea20 tx=0x7f085000eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:05.527 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850009710 con 0x7f085c0c9780 2023-10-26T05:47:05.527 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f085000ce70 con 0x7f085c0c9780 2023-10-26T05:47:05.528 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c06bbc0 con 0x7f085c0c9780 2023-10-26T05:47:05.528 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850010690 con 0x7f085c0c9780 2023-10-26T05:47:05.528 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c0c75d0 con 0x7f085c0c9780 2023-10-26T05:47:05.528 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f08500107f0 con 0x7f085c0c9780 2023-10-26T05:47:05.528 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.517+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0850014070 con 0x7f085c0c9780 2023-10-26T05:47:05.529 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.521+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c06c0e0 con 0x7f085c0c9780 2023-10-26T05:47:05.529 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.521+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f085c06c0e0 con 0x7f085c0c9780 2023-10-26T05:47:05.585 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.585+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f085c066d00 con 0x7f085c0c9780 2023-10-26T05:47:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.589+0000 7f084affd640 1 -- 172.21.15.86:0/2341844429 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0850026060 con 0x7f085c0c9780 2023-10-26T05:47:05.592 INFO:teuthology.orchestra.run.smithi086.stdout:4 2023-10-26T05:47:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 msgr2=0x7f085c14c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 0x7f085c14c6b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f085000ea20 tx=0x7f085000eee0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 shutdown_connections 2023-10-26T05:47:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f085c1591f0 0x7f085c0c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f085c0c9780 0x7f085c14c6b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 --2- 172.21.15.86:0/2341844429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f085c0b10f0 0x7f085c0c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.594 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 >> 172.21.15.86:0/2341844429 conn(0x7f085c06fd20 msgr2=0x7f085c14e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 shutdown_connections 2023-10-26T05:47:05.595 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.593+0000 7f0860a93640 1 -- 172.21.15.86:0/2341844429 wait complete. 2023-10-26T05:47:05.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-10-26T05:47:05.649 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:05.649+0000 7fef6bda1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T05:47:05.713 INFO:tasks.ceph.mgr.y.smithi071.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-26T05:47:05.713 INFO:tasks.ceph.mgr.y.smithi071.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-26T05:47:05.713 INFO:tasks.ceph.mgr.y.smithi071.stderr: from numpy import show_config as show_numpy_config 2023-10-26T05:47:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/903377431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e40af5c0 msgr2=0x7f87e40acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.720 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 --2- 172.21.15.86:0/903377431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e40af5c0 0x7f87e40acb20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f87e40b0970 tx=0x7f87dc01ccf0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/903377431 shutdown_connections 2023-10-26T05:47:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 --2- 172.21.15.86:0/903377431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e40af5c0 0x7f87e40acb20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 --2- 172.21.15.86:0/903377431 >> v2:172.21.15.71:3300/0 conn(0x7f87e4152780 0x7f87e4069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/903377431 >> 172.21.15.86:0/903377431 conn(0x7f87e40ca2c0 msgr2=0x7f87e40ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.721 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/903377431 shutdown_connections 2023-10-26T05:47:05.722 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/903377431 wait complete. 2023-10-26T05:47:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 Processor -- start 2023-10-26T05:47:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- start start 2023-10-26T05:47:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e40af5c0 0x7f87e4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 0x7f87e4155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87e41593a0 0x7f87e414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40b3690 con 0x7f87e40af5c0 2023-10-26T05:47:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4155c60 con 0x7f87e4152780 2023-10-26T05:47:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41534e0 con 0x7f87e41593a0 2023-10-26T05:47:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 0x7f87e4155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e3fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87e41593a0 0x7f87e414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 0x7f87e4155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43016/0 (socket says 172.21.15.86:43016) 2023-10-26T05:47:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e3fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87e41593a0 0x7f87e414a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:59788/0 (socket says 172.21.15.86:59788) 2023-10-26T05:47:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 -- 172.21.15.86:0/2993448130 learned_addr learned my addr 172.21.15.86:0/2993448130 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e37fe640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e40af5c0 0x7f87e4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 -- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87e41593a0 msgr2=0x7f87e414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87e41593a0 0x7f87e414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 -- 172.21.15.86:0/2993448130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e40af5c0 msgr2=0x7f87e4152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e40af5c0 0x7f87e4152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 -- 172.21.15.86:0/2993448130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0098e0 con 0x7f87e4152780 2023-10-26T05:47:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e2ffd640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 0x7f87e4155720 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f87d800b3c0 tx=0x7f87d800b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8011830 con 0x7f87e4152780 2023-10-26T05:47:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f87d8011aa0 con 0x7f87e4152780 2023-10-26T05:47:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e41537c0 con 0x7f87e4152780 2023-10-26T05:47:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8010380 con 0x7f87e4152780 2023-10-26T05:47:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4145590 con 0x7f87e4152780 2023-10-26T05:47:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f87d80107e0 con 0x7f87e4152780 2023-10-26T05:47:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.721+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f87d801ae40 con 0x7f87e4152780 2023-10-26T05:47:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.725+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e41457b0 con 0x7f87e4152780 2023-10-26T05:47:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.725+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f87d801a410 con 0x7f87e4152780 2023-10-26T05:47:05.787 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:05.780+0000 7fbeae4ae180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-26T05:47:05.789 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.789+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f87e4066d00 con 0x7f87e4152780 2023-10-26T05:47:05.797 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87cbfff640 1 -- 172.21.15.86:0/2993448130 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f87d8010c70 con 0x7f87e4152780 2023-10-26T05:47:05.797 INFO:teuthology.orchestra.run.smithi086.stdout:5 2023-10-26T05:47:05.798 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 msgr2=0x7f87e4155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 0x7f87e4155720 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f87d800b3c0 tx=0x7f87d800b880 comp rx=0 tx=0).stop 2023-10-26T05:47:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 shutdown_connections 2023-10-26T05:47:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87e41593a0 0x7f87e414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e4152780 0x7f87e4155720 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.799 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 --2- 172.21.15.86:0/2993448130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e40af5c0 0x7f87e4152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.800 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 >> 172.21.15.86:0/2993448130 conn(0x7f87e40ca2c0 msgr2=0x7f87e40ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.800 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 shutdown_connections 2023-10-26T05:47:05.800 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.797+0000 7f87e9a79640 1 -- 172.21.15.86:0/2993448130 wait complete. 2023-10-26T05:47:05.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-10-26T05:47:05.854 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:05.853+0000 7fef6bda1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T05:47:05.924 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:05.925+0000 7fef6bda1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T05:47:05.958 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:05.952+0000 7fbeae4ae180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-26T05:47:05.967 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 -- 172.21.15.86:0/2152157219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd80b2040 msgr2=0x7fdbd81587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 --2- 172.21.15.86:0/2152157219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd80b2040 0x7fdbd81587e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009900 tx=0x7fdbc801c7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 -- 172.21.15.86:0/2152157219 shutdown_connections 2023-10-26T05:47:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 --2- 172.21.15.86:0/2152157219 >> v2:172.21.15.71:3300/0 conn(0x7fdbd8158d20 0x7fdbd80b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.968 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 --2- 172.21.15.86:0/2152157219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd80b2040 0x7fdbd81587e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.969 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 -- 172.21.15.86:0/2152157219 >> 172.21.15.86:0/2152157219 conn(0x7fdbd80af7d0 msgr2=0x7fdbd80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:05.969 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 -- 172.21.15.86:0/2152157219 shutdown_connections 2023-10-26T05:47:05.969 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.965+0000 7fdbddcca640 1 -- 172.21.15.86:0/2152157219 wait complete. 2023-10-26T05:47:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 Processor -- start 2023-10-26T05:47:05.972 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- start start 2023-10-26T05:47:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd80b2040 0x7fdbd80a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbd8158d20 0x7fdbd8162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 0x7fdbd80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd8162cd0 con 0x7fdbd8158d20 2023-10-26T05:47:05.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd80a97a0 con 0x7fdbd815c540 2023-10-26T05:47:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd80a9a80 con 0x7fdbd80b2040 2023-10-26T05:47:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 0x7fdbd80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd80b2040 0x7fdbd80a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 0x7fdbd80cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43040/0 (socket says 172.21.15.86:43040) 2023-10-26T05:47:05.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd7fff640 1 -- 172.21.15.86:0/737080296 learned_addr learned my addr 172.21.15.86:0/737080296 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd6ffd640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbd8158d20 0x7fdbd8162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 -- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 msgr2=0x7fdbd80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 0x7fdbd80cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 -- 172.21.15.86:0/737080296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbd8158d20 msgr2=0x7fdbd8162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:05.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbd8158d20 0x7fdbd8162790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 -- 172.21.15.86:0/737080296 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8009560 con 0x7fdbd80b2040 2023-10-26T05:47:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd7fff640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 0x7fdbd80cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:05.976 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbd77fe640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd80b2040 0x7fdbd80a91c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c3c0 tx=0x7fdbcc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:05.977 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0052d0 con 0x7fdbd80b2040 2023-10-26T05:47:05.977 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdbcc007020 con 0x7fdbd80b2040 2023-10-26T05:47:05.977 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd80a9d60 con 0x7fdbd80b2040 2023-10-26T05:47:05.977 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc007430 con 0x7fdbd80b2040 2023-10-26T05:47:05.977 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd80a6850 con 0x7fdbd80b2040 2023-10-26T05:47:05.977 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdbcc006ae0 con 0x7fdbd80b2040 2023-10-26T05:47:05.978 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fdbcc011590 con 0x7fdbd80b2040 2023-10-26T05:47:05.978 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.969+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd8066d00 con 0x7fdbd80b2040 2023-10-26T05:47:05.978 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:05.973+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fdbcc020060 con 0x7fdbd80b2040 2023-10-26T05:47:06.041 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.041+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fdbd806a620 con 0x7fdbd80b2040 2023-10-26T05:47:06.047 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.045+0000 7fdbbffff640 1 -- 172.21.15.86:0/737080296 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdbcc003600 con 0x7fdbd80b2040 2023-10-26T05:47:06.047 INFO:teuthology.orchestra.run.smithi086.stdout:6 2023-10-26T05:47:06.048 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd80b2040 msgr2=0x7fdbd80a91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.049 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd80b2040 0x7fdbd80a91c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c3c0 tx=0x7fdbcc00c880 comp rx=0 tx=0).stop 2023-10-26T05:47:06.049 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 shutdown_connections 2023-10-26T05:47:06.049 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbd815c540 0x7fdbd80cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.049 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbd8158d20 0x7fdbd8162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.050 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 --2- 172.21.15.86:0/737080296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbd80b2040 0x7fdbd80a91c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.050 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 >> 172.21.15.86:0/737080296 conn(0x7fdbd80af7d0 msgr2=0x7fdbd8156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:06.050 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 shutdown_connections 2023-10-26T05:47:06.050 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.049+0000 7fdbddcca640 1 -- 172.21.15.86:0/737080296 wait complete. 2023-10-26T05:47:06.060 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:06.061+0000 7fef6bda1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T05:47:06.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-10-26T05:47:06.130 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:06.124+0000 7fbeae4ae180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-26T05:47:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 -- 172.21.15.86:0/767669838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40c9780 msgr2=0x7f76e40b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 --2- 172.21.15.86:0/767669838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40c9780 0x7f76e40b0bb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009840 tx=0x7f76cc01c700 comp rx=0 tx=0).stop 2023-10-26T05:47:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 -- 172.21.15.86:0/767669838 shutdown_connections 2023-10-26T05:47:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 --2- 172.21.15.86:0/767669838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40c9780 0x7f76e40b0bb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 -- 172.21.15.86:0/767669838 >> 172.21.15.86:0/767669838 conn(0x7f76e406fd20 msgr2=0x7f76e40cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 -- 172.21.15.86:0/767669838 shutdown_connections 2023-10-26T05:47:06.185 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.181+0000 7f76e995f640 1 -- 172.21.15.86:0/767669838 wait complete. 2023-10-26T05:47:06.187 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 Processor -- start 2023-10-26T05:47:06.187 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- start start 2023-10-26T05:47:06.187 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 0x7f76e414f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76e40c9780 0x7f76e40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76e41591f0 0x7f76e41476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e414ec70 con 0x7f76e40c9780 2023-10-26T05:47:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e414f640 con 0x7f76e40b10f0 2023-10-26T05:47:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e414e060 con 0x7f76e41591f0 2023-10-26T05:47:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 0x7f76e414f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 0x7f76e414f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43048/0 (socket says 172.21.15.86:43048) 2023-10-26T05:47:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 -- 172.21.15.86:0/1605817479 learned_addr learned my addr 172.21.15.86:0/1605817479 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-26T05:47:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e37fe640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76e41591f0 0x7f76e41476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e27fc640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76e40c9780 0x7f76e40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 -- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76e41591f0 msgr2=0x7f76e41476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76e41591f0 0x7f76e41476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 -- 172.21.15.86:0/1605817479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76e40c9780 msgr2=0x7f76e40b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76e40c9780 0x7f76e40b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 -- 172.21.15.86:0/1605817479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc0094a0 con 0x7f76e40b10f0 2023-10-26T05:47:06.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e27fc640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76e40c9780 0x7f76e40b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:06.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e2ffd640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 0x7f76e414f100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b410 tx=0x7f76d400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:06.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4004020 con 0x7f76e40b10f0 2023-10-26T05:47:06.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76d4004270 con 0x7f76e40b10f0 2023-10-26T05:47:06.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e414e340 con 0x7f76e40b10f0 2023-10-26T05:47:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4011890 con 0x7f76e40b10f0 2023-10-26T05:47:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e414e900 con 0x7f76e40b10f0 2023-10-26T05:47:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f76d4011cf0 con 0x7f76e40b10f0 2023-10-26T05:47:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f76d401aa70 con 0x7f76e40b10f0 2023-10-26T05:47:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.185+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e4066d00 con 0x7f76e40b10f0 2023-10-26T05:47:06.193 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.189+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f76d4015030 con 0x7f76e40b10f0 2023-10-26T05:47:06.199 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:06.192+0000 7fbeae4ae180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-26T05:47:06.255 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.253+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f76e415cb00 con 0x7f76e40b10f0 2023-10-26T05:47:06.261 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76dbfff640 1 -- 172.21.15.86:0/1605817479 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f76d401ad00 con 0x7f76e40b10f0 2023-10-26T05:47:06.261 INFO:teuthology.orchestra.run.smithi086.stdout:7 2023-10-26T05:47:06.262 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 msgr2=0x7f76e414f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.263 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 0x7f76e414f100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76d400b410 tx=0x7f76d400b8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.263 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 shutdown_connections 2023-10-26T05:47:06.263 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76e41591f0 0x7f76e41476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.263 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76e40c9780 0x7f76e40b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 --2- 172.21.15.86:0/1605817479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76e40b10f0 0x7f76e414f100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 >> 172.21.15.86:0/1605817479 conn(0x7f76e406fd20 msgr2=0x7f76e41568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 shutdown_connections 2023-10-26T05:47:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:06.261+0000 7f76e995f640 1 -- 172.21.15.86:0/1605817479 wait complete. 2023-10-26T05:47:06.278 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-26T05:47:06.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-26T05:47:06.281 INFO:tasks.ceph.osd.0:Started 2023-10-26T05:47:06.281 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-26T05:47:06.282 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-26T05:47:06.284 INFO:tasks.ceph.osd.1:Started 2023-10-26T05:47:06.284 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-26T05:47:06.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-26T05:47:06.287 INFO:tasks.ceph.osd.2:Started 2023-10-26T05:47:06.287 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-26T05:47:06.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-26T05:47:06.290 INFO:tasks.ceph.osd.3:Started 2023-10-26T05:47:06.290 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-26T05:47:06.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-26T05:47:06.293 INFO:tasks.ceph.osd.4:Started 2023-10-26T05:47:06.293 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-26T05:47:06.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-26T05:47:06.296 INFO:tasks.ceph.osd.5:Started 2023-10-26T05:47:06.297 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-26T05:47:06.297 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-26T05:47:06.300 INFO:tasks.ceph.osd.6:Started 2023-10-26T05:47:06.300 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-26T05:47:06.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-26T05:47:06.302 INFO:tasks.ceph.osd.7:Started 2023-10-26T05:47:06.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:06.348 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:06.340+0000 7fbeae4ae180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-26T05:47:06.357 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-26T05:47:06.357+0000 7f8f8f4ca640 -1 Falling back to public interface 2023-10-26T05:47:06.358 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-26T05:47:06.357+0000 7f1d4d2e3640 -1 Falling back to public interface 2023-10-26T05:47:06.361 INFO:tasks.ceph.osd.1.smithi071.stderr:2023-10-26T05:47:06.356+0000 7f30abaef640 -1 Falling back to public interface 2023-10-26T05:47:06.366 INFO:tasks.ceph.osd.0.smithi071.stderr:2023-10-26T05:47:06.360+0000 7f385a9df640 -1 Falling back to public interface 2023-10-26T05:47:06.367 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-26T05:47:06.365+0000 7f8f8f4ca640 -1 osd.4 0 log_to_monitors true 2023-10-26T05:47:06.368 INFO:tasks.ceph.osd.3.smithi071.stderr:2023-10-26T05:47:06.360+0000 7fad545d1640 -1 Falling back to public interface 2023-10-26T05:47:06.374 INFO:tasks.ceph.osd.2.smithi071.stderr:2023-10-26T05:47:06.368+0000 7f4d2053a640 -1 Falling back to public interface 2023-10-26T05:47:06.378 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-26T05:47:06.377+0000 7f1d4d2e3640 -1 osd.5 0 log_to_monitors true 2023-10-26T05:47:06.380 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-26T05:47:06.377+0000 7fac4c063640 -1 Falling back to public interface 2023-10-26T05:47:06.381 INFO:tasks.ceph.osd.1.smithi071.stderr:2023-10-26T05:47:06.376+0000 7f30abaef640 -1 osd.1 0 log_to_monitors true 2023-10-26T05:47:06.381 INFO:tasks.ceph.osd.3.smithi071.stderr:2023-10-26T05:47:06.376+0000 7fad545d1640 -1 osd.3 0 log_to_monitors true 2023-10-26T05:47:06.382 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-26T05:47:06.381+0000 7f54bee64640 -1 Falling back to public interface 2023-10-26T05:47:06.390 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-26T05:47:06.389+0000 7fac4c063640 -1 osd.7 0 log_to_monitors true 2023-10-26T05:47:06.391 INFO:tasks.ceph.osd.0.smithi071.stderr:2023-10-26T05:47:06.384+0000 7f385a9df640 -1 osd.0 0 log_to_monitors true 2023-10-26T05:47:06.396 INFO:tasks.ceph.osd.2.smithi071.stderr:2023-10-26T05:47:06.392+0000 7f4d2053a640 -1 osd.2 0 log_to_monitors true 2023-10-26T05:47:06.397 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-26T05:47:06.397+0000 7f54bee64640 -1 osd.6 0 log_to_monitors true 2023-10-26T05:47:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/1385919884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 msgr2=0x7f29fc06b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/1385919884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc06b4a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009930 tx=0x7f29ec01c830 comp rx=0 tx=0).stop 2023-10-26T05:47:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/1385919884 shutdown_connections 2023-10-26T05:47:06.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/1385919884 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29fc1595a0 0x7f29fc159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/1385919884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc06b4a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/1385919884 >> 172.21.15.71:0/1385919884 conn(0x7f29fc070170 msgr2=0x7f29fc070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/1385919884 shutdown_connections 2023-10-26T05:47:06.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/1385919884 wait complete. 2023-10-26T05:47:06.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 Processor -- start 2023-10-26T05:47:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- start start 2023-10-26T05:47:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc0ade80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29fc1591c0 0x7f29fc0abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:06.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29fc1595a0 0x7f29fc0ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0aa8e0 con 0x7f29fc06b010 2023-10-26T05:47:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0aabc0 con 0x7f29fc1591c0 2023-10-26T05:47:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29fc0aaea0 con 0x7f29fc1595a0 2023-10-26T05:47:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc0ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:06.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc0ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45384/0 (socket says 172.21.15.71:45384) 2023-10-26T05:47:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 -- 172.21.15.71:0/4041060571 learned_addr learned my addr 172.21.15.71:0/4041060571 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a0144e640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29fc1591c0 0x7f29fc0abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a02450640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29fc1595a0 0x7f29fc0ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 -- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29fc1595a0 msgr2=0x7f29fc0ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29fc1595a0 0x7f29fc0ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 -- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29fc1591c0 msgr2=0x7f29fc0abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29fc1591c0 0x7f29fc0abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 -- 172.21.15.71:0/4041060571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec009560 con 0x7f29fc06b010 2023-10-26T05:47:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a01c4f640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc0ade80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c500 tx=0x7f29f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4005220 con 0x7f29fc06b010 2023-10-26T05:47:06.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29f4005470 con 0x7f29fc06b010 2023-10-26T05:47:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4007290 con 0x7f29fc06b010 2023-10-26T05:47:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29fc0ab180 con 0x7f29fc06b010 2023-10-26T05:47:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29fc0ae3c0 con 0x7f29fc06b010 2023-10-26T05:47:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f29f40067e0 con 0x7f29fc06b010 2023-10-26T05:47:06.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.428+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f29f4003ab0 con 0x7f29fc06b010 2023-10-26T05:47:06.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.432+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29fc0b1ac0 con 0x7f29fc06b010 2023-10-26T05:47:06.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.432+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f29f4016020 con 0x7f29fc06b010 2023-10-26T05:47:06.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29fc066d00 con 0x7f29fc06b010 2023-10-26T05:47:06.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f29f27fc640 1 -- 172.21.15.71:0/4041060571 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f29f4003600 con 0x7f29fc06b010 2023-10-26T05:47:06.494 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:06.494 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":10,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:06.255185+0000","last_up_change":"0.000000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:06.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 msgr2=0x7f29fc0ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:06.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc0ade80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c500 tx=0x7f29f400c9c0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 shutdown_connections 2023-10-26T05:47:06.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29fc1595a0 0x7f29fc0ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29fc1591c0 0x7f29fc0abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 --2- 172.21.15.71:0/4041060571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29fc06b010 0x7f29fc0ade80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:06.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 >> 172.21.15.71:0/4041060571 conn(0x7f29fc070170 msgr2=0x7f29fc15a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:06.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 shutdown_connections 2023-10-26T05:47:06.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:06.488+0000 7f2a03eb3640 1 -- 172.21.15.71:0/4041060571 wait complete. 2023-10-26T05:47:06.509 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-26T05:47:06.510 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-26T05:47:06.696 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:06.693+0000 7fef6bda1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T05:47:06.770 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:06.769+0000 7fef6bda1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T05:47:06.941 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:06.941+0000 7fef6bda1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T05:47:06.973 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:06.968+0000 7fbeae4ae180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-26T05:47:07.013 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:07.013+0000 7fef6bda1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T05:47:07.048 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:07.040+0000 7fbeae4ae180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-26T05:47:07.216 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:07.208+0000 7fbeae4ae180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-26T05:47:07.290 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:07.284+0000 7fbeae4ae180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-26T05:47:07.302 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:07.301+0000 7fef6bda1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T05:47:07.316 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:07.361 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:07.361+0000 7fef6bda1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T05:47:07.437 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- 172.21.15.71:0/965322251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c141a70 msgr2=0x7eff3c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:07.438 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 --2- 172.21.15.71:0/965322251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c141a70 0x7eff3c0cd670 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009840 tx=0x7eff2c01c740 comp rx=0 tx=0).stop 2023-10-26T05:47:07.438 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- 172.21.15.71:0/965322251 shutdown_connections 2023-10-26T05:47:07.438 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 --2- 172.21.15.71:0/965322251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c141a70 0x7eff3c0cd670 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.438 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- 172.21.15.71:0/965322251 >> 172.21.15.71:0/965322251 conn(0x7eff3c0afeb0 msgr2=0x7eff3c0b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:07.439 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- 172.21.15.71:0/965322251 shutdown_connections 2023-10-26T05:47:07.439 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- 172.21.15.71:0/965322251 wait complete. 2023-10-26T05:47:07.442 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 Processor -- start 2023-10-26T05:47:07.442 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- start start 2023-10-26T05:47:07.442 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff3c0cdbb0 0x7eff3c0c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:07.442 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 0x7eff3c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:07.442 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c15c320 0x7eff3c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:07.443 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c139a50 con 0x7eff3c141a70 2023-10-26T05:47:07.443 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c139d30 con 0x7eff3c15c320 2023-10-26T05:47:07.443 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff415a9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c06cba0 con 0x7eff3c0cdbb0 2023-10-26T05:47:07.443 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 0x7eff3c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:07.443 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c15c320 0x7eff3c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:07.444 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 0x7eff3c155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45392/0 (socket says 172.21.15.71:45392) 2023-10-26T05:47:07.444 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3a7fc640 1 -- 172.21.15.71:0/4150734822 learned_addr learned my addr 172.21.15.71:0/4150734822 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:07.444 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 -- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff3c0cdbb0 msgr2=0x7eff3c0c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:07.444 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff3c0cdbb0 0x7eff3c0c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.444 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 -- 172.21.15.71:0/4150734822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 msgr2=0x7eff3c155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:07.445 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 0x7eff3c155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.445 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 -- 172.21.15.71:0/4150734822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0094a0 con 0x7eff3c15c320 2023-10-26T05:47:07.445 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3a7fc640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 0x7eff3c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:07.445 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.432+0000 7eff3b7fe640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c15c320 0x7eff3c15a440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c700 tx=0x7eff3400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:07.445 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34003b80 con 0x7eff3c15c320 2023-10-26T05:47:07.446 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff34003ce0 con 0x7eff3c15c320 2023-10-26T05:47:07.446 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c13a010 con 0x7eff3c15c320 2023-10-26T05:47:07.446 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34005330 con 0x7eff3c15c320 2023-10-26T05:47:07.446 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c156350 con 0x7eff3c15c320 2023-10-26T05:47:07.446 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 2) v1 ==== 42893+0+0 (secure 0 0 0) 0x7eff34005490 con 0x7eff3c15c320 2023-10-26T05:47:07.447 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c066d00 con 0x7eff3c15c320 2023-10-26T05:47:07.447 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7eff34046840 con 0x7eff3c15c320 2023-10-26T05:47:07.447 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.436+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7eff34005740 con 0x7eff3c15c320 2023-10-26T05:47:07.502 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff3c0cafb0 con 0x7eff3c15c320 2023-10-26T05:47:07.503 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff237fe640 1 -- 172.21.15.71:0/4150734822 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7eff3401a360 con 0x7eff3c15c320 2023-10-26T05:47:07.503 INFO:teuthology.misc.health.smithi071.stdout: 2023-10-26T05:47:07.504 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":11,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:07.257387+0000","last_up_change":"0.000000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:07.505 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c15c320 msgr2=0x7eff3c15a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:07.506 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c15c320 0x7eff3c15a440 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c700 tx=0x7eff3400cbc0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.506 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 shutdown_connections 2023-10-26T05:47:07.506 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff3c15c320 0x7eff3c15a440 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.507 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c141a70 0x7eff3c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.507 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 --2- 172.21.15.71:0/4150734822 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff3c0cdbb0 0x7eff3c0c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:07.507 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 >> 172.21.15.71:0/4150734822 conn(0x7eff3c0afeb0 msgr2=0x7eff3c0af180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:07.508 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 shutdown_connections 2023-10-26T05:47:07.508 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:07.496+0000 7eff415a9640 1 -- 172.21.15.71:0/4150734822 wait complete. 2023-10-26T05:47:07.518 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-26T05:47:07.590 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:07.584+0000 7fbeae4ae180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-26T05:47:07.649 INFO:tasks.ceph.mgr.y.smithi071.stderr:2023-10-26T05:47:07.644+0000 7fbeae4ae180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-26T05:47:08.295 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-26T05:47:08.293+0000 7f1d4928b640 -1 osd.5 0 waiting for initial osdmap 2023-10-26T05:47:08.297 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-26T05:47:08.297+0000 7f1d428d4640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.303 INFO:tasks.ceph.osd.0.smithi071.stderr:2023-10-26T05:47:08.296+0000 7f3856987640 -1 osd.0 0 waiting for initial osdmap 2023-10-26T05:47:08.303 INFO:tasks.ceph.osd.1.smithi071.stderr:2023-10-26T05:47:08.296+0000 7f30a7a97640 -1 osd.1 0 waiting for initial osdmap 2023-10-26T05:47:08.304 INFO:tasks.ceph.osd.3.smithi071.stderr:2023-10-26T05:47:08.296+0000 7fad50579640 -1 osd.3 0 waiting for initial osdmap 2023-10-26T05:47:08.306 INFO:tasks.ceph.osd.1.smithi071.stderr:2023-10-26T05:47:08.300+0000 7f30a10e0640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.307 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-26T05:47:08.305+0000 7fac4800b640 -1 osd.7 0 waiting for initial osdmap 2023-10-26T05:47:08.307 INFO:tasks.ceph.osd.0.smithi071.stderr:2023-10-26T05:47:08.300+0000 7f384ffd0640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.307 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-26T05:47:08.305+0000 7f8f8b472640 -1 osd.4 0 waiting for initial osdmap 2023-10-26T05:47:08.308 INFO:tasks.ceph.osd.3.smithi071.stderr:2023-10-26T05:47:08.300+0000 7fad49bc2640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.309 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-26T05:47:08.309+0000 7f8f84abb640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.309 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-26T05:47:08.309+0000 7fac41654640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.310 INFO:tasks.ceph.osd.2.smithi071.stderr:2023-10-26T05:47:08.304+0000 7f4d1c4e2640 -1 osd.2 0 waiting for initial osdmap 2023-10-26T05:47:08.310 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-26T05:47:08.309+0000 7f54bae0c640 -1 osd.6 0 waiting for initial osdmap 2023-10-26T05:47:08.312 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-26T05:47:08.313+0000 7f54b4455640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.312 INFO:tasks.ceph.osd.2.smithi071.stderr:2023-10-26T05:47:08.304+0000 7f4d15b2b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-26T05:47:08.419 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:08.417+0000 7fef13612640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-26T05:47:08.420 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-26T05:47:08.417+0000 7fef13612640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-26T05:47:14.325 DEBUG:teuthology.orchestra.run.smithi071:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:14.327 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:47:14.331 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:47:14.455 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 -- 172.21.15.71:0/3725832891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066c0047d0 msgr2=0x7f066c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.455 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 --2- 172.21.15.71:0/3725832891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066c0047d0 0x7f066c004c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0668009b40 tx=0x7f066801d230 comp rx=0 tx=0).stop 2023-10-26T05:47:14.455 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 -- 172.21.15.71:0/3725832891 shutdown_connections 2023-10-26T05:47:14.456 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 --2- 172.21.15.71:0/3725832891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f066c0047d0 0x7f066c004c50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.456 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 -- 172.21.15.71:0/3725832891 >> 172.21.15.71:0/3725832891 conn(0x7f0680070650 msgr2=0x7f0680070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:14.456 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 -- 172.21.15.71:0/3725832891 shutdown_connections 2023-10-26T05:47:14.456 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 -- 172.21.15.71:0/3725832891 wait complete. 2023-10-26T05:47:14.458 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 Processor -- start 2023-10-26T05:47:14.459 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.448+0000 7f06865c0640 1 -- start start 2023-10-26T05:47:14.459 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f066c0047d0 0x7f06800b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.459 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06800ab1d0 0x7f06800ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.459 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 0x7f068013e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.460 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06800ad010 con 0x7f068013e0a0 2023-10-26T05:47:14.460 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06800abb90 con 0x7f066c0047d0 2023-10-26T05:47:14.460 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068013edf0 con 0x7f06800ab1d0 2023-10-26T05:47:14.460 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 0x7f068013e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.460 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 0x7f068013e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46018/0 (socket says 172.21.15.71:46018) 2023-10-26T05:47:14.461 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 -- 172.21.15.71:0/128950788 learned_addr learned my addr 172.21.15.71:0/128950788 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:14.461 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067f7fe640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06800ab1d0 0x7f06800ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.461 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067ffff640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f066c0047d0 0x7f06800b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.461 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 -- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06800ab1d0 msgr2=0x7f06800ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.461 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06800ab1d0 0x7f06800ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.462 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 -- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f066c0047d0 msgr2=0x7f06800b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.462 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f066c0047d0 0x7f06800b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.462 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 -- 172.21.15.71:0/128950788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068013f0d0 con 0x7f068013e0a0 2023-10-26T05:47:14.462 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f0684b5d640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 0x7f068013e520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0680069d80 tx=0x7f067000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:14.463 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06700052d0 con 0x7f068013e0a0 2023-10-26T05:47:14.463 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0670005430 con 0x7f068013e0a0 2023-10-26T05:47:14.463 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0670003860 con 0x7f068013e0a0 2023-10-26T05:47:14.463 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068013f3b0 con 0x7f068013e0a0 2023-10-26T05:47:14.463 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068013f970 con 0x7f068013e0a0 2023-10-26T05:47:14.464 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0680066d00 con 0x7f068013e0a0 2023-10-26T05:47:14.464 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f06700039c0 con 0x7f068013e0a0 2023-10-26T05:47:14.465 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067cff9640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f06600664a0 0x7f0660068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.465 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0670082e00 con 0x7f068013e0a0 2023-10-26T05:47:14.465 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067ffff640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f06600664a0 0x7f0660068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.465 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.452+0000 7f067ffff640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f06600664a0 0x7f0660068960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0678001570 tx=0x7f067800c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:14.473 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.468+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f06700505d0 con 0x7f068013e0a0 2023-10-26T05:47:14.627 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.620+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06800b3120 con 0x7f068013e0a0 2023-10-26T05:47:14.628 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.620+0000 7f067cff9640 1 -- 172.21.15.71:0/128950788 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f0670017090 con 0x7f068013e0a0 2023-10-26T05:47:14.628 INFO:teuthology.misc.health.smithi071.stdout: 2023-10-26T05:47:14.629 INFO:teuthology.misc.health.smithi071.stdout:{"epoch":16,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:12.331843+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:14.630 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f06600664a0 msgr2=0x7f0660068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.631 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f06600664a0 0x7f0660068960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0678001570 tx=0x7f067800c040 comp rx=0 tx=0).stop 2023-10-26T05:47:14.631 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 msgr2=0x7f068013e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.631 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 0x7f068013e520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0680069d80 tx=0x7f067000c820 comp rx=0 tx=0).stop 2023-10-26T05:47:14.631 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 shutdown_connections 2023-10-26T05:47:14.632 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f06600664a0 0x7f0660068960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.632 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068013e0a0 0x7f068013e520 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.632 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06800ab1d0 0x7f06800ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.632 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 --2- 172.21.15.71:0/128950788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f066c0047d0 0x7f06800b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.632 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 >> 172.21.15.71:0/128950788 conn(0x7f0680070650 msgr2=0x7f0680158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:14.633 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 shutdown_connections 2023-10-26T05:47:14.633 INFO:teuthology.misc.health.smithi071.stderr:2023-10-26T05:47:14.624+0000 7f06865c0640 1 -- 172.21.15.71:0/128950788 wait complete. 2023-10-26T05:47:14.644 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-26T05:47:14.645 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-26T05:47:14.645 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:47:14.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:47:14.690 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:47:14.691 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:47:14.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:47:14.694 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:47:14.695 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:47:14.696 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:47:14.699 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:47:14.700 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:47:14.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:47:14.702 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:47:14.703 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-26T05:47:14.703 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:14.766 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:47:14.770 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:47:14.781 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:47:14.786 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:47:14.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/1631532131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 msgr2=0x7fbd540ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/1631532131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd540ce300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd540702d0 tx=0x7fbd4801c8a0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/1631532131 shutdown_connections 2023-10-26T05:47:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/1631532131 >> v2:172.21.15.86:3300/0 conn(0x7fbd5415d880 0x7fbd540cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/1631532131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd540ce300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/1631532131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd540cbad0 0x7fbd5406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/1631532131 >> 172.21.15.71:0/1631532131 conn(0x7fbd5406fd90 msgr2=0x7fbd540b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:14.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/1631532131 shutdown_connections 2023-10-26T05:47:14.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/1631532131 wait complete. 2023-10-26T05:47:14.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 Processor -- start 2023-10-26T05:47:14.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- start start 2023-10-26T05:47:14.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd540cbad0 0x7fbd541f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd541559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd5415d880 0x7fbd54150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54153a80 con 0x7fbd541594c0 2023-10-26T05:47:14.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd541f6de0 con 0x7fbd540cbad0 2023-10-26T05:47:14.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd541534c0 con 0x7fbd5415d880 2023-10-26T05:47:14.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd541559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd541559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46078/0 (socket says 172.21.15.71:46078) 2023-10-26T05:47:14.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 -- 172.21.15.71:0/2012900105 learned_addr learned my addr 172.21.15.71:0/2012900105 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:14.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd53fff640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd540cbad0 0x7fbd541f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd58ec7640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd5415d880 0x7fbd54150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 -- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd5415d880 msgr2=0x7fbd54150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd5415d880 0x7fbd54150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 -- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd540cbad0 msgr2=0x7fbd541f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd540cbad0 0x7fbd541f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 -- 172.21.15.71:0/2012900105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48009560 con 0x7fbd541594c0 2023-10-26T05:47:14.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd537fe640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd541559d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd4801af50 tx=0x7fbd480a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:14.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48005220 con 0x7fbd541594c0 2023-10-26T05:47:14.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd48005380 con 0x7fbd541594c0 2023-10-26T05:47:14.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd541537a0 con 0x7fbd541594c0 2023-10-26T05:47:14.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480046d0 con 0x7fbd541594c0 2023-10-26T05:47:14.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.812+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540c6610 con 0x7fbd541594c0 2023-10-26T05:47:14.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.816+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd540c6170 con 0x7fbd541594c0 2023-10-26T05:47:14.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.816+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbd480b4040 con 0x7fbd541594c0 2023-10-26T05:47:14.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.816+0000 7fbd50ff9640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd40066470 0x7fbd40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:14.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.816+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd4811dcc0 con 0x7fbd541594c0 2023-10-26T05:47:14.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.816+0000 7fbd53fff640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd40066470 0x7fbd40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:14.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.816+0000 7fbd53fff640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd40066470 0x7fbd40068930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009990 tx=0x7fbd44009110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:14.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.820+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbd480eb540 con 0x7fbd541594c0 2023-10-26T05:47:14.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.968+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd54066d00 con 0x7fbd541594c0 2023-10-26T05:47:14.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd50ff9640 1 -- 172.21.15.71:0/2012900105 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12766 (secure 0 0 0) 0x7fbd480b2090 con 0x7fbd541594c0 2023-10-26T05:47:14.978 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:14.978 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":16,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:12.331843+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:14.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd40066470 msgr2=0x7fbd40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd40066470 0x7fbd40068930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009990 tx=0x7fbd44009110 comp rx=0 tx=0).stop 2023-10-26T05:47:14.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 msgr2=0x7fbd541559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:14.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd541559d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbd4801af50 tx=0x7fbd480a57a0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 shutdown_connections 2023-10-26T05:47:14.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd5415d880 0x7fbd54150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd40066470 0x7fbd40068930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd541594c0 0x7fbd541559d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 --2- 172.21.15.71:0/2012900105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd540cbad0 0x7fbd541f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:14.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 >> 172.21.15.71:0/2012900105 conn(0x7fbd5406fd90 msgr2=0x7fbd540aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:14.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 shutdown_connections 2023-10-26T05:47:14.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:14.972+0000 7fbd5a92a640 1 -- 172.21.15.71:0/2012900105 wait complete. 2023-10-26T05:47:14.995 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:14.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.104+0000 7f42e0f1e640 1 -- 172.21.15.71:0/3509152394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42c8004d10 msgr2=0x7f42c8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.104+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/3509152394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42c8004d10 0x7f42c8005190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009e30 tx=0x7f42d401d150 comp rx=0 tx=0).stop 2023-10-26T05:47:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.104+0000 7f42dad9d640 1 -- 172.21.15.71:0/3509152394 reap_dead start 2023-10-26T05:47:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.104+0000 7f42e0f1e640 1 -- 172.21.15.71:0/3509152394 shutdown_connections 2023-10-26T05:47:15.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.104+0000 7f42e0f1e640 1 -- 172.21.15.71:0/3509152394 >> 172.21.15.71:0/3509152394 conn(0x7f42dc0aec60 msgr2=0x7f42dc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:15.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- 172.21.15.71:0/3509152394 shutdown_connections 2023-10-26T05:47:15.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- 172.21.15.71:0/3509152394 wait complete. 2023-10-26T05:47:15.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 Processor -- start 2023-10-26T05:47:15.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- start start 2023-10-26T05:47:15.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42dc14b920 0x7f42dc1f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42dc14bd00 0x7f42dc15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 0x7f42dc0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc15e370 con 0x7f42dc15dec0 2023-10-26T05:47:15.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c2000 con 0x7f42dc14b920 2023-10-26T05:47:15.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0c22e0 con 0x7f42dc14bd00 2023-10-26T05:47:15.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 0x7f42dc0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 0x7f42dc0c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46092/0 (socket says 172.21.15.71:46092) 2023-10-26T05:47:15.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 -- 172.21.15.71:0/547607499 learned_addr learned my addr 172.21.15.71:0/547607499 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:15.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42da59c640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42dc14b920 0x7f42dc1f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42d1d9b640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42dc14bd00 0x7f42dc15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 -- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42dc14bd00 msgr2=0x7f42dc15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42dc14bd00 0x7f42dc15d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 -- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42dc14b920 msgr2=0x7f42dc1f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42dc14b920 0x7f42dc1f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 -- 172.21.15.71:0/547607499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0c25c0 con 0x7f42dc15dec0 2023-10-26T05:47:15.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42dad9d640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 0x7f42dc0c4670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42d4002240 tx=0x7f42d40ada10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:15.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40adc90 con 0x7f42dc15dec0 2023-10-26T05:47:15.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f42d4004a50 con 0x7f42dc15dec0 2023-10-26T05:47:15.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004de0 con 0x7f42dc15dec0 2023-10-26T05:47:15.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc0c28a0 con 0x7f42dc15dec0 2023-10-26T05:47:15.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc0c4bb0 con 0x7f42dc15dec0 2023-10-26T05:47:15.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.108+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc066b10 con 0x7f42dc15dec0 2023-10-26T05:47:15.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.112+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f42d40b3070 con 0x7f42dc15dec0 2023-10-26T05:47:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.112+0000 7f42d37fe640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f42bc0664a0 0x7f42bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.112+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42d40a5e40 con 0x7f42dc15dec0 2023-10-26T05:47:15.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.112+0000 7f42da59c640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f42bc0664a0 0x7f42bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.112+0000 7f42da59c640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f42bc0664a0 0x7f42bc068960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f42cc001c30 tx=0x7f42cc000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:15.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.116+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f42d40ec1e0 con 0x7f42dc15dec0 2023-10-26T05:47:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.256+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42dc0721e0 con 0x7f42dc15dec0 2023-10-26T05:47:15.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.256+0000 7f42d37fe640 1 -- 172.21.15.71:0/547607499 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42d40b10a0 con 0x7f42dc15dec0 2023-10-26T05:47:15.264 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:15.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f42bc0664a0 msgr2=0x7f42bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f42bc0664a0 0x7f42bc068960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f42cc001c30 tx=0x7f42cc000ee0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 msgr2=0x7f42dc0c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 0x7f42dc0c4670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42d4002240 tx=0x7f42d40ada10 comp rx=0 tx=0).stop 2023-10-26T05:47:15.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 shutdown_connections 2023-10-26T05:47:15.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f42bc0664a0 0x7f42bc068960 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42dc15dec0 0x7f42dc0c4670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42dc14bd00 0x7f42dc15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 --2- 172.21.15.71:0/547607499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42dc14b920 0x7f42dc1f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 >> 172.21.15.71:0/547607499 conn(0x7f42dc0aec60 msgr2=0x7f42dc0b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:15.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 shutdown_connections 2023-10-26T05:47:15.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.260+0000 7f42e0f1e640 1 -- 172.21.15.71:0/547607499 wait complete. 2023-10-26T05:47:15.281 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:47:15.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 -- 172.21.15.71:0/800427998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c0b0040 msgr2=0x7fec4c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 --2- 172.21.15.71:0/800427998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c0b0040 0x7fec4c0b04b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06ed70 tx=0x7fec4401cd30 comp rx=0 tx=0).stop 2023-10-26T05:47:15.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 -- 172.21.15.71:0/800427998 shutdown_connections 2023-10-26T05:47:15.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 --2- 172.21.15.71:0/800427998 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c070fe0 0x7fec4c071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 --2- 172.21.15.71:0/800427998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c0b0040 0x7fec4c0b04b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 --2- 172.21.15.71:0/800427998 >> v2:172.21.15.86:3300/0 conn(0x7fec4c140cc0 0x7fec4c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 -- 172.21.15.71:0/800427998 >> 172.21.15.71:0/800427998 conn(0x7fec4c069770 msgr2=0x7fec4c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:15.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 -- 172.21.15.71:0/800427998 shutdown_connections 2023-10-26T05:47:15.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.400+0000 7fec5232e640 1 -- 172.21.15.71:0/800427998 wait complete. 2023-10-26T05:47:15.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 Processor -- start 2023-10-26T05:47:15.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- start start 2023-10-26T05:47:15.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec4c070fe0 0x7fec4c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c0b0040 0x7fec4c0c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 0x7fec4c15b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15ade0 con 0x7fec4c140cc0 2023-10-26T05:47:15.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c158130 con 0x7fec4c070fe0 2023-10-26T05:47:15.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c15a160 con 0x7fec4c0b0040 2023-10-26T05:47:15.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 0x7fec4c15b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 0x7fec4c15b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46120/0 (socket says 172.21.15.71:46120) 2023-10-26T05:47:15.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 -- 172.21.15.71:0/1073411593 learned_addr learned my addr 172.21.15.71:0/1073411593 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:15.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec4bfff640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec4c070fe0 0x7fec4c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec43fff640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c0b0040 0x7fec4c0c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 -- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c0b0040 msgr2=0x7fec4c0c1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c0b0040 0x7fec4c0c1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 -- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec4c070fe0 msgr2=0x7fec4c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec4c070fe0 0x7fec4c157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 -- 172.21.15.71:0/1073411593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440098e0 con 0x7fec4c140cc0 2023-10-26T05:47:15.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec43fff640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c0b0040 0x7fec4c0c1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:15.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec508cb640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 0x7fec4c15b200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec4401b3e0 tx=0x7fec440a5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:15.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44004e60 con 0x7fec4c140cc0 2023-10-26T05:47:15.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec440ad5b0 con 0x7fec4c140cc0 2023-10-26T05:47:15.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c15a440 con 0x7fec4c140cc0 2023-10-26T05:47:15.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec440ad980 con 0x7fec4c140cc0 2023-10-26T05:47:15.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c15aa00 con 0x7fec4c140cc0 2023-10-26T05:47:15.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c066b10 con 0x7fec4c140cc0 2023-10-26T05:47:15.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fec440b4040 con 0x7fec4c140cc0 2023-10-26T05:47:15.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec497fa640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec2c066470 0x7fec2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:15.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.404+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec4411e240 con 0x7fec4c140cc0 2023-10-26T05:47:15.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.408+0000 7fec4bfff640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec2c066470 0x7fec2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:15.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.408+0000 7fec4bfff640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec2c066470 0x7fec2c068930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0058d0 tx=0x7fec3c005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:15.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.408+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fec440ebad0 con 0x7fec4c140cc0 2023-10-26T05:47:15.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.584+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec4c15af20 con 0x7fec4c140cc0 2023-10-26T05:47:15.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.584+0000 7fec497fa640 1 -- 172.21.15.71:0/1073411593 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fec440bc540 con 0x7fec4c140cc0 2023-10-26T05:47:15.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:15.593 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-26T05:47:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 2 2023-10-26T05:47:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec2c066470 msgr2=0x7fec2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec2c066470 0x7fec2c068930 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0058d0 tx=0x7fec3c005f90 comp rx=0 tx=0).stop 2023-10-26T05:47:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 msgr2=0x7fec4c15b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 0x7fec4c15b200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec4401b3e0 tx=0x7fec440a5c20 comp rx=0 tx=0).stop 2023-10-26T05:47:15.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 shutdown_connections 2023-10-26T05:47:15.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec2c066470 0x7fec2c068930 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec4c140cc0 0x7fec4c15b200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec4c0b0040 0x7fec4c0c1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 --2- 172.21.15.71:0/1073411593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec4c070fe0 0x7fec4c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:15.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 >> 172.21.15.71:0/1073411593 conn(0x7fec4c069770 msgr2=0x7fec4c0ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 shutdown_connections 2023-10-26T05:47:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:15.588+0000 7fec5232e640 1 -- 172.21.15.71:0/1073411593 wait complete. 2023-10-26T05:47:17.609 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:47:17.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/3606543707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 msgr2=0x7fd8b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:17.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/3606543707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8b4003dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800af40 tx=0x7fd8b801e270 comp rx=0 tx=0).stop 2023-10-26T05:47:17.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c77fe640 1 -- 172.21.15.71:0/3606543707 reap_dead start 2023-10-26T05:47:17.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/3606543707 shutdown_connections 2023-10-26T05:47:17.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/3606543707 >> 172.21.15.71:0/3606543707 conn(0x7fd8c8069770 msgr2=0x7fd8c8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:17.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/3606543707 shutdown_connections 2023-10-26T05:47:17.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/3606543707 wait complete. 2023-10-26T05:47:17.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 Processor -- start 2023-10-26T05:47:17.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- start start 2023-10-26T05:47:17.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8c806a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:17.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8c80ae8c0 0x7fd8c80accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:17.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8c80ad1f0 0x7fd8c814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:17.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80ae430 con 0x7fd8b40039b0 2023-10-26T05:47:17.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c81625e0 con 0x7fd8c80ad1f0 2023-10-26T05:47:17.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80ad820 con 0x7fd8c80ae8c0 2023-10-26T05:47:17.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8c806a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:17.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8c806a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46140/0 (socket says 172.21.15.71:46140) 2023-10-26T05:47:17.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 -- 172.21.15.71:0/1392684728 learned_addr learned my addr 172.21.15.71:0/1392684728 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:17.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c77fe640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8c80ad1f0 0x7fd8c814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:17.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c67fc640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8c80ae8c0 0x7fd8c80accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:17.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 -- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8c80ae8c0 msgr2=0x7fd8c80accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:17.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8c80ae8c0 0x7fd8c80accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 -- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8c80ad1f0 msgr2=0x7fd8c814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:17.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8c80ad1f0 0x7fd8c814d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 -- 172.21.15.71:0/1392684728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c80adb00 con 0x7fd8b40039b0 2023-10-26T05:47:17.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c77fe640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8c80ad1f0 0x7fd8c814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:17.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8c6ffd640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8c806a860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800ae60 tx=0x7fd8b8004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:17.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80be070 con 0x7fd8b40039b0 2023-10-26T05:47:17.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd8b8004e80 con 0x7fd8b40039b0 2023-10-26T05:47:17.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c80adde0 con 0x7fd8b40039b0 2023-10-26T05:47:17.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8003bc0 con 0x7fd8b40039b0 2023-10-26T05:47:17.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.736+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c806ada0 con 0x7fd8b40039b0 2023-10-26T05:47:17.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.740+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c8066b10 con 0x7fd8b40039b0 2023-10-26T05:47:17.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.740+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd8b8005040 con 0x7fd8b40039b0 2023-10-26T05:47:17.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.740+0000 7fd8bf7fe640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8a8066470 0x7fd8a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:17.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.740+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8b811e830 con 0x7fd8b40039b0 2023-10-26T05:47:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.740+0000 7fd8c67fc640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8a8066470 0x7fd8a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.740+0000 7fd8c67fc640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8a8066470 0x7fd8a8068930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0001ec0 tx=0x7fd8b0001d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.744+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd8b80ec000 con 0x7fd8b40039b0 2023-10-26T05:47:17.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.920+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8c806af90 con 0x7fd8b40039b0 2023-10-26T05:47:17.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.920+0000 7fd8bf7fe640 1 -- 172.21.15.71:0/1392684728 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3688 (secure 0 0 0) 0x7fd8b80b2070 con 0x7fd8b40039b0 2023-10-26T05:47:17.927 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:17.928 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4369,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-26T05:47:17.928 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 3 2023-10-26T05:47:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8a8066470 msgr2=0x7fd8a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8a8066470 0x7fd8a8068930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0001ec0 tx=0x7fd8b0001d50 comp rx=0 tx=0).stop 2023-10-26T05:47:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 msgr2=0x7fd8c806a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8c806a860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800ae60 tx=0x7fd8b8004ae0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 shutdown_connections 2023-10-26T05:47:17.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8c80ad1f0 0x7fd8c814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8c80ae8c0 0x7fd8c80accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8a8066470 0x7fd8a8068930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 --2- 172.21.15.71:0/1392684728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8b40039b0 0x7fd8c806a860 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:17.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 >> 172.21.15.71:0/1392684728 conn(0x7fd8c8069770 msgr2=0x7fd8c815a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:17.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 shutdown_connections 2023-10-26T05:47:17.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:17.924+0000 7fd8cd5b8640 1 -- 172.21.15.71:0/1392684728 wait complete. 2023-10-26T05:47:17.945 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:18.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/2501801251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 msgr2=0x7f2630003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 --2- 172.21.15.71:0/2501801251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f2630003db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f264000c260 tx=0x7f264001cfb0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a9cf640 1 -- 172.21.15.71:0/2501801251 reap_dead start 2023-10-26T05:47:18.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/2501801251 shutdown_connections 2023-10-26T05:47:18.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/2501801251 >> 172.21.15.71:0/2501801251 conn(0x7f2644069770 msgr2=0x7f2644069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/2501801251 shutdown_connections 2023-10-26T05:47:18.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/2501801251 wait complete. 2023-10-26T05:47:18.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 Processor -- start 2023-10-26T05:47:18.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- start start 2023-10-26T05:47:18.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f26440a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26441383d0 0x7f26440a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f26440a7170 0x7f264406a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2644139c00 con 0x7f26300039b0 2023-10-26T05:47:18.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2644139ee0 con 0x7f26441383d0 2023-10-26T05:47:18.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264413a1c0 con 0x7f26440a7170 2023-10-26T05:47:18.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f26440a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f26440a8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46150/0 (socket says 172.21.15.71:46150) 2023-10-26T05:47:18.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 -- 172.21.15.71:0/1268814169 learned_addr learned my addr 172.21.15.71:0/1268814169 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:18.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 -- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f26440a7170 msgr2=0x7f264406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a9cf640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f26440a7170 0x7f264406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f26499cd640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26441383d0 0x7f26440a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f26440a7170 0x7f264406a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 -- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26441383d0 msgr2=0x7f26440a6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26441383d0 0x7f26440a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 -- 172.21.15.71:0/1268814169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264413a4a0 con 0x7f26300039b0 2023-10-26T05:47:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264a1ce640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f26440a8c30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f264001cde0 tx=0x7f2640001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400b3070 con 0x7f26300039b0 2023-10-26T05:47:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f26400015b0 con 0x7f26300039b0 2023-10-26T05:47:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26440a7610 con 0x7f26300039b0 2023-10-26T05:47:18.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640003600 con 0x7f26300039b0 2023-10-26T05:47:18.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26441398c0 con 0x7f26300039b0 2023-10-26T05:47:18.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.064+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2644066b10 con 0x7f26300039b0 2023-10-26T05:47:18.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.068+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f26400ab020 con 0x7f26300039b0 2023-10-26T05:47:18.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.068+0000 7f263affd640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2628066470 0x7f2628068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.068+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f264011d6a0 con 0x7f26300039b0 2023-10-26T05:47:18.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.068+0000 7f26499cd640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2628066470 0x7f2628068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.068+0000 7f26499cd640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2628066470 0x7f2628068930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f264406a300 tx=0x7f2634005f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.072+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f26400eae70 con 0x7f26300039b0 2023-10-26T05:47:18.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.220+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2644139d40 con 0x7f26300039b0 2023-10-26T05:47:18.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.220+0000 7f263affd640 1 -- 172.21.15.71:0/1268814169 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f2644139d40 con 0x7f26300039b0 2023-10-26T05:47:18.228 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:18.228 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":16,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:12.331843+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:18.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2628066470 msgr2=0x7f2628068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2628066470 0x7f2628068930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f264406a300 tx=0x7f2634005f90 comp rx=0 tx=0).stop 2023-10-26T05:47:18.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 msgr2=0x7f26440a8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f26440a8c30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f264001cde0 tx=0x7f2640001440 comp rx=0 tx=0).stop 2023-10-26T05:47:18.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 shutdown_connections 2023-10-26T05:47:18.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f26440a7170 0x7f264406a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26441383d0 0x7f26440a6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2628066470 0x7f2628068930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 --2- 172.21.15.71:0/1268814169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26300039b0 0x7f26440a8c30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 >> 172.21.15.71:0/1268814169 conn(0x7f2644069770 msgr2=0x7f264413f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 shutdown_connections 2023-10-26T05:47:18.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.224+0000 7f264c432640 1 -- 172.21.15.71:0/1268814169 wait complete. 2023-10-26T05:47:18.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:18.245 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:18.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/427118293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc1592a0 msgr2=0x7fcedc06ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 --2- 172.21.15.71:0/427118293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc1592a0 0x7fcedc06ee10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009840 tx=0x7fcecc01c7d0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/427118293 shutdown_connections 2023-10-26T05:47:18.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 --2- 172.21.15.71:0/427118293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc1592a0 0x7fcedc06ee10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/427118293 >> 172.21.15.71:0/427118293 conn(0x7fcedc06d320 msgr2=0x7fcedc06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/427118293 shutdown_connections 2023-10-26T05:47:18.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/427118293 wait complete. 2023-10-26T05:47:18.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 Processor -- start 2023-10-26T05:47:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- start start 2023-10-26T05:47:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0cbb50 0x7fcedc14ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcedc0ce940 0x7fcedc14b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 0x7fcedc157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0c7c60 con 0x7fcedc1592a0 2023-10-26T05:47:18.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc14bfc0 con 0x7fcedc0cbb50 2023-10-26T05:47:18.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc14b850 con 0x7fcedc0ce940 2023-10-26T05:47:18.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 0x7fcedc157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 0x7fcedc157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46152/0 (socket says 172.21.15.71:46152) 2023-10-26T05:47:18.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 -- 172.21.15.71:0/3526280431 learned_addr learned my addr 172.21.15.71:0/3526280431 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:18.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 -- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcedc0ce940 msgr2=0x7fcedc14b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcedc0ce940 0x7fcedc14b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 -- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0cbb50 msgr2=0x7fcedc14ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0cbb50 0x7fcedc14ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 -- 172.21.15.71:0/3526280431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0094a0 con 0x7fcedc1592a0 2023-10-26T05:47:18.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcedbfff640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 0x7fcedc157900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fced400c770 tx=0x7fced400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4003b80 con 0x7fcedc1592a0 2023-10-26T05:47:18.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc0af8a0 con 0x7fcedc1592a0 2023-10-26T05:47:18.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fced4003ce0 con 0x7fcedc1592a0 2023-10-26T05:47:18.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced400aab0 con 0x7fcedc1592a0 2023-10-26T05:47:18.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.364+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc0afe60 con 0x7fcedc1592a0 2023-10-26T05:47:18.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.368+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc066b10 con 0x7fcedc1592a0 2023-10-26T05:47:18.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.368+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fced400ac10 con 0x7fcedc1592a0 2023-10-26T05:47:18.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.368+0000 7fcec3fff640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcec80664a0 0x7fcec8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.368+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fced4083b70 con 0x7fcedc1592a0 2023-10-26T05:47:18.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.368+0000 7fcedb7fe640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcec80664a0 0x7fcec8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.368+0000 7fcedb7fe640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcec80664a0 0x7fcec8068960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0af000 tx=0x7fcecc019570 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.372+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fced40512b0 con 0x7fcedc1592a0 2023-10-26T05:47:18.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.524+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcedc066d00 con 0x7fcedc1592a0 2023-10-26T05:47:18.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.524+0000 7fcec3fff640 1 -- 172.21.15.71:0/3526280431 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fced40548d0 con 0x7fcedc1592a0 2023-10-26T05:47:18.531 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:18.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcec80664a0 msgr2=0x7fcec8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcec80664a0 0x7fcec8068960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0af000 tx=0x7fcecc019570 comp rx=0 tx=0).stop 2023-10-26T05:47:18.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 msgr2=0x7fcedc157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 0x7fcedc157900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fced400c770 tx=0x7fced400cc30 comp rx=0 tx=0).stop 2023-10-26T05:47:18.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 shutdown_connections 2023-10-26T05:47:18.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcec80664a0 0x7fcec8068960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcedc1592a0 0x7fcedc157900 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcedc0ce940 0x7fcedc14b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 --2- 172.21.15.71:0/3526280431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcedc0cbb50 0x7fcedc14ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 >> 172.21.15.71:0/3526280431 conn(0x7fcedc06d320 msgr2=0x7fcedc0654c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 shutdown_connections 2023-10-26T05:47:18.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.528+0000 7fcee20d1640 1 -- 172.21.15.71:0/3526280431 wait complete. 2023-10-26T05:47:18.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:47:18.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/990605345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc070a60 msgr2=0x7f2ccc15cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/990605345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc070a60 0x7f2ccc15cde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc06f740 tx=0x7f2cc001c860 comp rx=0 tx=0).stop 2023-10-26T05:47:18.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/990605345 shutdown_connections 2023-10-26T05:47:18.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/990605345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc070a60 0x7f2ccc15cde0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/990605345 >> v2:172.21.15.86:3300/0 conn(0x7f2ccc13f5a0 0x7f2ccc0b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/990605345 >> 172.21.15.71:0/990605345 conn(0x7f2ccc06ff40 msgr2=0x7f2ccc070350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/990605345 shutdown_connections 2023-10-26T05:47:18.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/990605345 wait complete. 2023-10-26T05:47:18.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 Processor -- start 2023-10-26T05:47:18.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- start start 2023-10-26T05:47:18.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 0x7f2ccc14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc070a60 0x7f2ccc14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ccc13f5a0 0x7f2ccc14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc15ce60 con 0x7f2ccc06b010 2023-10-26T05:47:18.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc15bfb0 con 0x7f2ccc070a60 2023-10-26T05:47:18.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc15c250 con 0x7f2ccc13f5a0 2023-10-26T05:47:18.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 0x7f2ccc14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 0x7f2ccc14ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46172/0 (socket says 172.21.15.71:46172) 2023-10-26T05:47:18.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 -- 172.21.15.71:0/1310031776 learned_addr learned my addr 172.21.15.71:0/1310031776 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:18.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2ccbfff640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc070a60 0x7f2ccc14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd1070640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ccc13f5a0 0x7f2ccc14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 -- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ccc13f5a0 msgr2=0x7f2ccc14f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ccc13f5a0 0x7f2ccc14f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 -- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc070a60 msgr2=0x7f2ccc14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc070a60 0x7f2ccc14f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 -- 172.21.15.71:0/1310031776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0009560 con 0x7f2ccc06b010 2023-10-26T05:47:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd1070640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ccc13f5a0 0x7f2ccc14f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd086f640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 0x7f2ccc14ef50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400eaa0 tx=0x7f2cc400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400ce10 con 0x7f2ccc06b010 2023-10-26T05:47:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2cc4011470 con 0x7f2ccc06b010 2023-10-26T05:47:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc15c530 con 0x7f2ccc06b010 2023-10-26T05:47:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4011860 con 0x7f2ccc06b010 2023-10-26T05:47:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.664+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc15caf0 con 0x7f2ccc06b010 2023-10-26T05:47:18.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.668+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc066d00 con 0x7f2ccc06b010 2023-10-26T05:47:18.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.668+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2cc40119c0 con 0x7f2ccc06b010 2023-10-26T05:47:18.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.668+0000 7f2cc97fa640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cb8066470 0x7f2cb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.668+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2cc4084730 con 0x7f2ccc06b010 2023-10-26T05:47:18.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.668+0000 7f2ccbfff640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cb8066470 0x7f2cb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.668+0000 7f2ccbfff640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cb8066470 0x7f2cb8068930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00af000 tx=0x7f2cc001b0a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.672+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2cc4055990 con 0x7f2ccc06b010 2023-10-26T05:47:18.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2ccc15a270 con 0x7f2ccc06b010 2023-10-26T05:47:18.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cc97fa640 1 -- 172.21.15.71:0/1310031776 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2cc401f090 con 0x7f2ccc06b010 2023-10-26T05:47:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:18.823 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:47:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:47:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:47:18.823 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:47:18.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:47:18.824 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:47:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cb8066470 msgr2=0x7f2cb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cb8066470 0x7f2cb8068930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00af000 tx=0x7f2cc001b0a0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 msgr2=0x7f2ccc14ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 0x7f2ccc14ef50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400eaa0 tx=0x7f2cc400ef60 comp rx=0 tx=0).stop 2023-10-26T05:47:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 shutdown_connections 2023-10-26T05:47:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ccc13f5a0 0x7f2ccc14f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ccc070a60 0x7f2ccc14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cb8066470 0x7f2cb8068930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 --2- 172.21.15.71:0/1310031776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ccc06b010 0x7f2ccc14ef50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 >> 172.21.15.71:0/1310031776 conn(0x7f2ccc06ff40 msgr2=0x7f2ccc0ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 shutdown_connections 2023-10-26T05:47:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.816+0000 7f2cd2ad3640 1 -- 172.21.15.71:0/1310031776 wait complete. 2023-10-26T05:47:18.840 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:47:18.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:47:18.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 -- 172.21.15.71:0/3552792300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 msgr2=0x7f1f4c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 --2- 172.21.15.71:0/3552792300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c06ac90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009900 tx=0x7f1f4001c850 comp rx=0 tx=0).stop 2023-10-26T05:47:18.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 -- 172.21.15.71:0/3552792300 shutdown_connections 2023-10-26T05:47:18.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 --2- 172.21.15.71:0/3552792300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c06ac90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 -- 172.21.15.71:0/3552792300 >> 172.21.15.71:0/3552792300 conn(0x7f1f4c0d13f0 msgr2=0x7f1f4c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:18.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 -- 172.21.15.71:0/3552792300 shutdown_connections 2023-10-26T05:47:18.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.948+0000 7f1f52472640 1 -- 172.21.15.71:0/3552792300 wait complete. 2023-10-26T05:47:18.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 Processor -- start 2023-10-26T05:47:18.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- start start 2023-10-26T05:47:18.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c167bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4c0cf2f0 0x7f1f4c0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f4c15b4d0 0x7f1f4c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c15b0f0 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c8f40 con 0x7f1f4c0cf2f0 2023-10-26T05:47:18.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c9190 con 0x7f1f4c15b4d0 2023-10-26T05:47:18.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c167bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c167bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46192/0 (socket says 172.21.15.71:46192) 2023-10-26T05:47:18.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 -- 172.21.15.71:0/476472088 learned_addr learned my addr 172.21.15.71:0/476472088 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:18.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f50a0f640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f4c15b4d0 0x7f1f4c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4b7fe640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4c0cf2f0 0x7f1f4c0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 -- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f4c15b4d0 msgr2=0x7f1f4c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f4c15b4d0 0x7f1f4c0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 -- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4c0cf2f0 msgr2=0x7f1f4c0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:18.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4c0cf2f0 0x7f1f4c0c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:18.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 -- 172.21.15.71:0/476472088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f40009560 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f50a0f640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f4c15b4d0 0x7f1f4c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:18.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4b7fe640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4c0cf2f0 0x7f1f4c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:18.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f4bfff640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c167bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00b410 tx=0x7f1f3c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011830 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1f3c011a80 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c158df0 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c011e90 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c168130 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c066b10 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.952+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1f3c01a400 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.956+0000 7f1f48ff9640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1f38066470 0x7f1f38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:18.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.956+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f3c083e50 con 0x7f1f4c06a7e0 2023-10-26T05:47:18.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.956+0000 7f1f4b7fe640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1f38066470 0x7f1f38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:18.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.956+0000 7f1f4b7fe640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1f38066470 0x7f1f38068930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f400af000 tx=0x7f1f4001b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:18.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:18.960+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1f3c0516b0 con 0x7f1f4c06a7e0 2023-10-26T05:47:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:19.108+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1f4c0721e0 con 0x7f1f4c06a7e0 2023-10-26T05:47:20.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.044+0000 7f1f48ff9640 1 -- 172.21.15.71:0/476472088 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1f3c015020 con 0x7f1f4c06a7e0 2023-10-26T05:47:20.052 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:47:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1f38066470 msgr2=0x7f1f38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1f38066470 0x7f1f38068930 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f400af000 tx=0x7f1f4001b090 comp rx=0 tx=0).stop 2023-10-26T05:47:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 msgr2=0x7f1f4c167bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:20.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c167bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00b410 tx=0x7f1f3c00b8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 shutdown_connections 2023-10-26T05:47:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f4c15b4d0 0x7f1f4c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f4c0cf2f0 0x7f1f4c0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1f38066470 0x7f1f38068930 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 --2- 172.21.15.71:0/476472088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f4c06a7e0 0x7f1f4c167bf0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00b410 tx=0x7f1f3c00b8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 >> 172.21.15.71:0/476472088 conn(0x7f1f4c0d13f0 msgr2=0x7f1f4c15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:20.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 shutdown_connections 2023-10-26T05:47:20.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.048+0000 7f1f52472640 1 -- 172.21.15.71:0/476472088 wait complete. 2023-10-26T05:47:20.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:47:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/391529255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 msgr2=0x7f6bbc0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/391529255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc0cc440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0ce850 tx=0x7f6bac01c870 comp rx=0 tx=0).stop 2023-10-26T05:47:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/391529255 shutdown_connections 2023-10-26T05:47:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/391529255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc0cc440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/391529255 >> v2:172.21.15.86:3300/0 conn(0x7f6bbc156e90 0x7f6bbc06c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/391529255 >> 172.21.15.71:0/391529255 conn(0x7f6bbc0cd770 msgr2=0x7f6bbc0cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/391529255 shutdown_connections 2023-10-26T05:47:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/391529255 wait complete. 2023-10-26T05:47:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 Processor -- start 2023-10-26T05:47:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- start start 2023-10-26T05:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bbc0cc980 0x7f6bbc14d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bbc156e90 0x7f6bbc070400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc14d9e0 con 0x7f6bbc15a280 2023-10-26T05:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc152360 con 0x7f6bbc0cc980 2023-10-26T05:47:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc152640 con 0x7f6bbc156e90 2023-10-26T05:47:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc070940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46198/0 (socket says 172.21.15.71:46198) 2023-10-26T05:47:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 -- 172.21.15.71:0/2255259711 learned_addr learned my addr 172.21.15.71:0/2255259711 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1373640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bbc0cc980 0x7f6bbc14d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:20.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc0b72640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bbc156e90 0x7f6bbc070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 -- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bbc156e90 msgr2=0x7f6bbc070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bbc156e90 0x7f6bbc070400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 -- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bbc0cc980 msgr2=0x7f6bbc14d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bbc0cc980 0x7f6bbc14d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 -- 172.21.15.71:0/2255259711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac009560 con 0x7f6bbc15a280 2023-10-26T05:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc1b74640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc070940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c770 tx=0x7f6bb400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4003b80 con 0x7f6bbc15a280 2023-10-26T05:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6bb4003ce0 con 0x7f6bbc15a280 2023-10-26T05:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc152920 con 0x7f6bbc15a280 2023-10-26T05:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb400aab0 con 0x7f6bbc15a280 2023-10-26T05:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc1515f0 con 0x7f6bbc15a280 2023-10-26T05:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.196+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc066b10 con 0x7f6bbc15a280 2023-10-26T05:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.200+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6bb400ac10 con 0x7f6bbc15a280 2023-10-26T05:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.200+0000 7f6bb1ffb640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ba0066470 0x7f6ba0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.200+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6bb4083c70 con 0x7f6bbc15a280 2023-10-26T05:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.200+0000 7f6bc1373640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ba0066470 0x7f6ba0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.200+0000 7f6bc1373640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ba0066470 0x7f6ba0068930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8005fd0 tx=0x7f6bb8005db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:20.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.204+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6bb4051340 con 0x7f6bbc15a280 2023-10-26T05:47:20.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:20.360+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6bbc066d00 con 0x7f6bbc15a280 2023-10-26T05:47:21.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bb1ffb640 1 -- 172.21.15.71:0/2255259711 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6bb4054960 con 0x7f6bbc15a280 2023-10-26T05:47:21.058 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:47:21.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ba0066470 msgr2=0x7f6ba0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:21.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ba0066470 0x7f6ba0068930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8005fd0 tx=0x7f6bb8005db0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 msgr2=0x7f6bbc070940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:21.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc070940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c770 tx=0x7f6bb400cc30 comp rx=0 tx=0).stop 2023-10-26T05:47:21.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 shutdown_connections 2023-10-26T05:47:21.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ba0066470 0x7f6ba0068930 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bbc15a280 0x7f6bbc070940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bbc156e90 0x7f6bbc070400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 --2- 172.21.15.71:0/2255259711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bbc0cc980 0x7f6bbc14d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.052+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 >> 172.21.15.71:0/2255259711 conn(0x7f6bbc0cd770 msgr2=0x7f6bbc153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:21.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.056+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 shutdown_connections 2023-10-26T05:47:21.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.056+0000 7f6bc35d7640 1 -- 172.21.15.71:0/2255259711 wait complete. 2023-10-26T05:47:21.087 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:47:21.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- 172.21.15.71:0/364312285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242c004d10 msgr2=0x7f242c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:21.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 --2- 172.21.15.71:0/364312285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f242c004d10 0x7f242c005190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f243c009ee0 tx=0x7f243c01d5d0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f24437fe640 1 -- 172.21.15.71:0/364312285 reap_dead start 2023-10-26T05:47:21.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- 172.21.15.71:0/364312285 shutdown_connections 2023-10-26T05:47:21.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- 172.21.15.71:0/364312285 >> 172.21.15.71:0/364312285 conn(0x7f24440cdd80 msgr2=0x7f24440ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:21.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- 172.21.15.71:0/364312285 shutdown_connections 2023-10-26T05:47:21.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- 172.21.15.71:0/364312285 wait complete. 2023-10-26T05:47:21.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 Processor -- start 2023-10-26T05:47:21.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- start start 2023-10-26T05:47:21.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242c004d10 0x7f2444156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:21.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 0x7f244414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24441f1870 0x7f24441f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244406a650 con 0x7f24440cc480 2023-10-26T05:47:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c64f0 con 0x7f242c004d10 2023-10-26T05:47:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.204+0000 7f2449925640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c67d0 con 0x7f24441f1870 2023-10-26T05:47:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 0x7f244414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:21.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 0x7f244414f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46206/0 (socket says 172.21.15.71:46206) 2023-10-26T05:47:21.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 -- 172.21.15.71:0/1522581284 learned_addr learned my addr 172.21.15.71:0/1522581284 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:21.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f2442ffd640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242c004d10 0x7f2444156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:21.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 -- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24441f1870 msgr2=0x7f24441f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:21.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24437fe640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24441f1870 0x7f24441f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:21.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24441f1870 0x7f24441f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 -- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242c004d10 msgr2=0x7f2444156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:21.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242c004d10 0x7f2444156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:21.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 -- 172.21.15.71:0/1522581284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440c6ab0 con 0x7f24440cc480 2023-10-26T05:47:21.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f24427fc640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 0x7f244414f7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24380126c0 tx=0x7f2438012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:21.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438010ae0 con 0x7f24440cc480 2023-10-26T05:47:21.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2438010c60 con 0x7f24440cc480 2023-10-26T05:47:21.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440c6d90 con 0x7f24440cc480 2023-10-26T05:47:21.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380136b0 con 0x7f24440cc480 2023-10-26T05:47:21.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244414fd20 con 0x7f24440cc480 2023-10-26T05:47:21.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2444066b10 con 0x7f24440cc480 2023-10-26T05:47:21.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2438019020 con 0x7f24440cc480 2023-10-26T05:47:21.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f242bfff640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2430066470 0x7f2430068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:21.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2438087b20 con 0x7f24440cc480 2023-10-26T05:47:21.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f2442ffd640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2430066470 0x7f2430068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:21.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.208+0000 7f2442ffd640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2430066470 0x7f2430068930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f244415e2d0 tx=0x7f2434006b40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:21.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.212+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2438055000 con 0x7f24440cc480 2023-10-26T05:47:21.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:21.400+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f24440721e0 con 0x7f24440cc480 2023-10-26T05:47:22.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.092+0000 7f242bfff640 1 -- 172.21.15.71:0/1522581284 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2438022020 con 0x7f24440cc480 2023-10-26T05:47:22.098 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 2 and data pool 3 2023-10-26T05:47:22.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2430066470 msgr2=0x7f2430068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2430066470 0x7f2430068930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f244415e2d0 tx=0x7f2434006b40 comp rx=0 tx=0).stop 2023-10-26T05:47:22.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 msgr2=0x7f244414f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 0x7f244414f7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24380126c0 tx=0x7f2438012b80 comp rx=0 tx=0).stop 2023-10-26T05:47:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 shutdown_connections 2023-10-26T05:47:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f24441f1870 0x7f24441f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2430066470 0x7f2430068930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24440cc480 0x7f244414f7e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 --2- 172.21.15.71:0/1522581284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f242c004d10 0x7f2444156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 >> 172.21.15.71:0/1522581284 conn(0x7f24440cdd80 msgr2=0x7f244415a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 shutdown_connections 2023-10-26T05:47:22.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.096+0000 7f2449925640 1 -- 172.21.15.71:0/1522581284 wait complete. 2023-10-26T05:47:22.128 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76cd864640 1 -- 172.21.15.71:0/2035456703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76b00047d0 msgr2=0x7f76b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76cd864640 1 --2- 172.21.15.71:0/2035456703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76b00047d0 0x7f76b0004c50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006f20 tx=0x7f76bc01f690 comp rx=0 tx=0).stop 2023-10-26T05:47:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76c77fe640 1 -- 172.21.15.71:0/2035456703 reap_dead start 2023-10-26T05:47:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76cd864640 1 -- 172.21.15.71:0/2035456703 shutdown_connections 2023-10-26T05:47:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76cd864640 1 -- 172.21.15.71:0/2035456703 >> 172.21.15.71:0/2035456703 conn(0x7f76c8069770 msgr2=0x7f76c8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76cd864640 1 -- 172.21.15.71:0/2035456703 shutdown_connections 2023-10-26T05:47:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.252+0000 7f76cd864640 1 -- 172.21.15.71:0/2035456703 wait complete. 2023-10-26T05:47:22.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 Processor -- start 2023-10-26T05:47:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- start start 2023-10-26T05:47:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76b00047d0 0x7f76c8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 0x7f76c80b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c80b0680 0x7f76c80ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c8139a30 con 0x7f76c8137190 2023-10-26T05:47:22.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80b1910 con 0x7f76b00047d0 2023-10-26T05:47:22.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c80b21a0 con 0x7f76c80b0680 2023-10-26T05:47:22.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 0x7f76c80b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 0x7f76c80b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53006/0 (socket says 172.21.15.71:53006) 2023-10-26T05:47:22.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 -- 172.21.15.71:0/801321535 learned_addr learned my addr 172.21.15.71:0/801321535 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:22.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c77fe640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c80b0680 0x7f76c80ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c6ffd640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76b00047d0 0x7f76c8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 -- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c80b0680 msgr2=0x7f76c80ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c80b0680 0x7f76c80ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 -- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76b00047d0 msgr2=0x7f76c8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76b00047d0 0x7f76c8150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 -- 172.21.15.71:0/801321535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c8142290 con 0x7f76c8137190 2023-10-26T05:47:22.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76c67fc640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 0x7f76c80b0140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76bc01f4c0 tx=0x7f76bc0065c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:22.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc003600 con 0x7f76c8137190 2023-10-26T05:47:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76bc003760 con 0x7f76c8137190 2023-10-26T05:47:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c8141fb0 con 0x7f76c8137190 2023-10-26T05:47:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc003b70 con 0x7f76c8137190 2023-10-26T05:47:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c80b1ec0 con 0x7f76c8137190 2023-10-26T05:47:22.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c8066b10 con 0x7f76c8137190 2023-10-26T05:47:22.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f76bc003cd0 con 0x7f76c8137190 2023-10-26T05:47:22.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.256+0000 7f76affff640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f76b4066470 0x7f76b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.260+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76bc121640 con 0x7f76c8137190 2023-10-26T05:47:22.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.260+0000 7f76c6ffd640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f76b4066470 0x7f76b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.260+0000 7f76c6ffd640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f76b4066470 0x7f76b4068930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76c81f4af0 tx=0x7f76b8009770 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:22.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.264+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f76bc0eeb50 con 0x7f76c8137190 2023-10-26T05:47:22.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.420+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76c813a9d0 con 0x7f76c8137190 2023-10-26T05:47:22.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.420+0000 7f76affff640 1 -- 172.21.15.71:0/801321535 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16434 (secure 0 0 0) 0x7f76bc0f2170 con 0x7f76c8137190 2023-10-26T05:47:22.429 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:22.429 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":19,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:21.405080+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:22.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f76b4066470 msgr2=0x7f76b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f76b4066470 0x7f76b4068930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f76c81f4af0 tx=0x7f76b8009770 comp rx=0 tx=0).stop 2023-10-26T05:47:22.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 msgr2=0x7f76c80b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 0x7f76c80b0140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76bc01f4c0 tx=0x7f76bc0065c0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 shutdown_connections 2023-10-26T05:47:22.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76c80b0680 0x7f76c80ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f76b4066470 0x7f76b4068930 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76c8137190 0x7f76c80b0140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 --2- 172.21.15.71:0/801321535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76b00047d0 0x7f76c8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 >> 172.21.15.71:0/801321535 conn(0x7f76c8069770 msgr2=0x7f76c80caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:22.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 shutdown_connections 2023-10-26T05:47:22.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.424+0000 7f76cd864640 1 -- 172.21.15.71:0/801321535 wait complete. 2023-10-26T05:47:22.447 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:22.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 -- 172.21.15.71:0/1297314594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0b2890 msgr2=0x7f0bfc0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 --2- 172.21.15.71:0/1297314594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0b2890 0x7f0bfc0caad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc06f190 tx=0x7f0be401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 -- 172.21.15.71:0/1297314594 shutdown_connections 2023-10-26T05:47:22.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 --2- 172.21.15.71:0/1297314594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0b2890 0x7f0bfc0caad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 --2- 172.21.15.71:0/1297314594 >> v2:172.21.15.86:3300/0 conn(0x7f0bfc13c2f0 0x7f0bfc0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 -- 172.21.15.71:0/1297314594 >> 172.21.15.71:0/1297314594 conn(0x7f0bfc069770 msgr2=0x7f0bfc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:22.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 -- 172.21.15.71:0/1297314594 shutdown_connections 2023-10-26T05:47:22.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.560+0000 7f0c01964640 1 -- 172.21.15.71:0/1297314594 wait complete. 2023-10-26T05:47:22.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 Processor -- start 2023-10-26T05:47:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- start start 2023-10-26T05:47:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bfc0b2890 0x7f0bfc14f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 0x7f0bfc14d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bfc13c2f0 0x7f0bfc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc14dcf0 con 0x7f0bfc0cb010 2023-10-26T05:47:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc139c80 con 0x7f0bfc0b2890 2023-10-26T05:47:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc139f60 con 0x7f0bfc13c2f0 2023-10-26T05:47:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 0x7f0bfc14d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 0x7f0bfc14d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53028/0 (socket says 172.21.15.71:53028) 2023-10-26T05:47:22.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 -- 172.21.15.71:0/4219417828 learned_addr learned my addr 172.21.15.71:0/4219417828 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bfaffd640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bfc0b2890 0x7f0bfc14f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bfb7fe640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bfc13c2f0 0x7f0bfc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 -- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bfc13c2f0 msgr2=0x7f0bfc0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bfc13c2f0 0x7f0bfc0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 -- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bfc0b2890 msgr2=0x7f0bfc14f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bfc0b2890 0x7f0bfc14f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 -- 172.21.15.71:0/4219417828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be40094a0 con 0x7f0bfc0cb010 2023-10-26T05:47:22.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bfb7fe640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bfc13c2f0 0x7f0bfc0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:22.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf3fff640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 0x7f0bfc14d7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be40193d0 tx=0x7f0be40a8490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:22.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40b4070 con 0x7f0bfc0cb010 2023-10-26T05:47:22.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0be40b5040 con 0x7f0bfc0cb010 2023-10-26T05:47:22.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bfc13a240 con 0x7f0bfc0cb010 2023-10-26T05:47:22.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40054c0 con 0x7f0bfc0cb010 2023-10-26T05:47:22.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bfc13a800 con 0x7f0bfc0cb010 2023-10-26T05:47:22.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bfc066b10 con 0x7f0bfc0cb010 2023-10-26T05:47:22.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.564+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0be40a6040 con 0x7f0bfc0cb010 2023-10-26T05:47:22.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.568+0000 7f0bf8ff9640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bdc066470 0x7f0bdc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.568+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0be411eca0 con 0x7f0bfc0cb010 2023-10-26T05:47:22.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.568+0000 7f0bfaffd640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bdc066470 0x7f0bdc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.568+0000 7f0bfaffd640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bdc066470 0x7f0bdc068930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0058d0 tx=0x7f0bec005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:22.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.572+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0be40ec240 con 0x7f0bfc0cb010 2023-10-26T05:47:22.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.724+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0bfc13a9f0 con 0x7f0bfc0cb010 2023-10-26T05:47:22.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0bf8ff9640 1 -- 172.21.15.71:0/4219417828 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16434 (secure 0 0 0) 0x7f0be40ef860 con 0x7f0bfc0cb010 2023-10-26T05:47:22.734 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:22.735 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":19,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:21.405080+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:22.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bdc066470 msgr2=0x7f0bdc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bdc066470 0x7f0bdc068930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0058d0 tx=0x7f0bec005860 comp rx=0 tx=0).stop 2023-10-26T05:47:22.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 msgr2=0x7f0bfc14d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 0x7f0bfc14d7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be40193d0 tx=0x7f0be40a8490 comp rx=0 tx=0).stop 2023-10-26T05:47:22.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 shutdown_connections 2023-10-26T05:47:22.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bfc13c2f0 0x7f0bfc0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bdc066470 0x7f0bdc068930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bfc0cb010 0x7f0bfc14d7b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 --2- 172.21.15.71:0/4219417828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bfc0b2890 0x7f0bfc14f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 >> 172.21.15.71:0/4219417828 conn(0x7f0bfc069770 msgr2=0x7f0bfc159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 shutdown_connections 2023-10-26T05:47:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.728+0000 7f0c01964640 1 -- 172.21.15.71:0/4219417828 wait complete. 2023-10-26T05:47:22.754 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:47:22.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/4230159608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840c99c0 msgr2=0x7f3c8406ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/4230159608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840c99c0 0x7f3c8406ffd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009900 tx=0x7f3c7401c830 comp rx=0 tx=0).stop 2023-10-26T05:47:22.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/4230159608 shutdown_connections 2023-10-26T05:47:22.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/4230159608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840c99c0 0x7f3c8406ffd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/4230159608 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 0x7f3c840ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/4230159608 >> 172.21.15.71:0/4230159608 conn(0x7f3c840af7d0 msgr2=0x7f3c840afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:22.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/4230159608 shutdown_connections 2023-10-26T05:47:22.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/4230159608 wait complete. 2023-10-26T05:47:22.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 Processor -- start 2023-10-26T05:47:22.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- start start 2023-10-26T05:47:22.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c840c99c0 0x7f3c8413a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 0x7f3c8413b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 0x7f3c8413d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84154fa0 con 0x7f3c840cbf40 2023-10-26T05:47:22.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84140140 con 0x7f3c840c99c0 2023-10-26T05:47:22.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8c2a2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841403e0 con 0x7f3c8413c1a0 2023-10-26T05:47:22.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8983d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 0x7f3c8413b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8983d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 0x7f3c8413b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53046/0 (socket says 172.21.15.71:53046) 2023-10-26T05:47:22.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8983d640 1 -- 172.21.15.71:0/2684393652 learned_addr learned my addr 172.21.15.71:0/2684393652 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:22.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8a03e640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c840c99c0 0x7f3c8413a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.872+0000 7f3c8a83f640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 0x7f3c8413d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8983d640 1 -- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 msgr2=0x7f3c8413d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8983d640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 0x7f3c8413d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8983d640 1 -- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c840c99c0 msgr2=0x7f3c8413a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:22.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8983d640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c840c99c0 0x7f3c8413a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:22.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8983d640 1 -- 172.21.15.71:0/2684393652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74009560 con 0x7f3c840cbf40 2023-10-26T05:47:22.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8a83f640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 0x7f3c8413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:22.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8983d640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 0x7f3c8413b130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009850 tx=0x7f3c740a65f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:22.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740bc070 con 0x7f3c840cbf40 2023-10-26T05:47:22.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c74004130 con 0x7f3c840cbf40 2023-10-26T05:47:22.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74005070 con 0x7f3c840cbf40 2023-10-26T05:47:22.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c841406c0 con 0x7f3c840cbf40 2023-10-26T05:47:22.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c841387f0 con 0x7f3c840cbf40 2023-10-26T05:47:22.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c84140b70 con 0x7f3c840cbf40 2023-10-26T05:47:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c740051d0 con 0x7f3c840cbf40 2023-10-26T05:47:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c7affd640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c68066470 0x7f3c68068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:22.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c740aa070 con 0x7f3c840cbf40 2023-10-26T05:47:22.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8a03e640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c68066470 0x7f3c68068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:22.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.876+0000 7f3c8a03e640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c68066470 0x7f3c68068930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c80005c50 tx=0x7f3c80005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:22.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:22.880+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3c740ebbd0 con 0x7f3c840cbf40 2023-10-26T05:47:23.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.064+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3c84066d00 con 0x7f3c840cbf40 2023-10-26T05:47:23.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.100+0000 7f3c7affd640 1 -- 172.21.15.71:0/2684393652 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3c740ef1f0 con 0x7f3c840cbf40 2023-10-26T05:47:23.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c68066470 msgr2=0x7f3c68068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:23.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c68066470 0x7f3c68068930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c80005c50 tx=0x7f3c80005be0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 msgr2=0x7f3c8413b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:23.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 0x7f3c8413b130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009850 tx=0x7f3c740a65f0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 shutdown_connections 2023-10-26T05:47:23.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c8413c1a0 0x7f3c8413d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c68066470 0x7f3c68068930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c840cbf40 0x7f3c8413b130 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 --2- 172.21.15.71:0/2684393652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c840c99c0 0x7f3c8413a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 >> 172.21.15.71:0/2684393652 conn(0x7f3c840af7d0 msgr2=0x7f3c840ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:23.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 shutdown_connections 2023-10-26T05:47:23.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.104+0000 7f3c8c2a2640 1 -- 172.21.15.71:0/2684393652 wait complete. 2023-10-26T05:47:23.125 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2023-10-26T05:47:23.126 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:47:23.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/2510297467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0d0850 msgr2=0x7f5eac15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:23.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/2510297467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0d0850 0x7f5eac15b3a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0009840 tx=0x7f5ea001c6b0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/2510297467 shutdown_connections 2023-10-26T05:47:23.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/2510297467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0d0850 0x7f5eac15b3a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/2510297467 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac0d0470 0x7f5eac06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/2510297467 >> 172.21.15.71:0/2510297467 conn(0x7f5eac0cdd80 msgr2=0x7f5eac0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:23.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/2510297467 shutdown_connections 2023-10-26T05:47:23.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/2510297467 wait complete. 2023-10-26T05:47:23.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 Processor -- start 2023-10-26T05:47:23.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- start start 2023-10-26T05:47:23.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 0x7f5eac0c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:23.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac0d0470 0x7f5eac14a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:23.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac0d0850 0x7f5eac158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac155e30 con 0x7f5eac0cd880 2023-10-26T05:47:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac156080 con 0x7f5eac0d0850 2023-10-26T05:47:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb3a8b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac156360 con 0x7f5eac0d0470 2023-10-26T05:47:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 0x7f5eac0c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:23.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 0x7f5eac0c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53066/0 (socket says 172.21.15.71:53066) 2023-10-26T05:47:23.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 -- 172.21.15.71:0/1818274727 learned_addr learned my addr 172.21.15.71:0/1818274727 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:23.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb2028640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac0d0850 0x7f5eac158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:23.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1026640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac0d0470 0x7f5eac14a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:23.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 -- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac0d0470 msgr2=0x7f5eac14a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:23.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac0d0470 0x7f5eac14a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 -- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac0d0850 msgr2=0x7f5eac158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:23.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac0d0850 0x7f5eac158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:23.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 -- 172.21.15.71:0/1818274727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00094a0 con 0x7f5eac0cd880 2023-10-26T05:47:23.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb2028640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac0d0850 0x7f5eac158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:23.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eb1827640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 0x7f5eac0c45f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009fd0 tx=0x7f5ea400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:23.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.236+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4011470 con 0x7f5eac0cd880 2023-10-26T05:47:23.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ea40116c0 con 0x7f5eac0cd880 2023-10-26T05:47:23.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac156640 con 0x7f5eac0cd880 2023-10-26T05:47:23.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4011ab0 con 0x7f5eac0cd880 2023-10-26T05:47:23.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac156c00 con 0x7f5eac0cd880 2023-10-26T05:47:23.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac066b10 con 0x7f5eac0cd880 2023-10-26T05:47:23.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ea4011c10 con 0x7f5eac0cd880 2023-10-26T05:47:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eaa7fc640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:23.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ea4084b70 con 0x7f5eac0cd880 2023-10-26T05:47:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eb1026640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.240+0000 7f5eb1026640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00af000 tx=0x7f5ea001aef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:23.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.244+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ea4052080 con 0x7f5eac0cd880 2023-10-26T05:47:23.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:23.432+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5eac0721e0 con 0x7f5eac0cd880 2023-10-26T05:47:24.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eaa7fc640 1 -- 172.21.15.71:0/1818274727 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5ea401f020 con 0x7f5eac0cd880 2023-10-26T05:47:24.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 msgr2=0x7f5e98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00af000 tx=0x7f5ea001aef0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 msgr2=0x7f5eac0c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 0x7f5eac0c45f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009fd0 tx=0x7f5ea400c270 comp rx=0 tx=0).stop 2023-10-26T05:47:24.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 shutdown_connections 2023-10-26T05:47:24.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eac0d0850 0x7f5eac158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eac0d0470 0x7f5eac14a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 --2- 172.21.15.71:0/1818274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eac0cd880 0x7f5eac0c45f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 >> 172.21.15.71:0/1818274727 conn(0x7f5eac0cdd80 msgr2=0x7f5eac158070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:24.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 shutdown_connections 2023-10-26T05:47:24.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.116+0000 7f5eb3a8b640 1 -- 172.21.15.71:0/1818274727 wait complete. 2023-10-26T05:47:24.143 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:47:24.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:47:24.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 -- 172.21.15.71:0/2469046414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80cb1e0 msgr2=0x7fc3b8159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/2469046414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b8159810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc3b806f190 tx=0x7fc3b001cb40 comp rx=0 tx=0).stop 2023-10-26T05:47:24.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 -- 172.21.15.71:0/2469046414 shutdown_connections 2023-10-26T05:47:24.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/2469046414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b8159810 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/2469046414 >> v2:172.21.15.86:3300/0 conn(0x7fc3b813c2f0 0x7fc3b80b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 -- 172.21.15.71:0/2469046414 >> 172.21.15.71:0/2469046414 conn(0x7fc3b8069770 msgr2=0x7fc3b8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 -- 172.21.15.71:0/2469046414 shutdown_connections 2023-10-26T05:47:24.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.260+0000 7fc3bdfed640 1 -- 172.21.15.71:0/2469046414 wait complete. 2023-10-26T05:47:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 Processor -- start 2023-10-26T05:47:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- start start 2023-10-26T05:47:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 0x7fc3b8137b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b813f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3b813c2f0 0x7fc3b8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b813e780 con 0x7fc3b80b2890 2023-10-26T05:47:24.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b813d8f0 con 0x7fc3b80cb1e0 2023-10-26T05:47:24.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b813db70 con 0x7fc3b813c2f0 2023-10-26T05:47:24.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 0x7fc3b8137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 0x7fc3b8137b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53080/0 (socket says 172.21.15.71:53080) 2023-10-26T05:47:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 -- 172.21.15.71:0/3503253352 learned_addr learned my addr 172.21.15.71:0/3503253352 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3affff640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b813f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b7fff640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3b813c2f0 0x7fc3b8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 -- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3b813c2f0 msgr2=0x7fc3b8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3b813c2f0 0x7fc3b8155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 -- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3b80cb1e0 msgr2=0x7fc3b813f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b813f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 -- 172.21.15.71:0/3503253352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b0009820 con 0x7fc3b80b2890 2023-10-26T05:47:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3affff640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b813f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:24.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b77fe640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 0x7fc3b8137b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c940 tx=0x7fc3a800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8004220 con 0x7fc3b80b2890 2023-10-26T05:47:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc3a8004470 con 0x7fc3b80b2890 2023-10-26T05:47:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b813de50 con 0x7fc3b80b2890 2023-10-26T05:47:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8007c10 con 0x7fc3b80b2890 2023-10-26T05:47:24.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b813e410 con 0x7fc3b80b2890 2023-10-26T05:47:24.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b8066b10 con 0x7fc3b80b2890 2023-10-26T05:47:24.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.264+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc3a8003600 con 0x7fc3b80b2890 2023-10-26T05:47:24.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.268+0000 7fc3b4ff9640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc398066470 0x7fc398068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.268+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a8084400 con 0x7fc3b80b2890 2023-10-26T05:47:24.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.268+0000 7fc3affff640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc398066470 0x7fc398068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.268+0000 7fc3affff640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc398066470 0x7fc398068930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8136db0 tx=0x7fc3a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:24.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.272+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc3a8051880 con 0x7fc3b80b2890 2023-10-26T05:47:24.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3b80721e0 con 0x7fc3b80b2890 2023-10-26T05:47:24.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3b4ff9640 1 -- 172.21.15.71:0/3503253352 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7fc3a8054ea0 con 0x7fc3b80b2890 2023-10-26T05:47:24.451 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:24.451 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:47:24.451 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:47:24.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc398066470 msgr2=0x7fc398068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc398066470 0x7fc398068930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8136db0 tx=0x7fc3a0009040 comp rx=0 tx=0).stop 2023-10-26T05:47:24.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 msgr2=0x7fc3b8137b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 0x7fc3b8137b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800c940 tx=0x7fc3a800ce00 comp rx=0 tx=0).stop 2023-10-26T05:47:24.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 shutdown_connections 2023-10-26T05:47:24.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3b813c2f0 0x7fc3b8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3b80cb1e0 0x7fc3b813f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc398066470 0x7fc398068930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 --2- 172.21.15.71:0/3503253352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b80b2890 0x7fc3b8137b30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 >> 172.21.15.71:0/3503253352 conn(0x7fc3b8069770 msgr2=0x7fc3b8069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:24.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 shutdown_connections 2023-10-26T05:47:24.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.444+0000 7fc3bdfed640 1 -- 172.21.15.71:0/3503253352 wait complete. 2023-10-26T05:47:24.468 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:24.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 -- 172.21.15.71:0/434268621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600d0470 msgr2=0x7fb26006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 --2- 172.21.15.71:0/434268621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600d0470 0x7fb26006aa70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb2600ccd20 tx=0x7fb25001c780 comp rx=0 tx=0).stop 2023-10-26T05:47:24.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 -- 172.21.15.71:0/434268621 shutdown_connections 2023-10-26T05:47:24.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 --2- 172.21.15.71:0/434268621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2600d0850 0x7fb26015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 --2- 172.21.15.71:0/434268621 >> v2:172.21.15.86:3300/0 conn(0x7fb2600cd880 0x7fb26015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 --2- 172.21.15.71:0/434268621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600d0470 0x7fb26006aa70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 -- 172.21.15.71:0/434268621 >> 172.21.15.71:0/434268621 conn(0x7fb2600cdd80 msgr2=0x7fb2600ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:24.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 -- 172.21.15.71:0/434268621 shutdown_connections 2023-10-26T05:47:24.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.568+0000 7fb265968640 1 -- 172.21.15.71:0/434268621 wait complete. 2023-10-26T05:47:24.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 Processor -- start 2023-10-26T05:47:24.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- start start 2023-10-26T05:47:24.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 0x7fb260157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2600d0470 0x7fb2600caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2600d0850 0x7fb2600c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600c8d00 con 0x7fb2600cd880 2023-10-26T05:47:24.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600c92c0 con 0x7fb2600d0850 2023-10-26T05:47:24.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2601559d0 con 0x7fb2600d0470 2023-10-26T05:47:24.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 0x7fb260157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 0x7fb260157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53094/0 (socket says 172.21.15.71:53094) 2023-10-26T05:47:24.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 -- 172.21.15.71:0/1575919182 learned_addr learned my addr 172.21.15.71:0/1575919182 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:24.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25f7fe640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2600d0850 0x7fb2600c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25e7fc640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2600d0470 0x7fb2600caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 -- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2600d0470 msgr2=0x7fb2600caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2600d0470 0x7fb2600caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 -- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2600d0850 msgr2=0x7fb2600c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2600d0850 0x7fb2600c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 -- 172.21.15.71:0/1575919182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250009560 con 0x7fb2600cd880 2023-10-26T05:47:24.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb25effd640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 0x7fb260157360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2500b0040 tx=0x7fb2500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:24.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250005230 con 0x7fb2600cd880 2023-10-26T05:47:24.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2500054a0 con 0x7fb2600cd880 2023-10-26T05:47:24.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250004930 con 0x7fb2600cd880 2023-10-26T05:47:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb260155cb0 con 0x7fb2600cd880 2023-10-26T05:47:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2601578a0 con 0x7fb2600cd880 2023-10-26T05:47:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb260066b10 con 0x7fb2600cd880 2023-10-26T05:47:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.572+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2500b4070 con 0x7fb2600cd880 2023-10-26T05:47:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.576+0000 7fb247fff640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb24c0664a0 0x7fb24c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.576+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb25011e250 con 0x7fb2600cd880 2023-10-26T05:47:24.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.576+0000 7fb25e7fc640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb24c0664a0 0x7fb24c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.576+0000 7fb25e7fc640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb24c0664a0 0x7fb24c068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2601f7f30 tx=0x7fb254005bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:24.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.580+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2500eb760 con 0x7fb2600cd880 2023-10-26T05:47:24.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.732+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb260155ea0 con 0x7fb2600cd880 2023-10-26T05:47:24.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb247fff640 1 -- 172.21.15.71:0/1575919182 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fb2500eed80 con 0x7fb2600cd880 2023-10-26T05:47:24.742 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:24.742 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:24.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb24c0664a0 msgr2=0x7fb24c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb24c0664a0 0x7fb24c068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2601f7f30 tx=0x7fb254005bd0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 msgr2=0x7fb260157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 0x7fb260157360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2500b0040 tx=0x7fb2500040c0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 shutdown_connections 2023-10-26T05:47:24.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2600d0850 0x7fb2600c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2600d0470 0x7fb2600caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb24c0664a0 0x7fb24c068960 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 --2- 172.21.15.71:0/1575919182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2600cd880 0x7fb260157360 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 >> 172.21.15.71:0/1575919182 conn(0x7fb2600cdd80 msgr2=0x7fb260156230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:24.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 shutdown_connections 2023-10-26T05:47:24.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.736+0000 7fb265968640 1 -- 172.21.15.71:0/1575919182 wait complete. 2023-10-26T05:47:24.761 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:47:24.762 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:24.795 INFO:tasks.ceph.mds.d.smithi086.stderr:2023-10-26T05:47:24.793+0000 7f06e55a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:47:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/2064497216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc80b1360 msgr2=0x7f4cc80b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 --2- 172.21.15.71:0/2064497216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc80b1360 0x7f4cc80b17d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0009c80 tx=0x7f4cc001cd50 comp rx=0 tx=0).stop 2023-10-26T05:47:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/2064497216 shutdown_connections 2023-10-26T05:47:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 --2- 172.21.15.71:0/2064497216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc80b1360 0x7f4cc80b17d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 --2- 172.21.15.71:0/2064497216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cc8070fe0 0x7f4cc8071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/2064497216 >> 172.21.15.71:0/2064497216 conn(0x7f4cc8069770 msgr2=0x7f4cc8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/2064497216 shutdown_connections 2023-10-26T05:47:24.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/2064497216 wait complete. 2023-10-26T05:47:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 Processor -- start 2023-10-26T05:47:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- start start 2023-10-26T05:47:24.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cc8070fe0 0x7f4cc8132370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cc80b1360 0x7f4cc8137520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 0x7f4cc806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8139af0 con 0x7f4cc8140cc0 2023-10-26T05:47:24.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc815a3d0 con 0x7f4cc80b1360 2023-10-26T05:47:24.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc80ab610 con 0x7f4cc8070fe0 2023-10-26T05:47:24.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 0x7f4cc806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 0x7f4cc806fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53108/0 (socket says 172.21.15.71:53108) 2023-10-26T05:47:24.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 -- 172.21.15.71:0/1144173222 learned_addr learned my addr 172.21.15.71:0/1144173222 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:24.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 -- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cc8070fe0 msgr2=0x7f4cc8132370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cc8070fe0 0x7f4cc8132370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 -- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cc80b1360 msgr2=0x7f4cc8137520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:24.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cc80b1360 0x7f4cc8137520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:24.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 -- 172.21.15.71:0/1144173222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00098e0 con 0x7f4cc8140cc0 2023-10-26T05:47:24.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccde16640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 0x7f4cc806fe80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4cc001a980 tx=0x7f4cc00ae7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:24.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00bc070 con 0x7f4cc8140cc0 2023-10-26T05:47:24.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4cc00aecb0 con 0x7f4cc8140cc0 2023-10-26T05:47:24.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc8159aa0 con 0x7f4cc8140cc0 2023-10-26T05:47:24.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0003be0 con 0x7f4cc8140cc0 2023-10-26T05:47:24.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.884+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc815a060 con 0x7f4cc8140cc0 2023-10-26T05:47:24.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.888+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8066b10 con 0x7f4cc8140cc0 2023-10-26T05:47:24.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.888+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4cc00b4040 con 0x7f4cc8140cc0 2023-10-26T05:47:24.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.888+0000 7f4cbdffb640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cac066470 0x7f4cac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:24.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.888+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cc011eeb0 con 0x7f4cc8140cc0 2023-10-26T05:47:24.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.888+0000 7f4ccd615640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cac066470 0x7f4cac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:24.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.888+0000 7f4ccd615640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cac066470 0x7f4cac068930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000f70 tx=0x7f4cc40079e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:24.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:24.892+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4cc00ec290 con 0x7f4cc8140cc0 2023-10-26T05:47:25.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.044+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4cc80cbc20 con 0x7f4cac066470 2023-10-26T05:47:25.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.064+0000 7f4cbdffb640 1 -- 172.21.15.71:0/1144173222 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248943 (secure 0 0 0) 0x7f4cc4005080 con 0x7f4cac066470 2023-10-26T05:47:25.070 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:25.073 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:25.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cac066470 msgr2=0x7f4cac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cac066470 0x7f4cac068930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000f70 tx=0x7f4cc40079e0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 msgr2=0x7f4cc806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 0x7f4cc806fe80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4cc001a980 tx=0x7f4cc00ae7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 shutdown_connections 2023-10-26T05:47:25.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cac066470 0x7f4cac068930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cc8140cc0 0x7f4cc806fe80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cc80b1360 0x7f4cc8137520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 --2- 172.21.15.71:0/1144173222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cc8070fe0 0x7f4cc8132370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 >> 172.21.15.71:0/1144173222 conn(0x7f4cc8069770 msgr2=0x7f4cc80b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:25.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 shutdown_connections 2023-10-26T05:47:25.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.068+0000 7f4ccf879640 1 -- 172.21.15.71:0/1144173222 wait complete. 2023-10-26T05:47:25.089 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-26T05:47:23.402285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560990,"statfs":{"total":1600000000,"available":1598454696,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001142"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-26T05:47:12.803951+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:12.803951+0000","last_peered":"2023-10-26T05:47:12.803951+0000","last_clean":"2023-10-26T05:47:12.803951+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:12.803951+0000","last_undegraded":"2023-10-26T05:47:12.803951+0000","last_fullsized":"2023-10-26T05:47:12.803951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":621,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194691,"statfs":{"total":200000000,"available":199364061,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":621,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194691,"statfs":{"total":200000000,"available":199364061,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:25.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:25.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/1221025144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 msgr2=0x7f23300caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 --2- 172.21.15.71:0/1221025144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 0x7f23300caad0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2324009900 tx=0x7f232401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/1221025144 shutdown_connections 2023-10-26T05:47:25.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 --2- 172.21.15.71:0/1221025144 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 0x7f23300caad0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 --2- 172.21.15.71:0/1221025144 >> v2:172.21.15.86:3300/0 conn(0x7f233013c2f0 0x7f23300b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/1221025144 >> 172.21.15.71:0/1221025144 conn(0x7f2330069770 msgr2=0x7f2330069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:25.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/1221025144 shutdown_connections 2023-10-26T05:47:25.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/1221025144 wait complete. 2023-10-26T05:47:25.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 Processor -- start 2023-10-26T05:47:25.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- start start 2023-10-26T05:47:25.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 0x7f2330134b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 0x7f233014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f233013c2f0 0x7f233014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233013b1e0 con 0x7f23300cb010 2023-10-26T05:47:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233013b670 con 0x7f233013c2f0 2023-10-26T05:47:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233014d8d0 con 0x7f23300b2890 2023-10-26T05:47:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 0x7f233014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 0x7f233014c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53124/0 (socket says 172.21.15.71:53124) 2023-10-26T05:47:25.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 -- 172.21.15.71:0/3928061559 learned_addr learned my addr 172.21.15.71:0/3928061559 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:25.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f233544d640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f233013c2f0 0x7f233014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2334c4c640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 0x7f2330134b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 -- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 msgr2=0x7f2330134b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 0x7f2330134b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 -- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f233013c2f0 msgr2=0x7f233014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f233013c2f0 0x7f233014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 -- 172.21.15.71:0/3928061559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23200094a0 con 0x7f23300cb010 2023-10-26T05:47:25.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232ffff640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 0x7f233014c0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f232401a4f0 tx=0x7f23240a66b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:25.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23240a6d70 con 0x7f23300cb010 2023-10-26T05:47:25.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2324004040 con 0x7f23300cb010 2023-10-26T05:47:25.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2324009560 con 0x7f23300cb010 2023-10-26T05:47:25.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2324004450 con 0x7f23300cb010 2023-10-26T05:47:25.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.240+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233013ab90 con 0x7f23300cb010 2023-10-26T05:47:25.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.244+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2330066b10 con 0x7f23300cb010 2023-10-26T05:47:25.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.244+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2324003600 con 0x7f23300cb010 2023-10-26T05:47:25.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.244+0000 7f232d7fa640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f231c066470 0x7f231c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.244+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23240aa070 con 0x7f23300cb010 2023-10-26T05:47:25.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.244+0000 7f2334c4c640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f231c066470 0x7f231c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.244+0000 7f2334c4c640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f231c066470 0x7f231c068930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2320009c60 tx=0x7f23200091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:25.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.248+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23240ebcc0 con 0x7f23300cb010 2023-10-26T05:47:25.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.384+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23300cb4d0 con 0x7f231c066470 2023-10-26T05:47:25.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.404+0000 7f232d7fa640 1 -- 172.21.15.71:0/3928061559 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248943 (secure 0 0 0) 0x7f23300cb4d0 con 0x7f231c066470 2023-10-26T05:47:25.413 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:25.414 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:25.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f231c066470 msgr2=0x7f231c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f231c066470 0x7f231c068930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2320009c60 tx=0x7f23200091e0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 msgr2=0x7f233014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 0x7f233014c0f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f232401a4f0 tx=0x7f23240a66b0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 shutdown_connections 2023-10-26T05:47:25.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f233013c2f0 0x7f233014c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f231c066470 0x7f231c068930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23300cb010 0x7f233014c0f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 --2- 172.21.15.71:0/3928061559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23300b2890 0x7f2330134b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 >> 172.21.15.71:0/3928061559 conn(0x7f2330069770 msgr2=0x7f23300acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 shutdown_connections 2023-10-26T05:47:25.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.408+0000 7f2336eb0640 1 -- 172.21.15.71:0/3928061559 wait complete. 2023-10-26T05:47:25.429 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-26T05:47:23.402285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560990,"statfs":{"total":1600000000,"available":1598454696,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001142"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-26T05:47:12.803951+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:12.803951+0000","last_peered":"2023-10-26T05:47:12.803951+0000","last_clean":"2023-10-26T05:47:12.803951+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:12.803951+0000","last_undegraded":"2023-10-26T05:47:12.803951+0000","last_fullsized":"2023-10-26T05:47:12.803951+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:21.048745+0000","last_change":"2023-10-26T05:47:21.048745+0000","last_active":"2023-10-26T05:47:21.048745+0000","last_peered":"2023-10-26T05:47:21.048745+0000","last_clean":"2023-10-26T05:47:21.048745+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:21.048745+0000","last_undegraded":"2023-10-26T05:47:21.048745+0000","last_fullsized":"2023-10-26T05:47:21.048745+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:47:20.038195+0000","last_change":"2023-10-26T05:47:20.038195+0000","last_active":"2023-10-26T05:47:20.038195+0000","last_peered":"2023-10-26T05:47:20.038195+0000","last_clean":"2023-10-26T05:47:20.038195+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:47:20.038195+0000","last_undegraded":"2023-10-26T05:47:20.038195+0000","last_fullsized":"2023-10-26T05:47:20.038195+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":621,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194691,"statfs":{"total":200000000,"available":199364061,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.075}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":44,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195267,"statfs":{"total":200000000,"available":199954429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":621,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194691,"statfs":{"total":200000000,"available":199364061,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:25.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/669117685 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00b2a60 msgr2=0x7fe0b00caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/669117685 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00b2a60 0x7fe0b00caad0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009900 tx=0x7fe0a401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/669117685 shutdown_connections 2023-10-26T05:47:25.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/669117685 >> v2:172.21.15.86:3300/0 conn(0x7fe0b00cb010 0x7fe0b0159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/669117685 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00b2a60 0x7fe0b00caad0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/669117685 >> 172.21.15.71:0/669117685 conn(0x7fe0b0069770 msgr2=0x7fe0b0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/669117685 shutdown_connections 2023-10-26T05:47:25.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/669117685 wait complete. 2023-10-26T05:47:25.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 Processor -- start 2023-10-26T05:47:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- start start 2023-10-26T05:47:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 0x7fe0b015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00cb010 0x7fe0b014bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b013c2f0 0x7fe0b0158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0151ea0 con 0x7fe0b00b2a60 2023-10-26T05:47:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014dd70 con 0x7fe0b013c2f0 2023-10-26T05:47:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b014cf20 con 0x7fe0b00cb010 2023-10-26T05:47:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 0x7fe0b015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 0x7fe0b015c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53128/0 (socket says 172.21.15.71:53128) 2023-10-26T05:47:25.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 -- 172.21.15.71:0/176812505 learned_addr learned my addr 172.21.15.71:0/176812505 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:25.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b53ac640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b013c2f0 0x7fe0b0158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0affff640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00cb010 0x7fe0b014bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 -- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00cb010 msgr2=0x7fe0b014bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00cb010 0x7fe0b014bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 -- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b013c2f0 msgr2=0x7fe0b0158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b013c2f0 0x7fe0b0158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 -- 172.21.15.71:0/176812505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4009560 con 0x7fe0b00b2a60 2023-10-26T05:47:25.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b53ac640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b013c2f0 0x7fe0b0158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:25.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b4bab640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 0x7fe0b015c7b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c370 tx=0x7fe0a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:25.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00052d0 con 0x7fe0b00b2a60 2023-10-26T05:47:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b014c680 con 0x7fe0b00b2a60 2023-10-26T05:47:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0a0007020 con 0x7fe0b00b2a60 2023-10-26T05:47:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0007430 con 0x7fe0b00b2a60 2023-10-26T05:47:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.588+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b014cc40 con 0x7fe0b00b2a60 2023-10-26T05:47:25.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.592+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b0066b10 con 0x7fe0b00b2a60 2023-10-26T05:47:25.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.592+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe0a00067e0 con 0x7fe0b00b2a60 2023-10-26T05:47:25.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.592+0000 7fe0ad7fa640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe09c0664a0 0x7fe09c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.592+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0a00835e0 con 0x7fe0b00b2a60 2023-10-26T05:47:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.592+0000 7fe0affff640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe09c0664a0 0x7fe09c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.592+0000 7fe0affff640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe09c0664a0 0x7fe09c068960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40af000 tx=0x7fe0a401b100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:25.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.596+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe0a00508b0 con 0x7fe0b00b2a60 2023-10-26T05:47:25.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.740+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0b00721e0 con 0x7fe09c0664a0 2023-10-26T05:47:25.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.760+0000 7fe0ad7fa640 1 -- 172.21.15.71:0/176812505 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260103 (secure 0 0 0) 0x7fe0b00721e0 con 0x7fe09c0664a0 2023-10-26T05:47:25.769 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:25.771 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:25.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe09c0664a0 msgr2=0x7fe09c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe09c0664a0 0x7fe09c068960 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40af000 tx=0x7fe0a401b100 comp rx=0 tx=0).stop 2023-10-26T05:47:25.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 msgr2=0x7fe0b015c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 0x7fe0b015c7b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c370 tx=0x7fe0a000c830 comp rx=0 tx=0).stop 2023-10-26T05:47:25.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 shutdown_connections 2023-10-26T05:47:25.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0b013c2f0 0x7fe0b0158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0b00cb010 0x7fe0b014bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe09c0664a0 0x7fe09c068960 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 --2- 172.21.15.71:0/176812505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0b00b2a60 0x7fe0b015c7b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 >> 172.21.15.71:0/176812505 conn(0x7fe0b0069770 msgr2=0x7fe0b00ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:25.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 shutdown_connections 2023-10-26T05:47:25.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.764+0000 7fe0b6e0f640 1 -- 172.21.15.71:0/176812505 wait complete. 2023-10-26T05:47:25.787 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-26T05:47:25.412242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248540,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.356123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:22.078297+0000","last_became_peered":"2023-10-26T05:47:22.078297+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032989+0000","last_change":"2023-10-26T05:47:22.050802+0000","last_active":"2023-10-26T05:47:23.032989+0000","last_peered":"2023-10-26T05:47:23.032989+0000","last_clean":"2023-10-26T05:47:23.032989+0000","last_became_active":"2023-10-26T05:47:22.049800+0000","last_became_peered":"2023-10-26T05:47:22.049800+0000","last_unstale":"2023-10-26T05:47:23.032989+0000","last_undegraded":"2023-10-26T05:47:23.032989+0000","last_fullsized":"2023-10-26T05:47:23.032989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595881+0000","last_change":"2023-10-26T05:47:22.059994+0000","last_active":"2023-10-26T05:47:23.595881+0000","last_peered":"2023-10-26T05:47:23.595881+0000","last_clean":"2023-10-26T05:47:23.595881+0000","last_became_active":"2023-10-26T05:47:22.058233+0000","last_became_peered":"2023-10-26T05:47:22.058233+0000","last_unstale":"2023-10-26T05:47:23.595881+0000","last_undegraded":"2023-10-26T05:47:23.595881+0000","last_fullsized":"2023-10-26T05:47:23.595881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:26.807782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296464+0000","last_change":"2023-10-26T05:47:22.069835+0000","last_active":"2023-10-26T05:47:23.296464+0000","last_peered":"2023-10-26T05:47:23.296464+0000","last_clean":"2023-10-26T05:47:23.296464+0000","last_became_active":"2023-10-26T05:47:22.069004+0000","last_became_peered":"2023-10-26T05:47:22.069004+0000","last_unstale":"2023-10-26T05:47:23.296464+0000","last_undegraded":"2023-10-26T05:47:23.296464+0000","last_fullsized":"2023-10-26T05:47:23.296464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299356+0000","last_change":"2023-10-26T05:47:22.074037+0000","last_active":"2023-10-26T05:47:23.299356+0000","last_peered":"2023-10-26T05:47:23.299356+0000","last_clean":"2023-10-26T05:47:23.299356+0000","last_became_active":"2023-10-26T05:47:22.073426+0000","last_became_peered":"2023-10-26T05:47:22.073426+0000","last_unstale":"2023-10-26T05:47:23.299356+0000","last_undegraded":"2023-10-26T05:47:23.299356+0000","last_fullsized":"2023-10-26T05:47:23.299356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297524+0000","last_change":"2023-10-26T05:47:22.056648+0000","last_active":"2023-10-26T05:47:23.297524+0000","last_peered":"2023-10-26T05:47:23.297524+0000","last_clean":"2023-10-26T05:47:23.297524+0000","last_became_active":"2023-10-26T05:47:22.051034+0000","last_became_peered":"2023-10-26T05:47:22.051034+0000","last_unstale":"2023-10-26T05:47:23.297524+0000","last_undegraded":"2023-10-26T05:47:23.297524+0000","last_fullsized":"2023-10-26T05:47:23.297524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300018+0000","last_change":"2023-10-26T05:47:22.073286+0000","last_active":"2023-10-26T05:47:23.300018+0000","last_peered":"2023-10-26T05:47:23.300018+0000","last_clean":"2023-10-26T05:47:23.300018+0000","last_became_active":"2023-10-26T05:47:22.072189+0000","last_became_peered":"2023-10-26T05:47:22.072189+0000","last_unstale":"2023-10-26T05:47:23.300018+0000","last_undegraded":"2023-10-26T05:47:23.300018+0000","last_fullsized":"2023-10-26T05:47:23.300018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:43.485135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290644+0000","last_change":"2023-10-26T05:47:22.054592+0000","last_active":"2023-10-26T05:47:23.290644+0000","last_peered":"2023-10-26T05:47:23.290644+0000","last_clean":"2023-10-26T05:47:23.290644+0000","last_became_active":"2023-10-26T05:47:22.048761+0000","last_became_peered":"2023-10-26T05:47:22.048761+0000","last_unstale":"2023-10-26T05:47:23.290644+0000","last_undegraded":"2023-10-26T05:47:23.290644+0000","last_fullsized":"2023-10-26T05:47:23.290644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296847+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.296847+0000","last_peered":"2023-10-26T05:47:23.296847+0000","last_clean":"2023-10-26T05:47:23.296847+0000","last_became_active":"2023-10-26T05:47:22.066674+0000","last_became_peered":"2023-10-26T05:47:22.066674+0000","last_unstale":"2023-10-26T05:47:23.296847+0000","last_undegraded":"2023-10-26T05:47:23.296847+0000","last_fullsized":"2023-10-26T05:47:23.296847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298450+0000","last_change":"2023-10-26T05:47:22.054456+0000","last_active":"2023-10-26T05:47:23.298450+0000","last_peered":"2023-10-26T05:47:23.298450+0000","last_clean":"2023-10-26T05:47:23.298450+0000","last_became_active":"2023-10-26T05:47:22.053876+0000","last_became_peered":"2023-10-26T05:47:22.053876+0000","last_unstale":"2023-10-26T05:47:23.298450+0000","last_undegraded":"2023-10-26T05:47:23.298450+0000","last_fullsized":"2023-10-26T05:47:23.298450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298522+0000","last_change":"2023-10-26T05:47:22.061734+0000","last_active":"2023-10-26T05:47:23.298522+0000","last_peered":"2023-10-26T05:47:23.298522+0000","last_clean":"2023-10-26T05:47:23.298522+0000","last_became_active":"2023-10-26T05:47:22.061125+0000","last_became_peered":"2023-10-26T05:47:22.061125+0000","last_unstale":"2023-10-26T05:47:23.298522+0000","last_undegraded":"2023-10-26T05:47:23.298522+0000","last_fullsized":"2023-10-26T05:47:23.298522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039953+0000","last_change":"2023-10-26T05:47:22.059027+0000","last_active":"2023-10-26T05:47:23.039953+0000","last_peered":"2023-10-26T05:47:23.039953+0000","last_clean":"2023-10-26T05:47:23.039953+0000","last_became_active":"2023-10-26T05:47:22.058395+0000","last_became_peered":"2023-10-26T05:47:22.058395+0000","last_unstale":"2023-10-26T05:47:23.039953+0000","last_undegraded":"2023-10-26T05:47:23.039953+0000","last_fullsized":"2023-10-26T05:47:23.039953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:26.058039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291056+0000","last_change":"2023-10-26T05:47:22.054240+0000","last_active":"2023-10-26T05:47:23.291056+0000","last_peered":"2023-10-26T05:47:23.291056+0000","last_clean":"2023-10-26T05:47:23.291056+0000","last_became_active":"2023-10-26T05:47:22.047954+0000","last_became_peered":"2023-10-26T05:47:22.047954+0000","last_unstale":"2023-10-26T05:47:23.291056+0000","last_undegraded":"2023-10-26T05:47:23.291056+0000","last_fullsized":"2023-10-26T05:47:23.291056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595789+0000","last_change":"2023-10-26T05:47:22.046495+0000","last_active":"2023-10-26T05:47:23.595789+0000","last_peered":"2023-10-26T05:47:23.595789+0000","last_clean":"2023-10-26T05:47:23.595789+0000","last_became_active":"2023-10-26T05:47:22.045867+0000","last_became_peered":"2023-10-26T05:47:22.045867+0000","last_unstale":"2023-10-26T05:47:23.595789+0000","last_undegraded":"2023-10-26T05:47:23.595789+0000","last_fullsized":"2023-10-26T05:47:23.595789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:36.941020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033315+0000","last_change":"2023-10-26T05:47:22.058776+0000","last_active":"2023-10-26T05:47:23.033315+0000","last_peered":"2023-10-26T05:47:23.033315+0000","last_clean":"2023-10-26T05:47:23.033315+0000","last_became_active":"2023-10-26T05:47:22.057978+0000","last_became_peered":"2023-10-26T05:47:22.057978+0000","last_unstale":"2023-10-26T05:47:23.033315+0000","last_undegraded":"2023-10-26T05:47:23.033315+0000","last_fullsized":"2023-10-26T05:47:23.033315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297178+0000","last_change":"2023-10-26T05:47:22.055995+0000","last_active":"2023-10-26T05:47:23.297178+0000","last_peered":"2023-10-26T05:47:23.297178+0000","last_clean":"2023-10-26T05:47:23.297178+0000","last_became_active":"2023-10-26T05:47:22.049810+0000","last_became_peered":"2023-10-26T05:47:22.049810+0000","last_unstale":"2023-10-26T05:47:23.297178+0000","last_undegraded":"2023-10-26T05:47:23.297178+0000","last_fullsized":"2023-10-26T05:47:23.297178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299268+0000","last_change":"2023-10-26T05:47:22.071995+0000","last_active":"2023-10-26T05:47:23.299268+0000","last_peered":"2023-10-26T05:47:23.299268+0000","last_clean":"2023-10-26T05:47:23.299268+0000","last_became_active":"2023-10-26T05:47:22.070349+0000","last_became_peered":"2023-10-26T05:47:22.070349+0000","last_unstale":"2023-10-26T05:47:23.299268+0000","last_undegraded":"2023-10-26T05:47:23.299268+0000","last_fullsized":"2023-10-26T05:47:23.299268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033881+0000","last_change":"2023-10-26T05:47:22.051874+0000","last_active":"2023-10-26T05:47:23.033881+0000","last_peered":"2023-10-26T05:47:23.033881+0000","last_clean":"2023-10-26T05:47:23.033881+0000","last_became_active":"2023-10-26T05:47:22.051002+0000","last_became_peered":"2023-10-26T05:47:22.051002+0000","last_unstale":"2023-10-26T05:47:23.033881+0000","last_undegraded":"2023-10-26T05:47:23.033881+0000","last_fullsized":"2023-10-26T05:47:23.033881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297488+0000","last_change":"2023-10-26T05:47:22.080998+0000","last_active":"2023-10-26T05:47:23.297488+0000","last_peered":"2023-10-26T05:47:23.297488+0000","last_clean":"2023-10-26T05:47:23.297488+0000","last_became_active":"2023-10-26T05:47:22.078649+0000","last_became_peered":"2023-10-26T05:47:22.078649+0000","last_unstale":"2023-10-26T05:47:23.297488+0000","last_undegraded":"2023-10-26T05:47:23.297488+0000","last_fullsized":"2023-10-26T05:47:23.297488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298814+0000","last_change":"2023-10-26T05:47:22.052419+0000","last_active":"2023-10-26T05:47:23.298814+0000","last_peered":"2023-10-26T05:47:23.298814+0000","last_clean":"2023-10-26T05:47:23.298814+0000","last_became_active":"2023-10-26T05:47:22.051751+0000","last_became_peered":"2023-10-26T05:47:22.051751+0000","last_unstale":"2023-10-26T05:47:23.298814+0000","last_undegraded":"2023-10-26T05:47:23.298814+0000","last_fullsized":"2023-10-26T05:47:23.298814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596450+0000","last_change":"2023-10-26T05:47:22.073364+0000","last_active":"2023-10-26T05:47:23.596450+0000","last_peered":"2023-10-26T05:47:23.596450+0000","last_clean":"2023-10-26T05:47:23.596450+0000","last_became_active":"2023-10-26T05:47:22.072258+0000","last_became_peered":"2023-10-26T05:47:22.072258+0000","last_unstale":"2023-10-26T05:47:23.596450+0000","last_undegraded":"2023-10-26T05:47:23.596450+0000","last_fullsized":"2023-10-26T05:47:23.596450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299915+0000","last_change":"2023-10-26T05:47:22.075501+0000","last_active":"2023-10-26T05:47:23.299915+0000","last_peered":"2023-10-26T05:47:23.299915+0000","last_clean":"2023-10-26T05:47:23.299915+0000","last_became_active":"2023-10-26T05:47:22.074808+0000","last_became_peered":"2023-10-26T05:47:22.074808+0000","last_unstale":"2023-10-26T05:47:23.299915+0000","last_undegraded":"2023-10-26T05:47:23.299915+0000","last_fullsized":"2023-10-26T05:47:23.299915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291547+0000","last_change":"2023-10-26T05:47:22.061162+0000","last_active":"2023-10-26T05:47:23.291547+0000","last_peered":"2023-10-26T05:47:23.291547+0000","last_clean":"2023-10-26T05:47:23.291547+0000","last_became_active":"2023-10-26T05:47:22.053623+0000","last_became_peered":"2023-10-26T05:47:22.053623+0000","last_unstale":"2023-10-26T05:47:23.291547+0000","last_undegraded":"2023-10-26T05:47:23.291547+0000","last_fullsized":"2023-10-26T05:47:23.291547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296810+0000","last_change":"2023-10-26T05:47:22.063651+0000","last_active":"2023-10-26T05:47:23.296810+0000","last_peered":"2023-10-26T05:47:23.296810+0000","last_clean":"2023-10-26T05:47:23.296810+0000","last_became_active":"2023-10-26T05:47:22.062599+0000","last_became_peered":"2023-10-26T05:47:22.062599+0000","last_unstale":"2023-10-26T05:47:23.296810+0000","last_undegraded":"2023-10-26T05:47:23.296810+0000","last_fullsized":"2023-10-26T05:47:23.296810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290912+0000","last_change":"2023-10-26T05:47:22.053793+0000","last_active":"2023-10-26T05:47:23.290912+0000","last_peered":"2023-10-26T05:47:23.290912+0000","last_clean":"2023-10-26T05:47:23.290912+0000","last_became_active":"2023-10-26T05:47:22.052294+0000","last_became_peered":"2023-10-26T05:47:22.052294+0000","last_unstale":"2023-10-26T05:47:23.290912+0000","last_undegraded":"2023-10-26T05:47:23.290912+0000","last_fullsized":"2023-10-26T05:47:23.290912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298905+0000","last_change":"2023-10-26T05:47:22.078582+0000","last_active":"2023-10-26T05:47:23.298905+0000","last_peered":"2023-10-26T05:47:23.298905+0000","last_clean":"2023-10-26T05:47:23.298905+0000","last_became_active":"2023-10-26T05:47:22.077883+0000","last_became_peered":"2023-10-26T05:47:22.077883+0000","last_unstale":"2023-10-26T05:47:23.298905+0000","last_undegraded":"2023-10-26T05:47:23.298905+0000","last_fullsized":"2023-10-26T05:47:23.298905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033302+0000","last_change":"2023-10-26T05:47:22.057439+0000","last_active":"2023-10-26T05:47:23.033302+0000","last_peered":"2023-10-26T05:47:23.033302+0000","last_clean":"2023-10-26T05:47:23.033302+0000","last_became_active":"2023-10-26T05:47:22.056731+0000","last_became_peered":"2023-10-26T05:47:22.056731+0000","last_unstale":"2023-10-26T05:47:23.033302+0000","last_undegraded":"2023-10-26T05:47:23.033302+0000","last_fullsized":"2023-10-26T05:47:23.033302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291223+0000","last_change":"2023-10-26T05:47:22.065579+0000","last_active":"2023-10-26T05:47:23.291223+0000","last_peered":"2023-10-26T05:47:23.291223+0000","last_clean":"2023-10-26T05:47:23.291223+0000","last_became_active":"2023-10-26T05:47:22.063798+0000","last_became_peered":"2023-10-26T05:47:22.063798+0000","last_unstale":"2023-10-26T05:47:23.291223+0000","last_undegraded":"2023-10-26T05:47:23.291223+0000","last_fullsized":"2023-10-26T05:47:23.291223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596104+0000","last_change":"2023-10-26T05:47:22.056700+0000","last_active":"2023-10-26T05:47:23.596104+0000","last_peered":"2023-10-26T05:47:23.596104+0000","last_clean":"2023-10-26T05:47:23.596104+0000","last_became_active":"2023-10-26T05:47:22.056216+0000","last_became_peered":"2023-10-26T05:47:22.056216+0000","last_unstale":"2023-10-26T05:47:23.596104+0000","last_undegraded":"2023-10-26T05:47:23.596104+0000","last_fullsized":"2023-10-26T05:47:23.596104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297162+0000","last_change":"2023-10-26T05:47:22.061465+0000","last_active":"2023-10-26T05:47:23.297162+0000","last_peered":"2023-10-26T05:47:23.297162+0000","last_clean":"2023-10-26T05:47:23.297162+0000","last_became_active":"2023-10-26T05:47:22.060699+0000","last_became_peered":"2023-10-26T05:47:22.060699+0000","last_unstale":"2023-10-26T05:47:23.297162+0000","last_undegraded":"2023-10-26T05:47:23.297162+0000","last_fullsized":"2023-10-26T05:47:23.297162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033512+0000","last_change":"2023-10-26T05:47:22.058403+0000","last_active":"2023-10-26T05:47:23.033512+0000","last_peered":"2023-10-26T05:47:23.033512+0000","last_clean":"2023-10-26T05:47:23.033512+0000","last_became_active":"2023-10-26T05:47:22.057792+0000","last_became_peered":"2023-10-26T05:47:22.057792+0000","last_unstale":"2023-10-26T05:47:23.033512+0000","last_undegraded":"2023-10-26T05:47:23.033512+0000","last_fullsized":"2023-10-26T05:47:23.033512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299297+0000","last_change":"2023-10-26T05:47:22.059009+0000","last_active":"2023-10-26T05:47:23.299297+0000","last_peered":"2023-10-26T05:47:23.299297+0000","last_clean":"2023-10-26T05:47:23.299297+0000","last_became_active":"2023-10-26T05:47:22.057498+0000","last_became_peered":"2023-10-26T05:47:22.057498+0000","last_unstale":"2023-10-26T05:47:23.299297+0000","last_undegraded":"2023-10-26T05:47:23.299297+0000","last_fullsized":"2023-10-26T05:47:23.299297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040301+0000","last_change":"2023-10-26T05:47:22.060276+0000","last_active":"2023-10-26T05:47:23.040301+0000","last_peered":"2023-10-26T05:47:23.040301+0000","last_clean":"2023-10-26T05:47:23.040301+0000","last_became_active":"2023-10-26T05:47:22.059444+0000","last_became_peered":"2023-10-26T05:47:22.059444+0000","last_unstale":"2023-10-26T05:47:23.040301+0000","last_undegraded":"2023-10-26T05:47:23.040301+0000","last_fullsized":"2023-10-26T05:47:23.040301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291438+0000","last_change":"2023-10-26T05:47:21.083390+0000","last_active":"2023-10-26T05:47:23.291438+0000","last_peered":"2023-10-26T05:47:23.291438+0000","last_clean":"2023-10-26T05:47:23.291438+0000","last_became_active":"2023-10-26T05:47:21.083255+0000","last_became_peered":"2023-10-26T05:47:21.083255+0000","last_unstale":"2023-10-26T05:47:23.291438+0000","last_undegraded":"2023-10-26T05:47:23.291438+0000","last_fullsized":"2023-10-26T05:47:23.291438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595999+0000","last_change":"2023-10-26T05:47:22.042083+0000","last_active":"2023-10-26T05:47:23.595999+0000","last_peered":"2023-10-26T05:47:23.595999+0000","last_clean":"2023-10-26T05:47:23.595999+0000","last_became_active":"2023-10-26T05:47:22.041458+0000","last_became_peered":"2023-10-26T05:47:22.041458+0000","last_unstale":"2023-10-26T05:47:23.595999+0000","last_undegraded":"2023-10-26T05:47:23.595999+0000","last_fullsized":"2023-10-26T05:47:23.595999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:18.110818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298965+0000","last_change":"2023-10-26T05:47:21.090954+0000","last_active":"2023-10-26T05:47:23.298965+0000","last_peered":"2023-10-26T05:47:23.298965+0000","last_clean":"2023-10-26T05:47:23.298965+0000","last_became_active":"2023-10-26T05:47:21.086959+0000","last_became_peered":"2023-10-26T05:47:21.086959+0000","last_unstale":"2023-10-26T05:47:23.298965+0000","last_undegraded":"2023-10-26T05:47:23.298965+0000","last_fullsized":"2023-10-26T05:47:23.298965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291059+0000","last_change":"2023-10-26T05:47:21.078842+0000","last_active":"2023-10-26T05:47:23.291059+0000","last_peered":"2023-10-26T05:47:23.291059+0000","last_clean":"2023-10-26T05:47:23.291059+0000","last_became_active":"2023-10-26T05:47:21.078498+0000","last_became_peered":"2023-10-26T05:47:21.078498+0000","last_unstale":"2023-10-26T05:47:23.291059+0000","last_undegraded":"2023-10-26T05:47:23.291059+0000","last_fullsized":"2023-10-26T05:47:23.291059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291157+0000","last_change":"2023-10-26T05:47:22.054327+0000","last_active":"2023-10-26T05:47:23.291157+0000","last_peered":"2023-10-26T05:47:23.291157+0000","last_clean":"2023-10-26T05:47:23.291157+0000","last_became_active":"2023-10-26T05:47:22.053185+0000","last_became_peered":"2023-10-26T05:47:22.053185+0000","last_unstale":"2023-10-26T05:47:23.291157+0000","last_undegraded":"2023-10-26T05:47:23.291157+0000","last_fullsized":"2023-10-26T05:47:23.291157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297471+0000","last_change":"2023-10-26T05:47:21.084041+0000","last_active":"2023-10-26T05:47:23.297471+0000","last_peered":"2023-10-26T05:47:23.297471+0000","last_clean":"2023-10-26T05:47:23.297471+0000","last_became_active":"2023-10-26T05:47:21.082189+0000","last_became_peered":"2023-10-26T05:47:21.082189+0000","last_unstale":"2023-10-26T05:47:23.297471+0000","last_undegraded":"2023-10-26T05:47:23.297471+0000","last_fullsized":"2023-10-26T05:47:23.297471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032584+0000","last_change":"2023-10-26T05:47:22.063104+0000","last_active":"2023-10-26T05:47:23.032584+0000","last_peered":"2023-10-26T05:47:23.032584+0000","last_clean":"2023-10-26T05:47:23.032584+0000","last_became_active":"2023-10-26T05:47:22.062483+0000","last_became_peered":"2023-10-26T05:47:22.062483+0000","last_unstale":"2023-10-26T05:47:23.032584+0000","last_undegraded":"2023-10-26T05:47:23.032584+0000","last_fullsized":"2023-10-26T05:47:23.032584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299320+0000","last_change":"2023-10-26T05:47:21.087323+0000","last_active":"2023-10-26T05:47:23.299320+0000","last_peered":"2023-10-26T05:47:23.299320+0000","last_clean":"2023-10-26T05:47:23.299320+0000","last_became_active":"2023-10-26T05:47:21.086686+0000","last_became_peered":"2023-10-26T05:47:21.086686+0000","last_unstale":"2023-10-26T05:47:23.299320+0000","last_undegraded":"2023-10-26T05:47:23.299320+0000","last_fullsized":"2023-10-26T05:47:23.299320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299593+0000","last_change":"2023-10-26T05:47:22.072429+0000","last_active":"2023-10-26T05:47:23.299593+0000","last_peered":"2023-10-26T05:47:23.299593+0000","last_clean":"2023-10-26T05:47:23.299593+0000","last_became_active":"2023-10-26T05:47:22.071355+0000","last_became_peered":"2023-10-26T05:47:22.071355+0000","last_unstale":"2023-10-26T05:47:23.299593+0000","last_undegraded":"2023-10-26T05:47:23.299593+0000","last_fullsized":"2023-10-26T05:47:23.299593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300874+0000","last_change":"2023-10-26T05:47:21.096076+0000","last_active":"2023-10-26T05:47:23.300874+0000","last_peered":"2023-10-26T05:47:23.300874+0000","last_clean":"2023-10-26T05:47:23.300874+0000","last_became_active":"2023-10-26T05:47:21.092744+0000","last_became_peered":"2023-10-26T05:47:21.092744+0000","last_unstale":"2023-10-26T05:47:23.300874+0000","last_undegraded":"2023-10-26T05:47:23.300874+0000","last_fullsized":"2023-10-26T05:47:23.300874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:26.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596655+0000","last_change":"2023-10-26T05:47:22.055264+0000","last_active":"2023-10-26T05:47:23.596655+0000","last_peered":"2023-10-26T05:47:23.596655+0000","last_clean":"2023-10-26T05:47:23.596655+0000","last_became_active":"2023-10-26T05:47:22.054516+0000","last_became_peered":"2023-10-26T05:47:22.054516+0000","last_unstale":"2023-10-26T05:47:23.596655+0000","last_undegraded":"2023-10-26T05:47:23.596655+0000","last_fullsized":"2023-10-26T05:47:23.596655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033126+0000","last_change":"2023-10-26T05:47:21.069747+0000","last_active":"2023-10-26T05:47:23.033126+0000","last_peered":"2023-10-26T05:47:23.033126+0000","last_clean":"2023-10-26T05:47:23.033126+0000","last_became_active":"2023-10-26T05:47:21.069525+0000","last_became_peered":"2023-10-26T05:47:21.069525+0000","last_unstale":"2023-10-26T05:47:23.033126+0000","last_undegraded":"2023-10-26T05:47:23.033126+0000","last_fullsized":"2023-10-26T05:47:23.033126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039554+0000","last_change":"2023-10-26T05:47:22.070886+0000","last_active":"2023-10-26T05:47:23.039554+0000","last_peered":"2023-10-26T05:47:23.039554+0000","last_clean":"2023-10-26T05:47:23.039554+0000","last_became_active":"2023-10-26T05:47:22.068908+0000","last_became_peered":"2023-10-26T05:47:22.068908+0000","last_unstale":"2023-10-26T05:47:23.039554+0000","last_undegraded":"2023-10-26T05:47:23.039554+0000","last_fullsized":"2023-10-26T05:47:23.039554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:31.276801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297743+0000","last_change":"2023-10-26T05:47:21.085627+0000","last_active":"2023-10-26T05:47:23.297743+0000","last_peered":"2023-10-26T05:47:23.297743+0000","last_clean":"2023-10-26T05:47:23.297743+0000","last_became_active":"2023-10-26T05:47:21.084306+0000","last_became_peered":"2023-10-26T05:47:21.084306+0000","last_unstale":"2023-10-26T05:47:23.297743+0000","last_undegraded":"2023-10-26T05:47:23.297743+0000","last_fullsized":"2023-10-26T05:47:23.297743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299433+0000","last_change":"2023-10-26T05:47:22.071700+0000","last_active":"2023-10-26T05:47:23.299433+0000","last_peered":"2023-10-26T05:47:23.299433+0000","last_clean":"2023-10-26T05:47:23.299433+0000","last_became_active":"2023-10-26T05:47:22.071056+0000","last_became_peered":"2023-10-26T05:47:22.071056+0000","last_unstale":"2023-10-26T05:47:23.299433+0000","last_undegraded":"2023-10-26T05:47:23.299433+0000","last_fullsized":"2023-10-26T05:47:23.299433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596128+0000","last_change":"2023-10-26T05:47:21.069670+0000","last_active":"2023-10-26T05:47:23.596128+0000","last_peered":"2023-10-26T05:47:23.596128+0000","last_clean":"2023-10-26T05:47:23.596128+0000","last_became_active":"2023-10-26T05:47:21.069297+0000","last_became_peered":"2023-10-26T05:47:21.069297+0000","last_unstale":"2023-10-26T05:47:23.596128+0000","last_undegraded":"2023-10-26T05:47:23.596128+0000","last_fullsized":"2023-10-26T05:47:23.596128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:15.639318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596815+0000","last_change":"2023-10-26T05:47:22.073645+0000","last_active":"2023-10-26T05:47:23.596815+0000","last_peered":"2023-10-26T05:47:23.596815+0000","last_clean":"2023-10-26T05:47:23.596815+0000","last_became_active":"2023-10-26T05:47:22.072989+0000","last_became_peered":"2023-10-26T05:47:22.072989+0000","last_unstale":"2023-10-26T05:47:23.596815+0000","last_undegraded":"2023-10-26T05:47:23.596815+0000","last_fullsized":"2023-10-26T05:47:23.596815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298358+0000","last_change":"2023-10-26T05:47:21.086438+0000","last_active":"2023-10-26T05:47:23.298358+0000","last_peered":"2023-10-26T05:47:23.298358+0000","last_clean":"2023-10-26T05:47:23.298358+0000","last_became_active":"2023-10-26T05:47:21.082053+0000","last_became_peered":"2023-10-26T05:47:21.082053+0000","last_unstale":"2023-10-26T05:47:23.298358+0000","last_undegraded":"2023-10-26T05:47:23.298358+0000","last_fullsized":"2023-10-26T05:47:23.298358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299687+0000","last_change":"2023-10-26T05:47:22.076444+0000","last_active":"2023-10-26T05:47:23.299687+0000","last_peered":"2023-10-26T05:47:23.299687+0000","last_clean":"2023-10-26T05:47:23.299687+0000","last_became_active":"2023-10-26T05:47:22.074572+0000","last_became_peered":"2023-10-26T05:47:22.074572+0000","last_unstale":"2023-10-26T05:47:23.299687+0000","last_undegraded":"2023-10-26T05:47:23.299687+0000","last_fullsized":"2023-10-26T05:47:23.299687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299516+0000","last_change":"2023-10-26T05:47:21.090350+0000","last_active":"2023-10-26T05:47:23.299516+0000","last_peered":"2023-10-26T05:47:23.299516+0000","last_clean":"2023-10-26T05:47:23.299516+0000","last_became_active":"2023-10-26T05:47:21.087109+0000","last_became_peered":"2023-10-26T05:47:21.087109+0000","last_unstale":"2023-10-26T05:47:23.299516+0000","last_undegraded":"2023-10-26T05:47:23.299516+0000","last_fullsized":"2023-10-26T05:47:23.299516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298687+0000","last_change":"2023-10-26T05:47:22.066147+0000","last_active":"2023-10-26T05:47:23.298687+0000","last_peered":"2023-10-26T05:47:23.298687+0000","last_clean":"2023-10-26T05:47:23.298687+0000","last_became_active":"2023-10-26T05:47:22.065497+0000","last_became_peered":"2023-10-26T05:47:22.065497+0000","last_unstale":"2023-10-26T05:47:23.298687+0000","last_undegraded":"2023-10-26T05:47:23.298687+0000","last_fullsized":"2023-10-26T05:47:23.298687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034102+0000","last_change":"2023-10-26T05:47:21.064585+0000","last_active":"2023-10-26T05:47:23.034102+0000","last_peered":"2023-10-26T05:47:23.034102+0000","last_clean":"2023-10-26T05:47:23.034102+0000","last_became_active":"2023-10-26T05:47:21.064395+0000","last_became_peered":"2023-10-26T05:47:21.064395+0000","last_unstale":"2023-10-26T05:47:23.034102+0000","last_undegraded":"2023-10-26T05:47:23.034102+0000","last_fullsized":"2023-10-26T05:47:23.034102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300524+0000","last_change":"2023-10-26T05:47:22.077148+0000","last_active":"2023-10-26T05:47:23.300524+0000","last_peered":"2023-10-26T05:47:23.300524+0000","last_clean":"2023-10-26T05:47:23.300524+0000","last_became_active":"2023-10-26T05:47:22.075279+0000","last_became_peered":"2023-10-26T05:47:22.075279+0000","last_unstale":"2023-10-26T05:47:23.300524+0000","last_undegraded":"2023-10-26T05:47:23.300524+0000","last_fullsized":"2023-10-26T05:47:23.300524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:12:08.628590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039791+0000","last_change":"2023-10-26T05:47:21.077376+0000","last_active":"2023-10-26T05:47:23.039791+0000","last_peered":"2023-10-26T05:47:23.039791+0000","last_clean":"2023-10-26T05:47:23.039791+0000","last_became_active":"2023-10-26T05:47:21.076766+0000","last_became_peered":"2023-10-26T05:47:21.076766+0000","last_unstale":"2023-10-26T05:47:23.039791+0000","last_undegraded":"2023-10-26T05:47:23.039791+0000","last_fullsized":"2023-10-26T05:47:23.039791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299945+0000","last_change":"2023-10-26T05:47:22.077007+0000","last_active":"2023-10-26T05:47:23.299945+0000","last_peered":"2023-10-26T05:47:23.299945+0000","last_clean":"2023-10-26T05:47:23.299945+0000","last_became_active":"2023-10-26T05:47:22.075297+0000","last_became_peered":"2023-10-26T05:47:22.075297+0000","last_unstale":"2023-10-26T05:47:23.299945+0000","last_undegraded":"2023-10-26T05:47:23.299945+0000","last_fullsized":"2023-10-26T05:47:23.299945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:15.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033816+0000","last_change":"2023-10-26T05:47:21.072137+0000","last_active":"2023-10-26T05:47:23.033816+0000","last_peered":"2023-10-26T05:47:23.033816+0000","last_clean":"2023-10-26T05:47:23.033816+0000","last_became_active":"2023-10-26T05:47:21.071720+0000","last_became_peered":"2023-10-26T05:47:21.071720+0000","last_unstale":"2023-10-26T05:47:23.033816+0000","last_undegraded":"2023-10-26T05:47:23.033816+0000","last_fullsized":"2023-10-26T05:47:23.033816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:09.715298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297159+0000","last_change":"2023-10-26T05:47:22.080581+0000","last_active":"2023-10-26T05:47:23.297159+0000","last_peered":"2023-10-26T05:47:23.297159+0000","last_clean":"2023-10-26T05:47:23.297159+0000","last_became_active":"2023-10-26T05:47:22.076913+0000","last_became_peered":"2023-10-26T05:47:22.076913+0000","last_unstale":"2023-10-26T05:47:23.297159+0000","last_undegraded":"2023-10-26T05:47:23.297159+0000","last_fullsized":"2023-10-26T05:47:23.297159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033027+0000","last_change":"2023-10-26T05:47:21.076982+0000","last_active":"2023-10-26T05:47:23.033027+0000","last_peered":"2023-10-26T05:47:23.033027+0000","last_clean":"2023-10-26T05:47:23.033027+0000","last_became_active":"2023-10-26T05:47:21.076695+0000","last_became_peered":"2023-10-26T05:47:21.076695+0000","last_unstale":"2023-10-26T05:47:23.033027+0000","last_undegraded":"2023-10-26T05:47:23.033027+0000","last_fullsized":"2023-10-26T05:47:23.033027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:57.904856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300280+0000","last_change":"2023-10-26T05:47:22.063246+0000","last_active":"2023-10-26T05:47:23.300280+0000","last_peered":"2023-10-26T05:47:23.300280+0000","last_clean":"2023-10-26T05:47:23.300280+0000","last_became_active":"2023-10-26T05:47:22.062499+0000","last_became_peered":"2023-10-26T05:47:22.062499+0000","last_unstale":"2023-10-26T05:47:23.300280+0000","last_undegraded":"2023-10-26T05:47:23.300280+0000","last_fullsized":"2023-10-26T05:47:23.300280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033647+0000","last_change":"2023-10-26T05:47:21.087999+0000","last_active":"2023-10-26T05:47:23.033647+0000","last_peered":"2023-10-26T05:47:23.033647+0000","last_clean":"2023-10-26T05:47:23.033647+0000","last_became_active":"2023-10-26T05:47:21.087744+0000","last_became_peered":"2023-10-26T05:47:21.087744+0000","last_unstale":"2023-10-26T05:47:23.033647+0000","last_undegraded":"2023-10-26T05:47:23.033647+0000","last_fullsized":"2023-10-26T05:47:23.033647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298194+0000","last_change":"2023-10-26T05:47:22.081324+0000","last_active":"2023-10-26T05:47:23.298194+0000","last_peered":"2023-10-26T05:47:23.298194+0000","last_clean":"2023-10-26T05:47:23.298194+0000","last_became_active":"2023-10-26T05:47:22.079073+0000","last_became_peered":"2023-10-26T05:47:22.079073+0000","last_unstale":"2023-10-26T05:47:23.298194+0000","last_undegraded":"2023-10-26T05:47:23.298194+0000","last_fullsized":"2023-10-26T05:47:23.298194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T05:47:22.066065+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:22.066065+0000","last_peered":"2023-10-26T05:47:22.066065+0000","last_clean":"2023-10-26T05:47:22.066065+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:22.066065+0000","last_undegraded":"2023-10-26T05:47:22.066065+0000","last_fullsized":"2023-10-26T05:47:22.066065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298749+0000","last_change":"2023-10-26T05:47:21.087000+0000","last_active":"2023-10-26T05:47:23.298749+0000","last_peered":"2023-10-26T05:47:23.298749+0000","last_clean":"2023-10-26T05:47:23.298749+0000","last_became_active":"2023-10-26T05:47:21.082935+0000","last_became_peered":"2023-10-26T05:47:21.082935+0000","last_unstale":"2023-10-26T05:47:23.298749+0000","last_undegraded":"2023-10-26T05:47:23.298749+0000","last_fullsized":"2023-10-26T05:47:23.298749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291206+0000","last_change":"2023-10-26T05:47:22.055580+0000","last_active":"2023-10-26T05:47:23.291206+0000","last_peered":"2023-10-26T05:47:23.291206+0000","last_clean":"2023-10-26T05:47:23.291206+0000","last_became_active":"2023-10-26T05:47:22.054587+0000","last_became_peered":"2023-10-26T05:47:22.054587+0000","last_unstale":"2023-10-26T05:47:23.291206+0000","last_undegraded":"2023-10-26T05:47:23.291206+0000","last_fullsized":"2023-10-26T05:47:23.291206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297907+0000","last_change":"2023-10-26T05:47:21.100233+0000","last_active":"2023-10-26T05:47:23.297907+0000","last_peered":"2023-10-26T05:47:23.297907+0000","last_clean":"2023-10-26T05:47:23.297907+0000","last_became_active":"2023-10-26T05:47:21.099318+0000","last_became_peered":"2023-10-26T05:47:21.099318+0000","last_unstale":"2023-10-26T05:47:23.297907+0000","last_undegraded":"2023-10-26T05:47:23.297907+0000","last_fullsized":"2023-10-26T05:47:23.297907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596567+0000","last_change":"2023-10-26T05:47:22.057190+0000","last_active":"2023-10-26T05:47:23.596567+0000","last_peered":"2023-10-26T05:47:23.596567+0000","last_clean":"2023-10-26T05:47:23.596567+0000","last_became_active":"2023-10-26T05:47:22.049709+0000","last_became_peered":"2023-10-26T05:47:22.049709+0000","last_unstale":"2023-10-26T05:47:23.596567+0000","last_undegraded":"2023-10-26T05:47:23.596567+0000","last_fullsized":"2023-10-26T05:47:23.596567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034304+0000","last_change":"2023-10-26T05:47:21.073026+0000","last_active":"2023-10-26T05:47:23.034304+0000","last_peered":"2023-10-26T05:47:23.034304+0000","last_clean":"2023-10-26T05:47:23.034304+0000","last_became_active":"2023-10-26T05:47:21.071577+0000","last_became_peered":"2023-10-26T05:47:21.071577+0000","last_unstale":"2023-10-26T05:47:23.034304+0000","last_undegraded":"2023-10-26T05:47:23.034304+0000","last_fullsized":"2023-10-26T05:47:23.034304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:25.047488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032708+0000","last_change":"2023-10-26T05:47:22.052469+0000","last_active":"2023-10-26T05:47:23.032708+0000","last_peered":"2023-10-26T05:47:23.032708+0000","last_clean":"2023-10-26T05:47:23.032708+0000","last_became_active":"2023-10-26T05:47:22.051614+0000","last_became_peered":"2023-10-26T05:47:22.051614+0000","last_unstale":"2023-10-26T05:47:23.032708+0000","last_undegraded":"2023-10-26T05:47:23.032708+0000","last_fullsized":"2023-10-26T05:47:23.032708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.301108+0000","last_change":"2023-10-26T05:47:21.096632+0000","last_active":"2023-10-26T05:47:23.301108+0000","last_peered":"2023-10-26T05:47:23.301108+0000","last_clean":"2023-10-26T05:47:23.301108+0000","last_became_active":"2023-10-26T05:47:21.093612+0000","last_became_peered":"2023-10-26T05:47:21.093612+0000","last_unstale":"2023-10-26T05:47:23.301108+0000","last_undegraded":"2023-10-26T05:47:23.301108+0000","last_fullsized":"2023-10-26T05:47:23.301108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039976+0000","last_change":"2023-10-26T05:47:22.060460+0000","last_active":"2023-10-26T05:47:23.039976+0000","last_peered":"2023-10-26T05:47:23.039976+0000","last_clean":"2023-10-26T05:47:23.039976+0000","last_became_active":"2023-10-26T05:47:22.059831+0000","last_became_peered":"2023-10-26T05:47:22.059831+0000","last_unstale":"2023-10-26T05:47:23.039976+0000","last_undegraded":"2023-10-26T05:47:23.039976+0000","last_fullsized":"2023-10-26T05:47:23.039976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033723+0000","last_change":"2023-10-26T05:47:21.071567+0000","last_active":"2023-10-26T05:47:23.033723+0000","last_peered":"2023-10-26T05:47:23.033723+0000","last_clean":"2023-10-26T05:47:23.033723+0000","last_became_active":"2023-10-26T05:47:21.071275+0000","last_became_peered":"2023-10-26T05:47:21.071275+0000","last_unstale":"2023-10-26T05:47:23.033723+0000","last_undegraded":"2023-10-26T05:47:23.033723+0000","last_fullsized":"2023-10-26T05:47:23.033723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297152+0000","last_change":"2023-10-26T05:47:22.063236+0000","last_active":"2023-10-26T05:47:23.297152+0000","last_peered":"2023-10-26T05:47:23.297152+0000","last_clean":"2023-10-26T05:47:23.297152+0000","last_became_active":"2023-10-26T05:47:22.061746+0000","last_became_peered":"2023-10-26T05:47:22.061746+0000","last_unstale":"2023-10-26T05:47:23.297152+0000","last_undegraded":"2023-10-26T05:47:23.297152+0000","last_fullsized":"2023-10-26T05:47:23.297152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:21.099849+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:21.097871+0000","last_became_peered":"2023-10-26T05:47:21.097871+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032981+0000","last_change":"2023-10-26T05:47:22.050271+0000","last_active":"2023-10-26T05:47:23.032981+0000","last_peered":"2023-10-26T05:47:23.032981+0000","last_clean":"2023-10-26T05:47:23.032981+0000","last_became_active":"2023-10-26T05:47:22.049654+0000","last_became_peered":"2023-10-26T05:47:22.049654+0000","last_unstale":"2023-10-26T05:47:23.032981+0000","last_undegraded":"2023-10-26T05:47:23.032981+0000","last_fullsized":"2023-10-26T05:47:23.032981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033347+0000","last_change":"2023-10-26T05:47:21.078711+0000","last_active":"2023-10-26T05:47:23.033347+0000","last_peered":"2023-10-26T05:47:23.033347+0000","last_clean":"2023-10-26T05:47:23.033347+0000","last_became_active":"2023-10-26T05:47:21.078403+0000","last_became_peered":"2023-10-26T05:47:21.078403+0000","last_unstale":"2023-10-26T05:47:23.033347+0000","last_undegraded":"2023-10-26T05:47:23.033347+0000","last_fullsized":"2023-10-26T05:47:23.033347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290995+0000","last_change":"2023-10-26T05:47:22.065870+0000","last_active":"2023-10-26T05:47:23.290995+0000","last_peered":"2023-10-26T05:47:23.290995+0000","last_clean":"2023-10-26T05:47:23.290995+0000","last_became_active":"2023-10-26T05:47:22.065412+0000","last_became_peered":"2023-10-26T05:47:22.065412+0000","last_unstale":"2023-10-26T05:47:23.290995+0000","last_undegraded":"2023-10-26T05:47:23.290995+0000","last_fullsized":"2023-10-26T05:47:23.290995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299744+0000","last_change":"2023-10-26T05:47:21.094764+0000","last_active":"2023-10-26T05:47:23.299744+0000","last_peered":"2023-10-26T05:47:23.299744+0000","last_clean":"2023-10-26T05:47:23.299744+0000","last_became_active":"2023-10-26T05:47:21.091991+0000","last_became_peered":"2023-10-26T05:47:21.091991+0000","last_unstale":"2023-10-26T05:47:23.299744+0000","last_undegraded":"2023-10-26T05:47:23.299744+0000","last_fullsized":"2023-10-26T05:47:23.299744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596211+0000","last_change":"2023-10-26T05:47:22.059433+0000","last_active":"2023-10-26T05:47:23.596211+0000","last_peered":"2023-10-26T05:47:23.596211+0000","last_clean":"2023-10-26T05:47:23.596211+0000","last_became_active":"2023-10-26T05:47:22.054466+0000","last_became_peered":"2023-10-26T05:47:22.054466+0000","last_unstale":"2023-10-26T05:47:23.596211+0000","last_undegraded":"2023-10-26T05:47:23.596211+0000","last_fullsized":"2023-10-26T05:47:23.596211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298877+0000","last_change":"2023-10-26T05:47:21.099000+0000","last_active":"2023-10-26T05:47:23.298877+0000","last_peered":"2023-10-26T05:47:23.298877+0000","last_clean":"2023-10-26T05:47:23.298877+0000","last_became_active":"2023-10-26T05:47:21.095552+0000","last_became_peered":"2023-10-26T05:47:21.095552+0000","last_unstale":"2023-10-26T05:47:23.298877+0000","last_undegraded":"2023-10-26T05:47:23.298877+0000","last_fullsized":"2023-10-26T05:47:23.298877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:49.348590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040667+0000","last_change":"2023-10-26T05:47:22.059642+0000","last_active":"2023-10-26T05:47:23.040667+0000","last_peered":"2023-10-26T05:47:23.040667+0000","last_clean":"2023-10-26T05:47:23.040667+0000","last_became_active":"2023-10-26T05:47:22.054716+0000","last_became_peered":"2023-10-26T05:47:22.054716+0000","last_unstale":"2023-10-26T05:47:23.040667+0000","last_undegraded":"2023-10-26T05:47:23.040667+0000","last_fullsized":"2023-10-26T05:47:23.040667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:10.310768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033526+0000","last_change":"2023-10-26T05:47:21.090099+0000","last_active":"2023-10-26T05:47:23.033526+0000","last_peered":"2023-10-26T05:47:23.033526+0000","last_clean":"2023-10-26T05:47:23.033526+0000","last_became_active":"2023-10-26T05:47:21.089837+0000","last_became_peered":"2023-10-26T05:47:21.089837+0000","last_unstale":"2023-10-26T05:47:23.033526+0000","last_undegraded":"2023-10-26T05:47:23.033526+0000","last_fullsized":"2023-10-26T05:47:23.033526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290991+0000","last_change":"2023-10-26T05:47:22.046792+0000","last_active":"2023-10-26T05:47:23.290991+0000","last_peered":"2023-10-26T05:47:23.290991+0000","last_clean":"2023-10-26T05:47:23.290991+0000","last_became_active":"2023-10-26T05:47:22.045151+0000","last_became_peered":"2023-10-26T05:47:22.045151+0000","last_unstale":"2023-10-26T05:47:23.290991+0000","last_undegraded":"2023-10-26T05:47:23.290991+0000","last_fullsized":"2023-10-26T05:47:23.290991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032792+0000","last_change":"2023-10-26T05:47:21.072214+0000","last_active":"2023-10-26T05:47:23.032792+0000","last_peered":"2023-10-26T05:47:23.032792+0000","last_clean":"2023-10-26T05:47:23.032792+0000","last_became_active":"2023-10-26T05:47:21.071876+0000","last_became_peered":"2023-10-26T05:47:21.071876+0000","last_unstale":"2023-10-26T05:47:23.032792+0000","last_undegraded":"2023-10-26T05:47:23.032792+0000","last_fullsized":"2023-10-26T05:47:23.032792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291391+0000","last_change":"2023-10-26T05:47:22.066034+0000","last_active":"2023-10-26T05:47:23.291391+0000","last_peered":"2023-10-26T05:47:23.291391+0000","last_clean":"2023-10-26T05:47:23.291391+0000","last_became_active":"2023-10-26T05:47:22.064885+0000","last_became_peered":"2023-10-26T05:47:22.064885+0000","last_unstale":"2023-10-26T05:47:23.291391+0000","last_undegraded":"2023-10-26T05:47:23.291391+0000","last_fullsized":"2023-10-26T05:47:23.291391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299704+0000","last_change":"2023-10-26T05:47:21.093006+0000","last_active":"2023-10-26T05:47:23.299704+0000","last_peered":"2023-10-26T05:47:23.299704+0000","last_clean":"2023-10-26T05:47:23.299704+0000","last_became_active":"2023-10-26T05:47:21.092342+0000","last_became_peered":"2023-10-26T05:47:21.092342+0000","last_unstale":"2023-10-26T05:47:23.299704+0000","last_undegraded":"2023-10-26T05:47:23.299704+0000","last_fullsized":"2023-10-26T05:47:23.299704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:12.988430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296978+0000","last_change":"2023-10-26T05:47:22.059661+0000","last_active":"2023-10-26T05:47:23.296978+0000","last_peered":"2023-10-26T05:47:23.296978+0000","last_clean":"2023-10-26T05:47:23.296978+0000","last_became_active":"2023-10-26T05:47:22.058379+0000","last_became_peered":"2023-10-26T05:47:22.058379+0000","last_unstale":"2023-10-26T05:47:23.296978+0000","last_undegraded":"2023-10-26T05:47:23.296978+0000","last_fullsized":"2023-10-26T05:47:23.296978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297352+0000","last_change":"2023-10-26T05:47:22.045989+0000","last_active":"2023-10-26T05:47:23.297352+0000","last_peered":"2023-10-26T05:47:23.297352+0000","last_clean":"2023-10-26T05:47:23.297352+0000","last_became_active":"2023-10-26T05:47:22.044537+0000","last_became_peered":"2023-10-26T05:47:22.044537+0000","last_unstale":"2023-10-26T05:47:23.297352+0000","last_undegraded":"2023-10-26T05:47:23.297352+0000","last_fullsized":"2023-10-26T05:47:23.297352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298638+0000","last_change":"2023-10-26T05:47:21.099538+0000","last_active":"2023-10-26T05:47:23.298638+0000","last_peered":"2023-10-26T05:47:23.298638+0000","last_clean":"2023-10-26T05:47:23.298638+0000","last_became_active":"2023-10-26T05:47:21.096351+0000","last_became_peered":"2023-10-26T05:47:21.096351+0000","last_unstale":"2023-10-26T05:47:23.298638+0000","last_undegraded":"2023-10-26T05:47:23.298638+0000","last_fullsized":"2023-10-26T05:47:23.298638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:21.625489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297831+0000","last_change":"2023-10-26T05:47:22.059627+0000","last_active":"2023-10-26T05:47:23.297831+0000","last_peered":"2023-10-26T05:47:23.297831+0000","last_clean":"2023-10-26T05:47:23.297831+0000","last_became_active":"2023-10-26T05:47:22.054248+0000","last_became_peered":"2023-10-26T05:47:22.054248+0000","last_unstale":"2023-10-26T05:47:23.297831+0000","last_undegraded":"2023-10-26T05:47:23.297831+0000","last_fullsized":"2023-10-26T05:47:23.297831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300077+0000","last_change":"2023-10-26T05:47:21.097760+0000","last_active":"2023-10-26T05:47:23.300077+0000","last_peered":"2023-10-26T05:47:23.300077+0000","last_clean":"2023-10-26T05:47:23.300077+0000","last_became_active":"2023-10-26T05:47:21.093680+0000","last_became_peered":"2023-10-26T05:47:21.093680+0000","last_unstale":"2023-10-26T05:47:23.300077+0000","last_undegraded":"2023-10-26T05:47:23.300077+0000","last_fullsized":"2023-10-26T05:47:23.300077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596384+0000","last_change":"2023-10-26T05:47:22.038934+0000","last_active":"2023-10-26T05:47:23.596384+0000","last_peered":"2023-10-26T05:47:23.596384+0000","last_clean":"2023-10-26T05:47:23.596384+0000","last_became_active":"2023-10-26T05:47:22.038330+0000","last_became_peered":"2023-10-26T05:47:22.038330+0000","last_unstale":"2023-10-26T05:47:23.596384+0000","last_undegraded":"2023-10-26T05:47:23.596384+0000","last_fullsized":"2023-10-26T05:47:23.596384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290969+0000","last_change":"2023-10-26T05:47:21.070756+0000","last_active":"2023-10-26T05:47:23.290969+0000","last_peered":"2023-10-26T05:47:23.290969+0000","last_clean":"2023-10-26T05:47:23.290969+0000","last_became_active":"2023-10-26T05:47:21.070211+0000","last_became_peered":"2023-10-26T05:47:21.070211+0000","last_unstale":"2023-10-26T05:47:23.290969+0000","last_undegraded":"2023-10-26T05:47:23.290969+0000","last_fullsized":"2023-10-26T05:47:23.290969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290817+0000","last_change":"2023-10-26T05:47:22.062259+0000","last_active":"2023-10-26T05:47:23.290817+0000","last_peered":"2023-10-26T05:47:23.290817+0000","last_clean":"2023-10-26T05:47:23.290817+0000","last_became_active":"2023-10-26T05:47:22.055535+0000","last_became_peered":"2023-10-26T05:47:22.055535+0000","last_unstale":"2023-10-26T05:47:23.290817+0000","last_undegraded":"2023-10-26T05:47:23.290817+0000","last_fullsized":"2023-10-26T05:47:23.290817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596691+0000","last_change":"2023-10-26T05:47:21.084394+0000","last_active":"2023-10-26T05:47:23.596691+0000","last_peered":"2023-10-26T05:47:23.596691+0000","last_clean":"2023-10-26T05:47:23.596691+0000","last_became_active":"2023-10-26T05:47:21.083775+0000","last_became_peered":"2023-10-26T05:47:21.083775+0000","last_unstale":"2023-10-26T05:47:23.596691+0000","last_undegraded":"2023-10-26T05:47:23.596691+0000","last_fullsized":"2023-10-26T05:47:23.596691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:25.792 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T05:47:25.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1271009804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880cd880 msgr2=0x7f308815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1271009804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880cd880 0x7f308815bab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f308808a140 tx=0x7f308001caa0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1271009804 shutdown_connections 2023-10-26T05:47:25.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1271009804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880cd880 0x7f308815bab0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1271009804 >> v2:172.21.15.86:3300/0 conn(0x7f30880d0470 0x7f308806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1271009804 >> 172.21.15.71:0/1271009804 conn(0x7f30880cdd80 msgr2=0x7f30880ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:25.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1271009804 shutdown_connections 2023-10-26T05:47:25.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1271009804 wait complete. 2023-10-26T05:47:25.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 Processor -- start 2023-10-26T05:47:25.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- start start 2023-10-26T05:47:25.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30880cd880 0x7f30880c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30880d0470 0x7f3088155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 0x7f30881575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3088156540 con 0x7f30880d0850 2023-10-26T05:47:25.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3088156820 con 0x7f30880d0470 2023-10-26T05:47:25.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3088156b00 con 0x7f30880cd880 2023-10-26T05:47:25.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 0x7f30881575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 0x7f30881575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53142/0 (socket says 172.21.15.71:53142) 2023-10-26T05:47:25.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 -- 172.21.15.71:0/1445665154 learned_addr learned my addr 172.21.15.71:0/1445665154 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:25.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 -- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30880cd880 msgr2=0x7f30880c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30880cd880 0x7f30880c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 -- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30880d0470 msgr2=0x7f3088155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:25.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30880d0470 0x7f3088155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:25.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 -- 172.21.15.71:0/1445665154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080009820 con 0x7f30880d0850 2023-10-26T05:47:25.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f30877fe640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 0x7f30881575e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3080098690 tx=0x7f30800a59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:25.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080003b80 con 0x7f30880d0850 2023-10-26T05:47:25.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3080003ce0 con 0x7f30880d0850 2023-10-26T05:47:25.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3088156de0 con 0x7f30880d0850 2023-10-26T05:47:25.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080004270 con 0x7f30880d0850 2023-10-26T05:47:25.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.940+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30880c5fe0 con 0x7f30880d0850 2023-10-26T05:47:25.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.944+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3088066b10 con 0x7f30880d0850 2023-10-26T05:47:25.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.944+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f30800043d0 con 0x7f30880d0850 2023-10-26T05:47:25.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.944+0000 7f307f7fe640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3068066470 0x7f3068068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:25.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.944+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f308011fa90 con 0x7f30880d0850 2023-10-26T05:47:25.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.944+0000 7f3086ffd640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3068066470 0x7f3068068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:25.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.944+0000 7f3086ffd640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3068066470 0x7f3068068930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30780058d0 tx=0x7f3078005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:25.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:25.948+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f30800ed110 con 0x7f30880d0850 2023-10-26T05:47:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3088157300 con 0x7f30880d0850 2023-10-26T05:47:26.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f307f7fe640 1 -- 172.21.15.71:0/1445665154 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1004 (secure 0 0 0) 0x7f30800b2400 con 0x7f30880d0850 2023-10-26T05:47:26.150 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:26.150 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1698299229,"num_in_osds":8,"osd_in_since":1698299226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":1751460,"bytes_avail":1598248540,"bytes_total":1600000000,"write_bytes_sec":2676,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4369}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-10-26T05:47:02.371725+0000","services":{}},"progress_events":{}} 2023-10-26T05:47:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3068066470 msgr2=0x7f3068068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3068066470 0x7f3068068930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30780058d0 tx=0x7f3078005860 comp rx=0 tx=0).stop 2023-10-26T05:47:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 msgr2=0x7f30881575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 0x7f30881575e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3080098690 tx=0x7f30800a59f0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 shutdown_connections 2023-10-26T05:47:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3068066470 0x7f3068068930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30880d0850 0x7f30881575e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30880d0470 0x7f3088155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 --2- 172.21.15.71:0/1445665154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30880cd880 0x7f30880c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 >> 172.21.15.71:0/1445665154 conn(0x7f30880cdd80 msgr2=0x7f30880b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 shutdown_connections 2023-10-26T05:47:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.144+0000 7f308d3c9640 1 -- 172.21.15.71:0/1445665154 wait complete. 2023-10-26T05:47:26.166 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T05:47:26.167 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:26.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- 172.21.15.71:0/725243515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 msgr2=0x7f292015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 --2- 172.21.15.71:0/725243515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 0x7f292015bab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2918009bc0 tx=0x7f291801cac0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- 172.21.15.71:0/725243515 shutdown_connections 2023-10-26T05:47:26.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 --2- 172.21.15.71:0/725243515 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 0x7f292015bab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 --2- 172.21.15.71:0/725243515 >> v2:172.21.15.86:3300/0 conn(0x7f29200d0470 0x7f292006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- 172.21.15.71:0/725243515 >> 172.21.15.71:0/725243515 conn(0x7f29200cdd80 msgr2=0x7f29200ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:26.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- 172.21.15.71:0/725243515 shutdown_connections 2023-10-26T05:47:26.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- 172.21.15.71:0/725243515 wait complete. 2023-10-26T05:47:26.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 Processor -- start 2023-10-26T05:47:26.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- start start 2023-10-26T05:47:26.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 0x7f292015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:26.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29200d0470 0x7f292014aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:26.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 0x7f29200c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:26.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2920159e30 con 0x7f29200d0850 2023-10-26T05:47:26.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292014af60 con 0x7f29200d0470 2023-10-26T05:47:26.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f2925360640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c53a0 con 0x7f29200cd880 2023-10-26T05:47:26.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.272+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 0x7f29200c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:26.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 0x7f29200c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53144/0 (socket says 172.21.15.71:53144) 2023-10-26T05:47:26.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 -- 172.21.15.71:0/2717383278 learned_addr learned my addr 172.21.15.71:0/2717383278 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:26.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291effd640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 0x7f292015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:26.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 -- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 msgr2=0x7f292015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291e7fc640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29200d0470 0x7f292014aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:26.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 0x7f292015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 -- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29200d0470 msgr2=0x7f292014aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29200d0470 0x7f292014aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 -- 172.21.15.71:0/2717383278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918009820 con 0x7f29200d0850 2023-10-26T05:47:26.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291f7fe640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 0x7f29200c8890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2918009b10 tx=0x7f2918003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:26.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918004a50 con 0x7f29200d0850 2023-10-26T05:47:26.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2918004ca0 con 0x7f29200d0850 2023-10-26T05:47:26.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2918004210 con 0x7f29200d0850 2023-10-26T05:47:26.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200c5680 con 0x7f29200d0850 2023-10-26T05:47:26.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200c5c40 con 0x7f29200d0850 2023-10-26T05:47:26.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2920066b10 con 0x7f29200d0850 2023-10-26T05:47:26.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29180b4040 con 0x7f29200d0850 2023-10-26T05:47:26.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f29177fe640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:26.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f291811de00 con 0x7f29200d0850 2023-10-26T05:47:26.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291effd640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:26.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.276+0000 7f291effd640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2910009ab0 tx=0x7f29100091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:26.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.280+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29180eb220 con 0x7f29200d0850 2023-10-26T05:47:26.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.420+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29200721e0 con 0x7f2900066470 2023-10-26T05:47:26.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.440+0000 7f29177fe640 1 -- 172.21.15.71:0/2717383278 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260103 (secure 0 0 0) 0x7f29200721e0 con 0x7f2900066470 2023-10-26T05:47:26.446 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:26.449 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:26.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 msgr2=0x7f2900068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2910009ab0 tx=0x7f29100091e0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 msgr2=0x7f29200c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 0x7f29200c8890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2918009b10 tx=0x7f2918003f40 comp rx=0 tx=0).stop 2023-10-26T05:47:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 shutdown_connections 2023-10-26T05:47:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29200d0850 0x7f29200c8890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29200d0470 0x7f292014aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 --2- 172.21.15.71:0/2717383278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29200cd880 0x7f292015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 >> 172.21.15.71:0/2717383278 conn(0x7f29200cdd80 msgr2=0x7f292015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 shutdown_connections 2023-10-26T05:47:26.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:26.444+0000 7f2925360640 1 -- 172.21.15.71:0/2717383278 wait complete. 2023-10-26T05:47:26.464 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-26T05:47:25.412242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248540,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.356123"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:22.078297+0000","last_became_peered":"2023-10-26T05:47:22.078297+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032989+0000","last_change":"2023-10-26T05:47:22.050802+0000","last_active":"2023-10-26T05:47:23.032989+0000","last_peered":"2023-10-26T05:47:23.032989+0000","last_clean":"2023-10-26T05:47:23.032989+0000","last_became_active":"2023-10-26T05:47:22.049800+0000","last_became_peered":"2023-10-26T05:47:22.049800+0000","last_unstale":"2023-10-26T05:47:23.032989+0000","last_undegraded":"2023-10-26T05:47:23.032989+0000","last_fullsized":"2023-10-26T05:47:23.032989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595881+0000","last_change":"2023-10-26T05:47:22.059994+0000","last_active":"2023-10-26T05:47:23.595881+0000","last_peered":"2023-10-26T05:47:23.595881+0000","last_clean":"2023-10-26T05:47:23.595881+0000","last_became_active":"2023-10-26T05:47:22.058233+0000","last_became_peered":"2023-10-26T05:47:22.058233+0000","last_unstale":"2023-10-26T05:47:23.595881+0000","last_undegraded":"2023-10-26T05:47:23.595881+0000","last_fullsized":"2023-10-26T05:47:23.595881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:26.807782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296464+0000","last_change":"2023-10-26T05:47:22.069835+0000","last_active":"2023-10-26T05:47:23.296464+0000","last_peered":"2023-10-26T05:47:23.296464+0000","last_clean":"2023-10-26T05:47:23.296464+0000","last_became_active":"2023-10-26T05:47:22.069004+0000","last_became_peered":"2023-10-26T05:47:22.069004+0000","last_unstale":"2023-10-26T05:47:23.296464+0000","last_undegraded":"2023-10-26T05:47:23.296464+0000","last_fullsized":"2023-10-26T05:47:23.296464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299356+0000","last_change":"2023-10-26T05:47:22.074037+0000","last_active":"2023-10-26T05:47:23.299356+0000","last_peered":"2023-10-26T05:47:23.299356+0000","last_clean":"2023-10-26T05:47:23.299356+0000","last_became_active":"2023-10-26T05:47:22.073426+0000","last_became_peered":"2023-10-26T05:47:22.073426+0000","last_unstale":"2023-10-26T05:47:23.299356+0000","last_undegraded":"2023-10-26T05:47:23.299356+0000","last_fullsized":"2023-10-26T05:47:23.299356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297524+0000","last_change":"2023-10-26T05:47:22.056648+0000","last_active":"2023-10-26T05:47:23.297524+0000","last_peered":"2023-10-26T05:47:23.297524+0000","last_clean":"2023-10-26T05:47:23.297524+0000","last_became_active":"2023-10-26T05:47:22.051034+0000","last_became_peered":"2023-10-26T05:47:22.051034+0000","last_unstale":"2023-10-26T05:47:23.297524+0000","last_undegraded":"2023-10-26T05:47:23.297524+0000","last_fullsized":"2023-10-26T05:47:23.297524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300018+0000","last_change":"2023-10-26T05:47:22.073286+0000","last_active":"2023-10-26T05:47:23.300018+0000","last_peered":"2023-10-26T05:47:23.300018+0000","last_clean":"2023-10-26T05:47:23.300018+0000","last_became_active":"2023-10-26T05:47:22.072189+0000","last_became_peered":"2023-10-26T05:47:22.072189+0000","last_unstale":"2023-10-26T05:47:23.300018+0000","last_undegraded":"2023-10-26T05:47:23.300018+0000","last_fullsized":"2023-10-26T05:47:23.300018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:43.485135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290644+0000","last_change":"2023-10-26T05:47:22.054592+0000","last_active":"2023-10-26T05:47:23.290644+0000","last_peered":"2023-10-26T05:47:23.290644+0000","last_clean":"2023-10-26T05:47:23.290644+0000","last_became_active":"2023-10-26T05:47:22.048761+0000","last_became_peered":"2023-10-26T05:47:22.048761+0000","last_unstale":"2023-10-26T05:47:23.290644+0000","last_undegraded":"2023-10-26T05:47:23.290644+0000","last_fullsized":"2023-10-26T05:47:23.290644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296847+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.296847+0000","last_peered":"2023-10-26T05:47:23.296847+0000","last_clean":"2023-10-26T05:47:23.296847+0000","last_became_active":"2023-10-26T05:47:22.066674+0000","last_became_peered":"2023-10-26T05:47:22.066674+0000","last_unstale":"2023-10-26T05:47:23.296847+0000","last_undegraded":"2023-10-26T05:47:23.296847+0000","last_fullsized":"2023-10-26T05:47:23.296847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298450+0000","last_change":"2023-10-26T05:47:22.054456+0000","last_active":"2023-10-26T05:47:23.298450+0000","last_peered":"2023-10-26T05:47:23.298450+0000","last_clean":"2023-10-26T05:47:23.298450+0000","last_became_active":"2023-10-26T05:47:22.053876+0000","last_became_peered":"2023-10-26T05:47:22.053876+0000","last_unstale":"2023-10-26T05:47:23.298450+0000","last_undegraded":"2023-10-26T05:47:23.298450+0000","last_fullsized":"2023-10-26T05:47:23.298450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298522+0000","last_change":"2023-10-26T05:47:22.061734+0000","last_active":"2023-10-26T05:47:23.298522+0000","last_peered":"2023-10-26T05:47:23.298522+0000","last_clean":"2023-10-26T05:47:23.298522+0000","last_became_active":"2023-10-26T05:47:22.061125+0000","last_became_peered":"2023-10-26T05:47:22.061125+0000","last_unstale":"2023-10-26T05:47:23.298522+0000","last_undegraded":"2023-10-26T05:47:23.298522+0000","last_fullsized":"2023-10-26T05:47:23.298522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039953+0000","last_change":"2023-10-26T05:47:22.059027+0000","last_active":"2023-10-26T05:47:23.039953+0000","last_peered":"2023-10-26T05:47:23.039953+0000","last_clean":"2023-10-26T05:47:23.039953+0000","last_became_active":"2023-10-26T05:47:22.058395+0000","last_became_peered":"2023-10-26T05:47:22.058395+0000","last_unstale":"2023-10-26T05:47:23.039953+0000","last_undegraded":"2023-10-26T05:47:23.039953+0000","last_fullsized":"2023-10-26T05:47:23.039953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:26.058039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291056+0000","last_change":"2023-10-26T05:47:22.054240+0000","last_active":"2023-10-26T05:47:23.291056+0000","last_peered":"2023-10-26T05:47:23.291056+0000","last_clean":"2023-10-26T05:47:23.291056+0000","last_became_active":"2023-10-26T05:47:22.047954+0000","last_became_peered":"2023-10-26T05:47:22.047954+0000","last_unstale":"2023-10-26T05:47:23.291056+0000","last_undegraded":"2023-10-26T05:47:23.291056+0000","last_fullsized":"2023-10-26T05:47:23.291056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595789+0000","last_change":"2023-10-26T05:47:22.046495+0000","last_active":"2023-10-26T05:47:23.595789+0000","last_peered":"2023-10-26T05:47:23.595789+0000","last_clean":"2023-10-26T05:47:23.595789+0000","last_became_active":"2023-10-26T05:47:22.045867+0000","last_became_peered":"2023-10-26T05:47:22.045867+0000","last_unstale":"2023-10-26T05:47:23.595789+0000","last_undegraded":"2023-10-26T05:47:23.595789+0000","last_fullsized":"2023-10-26T05:47:23.595789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:36.941020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033315+0000","last_change":"2023-10-26T05:47:22.058776+0000","last_active":"2023-10-26T05:47:23.033315+0000","last_peered":"2023-10-26T05:47:23.033315+0000","last_clean":"2023-10-26T05:47:23.033315+0000","last_became_active":"2023-10-26T05:47:22.057978+0000","last_became_peered":"2023-10-26T05:47:22.057978+0000","last_unstale":"2023-10-26T05:47:23.033315+0000","last_undegraded":"2023-10-26T05:47:23.033315+0000","last_fullsized":"2023-10-26T05:47:23.033315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297178+0000","last_change":"2023-10-26T05:47:22.055995+0000","last_active":"2023-10-26T05:47:23.297178+0000","last_peered":"2023-10-26T05:47:23.297178+0000","last_clean":"2023-10-26T05:47:23.297178+0000","last_became_active":"2023-10-26T05:47:22.049810+0000","last_became_peered":"2023-10-26T05:47:22.049810+0000","last_unstale":"2023-10-26T05:47:23.297178+0000","last_undegraded":"2023-10-26T05:47:23.297178+0000","last_fullsized":"2023-10-26T05:47:23.297178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299268+0000","last_change":"2023-10-26T05:47:22.071995+0000","last_active":"2023-10-26T05:47:23.299268+0000","last_peered":"2023-10-26T05:47:23.299268+0000","last_clean":"2023-10-26T05:47:23.299268+0000","last_became_active":"2023-10-26T05:47:22.070349+0000","last_became_peered":"2023-10-26T05:47:22.070349+0000","last_unstale":"2023-10-26T05:47:23.299268+0000","last_undegraded":"2023-10-26T05:47:23.299268+0000","last_fullsized":"2023-10-26T05:47:23.299268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033881+0000","last_change":"2023-10-26T05:47:22.051874+0000","last_active":"2023-10-26T05:47:23.033881+0000","last_peered":"2023-10-26T05:47:23.033881+0000","last_clean":"2023-10-26T05:47:23.033881+0000","last_became_active":"2023-10-26T05:47:22.051002+0000","last_became_peered":"2023-10-26T05:47:22.051002+0000","last_unstale":"2023-10-26T05:47:23.033881+0000","last_undegraded":"2023-10-26T05:47:23.033881+0000","last_fullsized":"2023-10-26T05:47:23.033881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297488+0000","last_change":"2023-10-26T05:47:22.080998+0000","last_active":"2023-10-26T05:47:23.297488+0000","last_peered":"2023-10-26T05:47:23.297488+0000","last_clean":"2023-10-26T05:47:23.297488+0000","last_became_active":"2023-10-26T05:47:22.078649+0000","last_became_peered":"2023-10-26T05:47:22.078649+0000","last_unstale":"2023-10-26T05:47:23.297488+0000","last_undegraded":"2023-10-26T05:47:23.297488+0000","last_fullsized":"2023-10-26T05:47:23.297488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298814+0000","last_change":"2023-10-26T05:47:22.052419+0000","last_active":"2023-10-26T05:47:23.298814+0000","last_peered":"2023-10-26T05:47:23.298814+0000","last_clean":"2023-10-26T05:47:23.298814+0000","last_became_active":"2023-10-26T05:47:22.051751+0000","last_became_peered":"2023-10-26T05:47:22.051751+0000","last_unstale":"2023-10-26T05:47:23.298814+0000","last_undegraded":"2023-10-26T05:47:23.298814+0000","last_fullsized":"2023-10-26T05:47:23.298814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596450+0000","last_change":"2023-10-26T05:47:22.073364+0000","last_active":"2023-10-26T05:47:23.596450+0000","last_peered":"2023-10-26T05:47:23.596450+0000","last_clean":"2023-10-26T05:47:23.596450+0000","last_became_active":"2023-10-26T05:47:22.072258+0000","last_became_peered":"2023-10-26T05:47:22.072258+0000","last_unstale":"2023-10-26T05:47:23.596450+0000","last_undegraded":"2023-10-26T05:47:23.596450+0000","last_fullsized":"2023-10-26T05:47:23.596450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299915+0000","last_change":"2023-10-26T05:47:22.075501+0000","last_active":"2023-10-26T05:47:23.299915+0000","last_peered":"2023-10-26T05:47:23.299915+0000","last_clean":"2023-10-26T05:47:23.299915+0000","last_became_active":"2023-10-26T05:47:22.074808+0000","last_became_peered":"2023-10-26T05:47:22.074808+0000","last_unstale":"2023-10-26T05:47:23.299915+0000","last_undegraded":"2023-10-26T05:47:23.299915+0000","last_fullsized":"2023-10-26T05:47:23.299915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291547+0000","last_change":"2023-10-26T05:47:22.061162+0000","last_active":"2023-10-26T05:47:23.291547+0000","last_peered":"2023-10-26T05:47:23.291547+0000","last_clean":"2023-10-26T05:47:23.291547+0000","last_became_active":"2023-10-26T05:47:22.053623+0000","last_became_peered":"2023-10-26T05:47:22.053623+0000","last_unstale":"2023-10-26T05:47:23.291547+0000","last_undegraded":"2023-10-26T05:47:23.291547+0000","last_fullsized":"2023-10-26T05:47:23.291547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296810+0000","last_change":"2023-10-26T05:47:22.063651+0000","last_active":"2023-10-26T05:47:23.296810+0000","last_peered":"2023-10-26T05:47:23.296810+0000","last_clean":"2023-10-26T05:47:23.296810+0000","last_became_active":"2023-10-26T05:47:22.062599+0000","last_became_peered":"2023-10-26T05:47:22.062599+0000","last_unstale":"2023-10-26T05:47:23.296810+0000","last_undegraded":"2023-10-26T05:47:23.296810+0000","last_fullsized":"2023-10-26T05:47:23.296810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290912+0000","last_change":"2023-10-26T05:47:22.053793+0000","last_active":"2023-10-26T05:47:23.290912+0000","last_peered":"2023-10-26T05:47:23.290912+0000","last_clean":"2023-10-26T05:47:23.290912+0000","last_became_active":"2023-10-26T05:47:22.052294+0000","last_became_peered":"2023-10-26T05:47:22.052294+0000","last_unstale":"2023-10-26T05:47:23.290912+0000","last_undegraded":"2023-10-26T05:47:23.290912+0000","last_fullsized":"2023-10-26T05:47:23.290912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298905+0000","last_change":"2023-10-26T05:47:22.078582+0000","last_active":"2023-10-26T05:47:23.298905+0000","last_peered":"2023-10-26T05:47:23.298905+0000","last_clean":"2023-10-26T05:47:23.298905+0000","last_became_active":"2023-10-26T05:47:22.077883+0000","last_became_peered":"2023-10-26T05:47:22.077883+0000","last_unstale":"2023-10-26T05:47:23.298905+0000","last_undegraded":"2023-10-26T05:47:23.298905+0000","last_fullsized":"2023-10-26T05:47:23.298905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033302+0000","last_change":"2023-10-26T05:47:22.057439+0000","last_active":"2023-10-26T05:47:23.033302+0000","last_peered":"2023-10-26T05:47:23.033302+0000","last_clean":"2023-10-26T05:47:23.033302+0000","last_became_active":"2023-10-26T05:47:22.056731+0000","last_became_peered":"2023-10-26T05:47:22.056731+0000","last_unstale":"2023-10-26T05:47:23.033302+0000","last_undegraded":"2023-10-26T05:47:23.033302+0000","last_fullsized":"2023-10-26T05:47:23.033302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291223+0000","last_change":"2023-10-26T05:47:22.065579+0000","last_active":"2023-10-26T05:47:23.291223+0000","last_peered":"2023-10-26T05:47:23.291223+0000","last_clean":"2023-10-26T05:47:23.291223+0000","last_became_active":"2023-10-26T05:47:22.063798+0000","last_became_peered":"2023-10-26T05:47:22.063798+0000","last_unstale":"2023-10-26T05:47:23.291223+0000","last_undegraded":"2023-10-26T05:47:23.291223+0000","last_fullsized":"2023-10-26T05:47:23.291223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596104+0000","last_change":"2023-10-26T05:47:22.056700+0000","last_active":"2023-10-26T05:47:23.596104+0000","last_peered":"2023-10-26T05:47:23.596104+0000","last_clean":"2023-10-26T05:47:23.596104+0000","last_became_active":"2023-10-26T05:47:22.056216+0000","last_became_peered":"2023-10-26T05:47:22.056216+0000","last_unstale":"2023-10-26T05:47:23.596104+0000","last_undegraded":"2023-10-26T05:47:23.596104+0000","last_fullsized":"2023-10-26T05:47:23.596104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297162+0000","last_change":"2023-10-26T05:47:22.061465+0000","last_active":"2023-10-26T05:47:23.297162+0000","last_peered":"2023-10-26T05:47:23.297162+0000","last_clean":"2023-10-26T05:47:23.297162+0000","last_became_active":"2023-10-26T05:47:22.060699+0000","last_became_peered":"2023-10-26T05:47:22.060699+0000","last_unstale":"2023-10-26T05:47:23.297162+0000","last_undegraded":"2023-10-26T05:47:23.297162+0000","last_fullsized":"2023-10-26T05:47:23.297162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033512+0000","last_change":"2023-10-26T05:47:22.058403+0000","last_active":"2023-10-26T05:47:23.033512+0000","last_peered":"2023-10-26T05:47:23.033512+0000","last_clean":"2023-10-26T05:47:23.033512+0000","last_became_active":"2023-10-26T05:47:22.057792+0000","last_became_peered":"2023-10-26T05:47:22.057792+0000","last_unstale":"2023-10-26T05:47:23.033512+0000","last_undegraded":"2023-10-26T05:47:23.033512+0000","last_fullsized":"2023-10-26T05:47:23.033512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299297+0000","last_change":"2023-10-26T05:47:22.059009+0000","last_active":"2023-10-26T05:47:23.299297+0000","last_peered":"2023-10-26T05:47:23.299297+0000","last_clean":"2023-10-26T05:47:23.299297+0000","last_became_active":"2023-10-26T05:47:22.057498+0000","last_became_peered":"2023-10-26T05:47:22.057498+0000","last_unstale":"2023-10-26T05:47:23.299297+0000","last_undegraded":"2023-10-26T05:47:23.299297+0000","last_fullsized":"2023-10-26T05:47:23.299297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040301+0000","last_change":"2023-10-26T05:47:22.060276+0000","last_active":"2023-10-26T05:47:23.040301+0000","last_peered":"2023-10-26T05:47:23.040301+0000","last_clean":"2023-10-26T05:47:23.040301+0000","last_became_active":"2023-10-26T05:47:22.059444+0000","last_became_peered":"2023-10-26T05:47:22.059444+0000","last_unstale":"2023-10-26T05:47:23.040301+0000","last_undegraded":"2023-10-26T05:47:23.040301+0000","last_fullsized":"2023-10-26T05:47:23.040301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291438+0000","last_change":"2023-10-26T05:47:21.083390+0000","last_active":"2023-10-26T05:47:23.291438+0000","last_peered":"2023-10-26T05:47:23.291438+0000","last_clean":"2023-10-26T05:47:23.291438+0000","last_became_active":"2023-10-26T05:47:21.083255+0000","last_became_peered":"2023-10-26T05:47:21.083255+0000","last_unstale":"2023-10-26T05:47:23.291438+0000","last_undegraded":"2023-10-26T05:47:23.291438+0000","last_fullsized":"2023-10-26T05:47:23.291438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595999+0000","last_change":"2023-10-26T05:47:22.042083+0000","last_active":"2023-10-26T05:47:23.595999+0000","last_peered":"2023-10-26T05:47:23.595999+0000","last_clean":"2023-10-26T05:47:23.595999+0000","last_became_active":"2023-10-26T05:47:22.041458+0000","last_became_peered":"2023-10-26T05:47:22.041458+0000","last_unstale":"2023-10-26T05:47:23.595999+0000","last_undegraded":"2023-10-26T05:47:23.595999+0000","last_fullsized":"2023-10-26T05:47:23.595999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:18.110818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298965+0000","last_change":"2023-10-26T05:47:21.090954+0000","last_active":"2023-10-26T05:47:23.298965+0000","last_peered":"2023-10-26T05:47:23.298965+0000","last_clean":"2023-10-26T05:47:23.298965+0000","last_became_active":"2023-10-26T05:47:21.086959+0000","last_became_peered":"2023-10-26T05:47:21.086959+0000","last_unstale":"2023-10-26T05:47:23.298965+0000","last_undegraded":"2023-10-26T05:47:23.298965+0000","last_fullsized":"2023-10-26T05:47:23.298965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291059+0000","last_change":"2023-10-26T05:47:21.078842+0000","last_active":"2023-10-26T05:47:23.291059+0000","last_peered":"2023-10-26T05:47:23.291059+0000","last_clean":"2023-10-26T05:47:23.291059+0000","last_became_active":"2023-10-26T05:47:21.078498+0000","last_became_peered":"2023-10-26T05:47:21.078498+0000","last_unstale":"2023-10-26T05:47:23.291059+0000","last_undegraded":"2023-10-26T05:47:23.291059+0000","last_fullsized":"2023-10-26T05:47:23.291059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291157+0000","last_change":"2023-10-26T05:47:22.054327+0000","last_active":"2023-10-26T05:47:23.291157+0000","last_peered":"2023-10-26T05:47:23.291157+0000","last_clean":"2023-10-26T05:47:23.291157+0000","last_became_active":"2023-10-26T05:47:22.053185+0000","last_became_peered":"2023-10-26T05:47:22.053185+0000","last_unstale":"2023-10-26T05:47:23.291157+0000","last_undegraded":"2023-10-26T05:47:23.291157+0000","last_fullsized":"2023-10-26T05:47:23.291157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297471+0000","last_change":"2023-10-26T05:47:21.084041+0000","last_active":"2023-10-26T05:47:23.297471+0000","last_peered":"2023-10-26T05:47:23.297471+0000","last_clean":"2023-10-26T05:47:23.297471+0000","last_became_active":"2023-10-26T05:47:21.082189+0000","last_became_peered":"2023-10-26T05:47:21.082189+0000","last_unstale":"2023-10-26T05:47:23.297471+0000","last_undegraded":"2023-10-26T05:47:23.297471+0000","last_fullsized":"2023-10-26T05:47:23.297471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032584+0000","last_change":"2023-10-26T05:47:22.063104+0000","last_active":"2023-10-26T05:47:23.032584+0000","last_peered":"2023-10-26T05:47:23.032584+0000","last_clean":"2023-10-26T05:47:23.032584+0000","last_became_active":"2023-10-26T05:47:22.062483+0000","last_became_peered":"2023-10-26T05:47:22.062483+0000","last_unstale":"2023-10-26T05:47:23.032584+0000","last_undegraded":"2023-10-26T05:47:23.032584+0000","last_fullsized":"2023-10-26T05:47:23.032584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299320+0000","last_change":"2023-10-26T05:47:21.087323+0000","last_active":"2023-10-26T05:47:23.299320+0000","last_peered":"2023-10-26T05:47:23.299320+0000","last_clean":"2023-10-26T05:47:23.299320+0000","last_became_active":"2023-10-26T05:47:21.086686+0000","last_became_peered":"2023-10-26T05:47:21.086686+0000","last_unstale":"2023-10-26T05:47:23.299320+0000","last_undegraded":"2023-10-26T05:47:23.299320+0000","last_fullsized":"2023-10-26T05:47:23.299320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299593+0000","last_change":"2023-10-26T05:47:22.072429+0000","last_active":"2023-10-26T05:47:23.299593+0000","last_peered":"2023-10-26T05:47:23.299593+0000","last_clean":"2023-10-26T05:47:23.299593+0000","last_became_active":"2023-10-26T05:47:22.071355+0000","last_became_peered":"2023-10-26T05:47:22.071355+0000","last_unstale":"2023-10-26T05:47:23.299593+0000","last_undegraded":"2023-10-26T05:47:23.299593+0000","last_fullsized":"2023-10-26T05:47:23.299593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300874+0000","last_change":"2023-10-26T05:47:21.096076+0000","last_active":"2023-10-26T05:47:23.300874+0000","last_peered":"2023-10-26T05:47:23.300874+0000","last_clean":"2023-10-26T05:47:23.300874+0000","last_became_active":"2023-10-26T05:47:21.092744+0000","last_became_peered":"2023-10-26T05:47:21.092744+0000","last_unstale":"2023-10-26T05:47:23.300874+0000","last_undegraded":"2023-10-26T05:47:23.300874+0000","last_fullsized":"2023-10-26T05:47:23.300874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:26.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596655+0000","last_change":"2023-10-26T05:47:22.055264+0000","last_active":"2023-10-26T05:47:23.596655+0000","last_peered":"2023-10-26T05:47:23.596655+0000","last_clean":"2023-10-26T05:47:23.596655+0000","last_became_active":"2023-10-26T05:47:22.054516+0000","last_became_peered":"2023-10-26T05:47:22.054516+0000","last_unstale":"2023-10-26T05:47:23.596655+0000","last_undegraded":"2023-10-26T05:47:23.596655+0000","last_fullsized":"2023-10-26T05:47:23.596655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033126+0000","last_change":"2023-10-26T05:47:21.069747+0000","last_active":"2023-10-26T05:47:23.033126+0000","last_peered":"2023-10-26T05:47:23.033126+0000","last_clean":"2023-10-26T05:47:23.033126+0000","last_became_active":"2023-10-26T05:47:21.069525+0000","last_became_peered":"2023-10-26T05:47:21.069525+0000","last_unstale":"2023-10-26T05:47:23.033126+0000","last_undegraded":"2023-10-26T05:47:23.033126+0000","last_fullsized":"2023-10-26T05:47:23.033126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039554+0000","last_change":"2023-10-26T05:47:22.070886+0000","last_active":"2023-10-26T05:47:23.039554+0000","last_peered":"2023-10-26T05:47:23.039554+0000","last_clean":"2023-10-26T05:47:23.039554+0000","last_became_active":"2023-10-26T05:47:22.068908+0000","last_became_peered":"2023-10-26T05:47:22.068908+0000","last_unstale":"2023-10-26T05:47:23.039554+0000","last_undegraded":"2023-10-26T05:47:23.039554+0000","last_fullsized":"2023-10-26T05:47:23.039554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:31.276801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297743+0000","last_change":"2023-10-26T05:47:21.085627+0000","last_active":"2023-10-26T05:47:23.297743+0000","last_peered":"2023-10-26T05:47:23.297743+0000","last_clean":"2023-10-26T05:47:23.297743+0000","last_became_active":"2023-10-26T05:47:21.084306+0000","last_became_peered":"2023-10-26T05:47:21.084306+0000","last_unstale":"2023-10-26T05:47:23.297743+0000","last_undegraded":"2023-10-26T05:47:23.297743+0000","last_fullsized":"2023-10-26T05:47:23.297743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299433+0000","last_change":"2023-10-26T05:47:22.071700+0000","last_active":"2023-10-26T05:47:23.299433+0000","last_peered":"2023-10-26T05:47:23.299433+0000","last_clean":"2023-10-26T05:47:23.299433+0000","last_became_active":"2023-10-26T05:47:22.071056+0000","last_became_peered":"2023-10-26T05:47:22.071056+0000","last_unstale":"2023-10-26T05:47:23.299433+0000","last_undegraded":"2023-10-26T05:47:23.299433+0000","last_fullsized":"2023-10-26T05:47:23.299433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596128+0000","last_change":"2023-10-26T05:47:21.069670+0000","last_active":"2023-10-26T05:47:23.596128+0000","last_peered":"2023-10-26T05:47:23.596128+0000","last_clean":"2023-10-26T05:47:23.596128+0000","last_became_active":"2023-10-26T05:47:21.069297+0000","last_became_peered":"2023-10-26T05:47:21.069297+0000","last_unstale":"2023-10-26T05:47:23.596128+0000","last_undegraded":"2023-10-26T05:47:23.596128+0000","last_fullsized":"2023-10-26T05:47:23.596128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:15.639318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596815+0000","last_change":"2023-10-26T05:47:22.073645+0000","last_active":"2023-10-26T05:47:23.596815+0000","last_peered":"2023-10-26T05:47:23.596815+0000","last_clean":"2023-10-26T05:47:23.596815+0000","last_became_active":"2023-10-26T05:47:22.072989+0000","last_became_peered":"2023-10-26T05:47:22.072989+0000","last_unstale":"2023-10-26T05:47:23.596815+0000","last_undegraded":"2023-10-26T05:47:23.596815+0000","last_fullsized":"2023-10-26T05:47:23.596815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298358+0000","last_change":"2023-10-26T05:47:21.086438+0000","last_active":"2023-10-26T05:47:23.298358+0000","last_peered":"2023-10-26T05:47:23.298358+0000","last_clean":"2023-10-26T05:47:23.298358+0000","last_became_active":"2023-10-26T05:47:21.082053+0000","last_became_peered":"2023-10-26T05:47:21.082053+0000","last_unstale":"2023-10-26T05:47:23.298358+0000","last_undegraded":"2023-10-26T05:47:23.298358+0000","last_fullsized":"2023-10-26T05:47:23.298358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299687+0000","last_change":"2023-10-26T05:47:22.076444+0000","last_active":"2023-10-26T05:47:23.299687+0000","last_peered":"2023-10-26T05:47:23.299687+0000","last_clean":"2023-10-26T05:47:23.299687+0000","last_became_active":"2023-10-26T05:47:22.074572+0000","last_became_peered":"2023-10-26T05:47:22.074572+0000","last_unstale":"2023-10-26T05:47:23.299687+0000","last_undegraded":"2023-10-26T05:47:23.299687+0000","last_fullsized":"2023-10-26T05:47:23.299687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299516+0000","last_change":"2023-10-26T05:47:21.090350+0000","last_active":"2023-10-26T05:47:23.299516+0000","last_peered":"2023-10-26T05:47:23.299516+0000","last_clean":"2023-10-26T05:47:23.299516+0000","last_became_active":"2023-10-26T05:47:21.087109+0000","last_became_peered":"2023-10-26T05:47:21.087109+0000","last_unstale":"2023-10-26T05:47:23.299516+0000","last_undegraded":"2023-10-26T05:47:23.299516+0000","last_fullsized":"2023-10-26T05:47:23.299516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298687+0000","last_change":"2023-10-26T05:47:22.066147+0000","last_active":"2023-10-26T05:47:23.298687+0000","last_peered":"2023-10-26T05:47:23.298687+0000","last_clean":"2023-10-26T05:47:23.298687+0000","last_became_active":"2023-10-26T05:47:22.065497+0000","last_became_peered":"2023-10-26T05:47:22.065497+0000","last_unstale":"2023-10-26T05:47:23.298687+0000","last_undegraded":"2023-10-26T05:47:23.298687+0000","last_fullsized":"2023-10-26T05:47:23.298687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034102+0000","last_change":"2023-10-26T05:47:21.064585+0000","last_active":"2023-10-26T05:47:23.034102+0000","last_peered":"2023-10-26T05:47:23.034102+0000","last_clean":"2023-10-26T05:47:23.034102+0000","last_became_active":"2023-10-26T05:47:21.064395+0000","last_became_peered":"2023-10-26T05:47:21.064395+0000","last_unstale":"2023-10-26T05:47:23.034102+0000","last_undegraded":"2023-10-26T05:47:23.034102+0000","last_fullsized":"2023-10-26T05:47:23.034102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300524+0000","last_change":"2023-10-26T05:47:22.077148+0000","last_active":"2023-10-26T05:47:23.300524+0000","last_peered":"2023-10-26T05:47:23.300524+0000","last_clean":"2023-10-26T05:47:23.300524+0000","last_became_active":"2023-10-26T05:47:22.075279+0000","last_became_peered":"2023-10-26T05:47:22.075279+0000","last_unstale":"2023-10-26T05:47:23.300524+0000","last_undegraded":"2023-10-26T05:47:23.300524+0000","last_fullsized":"2023-10-26T05:47:23.300524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:12:08.628590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039791+0000","last_change":"2023-10-26T05:47:21.077376+0000","last_active":"2023-10-26T05:47:23.039791+0000","last_peered":"2023-10-26T05:47:23.039791+0000","last_clean":"2023-10-26T05:47:23.039791+0000","last_became_active":"2023-10-26T05:47:21.076766+0000","last_became_peered":"2023-10-26T05:47:21.076766+0000","last_unstale":"2023-10-26T05:47:23.039791+0000","last_undegraded":"2023-10-26T05:47:23.039791+0000","last_fullsized":"2023-10-26T05:47:23.039791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299945+0000","last_change":"2023-10-26T05:47:22.077007+0000","last_active":"2023-10-26T05:47:23.299945+0000","last_peered":"2023-10-26T05:47:23.299945+0000","last_clean":"2023-10-26T05:47:23.299945+0000","last_became_active":"2023-10-26T05:47:22.075297+0000","last_became_peered":"2023-10-26T05:47:22.075297+0000","last_unstale":"2023-10-26T05:47:23.299945+0000","last_undegraded":"2023-10-26T05:47:23.299945+0000","last_fullsized":"2023-10-26T05:47:23.299945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:15.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033816+0000","last_change":"2023-10-26T05:47:21.072137+0000","last_active":"2023-10-26T05:47:23.033816+0000","last_peered":"2023-10-26T05:47:23.033816+0000","last_clean":"2023-10-26T05:47:23.033816+0000","last_became_active":"2023-10-26T05:47:21.071720+0000","last_became_peered":"2023-10-26T05:47:21.071720+0000","last_unstale":"2023-10-26T05:47:23.033816+0000","last_undegraded":"2023-10-26T05:47:23.033816+0000","last_fullsized":"2023-10-26T05:47:23.033816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:09.715298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297159+0000","last_change":"2023-10-26T05:47:22.080581+0000","last_active":"2023-10-26T05:47:23.297159+0000","last_peered":"2023-10-26T05:47:23.297159+0000","last_clean":"2023-10-26T05:47:23.297159+0000","last_became_active":"2023-10-26T05:47:22.076913+0000","last_became_peered":"2023-10-26T05:47:22.076913+0000","last_unstale":"2023-10-26T05:47:23.297159+0000","last_undegraded":"2023-10-26T05:47:23.297159+0000","last_fullsized":"2023-10-26T05:47:23.297159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033027+0000","last_change":"2023-10-26T05:47:21.076982+0000","last_active":"2023-10-26T05:47:23.033027+0000","last_peered":"2023-10-26T05:47:23.033027+0000","last_clean":"2023-10-26T05:47:23.033027+0000","last_became_active":"2023-10-26T05:47:21.076695+0000","last_became_peered":"2023-10-26T05:47:21.076695+0000","last_unstale":"2023-10-26T05:47:23.033027+0000","last_undegraded":"2023-10-26T05:47:23.033027+0000","last_fullsized":"2023-10-26T05:47:23.033027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:57.904856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300280+0000","last_change":"2023-10-26T05:47:22.063246+0000","last_active":"2023-10-26T05:47:23.300280+0000","last_peered":"2023-10-26T05:47:23.300280+0000","last_clean":"2023-10-26T05:47:23.300280+0000","last_became_active":"2023-10-26T05:47:22.062499+0000","last_became_peered":"2023-10-26T05:47:22.062499+0000","last_unstale":"2023-10-26T05:47:23.300280+0000","last_undegraded":"2023-10-26T05:47:23.300280+0000","last_fullsized":"2023-10-26T05:47:23.300280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033647+0000","last_change":"2023-10-26T05:47:21.087999+0000","last_active":"2023-10-26T05:47:23.033647+0000","last_peered":"2023-10-26T05:47:23.033647+0000","last_clean":"2023-10-26T05:47:23.033647+0000","last_became_active":"2023-10-26T05:47:21.087744+0000","last_became_peered":"2023-10-26T05:47:21.087744+0000","last_unstale":"2023-10-26T05:47:23.033647+0000","last_undegraded":"2023-10-26T05:47:23.033647+0000","last_fullsized":"2023-10-26T05:47:23.033647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298194+0000","last_change":"2023-10-26T05:47:22.081324+0000","last_active":"2023-10-26T05:47:23.298194+0000","last_peered":"2023-10-26T05:47:23.298194+0000","last_clean":"2023-10-26T05:47:23.298194+0000","last_became_active":"2023-10-26T05:47:22.079073+0000","last_became_peered":"2023-10-26T05:47:22.079073+0000","last_unstale":"2023-10-26T05:47:23.298194+0000","last_undegraded":"2023-10-26T05:47:23.298194+0000","last_fullsized":"2023-10-26T05:47:23.298194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T05:47:22.066065+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:22.066065+0000","last_peered":"2023-10-26T05:47:22.066065+0000","last_clean":"2023-10-26T05:47:22.066065+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:22.066065+0000","last_undegraded":"2023-10-26T05:47:22.066065+0000","last_fullsized":"2023-10-26T05:47:22.066065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298749+0000","last_change":"2023-10-26T05:47:21.087000+0000","last_active":"2023-10-26T05:47:23.298749+0000","last_peered":"2023-10-26T05:47:23.298749+0000","last_clean":"2023-10-26T05:47:23.298749+0000","last_became_active":"2023-10-26T05:47:21.082935+0000","last_became_peered":"2023-10-26T05:47:21.082935+0000","last_unstale":"2023-10-26T05:47:23.298749+0000","last_undegraded":"2023-10-26T05:47:23.298749+0000","last_fullsized":"2023-10-26T05:47:23.298749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291206+0000","last_change":"2023-10-26T05:47:22.055580+0000","last_active":"2023-10-26T05:47:23.291206+0000","last_peered":"2023-10-26T05:47:23.291206+0000","last_clean":"2023-10-26T05:47:23.291206+0000","last_became_active":"2023-10-26T05:47:22.054587+0000","last_became_peered":"2023-10-26T05:47:22.054587+0000","last_unstale":"2023-10-26T05:47:23.291206+0000","last_undegraded":"2023-10-26T05:47:23.291206+0000","last_fullsized":"2023-10-26T05:47:23.291206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297907+0000","last_change":"2023-10-26T05:47:21.100233+0000","last_active":"2023-10-26T05:47:23.297907+0000","last_peered":"2023-10-26T05:47:23.297907+0000","last_clean":"2023-10-26T05:47:23.297907+0000","last_became_active":"2023-10-26T05:47:21.099318+0000","last_became_peered":"2023-10-26T05:47:21.099318+0000","last_unstale":"2023-10-26T05:47:23.297907+0000","last_undegraded":"2023-10-26T05:47:23.297907+0000","last_fullsized":"2023-10-26T05:47:23.297907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596567+0000","last_change":"2023-10-26T05:47:22.057190+0000","last_active":"2023-10-26T05:47:23.596567+0000","last_peered":"2023-10-26T05:47:23.596567+0000","last_clean":"2023-10-26T05:47:23.596567+0000","last_became_active":"2023-10-26T05:47:22.049709+0000","last_became_peered":"2023-10-26T05:47:22.049709+0000","last_unstale":"2023-10-26T05:47:23.596567+0000","last_undegraded":"2023-10-26T05:47:23.596567+0000","last_fullsized":"2023-10-26T05:47:23.596567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034304+0000","last_change":"2023-10-26T05:47:21.073026+0000","last_active":"2023-10-26T05:47:23.034304+0000","last_peered":"2023-10-26T05:47:23.034304+0000","last_clean":"2023-10-26T05:47:23.034304+0000","last_became_active":"2023-10-26T05:47:21.071577+0000","last_became_peered":"2023-10-26T05:47:21.071577+0000","last_unstale":"2023-10-26T05:47:23.034304+0000","last_undegraded":"2023-10-26T05:47:23.034304+0000","last_fullsized":"2023-10-26T05:47:23.034304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:25.047488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032708+0000","last_change":"2023-10-26T05:47:22.052469+0000","last_active":"2023-10-26T05:47:23.032708+0000","last_peered":"2023-10-26T05:47:23.032708+0000","last_clean":"2023-10-26T05:47:23.032708+0000","last_became_active":"2023-10-26T05:47:22.051614+0000","last_became_peered":"2023-10-26T05:47:22.051614+0000","last_unstale":"2023-10-26T05:47:23.032708+0000","last_undegraded":"2023-10-26T05:47:23.032708+0000","last_fullsized":"2023-10-26T05:47:23.032708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.301108+0000","last_change":"2023-10-26T05:47:21.096632+0000","last_active":"2023-10-26T05:47:23.301108+0000","last_peered":"2023-10-26T05:47:23.301108+0000","last_clean":"2023-10-26T05:47:23.301108+0000","last_became_active":"2023-10-26T05:47:21.093612+0000","last_became_peered":"2023-10-26T05:47:21.093612+0000","last_unstale":"2023-10-26T05:47:23.301108+0000","last_undegraded":"2023-10-26T05:47:23.301108+0000","last_fullsized":"2023-10-26T05:47:23.301108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039976+0000","last_change":"2023-10-26T05:47:22.060460+0000","last_active":"2023-10-26T05:47:23.039976+0000","last_peered":"2023-10-26T05:47:23.039976+0000","last_clean":"2023-10-26T05:47:23.039976+0000","last_became_active":"2023-10-26T05:47:22.059831+0000","last_became_peered":"2023-10-26T05:47:22.059831+0000","last_unstale":"2023-10-26T05:47:23.039976+0000","last_undegraded":"2023-10-26T05:47:23.039976+0000","last_fullsized":"2023-10-26T05:47:23.039976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033723+0000","last_change":"2023-10-26T05:47:21.071567+0000","last_active":"2023-10-26T05:47:23.033723+0000","last_peered":"2023-10-26T05:47:23.033723+0000","last_clean":"2023-10-26T05:47:23.033723+0000","last_became_active":"2023-10-26T05:47:21.071275+0000","last_became_peered":"2023-10-26T05:47:21.071275+0000","last_unstale":"2023-10-26T05:47:23.033723+0000","last_undegraded":"2023-10-26T05:47:23.033723+0000","last_fullsized":"2023-10-26T05:47:23.033723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297152+0000","last_change":"2023-10-26T05:47:22.063236+0000","last_active":"2023-10-26T05:47:23.297152+0000","last_peered":"2023-10-26T05:47:23.297152+0000","last_clean":"2023-10-26T05:47:23.297152+0000","last_became_active":"2023-10-26T05:47:22.061746+0000","last_became_peered":"2023-10-26T05:47:22.061746+0000","last_unstale":"2023-10-26T05:47:23.297152+0000","last_undegraded":"2023-10-26T05:47:23.297152+0000","last_fullsized":"2023-10-26T05:47:23.297152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:21.099849+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:21.097871+0000","last_became_peered":"2023-10-26T05:47:21.097871+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032981+0000","last_change":"2023-10-26T05:47:22.050271+0000","last_active":"2023-10-26T05:47:23.032981+0000","last_peered":"2023-10-26T05:47:23.032981+0000","last_clean":"2023-10-26T05:47:23.032981+0000","last_became_active":"2023-10-26T05:47:22.049654+0000","last_became_peered":"2023-10-26T05:47:22.049654+0000","last_unstale":"2023-10-26T05:47:23.032981+0000","last_undegraded":"2023-10-26T05:47:23.032981+0000","last_fullsized":"2023-10-26T05:47:23.032981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033347+0000","last_change":"2023-10-26T05:47:21.078711+0000","last_active":"2023-10-26T05:47:23.033347+0000","last_peered":"2023-10-26T05:47:23.033347+0000","last_clean":"2023-10-26T05:47:23.033347+0000","last_became_active":"2023-10-26T05:47:21.078403+0000","last_became_peered":"2023-10-26T05:47:21.078403+0000","last_unstale":"2023-10-26T05:47:23.033347+0000","last_undegraded":"2023-10-26T05:47:23.033347+0000","last_fullsized":"2023-10-26T05:47:23.033347+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290995+0000","last_change":"2023-10-26T05:47:22.065870+0000","last_active":"2023-10-26T05:47:23.290995+0000","last_peered":"2023-10-26T05:47:23.290995+0000","last_clean":"2023-10-26T05:47:23.290995+0000","last_became_active":"2023-10-26T05:47:22.065412+0000","last_became_peered":"2023-10-26T05:47:22.065412+0000","last_unstale":"2023-10-26T05:47:23.290995+0000","last_undegraded":"2023-10-26T05:47:23.290995+0000","last_fullsized":"2023-10-26T05:47:23.290995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299744+0000","last_change":"2023-10-26T05:47:21.094764+0000","last_active":"2023-10-26T05:47:23.299744+0000","last_peered":"2023-10-26T05:47:23.299744+0000","last_clean":"2023-10-26T05:47:23.299744+0000","last_became_active":"2023-10-26T05:47:21.091991+0000","last_became_peered":"2023-10-26T05:47:21.091991+0000","last_unstale":"2023-10-26T05:47:23.299744+0000","last_undegraded":"2023-10-26T05:47:23.299744+0000","last_fullsized":"2023-10-26T05:47:23.299744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596211+0000","last_change":"2023-10-26T05:47:22.059433+0000","last_active":"2023-10-26T05:47:23.596211+0000","last_peered":"2023-10-26T05:47:23.596211+0000","last_clean":"2023-10-26T05:47:23.596211+0000","last_became_active":"2023-10-26T05:47:22.054466+0000","last_became_peered":"2023-10-26T05:47:22.054466+0000","last_unstale":"2023-10-26T05:47:23.596211+0000","last_undegraded":"2023-10-26T05:47:23.596211+0000","last_fullsized":"2023-10-26T05:47:23.596211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298877+0000","last_change":"2023-10-26T05:47:21.099000+0000","last_active":"2023-10-26T05:47:23.298877+0000","last_peered":"2023-10-26T05:47:23.298877+0000","last_clean":"2023-10-26T05:47:23.298877+0000","last_became_active":"2023-10-26T05:47:21.095552+0000","last_became_peered":"2023-10-26T05:47:21.095552+0000","last_unstale":"2023-10-26T05:47:23.298877+0000","last_undegraded":"2023-10-26T05:47:23.298877+0000","last_fullsized":"2023-10-26T05:47:23.298877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:49.348590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040667+0000","last_change":"2023-10-26T05:47:22.059642+0000","last_active":"2023-10-26T05:47:23.040667+0000","last_peered":"2023-10-26T05:47:23.040667+0000","last_clean":"2023-10-26T05:47:23.040667+0000","last_became_active":"2023-10-26T05:47:22.054716+0000","last_became_peered":"2023-10-26T05:47:22.054716+0000","last_unstale":"2023-10-26T05:47:23.040667+0000","last_undegraded":"2023-10-26T05:47:23.040667+0000","last_fullsized":"2023-10-26T05:47:23.040667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:10.310768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033526+0000","last_change":"2023-10-26T05:47:21.090099+0000","last_active":"2023-10-26T05:47:23.033526+0000","last_peered":"2023-10-26T05:47:23.033526+0000","last_clean":"2023-10-26T05:47:23.033526+0000","last_became_active":"2023-10-26T05:47:21.089837+0000","last_became_peered":"2023-10-26T05:47:21.089837+0000","last_unstale":"2023-10-26T05:47:23.033526+0000","last_undegraded":"2023-10-26T05:47:23.033526+0000","last_fullsized":"2023-10-26T05:47:23.033526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290991+0000","last_change":"2023-10-26T05:47:22.046792+0000","last_active":"2023-10-26T05:47:23.290991+0000","last_peered":"2023-10-26T05:47:23.290991+0000","last_clean":"2023-10-26T05:47:23.290991+0000","last_became_active":"2023-10-26T05:47:22.045151+0000","last_became_peered":"2023-10-26T05:47:22.045151+0000","last_unstale":"2023-10-26T05:47:23.290991+0000","last_undegraded":"2023-10-26T05:47:23.290991+0000","last_fullsized":"2023-10-26T05:47:23.290991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032792+0000","last_change":"2023-10-26T05:47:21.072214+0000","last_active":"2023-10-26T05:47:23.032792+0000","last_peered":"2023-10-26T05:47:23.032792+0000","last_clean":"2023-10-26T05:47:23.032792+0000","last_became_active":"2023-10-26T05:47:21.071876+0000","last_became_peered":"2023-10-26T05:47:21.071876+0000","last_unstale":"2023-10-26T05:47:23.032792+0000","last_undegraded":"2023-10-26T05:47:23.032792+0000","last_fullsized":"2023-10-26T05:47:23.032792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291391+0000","last_change":"2023-10-26T05:47:22.066034+0000","last_active":"2023-10-26T05:47:23.291391+0000","last_peered":"2023-10-26T05:47:23.291391+0000","last_clean":"2023-10-26T05:47:23.291391+0000","last_became_active":"2023-10-26T05:47:22.064885+0000","last_became_peered":"2023-10-26T05:47:22.064885+0000","last_unstale":"2023-10-26T05:47:23.291391+0000","last_undegraded":"2023-10-26T05:47:23.291391+0000","last_fullsized":"2023-10-26T05:47:23.291391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299704+0000","last_change":"2023-10-26T05:47:21.093006+0000","last_active":"2023-10-26T05:47:23.299704+0000","last_peered":"2023-10-26T05:47:23.299704+0000","last_clean":"2023-10-26T05:47:23.299704+0000","last_became_active":"2023-10-26T05:47:21.092342+0000","last_became_peered":"2023-10-26T05:47:21.092342+0000","last_unstale":"2023-10-26T05:47:23.299704+0000","last_undegraded":"2023-10-26T05:47:23.299704+0000","last_fullsized":"2023-10-26T05:47:23.299704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:12.988430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296978+0000","last_change":"2023-10-26T05:47:22.059661+0000","last_active":"2023-10-26T05:47:23.296978+0000","last_peered":"2023-10-26T05:47:23.296978+0000","last_clean":"2023-10-26T05:47:23.296978+0000","last_became_active":"2023-10-26T05:47:22.058379+0000","last_became_peered":"2023-10-26T05:47:22.058379+0000","last_unstale":"2023-10-26T05:47:23.296978+0000","last_undegraded":"2023-10-26T05:47:23.296978+0000","last_fullsized":"2023-10-26T05:47:23.296978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297352+0000","last_change":"2023-10-26T05:47:22.045989+0000","last_active":"2023-10-26T05:47:23.297352+0000","last_peered":"2023-10-26T05:47:23.297352+0000","last_clean":"2023-10-26T05:47:23.297352+0000","last_became_active":"2023-10-26T05:47:22.044537+0000","last_became_peered":"2023-10-26T05:47:22.044537+0000","last_unstale":"2023-10-26T05:47:23.297352+0000","last_undegraded":"2023-10-26T05:47:23.297352+0000","last_fullsized":"2023-10-26T05:47:23.297352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298638+0000","last_change":"2023-10-26T05:47:21.099538+0000","last_active":"2023-10-26T05:47:23.298638+0000","last_peered":"2023-10-26T05:47:23.298638+0000","last_clean":"2023-10-26T05:47:23.298638+0000","last_became_active":"2023-10-26T05:47:21.096351+0000","last_became_peered":"2023-10-26T05:47:21.096351+0000","last_unstale":"2023-10-26T05:47:23.298638+0000","last_undegraded":"2023-10-26T05:47:23.298638+0000","last_fullsized":"2023-10-26T05:47:23.298638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:21.625489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297831+0000","last_change":"2023-10-26T05:47:22.059627+0000","last_active":"2023-10-26T05:47:23.297831+0000","last_peered":"2023-10-26T05:47:23.297831+0000","last_clean":"2023-10-26T05:47:23.297831+0000","last_became_active":"2023-10-26T05:47:22.054248+0000","last_became_peered":"2023-10-26T05:47:22.054248+0000","last_unstale":"2023-10-26T05:47:23.297831+0000","last_undegraded":"2023-10-26T05:47:23.297831+0000","last_fullsized":"2023-10-26T05:47:23.297831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300077+0000","last_change":"2023-10-26T05:47:21.097760+0000","last_active":"2023-10-26T05:47:23.300077+0000","last_peered":"2023-10-26T05:47:23.300077+0000","last_clean":"2023-10-26T05:47:23.300077+0000","last_became_active":"2023-10-26T05:47:21.093680+0000","last_became_peered":"2023-10-26T05:47:21.093680+0000","last_unstale":"2023-10-26T05:47:23.300077+0000","last_undegraded":"2023-10-26T05:47:23.300077+0000","last_fullsized":"2023-10-26T05:47:23.300077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596384+0000","last_change":"2023-10-26T05:47:22.038934+0000","last_active":"2023-10-26T05:47:23.596384+0000","last_peered":"2023-10-26T05:47:23.596384+0000","last_clean":"2023-10-26T05:47:23.596384+0000","last_became_active":"2023-10-26T05:47:22.038330+0000","last_became_peered":"2023-10-26T05:47:22.038330+0000","last_unstale":"2023-10-26T05:47:23.596384+0000","last_undegraded":"2023-10-26T05:47:23.596384+0000","last_fullsized":"2023-10-26T05:47:23.596384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290969+0000","last_change":"2023-10-26T05:47:21.070756+0000","last_active":"2023-10-26T05:47:23.290969+0000","last_peered":"2023-10-26T05:47:23.290969+0000","last_clean":"2023-10-26T05:47:23.290969+0000","last_became_active":"2023-10-26T05:47:21.070211+0000","last_became_peered":"2023-10-26T05:47:21.070211+0000","last_unstale":"2023-10-26T05:47:23.290969+0000","last_undegraded":"2023-10-26T05:47:23.290969+0000","last_fullsized":"2023-10-26T05:47:23.290969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290817+0000","last_change":"2023-10-26T05:47:22.062259+0000","last_active":"2023-10-26T05:47:23.290817+0000","last_peered":"2023-10-26T05:47:23.290817+0000","last_clean":"2023-10-26T05:47:23.290817+0000","last_became_active":"2023-10-26T05:47:22.055535+0000","last_became_peered":"2023-10-26T05:47:22.055535+0000","last_unstale":"2023-10-26T05:47:23.290817+0000","last_undegraded":"2023-10-26T05:47:23.290817+0000","last_fullsized":"2023-10-26T05:47:23.290817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596691+0000","last_change":"2023-10-26T05:47:21.084394+0000","last_active":"2023-10-26T05:47:23.596691+0000","last_peered":"2023-10-26T05:47:23.596691+0000","last_clean":"2023-10-26T05:47:23.596691+0000","last_became_active":"2023-10-26T05:47:21.083775+0000","last_became_peered":"2023-10-26T05:47:21.083775+0000","last_unstale":"2023-10-26T05:47:23.596691+0000","last_undegraded":"2023-10-26T05:47:23.596691+0000","last_fullsized":"2023-10-26T05:47:23.596691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:29.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:29.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/251634716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c13c2f0 msgr2=0x7fa73c0b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 --2- 172.21.15.71:0/251634716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c13c2f0 0x7fa73c0b2350 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009900 tx=0x7fa72c01c8c0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/251634716 shutdown_connections 2023-10-26T05:47:29.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 --2- 172.21.15.71:0/251634716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0b2890 0x7fa73c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 --2- 172.21.15.71:0/251634716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c13c2f0 0x7fa73c0b2350 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/251634716 >> 172.21.15.71:0/251634716 conn(0x7fa73c069770 msgr2=0x7fa73c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:29.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/251634716 shutdown_connections 2023-10-26T05:47:29.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/251634716 wait complete. 2023-10-26T05:47:29.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 Processor -- start 2023-10-26T05:47:29.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- start start 2023-10-26T05:47:29.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 0x7fa73c131a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0cb010 0x7fa73c163430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c13c2f0 0x7fa73c163970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c15b150 con 0x7fa73c0b2890 2023-10-26T05:47:29.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0b0900 con 0x7fa73c13c2f0 2023-10-26T05:47:29.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0b0620 con 0x7fa73c0cb010 2023-10-26T05:47:29.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 0x7fa73c131a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 0x7fa73c131a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53158/0 (socket says 172.21.15.71:53158) 2023-10-26T05:47:29.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 -- 172.21.15.71:0/1794794889 learned_addr learned my addr 172.21.15.71:0/1794794889 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:29.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa739d9b640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0cb010 0x7fa73c163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73ad9d640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c13c2f0 0x7fa73c163970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 -- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0cb010 msgr2=0x7fa73c163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0cb010 0x7fa73c163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 -- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c13c2f0 msgr2=0x7fa73c163970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c13c2f0 0x7fa73c163970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 -- 172.21.15.71:0/1794794889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c009560 con 0x7fa73c0b2890 2023-10-26T05:47:29.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa739d9b640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0cb010 0x7fa73c163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:29.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73ad9d640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c13c2f0 0x7fa73c163970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:29.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa73a59c640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 0x7fa73c131a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa72c01a4f0 tx=0x7fa72c0a6af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:29.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0bc070 con 0x7fa73c0b2890 2023-10-26T05:47:29.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa72c004130 con 0x7fa73c0b2890 2023-10-26T05:47:29.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c15ab00 con 0x7fa73c0b2890 2023-10-26T05:47:29.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c004500 con 0x7fa73c0b2890 2023-10-26T05:47:29.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.600+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c131f80 con 0x7fa73c0b2890 2023-10-26T05:47:29.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.604+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c066b10 con 0x7fa73c0b2890 2023-10-26T05:47:29.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.604+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa72c003600 con 0x7fa73c0b2890 2023-10-26T05:47:29.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.604+0000 7fa72affd640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa71c066470 0x7fa71c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.604+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa72c0aa070 con 0x7fa73c0b2890 2023-10-26T05:47:29.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.604+0000 7fa739d9b640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa71c066470 0x7fa71c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.604+0000 7fa739d9b640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa71c066470 0x7fa71c068930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa730005fd0 tx=0x7fa730005d60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:29.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.608+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa72c0ebde0 con 0x7fa73c0b2890 2023-10-26T05:47:29.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.756+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa73c15a980 con 0x7fa71c066470 2023-10-26T05:47:29.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.776+0000 7fa72affd640 1 -- 172.21.15.71:0/1794794889 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260171 (secure 0 0 0) 0x7fa73000da50 con 0x7fa71c066470 2023-10-26T05:47:29.786 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:29.787 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:29.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa71c066470 msgr2=0x7fa71c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa71c066470 0x7fa71c068930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa730005fd0 tx=0x7fa730005d60 comp rx=0 tx=0).stop 2023-10-26T05:47:29.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 msgr2=0x7fa73c131a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 0x7fa73c131a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa72c01a4f0 tx=0x7fa72c0a6af0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 shutdown_connections 2023-10-26T05:47:29.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c13c2f0 0x7fa73c163970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0cb010 0x7fa73c163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa71c066470 0x7fa71c068930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 --2- 172.21.15.71:0/1794794889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa73c0b2890 0x7fa73c131a40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 >> 172.21.15.71:0/1794794889 conn(0x7fa73c069770 msgr2=0x7fa73c0ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:29.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 shutdown_connections 2023-10-26T05:47:29.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.780+0000 7fa740a0e640 1 -- 172.21.15.71:0/1794794889 wait complete. 2023-10-26T05:47:29.804 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-26T05:47:29.413757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248540,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.012039"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:22.078297+0000","last_became_peered":"2023-10-26T05:47:22.078297+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032989+0000","last_change":"2023-10-26T05:47:22.050802+0000","last_active":"2023-10-26T05:47:23.032989+0000","last_peered":"2023-10-26T05:47:23.032989+0000","last_clean":"2023-10-26T05:47:23.032989+0000","last_became_active":"2023-10-26T05:47:22.049800+0000","last_became_peered":"2023-10-26T05:47:22.049800+0000","last_unstale":"2023-10-26T05:47:23.032989+0000","last_undegraded":"2023-10-26T05:47:23.032989+0000","last_fullsized":"2023-10-26T05:47:23.032989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595881+0000","last_change":"2023-10-26T05:47:22.059994+0000","last_active":"2023-10-26T05:47:23.595881+0000","last_peered":"2023-10-26T05:47:23.595881+0000","last_clean":"2023-10-26T05:47:23.595881+0000","last_became_active":"2023-10-26T05:47:22.058233+0000","last_became_peered":"2023-10-26T05:47:22.058233+0000","last_unstale":"2023-10-26T05:47:23.595881+0000","last_undegraded":"2023-10-26T05:47:23.595881+0000","last_fullsized":"2023-10-26T05:47:23.595881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:26.807782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296464+0000","last_change":"2023-10-26T05:47:22.069835+0000","last_active":"2023-10-26T05:47:23.296464+0000","last_peered":"2023-10-26T05:47:23.296464+0000","last_clean":"2023-10-26T05:47:23.296464+0000","last_became_active":"2023-10-26T05:47:22.069004+0000","last_became_peered":"2023-10-26T05:47:22.069004+0000","last_unstale":"2023-10-26T05:47:23.296464+0000","last_undegraded":"2023-10-26T05:47:23.296464+0000","last_fullsized":"2023-10-26T05:47:23.296464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299356+0000","last_change":"2023-10-26T05:47:22.074037+0000","last_active":"2023-10-26T05:47:23.299356+0000","last_peered":"2023-10-26T05:47:23.299356+0000","last_clean":"2023-10-26T05:47:23.299356+0000","last_became_active":"2023-10-26T05:47:22.073426+0000","last_became_peered":"2023-10-26T05:47:22.073426+0000","last_unstale":"2023-10-26T05:47:23.299356+0000","last_undegraded":"2023-10-26T05:47:23.299356+0000","last_fullsized":"2023-10-26T05:47:23.299356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297524+0000","last_change":"2023-10-26T05:47:22.056648+0000","last_active":"2023-10-26T05:47:23.297524+0000","last_peered":"2023-10-26T05:47:23.297524+0000","last_clean":"2023-10-26T05:47:23.297524+0000","last_became_active":"2023-10-26T05:47:22.051034+0000","last_became_peered":"2023-10-26T05:47:22.051034+0000","last_unstale":"2023-10-26T05:47:23.297524+0000","last_undegraded":"2023-10-26T05:47:23.297524+0000","last_fullsized":"2023-10-26T05:47:23.297524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300018+0000","last_change":"2023-10-26T05:47:22.073286+0000","last_active":"2023-10-26T05:47:23.300018+0000","last_peered":"2023-10-26T05:47:23.300018+0000","last_clean":"2023-10-26T05:47:23.300018+0000","last_became_active":"2023-10-26T05:47:22.072189+0000","last_became_peered":"2023-10-26T05:47:22.072189+0000","last_unstale":"2023-10-26T05:47:23.300018+0000","last_undegraded":"2023-10-26T05:47:23.300018+0000","last_fullsized":"2023-10-26T05:47:23.300018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:43.485135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290644+0000","last_change":"2023-10-26T05:47:22.054592+0000","last_active":"2023-10-26T05:47:23.290644+0000","last_peered":"2023-10-26T05:47:23.290644+0000","last_clean":"2023-10-26T05:47:23.290644+0000","last_became_active":"2023-10-26T05:47:22.048761+0000","last_became_peered":"2023-10-26T05:47:22.048761+0000","last_unstale":"2023-10-26T05:47:23.290644+0000","last_undegraded":"2023-10-26T05:47:23.290644+0000","last_fullsized":"2023-10-26T05:47:23.290644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296847+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.296847+0000","last_peered":"2023-10-26T05:47:23.296847+0000","last_clean":"2023-10-26T05:47:23.296847+0000","last_became_active":"2023-10-26T05:47:22.066674+0000","last_became_peered":"2023-10-26T05:47:22.066674+0000","last_unstale":"2023-10-26T05:47:23.296847+0000","last_undegraded":"2023-10-26T05:47:23.296847+0000","last_fullsized":"2023-10-26T05:47:23.296847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298450+0000","last_change":"2023-10-26T05:47:22.054456+0000","last_active":"2023-10-26T05:47:23.298450+0000","last_peered":"2023-10-26T05:47:23.298450+0000","last_clean":"2023-10-26T05:47:23.298450+0000","last_became_active":"2023-10-26T05:47:22.053876+0000","last_became_peered":"2023-10-26T05:47:22.053876+0000","last_unstale":"2023-10-26T05:47:23.298450+0000","last_undegraded":"2023-10-26T05:47:23.298450+0000","last_fullsized":"2023-10-26T05:47:23.298450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298522+0000","last_change":"2023-10-26T05:47:22.061734+0000","last_active":"2023-10-26T05:47:23.298522+0000","last_peered":"2023-10-26T05:47:23.298522+0000","last_clean":"2023-10-26T05:47:23.298522+0000","last_became_active":"2023-10-26T05:47:22.061125+0000","last_became_peered":"2023-10-26T05:47:22.061125+0000","last_unstale":"2023-10-26T05:47:23.298522+0000","last_undegraded":"2023-10-26T05:47:23.298522+0000","last_fullsized":"2023-10-26T05:47:23.298522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039953+0000","last_change":"2023-10-26T05:47:22.059027+0000","last_active":"2023-10-26T05:47:23.039953+0000","last_peered":"2023-10-26T05:47:23.039953+0000","last_clean":"2023-10-26T05:47:23.039953+0000","last_became_active":"2023-10-26T05:47:22.058395+0000","last_became_peered":"2023-10-26T05:47:22.058395+0000","last_unstale":"2023-10-26T05:47:23.039953+0000","last_undegraded":"2023-10-26T05:47:23.039953+0000","last_fullsized":"2023-10-26T05:47:23.039953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:26.058039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291056+0000","last_change":"2023-10-26T05:47:22.054240+0000","last_active":"2023-10-26T05:47:23.291056+0000","last_peered":"2023-10-26T05:47:23.291056+0000","last_clean":"2023-10-26T05:47:23.291056+0000","last_became_active":"2023-10-26T05:47:22.047954+0000","last_became_peered":"2023-10-26T05:47:22.047954+0000","last_unstale":"2023-10-26T05:47:23.291056+0000","last_undegraded":"2023-10-26T05:47:23.291056+0000","last_fullsized":"2023-10-26T05:47:23.291056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595789+0000","last_change":"2023-10-26T05:47:22.046495+0000","last_active":"2023-10-26T05:47:23.595789+0000","last_peered":"2023-10-26T05:47:23.595789+0000","last_clean":"2023-10-26T05:47:23.595789+0000","last_became_active":"2023-10-26T05:47:22.045867+0000","last_became_peered":"2023-10-26T05:47:22.045867+0000","last_unstale":"2023-10-26T05:47:23.595789+0000","last_undegraded":"2023-10-26T05:47:23.595789+0000","last_fullsized":"2023-10-26T05:47:23.595789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:36.941020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033315+0000","last_change":"2023-10-26T05:47:22.058776+0000","last_active":"2023-10-26T05:47:23.033315+0000","last_peered":"2023-10-26T05:47:23.033315+0000","last_clean":"2023-10-26T05:47:23.033315+0000","last_became_active":"2023-10-26T05:47:22.057978+0000","last_became_peered":"2023-10-26T05:47:22.057978+0000","last_unstale":"2023-10-26T05:47:23.033315+0000","last_undegraded":"2023-10-26T05:47:23.033315+0000","last_fullsized":"2023-10-26T05:47:23.033315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297178+0000","last_change":"2023-10-26T05:47:22.055995+0000","last_active":"2023-10-26T05:47:23.297178+0000","last_peered":"2023-10-26T05:47:23.297178+0000","last_clean":"2023-10-26T05:47:23.297178+0000","last_became_active":"2023-10-26T05:47:22.049810+0000","last_became_peered":"2023-10-26T05:47:22.049810+0000","last_unstale":"2023-10-26T05:47:23.297178+0000","last_undegraded":"2023-10-26T05:47:23.297178+0000","last_fullsized":"2023-10-26T05:47:23.297178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299268+0000","last_change":"2023-10-26T05:47:22.071995+0000","last_active":"2023-10-26T05:47:23.299268+0000","last_peered":"2023-10-26T05:47:23.299268+0000","last_clean":"2023-10-26T05:47:23.299268+0000","last_became_active":"2023-10-26T05:47:22.070349+0000","last_became_peered":"2023-10-26T05:47:22.070349+0000","last_unstale":"2023-10-26T05:47:23.299268+0000","last_undegraded":"2023-10-26T05:47:23.299268+0000","last_fullsized":"2023-10-26T05:47:23.299268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033881+0000","last_change":"2023-10-26T05:47:22.051874+0000","last_active":"2023-10-26T05:47:23.033881+0000","last_peered":"2023-10-26T05:47:23.033881+0000","last_clean":"2023-10-26T05:47:23.033881+0000","last_became_active":"2023-10-26T05:47:22.051002+0000","last_became_peered":"2023-10-26T05:47:22.051002+0000","last_unstale":"2023-10-26T05:47:23.033881+0000","last_undegraded":"2023-10-26T05:47:23.033881+0000","last_fullsized":"2023-10-26T05:47:23.033881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297488+0000","last_change":"2023-10-26T05:47:22.080998+0000","last_active":"2023-10-26T05:47:23.297488+0000","last_peered":"2023-10-26T05:47:23.297488+0000","last_clean":"2023-10-26T05:47:23.297488+0000","last_became_active":"2023-10-26T05:47:22.078649+0000","last_became_peered":"2023-10-26T05:47:22.078649+0000","last_unstale":"2023-10-26T05:47:23.297488+0000","last_undegraded":"2023-10-26T05:47:23.297488+0000","last_fullsized":"2023-10-26T05:47:23.297488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298814+0000","last_change":"2023-10-26T05:47:22.052419+0000","last_active":"2023-10-26T05:47:23.298814+0000","last_peered":"2023-10-26T05:47:23.298814+0000","last_clean":"2023-10-26T05:47:23.298814+0000","last_became_active":"2023-10-26T05:47:22.051751+0000","last_became_peered":"2023-10-26T05:47:22.051751+0000","last_unstale":"2023-10-26T05:47:23.298814+0000","last_undegraded":"2023-10-26T05:47:23.298814+0000","last_fullsized":"2023-10-26T05:47:23.298814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596450+0000","last_change":"2023-10-26T05:47:22.073364+0000","last_active":"2023-10-26T05:47:23.596450+0000","last_peered":"2023-10-26T05:47:23.596450+0000","last_clean":"2023-10-26T05:47:23.596450+0000","last_became_active":"2023-10-26T05:47:22.072258+0000","last_became_peered":"2023-10-26T05:47:22.072258+0000","last_unstale":"2023-10-26T05:47:23.596450+0000","last_undegraded":"2023-10-26T05:47:23.596450+0000","last_fullsized":"2023-10-26T05:47:23.596450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299915+0000","last_change":"2023-10-26T05:47:22.075501+0000","last_active":"2023-10-26T05:47:23.299915+0000","last_peered":"2023-10-26T05:47:23.299915+0000","last_clean":"2023-10-26T05:47:23.299915+0000","last_became_active":"2023-10-26T05:47:22.074808+0000","last_became_peered":"2023-10-26T05:47:22.074808+0000","last_unstale":"2023-10-26T05:47:23.299915+0000","last_undegraded":"2023-10-26T05:47:23.299915+0000","last_fullsized":"2023-10-26T05:47:23.299915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291547+0000","last_change":"2023-10-26T05:47:22.061162+0000","last_active":"2023-10-26T05:47:23.291547+0000","last_peered":"2023-10-26T05:47:23.291547+0000","last_clean":"2023-10-26T05:47:23.291547+0000","last_became_active":"2023-10-26T05:47:22.053623+0000","last_became_peered":"2023-10-26T05:47:22.053623+0000","last_unstale":"2023-10-26T05:47:23.291547+0000","last_undegraded":"2023-10-26T05:47:23.291547+0000","last_fullsized":"2023-10-26T05:47:23.291547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296810+0000","last_change":"2023-10-26T05:47:22.063651+0000","last_active":"2023-10-26T05:47:23.296810+0000","last_peered":"2023-10-26T05:47:23.296810+0000","last_clean":"2023-10-26T05:47:23.296810+0000","last_became_active":"2023-10-26T05:47:22.062599+0000","last_became_peered":"2023-10-26T05:47:22.062599+0000","last_unstale":"2023-10-26T05:47:23.296810+0000","last_undegraded":"2023-10-26T05:47:23.296810+0000","last_fullsized":"2023-10-26T05:47:23.296810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290912+0000","last_change":"2023-10-26T05:47:22.053793+0000","last_active":"2023-10-26T05:47:23.290912+0000","last_peered":"2023-10-26T05:47:23.290912+0000","last_clean":"2023-10-26T05:47:23.290912+0000","last_became_active":"2023-10-26T05:47:22.052294+0000","last_became_peered":"2023-10-26T05:47:22.052294+0000","last_unstale":"2023-10-26T05:47:23.290912+0000","last_undegraded":"2023-10-26T05:47:23.290912+0000","last_fullsized":"2023-10-26T05:47:23.290912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298905+0000","last_change":"2023-10-26T05:47:22.078582+0000","last_active":"2023-10-26T05:47:23.298905+0000","last_peered":"2023-10-26T05:47:23.298905+0000","last_clean":"2023-10-26T05:47:23.298905+0000","last_became_active":"2023-10-26T05:47:22.077883+0000","last_became_peered":"2023-10-26T05:47:22.077883+0000","last_unstale":"2023-10-26T05:47:23.298905+0000","last_undegraded":"2023-10-26T05:47:23.298905+0000","last_fullsized":"2023-10-26T05:47:23.298905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033302+0000","last_change":"2023-10-26T05:47:22.057439+0000","last_active":"2023-10-26T05:47:23.033302+0000","last_peered":"2023-10-26T05:47:23.033302+0000","last_clean":"2023-10-26T05:47:23.033302+0000","last_became_active":"2023-10-26T05:47:22.056731+0000","last_became_peered":"2023-10-26T05:47:22.056731+0000","last_unstale":"2023-10-26T05:47:23.033302+0000","last_undegraded":"2023-10-26T05:47:23.033302+0000","last_fullsized":"2023-10-26T05:47:23.033302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291223+0000","last_change":"2023-10-26T05:47:22.065579+0000","last_active":"2023-10-26T05:47:23.291223+0000","last_peered":"2023-10-26T05:47:23.291223+0000","last_clean":"2023-10-26T05:47:23.291223+0000","last_became_active":"2023-10-26T05:47:22.063798+0000","last_became_peered":"2023-10-26T05:47:22.063798+0000","last_unstale":"2023-10-26T05:47:23.291223+0000","last_undegraded":"2023-10-26T05:47:23.291223+0000","last_fullsized":"2023-10-26T05:47:23.291223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596104+0000","last_change":"2023-10-26T05:47:22.056700+0000","last_active":"2023-10-26T05:47:23.596104+0000","last_peered":"2023-10-26T05:47:23.596104+0000","last_clean":"2023-10-26T05:47:23.596104+0000","last_became_active":"2023-10-26T05:47:22.056216+0000","last_became_peered":"2023-10-26T05:47:22.056216+0000","last_unstale":"2023-10-26T05:47:23.596104+0000","last_undegraded":"2023-10-26T05:47:23.596104+0000","last_fullsized":"2023-10-26T05:47:23.596104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297162+0000","last_change":"2023-10-26T05:47:22.061465+0000","last_active":"2023-10-26T05:47:23.297162+0000","last_peered":"2023-10-26T05:47:23.297162+0000","last_clean":"2023-10-26T05:47:23.297162+0000","last_became_active":"2023-10-26T05:47:22.060699+0000","last_became_peered":"2023-10-26T05:47:22.060699+0000","last_unstale":"2023-10-26T05:47:23.297162+0000","last_undegraded":"2023-10-26T05:47:23.297162+0000","last_fullsized":"2023-10-26T05:47:23.297162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033512+0000","last_change":"2023-10-26T05:47:22.058403+0000","last_active":"2023-10-26T05:47:23.033512+0000","last_peered":"2023-10-26T05:47:23.033512+0000","last_clean":"2023-10-26T05:47:23.033512+0000","last_became_active":"2023-10-26T05:47:22.057792+0000","last_became_peered":"2023-10-26T05:47:22.057792+0000","last_unstale":"2023-10-26T05:47:23.033512+0000","last_undegraded":"2023-10-26T05:47:23.033512+0000","last_fullsized":"2023-10-26T05:47:23.033512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299297+0000","last_change":"2023-10-26T05:47:22.059009+0000","last_active":"2023-10-26T05:47:23.299297+0000","last_peered":"2023-10-26T05:47:23.299297+0000","last_clean":"2023-10-26T05:47:23.299297+0000","last_became_active":"2023-10-26T05:47:22.057498+0000","last_became_peered":"2023-10-26T05:47:22.057498+0000","last_unstale":"2023-10-26T05:47:23.299297+0000","last_undegraded":"2023-10-26T05:47:23.299297+0000","last_fullsized":"2023-10-26T05:47:23.299297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040301+0000","last_change":"2023-10-26T05:47:22.060276+0000","last_active":"2023-10-26T05:47:23.040301+0000","last_peered":"2023-10-26T05:47:23.040301+0000","last_clean":"2023-10-26T05:47:23.040301+0000","last_became_active":"2023-10-26T05:47:22.059444+0000","last_became_peered":"2023-10-26T05:47:22.059444+0000","last_unstale":"2023-10-26T05:47:23.040301+0000","last_undegraded":"2023-10-26T05:47:23.040301+0000","last_fullsized":"2023-10-26T05:47:23.040301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291438+0000","last_change":"2023-10-26T05:47:21.083390+0000","last_active":"2023-10-26T05:47:23.291438+0000","last_peered":"2023-10-26T05:47:23.291438+0000","last_clean":"2023-10-26T05:47:23.291438+0000","last_became_active":"2023-10-26T05:47:21.083255+0000","last_became_peered":"2023-10-26T05:47:21.083255+0000","last_unstale":"2023-10-26T05:47:23.291438+0000","last_undegraded":"2023-10-26T05:47:23.291438+0000","last_fullsized":"2023-10-26T05:47:23.291438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595999+0000","last_change":"2023-10-26T05:47:22.042083+0000","last_active":"2023-10-26T05:47:23.595999+0000","last_peered":"2023-10-26T05:47:23.595999+0000","last_clean":"2023-10-26T05:47:23.595999+0000","last_became_active":"2023-10-26T05:47:22.041458+0000","last_became_peered":"2023-10-26T05:47:22.041458+0000","last_unstale":"2023-10-26T05:47:23.595999+0000","last_undegraded":"2023-10-26T05:47:23.595999+0000","last_fullsized":"2023-10-26T05:47:23.595999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:18.110818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298965+0000","last_change":"2023-10-26T05:47:21.090954+0000","last_active":"2023-10-26T05:47:23.298965+0000","last_peered":"2023-10-26T05:47:23.298965+0000","last_clean":"2023-10-26T05:47:23.298965+0000","last_became_active":"2023-10-26T05:47:21.086959+0000","last_became_peered":"2023-10-26T05:47:21.086959+0000","last_unstale":"2023-10-26T05:47:23.298965+0000","last_undegraded":"2023-10-26T05:47:23.298965+0000","last_fullsized":"2023-10-26T05:47:23.298965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291059+0000","last_change":"2023-10-26T05:47:21.078842+0000","last_active":"2023-10-26T05:47:23.291059+0000","last_peered":"2023-10-26T05:47:23.291059+0000","last_clean":"2023-10-26T05:47:23.291059+0000","last_became_active":"2023-10-26T05:47:21.078498+0000","last_became_peered":"2023-10-26T05:47:21.078498+0000","last_unstale":"2023-10-26T05:47:23.291059+0000","last_undegraded":"2023-10-26T05:47:23.291059+0000","last_fullsized":"2023-10-26T05:47:23.291059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291157+0000","last_change":"2023-10-26T05:47:22.054327+0000","last_active":"2023-10-26T05:47:23.291157+0000","last_peered":"2023-10-26T05:47:23.291157+0000","last_clean":"2023-10-26T05:47:23.291157+0000","last_became_active":"2023-10-26T05:47:22.053185+0000","last_became_peered":"2023-10-26T05:47:22.053185+0000","last_unstale":"2023-10-26T05:47:23.291157+0000","last_undegraded":"2023-10-26T05:47:23.291157+0000","last_fullsized":"2023-10-26T05:47:23.291157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297471+0000","last_change":"2023-10-26T05:47:21.084041+0000","last_active":"2023-10-26T05:47:23.297471+0000","last_peered":"2023-10-26T05:47:23.297471+0000","last_clean":"2023-10-26T05:47:23.297471+0000","last_became_active":"2023-10-26T05:47:21.082189+0000","last_became_peered":"2023-10-26T05:47:21.082189+0000","last_unstale":"2023-10-26T05:47:23.297471+0000","last_undegraded":"2023-10-26T05:47:23.297471+0000","last_fullsized":"2023-10-26T05:47:23.297471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032584+0000","last_change":"2023-10-26T05:47:22.063104+0000","last_active":"2023-10-26T05:47:23.032584+0000","last_peered":"2023-10-26T05:47:23.032584+0000","last_clean":"2023-10-26T05:47:23.032584+0000","last_became_active":"2023-10-26T05:47:22.062483+0000","last_became_peered":"2023-10-26T05:47:22.062483+0000","last_unstale":"2023-10-26T05:47:23.032584+0000","last_undegraded":"2023-10-26T05:47:23.032584+0000","last_fullsized":"2023-10-26T05:47:23.032584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299320+0000","last_change":"2023-10-26T05:47:21.087323+0000","last_active":"2023-10-26T05:47:23.299320+0000","last_peered":"2023-10-26T05:47:23.299320+0000","last_clean":"2023-10-26T05:47:23.299320+0000","last_became_active":"2023-10-26T05:47:21.086686+0000","last_became_peered":"2023-10-26T05:47:21.086686+0000","last_unstale":"2023-10-26T05:47:23.299320+0000","last_undegraded":"2023-10-26T05:47:23.299320+0000","last_fullsized":"2023-10-26T05:47:23.299320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299593+0000","last_change":"2023-10-26T05:47:22.072429+0000","last_active":"2023-10-26T05:47:23.299593+0000","last_peered":"2023-10-26T05:47:23.299593+0000","last_clean":"2023-10-26T05:47:23.299593+0000","last_became_active":"2023-10-26T05:47:22.071355+0000","last_became_peered":"2023-10-26T05:47:22.071355+0000","last_unstale":"2023-10-26T05:47:23.299593+0000","last_undegraded":"2023-10-26T05:47:23.299593+0000","last_fullsized":"2023-10-26T05:47:23.299593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300874+0000","last_change":"2023-10-26T05:47:21.096076+0000","last_active":"2023-10-26T05:47:23.300874+0000","last_peered":"2023-10-26T05:47:23.300874+0000","last_clean":"2023-10-26T05:47:23.300874+0000","last_became_active":"2023-10-26T05:47:21.092744+0000","last_became_peered":"2023-10-26T05:47:21.092744+0000","last_unstale":"2023-10-26T05:47:23.300874+0000","last_undegraded":"2023-10-26T05:47:23.300874+0000","last_fullsized":"2023-10-26T05:47:23.300874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:26.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596655+0000","last_change":"2023-10-26T05:47:22.055264+0000","last_active":"2023-10-26T05:47:23.596655+0000","last_peered":"2023-10-26T05:47:23.596655+0000","last_clean":"2023-10-26T05:47:23.596655+0000","last_became_active":"2023-10-26T05:47:22.054516+0000","last_became_peered":"2023-10-26T05:47:22.054516+0000","last_unstale":"2023-10-26T05:47:23.596655+0000","last_undegraded":"2023-10-26T05:47:23.596655+0000","last_fullsized":"2023-10-26T05:47:23.596655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033126+0000","last_change":"2023-10-26T05:47:21.069747+0000","last_active":"2023-10-26T05:47:23.033126+0000","last_peered":"2023-10-26T05:47:23.033126+0000","last_clean":"2023-10-26T05:47:23.033126+0000","last_became_active":"2023-10-26T05:47:21.069525+0000","last_became_peered":"2023-10-26T05:47:21.069525+0000","last_unstale":"2023-10-26T05:47:23.033126+0000","last_undegraded":"2023-10-26T05:47:23.033126+0000","last_fullsized":"2023-10-26T05:47:23.033126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039554+0000","last_change":"2023-10-26T05:47:22.070886+0000","last_active":"2023-10-26T05:47:23.039554+0000","last_peered":"2023-10-26T05:47:23.039554+0000","last_clean":"2023-10-26T05:47:23.039554+0000","last_became_active":"2023-10-26T05:47:22.068908+0000","last_became_peered":"2023-10-26T05:47:22.068908+0000","last_unstale":"2023-10-26T05:47:23.039554+0000","last_undegraded":"2023-10-26T05:47:23.039554+0000","last_fullsized":"2023-10-26T05:47:23.039554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:31.276801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297743+0000","last_change":"2023-10-26T05:47:21.085627+0000","last_active":"2023-10-26T05:47:23.297743+0000","last_peered":"2023-10-26T05:47:23.297743+0000","last_clean":"2023-10-26T05:47:23.297743+0000","last_became_active":"2023-10-26T05:47:21.084306+0000","last_became_peered":"2023-10-26T05:47:21.084306+0000","last_unstale":"2023-10-26T05:47:23.297743+0000","last_undegraded":"2023-10-26T05:47:23.297743+0000","last_fullsized":"2023-10-26T05:47:23.297743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299433+0000","last_change":"2023-10-26T05:47:22.071700+0000","last_active":"2023-10-26T05:47:23.299433+0000","last_peered":"2023-10-26T05:47:23.299433+0000","last_clean":"2023-10-26T05:47:23.299433+0000","last_became_active":"2023-10-26T05:47:22.071056+0000","last_became_peered":"2023-10-26T05:47:22.071056+0000","last_unstale":"2023-10-26T05:47:23.299433+0000","last_undegraded":"2023-10-26T05:47:23.299433+0000","last_fullsized":"2023-10-26T05:47:23.299433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596128+0000","last_change":"2023-10-26T05:47:21.069670+0000","last_active":"2023-10-26T05:47:23.596128+0000","last_peered":"2023-10-26T05:47:23.596128+0000","last_clean":"2023-10-26T05:47:23.596128+0000","last_became_active":"2023-10-26T05:47:21.069297+0000","last_became_peered":"2023-10-26T05:47:21.069297+0000","last_unstale":"2023-10-26T05:47:23.596128+0000","last_undegraded":"2023-10-26T05:47:23.596128+0000","last_fullsized":"2023-10-26T05:47:23.596128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:15.639318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596815+0000","last_change":"2023-10-26T05:47:22.073645+0000","last_active":"2023-10-26T05:47:23.596815+0000","last_peered":"2023-10-26T05:47:23.596815+0000","last_clean":"2023-10-26T05:47:23.596815+0000","last_became_active":"2023-10-26T05:47:22.072989+0000","last_became_peered":"2023-10-26T05:47:22.072989+0000","last_unstale":"2023-10-26T05:47:23.596815+0000","last_undegraded":"2023-10-26T05:47:23.596815+0000","last_fullsized":"2023-10-26T05:47:23.596815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298358+0000","last_change":"2023-10-26T05:47:21.086438+0000","last_active":"2023-10-26T05:47:23.298358+0000","last_peered":"2023-10-26T05:47:23.298358+0000","last_clean":"2023-10-26T05:47:23.298358+0000","last_became_active":"2023-10-26T05:47:21.082053+0000","last_became_peered":"2023-10-26T05:47:21.082053+0000","last_unstale":"2023-10-26T05:47:23.298358+0000","last_undegraded":"2023-10-26T05:47:23.298358+0000","last_fullsized":"2023-10-26T05:47:23.298358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299687+0000","last_change":"2023-10-26T05:47:22.076444+0000","last_active":"2023-10-26T05:47:23.299687+0000","last_peered":"2023-10-26T05:47:23.299687+0000","last_clean":"2023-10-26T05:47:23.299687+0000","last_became_active":"2023-10-26T05:47:22.074572+0000","last_became_peered":"2023-10-26T05:47:22.074572+0000","last_unstale":"2023-10-26T05:47:23.299687+0000","last_undegraded":"2023-10-26T05:47:23.299687+0000","last_fullsized":"2023-10-26T05:47:23.299687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299516+0000","last_change":"2023-10-26T05:47:21.090350+0000","last_active":"2023-10-26T05:47:23.299516+0000","last_peered":"2023-10-26T05:47:23.299516+0000","last_clean":"2023-10-26T05:47:23.299516+0000","last_became_active":"2023-10-26T05:47:21.087109+0000","last_became_peered":"2023-10-26T05:47:21.087109+0000","last_unstale":"2023-10-26T05:47:23.299516+0000","last_undegraded":"2023-10-26T05:47:23.299516+0000","last_fullsized":"2023-10-26T05:47:23.299516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298687+0000","last_change":"2023-10-26T05:47:22.066147+0000","last_active":"2023-10-26T05:47:23.298687+0000","last_peered":"2023-10-26T05:47:23.298687+0000","last_clean":"2023-10-26T05:47:23.298687+0000","last_became_active":"2023-10-26T05:47:22.065497+0000","last_became_peered":"2023-10-26T05:47:22.065497+0000","last_unstale":"2023-10-26T05:47:23.298687+0000","last_undegraded":"2023-10-26T05:47:23.298687+0000","last_fullsized":"2023-10-26T05:47:23.298687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034102+0000","last_change":"2023-10-26T05:47:21.064585+0000","last_active":"2023-10-26T05:47:23.034102+0000","last_peered":"2023-10-26T05:47:23.034102+0000","last_clean":"2023-10-26T05:47:23.034102+0000","last_became_active":"2023-10-26T05:47:21.064395+0000","last_became_peered":"2023-10-26T05:47:21.064395+0000","last_unstale":"2023-10-26T05:47:23.034102+0000","last_undegraded":"2023-10-26T05:47:23.034102+0000","last_fullsized":"2023-10-26T05:47:23.034102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300524+0000","last_change":"2023-10-26T05:47:22.077148+0000","last_active":"2023-10-26T05:47:23.300524+0000","last_peered":"2023-10-26T05:47:23.300524+0000","last_clean":"2023-10-26T05:47:23.300524+0000","last_became_active":"2023-10-26T05:47:22.075279+0000","last_became_peered":"2023-10-26T05:47:22.075279+0000","last_unstale":"2023-10-26T05:47:23.300524+0000","last_undegraded":"2023-10-26T05:47:23.300524+0000","last_fullsized":"2023-10-26T05:47:23.300524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:12:08.628590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039791+0000","last_change":"2023-10-26T05:47:21.077376+0000","last_active":"2023-10-26T05:47:23.039791+0000","last_peered":"2023-10-26T05:47:23.039791+0000","last_clean":"2023-10-26T05:47:23.039791+0000","last_became_active":"2023-10-26T05:47:21.076766+0000","last_became_peered":"2023-10-26T05:47:21.076766+0000","last_unstale":"2023-10-26T05:47:23.039791+0000","last_undegraded":"2023-10-26T05:47:23.039791+0000","last_fullsized":"2023-10-26T05:47:23.039791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299945+0000","last_change":"2023-10-26T05:47:22.077007+0000","last_active":"2023-10-26T05:47:23.299945+0000","last_peered":"2023-10-26T05:47:23.299945+0000","last_clean":"2023-10-26T05:47:23.299945+0000","last_became_active":"2023-10-26T05:47:22.075297+0000","last_became_peered":"2023-10-26T05:47:22.075297+0000","last_unstale":"2023-10-26T05:47:23.299945+0000","last_undegraded":"2023-10-26T05:47:23.299945+0000","last_fullsized":"2023-10-26T05:47:23.299945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:15.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033816+0000","last_change":"2023-10-26T05:47:21.072137+0000","last_active":"2023-10-26T05:47:23.033816+0000","last_peered":"2023-10-26T05:47:23.033816+0000","last_clean":"2023-10-26T05:47:23.033816+0000","last_became_active":"2023-10-26T05:47:21.071720+0000","last_became_peered":"2023-10-26T05:47:21.071720+0000","last_unstale":"2023-10-26T05:47:23.033816+0000","last_undegraded":"2023-10-26T05:47:23.033816+0000","last_fullsized":"2023-10-26T05:47:23.033816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:09.715298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297159+0000","last_change":"2023-10-26T05:47:22.080581+0000","last_active":"2023-10-26T05:47:23.297159+0000","last_peered":"2023-10-26T05:47:23.297159+0000","last_clean":"2023-10-26T05:47:23.297159+0000","last_became_active":"2023-10-26T05:47:22.076913+0000","last_became_peered":"2023-10-26T05:47:22.076913+0000","last_unstale":"2023-10-26T05:47:23.297159+0000","last_undegraded":"2023-10-26T05:47:23.297159+0000","last_fullsized":"2023-10-26T05:47:23.297159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033027+0000","last_change":"2023-10-26T05:47:21.076982+0000","last_active":"2023-10-26T05:47:23.033027+0000","last_peered":"2023-10-26T05:47:23.033027+0000","last_clean":"2023-10-26T05:47:23.033027+0000","last_became_active":"2023-10-26T05:47:21.076695+0000","last_became_peered":"2023-10-26T05:47:21.076695+0000","last_unstale":"2023-10-26T05:47:23.033027+0000","last_undegraded":"2023-10-26T05:47:23.033027+0000","last_fullsized":"2023-10-26T05:47:23.033027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:57.904856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300280+0000","last_change":"2023-10-26T05:47:22.063246+0000","last_active":"2023-10-26T05:47:23.300280+0000","last_peered":"2023-10-26T05:47:23.300280+0000","last_clean":"2023-10-26T05:47:23.300280+0000","last_became_active":"2023-10-26T05:47:22.062499+0000","last_became_peered":"2023-10-26T05:47:22.062499+0000","last_unstale":"2023-10-26T05:47:23.300280+0000","last_undegraded":"2023-10-26T05:47:23.300280+0000","last_fullsized":"2023-10-26T05:47:23.300280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033647+0000","last_change":"2023-10-26T05:47:21.087999+0000","last_active":"2023-10-26T05:47:23.033647+0000","last_peered":"2023-10-26T05:47:23.033647+0000","last_clean":"2023-10-26T05:47:23.033647+0000","last_became_active":"2023-10-26T05:47:21.087744+0000","last_became_peered":"2023-10-26T05:47:21.087744+0000","last_unstale":"2023-10-26T05:47:23.033647+0000","last_undegraded":"2023-10-26T05:47:23.033647+0000","last_fullsized":"2023-10-26T05:47:23.033647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298194+0000","last_change":"2023-10-26T05:47:22.081324+0000","last_active":"2023-10-26T05:47:23.298194+0000","last_peered":"2023-10-26T05:47:23.298194+0000","last_clean":"2023-10-26T05:47:23.298194+0000","last_became_active":"2023-10-26T05:47:22.079073+0000","last_became_peered":"2023-10-26T05:47:22.079073+0000","last_unstale":"2023-10-26T05:47:23.298194+0000","last_undegraded":"2023-10-26T05:47:23.298194+0000","last_fullsized":"2023-10-26T05:47:23.298194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T05:47:22.066065+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:22.066065+0000","last_peered":"2023-10-26T05:47:22.066065+0000","last_clean":"2023-10-26T05:47:22.066065+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:22.066065+0000","last_undegraded":"2023-10-26T05:47:22.066065+0000","last_fullsized":"2023-10-26T05:47:22.066065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298749+0000","last_change":"2023-10-26T05:47:21.087000+0000","last_active":"2023-10-26T05:47:23.298749+0000","last_peered":"2023-10-26T05:47:23.298749+0000","last_clean":"2023-10-26T05:47:23.298749+0000","last_became_active":"2023-10-26T05:47:21.082935+0000","last_became_peered":"2023-10-26T05:47:21.082935+0000","last_unstale":"2023-10-26T05:47:23.298749+0000","last_undegraded":"2023-10-26T05:47:23.298749+0000","last_fullsized":"2023-10-26T05:47:23.298749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291206+0000","last_change":"2023-10-26T05:47:22.055580+0000","last_active":"2023-10-26T05:47:23.291206+0000","last_peered":"2023-10-26T05:47:23.291206+0000","last_clean":"2023-10-26T05:47:23.291206+0000","last_became_active":"2023-10-26T05:47:22.054587+0000","last_became_peered":"2023-10-26T05:47:22.054587+0000","last_unstale":"2023-10-26T05:47:23.291206+0000","last_undegraded":"2023-10-26T05:47:23.291206+0000","last_fullsized":"2023-10-26T05:47:23.291206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297907+0000","last_change":"2023-10-26T05:47:21.100233+0000","last_active":"2023-10-26T05:47:23.297907+0000","last_peered":"2023-10-26T05:47:23.297907+0000","last_clean":"2023-10-26T05:47:23.297907+0000","last_became_active":"2023-10-26T05:47:21.099318+0000","last_became_peered":"2023-10-26T05:47:21.099318+0000","last_unstale":"2023-10-26T05:47:23.297907+0000","last_undegraded":"2023-10-26T05:47:23.297907+0000","last_fullsized":"2023-10-26T05:47:23.297907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596567+0000","last_change":"2023-10-26T05:47:22.057190+0000","last_active":"2023-10-26T05:47:23.596567+0000","last_peered":"2023-10-26T05:47:23.596567+0000","last_clean":"2023-10-26T05:47:23.596567+0000","last_became_active":"2023-10-26T05:47:22.049709+0000","last_became_peered":"2023-10-26T05:47:22.049709+0000","last_unstale":"2023-10-26T05:47:23.596567+0000","last_undegraded":"2023-10-26T05:47:23.596567+0000","last_fullsized":"2023-10-26T05:47:23.596567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034304+0000","last_change":"2023-10-26T05:47:21.073026+0000","last_active":"2023-10-26T05:47:23.034304+0000","last_peered":"2023-10-26T05:47:23.034304+0000","last_clean":"2023-10-26T05:47:23.034304+0000","last_became_active":"2023-10-26T05:47:21.071577+0000","last_became_peered":"2023-10-26T05:47:21.071577+0000","last_unstale":"2023-10-26T05:47:23.034304+0000","last_undegraded":"2023-10-26T05:47:23.034304+0000","last_fullsized":"2023-10-26T05:47:23.034304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:25.047488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032708+0000","last_change":"2023-10-26T05:47:22.052469+0000","last_active":"2023-10-26T05:47:23.032708+0000","last_peered":"2023-10-26T05:47:23.032708+0000","last_clean":"2023-10-26T05:47:23.032708+0000","last_became_active":"2023-10-26T05:47:22.051614+0000","last_became_peered":"2023-10-26T05:47:22.051614+0000","last_unstale":"2023-10-26T05:47:23.032708+0000","last_undegraded":"2023-10-26T05:47:23.032708+0000","last_fullsized":"2023-10-26T05:47:23.032708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.301108+0000","last_change":"2023-10-26T05:47:21.096632+0000","last_active":"2023-10-26T05:47:23.301108+0000","last_peered":"2023-10-26T05:47:23.301108+0000","last_clean":"2023-10-26T05:47:23.301108+0000","last_became_active":"2023-10-26T05:47:21.093612+0000","last_became_peered":"2023-10-26T05:47:21.093612+0000","last_unstale":"2023-10-26T05:47:23.301108+0000","last_undegraded":"2023-10-26T05:47:23.301108+0000","last_fullsized":"2023-10-26T05:47:23.301108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039976+0000","last_change":"2023-10-26T05:47:22.060460+0000","last_active":"2023-10-26T05:47:23.039976+0000","last_peered":"2023-10-26T05:47:23.039976+0000","last_clean":"2023-10-26T05:47:23.039976+0000","last_became_active":"2023-10-26T05:47:22.059831+0000","last_became_peered":"2023-10-26T05:47:22.059831+0000","last_unstale":"2023-10-26T05:47:23.039976+0000","last_undegraded":"2023-10-26T05:47:23.039976+0000","last_fullsized":"2023-10-26T05:47:23.039976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033723+0000","last_change":"2023-10-26T05:47:21.071567+0000","last_active":"2023-10-26T05:47:23.033723+0000","last_peered":"2023-10-26T05:47:23.033723+0000","last_clean":"2023-10-26T05:47:23.033723+0000","last_became_active":"2023-10-26T05:47:21.071275+0000","last_became_peered":"2023-10-26T05:47:21.071275+0000","last_unstale":"2023-10-26T05:47:23.033723+0000","last_undegraded":"2023-10-26T05:47:23.033723+0000","last_fullsized":"2023-10-26T05:47:23.033723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297152+0000","last_change":"2023-10-26T05:47:22.063236+0000","last_active":"2023-10-26T05:47:23.297152+0000","last_peered":"2023-10-26T05:47:23.297152+0000","last_clean":"2023-10-26T05:47:23.297152+0000","last_became_active":"2023-10-26T05:47:22.061746+0000","last_became_peered":"2023-10-26T05:47:22.061746+0000","last_unstale":"2023-10-26T05:47:23.297152+0000","last_undegraded":"2023-10-26T05:47:23.297152+0000","last_fullsized":"2023-10-26T05:47:23.297152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:21.099849+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:21.097871+0000","last_became_peered":"2023-10-26T05:47:21.097871+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032981+0000","last_change":"2023-10-26T05:47:22.050271+0000","last_active":"2023-10-26T05:47:23.032981+0000","last_peered":"2023-10-26T05:47:23.032981+0000","last_clean":"2023-10-26T05:47:23.032981+0000","last_became_active":"2023-10-26T05:47:22.049654+0000","last_became_peered":"2023-10-26T05:47:22.049654+0000","last_unstale":"2023-10-26T05:47:23.032981+0000","last_undegraded":"2023-10-26T05:47:23.032981+0000","last_fullsized":"2023-10-26T05:47:23.032981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:24.404662+0000","last_change":"2023-10-26T05:47:21.078711+0000","last_active":"2023-10-26T05:47:24.404662+0000","last_peered":"2023-10-26T05:47:24.404662+0000","last_clean":"2023-10-26T05:47:24.404662+0000","last_became_active":"2023-10-26T05:47:21.078403+0000","last_became_peered":"2023-10-26T05:47:21.078403+0000","last_unstale":"2023-10-26T05:47:24.404662+0000","last_undegraded":"2023-10-26T05:47:24.404662+0000","last_fullsized":"2023-10-26T05:47:24.404662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290995+0000","last_change":"2023-10-26T05:47:22.065870+0000","last_active":"2023-10-26T05:47:23.290995+0000","last_peered":"2023-10-26T05:47:23.290995+0000","last_clean":"2023-10-26T05:47:23.290995+0000","last_became_active":"2023-10-26T05:47:22.065412+0000","last_became_peered":"2023-10-26T05:47:22.065412+0000","last_unstale":"2023-10-26T05:47:23.290995+0000","last_undegraded":"2023-10-26T05:47:23.290995+0000","last_fullsized":"2023-10-26T05:47:23.290995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299744+0000","last_change":"2023-10-26T05:47:21.094764+0000","last_active":"2023-10-26T05:47:23.299744+0000","last_peered":"2023-10-26T05:47:23.299744+0000","last_clean":"2023-10-26T05:47:23.299744+0000","last_became_active":"2023-10-26T05:47:21.091991+0000","last_became_peered":"2023-10-26T05:47:21.091991+0000","last_unstale":"2023-10-26T05:47:23.299744+0000","last_undegraded":"2023-10-26T05:47:23.299744+0000","last_fullsized":"2023-10-26T05:47:23.299744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596211+0000","last_change":"2023-10-26T05:47:22.059433+0000","last_active":"2023-10-26T05:47:23.596211+0000","last_peered":"2023-10-26T05:47:23.596211+0000","last_clean":"2023-10-26T05:47:23.596211+0000","last_became_active":"2023-10-26T05:47:22.054466+0000","last_became_peered":"2023-10-26T05:47:22.054466+0000","last_unstale":"2023-10-26T05:47:23.596211+0000","last_undegraded":"2023-10-26T05:47:23.596211+0000","last_fullsized":"2023-10-26T05:47:23.596211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298877+0000","last_change":"2023-10-26T05:47:21.099000+0000","last_active":"2023-10-26T05:47:23.298877+0000","last_peered":"2023-10-26T05:47:23.298877+0000","last_clean":"2023-10-26T05:47:23.298877+0000","last_became_active":"2023-10-26T05:47:21.095552+0000","last_became_peered":"2023-10-26T05:47:21.095552+0000","last_unstale":"2023-10-26T05:47:23.298877+0000","last_undegraded":"2023-10-26T05:47:23.298877+0000","last_fullsized":"2023-10-26T05:47:23.298877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:49.348590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040667+0000","last_change":"2023-10-26T05:47:22.059642+0000","last_active":"2023-10-26T05:47:23.040667+0000","last_peered":"2023-10-26T05:47:23.040667+0000","last_clean":"2023-10-26T05:47:23.040667+0000","last_became_active":"2023-10-26T05:47:22.054716+0000","last_became_peered":"2023-10-26T05:47:22.054716+0000","last_unstale":"2023-10-26T05:47:23.040667+0000","last_undegraded":"2023-10-26T05:47:23.040667+0000","last_fullsized":"2023-10-26T05:47:23.040667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:10.310768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033526+0000","last_change":"2023-10-26T05:47:21.090099+0000","last_active":"2023-10-26T05:47:23.033526+0000","last_peered":"2023-10-26T05:47:23.033526+0000","last_clean":"2023-10-26T05:47:23.033526+0000","last_became_active":"2023-10-26T05:47:21.089837+0000","last_became_peered":"2023-10-26T05:47:21.089837+0000","last_unstale":"2023-10-26T05:47:23.033526+0000","last_undegraded":"2023-10-26T05:47:23.033526+0000","last_fullsized":"2023-10-26T05:47:23.033526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290991+0000","last_change":"2023-10-26T05:47:22.046792+0000","last_active":"2023-10-26T05:47:23.290991+0000","last_peered":"2023-10-26T05:47:23.290991+0000","last_clean":"2023-10-26T05:47:23.290991+0000","last_became_active":"2023-10-26T05:47:22.045151+0000","last_became_peered":"2023-10-26T05:47:22.045151+0000","last_unstale":"2023-10-26T05:47:23.290991+0000","last_undegraded":"2023-10-26T05:47:23.290991+0000","last_fullsized":"2023-10-26T05:47:23.290991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032792+0000","last_change":"2023-10-26T05:47:21.072214+0000","last_active":"2023-10-26T05:47:23.032792+0000","last_peered":"2023-10-26T05:47:23.032792+0000","last_clean":"2023-10-26T05:47:23.032792+0000","last_became_active":"2023-10-26T05:47:21.071876+0000","last_became_peered":"2023-10-26T05:47:21.071876+0000","last_unstale":"2023-10-26T05:47:23.032792+0000","last_undegraded":"2023-10-26T05:47:23.032792+0000","last_fullsized":"2023-10-26T05:47:23.032792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291391+0000","last_change":"2023-10-26T05:47:22.066034+0000","last_active":"2023-10-26T05:47:23.291391+0000","last_peered":"2023-10-26T05:47:23.291391+0000","last_clean":"2023-10-26T05:47:23.291391+0000","last_became_active":"2023-10-26T05:47:22.064885+0000","last_became_peered":"2023-10-26T05:47:22.064885+0000","last_unstale":"2023-10-26T05:47:23.291391+0000","last_undegraded":"2023-10-26T05:47:23.291391+0000","last_fullsized":"2023-10-26T05:47:23.291391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299704+0000","last_change":"2023-10-26T05:47:21.093006+0000","last_active":"2023-10-26T05:47:23.299704+0000","last_peered":"2023-10-26T05:47:23.299704+0000","last_clean":"2023-10-26T05:47:23.299704+0000","last_became_active":"2023-10-26T05:47:21.092342+0000","last_became_peered":"2023-10-26T05:47:21.092342+0000","last_unstale":"2023-10-26T05:47:23.299704+0000","last_undegraded":"2023-10-26T05:47:23.299704+0000","last_fullsized":"2023-10-26T05:47:23.299704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:12.988430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296978+0000","last_change":"2023-10-26T05:47:22.059661+0000","last_active":"2023-10-26T05:47:23.296978+0000","last_peered":"2023-10-26T05:47:23.296978+0000","last_clean":"2023-10-26T05:47:23.296978+0000","last_became_active":"2023-10-26T05:47:22.058379+0000","last_became_peered":"2023-10-26T05:47:22.058379+0000","last_unstale":"2023-10-26T05:47:23.296978+0000","last_undegraded":"2023-10-26T05:47:23.296978+0000","last_fullsized":"2023-10-26T05:47:23.296978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297352+0000","last_change":"2023-10-26T05:47:22.045989+0000","last_active":"2023-10-26T05:47:23.297352+0000","last_peered":"2023-10-26T05:47:23.297352+0000","last_clean":"2023-10-26T05:47:23.297352+0000","last_became_active":"2023-10-26T05:47:22.044537+0000","last_became_peered":"2023-10-26T05:47:22.044537+0000","last_unstale":"2023-10-26T05:47:23.297352+0000","last_undegraded":"2023-10-26T05:47:23.297352+0000","last_fullsized":"2023-10-26T05:47:23.297352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298638+0000","last_change":"2023-10-26T05:47:21.099538+0000","last_active":"2023-10-26T05:47:23.298638+0000","last_peered":"2023-10-26T05:47:23.298638+0000","last_clean":"2023-10-26T05:47:23.298638+0000","last_became_active":"2023-10-26T05:47:21.096351+0000","last_became_peered":"2023-10-26T05:47:21.096351+0000","last_unstale":"2023-10-26T05:47:23.298638+0000","last_undegraded":"2023-10-26T05:47:23.298638+0000","last_fullsized":"2023-10-26T05:47:23.298638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:21.625489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297831+0000","last_change":"2023-10-26T05:47:22.059627+0000","last_active":"2023-10-26T05:47:23.297831+0000","last_peered":"2023-10-26T05:47:23.297831+0000","last_clean":"2023-10-26T05:47:23.297831+0000","last_became_active":"2023-10-26T05:47:22.054248+0000","last_became_peered":"2023-10-26T05:47:22.054248+0000","last_unstale":"2023-10-26T05:47:23.297831+0000","last_undegraded":"2023-10-26T05:47:23.297831+0000","last_fullsized":"2023-10-26T05:47:23.297831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300077+0000","last_change":"2023-10-26T05:47:21.097760+0000","last_active":"2023-10-26T05:47:23.300077+0000","last_peered":"2023-10-26T05:47:23.300077+0000","last_clean":"2023-10-26T05:47:23.300077+0000","last_became_active":"2023-10-26T05:47:21.093680+0000","last_became_peered":"2023-10-26T05:47:21.093680+0000","last_unstale":"2023-10-26T05:47:23.300077+0000","last_undegraded":"2023-10-26T05:47:23.300077+0000","last_fullsized":"2023-10-26T05:47:23.300077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596384+0000","last_change":"2023-10-26T05:47:22.038934+0000","last_active":"2023-10-26T05:47:23.596384+0000","last_peered":"2023-10-26T05:47:23.596384+0000","last_clean":"2023-10-26T05:47:23.596384+0000","last_became_active":"2023-10-26T05:47:22.038330+0000","last_became_peered":"2023-10-26T05:47:22.038330+0000","last_unstale":"2023-10-26T05:47:23.596384+0000","last_undegraded":"2023-10-26T05:47:23.596384+0000","last_fullsized":"2023-10-26T05:47:23.596384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290969+0000","last_change":"2023-10-26T05:47:21.070756+0000","last_active":"2023-10-26T05:47:23.290969+0000","last_peered":"2023-10-26T05:47:23.290969+0000","last_clean":"2023-10-26T05:47:23.290969+0000","last_became_active":"2023-10-26T05:47:21.070211+0000","last_became_peered":"2023-10-26T05:47:21.070211+0000","last_unstale":"2023-10-26T05:47:23.290969+0000","last_undegraded":"2023-10-26T05:47:23.290969+0000","last_fullsized":"2023-10-26T05:47:23.290969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290817+0000","last_change":"2023-10-26T05:47:22.062259+0000","last_active":"2023-10-26T05:47:23.290817+0000","last_peered":"2023-10-26T05:47:23.290817+0000","last_clean":"2023-10-26T05:47:23.290817+0000","last_became_active":"2023-10-26T05:47:22.055535+0000","last_became_peered":"2023-10-26T05:47:22.055535+0000","last_unstale":"2023-10-26T05:47:23.290817+0000","last_undegraded":"2023-10-26T05:47:23.290817+0000","last_fullsized":"2023-10-26T05:47:23.290817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596691+0000","last_change":"2023-10-26T05:47:21.084394+0000","last_active":"2023-10-26T05:47:23.596691+0000","last_peered":"2023-10-26T05:47:23.596691+0000","last_clean":"2023-10-26T05:47:23.596691+0000","last_became_active":"2023-10-26T05:47:21.083775+0000","last_became_peered":"2023-10-26T05:47:21.083775+0000","last_unstale":"2023-10-26T05:47:23.596691+0000","last_undegraded":"2023-10-26T05:47:23.596691+0000","last_fullsized":"2023-10-26T05:47:23.596691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.325}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.163}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.23}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:29.809 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:47:29.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2023-10-26T05:47:29.810 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-26T05:47:29.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 -- 172.21.15.71:0/1839818196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413dd20 msgr2=0x7ff48406f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 --2- 172.21.15.71:0/1839818196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413dd20 0x7ff48406f8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009bc0 tx=0x7ff47c01cb50 comp rx=0 tx=0).stop 2023-10-26T05:47:29.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 -- 172.21.15.71:0/1839818196 shutdown_connections 2023-10-26T05:47:29.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 --2- 172.21.15.71:0/1839818196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413dd20 0x7ff48406f8b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 --2- 172.21.15.71:0/1839818196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4841590b0 0x7ff48413d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 -- 172.21.15.71:0/1839818196 >> 172.21.15.71:0/1839818196 conn(0x7ff48406b8c0 msgr2=0x7ff48406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:29.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 -- 172.21.15.71:0/1839818196 shutdown_connections 2023-10-26T05:47:29.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.956+0000 7ff48bced640 1 -- 172.21.15.71:0/1839818196 wait complete. 2023-10-26T05:47:29.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 Processor -- start 2023-10-26T05:47:29.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- start start 2023-10-26T05:47:29.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 0x7ff48415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff48413dd20 0x7ff484136340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4841590b0 0x7ff484136880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48414fc00 con 0x7ff48413b0f0 2023-10-26T05:47:29.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415c150 con 0x7ff4841590b0 2023-10-26T05:47:29.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48415c680 con 0x7ff48413dd20 2023-10-26T05:47:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 0x7ff48415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 0x7ff48415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53166/0 (socket says 172.21.15.71:53166) 2023-10-26T05:47:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 -- 172.21.15.71:0/2556343678 learned_addr learned my addr 172.21.15.71:0/2556343678 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48a28a640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4841590b0 0x7ff484136880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 -- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff48413dd20 msgr2=0x7ff484136340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff48413dd20 0x7ff484136340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 -- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4841590b0 msgr2=0x7ff484136880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:29.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4841590b0 0x7ff484136880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:29.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 -- 172.21.15.71:0/2556343678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c009820 con 0x7ff48413b0f0 2023-10-26T05:47:29.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff489a89640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 0x7ff48415b380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c460 tx=0x7ff48000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480004230 con 0x7ff48413b0f0 2023-10-26T05:47:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4800044a0 con 0x7ff48413b0f0 2023-10-26T05:47:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48415b8c0 con 0x7ff48413b0f0 2023-10-26T05:47:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480003860 con 0x7ff48413b0f0 2023-10-26T05:47:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48414f890 con 0x7ff48413b0f0 2023-10-26T05:47:29.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff484066b10 con 0x7ff48413b0f0 2023-10-26T05:47:29.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.960+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4800039c0 con 0x7ff48413b0f0 2023-10-26T05:47:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.964+0000 7ff47a7fc640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff4680664a0 0x7ff468068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.964+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4800836d0 con 0x7ff48413b0f0 2023-10-26T05:47:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.964+0000 7ff489288640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff4680664a0 0x7ff468068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.964+0000 7ff489288640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff4680664a0 0x7ff468068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff48406bdf0 tx=0x7ff474005b80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:29.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:29.968+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff4800546a0 con 0x7ff48413b0f0 2023-10-26T05:47:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.124+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7ff48406c7d0 con 0x7ff48413b0f0 2023-10-26T05:47:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff47a7fc640 1 -- 172.21.15.71:0/2556343678 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v18) v1 ==== 86+0+146 (secure 0 0 0) 0x7ff480017090 con 0x7ff48413b0f0 2023-10-26T05:47:30.133 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:30.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff4680664a0 msgr2=0x7ff468068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff4680664a0 0x7ff468068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff48406bdf0 tx=0x7ff474005b80 comp rx=0 tx=0).stop 2023-10-26T05:47:30.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 msgr2=0x7ff48415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 0x7ff48415b380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c460 tx=0x7ff48000c920 comp rx=0 tx=0).stop 2023-10-26T05:47:30.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 shutdown_connections 2023-10-26T05:47:30.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4841590b0 0x7ff484136880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff48413dd20 0x7ff484136340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff4680664a0 0x7ff468068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 --2- 172.21.15.71:0/2556343678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff48413b0f0 0x7ff48415b380 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 >> 172.21.15.71:0/2556343678 conn(0x7ff48406b8c0 msgr2=0x7ff4840b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:30.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 shutdown_connections 2023-10-26T05:47:30.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.128+0000 7ff48bced640 1 -- 172.21.15.71:0/2556343678 wait complete. 2023-10-26T05:47:30.150 INFO:teuthology.orchestra.run.smithi071.stdout:[{"version":18,"timestamp":"2023-10-26T05:47:03.897577+0000","name":"","changes":[{"name":"osd/osd_op_complaint_time","new_value":"180.000000"}]}] 2023-10-26T05:47:30.151 INFO:tasks.ceph_manager:config epoch is 18 2023-10-26T05:47:30.155 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-26T05:47:30.155 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-26T05:47:30.156 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-26T05:47:30.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/2164566580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 msgr2=0x7f4b1806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/2164566580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b1806e410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4b180cf020 tx=0x7f4b0c01c550 comp rx=0 tx=0).stop 2023-10-26T05:47:30.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/2164566580 shutdown_connections 2023-10-26T05:47:30.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/2164566580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b1806e950 0x7f4b1815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/2164566580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b1806e410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/2164566580 >> v2:172.21.15.86:3300/0 conn(0x7f4b18143e40 0x7f4b180cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/2164566580 >> 172.21.15.71:0/2164566580 conn(0x7f4b180afeb0 msgr2=0x7f4b180b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:30.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/2164566580 shutdown_connections 2023-10-26T05:47:30.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/2164566580 wait complete. 2023-10-26T05:47:30.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 Processor -- start 2023-10-26T05:47:30.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- start start 2023-10-26T05:47:30.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1806e950 0x7f4b181579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b18143e40 0x7f4b18154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b18139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18139b90 con 0x7f4b18144600 2023-10-26T05:47:30.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18158080 con 0x7f4b1806e950 2023-10-26T05:47:30.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18158360 con 0x7f4b18143e40 2023-10-26T05:47:30.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b18139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b18139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53186/0 (socket says 172.21.15.71:53186) 2023-10-26T05:47:30.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 -- 172.21.15.71:0/1630844907 learned_addr learned my addr 172.21.15.71:0/1630844907 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:30.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1d597640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1806e950 0x7f4b181579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1cd96640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b18143e40 0x7f4b18154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 -- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b18143e40 msgr2=0x7f4b18154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b18143e40 0x7f4b18154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 -- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1806e950 msgr2=0x7f4b181579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1806e950 0x7f4b181579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 -- 172.21.15.71:0/1630844907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c009560 con 0x7f4b18144600 2023-10-26T05:47:30.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1d597640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1806e950 0x7f4b181579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:30.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1dd98640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b18139170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000d750 tx=0x7f4b1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:30.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10009c30 con 0x7f4b18144600 2023-10-26T05:47:30.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b10009e80 con 0x7f4b18144600 2023-10-26T05:47:30.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b18158640 con 0x7f4b18144600 2023-10-26T05:47:30.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10005280 con 0x7f4b18144600 2023-10-26T05:47:30.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.260+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18155db0 con 0x7f4b18144600 2023-10-26T05:47:30.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.264+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b18066d00 con 0x7f4b18144600 2023-10-26T05:47:30.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.264+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b100053e0 con 0x7f4b18144600 2023-10-26T05:47:30.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.264+0000 7f4b15ffb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b04066470 0x7f4b04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.264+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b10084050 con 0x7f4b18144600 2023-10-26T05:47:30.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.264+0000 7f4b1d597640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b04066470 0x7f4b04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.264+0000 7f4b1d597640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b04066470 0x7f4b04068930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b1813ab90 tx=0x7f4b080093c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:30.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.268+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4b100514d0 con 0x7f4b18144600 2023-10-26T05:47:30.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.444+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4b180ada30 con 0x7f4b18144600 2023-10-26T05:47:30.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.452+0000 7f4b15ffb640 1 -- 172.21.15.71:0/1630844907 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164429 (secure 0 0 0) 0x7f4b10017090 con 0x7f4b18144600 2023-10-26T05:47:30.461 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:30.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b04066470 msgr2=0x7f4b04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b04066470 0x7f4b04068930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b1813ab90 tx=0x7f4b080093c0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 msgr2=0x7f4b18139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b18139170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000d750 tx=0x7f4b1000dc10 comp rx=0 tx=0).stop 2023-10-26T05:47:30.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 shutdown_connections 2023-10-26T05:47:30.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b04066470 0x7f4b04068930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b18144600 0x7f4b18139170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b18143e40 0x7f4b18154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 --2- 172.21.15.71:0/1630844907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b1806e950 0x7f4b181579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 >> 172.21.15.71:0/1630844907 conn(0x7f4b180afeb0 msgr2=0x7f4b1815de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:30.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 shutdown_connections 2023-10-26T05:47:30.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.456+0000 7f4b1f7fb640 1 -- 172.21.15.71:0/1630844907 wait complete. 2023-10-26T05:47:30.478 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":5,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1120505646},{"type":"v1","addr":"172.21.15.86:6833","nonce":1120505646}]},"active_addr":"172.21.15.86:6833/1120505646","active_change":"2023-10-26T05:47:07.381249+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4158,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3646245716}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":718546707}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2862200710}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2226013227}]}]} 2023-10-26T05:47:30.483 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-26T05:47:30.483 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-26T05:47:30.483 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:30.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91e59c640 1 -- 172.21.15.71:0/794344879 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100a6d30 con 0x7fc92013c2f0 2023-10-26T05:47:30.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- 172.21.15.71:0/794344879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92013c2f0 msgr2=0x7fc9200b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 --2- 172.21.15.71:0/794344879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92013c2f0 0x7fc9200b2350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc910009840 tx=0x7fc91001c800 comp rx=0 tx=0).stop 2023-10-26T05:47:30.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- 172.21.15.71:0/794344879 shutdown_connections 2023-10-26T05:47:30.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 --2- 172.21.15.71:0/794344879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9200cb1e0 0x7fc920159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 --2- 172.21.15.71:0/794344879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc92013c2f0 0x7fc9200b2350 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- 172.21.15.71:0/794344879 >> 172.21.15.71:0/794344879 conn(0x7fc920069770 msgr2=0x7fc920069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:30.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- 172.21.15.71:0/794344879 shutdown_connections 2023-10-26T05:47:30.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- 172.21.15.71:0/794344879 wait complete. 2023-10-26T05:47:30.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 Processor -- start 2023-10-26T05:47:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- start start 2023-10-26T05:47:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 0x7fc9201622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200cb1e0 0x7fc920162830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92013c2f0 0x7fc9200b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920155b80 con 0x7fc9200b2890 2023-10-26T05:47:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920154f70 con 0x7fc9200cb1e0 2023-10-26T05:47:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc92518d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc920155250 con 0x7fc92013c2f0 2023-10-26T05:47:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 0x7fc9201622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 0x7fc9201622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53192/0 (socket says 172.21.15.71:53192) 2023-10-26T05:47:30.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 -- 172.21.15.71:0/3692724919 learned_addr learned my addr 172.21.15.71:0/3692724919 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:30.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc917fff640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200cb1e0 0x7fc920162830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91f59e640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92013c2f0 0x7fc9200b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 -- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92013c2f0 msgr2=0x7fc9200b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92013c2f0 0x7fc9200b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 -- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200cb1e0 msgr2=0x7fc920162830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200cb1e0 0x7fc920162830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc91ed9d640 1 -- 172.21.15.71:0/3692724919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100094a0 con 0x7fc9200b2890 2023-10-26T05:47:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.588+0000 7fc917fff640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200cb1e0 0x7fc920162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc91ed9d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 0x7fc9201622f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc91001d0a0 tx=0x7fc9100a6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910004240 con 0x7fc9200b2890 2023-10-26T05:47:30.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc910004490 con 0x7fc9200b2890 2023-10-26T05:47:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc920155530 con 0x7fc9200b2890 2023-10-26T05:47:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910005280 con 0x7fc9200b2890 2023-10-26T05:47:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9200b0da0 con 0x7fc9200b2890 2023-10-26T05:47:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920066b10 con 0x7fc9200b2890 2023-10-26T05:47:30.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc9100053e0 con 0x7fc9200b2890 2023-10-26T05:47:30.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc9177fe640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc9000664c0 0x7fc900068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9100aa070 con 0x7fc9200b2890 2023-10-26T05:47:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc917fff640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc9000664c0 0x7fc900068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.592+0000 7fc917fff640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc9000664c0 0x7fc900068980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9200ae190 tx=0x7fc908009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.596+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc9100ebba0 con 0x7fc9200b2890 2023-10-26T05:47:30.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.736+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc920155390 con 0x7fc9200b2890 2023-10-26T05:47:30.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.736+0000 7fc9177fe640 1 -- 172.21.15.71:0/3692724919 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fc920155390 con 0x7fc9200b2890 2023-10-26T05:47:30.744 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:30.745 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:30.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc9000664c0 msgr2=0x7fc900068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc9000664c0 0x7fc900068980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9200ae190 tx=0x7fc908009040 comp rx=0 tx=0).stop 2023-10-26T05:47:30.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 msgr2=0x7fc9201622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 0x7fc9201622f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc91001d0a0 tx=0x7fc9100a6550 comp rx=0 tx=0).stop 2023-10-26T05:47:30.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 shutdown_connections 2023-10-26T05:47:30.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc92013c2f0 0x7fc9200b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9200cb1e0 0x7fc920162830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc9000664c0 0x7fc900068980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 --2- 172.21.15.71:0/3692724919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9200b2890 0x7fc9201622f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 >> 172.21.15.71:0/3692724919 conn(0x7fc920069770 msgr2=0x7fc9200ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:30.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 shutdown_connections 2023-10-26T05:47:30.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.740+0000 7fc92518d640 1 -- 172.21.15.71:0/3692724919 wait complete. 2023-10-26T05:47:30.761 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-26T05:47:30.761 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- 172.21.15.71:0/3858016771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c003ef0 msgr2=0x7f623c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 --2- 172.21.15.71:0/3858016771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f623c003ef0 0x7f623c0047d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6248006fc0 tx=0x7f624801cf80 comp rx=0 tx=0).stop 2023-10-26T05:47:30.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 -- 172.21.15.71:0/3858016771 reap_dead start 2023-10-26T05:47:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- 172.21.15.71:0/3858016771 shutdown_connections 2023-10-26T05:47:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- 172.21.15.71:0/3858016771 >> 172.21.15.71:0/3858016771 conn(0x7f625406fdc0 msgr2=0x7f62540b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- 172.21.15.71:0/3858016771 shutdown_connections 2023-10-26T05:47:30.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- 172.21.15.71:0/3858016771 wait complete. 2023-10-26T05:47:30.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 Processor -- start 2023-10-26T05:47:30.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- start start 2023-10-26T05:47:30.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c003ef0 0x7f62540c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62540c2d30 0x7f62541e3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 0x7f62540c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625415d6f0 con 0x7f62540afdd0 2023-10-26T05:47:30.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c32d0 con 0x7f623c003ef0 2023-10-26T05:47:30.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6258804640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c53b0 con 0x7f62540c2d30 2023-10-26T05:47:30.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 0x7f62540c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 0x7f62540c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53210/0 (socket says 172.21.15.71:53210) 2023-10-26T05:47:30.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 -- 172.21.15.71:0/2090872078 learned_addr learned my addr 172.21.15.71:0/2090872078 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:30.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f625259c640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c003ef0 0x7f62540c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6251d9b640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62540c2d30 0x7f62541e3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 -- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62540c2d30 msgr2=0x7f62541e3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62540c2d30 0x7f62541e3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 -- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c003ef0 msgr2=0x7f62540c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:30.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c003ef0 0x7f62540c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:30.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 -- 172.21.15.71:0/2090872078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540c5690 con 0x7f62540afdd0 2023-10-26T05:47:30.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.880+0000 7f6252d9d640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 0x7f62540c7250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f62540c6180 tx=0x7f624c00fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:30.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0118a0 con 0x7f62540afdd0 2023-10-26T05:47:30.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f624c011af0 con 0x7f62540afdd0 2023-10-26T05:47:30.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62540c5970 con 0x7f62540afdd0 2023-10-26T05:47:30.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c012470 con 0x7f62540afdd0 2023-10-26T05:47:30.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541e4350 con 0x7f62540afdd0 2023-10-26T05:47:30.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254066d00 con 0x7f62540afdd0 2023-10-26T05:47:30.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f624c0125d0 con 0x7f62540afdd0 2023-10-26T05:47:30.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6242ffd640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6234066470 0x7f6234068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:30.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f624c0870f0 con 0x7f62540afdd0 2023-10-26T05:47:30.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f625259c640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6234066470 0x7f6234068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:30.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.884+0000 7f625259c640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6234066470 0x7f6234068930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6244005b50 tx=0x7f6244005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:30.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:30.888+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f624c054570 con 0x7f62540afdd0 2023-10-26T05:47:31.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.028+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f625415a5e0 con 0x7f62540afdd0 2023-10-26T05:47:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.032+0000 7f6242ffd640 1 -- 172.21.15.71:0/2090872078 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f624c057b90 con 0x7f62540afdd0 2023-10-26T05:47:31.039 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:31.039 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:31.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.032+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6234066470 msgr2=0x7f6234068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.032+0000 7f6258804640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6234066470 0x7f6234068930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6244005b50 tx=0x7f6244005be0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 msgr2=0x7f62540c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 0x7f62540c7250 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f62540c6180 tx=0x7f624c00fa30 comp rx=0 tx=0).stop 2023-10-26T05:47:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 shutdown_connections 2023-10-26T05:47:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6234066470 0x7f6234068930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62540afdd0 0x7f62540c7250 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62540c2d30 0x7f62541e3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 --2- 172.21.15.71:0/2090872078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f623c003ef0 0x7f62540c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 >> 172.21.15.71:0/2090872078 conn(0x7f625406fdc0 msgr2=0x7f6254157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 shutdown_connections 2023-10-26T05:47:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.036+0000 7f6258804640 1 -- 172.21.15.71:0/2090872078 wait complete. 2023-10-26T05:47:31.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-26T05:47:31.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-26T05:47:31.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-26T05:47:31.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-26T05:47:31.058 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-26T05:47:31.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-26T05:47:31.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-26T05:47:31.060 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-26T05:47:31.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 -- 172.21.15.71:0/1483356514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85680b1a90 msgr2=0x7f856806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 --2- 172.21.15.71:0/1483356514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85680b1a90 0x7f856806d920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85680ccc80 tx=0x7f855801c830 comp rx=0 tx=0).stop 2023-10-26T05:47:31.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 -- 172.21.15.71:0/1483356514 shutdown_connections 2023-10-26T05:47:31.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 --2- 172.21.15.71:0/1483356514 >> v2:172.21.15.86:3300/0 conn(0x7f856815bf20 0x7f85680afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 --2- 172.21.15.71:0/1483356514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85680b1a90 0x7f856806d920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 -- 172.21.15.71:0/1483356514 >> 172.21.15.71:0/1483356514 conn(0x7f856806a6b0 msgr2=0x7f856806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 -- 172.21.15.71:0/1483356514 shutdown_connections 2023-10-26T05:47:31.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 -- 172.21.15.71:0/1483356514 wait complete. 2023-10-26T05:47:31.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 Processor -- start 2023-10-26T05:47:31.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.228+0000 7f856ed16640 1 -- start start 2023-10-26T05:47:31.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85680b1a90 0x7f85680ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85680b1e70 0x7f8568168990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f856815bf20 0x7f856815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856815a500 con 0x7f856815bf20 2023-10-26T05:47:31.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856cab2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85680b1a90 0x7f85680ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f856815bf20 0x7f856815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f856815bf20 0x7f856815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53234/0 (socket says 172.21.15.71:53234) 2023-10-26T05:47:31.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 -- 172.21.15.71:0/338098506 learned_addr learned my addr 172.21.15.71:0/338098506 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85680c95f0 con 0x7f85680b1a90 2023-10-26T05:47:31.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8568159930 con 0x7f85680b1e70 2023-10-26T05:47:31.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 -- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85680b1e70 msgr2=0x7f8568168990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-26T05:47:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85680b1e70 0x7f8568168990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 -- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85680b1a90 msgr2=0x7f85680ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85680b1a90 0x7f85680ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8558009560 con 0x7f856815bf20 2023-10-26T05:47:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856d2b3640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f856815bf20 0x7f856815a990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f855c00d660 tx=0x7f855c00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c005220 con 0x7f856815bf20 2023-10-26T05:47:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8568159bd0 con 0x7f856815bf20 2023-10-26T05:47:31.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856815a190 con 0x7f856815bf20 2023-10-26T05:47:31.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 -- 172.21.15.71:0/4158246551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75280535f0 msgr2=0x7f75280541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 --2- 172.21.15.71:0/4158246551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75280535f0 0x7f75280541d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f753002caf0 tx=0x7f7530097490 comp rx=0 tx=0).stop 2023-10-26T05:47:31.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 -- 172.21.15.71:0/4158246551 shutdown_connections 2023-10-26T05:47:31.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 --2- 172.21.15.71:0/4158246551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75280535f0 0x7f75280541d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 --2- 172.21.15.71:0/4158246551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7528006e60 0x7f7528053c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 -- 172.21.15.71:0/4158246551 >> 172.21.15.71:0/4158246551 conn(0x7f7528009f90 msgr2=0x7f752800a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 -- 172.21.15.71:0/4158246551 shutdown_connections 2023-10-26T05:47:31.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.232+0000 7f753775a640 1 -- 172.21.15.71:0/4158246551 wait complete. 2023-10-26T05:47:31.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f856804da80 con 0x7f856815bf20 2023-10-26T05:47:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- 172.21.15.71:0/317872876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c06c320 msgr2=0x7f073c06c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- 172.21.15.71:0/317872876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c06c320 0x7f073c06c790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f073c06b320 tx=0x7f073401cb30 comp rx=0 tx=0).stop 2023-10-26T05:47:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- 172.21.15.71:0/317872876 shutdown_connections 2023-10-26T05:47:31.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- 172.21.15.71:0/317872876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c06c320 0x7f073c06c790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- 172.21.15.71:0/317872876 >> v2:172.21.15.86:3300/0 conn(0x7f073c070fb0 0x7f073c0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- 172.21.15.71:0/317872876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c157670 0x7f073c06fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- 172.21.15.71:0/317872876 >> 172.21.15.71:0/317872876 conn(0x7f073c069770 msgr2=0x7f073c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- 172.21.15.71:0/317872876 shutdown_connections 2023-10-26T05:47:31.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- 172.21.15.71:0/317872876 wait complete. 2023-10-26T05:47:31.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 Processor -- start 2023-10-26T05:47:31.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f855c005490 con 0x7f856815bf20 2023-10-26T05:47:31.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 -- start start 2023-10-26T05:47:31.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 -- 172.21.15.71:0/1482509836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c070b90 msgr2=0x7ff02c0cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 --2- 172.21.15.71:0/1482509836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c070b90 0x7ff02c0cba70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff02c065e40 tx=0x7ff02801c880 comp rx=0 tx=0).stop 2023-10-26T05:47:31.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 Processor -- start 2023-10-26T05:47:31.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 -- 172.21.15.71:0/1482509836 shutdown_connections 2023-10-26T05:47:31.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 --2- 172.21.15.71:0/1482509836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c070b90 0x7ff02c0cba70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 --2- 172.21.15.71:0/1482509836 >> v2:172.21.15.86:3300/0 conn(0x7ff02c155a40 0x7ff02c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 -- 172.21.15.71:0/1482509836 >> 172.21.15.71:0/1482509836 conn(0x7ff02c06a6b0 msgr2=0x7ff02c06aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c009a80 con 0x7f856815bf20 2023-10-26T05:47:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 -- 172.21.15.71:0/1482509836 shutdown_connections 2023-10-26T05:47:31.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528006e60 0x7f75280476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7ff033292640 1 -- 172.21.15.71:0/1482509836 wait complete. 2023-10-26T05:47:31.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75280535f0 0x7f7528048570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f75354f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528006e60 0x7f75280476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f75354f6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528006e60 0x7f75280476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:58348/0 (socket says 172.21.15.71:58348) 2023-10-26T05:47:31.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f855c009be0 con 0x7f856815bf20 2023-10-26T05:47:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7fc2dd33b640 1 -- 172.21.15.71:0/3671902592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d003ab40 msgr2=0x7fc2d0006d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7fc2dd33b640 1 --2- 172.21.15.71:0/3671902592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d003ab40 0x7fc2d0006d20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc009950 tx=0x7fc2cc01f0d0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75280df740 0x7f75280cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75280d9eb0 con 0x7f75280535f0 2023-10-26T05:47:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752804b050 con 0x7f7528006e60 2023-10-26T05:47:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f753775a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752804b330 con 0x7f75280df740 2023-10-26T05:47:31.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- start start 2023-10-26T05:47:31.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c06c320 0x7f073c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c070fb0 0x7f073c146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c157670 0x7f073c14f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1543d0 con 0x7f073c070fb0 2023-10-26T05:47:31.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.236+0000 7f0740a99640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c148f10 con 0x7f073c06c320 2023-10-26T05:47:31.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0740a99640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1491f0 con 0x7f073c157670 2023-10-26T05:47:31.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 -- 172.21.15.71:0/3671902592 shutdown_connections 2023-10-26T05:47:31.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 --2- 172.21.15.71:0/3671902592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d003ab40 0x7fc2d0006d20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 -- 172.21.15.71:0/3671902592 >> 172.21.15.71:0/3671902592 conn(0x7fc2d0054ef0 msgr2=0x7fc2d00552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75280535f0 0x7f7528048570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75280535f0 0x7f7528048570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53244/0 (socket says 172.21.15.71:53244) 2023-10-26T05:47:31.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 -- 172.21.15.71:0/1911041596 learned_addr learned my addr 172.21.15.71:0/1911041596 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c070fb0 0x7f073c146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c070fb0 0x7f073c146270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53266/0 (socket says 172.21.15.71:53266) 2023-10-26T05:47:31.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 -- 172.21.15.71:0/807091358 learned_addr learned my addr 172.21.15.71:0/807091358 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 -- 172.21.15.71:0/3671902592 shutdown_connections 2023-10-26T05:47:31.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 -- 172.21.15.71:0/3671902592 wait complete. 2023-10-26T05:47:31.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 Processor -- start 2023-10-26T05:47:31.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 -- start start 2023-10-26T05:47:31.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff02c070b90 0x7ff02c1492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c0b1090 0x7ff02c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03102e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff02c070b90 0x7ff02c1492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03102e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff02c070b90 0x7ff02c1492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42164/0 (socket says 172.21.15.71:42164) 2023-10-26T05:47:31.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02c155a40 0x7ff02c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c0b1090 0x7ff02c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c0b1090 0x7ff02c14e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53274/0 (socket says 172.21.15.71:53274) 2023-10-26T05:47:31.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03102e640 1 -- 172.21.15.71:0/1620177493 learned_addr learned my addr 172.21.15.71:0/1620177493 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c14b470 con 0x7ff02c0b1090 2023-10-26T05:47:31.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c14b0d0 con 0x7ff02c155a40 2023-10-26T05:47:31.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02c1574b0 con 0x7ff02c070b90 2023-10-26T05:47:31.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03182f640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02c155a40 0x7ff02c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 -- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff02c070b90 msgr2=0x7ff02c1492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff02c070b90 0x7ff02c1492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 -- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02c155a40 msgr2=0x7ff02c14eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff02c155a40 0x7ff02c14eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009560 con 0x7ff02c0b1090 2023-10-26T05:47:31.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f85657fa640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8554066540 0x7f8554068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f855c0843d0 con 0x7f856815bf20 2023-10-26T05:47:31.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f85657fa640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f855406e5f0 0x7f8554070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f856cab2640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8554066540 0x7f8554068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85540710c0 con 0x7f855406e5f0 2023-10-26T05:47:31.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f855c084780 con 0x7f856815bf20 2023-10-26T05:47:31.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f8567fff640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f855406e5f0 0x7f8554070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f8567fff640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f855406e5f0 0x7f8554070a10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7535cf7640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75280df740 0x7f75280cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 -- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75280df740 msgr2=0x7f75280cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75280df740 0x7f75280cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 -- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528006e60 msgr2=0x7f75280476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7528006e60 0x7f75280476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 -- 172.21.15.71:0/1911041596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753004ea10 con 0x7f75280535f0 2023-10-26T05:47:31.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7534cf5640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75280535f0 0x7f7528048570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f752000d950 tx=0x7f752000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520009c30 con 0x7f75280535f0 2023-10-26T05:47:31.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7520009e80 con 0x7f75280535f0 2023-10-26T05:47:31.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75280d9860 con 0x7f75280535f0 2023-10-26T05:47:31.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75200106a0 con 0x7f75280535f0 2023-10-26T05:47:31.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7528048ab0 con 0x7f75280535f0 2023-10-26T05:47:31.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f073a59c640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c06c320 0x7f073c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f073ad9d640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c157670 0x7f073c14f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 -- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c157670 msgr2=0x7f073c14f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c157670 0x7f073c14f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 -- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c06c320 msgr2=0x7f073c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c06c320 0x7f073c156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 -- 172.21.15.71:0/807091358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734009820 con 0x7f073c070fb0 2023-10-26T05:47:31.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f073ad9d640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c157670 0x7f073c14f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:31.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f073a59c640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c06c320 0x7f073c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:31.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0739d9b640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c070fb0 0x7f073c146270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f073000e9f0 tx=0x7f073000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073000ccc0 con 0x7f073c070fb0 2023-10-26T05:47:31.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0730010430 con 0x7f073c070fb0 2023-10-26T05:47:31.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730010800 con 0x7f073c070fb0 2023-10-26T05:47:31.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c154620 con 0x7f073c070fb0 2023-10-26T05:47:31.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7ff03082d640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c0b1090 0x7ff02c14e860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff02400bd50 tx=0x7ff02400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1467b0 con 0x7f073c070fb0 2023-10-26T05:47:31.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 Processor -- start 2023-10-26T05:47:31.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.240+0000 7fc2dd33b640 1 -- start start 2023-10-26T05:47:31.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2d0004a10 0x7fc2d00ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d003ab40 0x7fc2d00edc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d004fbc0 0x7fc2d0049840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00de380 con 0x7fc2d004fbc0 2023-10-26T05:47:31.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00ee1d0 con 0x7fc2d003ab40 2023-10-26T05:47:31.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00de0a0 con 0x7fc2d0004a10 2023-10-26T05:47:31.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400ce90 con 0x7ff02c0b1090 2023-10-26T05:47:31.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff02c157790 con 0x7ff02c0b1090 2023-10-26T05:47:31.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff02c149810 con 0x7ff02c0b1090 2023-10-26T05:47:31.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75280052a0 con 0x7f75280535f0 2023-10-26T05:47:31.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- 172.21.15.71:0/2800549131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42600e30a0 msgr2=0x7f4260005ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 --2- 172.21.15.71:0/2800549131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42600e30a0 0x7f4260005ec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4264009900 tx=0x7f426401f070 comp rx=0 tx=0).stop 2023-10-26T05:47:31.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d003ab40 0x7fc2d00edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d003ab40 0x7fc2d00edc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:58378/0 (socket says 172.21.15.71:58378) 2023-10-26T05:47:31.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 -- 172.21.15.71:0/2259356936 learned_addr learned my addr 172.21.15.71:0/2259356936 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d6ffd640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2d0004a10 0x7fc2d00ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f25ff808640 1 -- 172.21.15.71:0/2030171348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 msgr2=0x7f25f81583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f25ff808640 1 --2- 172.21.15.71:0/2030171348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f81583c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c2e0 tx=0x7f25f001de00 comp rx=0 tx=0).stop 2023-10-26T05:47:31.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- 172.21.15.71:0/2800549131 shutdown_connections 2023-10-26T05:47:31.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 --2- 172.21.15.71:0/2800549131 >> v2:172.21.15.71:3300/0 conn(0x7f426004f2d0 0x7f42600e3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 --2- 172.21.15.71:0/2800549131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42600e30a0 0x7f4260005ec0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- 172.21.15.71:0/2800549131 >> 172.21.15.71:0/2800549131 conn(0x7f4260009f90 msgr2=0x7f426000a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d77fe640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d004fbc0 0x7fc2d0049840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 -- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2d0004a10 msgr2=0x7fc2d00ef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2d0004a10 0x7fc2d00ef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 -- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d004fbc0 msgr2=0x7fc2d0049840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2d004fbc0 0x7fc2d0049840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 -- 172.21.15.71:0/2259356936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc009560 con 0x7fc2d003ab40 2023-10-26T05:47:31.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff02c164f00 con 0x7ff02c0b1090 2023-10-26T05:47:31.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- 172.21.15.71:0/2800549131 shutdown_connections 2023-10-26T05:47:31.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f856cab2640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8554066540 0x7f8554068a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85580af000 tx=0x7f855801b070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== osd.4 v2:172.21.15.86:6800/893150098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f85540710c0 con 0x7f855406e5f0 2023-10-26T05:47:31.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f073c04da80 con 0x7f073c070fb0 2023-10-26T05:47:31.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0730010960 con 0x7f073c070fb0 2023-10-26T05:47:31.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2d67fc640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d003ab40 0x7fc2d00edc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098a0 tx=0x7fc2cc005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7520010800 con 0x7f75280535f0 2023-10-26T05:47:31.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7525ffb640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7514066470 0x7f7514068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f072affd640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f071c066470 0x7f071c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0730014070 con 0x7f073c070fb0 2023-10-26T05:47:31.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- 172.21.15.71:0/2800549131 wait complete. 2023-10-26T05:47:31.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 Processor -- start 2023-10-26T05:47:31.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75200842d0 con 0x7f75280535f0 2023-10-26T05:47:31.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7525ffb640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f751406e610 0x7f7514070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 --> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75140710e0 con 0x7f751406e610 2023-10-26T05:47:31.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f75354f6640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7514066470 0x7f7514068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff024014440 con 0x7ff02c0b1090 2023-10-26T05:47:31.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7520051620 con 0x7f75280535f0 2023-10-26T05:47:31.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f072affd640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f071c06e610 0x7f071c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 --> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f071c0710e0 con 0x7f071c06e610 2023-10-26T05:47:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0730051ea0 con 0x7f073c070fb0 2023-10-26T05:47:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc0b0b80 con 0x7fc2d003ab40 2023-10-26T05:47:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f7535cf7640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f751406e610 0x7f7514070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f073a59c640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f071c066470 0x7f071c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f073ad9d640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f071c06e610 0x7f071c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d0035c60 con 0x7fc2d003ab40 2023-10-26T05:47:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d0049320 con 0x7fc2d003ab40 2023-10-26T05:47:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240148d0 con 0x7ff02c0b1090 2023-10-26T05:47:31.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc2cc0b0ce0 con 0x7fc2d003ab40 2023-10-26T05:47:31.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 -- 172.21.15.71:0/2030171348 shutdown_connections 2023-10-26T05:47:31.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 --2- 172.21.15.71:0/2030171348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f81583c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- start start 2023-10-26T05:47:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426004f2d0 0x7f42600e1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600544d0 0x7f426004d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc005280 con 0x7fc2d003ab40 2023-10-26T05:47:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff02401d3b0 con 0x7ff02c0b1090 2023-10-26T05:47:31.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42600e30a0 0x7f4260039d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600e16d0 con 0x7f42600e30a0 2023-10-26T05:47:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260043940 con 0x7f42600544d0 2023-10-26T05:47:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.244+0000 7f426e086640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4260043c20 con 0x7f426004f2d0 2023-10-26T05:47:31.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f073a59c640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f071c066470 0x7f071c068930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f072c009f30 tx=0x7f072c009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 --2- 172.21.15.71:0/2030171348 >> v2:172.21.15.71:3300/0 conn(0x7f25f8070560 0x7f25f806a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 -- 172.21.15.71:0/2030171348 >> 172.21.15.71:0/2030171348 conn(0x7f25f80aec60 msgr2=0x7f25f80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42600e30a0 0x7f4260039d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d084640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426004f2d0 0x7f42600e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42600e30a0 0x7f4260039d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53286/0 (socket says 172.21.15.71:53286) 2023-10-26T05:47:31.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d084640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426004f2d0 0x7f42600e1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42178/0 (socket says 172.21.15.71:42178) 2023-10-26T05:47:31.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 -- 172.21.15.71:0/398270931 learned_addr learned my addr 172.21.15.71:0/398270931 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f073ad9d640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f071c06e610 0x7f071c070a30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f75354f6640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7514066470 0x7f7514068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f752c0011f0 tx=0x7f752c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff021ffb640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0100665c0 0x7ff010068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff024084420 con 0x7ff02c0b1090 2023-10-26T05:47:31.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f7535cf7640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f751406e610 0x7f7514070a30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff021ffb640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7ff01006e5f0 0x7ff010070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0100710c0 con 0x7ff01006e5f0 2023-10-26T05:47:31.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 -- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426004f2d0 msgr2=0x7f42600e1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f426004f2d0 0x7f42600e1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 -- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600544d0 msgr2=0x7f426004d5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-26T05:47:31.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42600544d0 0x7f426004d5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 -- 172.21.15.71:0/398270931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4264009560 con 0x7f42600e30a0 2023-10-26T05:47:31.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426d885640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42600e30a0 0x7f4260039d30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4268066ca0 tx=0x7f42680670d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 -- 172.21.15.71:0/2030171348 shutdown_connections 2023-10-26T05:47:31.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 -- 172.21.15.71:0/2030171348 wait complete. 2023-10-26T05:47:31.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff03102e640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0100665c0 0x7ff010068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426808d070 con 0x7f42600e30a0 2023-10-26T05:47:31.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4268069120 con 0x7f42600e30a0 2023-10-26T05:47:31.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268072a90 con 0x7f42600e30a0 2023-10-26T05:47:31.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4260043f00 con 0x7f42600e30a0 2023-10-26T05:47:31.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42600444c0 con 0x7f42600e30a0 2023-10-26T05:47:31.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== osd.6 v2:172.21.15.86:6824/1847760170 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f071c0710e0 con 0x7f071c06e610 2023-10-26T05:47:31.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== osd.5 v2:172.21.15.86:6808/88051276 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f75140710e0 con 0x7f751406e610 2023-10-26T05:47:31.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7ff03102e640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0100665c0 0x7ff010068a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c00b410 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc2cc0b7070 con 0x7fc2d003ab40 2023-10-26T05:47:31.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 Processor -- start 2023-10-26T05:47:31.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f25ff808640 1 -- start start 2023-10-26T05:47:31.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4260043ab0 con 0x7f42600e30a0 2023-10-26T05:47:31.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7fc2bf7fe640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc2c4066540 0x7fc2c4068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.248+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2cc121320 con 0x7fc2d003ab40 2023-10-26T05:47:31.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7fc2d6ffd640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc2c4066540 0x7fc2c4068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7fc2d6ffd640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc2c4066540 0x7fc2c4068a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0034440 tx=0x7fc2c80090e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25ff808640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f8136fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25ff808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8070560 0x7f25f8139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25ff808640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f8140c40 0x7f25f80cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25ff808640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f813b3b0 con 0x7f25f8140c40 2023-10-26T05:47:31.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7fc2dd33b640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fc2b80015f0 0x7fc2b8003ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 --> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2b8006b50 con 0x7fc2b80015f0 2023-10-26T05:47:31.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f8136fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f8136fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42192/0 (socket says 172.21.15.71:42192) 2023-10-26T05:47:31.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 -- 172.21.15.71:0/2900597994 learned_addr learned my addr 172.21.15.71:0/2900597994 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fcda3640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8070560 0x7f25f8139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff024051930 con 0x7ff02c0b1090 2023-10-26T05:47:31.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25ff808640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8137510 con 0x7f25f8070560 2023-10-26T05:47:31.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7ff03182f640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7ff01006e5f0 0x7ff010070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25ff808640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f813a7a0 con 0x7f25f806acc0 2023-10-26T05:47:31.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4268090070 con 0x7f42600e30a0 2023-10-26T05:47:31.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7ff03182f640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7ff01006e5f0 0x7ff010070a10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 -- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8070560 msgr2=0x7f25f8139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25f8070560 0x7f25f8139dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 -- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f8140c40 msgr2=0x7f25f80cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25f8140c40 0x7f25f80cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f000c050 con 0x7f25f806acc0 2023-10-26T05:47:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25fd5a4640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f8136fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c370 tx=0x7f25f400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4004230 con 0x7f25f806acc0 2023-10-26T05:47:31.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25f4004480 con 0x7f25f806acc0 2023-10-26T05:47:31.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4005260 con 0x7f25f806acc0 2023-10-26T05:47:31.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f425dffb640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f424c066570 0x7f424c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42680fbc20 con 0x7f42600e30a0 2023-10-26T05:47:31.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f425dffb640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f424c06e790 0x7f424c070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 --> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f424c071260 con 0x7f424c06e790 2023-10-26T05:47:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42680fbfd0 con 0x7f42600e30a0 2023-10-26T05:47:31.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.252+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== osd.1 v2:172.21.15.71:6800/619557138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7ff0100710c0 con 0x7ff01006e5f0 2023-10-26T05:47:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.256+0000 7fc2d77fe640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fc2b80015f0 0x7fc2b8003ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.256+0000 7fc2d77fe640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fc2b80015f0 0x7fc2b8003ab0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.256+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== osd.0 v2:172.21.15.71:6808/4161483918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fc2b8006b50 con 0x7fc2b80015f0 2023-10-26T05:47:31.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.256+0000 7f426c883640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f424c06e790 0x7f424c070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.256+0000 7f426c883640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f424c06e790 0x7f424c070bb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.256+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== osd.3 v2:172.21.15.71:6811/1313958381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f424c071260 con 0x7f424c06e790 2023-10-26T05:47:31.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f426d084640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f424c066570 0x7f424c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f25ff808640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f813aa80 con 0x7f25f806acc0 2023-10-26T05:47:31.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f25ff808640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f813b040 con 0x7f25f806acc0 2023-10-26T05:47:31.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f426d084640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f424c066570 0x7f424c068a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42640b2000 tx=0x7f42640a8f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25f8159630 con 0x7f25f806acc0 2023-10-26T05:47:31.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 --> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f856815a640 con 0x7f855406e5f0 2023-10-26T05:47:31.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.264+0000 7f85657fa640 1 -- 172.21.15.71:0/338098506 <== osd.4 v2:172.21.15.86:6800/893150098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f856815a640 con 0x7f855406e5f0 2023-10-26T05:47:31.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f855406e5f0 msgr2=0x7f8554070a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f855406e5f0 0x7f8554070a10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8554066540 msgr2=0x7f8554068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8554066540 0x7f8554068a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85580af000 tx=0x7f855801b070 comp rx=0 tx=0).stop 2023-10-26T05:47:31.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f856815bf20 msgr2=0x7f856815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 --2- 172.21.15.71:0/338098506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f856815bf20 0x7f856815a990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f855c00d660 tx=0x7f855c00db20 comp rx=0 tx=0).stop 2023-10-26T05:47:31.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856d2b3640 1 -- 172.21.15.71:0/338098506 reap_dead start 2023-10-26T05:47:31.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 shutdown_connections 2023-10-26T05:47:31.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 >> 172.21.15.71:0/338098506 conn(0x7f856806a6b0 msgr2=0x7f856813b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 shutdown_connections 2023-10-26T05:47:31.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.268+0000 7f856ed16640 1 -- 172.21.15.71:0/338098506 wait complete. 2023-10-26T05:47:31.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f25f40053c0 con 0x7f25f806acc0 2023-10-26T05:47:31.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25dc066470 0x7f25dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25f4084060 con 0x7f25f806acc0 2023-10-26T05:47:31.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25fcda3640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25dc066470 0x7f25dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f25dc06e610 0x7f25dc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25fdda5640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f25dc06e610 0x7f25dc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25fcda3640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25dc066470 0x7f25dc068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25f8071860 tx=0x7f25e800a2d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25fdda5640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f25dc06e610 0x7f25dc070a30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25dc0710e0 con 0x7f25dc06e610 2023-10-26T05:47:31.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25f4084410 con 0x7f25f806acc0 2023-10-26T05:47:31.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== osd.2 v2:172.21.15.71:6824/2687547983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f25dc0710e0 con 0x7f25dc06e610 2023-10-26T05:47:31.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f61bffb5640 1 -- 172.21.15.71:0/209290961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 msgr2=0x7f61b80af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.272+0000 7f61bffb5640 1 --2- 172.21.15.71:0/209290961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af1e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009840 tx=0x7f61a801c790 comp rx=0 tx=0).stop 2023-10-26T05:47:31.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 --> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75280d9d10 con 0x7f751406e610 2023-10-26T05:47:31.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 --> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f073c165760 con 0x7f071c06e610 2023-10-26T05:47:31.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 -- 172.21.15.71:0/209290961 shutdown_connections 2023-10-26T05:47:31.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 --2- 172.21.15.71:0/209290961 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b806bfd0 0x7f61b80af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 --2- 172.21.15.71:0/209290961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af1e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 -- 172.21.15.71:0/209290961 >> 172.21.15.71:0/209290961 conn(0x7f61b80addc0 msgr2=0x7f61b80b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 -- 172.21.15.71:0/209290961 shutdown_connections 2023-10-26T05:47:31.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 -- 172.21.15.71:0/209290961 wait complete. 2023-10-26T05:47:31.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f072affd640 1 -- 172.21.15.71:0/807091358 <== osd.6 v2:172.21.15.86:6824/1847760170 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f073c165760 con 0x7f071c06e610 2023-10-26T05:47:31.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f7525ffb640 1 -- 172.21.15.71:0/1911041596 <== osd.5 v2:172.21.15.86:6808/88051276 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f753006b130 con 0x7f751406e610 2023-10-26T05:47:31.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f751406e610 msgr2=0x7f7514070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f751406e610 0x7f7514070a30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7514066470 msgr2=0x7f7514068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7514066470 0x7f7514068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f752c0011f0 tx=0x7f752c009040 comp rx=0 tx=0).stop 2023-10-26T05:47:31.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75280535f0 msgr2=0x7f7528048570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 --2- 172.21.15.71:0/1911041596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75280535f0 0x7f7528048570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f752000d950 tx=0x7f752000de10 comp rx=0 tx=0).stop 2023-10-26T05:47:31.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f7535cf7640 1 -- 172.21.15.71:0/1911041596 reap_dead start 2023-10-26T05:47:31.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 shutdown_connections 2023-10-26T05:47:31.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 >> 172.21.15.71:0/1911041596 conn(0x7f7528009f90 msgr2=0x7f75280dcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 shutdown_connections 2023-10-26T05:47:31.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f753775a640 1 -- 172.21.15.71:0/1911041596 wait complete. 2023-10-26T05:47:31.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 --> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff02c14b5b0 con 0x7ff01006e5f0 2023-10-26T05:47:31.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 Processor -- start 2023-10-26T05:47:31.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7f61bffb5640 1 -- start start 2023-10-26T05:47:31.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.276+0000 7ff021ffb640 1 -- 172.21.15.71:0/1620177493 <== osd.1 v2:172.21.15.71:6800/619557138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff02c14b5b0 con 0x7ff01006e5f0 2023-10-26T05:47:31.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7ff01006e5f0 msgr2=0x7ff010070a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7ff01006e5f0 0x7ff010070a10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0100665c0 msgr2=0x7ff010068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0100665c0 0x7ff010068a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c00b410 comp rx=0 tx=0).stop 2023-10-26T05:47:31.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c0b1090 msgr2=0x7ff02c14e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 --2- 172.21.15.71:0/1620177493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff02c0b1090 0x7ff02c14e860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff02400bd50 tx=0x7ff02400be40 comp rx=0 tx=0).stop 2023-10-26T05:47:31.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff03182f640 1 -- 172.21.15.71:0/1620177493 reap_dead start 2023-10-26T05:47:31.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 shutdown_connections 2023-10-26T05:47:31.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 >> 172.21.15.71:0/1620177493 conn(0x7ff02c06a6b0 msgr2=0x7ff02c15af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bffb5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b806bfd0 0x7f61b8149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 shutdown_connections 2023-10-26T05:47:31.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bffb5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80cb900 0x7f61b81537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7ff033292640 1 -- 172.21.15.71:0/1620177493 wait complete. 2023-10-26T05:47:31.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bffb5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bdd51640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b806bfd0 0x7f61b8149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bdd51640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b806bfd0 0x7f61b8149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42206/0 (socket says 172.21.15.71:42206) 2023-10-26T05:47:31.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53302/0 (socket says 172.21.15.71:53302) 2023-10-26T05:47:31.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bd550640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80cb900 0x7f61b81537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bdd51640 1 -- 172.21.15.71:0/2616634174 learned_addr learned my addr 172.21.15.71:0/2616634174 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8149c90 con 0x7f61b81590f0 2023-10-26T05:47:31.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8153cf0 con 0x7f61b80cb900 2023-10-26T05:47:31.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80b0f40 con 0x7f61b806bfd0 2023-10-26T05:47:31.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f071c06e610 msgr2=0x7f071c070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f071c06e610 0x7f071c070a30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 --> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc2b8005bd0 con 0x7fc2b80015f0 2023-10-26T05:47:31.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 --> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4260043ab0 con 0x7f424c06e790 2023-10-26T05:47:31.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f071c066470 msgr2=0x7f071c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f071c066470 0x7f071c068930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f072c009f30 tx=0x7f072c009260 comp rx=0 tx=0).stop 2023-10-26T05:47:31.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c070fb0 msgr2=0x7f073c146270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 --2- 172.21.15.71:0/807091358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f073c070fb0 0x7f073c146270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f073000e9f0 tx=0x7f073000eeb0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f073ad9d640 1 -- 172.21.15.71:0/807091358 reap_dead start 2023-10-26T05:47:31.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 shutdown_connections 2023-10-26T05:47:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 >> 172.21.15.71:0/807091358 conn(0x7f073c069770 msgr2=0x7f073c149e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 shutdown_connections 2023-10-26T05:47:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f0740a99640 1 -- 172.21.15.71:0/807091358 wait complete. 2023-10-26T05:47:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2bf7fe640 1 -- 172.21.15.71:0/2259356936 <== osd.0 v2:172.21.15.71:6808/4161483918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc2b8005bd0 con 0x7fc2b80015f0 2023-10-26T05:47:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f425dffb640 1 -- 172.21.15.71:0/398270931 <== osd.3 v2:172.21.15.71:6811/1313958381 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4260043ab0 con 0x7f424c06e790 2023-10-26T05:47:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 -- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b806bfd0 msgr2=0x7f61b8149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b806bfd0 0x7f61b8149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 -- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80cb900 msgr2=0x7f61b81537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80cb900 0x7f61b81537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80094a0 con 0x7f61b81590f0 2023-10-26T05:47:31.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61bd550640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80cb900 0x7f61b81537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:31.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fc2b80015f0 msgr2=0x7fc2b8003ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fc2b80015f0 0x7fc2b8003ab0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc2c4066540 msgr2=0x7fc2c4068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc2c4066540 0x7fc2c4068a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0034440 tx=0x7fc2c80090e0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d003ab40 msgr2=0x7fc2d00edc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 --2- 172.21.15.71:0/2259356936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d003ab40 0x7fc2d00edc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc0098a0 tx=0x7fc2cc005e50 comp rx=0 tx=0).stop 2023-10-26T05:47:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2d77fe640 1 -- 172.21.15.71:0/2259356936 reap_dead start 2023-10-26T05:47:31.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 shutdown_connections 2023-10-26T05:47:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 >> 172.21.15.71:0/2259356936 conn(0x7fc2d0054ef0 msgr2=0x7fc2d0005c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 shutdown_connections 2023-10-26T05:47:31.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7fc2dd33b640 1 -- 172.21.15.71:0/2259356936 wait complete. 2023-10-26T05:47:31.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.280+0000 7f61be552640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f61b000ea60 tx=0x7f61b000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004b30 con 0x7f61b81590f0 2023-10-26T05:47:31.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b80b1220 con 0x7f61b81590f0 2023-10-26T05:47:31.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b80b17e0 con 0x7f61b81590f0 2023-10-26T05:47:31.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61b0004d80 con 0x7f61b81590f0 2023-10-26T05:47:31.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0015c40 con 0x7f61b81590f0 2023-10-26T05:47:31.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61b80cd440 con 0x7f61b81590f0 2023-10-26T05:47:31.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.284+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61b000c970 con 0x7f61b81590f0 2023-10-26T05:47:31.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61ae7fc640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f619c066470 0x7f619c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.358 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-26T05:47:31.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b0088ed0 con 0x7f61b81590f0 2023-10-26T05:47:31.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61bdd51640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f619c066470 0x7f619c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61ae7fc640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f619c06e610 0x7f619c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f619c0710e0 con 0x7f619c06e610 2023-10-26T05:47:31.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b008b590 con 0x7f61b81590f0 2023-10-26T05:47:31.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61bd550640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f619c06e610 0x7f619c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.288+0000 7f61bdd51640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f619c066470 0x7f619c068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61a801d030 tx=0x7f61a8019530 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f424c06e790 msgr2=0x7f424c070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f424c06e790 0x7f424c070bb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f424c066570 msgr2=0x7f424c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f424c066570 0x7f424c068a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42640b2000 tx=0x7f42640a8f90 comp rx=0 tx=0).stop 2023-10-26T05:47:31.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42600e30a0 msgr2=0x7f4260039d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 --2- 172.21.15.71:0/398270931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42600e30a0 0x7f4260039d30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4268066ca0 tx=0x7f42680670d0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426d885640 1 -- 172.21.15.71:0/398270931 reap_dead start 2023-10-26T05:47:31.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 shutdown_connections 2023-10-26T05:47:31.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 >> 172.21.15.71:0/398270931 conn(0x7f4260009f90 msgr2=0x7f426004aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 shutdown_connections 2023-10-26T05:47:31.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f426e086640 1 -- 172.21.15.71:0/398270931 wait complete. 2023-10-26T05:47:31.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f61bd550640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f619c06e610 0x7f619c070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.292+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== osd.7 v2:172.21.15.86:6816/2184210851 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f619c0710e0 con 0x7f619c06e610 2023-10-26T05:47:31.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 --> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25f8159630 con 0x7f25dc06e610 2023-10-26T05:47:31.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25edffb640 1 -- 172.21.15.71:0/2900597994 <== osd.2 v2:172.21.15.71:6824/2687547983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25f00ae3f0 con 0x7f25dc06e610 2023-10-26T05:47:31.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f25dc06e610 msgr2=0x7f25dc070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f25dc06e610 0x7f25dc070a30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25dc066470 msgr2=0x7f25dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25dc066470 0x7f25dc068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25f8071860 tx=0x7f25e800a2d0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 msgr2=0x7f25f8136fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 --2- 172.21.15.71:0/2900597994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25f806acc0 0x7f25f8136fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c370 tx=0x7f25f400c830 comp rx=0 tx=0).stop 2023-10-26T05:47:31.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25fdda5640 1 -- 172.21.15.71:0/2900597994 reap_dead start 2023-10-26T05:47:31.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 shutdown_connections 2023-10-26T05:47:31.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 >> 172.21.15.71:0/2900597994 conn(0x7f25f80aec60 msgr2=0x7f25f806fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 shutdown_connections 2023-10-26T05:47:31.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.296+0000 7f25db7fe640 1 -- 172.21.15.71:0/2900597994 wait complete. 2023-10-26T05:47:31.369 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.369 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-26T05:47:31.370 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.371 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-26T05:47:31.372 INFO:teuthology.orchestra.run.smithi071.stdout:55834574855 2023-10-26T05:47:31.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-26T05:47:31.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.312+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 --> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61b8071cd0 con 0x7f619c06e610 2023-10-26T05:47:31.373 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.374 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-26T05:47:31.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61ae7fc640 1 -- 172.21.15.71:0/2616634174 <== osd.7 v2:172.21.15.86:6816/2184210851 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61b8071cd0 con 0x7f619c06e610 2023-10-26T05:47:31.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f619c06e610 msgr2=0x7f619c070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f619c06e610 0x7f619c070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f619c066470 msgr2=0x7f619c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f619c066470 0x7f619c068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61a801d030 tx=0x7f61a8019530 comp rx=0 tx=0).stop 2023-10-26T05:47:31.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 msgr2=0x7f61b80af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 --2- 172.21.15.71:0/2616634174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b81590f0 0x7f61b80af490 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f61b000ea60 tx=0x7f61b000ef20 comp rx=0 tx=0).stop 2023-10-26T05:47:31.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61be552640 1 -- 172.21.15.71:0/2616634174 reap_dead start 2023-10-26T05:47:31.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 shutdown_connections 2023-10-26T05:47:31.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 >> 172.21.15.71:0/2616634174 conn(0x7f61b80addc0 msgr2=0x7f61b814b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 shutdown_connections 2023-10-26T05:47:31.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.316+0000 7f61bffb5640 1 -- 172.21.15.71:0/2616634174 wait complete. 2023-10-26T05:47:31.378 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-26T05:47:31.381 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.381 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-26T05:47:31.382 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-26T05:47:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 -- 172.21.15.71:0/3264958706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2415da70 msgr2=0x7f8a240cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3264958706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2415da70 0x7f8a240cb9c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009c80 tx=0x7f8a1c01cd70 comp rx=0 tx=0).stop 2023-10-26T05:47:31.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 -- 172.21.15.71:0/2154124771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c2006a780 msgr2=0x7f4c20158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/2154124771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c2006a780 0x7f4c20158e70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009900 tx=0x7f4c1001c8e0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 -- 172.21.15.71:0/3264958706 shutdown_connections 2023-10-26T05:47:31.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3264958706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2415da70 0x7f8a240cb9c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 -- 172.21.15.71:0/3264958706 >> 172.21.15.71:0/3264958706 conn(0x7f8a240afe40 msgr2=0x7f8a240b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 -- 172.21.15.71:0/2154124771 shutdown_connections 2023-10-26T05:47:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/2154124771 >> v2:172.21.15.86:3300/0 conn(0x7f4c200c9680 0x7f4c2006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/2154124771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c2006a780 0x7f4c20158e70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 -- 172.21.15.71:0/2154124771 >> 172.21.15.71:0/2154124771 conn(0x7f4c2006c6c0 msgr2=0x7f4c2006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 -- 172.21.15.71:0/3264958706 shutdown_connections 2023-10-26T05:47:31.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f8a23fff640 1 -- 172.21.15.71:0/3264958706 wait complete. 2023-10-26T05:47:31.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 -- 172.21.15.71:0/2154124771 shutdown_connections 2023-10-26T05:47:31.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.584+0000 7f4c1ffff640 1 -- 172.21.15.71:0/2154124771 wait complete. 2023-10-26T05:47:31.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 Processor -- start 2023-10-26T05:47:31.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 Processor -- start 2023-10-26T05:47:31.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 -- start start 2023-10-26T05:47:31.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2406e670 0x7f8a241372f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 0x7f8a241560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a2415da70 0x7f8a24159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a241576b0 con 0x7f8a2406e670 2023-10-26T05:47:31.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24156aa0 con 0x7f8a2406ea50 2023-10-26T05:47:31.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a24156d80 con 0x7f8a2415da70 2023-10-26T05:47:31.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 -- 172.21.15.71:0/3442433007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa8036ff0 msgr2=0x7f2fa8039ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 --2- 172.21.15.71:0/3442433007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa8036ff0 0x7f2fa8039ba0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800c000 tx=0x7f2f9c01c830 comp rx=0 tx=0).stop 2023-10-26T05:47:31.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 -- 172.21.15.71:0/2315417225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 msgr2=0x7f5dd800c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 --2- 172.21.15.71:0/2315417225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dd800c9c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5de8006fc0 tx=0x7f5de801f190 comp rx=0 tx=0).stop 2023-10-26T05:47:31.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 -- 172.21.15.71:0/2477153298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f50150110 msgr2=0x7f0f500cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 --2- 172.21.15.71:0/2477153298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f50150110 0x7f0f500cf360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f500c5f00 tx=0x7f0f4401c770 comp rx=0 tx=0).stop 2023-10-26T05:47:31.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 -- 172.21.15.71:0/3442433007 shutdown_connections 2023-10-26T05:47:31.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 --2- 172.21.15.71:0/3442433007 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80378c0 0x7f2fa8053480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 --2- 172.21.15.71:0/3442433007 >> v2:172.21.15.86:3300/0 conn(0x7f2fa80373d0 0x7f2fa8057670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 --2- 172.21.15.71:0/3442433007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa8036ff0 0x7f2fa8039ba0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 -- 172.21.15.71:0/3442433007 >> 172.21.15.71:0/3442433007 conn(0x7f2fa8020020 msgr2=0x7f2fa8056d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 0x7f8a241560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 0x7f8a241560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:58458/0 (socket says 172.21.15.71:58458) 2023-10-26T05:47:31.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 -- 172.21.15.71:0/3102159409 learned_addr learned my addr 172.21.15.71:0/3102159409 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 -- 172.21.15.71:0/2477153298 shutdown_connections 2023-10-26T05:47:31.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 --2- 172.21.15.71:0/2477153298 >> v2:172.21.15.86:3300/0 conn(0x7f0f500b0510 0x7f0f500b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 --2- 172.21.15.71:0/2477153298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f50150110 0x7f0f500cf360 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 -- 172.21.15.71:0/2477153298 >> 172.21.15.71:0/2477153298 conn(0x7f0f5006f550 msgr2=0x7f0f5006f960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a237fe640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a2415da70 0x7f8a24159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a22ffd640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2406e670 0x7f8a241372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 -- 172.21.15.71:0/3442433007 shutdown_connections 2023-10-26T05:47:31.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 -- 172.21.15.71:0/3442433007 wait complete. 2023-10-26T05:47:31.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 -- 172.21.15.71:0/2315417225 shutdown_connections 2023-10-26T05:47:31.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 --2- 172.21.15.71:0/2315417225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dd800c9c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 -- 172.21.15.71:0/2315417225 >> 172.21.15.71:0/2315417225 conn(0x7f5dec06c6c0 msgr2=0x7f5dec06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 -- 172.21.15.71:0/2315417225 shutdown_connections 2023-10-26T05:47:31.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 -- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a2415da70 msgr2=0x7f8a24159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a2415da70 0x7f8a24159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 -- 172.21.15.71:0/3102159409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2406e670 msgr2=0x7f8a241372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2406e670 0x7f8a241372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 -- 172.21.15.71:0/3102159409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c0098e0 con 0x7f8a2406ea50 2023-10-26T05:47:31.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1bfff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 0x7f8a241560c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00c500 tx=0x7f8a0c00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 -- 172.21.15.71:0/2315417225 wait complete. 2023-10-26T05:47:31.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 -- 172.21.15.71:0/2477153298 shutdown_connections 2023-10-26T05:47:31.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 -- start start 2023-10-26T05:47:31.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c005220 con 0x7f8a2406ea50 2023-10-26T05:47:31.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8a0c005470 con 0x7f8a2406ea50 2023-10-26T05:47:31.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c007290 con 0x7f8a2406ea50 2023-10-26T05:47:31.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a24157060 con 0x7f8a2406ea50 2023-10-26T05:47:31.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2415b4d0 con 0x7f8a2406ea50 2023-10-26T05:47:31.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 Processor -- start 2023-10-26T05:47:31.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 Processor -- start 2023-10-26T05:47:31.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 -- 172.21.15.71:0/2477153298 wait complete. 2023-10-26T05:47:31.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 Processor -- start 2023-10-26T05:47:31.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2006a780 0x7f4c20154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c20070510 0x7f4c20146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 0x7f4c201490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20130a70 con 0x7f4c200c9680 2023-10-26T05:47:31.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20155000 con 0x7f4c2006a780 2023-10-26T05:47:31.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1ffff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20130ea0 con 0x7f4c20070510 2023-10-26T05:47:31.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 0x7f4c201490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 0x7f4c201490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53374/0 (socket says 172.21.15.71:53374) 2023-10-26T05:47:31.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f4c1f7fe640 1 -- 172.21.15.71:0/276051562 learned_addr learned my addr 172.21.15.71:0/276051562 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f5df302c640 1 -- start start 2023-10-26T05:47:31.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1e7fc640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c20070510 0x7f4c20146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1effd640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2006a780 0x7f4c20154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 -- 172.21.15.71:0/2812998879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8754004df0 msgr2=0x7f87540071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2812998879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8754004df0 0x7f87540071e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8750009d10 tx=0x7f875001d3a0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 -- 172.21.15.71:0/2812998879 shutdown_connections 2023-10-26T05:47:31.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2812998879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8754004df0 0x7f87540071e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 -- 172.21.15.71:0/2812998879 >> 172.21.15.71:0/2812998879 conn(0x7f87680b2870 msgr2=0x7f87680aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1f7fe640 1 -- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c20070510 msgr2=0x7f4c20146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1f7fe640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c20070510 0x7f4c20146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1f7fe640 1 -- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2006a780 msgr2=0x7f4c20154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1f7fe640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2006a780 0x7f4c20154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1f7fe640 1 -- 172.21.15.71:0/276051562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c10009560 con 0x7f4c200c9680 2023-10-26T05:47:31.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1effd640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2006a780 0x7f4c20154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:31.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f0f54882640 1 -- start start 2023-10-26T05:47:31.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f500b0510 0x7f0f500c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 0x7f0f500b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f50150110 0x7f0f500b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501f9ae0 con 0x7f0f500b0510 2023-10-26T05:47:31.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50163650 con 0x7f0f50150110 2023-10-26T05:47:31.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501f9800 con 0x7f0f500cf8a0 2023-10-26T05:47:31.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.588+0000 7f2fb6c91640 1 -- start start 2023-10-26T05:47:31.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8036ff0 0x7f2fa80e0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80373d0 0x7f2fa8007e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa80378c0 0x7f2fa80dac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8008770 con 0x7f2fa80378c0 2023-10-26T05:47:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80092f0 con 0x7f2fa8036ff0 2023-10-26T05:47:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa8009780 con 0x7f2fa80373d0 2023-10-26T05:47:31.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1f7fe640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 0x7f4c201490b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800c790 tx=0x7f4c1800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 0x7f0f500b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80373d0 0x7f2fa8007e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb522e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa80378c0 0x7f2fa80dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a24066b10 con 0x7f8a2406ea50 2023-10-26T05:47:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb522e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa80378c0 0x7f2fa80dac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53400/0 (socket says 172.21.15.71:53400) 2023-10-26T05:47:31.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb522e640 1 -- 172.21.15.71:0/1833346931 learned_addr learned my addr 172.21.15.71:0/1833346931 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800cea0 con 0x7f4c200c9680 2023-10-26T05:47:31.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4c18003b80 con 0x7f4c200c9680 2023-10-26T05:47:31.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 0x7f0f500b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42270/0 (socket says 172.21.15.71:42270) 2023-10-26T05:47:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 -- 172.21.15.71:0/1181375032 learned_addr learned my addr 172.21.15.71:0/1181375032 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800a850 con 0x7f4c200c9680 2023-10-26T05:47:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df302c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dec15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df302c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dec15c5e0 0x7f5dec1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df302c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec15e510 0x7f5dec12c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df302c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec15fb50 con 0x7f5dd800a500 2023-10-26T05:47:31.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df302c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec156e00 con 0x7f5dec15e510 2023-10-26T05:47:31.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df302c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec15d470 con 0x7f5dec15c5e0 2023-10-26T05:47:31.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df202a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dec15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df202a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dec15ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53414/0 (socket says 172.21.15.71:53414) 2023-10-26T05:47:31.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df202a640 1 -- 172.21.15.71:0/479313230 learned_addr learned my addr 172.21.15.71:0/479313230 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb4a2d640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8036ff0 0x7f2fa80e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec15e510 0x7f5dec12c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df1829640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dec15c5e0 0x7f5dec1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c20131180 con 0x7f4c200c9680 2023-10-26T05:47:31.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c20131740 con 0x7f4c200c9680 2023-10-26T05:47:31.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 -- 172.21.15.71:0/2812998879 shutdown_connections 2023-10-26T05:47:31.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 -- 172.21.15.71:0/2812998879 wait complete. 2023-10-26T05:47:31.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 -- 172.21.15.71:0/603257366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650100f30 msgr2=0x7f1650052a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 --2- 172.21.15.71:0/603257366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650100f30 0x7f1650052a20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f164c009900 tx=0x7f164c01eac0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 -- 172.21.15.71:0/603257366 shutdown_connections 2023-10-26T05:47:31.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 --2- 172.21.15.71:0/603257366 >> v2:172.21.15.71:3300/0 conn(0x7f16500535c0 0x7f1650007160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 --2- 172.21.15.71:0/603257366 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650100f30 0x7f1650052a20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 -- 172.21.15.71:0/603257366 >> 172.21.15.71:0/603257366 conn(0x7f1650009f90 msgr2=0x7f165000a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8a0c0073f0 con 0x7f8a2406ea50 2023-10-26T05:47:31.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f8a1b7fe640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8a04066470 0x7f8a04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a0c083670 con 0x7f8a2406ea50 2023-10-26T05:47:31.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f8a22ffd640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8a04066470 0x7f8a04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 -- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8036ff0 msgr2=0x7f2fa80e0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8036ff0 0x7f2fa80e0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 -- 172.21.15.71:0/1833346931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa80378c0 msgr2=0x7f2fa80dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa80378c0 0x7f2fa80dac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 -- 172.21.15.71:0/1833346931 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c009560 con 0x7f2fa80373d0 2023-10-26T05:47:31.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2faffff640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80373d0 0x7f2fa8007e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d780 tx=0x7f2fa400db80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4005220 con 0x7f2fa80373d0 2023-10-26T05:47:31.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 -- 172.21.15.71:0/1833346931 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa80089c0 con 0x7f2fa80373d0 2023-10-26T05:47:31.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fb6c91640 1 -- 172.21.15.71:0/1833346931 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa8008f80 con 0x7f2fa80373d0 2023-10-26T05:47:31.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fa4005380 con 0x7f2fa80373d0 2023-10-26T05:47:31.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40097c0 con 0x7f2fa80373d0 2023-10-26T05:47:31.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 -- 172.21.15.71:0/603257366 shutdown_connections 2023-10-26T05:47:31.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f165da2e640 1 -- 172.21.15.71:0/603257366 wait complete. 2023-10-26T05:47:31.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 -- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dec15c5e0 msgr2=0x7f5dec1568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dec15c5e0 0x7f5dec1568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 -- 172.21.15.71:0/479313230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 msgr2=0x7f5dec15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dec15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 -- 172.21.15.71:0/479313230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec0b0ed0 con 0x7f5dec15e510 2023-10-26T05:47:31.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df282b640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec15e510 0x7f5dec12c190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dec15ad40 tx=0x7f5de80011b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f5df1829640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dec15c5e0 0x7f5dec1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:47:31.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 -- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f50150110 msgr2=0x7f0f500b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f50150110 0x7f0f500b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 -- 172.21.15.71:0/1181375032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f500b0510 msgr2=0x7f0f500c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f500b0510 0x7f0f500c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 -- 172.21.15.71:0/1181375032 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f440094a0 con 0x7f0f500cf8a0 2023-10-26T05:47:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f4e59c640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 0x7f0f500b2fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c480 tx=0x7f0f4000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400068a0 con 0x7f0f500cf8a0 2023-10-26T05:47:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f40006b10 con 0x7f0f500cf8a0 2023-10-26T05:47:31.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f50163000 con 0x7f0f500cf8a0 2023-10-26T05:47:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f876cbe5640 1 Processor -- start 2023-10-26T05:47:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c00005260 con 0x7f4c200c9680 2023-10-26T05:47:31.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 Processor -- start 2023-10-26T05:47:31.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f500c1c20 con 0x7f0f500cf8a0 2023-10-26T05:47:31.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.592+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40007020 con 0x7f0f500cf8a0 2023-10-26T05:47:31.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f8a22ffd640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8a04066470 0x7f8a04068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000f70 tx=0x7f8a14009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 -- start start 2023-10-26T05:47:31.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8754004df0 0x7f876814e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 0x7f87680991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f87677fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8754004df0 0x7f876814e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f87677fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8754004df0 0x7f876814e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42304/0 (socket says 172.21.15.71:42304) 2023-10-26T05:47:31.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 -- start start 2023-10-26T05:47:31.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16500535c0 0x7f16500f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 0x7f165004c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650037b20 0x7f165004c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 0x7f87680991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 0x7f87680991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53442/0 (socket says 172.21.15.71:53442) 2023-10-26T05:47:31.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1650038090 con 0x7f16500535c0 2023-10-26T05:47:31.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1650038370 con 0x7f1650100b50 2023-10-26T05:47:31.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165da2e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1650038650 con 0x7f1650037b20 2023-10-26T05:47:31.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8768099700 0x7f8768159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 0x7f165004c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f2fb6c91640 1 -- 172.21.15.71:0/1833346931 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f90005260 con 0x7f2fa80373d0 2023-10-26T05:47:31.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165d22d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650037b20 0x7f165004c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876809a920 con 0x7f87680aecd0 2023-10-26T05:47:31.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876814e8f0 con 0x7f8768099700 2023-10-26T05:47:31.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f876cbe5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8768099ca0 con 0x7f8754004df0 2023-10-26T05:47:31.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 -- 172.21.15.71:0/2076579022 learned_addr learned my addr 172.21.15.71:0/2076579022 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4c1800a9b0 con 0x7f4c200c9680 2023-10-26T05:47:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 0x7f165004c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:58488/0 (socket says 172.21.15.71:58488) 2023-10-26T05:47:31.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f165d22d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650037b20 0x7f165004c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:42306/0 (socket says 172.21.15.71:42306) 2023-10-26T05:47:31.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 -- 172.21.15.71:0/2295396426 learned_addr learned my addr 172.21.15.71:0/2295396426 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f8767fff640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8768099700 0x7f8768159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f4c077fe640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4c0c066540 0x7f4c0c068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2fa4009920 con 0x7f2fa80373d0 2023-10-26T05:47:31.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c180848e0 con 0x7f4c200c9680 2023-10-26T05:47:31.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 -- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650037b20 msgr2=0x7f165004c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650037b20 0x7f165004c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 -- 172.21.15.71:0/2295396426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16500535c0 msgr2=0x7f16500f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16500535c0 0x7f16500f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 -- 172.21.15.71:0/2295396426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c009560 con 0x7f1650100b50 2023-10-26T05:47:31.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 -- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8754004df0 msgr2=0x7f876814e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8754004df0 0x7f876814e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 -- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8768099700 msgr2=0x7f8768159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8768099700 0x7f8768159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 -- 172.21.15.71:0/2076579022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8768099f80 con 0x7f87680aecd0 2023-10-26T05:47:31.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f4c1effd640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4c0c066540 0x7f4c0c068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f2fad7fa640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2f940664a0 0x7f2f94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f1657fff640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 0x7f165004c1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f164c009850 tx=0x7f164c004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f875ffff640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 0x7f87680991c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8758003e70 tx=0x7f8758001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fa4083be0 con 0x7f2fa80373d0 2023-10-26T05:47:31.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f2fb4a2d640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2f940664a0 0x7f2f94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.596+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f30005260 con 0x7f0f500cf8a0 2023-10-26T05:47:31.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f2fb4a2d640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2f940664a0 0x7f2f94068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0af000 tx=0x7f2f9c01b070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f4c1effd640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4c0c066540 0x7f4c0c068a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c100af000 tx=0x7f4c1001b120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f40007180 con 0x7f0f500cf8a0 2023-10-26T05:47:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f0f377fe640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f3c066470 0x7f0f3c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f40083de0 con 0x7f0f500cf8a0 2023-10-26T05:47:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758011840 con 0x7f87680aecd0 2023-10-26T05:47:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f0f4ed9d640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f3c066470 0x7f0f3c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4c18051d40 con 0x7f4c200c9680 2023-10-26T05:47:31.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f87580119a0 con 0x7f87680aecd0 2023-10-26T05:47:31.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876809a260 con 0x7f87680aecd0 2023-10-26T05:47:31.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8768159fd0 con 0x7f87680aecd0 2023-10-26T05:47:31.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758011d90 con 0x7f87680aecd0 2023-10-26T05:47:31.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f0f4ed9d640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f3c066470 0x7f0f3c068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f440af000 tx=0x7f0f44019510 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876804da80 con 0x7f87680aecd0 2023-10-26T05:47:31.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87580103c0 con 0x7f87680aecd0 2023-10-26T05:47:31.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f8764ff9640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87480664a0 0x7f8748068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87580856a0 con 0x7f87680aecd0 2023-10-26T05:47:31.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.600+0000 7f87677fe640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87480664a0 0x7f8748068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.604+0000 7f87677fe640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87480664a0 0x7f8748068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f87500b0000 tx=0x7f875001a140 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.604+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8758052120 con 0x7f87680aecd0 2023-10-26T05:47:31.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c0a7aa0 con 0x7f1650100b50 2023-10-26T05:47:31.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1650038930 con 0x7f1650100b50 2023-10-26T05:47:31.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16500f1420 con 0x7f1650100b50 2023-10-26T05:47:31.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8a0c050ba0 con 0x7f8a2406ea50 2023-10-26T05:47:31.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80b4070 con 0x7f5dec15e510 2023-10-26T05:47:31.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5de8001440 con 0x7f5dec15e510 2023-10-26T05:47:31.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8006140 con 0x7f5dec15e510 2023-10-26T05:47:31.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec15d750 con 0x7f5dec15e510 2023-10-26T05:47:31.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec15dd10 con 0x7f5dec15e510 2023-10-26T05:47:31.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f164c005030 con 0x7f1650100b50 2023-10-26T05:47:31.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c0053a0 con 0x7f1650100b50 2023-10-26T05:47:31.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.608+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec066b10 con 0x7f5dec15e510 2023-10-26T05:47:31.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1638005260 con 0x7f1650100b50 2023-10-26T05:47:31.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5de80ac030 con 0x7f5dec15e510 2023-10-26T05:47:31.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2fa40514e0 con 0x7f2fa80373d0 2023-10-26T05:47:31.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f5de2ffd640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5dd0066570 0x7f5dd0068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5de811e2e0 con 0x7f5dec15e510 2023-10-26T05:47:31.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f5df202a640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5dd0066570 0x7f5dd0068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f164c0046d0 con 0x7f1650100b50 2023-10-26T05:47:31.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f16557fa640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f163c066470 0x7f163c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f164c11dff0 con 0x7f1650100b50 2023-10-26T05:47:31.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f5df202a640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5dd0066570 0x7f5dd0068a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004f30 tx=0x7f5ddc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.612+0000 7f165ca2c640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f163c066470 0x7f163c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.616+0000 7f165ca2c640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f163c066470 0x7f163c068930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1658068e70 tx=0x7f1658077c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5de80eb7d0 con 0x7f5dec15e510 2023-10-26T05:47:31.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f400512f0 con 0x7f0f500cf8a0 2023-10-26T05:47:31.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 -- 172.21.15.71:0/265695139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59841571a0 msgr2=0x7f598406a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 --2- 172.21.15.71:0/265695139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59841571a0 0x7f598406a150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f598406aa70 tx=0x7f598001d8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 -- 172.21.15.71:0/265695139 shutdown_connections 2023-10-26T05:47:31.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 --2- 172.21.15.71:0/265695139 >> v2:172.21.15.86:3300/0 conn(0x7f59840b0530 0x7f59840cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 --2- 172.21.15.71:0/265695139 >> v2:172.21.15.71:3300/0 conn(0x7f598406a690 0x7f59840afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 --2- 172.21.15.71:0/265695139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59841571a0 0x7f598406a150 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f598406aa70 tx=0x7f598001d8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 -- 172.21.15.71:0/265695139 >> 172.21.15.71:0/265695139 conn(0x7f59840b2bc0 msgr2=0x7f59840aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 -- 172.21.15.71:0/265695139 shutdown_connections 2023-10-26T05:47:31.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.624+0000 7f598a466640 1 -- 172.21.15.71:0/265695139 wait complete. 2023-10-26T05:47:31.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 Processor -- start 2023-10-26T05:47:31.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 -- start start 2023-10-26T05:47:31.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 0x7f598414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59840b0530 0x7f598413eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5984097a70 0x7f598414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f5989464640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 0x7f598414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f5989464640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 0x7f598414e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53458/0 (socket says 172.21.15.71:53458) 2023-10-26T05:47:31.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f5989464640 1 -- 172.21.15.71:0/3859597047 learned_addr learned my addr 172.21.15.71:0/3859597047 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:31.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984098b80 con 0x7f598406a690 2023-10-26T05:47:31.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598414e9a0 con 0x7f59840b0530 2023-10-26T05:47:31.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598413f0a0 con 0x7f5984097a70 2023-10-26T05:47:31.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f5988c63640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59840b0530 0x7f598413eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.628+0000 7f5989c65640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5984097a70 0x7f598414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989464640 1 -- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5984097a70 msgr2=0x7f598414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989464640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5984097a70 0x7f598414b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989464640 1 -- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59840b0530 msgr2=0x7f598413eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989464640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59840b0530 0x7f598413eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989464640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5980007450 con 0x7f598406a690 2023-10-26T05:47:31.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989c65640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5984097a70 0x7f598414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:31.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5989464640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 0x7f598414e460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f598001b500 tx=0x7f59800a6650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980007c60 con 0x7f598406a690 2023-10-26T05:47:31.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5980003980 con 0x7f598406a690 2023-10-26T05:47:31.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980003d50 con 0x7f598406a690 2023-10-26T05:47:31.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5984097f00 con 0x7f598406a690 2023-10-26T05:47:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59840984c0 con 0x7f598406a690 2023-10-26T05:47:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.632+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598404da80 con 0x7f598406a690 2023-10-26T05:47:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.636+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f59800a6840 con 0x7f598406a690 2023-10-26T05:47:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.636+0000 7f5979ffb640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:31.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.636+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59800eedc0 con 0x7f598406a690 2023-10-26T05:47:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.636+0000 7f5988c63640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:31.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.636+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f164c0eb490 con 0x7f1650100b50 2023-10-26T05:47:31.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.636+0000 7f5988c63640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59680664a0 0x7f5968068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5984152d80 tx=0x7f597400b390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:31.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.644+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59800eaa00 con 0x7f598406a690 2023-10-26T05:47:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.892+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4c000050d0 con 0x7f4c200c9680 2023-10-26T05:47:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.892+0000 7f4c077fe640 1 -- 172.21.15.71:0/276051562 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c18055360 con 0x7f4c200c9680 2023-10-26T05:47:31.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.892+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8a24157820 con 0x7f8a2406ea50 2023-10-26T05:47:31.900 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f8a1b7fe640 1 -- 172.21.15.71:0/3102159409 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a0c0541c0 con 0x7f8a2406ea50 2023-10-26T05:47:31.902 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.902 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f87680af4d0 con 0x7f87680aecd0 2023-10-26T05:47:31.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f8764ff9640 1 -- 172.21.15.71:0/2076579022 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f875801f020 con 0x7f87680aecd0 2023-10-26T05:47:31.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4c0c066540 msgr2=0x7f4c0c068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4c0c066540 0x7f4c0c068a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c100af000 tx=0x7f4c1001b120 comp rx=0 tx=0).stop 2023-10-26T05:47:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 msgr2=0x7f4c201490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 0x7f4c201490b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800c790 tx=0x7f4c1800cc50 comp rx=0 tx=0).stop 2023-10-26T05:47:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 shutdown_connections 2023-10-26T05:47:31.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4c0c066540 0x7f4c0c068a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c200c9680 0x7f4c201490b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4c20070510 0x7f4c20146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 --2- 172.21.15.71:0/276051562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c2006a780 0x7f4c20154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 >> 172.21.15.71:0/276051562 conn(0x7f4c2006c6c0 msgr2=0x7f4c2014fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 shutdown_connections 2023-10-26T05:47:31.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f4c1ffff640 1 -- 172.21.15.71:0/276051562 wait complete. 2023-10-26T05:47:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8a04066470 msgr2=0x7f8a04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8a04066470 0x7f8a04068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000f70 tx=0x7f8a14009040 comp rx=0 tx=0).stop 2023-10-26T05:47:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 msgr2=0x7f8a241560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.896+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 0x7f8a241560c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00c500 tx=0x7f8a0c00c9c0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 shutdown_connections 2023-10-26T05:47:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8a2415da70 0x7f8a24159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a2406ea50 0x7f8a241560c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8a04066470 0x7f8a04068930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 --2- 172.21.15.71:0/3102159409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a2406e670 0x7f8a241372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 >> 172.21.15.71:0/3102159409 conn(0x7f8a240afe40 msgr2=0x7f8a24070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87480664a0 msgr2=0x7f8748068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87480664a0 0x7f8748068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f87500b0000 tx=0x7f875001a140 comp rx=0 tx=0).stop 2023-10-26T05:47:31.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 msgr2=0x7f87680991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 0x7f87680991c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8758003e70 tx=0x7f8758001640 comp rx=0 tx=0).stop 2023-10-26T05:47:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 shutdown_connections 2023-10-26T05:47:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8768099700 0x7f8768159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87480664a0 0x7f8748068960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87680aecd0 0x7f87680991c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 --2- 172.21.15.71:0/2076579022 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8754004df0 0x7f876814e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 >> 172.21.15.71:0/2076579022 conn(0x7f87680b2870 msgr2=0x7f87680b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 shutdown_connections 2023-10-26T05:47:31.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 shutdown_connections 2023-10-26T05:47:31.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f876cbe5640 1 -- 172.21.15.71:0/2076579022 wait complete. 2023-10-26T05:47:31.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.900+0000 7f8a23fff640 1 -- 172.21.15.71:0/3102159409 wait complete. 2023-10-26T05:47:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.912+0000 7f2fb6c91640 1 -- 172.21.15.71:0/1833346931 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2f900050d0 con 0x7f2fa80373d0 2023-10-26T05:47:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.912+0000 7f2fad7fa640 1 -- 172.21.15.71:0/1833346931 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fa4017090 con 0x7f2fa80373d0 2023-10-26T05:47:31.921 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 -- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2f940664a0 msgr2=0x7f2f94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2f940664a0 0x7f2f94068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0af000 tx=0x7f2f9c01b070 comp rx=0 tx=0).stop 2023-10-26T05:47:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 -- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80373d0 msgr2=0x7f2fa8007e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80373d0 0x7f2fa8007e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d780 tx=0x7f2fa400db80 comp rx=0 tx=0).stop 2023-10-26T05:47:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 -- 172.21.15.71:0/1833346931 shutdown_connections 2023-10-26T05:47:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2f940664a0 0x7f2f94068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fa80378c0 0x7f2fa80dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fa80373d0 0x7f2fa8007e20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 --2- 172.21.15.71:0/1833346931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fa8036ff0 0x7f2fa80e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 -- 172.21.15.71:0/1833346931 >> 172.21.15.71:0/1833346931 conn(0x7f2fa8020020 msgr2=0x7f2fa800c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 -- 172.21.15.71:0/1833346931 shutdown_connections 2023-10-26T05:47:31.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.916+0000 7f2f9affd640 1 -- 172.21.15.71:0/1833346931 wait complete. 2023-10-26T05:47:31.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-26T05:47:31.933 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-26T05:47:31.934 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.932+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0f300050d0 con 0x7f0f500cf8a0 2023-10-26T05:47:31.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.932+0000 7f0f377fe640 1 -- 172.21.15.71:0/1181375032 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f40016020 con 0x7f0f500cf8a0 2023-10-26T05:47:31.939 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f3c066470 msgr2=0x7f0f3c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f3c066470 0x7f0f3c068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f440af000 tx=0x7f0f44019510 comp rx=0 tx=0).stop 2023-10-26T05:47:31.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 msgr2=0x7f0f500b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 0x7f0f500b2fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c480 tx=0x7f0f4000c940 comp rx=0 tx=0).stop 2023-10-26T05:47:31.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-26T05:47:31.943 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 shutdown_connections 2023-10-26T05:47:31.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f50150110 0x7f0f500b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f500cf8a0 0x7f0f500b2fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f3c066470 0x7f0f3c068930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 --2- 172.21.15.71:0/1181375032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f500b0510 0x7f0f500c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 >> 172.21.15.71:0/1181375032 conn(0x7f0f5006f550 msgr2=0x7f0f500cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 shutdown_connections 2023-10-26T05:47:31.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f0f54882640 1 -- 172.21.15.71:0/1181375032 wait complete. 2023-10-26T05:47:31.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5dec04da80 con 0x7f5dec15e510 2023-10-26T05:47:31.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.936+0000 7f5de2ffd640 1 -- 172.21.15.71:0/479313230 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5de80eedf0 con 0x7f5dec15e510 2023-10-26T05:47:31.946 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5dd0066570 msgr2=0x7f5dd0068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5dd0066570 0x7f5dd0068a30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004f30 tx=0x7f5ddc009040 comp rx=0 tx=0).stop 2023-10-26T05:47:31.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec15e510 msgr2=0x7f5dec12c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec15e510 0x7f5dec12c190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5dec15ad40 tx=0x7f5de80011b0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 shutdown_connections 2023-10-26T05:47:31.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dec15e510 0x7f5dec12c190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dec15c5e0 0x7f5dec1568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5dd0066570 0x7f5dd0068a30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 --2- 172.21.15.71:0/479313230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd800a500 0x7f5dec15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 >> 172.21.15.71:0/479313230 conn(0x7f5dec06c6c0 msgr2=0x7f5dec0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 shutdown_connections 2023-10-26T05:47:31.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f5df302c640 1 -- 172.21.15.71:0/479313230 wait complete. 2023-10-26T05:47:31.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f16380057e0 con 0x7f1650100b50 2023-10-26T05:47:31.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.940+0000 7f16557fa640 1 -- 172.21.15.71:0/2295396426 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f164c0eeab0 con 0x7f1650100b50 2023-10-26T05:47:31.950 INFO:teuthology.orchestra.run.smithi071.stdout:55834574855 2023-10-26T05:47:31.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f163c066470 msgr2=0x7f163c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f163c066470 0x7f163c068930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1658068e70 tx=0x7f1658077c90 comp rx=0 tx=0).stop 2023-10-26T05:47:31.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 msgr2=0x7f165004c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 0x7f165004c1d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f164c009850 tx=0x7f164c004620 comp rx=0 tx=0).stop 2023-10-26T05:47:31.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 shutdown_connections 2023-10-26T05:47:31.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1650037b20 0x7f165004c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1650100b50 0x7f165004c1d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f163c066470 0x7f163c068930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 --2- 172.21.15.71:0/2295396426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16500535c0 0x7f16500f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 >> 172.21.15.71:0/2295396426 conn(0x7f1650009f90 msgr2=0x7f16500485d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 shutdown_connections 2023-10-26T05:47:31.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.944+0000 7f165da2e640 1 -- 172.21.15.71:0/2295396426 wait complete. 2023-10-26T05:47:31.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-26T05:47:31.954 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.948+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f598415ac40 con 0x7f598406a690 2023-10-26T05:47:31.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.948+0000 7f5979ffb640 1 -- 172.21.15.71:0/3859597047 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59800ee020 con 0x7f598406a690 2023-10-26T05:47:31.955 INFO:teuthology.orchestra.run.smithi071.stdout:55834574854 2023-10-26T05:47:31.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59680664a0 msgr2=0x7f5968068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59680664a0 0x7f5968068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5984152d80 tx=0x7f597400b390 comp rx=0 tx=0).stop 2023-10-26T05:47:31.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 msgr2=0x7f598414e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:31.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 0x7f598414e460 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f598001b500 tx=0x7f59800a6650 comp rx=0 tx=0).stop 2023-10-26T05:47:31.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 shutdown_connections 2023-10-26T05:47:31.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5984097a70 0x7f598414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59840b0530 0x7f598413eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59680664a0 0x7f5968068960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 --2- 172.21.15.71:0/3859597047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f598406a690 0x7f598414e460 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:31.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 >> 172.21.15.71:0/3859597047 conn(0x7f59840b2bc0 msgr2=0x7f5984153cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:31.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 shutdown_connections 2023-10-26T05:47:31.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:31.952+0000 7f598a466640 1 -- 172.21.15.71:0/3859597047 wait complete. 2023-10-26T05:47:31.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-26T05:47:31.966 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-26T05:47:31.970 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-26T05:47:31.976 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-26T05:47:31.977 DEBUG:teuthology.parallel:result is None 2023-10-26T05:47:31.978 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-26T05:47:31.978 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:32.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/551413728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0850 msgr2=0x7f86e815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 --2- 172.21.15.71:0/551413728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0850 0x7f86e815b3a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009840 tx=0x7f86dc01c770 comp rx=0 tx=0).stop 2023-10-26T05:47:32.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/551413728 shutdown_connections 2023-10-26T05:47:32.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 --2- 172.21.15.71:0/551413728 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86e80cd880 0x7f86e815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 --2- 172.21.15.71:0/551413728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0850 0x7f86e815b3a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/551413728 >> 172.21.15.71:0/551413728 conn(0x7f86e80cdd80 msgr2=0x7f86e80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:32.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/551413728 shutdown_connections 2023-10-26T05:47:32.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/551413728 wait complete. 2023-10-26T05:47:32.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 Processor -- start 2023-10-26T05:47:32.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- start start 2023-10-26T05:47:32.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 0x7f86e81544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0470 0x7f86e80ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86e80d0850 0x7f86e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c9710 con 0x7f86e80cd880 2023-10-26T05:47:32.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e815e420 con 0x7f86e80d0470 2023-10-26T05:47:32.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c9f90 con 0x7f86e80d0850 2023-10-26T05:47:32.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 0x7f86e81544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 0x7f86e81544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55666/0 (socket says 172.21.15.71:55666) 2023-10-26T05:47:32.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 -- 172.21.15.71:0/2355776186 learned_addr learned my addr 172.21.15.71:0/2355776186 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:32.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e7fff640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86e80d0850 0x7f86e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e6ffd640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0470 0x7f86e80ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 -- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86e80d0850 msgr2=0x7f86e815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86e80d0850 0x7f86e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 -- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0470 msgr2=0x7f86e80ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0470 0x7f86e80ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 -- 172.21.15.71:0/2355776186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009820 con 0x7f86e80cd880 2023-10-26T05:47:32.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86e77fe640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 0x7f86e81544b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c430 tx=0x7f86d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:32.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011790 con 0x7f86e80cd880 2023-10-26T05:47:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86d8011a00 con 0x7f86e80cd880 2023-10-26T05:47:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86dc0094a0 con 0x7f86e80cd880 2023-10-26T05:47:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011e10 con 0x7f86e80cd880 2023-10-26T05:47:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e815e1f0 con 0x7f86e80cd880 2023-10-26T05:47:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.124+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e8066b10 con 0x7f86e80cd880 2023-10-26T05:47:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.128+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86d801c460 con 0x7f86e80cd880 2023-10-26T05:47:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.128+0000 7f86cffff640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86d4066470 0x7f86d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.128+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86d8083fe0 con 0x7f86e80cd880 2023-10-26T05:47:32.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.128+0000 7f86e6ffd640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86d4066470 0x7f86d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.128+0000 7f86e6ffd640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86d4066470 0x7f86d4068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0af000 tx=0x7f86dc01afb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:32.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.132+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86d80513e0 con 0x7f86e80cd880 2023-10-26T05:47:32.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.268+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86e80721e0 con 0x7f86d4066470 2023-10-26T05:47:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.288+0000 7f86cffff640 1 -- 172.21.15.71:0/2355776186 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260176 (secure 0 0 0) 0x7f86e80721e0 con 0x7f86d4066470 2023-10-26T05:47:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:32.297 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:32.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.288+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86d4066470 msgr2=0x7f86d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.288+0000 7f86edd81640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86d4066470 0x7f86d4068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0af000 tx=0x7f86dc01afb0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 msgr2=0x7f86e81544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 0x7f86e81544b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c430 tx=0x7f86d800c8f0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 shutdown_connections 2023-10-26T05:47:32.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86e80d0850 0x7f86e815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e80d0470 0x7f86e80ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86d4066470 0x7f86d4068930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 --2- 172.21.15.71:0/2355776186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e80cd880 0x7f86e81544b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 >> 172.21.15.71:0/2355776186 conn(0x7f86e80cdd80 msgr2=0x7f86e80b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:32.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 shutdown_connections 2023-10-26T05:47:32.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.292+0000 7f86edd81640 1 -- 172.21.15.71:0/2355776186 wait complete. 2023-10-26T05:47:32.311 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-26T05:47:31.414430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248540,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.379089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:22.078297+0000","last_became_peered":"2023-10-26T05:47:22.078297+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032989+0000","last_change":"2023-10-26T05:47:22.050802+0000","last_active":"2023-10-26T05:47:23.032989+0000","last_peered":"2023-10-26T05:47:23.032989+0000","last_clean":"2023-10-26T05:47:23.032989+0000","last_became_active":"2023-10-26T05:47:22.049800+0000","last_became_peered":"2023-10-26T05:47:22.049800+0000","last_unstale":"2023-10-26T05:47:23.032989+0000","last_undegraded":"2023-10-26T05:47:23.032989+0000","last_fullsized":"2023-10-26T05:47:23.032989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595881+0000","last_change":"2023-10-26T05:47:22.059994+0000","last_active":"2023-10-26T05:47:23.595881+0000","last_peered":"2023-10-26T05:47:23.595881+0000","last_clean":"2023-10-26T05:47:23.595881+0000","last_became_active":"2023-10-26T05:47:22.058233+0000","last_became_peered":"2023-10-26T05:47:22.058233+0000","last_unstale":"2023-10-26T05:47:23.595881+0000","last_undegraded":"2023-10-26T05:47:23.595881+0000","last_fullsized":"2023-10-26T05:47:23.595881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:26.807782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296464+0000","last_change":"2023-10-26T05:47:22.069835+0000","last_active":"2023-10-26T05:47:23.296464+0000","last_peered":"2023-10-26T05:47:23.296464+0000","last_clean":"2023-10-26T05:47:23.296464+0000","last_became_active":"2023-10-26T05:47:22.069004+0000","last_became_peered":"2023-10-26T05:47:22.069004+0000","last_unstale":"2023-10-26T05:47:23.296464+0000","last_undegraded":"2023-10-26T05:47:23.296464+0000","last_fullsized":"2023-10-26T05:47:23.296464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299356+0000","last_change":"2023-10-26T05:47:22.074037+0000","last_active":"2023-10-26T05:47:23.299356+0000","last_peered":"2023-10-26T05:47:23.299356+0000","last_clean":"2023-10-26T05:47:23.299356+0000","last_became_active":"2023-10-26T05:47:22.073426+0000","last_became_peered":"2023-10-26T05:47:22.073426+0000","last_unstale":"2023-10-26T05:47:23.299356+0000","last_undegraded":"2023-10-26T05:47:23.299356+0000","last_fullsized":"2023-10-26T05:47:23.299356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297524+0000","last_change":"2023-10-26T05:47:22.056648+0000","last_active":"2023-10-26T05:47:23.297524+0000","last_peered":"2023-10-26T05:47:23.297524+0000","last_clean":"2023-10-26T05:47:23.297524+0000","last_became_active":"2023-10-26T05:47:22.051034+0000","last_became_peered":"2023-10-26T05:47:22.051034+0000","last_unstale":"2023-10-26T05:47:23.297524+0000","last_undegraded":"2023-10-26T05:47:23.297524+0000","last_fullsized":"2023-10-26T05:47:23.297524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300018+0000","last_change":"2023-10-26T05:47:22.073286+0000","last_active":"2023-10-26T05:47:23.300018+0000","last_peered":"2023-10-26T05:47:23.300018+0000","last_clean":"2023-10-26T05:47:23.300018+0000","last_became_active":"2023-10-26T05:47:22.072189+0000","last_became_peered":"2023-10-26T05:47:22.072189+0000","last_unstale":"2023-10-26T05:47:23.300018+0000","last_undegraded":"2023-10-26T05:47:23.300018+0000","last_fullsized":"2023-10-26T05:47:23.300018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:43.485135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290644+0000","last_change":"2023-10-26T05:47:22.054592+0000","last_active":"2023-10-26T05:47:23.290644+0000","last_peered":"2023-10-26T05:47:23.290644+0000","last_clean":"2023-10-26T05:47:23.290644+0000","last_became_active":"2023-10-26T05:47:22.048761+0000","last_became_peered":"2023-10-26T05:47:22.048761+0000","last_unstale":"2023-10-26T05:47:23.290644+0000","last_undegraded":"2023-10-26T05:47:23.290644+0000","last_fullsized":"2023-10-26T05:47:23.290644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296847+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.296847+0000","last_peered":"2023-10-26T05:47:23.296847+0000","last_clean":"2023-10-26T05:47:23.296847+0000","last_became_active":"2023-10-26T05:47:22.066674+0000","last_became_peered":"2023-10-26T05:47:22.066674+0000","last_unstale":"2023-10-26T05:47:23.296847+0000","last_undegraded":"2023-10-26T05:47:23.296847+0000","last_fullsized":"2023-10-26T05:47:23.296847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298450+0000","last_change":"2023-10-26T05:47:22.054456+0000","last_active":"2023-10-26T05:47:23.298450+0000","last_peered":"2023-10-26T05:47:23.298450+0000","last_clean":"2023-10-26T05:47:23.298450+0000","last_became_active":"2023-10-26T05:47:22.053876+0000","last_became_peered":"2023-10-26T05:47:22.053876+0000","last_unstale":"2023-10-26T05:47:23.298450+0000","last_undegraded":"2023-10-26T05:47:23.298450+0000","last_fullsized":"2023-10-26T05:47:23.298450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298522+0000","last_change":"2023-10-26T05:47:22.061734+0000","last_active":"2023-10-26T05:47:23.298522+0000","last_peered":"2023-10-26T05:47:23.298522+0000","last_clean":"2023-10-26T05:47:23.298522+0000","last_became_active":"2023-10-26T05:47:22.061125+0000","last_became_peered":"2023-10-26T05:47:22.061125+0000","last_unstale":"2023-10-26T05:47:23.298522+0000","last_undegraded":"2023-10-26T05:47:23.298522+0000","last_fullsized":"2023-10-26T05:47:23.298522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039953+0000","last_change":"2023-10-26T05:47:22.059027+0000","last_active":"2023-10-26T05:47:23.039953+0000","last_peered":"2023-10-26T05:47:23.039953+0000","last_clean":"2023-10-26T05:47:23.039953+0000","last_became_active":"2023-10-26T05:47:22.058395+0000","last_became_peered":"2023-10-26T05:47:22.058395+0000","last_unstale":"2023-10-26T05:47:23.039953+0000","last_undegraded":"2023-10-26T05:47:23.039953+0000","last_fullsized":"2023-10-26T05:47:23.039953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:26.058039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291056+0000","last_change":"2023-10-26T05:47:22.054240+0000","last_active":"2023-10-26T05:47:23.291056+0000","last_peered":"2023-10-26T05:47:23.291056+0000","last_clean":"2023-10-26T05:47:23.291056+0000","last_became_active":"2023-10-26T05:47:22.047954+0000","last_became_peered":"2023-10-26T05:47:22.047954+0000","last_unstale":"2023-10-26T05:47:23.291056+0000","last_undegraded":"2023-10-26T05:47:23.291056+0000","last_fullsized":"2023-10-26T05:47:23.291056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595789+0000","last_change":"2023-10-26T05:47:22.046495+0000","last_active":"2023-10-26T05:47:23.595789+0000","last_peered":"2023-10-26T05:47:23.595789+0000","last_clean":"2023-10-26T05:47:23.595789+0000","last_became_active":"2023-10-26T05:47:22.045867+0000","last_became_peered":"2023-10-26T05:47:22.045867+0000","last_unstale":"2023-10-26T05:47:23.595789+0000","last_undegraded":"2023-10-26T05:47:23.595789+0000","last_fullsized":"2023-10-26T05:47:23.595789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:36.941020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033315+0000","last_change":"2023-10-26T05:47:22.058776+0000","last_active":"2023-10-26T05:47:23.033315+0000","last_peered":"2023-10-26T05:47:23.033315+0000","last_clean":"2023-10-26T05:47:23.033315+0000","last_became_active":"2023-10-26T05:47:22.057978+0000","last_became_peered":"2023-10-26T05:47:22.057978+0000","last_unstale":"2023-10-26T05:47:23.033315+0000","last_undegraded":"2023-10-26T05:47:23.033315+0000","last_fullsized":"2023-10-26T05:47:23.033315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297178+0000","last_change":"2023-10-26T05:47:22.055995+0000","last_active":"2023-10-26T05:47:23.297178+0000","last_peered":"2023-10-26T05:47:23.297178+0000","last_clean":"2023-10-26T05:47:23.297178+0000","last_became_active":"2023-10-26T05:47:22.049810+0000","last_became_peered":"2023-10-26T05:47:22.049810+0000","last_unstale":"2023-10-26T05:47:23.297178+0000","last_undegraded":"2023-10-26T05:47:23.297178+0000","last_fullsized":"2023-10-26T05:47:23.297178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299268+0000","last_change":"2023-10-26T05:47:22.071995+0000","last_active":"2023-10-26T05:47:23.299268+0000","last_peered":"2023-10-26T05:47:23.299268+0000","last_clean":"2023-10-26T05:47:23.299268+0000","last_became_active":"2023-10-26T05:47:22.070349+0000","last_became_peered":"2023-10-26T05:47:22.070349+0000","last_unstale":"2023-10-26T05:47:23.299268+0000","last_undegraded":"2023-10-26T05:47:23.299268+0000","last_fullsized":"2023-10-26T05:47:23.299268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033881+0000","last_change":"2023-10-26T05:47:22.051874+0000","last_active":"2023-10-26T05:47:23.033881+0000","last_peered":"2023-10-26T05:47:23.033881+0000","last_clean":"2023-10-26T05:47:23.033881+0000","last_became_active":"2023-10-26T05:47:22.051002+0000","last_became_peered":"2023-10-26T05:47:22.051002+0000","last_unstale":"2023-10-26T05:47:23.033881+0000","last_undegraded":"2023-10-26T05:47:23.033881+0000","last_fullsized":"2023-10-26T05:47:23.033881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297488+0000","last_change":"2023-10-26T05:47:22.080998+0000","last_active":"2023-10-26T05:47:23.297488+0000","last_peered":"2023-10-26T05:47:23.297488+0000","last_clean":"2023-10-26T05:47:23.297488+0000","last_became_active":"2023-10-26T05:47:22.078649+0000","last_became_peered":"2023-10-26T05:47:22.078649+0000","last_unstale":"2023-10-26T05:47:23.297488+0000","last_undegraded":"2023-10-26T05:47:23.297488+0000","last_fullsized":"2023-10-26T05:47:23.297488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298814+0000","last_change":"2023-10-26T05:47:22.052419+0000","last_active":"2023-10-26T05:47:23.298814+0000","last_peered":"2023-10-26T05:47:23.298814+0000","last_clean":"2023-10-26T05:47:23.298814+0000","last_became_active":"2023-10-26T05:47:22.051751+0000","last_became_peered":"2023-10-26T05:47:22.051751+0000","last_unstale":"2023-10-26T05:47:23.298814+0000","last_undegraded":"2023-10-26T05:47:23.298814+0000","last_fullsized":"2023-10-26T05:47:23.298814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596450+0000","last_change":"2023-10-26T05:47:22.073364+0000","last_active":"2023-10-26T05:47:23.596450+0000","last_peered":"2023-10-26T05:47:23.596450+0000","last_clean":"2023-10-26T05:47:23.596450+0000","last_became_active":"2023-10-26T05:47:22.072258+0000","last_became_peered":"2023-10-26T05:47:22.072258+0000","last_unstale":"2023-10-26T05:47:23.596450+0000","last_undegraded":"2023-10-26T05:47:23.596450+0000","last_fullsized":"2023-10-26T05:47:23.596450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299915+0000","last_change":"2023-10-26T05:47:22.075501+0000","last_active":"2023-10-26T05:47:23.299915+0000","last_peered":"2023-10-26T05:47:23.299915+0000","last_clean":"2023-10-26T05:47:23.299915+0000","last_became_active":"2023-10-26T05:47:22.074808+0000","last_became_peered":"2023-10-26T05:47:22.074808+0000","last_unstale":"2023-10-26T05:47:23.299915+0000","last_undegraded":"2023-10-26T05:47:23.299915+0000","last_fullsized":"2023-10-26T05:47:23.299915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291547+0000","last_change":"2023-10-26T05:47:22.061162+0000","last_active":"2023-10-26T05:47:23.291547+0000","last_peered":"2023-10-26T05:47:23.291547+0000","last_clean":"2023-10-26T05:47:23.291547+0000","last_became_active":"2023-10-26T05:47:22.053623+0000","last_became_peered":"2023-10-26T05:47:22.053623+0000","last_unstale":"2023-10-26T05:47:23.291547+0000","last_undegraded":"2023-10-26T05:47:23.291547+0000","last_fullsized":"2023-10-26T05:47:23.291547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296810+0000","last_change":"2023-10-26T05:47:22.063651+0000","last_active":"2023-10-26T05:47:23.296810+0000","last_peered":"2023-10-26T05:47:23.296810+0000","last_clean":"2023-10-26T05:47:23.296810+0000","last_became_active":"2023-10-26T05:47:22.062599+0000","last_became_peered":"2023-10-26T05:47:22.062599+0000","last_unstale":"2023-10-26T05:47:23.296810+0000","last_undegraded":"2023-10-26T05:47:23.296810+0000","last_fullsized":"2023-10-26T05:47:23.296810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290912+0000","last_change":"2023-10-26T05:47:22.053793+0000","last_active":"2023-10-26T05:47:23.290912+0000","last_peered":"2023-10-26T05:47:23.290912+0000","last_clean":"2023-10-26T05:47:23.290912+0000","last_became_active":"2023-10-26T05:47:22.052294+0000","last_became_peered":"2023-10-26T05:47:22.052294+0000","last_unstale":"2023-10-26T05:47:23.290912+0000","last_undegraded":"2023-10-26T05:47:23.290912+0000","last_fullsized":"2023-10-26T05:47:23.290912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298905+0000","last_change":"2023-10-26T05:47:22.078582+0000","last_active":"2023-10-26T05:47:23.298905+0000","last_peered":"2023-10-26T05:47:23.298905+0000","last_clean":"2023-10-26T05:47:23.298905+0000","last_became_active":"2023-10-26T05:47:22.077883+0000","last_became_peered":"2023-10-26T05:47:22.077883+0000","last_unstale":"2023-10-26T05:47:23.298905+0000","last_undegraded":"2023-10-26T05:47:23.298905+0000","last_fullsized":"2023-10-26T05:47:23.298905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033302+0000","last_change":"2023-10-26T05:47:22.057439+0000","last_active":"2023-10-26T05:47:23.033302+0000","last_peered":"2023-10-26T05:47:23.033302+0000","last_clean":"2023-10-26T05:47:23.033302+0000","last_became_active":"2023-10-26T05:47:22.056731+0000","last_became_peered":"2023-10-26T05:47:22.056731+0000","last_unstale":"2023-10-26T05:47:23.033302+0000","last_undegraded":"2023-10-26T05:47:23.033302+0000","last_fullsized":"2023-10-26T05:47:23.033302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291223+0000","last_change":"2023-10-26T05:47:22.065579+0000","last_active":"2023-10-26T05:47:23.291223+0000","last_peered":"2023-10-26T05:47:23.291223+0000","last_clean":"2023-10-26T05:47:23.291223+0000","last_became_active":"2023-10-26T05:47:22.063798+0000","last_became_peered":"2023-10-26T05:47:22.063798+0000","last_unstale":"2023-10-26T05:47:23.291223+0000","last_undegraded":"2023-10-26T05:47:23.291223+0000","last_fullsized":"2023-10-26T05:47:23.291223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596104+0000","last_change":"2023-10-26T05:47:22.056700+0000","last_active":"2023-10-26T05:47:23.596104+0000","last_peered":"2023-10-26T05:47:23.596104+0000","last_clean":"2023-10-26T05:47:23.596104+0000","last_became_active":"2023-10-26T05:47:22.056216+0000","last_became_peered":"2023-10-26T05:47:22.056216+0000","last_unstale":"2023-10-26T05:47:23.596104+0000","last_undegraded":"2023-10-26T05:47:23.596104+0000","last_fullsized":"2023-10-26T05:47:23.596104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297162+0000","last_change":"2023-10-26T05:47:22.061465+0000","last_active":"2023-10-26T05:47:23.297162+0000","last_peered":"2023-10-26T05:47:23.297162+0000","last_clean":"2023-10-26T05:47:23.297162+0000","last_became_active":"2023-10-26T05:47:22.060699+0000","last_became_peered":"2023-10-26T05:47:22.060699+0000","last_unstale":"2023-10-26T05:47:23.297162+0000","last_undegraded":"2023-10-26T05:47:23.297162+0000","last_fullsized":"2023-10-26T05:47:23.297162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033512+0000","last_change":"2023-10-26T05:47:22.058403+0000","last_active":"2023-10-26T05:47:23.033512+0000","last_peered":"2023-10-26T05:47:23.033512+0000","last_clean":"2023-10-26T05:47:23.033512+0000","last_became_active":"2023-10-26T05:47:22.057792+0000","last_became_peered":"2023-10-26T05:47:22.057792+0000","last_unstale":"2023-10-26T05:47:23.033512+0000","last_undegraded":"2023-10-26T05:47:23.033512+0000","last_fullsized":"2023-10-26T05:47:23.033512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299297+0000","last_change":"2023-10-26T05:47:22.059009+0000","last_active":"2023-10-26T05:47:23.299297+0000","last_peered":"2023-10-26T05:47:23.299297+0000","last_clean":"2023-10-26T05:47:23.299297+0000","last_became_active":"2023-10-26T05:47:22.057498+0000","last_became_peered":"2023-10-26T05:47:22.057498+0000","last_unstale":"2023-10-26T05:47:23.299297+0000","last_undegraded":"2023-10-26T05:47:23.299297+0000","last_fullsized":"2023-10-26T05:47:23.299297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040301+0000","last_change":"2023-10-26T05:47:22.060276+0000","last_active":"2023-10-26T05:47:23.040301+0000","last_peered":"2023-10-26T05:47:23.040301+0000","last_clean":"2023-10-26T05:47:23.040301+0000","last_became_active":"2023-10-26T05:47:22.059444+0000","last_became_peered":"2023-10-26T05:47:22.059444+0000","last_unstale":"2023-10-26T05:47:23.040301+0000","last_undegraded":"2023-10-26T05:47:23.040301+0000","last_fullsized":"2023-10-26T05:47:23.040301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291438+0000","last_change":"2023-10-26T05:47:21.083390+0000","last_active":"2023-10-26T05:47:23.291438+0000","last_peered":"2023-10-26T05:47:23.291438+0000","last_clean":"2023-10-26T05:47:23.291438+0000","last_became_active":"2023-10-26T05:47:21.083255+0000","last_became_peered":"2023-10-26T05:47:21.083255+0000","last_unstale":"2023-10-26T05:47:23.291438+0000","last_undegraded":"2023-10-26T05:47:23.291438+0000","last_fullsized":"2023-10-26T05:47:23.291438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595999+0000","last_change":"2023-10-26T05:47:22.042083+0000","last_active":"2023-10-26T05:47:23.595999+0000","last_peered":"2023-10-26T05:47:23.595999+0000","last_clean":"2023-10-26T05:47:23.595999+0000","last_became_active":"2023-10-26T05:47:22.041458+0000","last_became_peered":"2023-10-26T05:47:22.041458+0000","last_unstale":"2023-10-26T05:47:23.595999+0000","last_undegraded":"2023-10-26T05:47:23.595999+0000","last_fullsized":"2023-10-26T05:47:23.595999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:18.110818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298965+0000","last_change":"2023-10-26T05:47:21.090954+0000","last_active":"2023-10-26T05:47:23.298965+0000","last_peered":"2023-10-26T05:47:23.298965+0000","last_clean":"2023-10-26T05:47:23.298965+0000","last_became_active":"2023-10-26T05:47:21.086959+0000","last_became_peered":"2023-10-26T05:47:21.086959+0000","last_unstale":"2023-10-26T05:47:23.298965+0000","last_undegraded":"2023-10-26T05:47:23.298965+0000","last_fullsized":"2023-10-26T05:47:23.298965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291059+0000","last_change":"2023-10-26T05:47:21.078842+0000","last_active":"2023-10-26T05:47:23.291059+0000","last_peered":"2023-10-26T05:47:23.291059+0000","last_clean":"2023-10-26T05:47:23.291059+0000","last_became_active":"2023-10-26T05:47:21.078498+0000","last_became_peered":"2023-10-26T05:47:21.078498+0000","last_unstale":"2023-10-26T05:47:23.291059+0000","last_undegraded":"2023-10-26T05:47:23.291059+0000","last_fullsized":"2023-10-26T05:47:23.291059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291157+0000","last_change":"2023-10-26T05:47:22.054327+0000","last_active":"2023-10-26T05:47:23.291157+0000","last_peered":"2023-10-26T05:47:23.291157+0000","last_clean":"2023-10-26T05:47:23.291157+0000","last_became_active":"2023-10-26T05:47:22.053185+0000","last_became_peered":"2023-10-26T05:47:22.053185+0000","last_unstale":"2023-10-26T05:47:23.291157+0000","last_undegraded":"2023-10-26T05:47:23.291157+0000","last_fullsized":"2023-10-26T05:47:23.291157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297471+0000","last_change":"2023-10-26T05:47:21.084041+0000","last_active":"2023-10-26T05:47:23.297471+0000","last_peered":"2023-10-26T05:47:23.297471+0000","last_clean":"2023-10-26T05:47:23.297471+0000","last_became_active":"2023-10-26T05:47:21.082189+0000","last_became_peered":"2023-10-26T05:47:21.082189+0000","last_unstale":"2023-10-26T05:47:23.297471+0000","last_undegraded":"2023-10-26T05:47:23.297471+0000","last_fullsized":"2023-10-26T05:47:23.297471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032584+0000","last_change":"2023-10-26T05:47:22.063104+0000","last_active":"2023-10-26T05:47:23.032584+0000","last_peered":"2023-10-26T05:47:23.032584+0000","last_clean":"2023-10-26T05:47:23.032584+0000","last_became_active":"2023-10-26T05:47:22.062483+0000","last_became_peered":"2023-10-26T05:47:22.062483+0000","last_unstale":"2023-10-26T05:47:23.032584+0000","last_undegraded":"2023-10-26T05:47:23.032584+0000","last_fullsized":"2023-10-26T05:47:23.032584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299320+0000","last_change":"2023-10-26T05:47:21.087323+0000","last_active":"2023-10-26T05:47:23.299320+0000","last_peered":"2023-10-26T05:47:23.299320+0000","last_clean":"2023-10-26T05:47:23.299320+0000","last_became_active":"2023-10-26T05:47:21.086686+0000","last_became_peered":"2023-10-26T05:47:21.086686+0000","last_unstale":"2023-10-26T05:47:23.299320+0000","last_undegraded":"2023-10-26T05:47:23.299320+0000","last_fullsized":"2023-10-26T05:47:23.299320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299593+0000","last_change":"2023-10-26T05:47:22.072429+0000","last_active":"2023-10-26T05:47:23.299593+0000","last_peered":"2023-10-26T05:47:23.299593+0000","last_clean":"2023-10-26T05:47:23.299593+0000","last_became_active":"2023-10-26T05:47:22.071355+0000","last_became_peered":"2023-10-26T05:47:22.071355+0000","last_unstale":"2023-10-26T05:47:23.299593+0000","last_undegraded":"2023-10-26T05:47:23.299593+0000","last_fullsized":"2023-10-26T05:47:23.299593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300874+0000","last_change":"2023-10-26T05:47:21.096076+0000","last_active":"2023-10-26T05:47:23.300874+0000","last_peered":"2023-10-26T05:47:23.300874+0000","last_clean":"2023-10-26T05:47:23.300874+0000","last_became_active":"2023-10-26T05:47:21.092744+0000","last_became_peered":"2023-10-26T05:47:21.092744+0000","last_unstale":"2023-10-26T05:47:23.300874+0000","last_undegraded":"2023-10-26T05:47:23.300874+0000","last_fullsized":"2023-10-26T05:47:23.300874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:26.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596655+0000","last_change":"2023-10-26T05:47:22.055264+0000","last_active":"2023-10-26T05:47:23.596655+0000","last_peered":"2023-10-26T05:47:23.596655+0000","last_clean":"2023-10-26T05:47:23.596655+0000","last_became_active":"2023-10-26T05:47:22.054516+0000","last_became_peered":"2023-10-26T05:47:22.054516+0000","last_unstale":"2023-10-26T05:47:23.596655+0000","last_undegraded":"2023-10-26T05:47:23.596655+0000","last_fullsized":"2023-10-26T05:47:23.596655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033126+0000","last_change":"2023-10-26T05:47:21.069747+0000","last_active":"2023-10-26T05:47:23.033126+0000","last_peered":"2023-10-26T05:47:23.033126+0000","last_clean":"2023-10-26T05:47:23.033126+0000","last_became_active":"2023-10-26T05:47:21.069525+0000","last_became_peered":"2023-10-26T05:47:21.069525+0000","last_unstale":"2023-10-26T05:47:23.033126+0000","last_undegraded":"2023-10-26T05:47:23.033126+0000","last_fullsized":"2023-10-26T05:47:23.033126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039554+0000","last_change":"2023-10-26T05:47:22.070886+0000","last_active":"2023-10-26T05:47:23.039554+0000","last_peered":"2023-10-26T05:47:23.039554+0000","last_clean":"2023-10-26T05:47:23.039554+0000","last_became_active":"2023-10-26T05:47:22.068908+0000","last_became_peered":"2023-10-26T05:47:22.068908+0000","last_unstale":"2023-10-26T05:47:23.039554+0000","last_undegraded":"2023-10-26T05:47:23.039554+0000","last_fullsized":"2023-10-26T05:47:23.039554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:31.276801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297743+0000","last_change":"2023-10-26T05:47:21.085627+0000","last_active":"2023-10-26T05:47:23.297743+0000","last_peered":"2023-10-26T05:47:23.297743+0000","last_clean":"2023-10-26T05:47:23.297743+0000","last_became_active":"2023-10-26T05:47:21.084306+0000","last_became_peered":"2023-10-26T05:47:21.084306+0000","last_unstale":"2023-10-26T05:47:23.297743+0000","last_undegraded":"2023-10-26T05:47:23.297743+0000","last_fullsized":"2023-10-26T05:47:23.297743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299433+0000","last_change":"2023-10-26T05:47:22.071700+0000","last_active":"2023-10-26T05:47:23.299433+0000","last_peered":"2023-10-26T05:47:23.299433+0000","last_clean":"2023-10-26T05:47:23.299433+0000","last_became_active":"2023-10-26T05:47:22.071056+0000","last_became_peered":"2023-10-26T05:47:22.071056+0000","last_unstale":"2023-10-26T05:47:23.299433+0000","last_undegraded":"2023-10-26T05:47:23.299433+0000","last_fullsized":"2023-10-26T05:47:23.299433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596128+0000","last_change":"2023-10-26T05:47:21.069670+0000","last_active":"2023-10-26T05:47:23.596128+0000","last_peered":"2023-10-26T05:47:23.596128+0000","last_clean":"2023-10-26T05:47:23.596128+0000","last_became_active":"2023-10-26T05:47:21.069297+0000","last_became_peered":"2023-10-26T05:47:21.069297+0000","last_unstale":"2023-10-26T05:47:23.596128+0000","last_undegraded":"2023-10-26T05:47:23.596128+0000","last_fullsized":"2023-10-26T05:47:23.596128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:15.639318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596815+0000","last_change":"2023-10-26T05:47:22.073645+0000","last_active":"2023-10-26T05:47:23.596815+0000","last_peered":"2023-10-26T05:47:23.596815+0000","last_clean":"2023-10-26T05:47:23.596815+0000","last_became_active":"2023-10-26T05:47:22.072989+0000","last_became_peered":"2023-10-26T05:47:22.072989+0000","last_unstale":"2023-10-26T05:47:23.596815+0000","last_undegraded":"2023-10-26T05:47:23.596815+0000","last_fullsized":"2023-10-26T05:47:23.596815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298358+0000","last_change":"2023-10-26T05:47:21.086438+0000","last_active":"2023-10-26T05:47:23.298358+0000","last_peered":"2023-10-26T05:47:23.298358+0000","last_clean":"2023-10-26T05:47:23.298358+0000","last_became_active":"2023-10-26T05:47:21.082053+0000","last_became_peered":"2023-10-26T05:47:21.082053+0000","last_unstale":"2023-10-26T05:47:23.298358+0000","last_undegraded":"2023-10-26T05:47:23.298358+0000","last_fullsized":"2023-10-26T05:47:23.298358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299687+0000","last_change":"2023-10-26T05:47:22.076444+0000","last_active":"2023-10-26T05:47:23.299687+0000","last_peered":"2023-10-26T05:47:23.299687+0000","last_clean":"2023-10-26T05:47:23.299687+0000","last_became_active":"2023-10-26T05:47:22.074572+0000","last_became_peered":"2023-10-26T05:47:22.074572+0000","last_unstale":"2023-10-26T05:47:23.299687+0000","last_undegraded":"2023-10-26T05:47:23.299687+0000","last_fullsized":"2023-10-26T05:47:23.299687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299516+0000","last_change":"2023-10-26T05:47:21.090350+0000","last_active":"2023-10-26T05:47:23.299516+0000","last_peered":"2023-10-26T05:47:23.299516+0000","last_clean":"2023-10-26T05:47:23.299516+0000","last_became_active":"2023-10-26T05:47:21.087109+0000","last_became_peered":"2023-10-26T05:47:21.087109+0000","last_unstale":"2023-10-26T05:47:23.299516+0000","last_undegraded":"2023-10-26T05:47:23.299516+0000","last_fullsized":"2023-10-26T05:47:23.299516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298687+0000","last_change":"2023-10-26T05:47:22.066147+0000","last_active":"2023-10-26T05:47:23.298687+0000","last_peered":"2023-10-26T05:47:23.298687+0000","last_clean":"2023-10-26T05:47:23.298687+0000","last_became_active":"2023-10-26T05:47:22.065497+0000","last_became_peered":"2023-10-26T05:47:22.065497+0000","last_unstale":"2023-10-26T05:47:23.298687+0000","last_undegraded":"2023-10-26T05:47:23.298687+0000","last_fullsized":"2023-10-26T05:47:23.298687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034102+0000","last_change":"2023-10-26T05:47:21.064585+0000","last_active":"2023-10-26T05:47:23.034102+0000","last_peered":"2023-10-26T05:47:23.034102+0000","last_clean":"2023-10-26T05:47:23.034102+0000","last_became_active":"2023-10-26T05:47:21.064395+0000","last_became_peered":"2023-10-26T05:47:21.064395+0000","last_unstale":"2023-10-26T05:47:23.034102+0000","last_undegraded":"2023-10-26T05:47:23.034102+0000","last_fullsized":"2023-10-26T05:47:23.034102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300524+0000","last_change":"2023-10-26T05:47:22.077148+0000","last_active":"2023-10-26T05:47:23.300524+0000","last_peered":"2023-10-26T05:47:23.300524+0000","last_clean":"2023-10-26T05:47:23.300524+0000","last_became_active":"2023-10-26T05:47:22.075279+0000","last_became_peered":"2023-10-26T05:47:22.075279+0000","last_unstale":"2023-10-26T05:47:23.300524+0000","last_undegraded":"2023-10-26T05:47:23.300524+0000","last_fullsized":"2023-10-26T05:47:23.300524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:12:08.628590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039791+0000","last_change":"2023-10-26T05:47:21.077376+0000","last_active":"2023-10-26T05:47:23.039791+0000","last_peered":"2023-10-26T05:47:23.039791+0000","last_clean":"2023-10-26T05:47:23.039791+0000","last_became_active":"2023-10-26T05:47:21.076766+0000","last_became_peered":"2023-10-26T05:47:21.076766+0000","last_unstale":"2023-10-26T05:47:23.039791+0000","last_undegraded":"2023-10-26T05:47:23.039791+0000","last_fullsized":"2023-10-26T05:47:23.039791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299945+0000","last_change":"2023-10-26T05:47:22.077007+0000","last_active":"2023-10-26T05:47:23.299945+0000","last_peered":"2023-10-26T05:47:23.299945+0000","last_clean":"2023-10-26T05:47:23.299945+0000","last_became_active":"2023-10-26T05:47:22.075297+0000","last_became_peered":"2023-10-26T05:47:22.075297+0000","last_unstale":"2023-10-26T05:47:23.299945+0000","last_undegraded":"2023-10-26T05:47:23.299945+0000","last_fullsized":"2023-10-26T05:47:23.299945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:15.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033816+0000","last_change":"2023-10-26T05:47:21.072137+0000","last_active":"2023-10-26T05:47:23.033816+0000","last_peered":"2023-10-26T05:47:23.033816+0000","last_clean":"2023-10-26T05:47:23.033816+0000","last_became_active":"2023-10-26T05:47:21.071720+0000","last_became_peered":"2023-10-26T05:47:21.071720+0000","last_unstale":"2023-10-26T05:47:23.033816+0000","last_undegraded":"2023-10-26T05:47:23.033816+0000","last_fullsized":"2023-10-26T05:47:23.033816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:09.715298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297159+0000","last_change":"2023-10-26T05:47:22.080581+0000","last_active":"2023-10-26T05:47:23.297159+0000","last_peered":"2023-10-26T05:47:23.297159+0000","last_clean":"2023-10-26T05:47:23.297159+0000","last_became_active":"2023-10-26T05:47:22.076913+0000","last_became_peered":"2023-10-26T05:47:22.076913+0000","last_unstale":"2023-10-26T05:47:23.297159+0000","last_undegraded":"2023-10-26T05:47:23.297159+0000","last_fullsized":"2023-10-26T05:47:23.297159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033027+0000","last_change":"2023-10-26T05:47:21.076982+0000","last_active":"2023-10-26T05:47:23.033027+0000","last_peered":"2023-10-26T05:47:23.033027+0000","last_clean":"2023-10-26T05:47:23.033027+0000","last_became_active":"2023-10-26T05:47:21.076695+0000","last_became_peered":"2023-10-26T05:47:21.076695+0000","last_unstale":"2023-10-26T05:47:23.033027+0000","last_undegraded":"2023-10-26T05:47:23.033027+0000","last_fullsized":"2023-10-26T05:47:23.033027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:57.904856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300280+0000","last_change":"2023-10-26T05:47:22.063246+0000","last_active":"2023-10-26T05:47:23.300280+0000","last_peered":"2023-10-26T05:47:23.300280+0000","last_clean":"2023-10-26T05:47:23.300280+0000","last_became_active":"2023-10-26T05:47:22.062499+0000","last_became_peered":"2023-10-26T05:47:22.062499+0000","last_unstale":"2023-10-26T05:47:23.300280+0000","last_undegraded":"2023-10-26T05:47:23.300280+0000","last_fullsized":"2023-10-26T05:47:23.300280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033647+0000","last_change":"2023-10-26T05:47:21.087999+0000","last_active":"2023-10-26T05:47:23.033647+0000","last_peered":"2023-10-26T05:47:23.033647+0000","last_clean":"2023-10-26T05:47:23.033647+0000","last_became_active":"2023-10-26T05:47:21.087744+0000","last_became_peered":"2023-10-26T05:47:21.087744+0000","last_unstale":"2023-10-26T05:47:23.033647+0000","last_undegraded":"2023-10-26T05:47:23.033647+0000","last_fullsized":"2023-10-26T05:47:23.033647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298194+0000","last_change":"2023-10-26T05:47:22.081324+0000","last_active":"2023-10-26T05:47:23.298194+0000","last_peered":"2023-10-26T05:47:23.298194+0000","last_clean":"2023-10-26T05:47:23.298194+0000","last_became_active":"2023-10-26T05:47:22.079073+0000","last_became_peered":"2023-10-26T05:47:22.079073+0000","last_unstale":"2023-10-26T05:47:23.298194+0000","last_undegraded":"2023-10-26T05:47:23.298194+0000","last_fullsized":"2023-10-26T05:47:23.298194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T05:47:22.066065+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:22.066065+0000","last_peered":"2023-10-26T05:47:22.066065+0000","last_clean":"2023-10-26T05:47:22.066065+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:22.066065+0000","last_undegraded":"2023-10-26T05:47:22.066065+0000","last_fullsized":"2023-10-26T05:47:22.066065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298749+0000","last_change":"2023-10-26T05:47:21.087000+0000","last_active":"2023-10-26T05:47:23.298749+0000","last_peered":"2023-10-26T05:47:23.298749+0000","last_clean":"2023-10-26T05:47:23.298749+0000","last_became_active":"2023-10-26T05:47:21.082935+0000","last_became_peered":"2023-10-26T05:47:21.082935+0000","last_unstale":"2023-10-26T05:47:23.298749+0000","last_undegraded":"2023-10-26T05:47:23.298749+0000","last_fullsized":"2023-10-26T05:47:23.298749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291206+0000","last_change":"2023-10-26T05:47:22.055580+0000","last_active":"2023-10-26T05:47:23.291206+0000","last_peered":"2023-10-26T05:47:23.291206+0000","last_clean":"2023-10-26T05:47:23.291206+0000","last_became_active":"2023-10-26T05:47:22.054587+0000","last_became_peered":"2023-10-26T05:47:22.054587+0000","last_unstale":"2023-10-26T05:47:23.291206+0000","last_undegraded":"2023-10-26T05:47:23.291206+0000","last_fullsized":"2023-10-26T05:47:23.291206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297907+0000","last_change":"2023-10-26T05:47:21.100233+0000","last_active":"2023-10-26T05:47:23.297907+0000","last_peered":"2023-10-26T05:47:23.297907+0000","last_clean":"2023-10-26T05:47:23.297907+0000","last_became_active":"2023-10-26T05:47:21.099318+0000","last_became_peered":"2023-10-26T05:47:21.099318+0000","last_unstale":"2023-10-26T05:47:23.297907+0000","last_undegraded":"2023-10-26T05:47:23.297907+0000","last_fullsized":"2023-10-26T05:47:23.297907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596567+0000","last_change":"2023-10-26T05:47:22.057190+0000","last_active":"2023-10-26T05:47:23.596567+0000","last_peered":"2023-10-26T05:47:23.596567+0000","last_clean":"2023-10-26T05:47:23.596567+0000","last_became_active":"2023-10-26T05:47:22.049709+0000","last_became_peered":"2023-10-26T05:47:22.049709+0000","last_unstale":"2023-10-26T05:47:23.596567+0000","last_undegraded":"2023-10-26T05:47:23.596567+0000","last_fullsized":"2023-10-26T05:47:23.596567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034304+0000","last_change":"2023-10-26T05:47:21.073026+0000","last_active":"2023-10-26T05:47:23.034304+0000","last_peered":"2023-10-26T05:47:23.034304+0000","last_clean":"2023-10-26T05:47:23.034304+0000","last_became_active":"2023-10-26T05:47:21.071577+0000","last_became_peered":"2023-10-26T05:47:21.071577+0000","last_unstale":"2023-10-26T05:47:23.034304+0000","last_undegraded":"2023-10-26T05:47:23.034304+0000","last_fullsized":"2023-10-26T05:47:23.034304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:25.047488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032708+0000","last_change":"2023-10-26T05:47:22.052469+0000","last_active":"2023-10-26T05:47:23.032708+0000","last_peered":"2023-10-26T05:47:23.032708+0000","last_clean":"2023-10-26T05:47:23.032708+0000","last_became_active":"2023-10-26T05:47:22.051614+0000","last_became_peered":"2023-10-26T05:47:22.051614+0000","last_unstale":"2023-10-26T05:47:23.032708+0000","last_undegraded":"2023-10-26T05:47:23.032708+0000","last_fullsized":"2023-10-26T05:47:23.032708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.301108+0000","last_change":"2023-10-26T05:47:21.096632+0000","last_active":"2023-10-26T05:47:23.301108+0000","last_peered":"2023-10-26T05:47:23.301108+0000","last_clean":"2023-10-26T05:47:23.301108+0000","last_became_active":"2023-10-26T05:47:21.093612+0000","last_became_peered":"2023-10-26T05:47:21.093612+0000","last_unstale":"2023-10-26T05:47:23.301108+0000","last_undegraded":"2023-10-26T05:47:23.301108+0000","last_fullsized":"2023-10-26T05:47:23.301108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039976+0000","last_change":"2023-10-26T05:47:22.060460+0000","last_active":"2023-10-26T05:47:23.039976+0000","last_peered":"2023-10-26T05:47:23.039976+0000","last_clean":"2023-10-26T05:47:23.039976+0000","last_became_active":"2023-10-26T05:47:22.059831+0000","last_became_peered":"2023-10-26T05:47:22.059831+0000","last_unstale":"2023-10-26T05:47:23.039976+0000","last_undegraded":"2023-10-26T05:47:23.039976+0000","last_fullsized":"2023-10-26T05:47:23.039976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033723+0000","last_change":"2023-10-26T05:47:21.071567+0000","last_active":"2023-10-26T05:47:23.033723+0000","last_peered":"2023-10-26T05:47:23.033723+0000","last_clean":"2023-10-26T05:47:23.033723+0000","last_became_active":"2023-10-26T05:47:21.071275+0000","last_became_peered":"2023-10-26T05:47:21.071275+0000","last_unstale":"2023-10-26T05:47:23.033723+0000","last_undegraded":"2023-10-26T05:47:23.033723+0000","last_fullsized":"2023-10-26T05:47:23.033723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297152+0000","last_change":"2023-10-26T05:47:22.063236+0000","last_active":"2023-10-26T05:47:23.297152+0000","last_peered":"2023-10-26T05:47:23.297152+0000","last_clean":"2023-10-26T05:47:23.297152+0000","last_became_active":"2023-10-26T05:47:22.061746+0000","last_became_peered":"2023-10-26T05:47:22.061746+0000","last_unstale":"2023-10-26T05:47:23.297152+0000","last_undegraded":"2023-10-26T05:47:23.297152+0000","last_fullsized":"2023-10-26T05:47:23.297152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:21.099849+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:21.097871+0000","last_became_peered":"2023-10-26T05:47:21.097871+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032981+0000","last_change":"2023-10-26T05:47:22.050271+0000","last_active":"2023-10-26T05:47:23.032981+0000","last_peered":"2023-10-26T05:47:23.032981+0000","last_clean":"2023-10-26T05:47:23.032981+0000","last_became_active":"2023-10-26T05:47:22.049654+0000","last_became_peered":"2023-10-26T05:47:22.049654+0000","last_unstale":"2023-10-26T05:47:23.032981+0000","last_undegraded":"2023-10-26T05:47:23.032981+0000","last_fullsized":"2023-10-26T05:47:23.032981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:24.404662+0000","last_change":"2023-10-26T05:47:21.078711+0000","last_active":"2023-10-26T05:47:24.404662+0000","last_peered":"2023-10-26T05:47:24.404662+0000","last_clean":"2023-10-26T05:47:24.404662+0000","last_became_active":"2023-10-26T05:47:21.078403+0000","last_became_peered":"2023-10-26T05:47:21.078403+0000","last_unstale":"2023-10-26T05:47:24.404662+0000","last_undegraded":"2023-10-26T05:47:24.404662+0000","last_fullsized":"2023-10-26T05:47:24.404662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290995+0000","last_change":"2023-10-26T05:47:22.065870+0000","last_active":"2023-10-26T05:47:23.290995+0000","last_peered":"2023-10-26T05:47:23.290995+0000","last_clean":"2023-10-26T05:47:23.290995+0000","last_became_active":"2023-10-26T05:47:22.065412+0000","last_became_peered":"2023-10-26T05:47:22.065412+0000","last_unstale":"2023-10-26T05:47:23.290995+0000","last_undegraded":"2023-10-26T05:47:23.290995+0000","last_fullsized":"2023-10-26T05:47:23.290995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299744+0000","last_change":"2023-10-26T05:47:21.094764+0000","last_active":"2023-10-26T05:47:23.299744+0000","last_peered":"2023-10-26T05:47:23.299744+0000","last_clean":"2023-10-26T05:47:23.299744+0000","last_became_active":"2023-10-26T05:47:21.091991+0000","last_became_peered":"2023-10-26T05:47:21.091991+0000","last_unstale":"2023-10-26T05:47:23.299744+0000","last_undegraded":"2023-10-26T05:47:23.299744+0000","last_fullsized":"2023-10-26T05:47:23.299744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596211+0000","last_change":"2023-10-26T05:47:22.059433+0000","last_active":"2023-10-26T05:47:23.596211+0000","last_peered":"2023-10-26T05:47:23.596211+0000","last_clean":"2023-10-26T05:47:23.596211+0000","last_became_active":"2023-10-26T05:47:22.054466+0000","last_became_peered":"2023-10-26T05:47:22.054466+0000","last_unstale":"2023-10-26T05:47:23.596211+0000","last_undegraded":"2023-10-26T05:47:23.596211+0000","last_fullsized":"2023-10-26T05:47:23.596211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298877+0000","last_change":"2023-10-26T05:47:21.099000+0000","last_active":"2023-10-26T05:47:23.298877+0000","last_peered":"2023-10-26T05:47:23.298877+0000","last_clean":"2023-10-26T05:47:23.298877+0000","last_became_active":"2023-10-26T05:47:21.095552+0000","last_became_peered":"2023-10-26T05:47:21.095552+0000","last_unstale":"2023-10-26T05:47:23.298877+0000","last_undegraded":"2023-10-26T05:47:23.298877+0000","last_fullsized":"2023-10-26T05:47:23.298877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:49.348590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040667+0000","last_change":"2023-10-26T05:47:22.059642+0000","last_active":"2023-10-26T05:47:23.040667+0000","last_peered":"2023-10-26T05:47:23.040667+0000","last_clean":"2023-10-26T05:47:23.040667+0000","last_became_active":"2023-10-26T05:47:22.054716+0000","last_became_peered":"2023-10-26T05:47:22.054716+0000","last_unstale":"2023-10-26T05:47:23.040667+0000","last_undegraded":"2023-10-26T05:47:23.040667+0000","last_fullsized":"2023-10-26T05:47:23.040667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:10.310768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033526+0000","last_change":"2023-10-26T05:47:21.090099+0000","last_active":"2023-10-26T05:47:23.033526+0000","last_peered":"2023-10-26T05:47:23.033526+0000","last_clean":"2023-10-26T05:47:23.033526+0000","last_became_active":"2023-10-26T05:47:21.089837+0000","last_became_peered":"2023-10-26T05:47:21.089837+0000","last_unstale":"2023-10-26T05:47:23.033526+0000","last_undegraded":"2023-10-26T05:47:23.033526+0000","last_fullsized":"2023-10-26T05:47:23.033526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290991+0000","last_change":"2023-10-26T05:47:22.046792+0000","last_active":"2023-10-26T05:47:23.290991+0000","last_peered":"2023-10-26T05:47:23.290991+0000","last_clean":"2023-10-26T05:47:23.290991+0000","last_became_active":"2023-10-26T05:47:22.045151+0000","last_became_peered":"2023-10-26T05:47:22.045151+0000","last_unstale":"2023-10-26T05:47:23.290991+0000","last_undegraded":"2023-10-26T05:47:23.290991+0000","last_fullsized":"2023-10-26T05:47:23.290991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032792+0000","last_change":"2023-10-26T05:47:21.072214+0000","last_active":"2023-10-26T05:47:23.032792+0000","last_peered":"2023-10-26T05:47:23.032792+0000","last_clean":"2023-10-26T05:47:23.032792+0000","last_became_active":"2023-10-26T05:47:21.071876+0000","last_became_peered":"2023-10-26T05:47:21.071876+0000","last_unstale":"2023-10-26T05:47:23.032792+0000","last_undegraded":"2023-10-26T05:47:23.032792+0000","last_fullsized":"2023-10-26T05:47:23.032792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291391+0000","last_change":"2023-10-26T05:47:22.066034+0000","last_active":"2023-10-26T05:47:23.291391+0000","last_peered":"2023-10-26T05:47:23.291391+0000","last_clean":"2023-10-26T05:47:23.291391+0000","last_became_active":"2023-10-26T05:47:22.064885+0000","last_became_peered":"2023-10-26T05:47:22.064885+0000","last_unstale":"2023-10-26T05:47:23.291391+0000","last_undegraded":"2023-10-26T05:47:23.291391+0000","last_fullsized":"2023-10-26T05:47:23.291391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299704+0000","last_change":"2023-10-26T05:47:21.093006+0000","last_active":"2023-10-26T05:47:23.299704+0000","last_peered":"2023-10-26T05:47:23.299704+0000","last_clean":"2023-10-26T05:47:23.299704+0000","last_became_active":"2023-10-26T05:47:21.092342+0000","last_became_peered":"2023-10-26T05:47:21.092342+0000","last_unstale":"2023-10-26T05:47:23.299704+0000","last_undegraded":"2023-10-26T05:47:23.299704+0000","last_fullsized":"2023-10-26T05:47:23.299704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:12.988430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296978+0000","last_change":"2023-10-26T05:47:22.059661+0000","last_active":"2023-10-26T05:47:23.296978+0000","last_peered":"2023-10-26T05:47:23.296978+0000","last_clean":"2023-10-26T05:47:23.296978+0000","last_became_active":"2023-10-26T05:47:22.058379+0000","last_became_peered":"2023-10-26T05:47:22.058379+0000","last_unstale":"2023-10-26T05:47:23.296978+0000","last_undegraded":"2023-10-26T05:47:23.296978+0000","last_fullsized":"2023-10-26T05:47:23.296978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297352+0000","last_change":"2023-10-26T05:47:22.045989+0000","last_active":"2023-10-26T05:47:23.297352+0000","last_peered":"2023-10-26T05:47:23.297352+0000","last_clean":"2023-10-26T05:47:23.297352+0000","last_became_active":"2023-10-26T05:47:22.044537+0000","last_became_peered":"2023-10-26T05:47:22.044537+0000","last_unstale":"2023-10-26T05:47:23.297352+0000","last_undegraded":"2023-10-26T05:47:23.297352+0000","last_fullsized":"2023-10-26T05:47:23.297352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298638+0000","last_change":"2023-10-26T05:47:21.099538+0000","last_active":"2023-10-26T05:47:23.298638+0000","last_peered":"2023-10-26T05:47:23.298638+0000","last_clean":"2023-10-26T05:47:23.298638+0000","last_became_active":"2023-10-26T05:47:21.096351+0000","last_became_peered":"2023-10-26T05:47:21.096351+0000","last_unstale":"2023-10-26T05:47:23.298638+0000","last_undegraded":"2023-10-26T05:47:23.298638+0000","last_fullsized":"2023-10-26T05:47:23.298638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:21.625489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297831+0000","last_change":"2023-10-26T05:47:22.059627+0000","last_active":"2023-10-26T05:47:23.297831+0000","last_peered":"2023-10-26T05:47:23.297831+0000","last_clean":"2023-10-26T05:47:23.297831+0000","last_became_active":"2023-10-26T05:47:22.054248+0000","last_became_peered":"2023-10-26T05:47:22.054248+0000","last_unstale":"2023-10-26T05:47:23.297831+0000","last_undegraded":"2023-10-26T05:47:23.297831+0000","last_fullsized":"2023-10-26T05:47:23.297831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300077+0000","last_change":"2023-10-26T05:47:21.097760+0000","last_active":"2023-10-26T05:47:23.300077+0000","last_peered":"2023-10-26T05:47:23.300077+0000","last_clean":"2023-10-26T05:47:23.300077+0000","last_became_active":"2023-10-26T05:47:21.093680+0000","last_became_peered":"2023-10-26T05:47:21.093680+0000","last_unstale":"2023-10-26T05:47:23.300077+0000","last_undegraded":"2023-10-26T05:47:23.300077+0000","last_fullsized":"2023-10-26T05:47:23.300077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596384+0000","last_change":"2023-10-26T05:47:22.038934+0000","last_active":"2023-10-26T05:47:23.596384+0000","last_peered":"2023-10-26T05:47:23.596384+0000","last_clean":"2023-10-26T05:47:23.596384+0000","last_became_active":"2023-10-26T05:47:22.038330+0000","last_became_peered":"2023-10-26T05:47:22.038330+0000","last_unstale":"2023-10-26T05:47:23.596384+0000","last_undegraded":"2023-10-26T05:47:23.596384+0000","last_fullsized":"2023-10-26T05:47:23.596384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290969+0000","last_change":"2023-10-26T05:47:21.070756+0000","last_active":"2023-10-26T05:47:23.290969+0000","last_peered":"2023-10-26T05:47:23.290969+0000","last_clean":"2023-10-26T05:47:23.290969+0000","last_became_active":"2023-10-26T05:47:21.070211+0000","last_became_peered":"2023-10-26T05:47:21.070211+0000","last_unstale":"2023-10-26T05:47:23.290969+0000","last_undegraded":"2023-10-26T05:47:23.290969+0000","last_fullsized":"2023-10-26T05:47:23.290969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290817+0000","last_change":"2023-10-26T05:47:22.062259+0000","last_active":"2023-10-26T05:47:23.290817+0000","last_peered":"2023-10-26T05:47:23.290817+0000","last_clean":"2023-10-26T05:47:23.290817+0000","last_became_active":"2023-10-26T05:47:22.055535+0000","last_became_peered":"2023-10-26T05:47:22.055535+0000","last_unstale":"2023-10-26T05:47:23.290817+0000","last_undegraded":"2023-10-26T05:47:23.290817+0000","last_fullsized":"2023-10-26T05:47:23.290817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596691+0000","last_change":"2023-10-26T05:47:21.084394+0000","last_active":"2023-10-26T05:47:23.596691+0000","last_peered":"2023-10-26T05:47:23.596691+0000","last_clean":"2023-10-26T05:47:23.596691+0000","last_became_active":"2023-10-26T05:47:21.083775+0000","last_became_peered":"2023-10-26T05:47:21.083775+0000","last_unstale":"2023-10-26T05:47:23.596691+0000","last_undegraded":"2023-10-26T05:47:23.596691+0000","last_fullsized":"2023-10-26T05:47:23.596691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:32.317 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:47:32.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/2114271123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 msgr2=0x7f3be80b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 --2- 172.21.15.71:0/2114271123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be80b0560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009840 tx=0x7f3bd801c740 comp rx=0 tx=0).stop 2023-10-26T05:47:32.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/2114271123 shutdown_connections 2023-10-26T05:47:32.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 --2- 172.21.15.71:0/2114271123 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3be815d560 0x7f3be815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 --2- 172.21.15.71:0/2114271123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be80b0560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/2114271123 >> 172.21.15.71:0/2114271123 conn(0x7f3be80b0f20 msgr2=0x7f3be80b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:32.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/2114271123 shutdown_connections 2023-10-26T05:47:32.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/2114271123 wait complete. 2023-10-26T05:47:32.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 Processor -- start 2023-10-26T05:47:32.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- start start 2023-10-26T05:47:32.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be80cede0 0x7f3be8143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be8148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3be815d560 0x7f3be806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8143ed0 con 0x7f3be815c720 2023-10-26T05:47:32.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8144ee0 con 0x7f3be80cede0 2023-10-26T05:47:32.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8144030 con 0x7f3be815d560 2023-10-26T05:47:32.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be8148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be8148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55684/0 (socket says 172.21.15.71:55684) 2023-10-26T05:47:32.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 -- 172.21.15.71:0/3893028094 learned_addr learned my addr 172.21.15.71:0/3893028094 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:32.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 -- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3be815d560 msgr2=0x7f3be806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3be815d560 0x7f3be806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 -- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be80cede0 msgr2=0x7f3be8143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be80cede0 0x7f3be8143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 -- 172.21.15.71:0/3893028094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80094a0 con 0x7f3be815c720 2023-10-26T05:47:32.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be77fe640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be8148170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c480 tx=0x7f3bdc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:32.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc012830 con 0x7f3be815c720 2023-10-26T05:47:32.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3bdc012a80 con 0x7f3be815c720 2023-10-26T05:47:32.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be8145370 con 0x7f3be815c720 2023-10-26T05:47:32.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc011380 con 0x7f3be815c720 2023-10-26T05:47:32.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be81442d0 con 0x7f3be815c720 2023-10-26T05:47:32.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.464+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be8066d00 con 0x7f3be815c720 2023-10-26T05:47:32.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.468+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3bdc0114e0 con 0x7f3be815c720 2023-10-26T05:47:32.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.468+0000 7f3be4ff9640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bd4066470 0x7f3bd4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.468+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bdc083ec0 con 0x7f3be815c720 2023-10-26T05:47:32.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.468+0000 7f3be7fff640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bd4066470 0x7f3bd4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.468+0000 7f3be7fff640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bd4066470 0x7f3bd4068930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80af000 tx=0x7f3bd80194e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:32.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.472+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3bdc0513d0 con 0x7f3be815c720 2023-10-26T05:47:32.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.612+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3be8145050 con 0x7f3bd4066470 2023-10-26T05:47:32.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.632+0000 7f3be4ff9640 1 -- 172.21.15.71:0/3893028094 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260176 (secure 0 0 0) 0x7f3be8145050 con 0x7f3bd4066470 2023-10-26T05:47:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:47:32.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bd4066470 msgr2=0x7f3bd4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bd4066470 0x7f3bd4068930 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3bd80af000 tx=0x7f3bd80194e0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 msgr2=0x7f3be8148170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be8148170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c480 tx=0x7f3bdc00c940 comp rx=0 tx=0).stop 2023-10-26T05:47:32.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 shutdown_connections 2023-10-26T05:47:32.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3be815d560 0x7f3be806dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bd4066470 0x7f3bd4068930 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3be815c720 0x7f3be8148170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 --2- 172.21.15.71:0/3893028094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3be80cede0 0x7f3be8143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 >> 172.21.15.71:0/3893028094 conn(0x7f3be80b0f20 msgr2=0x7f3be806eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 shutdown_connections 2023-10-26T05:47:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.636+0000 7f3bee8da640 1 -- 172.21.15.71:0/3893028094 wait complete. 2023-10-26T05:47:32.658 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-26T05:47:31.414430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248540,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.379089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:22.078297+0000","last_became_peered":"2023-10-26T05:47:22.078297+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032989+0000","last_change":"2023-10-26T05:47:22.050802+0000","last_active":"2023-10-26T05:47:23.032989+0000","last_peered":"2023-10-26T05:47:23.032989+0000","last_clean":"2023-10-26T05:47:23.032989+0000","last_became_active":"2023-10-26T05:47:22.049800+0000","last_became_peered":"2023-10-26T05:47:22.049800+0000","last_unstale":"2023-10-26T05:47:23.032989+0000","last_undegraded":"2023-10-26T05:47:23.032989+0000","last_fullsized":"2023-10-26T05:47:23.032989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595881+0000","last_change":"2023-10-26T05:47:22.059994+0000","last_active":"2023-10-26T05:47:23.595881+0000","last_peered":"2023-10-26T05:47:23.595881+0000","last_clean":"2023-10-26T05:47:23.595881+0000","last_became_active":"2023-10-26T05:47:22.058233+0000","last_became_peered":"2023-10-26T05:47:22.058233+0000","last_unstale":"2023-10-26T05:47:23.595881+0000","last_undegraded":"2023-10-26T05:47:23.595881+0000","last_fullsized":"2023-10-26T05:47:23.595881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:26.807782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296464+0000","last_change":"2023-10-26T05:47:22.069835+0000","last_active":"2023-10-26T05:47:23.296464+0000","last_peered":"2023-10-26T05:47:23.296464+0000","last_clean":"2023-10-26T05:47:23.296464+0000","last_became_active":"2023-10-26T05:47:22.069004+0000","last_became_peered":"2023-10-26T05:47:22.069004+0000","last_unstale":"2023-10-26T05:47:23.296464+0000","last_undegraded":"2023-10-26T05:47:23.296464+0000","last_fullsized":"2023-10-26T05:47:23.296464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299356+0000","last_change":"2023-10-26T05:47:22.074037+0000","last_active":"2023-10-26T05:47:23.299356+0000","last_peered":"2023-10-26T05:47:23.299356+0000","last_clean":"2023-10-26T05:47:23.299356+0000","last_became_active":"2023-10-26T05:47:22.073426+0000","last_became_peered":"2023-10-26T05:47:22.073426+0000","last_unstale":"2023-10-26T05:47:23.299356+0000","last_undegraded":"2023-10-26T05:47:23.299356+0000","last_fullsized":"2023-10-26T05:47:23.299356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297524+0000","last_change":"2023-10-26T05:47:22.056648+0000","last_active":"2023-10-26T05:47:23.297524+0000","last_peered":"2023-10-26T05:47:23.297524+0000","last_clean":"2023-10-26T05:47:23.297524+0000","last_became_active":"2023-10-26T05:47:22.051034+0000","last_became_peered":"2023-10-26T05:47:22.051034+0000","last_unstale":"2023-10-26T05:47:23.297524+0000","last_undegraded":"2023-10-26T05:47:23.297524+0000","last_fullsized":"2023-10-26T05:47:23.297524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300018+0000","last_change":"2023-10-26T05:47:22.073286+0000","last_active":"2023-10-26T05:47:23.300018+0000","last_peered":"2023-10-26T05:47:23.300018+0000","last_clean":"2023-10-26T05:47:23.300018+0000","last_became_active":"2023-10-26T05:47:22.072189+0000","last_became_peered":"2023-10-26T05:47:22.072189+0000","last_unstale":"2023-10-26T05:47:23.300018+0000","last_undegraded":"2023-10-26T05:47:23.300018+0000","last_fullsized":"2023-10-26T05:47:23.300018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:43.485135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290644+0000","last_change":"2023-10-26T05:47:22.054592+0000","last_active":"2023-10-26T05:47:23.290644+0000","last_peered":"2023-10-26T05:47:23.290644+0000","last_clean":"2023-10-26T05:47:23.290644+0000","last_became_active":"2023-10-26T05:47:22.048761+0000","last_became_peered":"2023-10-26T05:47:22.048761+0000","last_unstale":"2023-10-26T05:47:23.290644+0000","last_undegraded":"2023-10-26T05:47:23.290644+0000","last_fullsized":"2023-10-26T05:47:23.290644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296847+0000","last_change":"2023-10-26T05:47:22.080023+0000","last_active":"2023-10-26T05:47:23.296847+0000","last_peered":"2023-10-26T05:47:23.296847+0000","last_clean":"2023-10-26T05:47:23.296847+0000","last_became_active":"2023-10-26T05:47:22.066674+0000","last_became_peered":"2023-10-26T05:47:22.066674+0000","last_unstale":"2023-10-26T05:47:23.296847+0000","last_undegraded":"2023-10-26T05:47:23.296847+0000","last_fullsized":"2023-10-26T05:47:23.296847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298450+0000","last_change":"2023-10-26T05:47:22.054456+0000","last_active":"2023-10-26T05:47:23.298450+0000","last_peered":"2023-10-26T05:47:23.298450+0000","last_clean":"2023-10-26T05:47:23.298450+0000","last_became_active":"2023-10-26T05:47:22.053876+0000","last_became_peered":"2023-10-26T05:47:22.053876+0000","last_unstale":"2023-10-26T05:47:23.298450+0000","last_undegraded":"2023-10-26T05:47:23.298450+0000","last_fullsized":"2023-10-26T05:47:23.298450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298522+0000","last_change":"2023-10-26T05:47:22.061734+0000","last_active":"2023-10-26T05:47:23.298522+0000","last_peered":"2023-10-26T05:47:23.298522+0000","last_clean":"2023-10-26T05:47:23.298522+0000","last_became_active":"2023-10-26T05:47:22.061125+0000","last_became_peered":"2023-10-26T05:47:22.061125+0000","last_unstale":"2023-10-26T05:47:23.298522+0000","last_undegraded":"2023-10-26T05:47:23.298522+0000","last_fullsized":"2023-10-26T05:47:23.298522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039953+0000","last_change":"2023-10-26T05:47:22.059027+0000","last_active":"2023-10-26T05:47:23.039953+0000","last_peered":"2023-10-26T05:47:23.039953+0000","last_clean":"2023-10-26T05:47:23.039953+0000","last_became_active":"2023-10-26T05:47:22.058395+0000","last_became_peered":"2023-10-26T05:47:22.058395+0000","last_unstale":"2023-10-26T05:47:23.039953+0000","last_undegraded":"2023-10-26T05:47:23.039953+0000","last_fullsized":"2023-10-26T05:47:23.039953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:26.058039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291056+0000","last_change":"2023-10-26T05:47:22.054240+0000","last_active":"2023-10-26T05:47:23.291056+0000","last_peered":"2023-10-26T05:47:23.291056+0000","last_clean":"2023-10-26T05:47:23.291056+0000","last_became_active":"2023-10-26T05:47:22.047954+0000","last_became_peered":"2023-10-26T05:47:22.047954+0000","last_unstale":"2023-10-26T05:47:23.291056+0000","last_undegraded":"2023-10-26T05:47:23.291056+0000","last_fullsized":"2023-10-26T05:47:23.291056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595789+0000","last_change":"2023-10-26T05:47:22.046495+0000","last_active":"2023-10-26T05:47:23.595789+0000","last_peered":"2023-10-26T05:47:23.595789+0000","last_clean":"2023-10-26T05:47:23.595789+0000","last_became_active":"2023-10-26T05:47:22.045867+0000","last_became_peered":"2023-10-26T05:47:22.045867+0000","last_unstale":"2023-10-26T05:47:23.595789+0000","last_undegraded":"2023-10-26T05:47:23.595789+0000","last_fullsized":"2023-10-26T05:47:23.595789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:36.941020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033315+0000","last_change":"2023-10-26T05:47:22.058776+0000","last_active":"2023-10-26T05:47:23.033315+0000","last_peered":"2023-10-26T05:47:23.033315+0000","last_clean":"2023-10-26T05:47:23.033315+0000","last_became_active":"2023-10-26T05:47:22.057978+0000","last_became_peered":"2023-10-26T05:47:22.057978+0000","last_unstale":"2023-10-26T05:47:23.033315+0000","last_undegraded":"2023-10-26T05:47:23.033315+0000","last_fullsized":"2023-10-26T05:47:23.033315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297178+0000","last_change":"2023-10-26T05:47:22.055995+0000","last_active":"2023-10-26T05:47:23.297178+0000","last_peered":"2023-10-26T05:47:23.297178+0000","last_clean":"2023-10-26T05:47:23.297178+0000","last_became_active":"2023-10-26T05:47:22.049810+0000","last_became_peered":"2023-10-26T05:47:22.049810+0000","last_unstale":"2023-10-26T05:47:23.297178+0000","last_undegraded":"2023-10-26T05:47:23.297178+0000","last_fullsized":"2023-10-26T05:47:23.297178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299268+0000","last_change":"2023-10-26T05:47:22.071995+0000","last_active":"2023-10-26T05:47:23.299268+0000","last_peered":"2023-10-26T05:47:23.299268+0000","last_clean":"2023-10-26T05:47:23.299268+0000","last_became_active":"2023-10-26T05:47:22.070349+0000","last_became_peered":"2023-10-26T05:47:22.070349+0000","last_unstale":"2023-10-26T05:47:23.299268+0000","last_undegraded":"2023-10-26T05:47:23.299268+0000","last_fullsized":"2023-10-26T05:47:23.299268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033881+0000","last_change":"2023-10-26T05:47:22.051874+0000","last_active":"2023-10-26T05:47:23.033881+0000","last_peered":"2023-10-26T05:47:23.033881+0000","last_clean":"2023-10-26T05:47:23.033881+0000","last_became_active":"2023-10-26T05:47:22.051002+0000","last_became_peered":"2023-10-26T05:47:22.051002+0000","last_unstale":"2023-10-26T05:47:23.033881+0000","last_undegraded":"2023-10-26T05:47:23.033881+0000","last_fullsized":"2023-10-26T05:47:23.033881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297488+0000","last_change":"2023-10-26T05:47:22.080998+0000","last_active":"2023-10-26T05:47:23.297488+0000","last_peered":"2023-10-26T05:47:23.297488+0000","last_clean":"2023-10-26T05:47:23.297488+0000","last_became_active":"2023-10-26T05:47:22.078649+0000","last_became_peered":"2023-10-26T05:47:22.078649+0000","last_unstale":"2023-10-26T05:47:23.297488+0000","last_undegraded":"2023-10-26T05:47:23.297488+0000","last_fullsized":"2023-10-26T05:47:23.297488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:05:54.913525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298814+0000","last_change":"2023-10-26T05:47:22.052419+0000","last_active":"2023-10-26T05:47:23.298814+0000","last_peered":"2023-10-26T05:47:23.298814+0000","last_clean":"2023-10-26T05:47:23.298814+0000","last_became_active":"2023-10-26T05:47:22.051751+0000","last_became_peered":"2023-10-26T05:47:22.051751+0000","last_unstale":"2023-10-26T05:47:23.298814+0000","last_undegraded":"2023-10-26T05:47:23.298814+0000","last_fullsized":"2023-10-26T05:47:23.298814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596450+0000","last_change":"2023-10-26T05:47:22.073364+0000","last_active":"2023-10-26T05:47:23.596450+0000","last_peered":"2023-10-26T05:47:23.596450+0000","last_clean":"2023-10-26T05:47:23.596450+0000","last_became_active":"2023-10-26T05:47:22.072258+0000","last_became_peered":"2023-10-26T05:47:22.072258+0000","last_unstale":"2023-10-26T05:47:23.596450+0000","last_undegraded":"2023-10-26T05:47:23.596450+0000","last_fullsized":"2023-10-26T05:47:23.596450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299915+0000","last_change":"2023-10-26T05:47:22.075501+0000","last_active":"2023-10-26T05:47:23.299915+0000","last_peered":"2023-10-26T05:47:23.299915+0000","last_clean":"2023-10-26T05:47:23.299915+0000","last_became_active":"2023-10-26T05:47:22.074808+0000","last_became_peered":"2023-10-26T05:47:22.074808+0000","last_unstale":"2023-10-26T05:47:23.299915+0000","last_undegraded":"2023-10-26T05:47:23.299915+0000","last_fullsized":"2023-10-26T05:47:23.299915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291547+0000","last_change":"2023-10-26T05:47:22.061162+0000","last_active":"2023-10-26T05:47:23.291547+0000","last_peered":"2023-10-26T05:47:23.291547+0000","last_clean":"2023-10-26T05:47:23.291547+0000","last_became_active":"2023-10-26T05:47:22.053623+0000","last_became_peered":"2023-10-26T05:47:22.053623+0000","last_unstale":"2023-10-26T05:47:23.291547+0000","last_undegraded":"2023-10-26T05:47:23.291547+0000","last_fullsized":"2023-10-26T05:47:23.291547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296810+0000","last_change":"2023-10-26T05:47:22.063651+0000","last_active":"2023-10-26T05:47:23.296810+0000","last_peered":"2023-10-26T05:47:23.296810+0000","last_clean":"2023-10-26T05:47:23.296810+0000","last_became_active":"2023-10-26T05:47:22.062599+0000","last_became_peered":"2023-10-26T05:47:22.062599+0000","last_unstale":"2023-10-26T05:47:23.296810+0000","last_undegraded":"2023-10-26T05:47:23.296810+0000","last_fullsized":"2023-10-26T05:47:23.296810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290912+0000","last_change":"2023-10-26T05:47:22.053793+0000","last_active":"2023-10-26T05:47:23.290912+0000","last_peered":"2023-10-26T05:47:23.290912+0000","last_clean":"2023-10-26T05:47:23.290912+0000","last_became_active":"2023-10-26T05:47:22.052294+0000","last_became_peered":"2023-10-26T05:47:22.052294+0000","last_unstale":"2023-10-26T05:47:23.290912+0000","last_undegraded":"2023-10-26T05:47:23.290912+0000","last_fullsized":"2023-10-26T05:47:23.290912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298905+0000","last_change":"2023-10-26T05:47:22.078582+0000","last_active":"2023-10-26T05:47:23.298905+0000","last_peered":"2023-10-26T05:47:23.298905+0000","last_clean":"2023-10-26T05:47:23.298905+0000","last_became_active":"2023-10-26T05:47:22.077883+0000","last_became_peered":"2023-10-26T05:47:22.077883+0000","last_unstale":"2023-10-26T05:47:23.298905+0000","last_undegraded":"2023-10-26T05:47:23.298905+0000","last_fullsized":"2023-10-26T05:47:23.298905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033302+0000","last_change":"2023-10-26T05:47:22.057439+0000","last_active":"2023-10-26T05:47:23.033302+0000","last_peered":"2023-10-26T05:47:23.033302+0000","last_clean":"2023-10-26T05:47:23.033302+0000","last_became_active":"2023-10-26T05:47:22.056731+0000","last_became_peered":"2023-10-26T05:47:22.056731+0000","last_unstale":"2023-10-26T05:47:23.033302+0000","last_undegraded":"2023-10-26T05:47:23.033302+0000","last_fullsized":"2023-10-26T05:47:23.033302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291223+0000","last_change":"2023-10-26T05:47:22.065579+0000","last_active":"2023-10-26T05:47:23.291223+0000","last_peered":"2023-10-26T05:47:23.291223+0000","last_clean":"2023-10-26T05:47:23.291223+0000","last_became_active":"2023-10-26T05:47:22.063798+0000","last_became_peered":"2023-10-26T05:47:22.063798+0000","last_unstale":"2023-10-26T05:47:23.291223+0000","last_undegraded":"2023-10-26T05:47:23.291223+0000","last_fullsized":"2023-10-26T05:47:23.291223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596104+0000","last_change":"2023-10-26T05:47:22.056700+0000","last_active":"2023-10-26T05:47:23.596104+0000","last_peered":"2023-10-26T05:47:23.596104+0000","last_clean":"2023-10-26T05:47:23.596104+0000","last_became_active":"2023-10-26T05:47:22.056216+0000","last_became_peered":"2023-10-26T05:47:22.056216+0000","last_unstale":"2023-10-26T05:47:23.596104+0000","last_undegraded":"2023-10-26T05:47:23.596104+0000","last_fullsized":"2023-10-26T05:47:23.596104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297162+0000","last_change":"2023-10-26T05:47:22.061465+0000","last_active":"2023-10-26T05:47:23.297162+0000","last_peered":"2023-10-26T05:47:23.297162+0000","last_clean":"2023-10-26T05:47:23.297162+0000","last_became_active":"2023-10-26T05:47:22.060699+0000","last_became_peered":"2023-10-26T05:47:22.060699+0000","last_unstale":"2023-10-26T05:47:23.297162+0000","last_undegraded":"2023-10-26T05:47:23.297162+0000","last_fullsized":"2023-10-26T05:47:23.297162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:30:33.956923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033512+0000","last_change":"2023-10-26T05:47:22.058403+0000","last_active":"2023-10-26T05:47:23.033512+0000","last_peered":"2023-10-26T05:47:23.033512+0000","last_clean":"2023-10-26T05:47:23.033512+0000","last_became_active":"2023-10-26T05:47:22.057792+0000","last_became_peered":"2023-10-26T05:47:22.057792+0000","last_unstale":"2023-10-26T05:47:23.033512+0000","last_undegraded":"2023-10-26T05:47:23.033512+0000","last_fullsized":"2023-10-26T05:47:23.033512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299297+0000","last_change":"2023-10-26T05:47:22.059009+0000","last_active":"2023-10-26T05:47:23.299297+0000","last_peered":"2023-10-26T05:47:23.299297+0000","last_clean":"2023-10-26T05:47:23.299297+0000","last_became_active":"2023-10-26T05:47:22.057498+0000","last_became_peered":"2023-10-26T05:47:22.057498+0000","last_unstale":"2023-10-26T05:47:23.299297+0000","last_undegraded":"2023-10-26T05:47:23.299297+0000","last_fullsized":"2023-10-26T05:47:23.299297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040301+0000","last_change":"2023-10-26T05:47:22.060276+0000","last_active":"2023-10-26T05:47:23.040301+0000","last_peered":"2023-10-26T05:47:23.040301+0000","last_clean":"2023-10-26T05:47:23.040301+0000","last_became_active":"2023-10-26T05:47:22.059444+0000","last_became_peered":"2023-10-26T05:47:22.059444+0000","last_unstale":"2023-10-26T05:47:23.040301+0000","last_undegraded":"2023-10-26T05:47:23.040301+0000","last_fullsized":"2023-10-26T05:47:23.040301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:13.998981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291438+0000","last_change":"2023-10-26T05:47:21.083390+0000","last_active":"2023-10-26T05:47:23.291438+0000","last_peered":"2023-10-26T05:47:23.291438+0000","last_clean":"2023-10-26T05:47:23.291438+0000","last_became_active":"2023-10-26T05:47:21.083255+0000","last_became_peered":"2023-10-26T05:47:21.083255+0000","last_unstale":"2023-10-26T05:47:23.291438+0000","last_undegraded":"2023-10-26T05:47:23.291438+0000","last_fullsized":"2023-10-26T05:47:23.291438+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.595999+0000","last_change":"2023-10-26T05:47:22.042083+0000","last_active":"2023-10-26T05:47:23.595999+0000","last_peered":"2023-10-26T05:47:23.595999+0000","last_clean":"2023-10-26T05:47:23.595999+0000","last_became_active":"2023-10-26T05:47:22.041458+0000","last_became_peered":"2023-10-26T05:47:22.041458+0000","last_unstale":"2023-10-26T05:47:23.595999+0000","last_undegraded":"2023-10-26T05:47:23.595999+0000","last_fullsized":"2023-10-26T05:47:23.595999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:18.110818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298965+0000","last_change":"2023-10-26T05:47:21.090954+0000","last_active":"2023-10-26T05:47:23.298965+0000","last_peered":"2023-10-26T05:47:23.298965+0000","last_clean":"2023-10-26T05:47:23.298965+0000","last_became_active":"2023-10-26T05:47:21.086959+0000","last_became_peered":"2023-10-26T05:47:21.086959+0000","last_unstale":"2023-10-26T05:47:23.298965+0000","last_undegraded":"2023-10-26T05:47:23.298965+0000","last_fullsized":"2023-10-26T05:47:23.298965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291059+0000","last_change":"2023-10-26T05:47:21.078842+0000","last_active":"2023-10-26T05:47:23.291059+0000","last_peered":"2023-10-26T05:47:23.291059+0000","last_clean":"2023-10-26T05:47:23.291059+0000","last_became_active":"2023-10-26T05:47:21.078498+0000","last_became_peered":"2023-10-26T05:47:21.078498+0000","last_unstale":"2023-10-26T05:47:23.291059+0000","last_undegraded":"2023-10-26T05:47:23.291059+0000","last_fullsized":"2023-10-26T05:47:23.291059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291157+0000","last_change":"2023-10-26T05:47:22.054327+0000","last_active":"2023-10-26T05:47:23.291157+0000","last_peered":"2023-10-26T05:47:23.291157+0000","last_clean":"2023-10-26T05:47:23.291157+0000","last_became_active":"2023-10-26T05:47:22.053185+0000","last_became_peered":"2023-10-26T05:47:22.053185+0000","last_unstale":"2023-10-26T05:47:23.291157+0000","last_undegraded":"2023-10-26T05:47:23.291157+0000","last_fullsized":"2023-10-26T05:47:23.291157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297471+0000","last_change":"2023-10-26T05:47:21.084041+0000","last_active":"2023-10-26T05:47:23.297471+0000","last_peered":"2023-10-26T05:47:23.297471+0000","last_clean":"2023-10-26T05:47:23.297471+0000","last_became_active":"2023-10-26T05:47:21.082189+0000","last_became_peered":"2023-10-26T05:47:21.082189+0000","last_unstale":"2023-10-26T05:47:23.297471+0000","last_undegraded":"2023-10-26T05:47:23.297471+0000","last_fullsized":"2023-10-26T05:47:23.297471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032584+0000","last_change":"2023-10-26T05:47:22.063104+0000","last_active":"2023-10-26T05:47:23.032584+0000","last_peered":"2023-10-26T05:47:23.032584+0000","last_clean":"2023-10-26T05:47:23.032584+0000","last_became_active":"2023-10-26T05:47:22.062483+0000","last_became_peered":"2023-10-26T05:47:22.062483+0000","last_unstale":"2023-10-26T05:47:23.032584+0000","last_undegraded":"2023-10-26T05:47:23.032584+0000","last_fullsized":"2023-10-26T05:47:23.032584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299320+0000","last_change":"2023-10-26T05:47:21.087323+0000","last_active":"2023-10-26T05:47:23.299320+0000","last_peered":"2023-10-26T05:47:23.299320+0000","last_clean":"2023-10-26T05:47:23.299320+0000","last_became_active":"2023-10-26T05:47:21.086686+0000","last_became_peered":"2023-10-26T05:47:21.086686+0000","last_unstale":"2023-10-26T05:47:23.299320+0000","last_undegraded":"2023-10-26T05:47:23.299320+0000","last_fullsized":"2023-10-26T05:47:23.299320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299593+0000","last_change":"2023-10-26T05:47:22.072429+0000","last_active":"2023-10-26T05:47:23.299593+0000","last_peered":"2023-10-26T05:47:23.299593+0000","last_clean":"2023-10-26T05:47:23.299593+0000","last_became_active":"2023-10-26T05:47:22.071355+0000","last_became_peered":"2023-10-26T05:47:22.071355+0000","last_unstale":"2023-10-26T05:47:23.299593+0000","last_undegraded":"2023-10-26T05:47:23.299593+0000","last_fullsized":"2023-10-26T05:47:23.299593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:55.408170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300874+0000","last_change":"2023-10-26T05:47:21.096076+0000","last_active":"2023-10-26T05:47:23.300874+0000","last_peered":"2023-10-26T05:47:23.300874+0000","last_clean":"2023-10-26T05:47:23.300874+0000","last_became_active":"2023-10-26T05:47:21.092744+0000","last_became_peered":"2023-10-26T05:47:21.092744+0000","last_unstale":"2023-10-26T05:47:23.300874+0000","last_undegraded":"2023-10-26T05:47:23.300874+0000","last_fullsized":"2023-10-26T05:47:23.300874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:26.546812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596655+0000","last_change":"2023-10-26T05:47:22.055264+0000","last_active":"2023-10-26T05:47:23.596655+0000","last_peered":"2023-10-26T05:47:23.596655+0000","last_clean":"2023-10-26T05:47:23.596655+0000","last_became_active":"2023-10-26T05:47:22.054516+0000","last_became_peered":"2023-10-26T05:47:22.054516+0000","last_unstale":"2023-10-26T05:47:23.596655+0000","last_undegraded":"2023-10-26T05:47:23.596655+0000","last_fullsized":"2023-10-26T05:47:23.596655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:42:31.528661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033126+0000","last_change":"2023-10-26T05:47:21.069747+0000","last_active":"2023-10-26T05:47:23.033126+0000","last_peered":"2023-10-26T05:47:23.033126+0000","last_clean":"2023-10-26T05:47:23.033126+0000","last_became_active":"2023-10-26T05:47:21.069525+0000","last_became_peered":"2023-10-26T05:47:21.069525+0000","last_unstale":"2023-10-26T05:47:23.033126+0000","last_undegraded":"2023-10-26T05:47:23.033126+0000","last_fullsized":"2023-10-26T05:47:23.033126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039554+0000","last_change":"2023-10-26T05:47:22.070886+0000","last_active":"2023-10-26T05:47:23.039554+0000","last_peered":"2023-10-26T05:47:23.039554+0000","last_clean":"2023-10-26T05:47:23.039554+0000","last_became_active":"2023-10-26T05:47:22.068908+0000","last_became_peered":"2023-10-26T05:47:22.068908+0000","last_unstale":"2023-10-26T05:47:23.039554+0000","last_undegraded":"2023-10-26T05:47:23.039554+0000","last_fullsized":"2023-10-26T05:47:23.039554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:31.276801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297743+0000","last_change":"2023-10-26T05:47:21.085627+0000","last_active":"2023-10-26T05:47:23.297743+0000","last_peered":"2023-10-26T05:47:23.297743+0000","last_clean":"2023-10-26T05:47:23.297743+0000","last_became_active":"2023-10-26T05:47:21.084306+0000","last_became_peered":"2023-10-26T05:47:21.084306+0000","last_unstale":"2023-10-26T05:47:23.297743+0000","last_undegraded":"2023-10-26T05:47:23.297743+0000","last_fullsized":"2023-10-26T05:47:23.297743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299433+0000","last_change":"2023-10-26T05:47:22.071700+0000","last_active":"2023-10-26T05:47:23.299433+0000","last_peered":"2023-10-26T05:47:23.299433+0000","last_clean":"2023-10-26T05:47:23.299433+0000","last_became_active":"2023-10-26T05:47:22.071056+0000","last_became_peered":"2023-10-26T05:47:22.071056+0000","last_unstale":"2023-10-26T05:47:23.299433+0000","last_undegraded":"2023-10-26T05:47:23.299433+0000","last_fullsized":"2023-10-26T05:47:23.299433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:08:09.742201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596128+0000","last_change":"2023-10-26T05:47:21.069670+0000","last_active":"2023-10-26T05:47:23.596128+0000","last_peered":"2023-10-26T05:47:23.596128+0000","last_clean":"2023-10-26T05:47:23.596128+0000","last_became_active":"2023-10-26T05:47:21.069297+0000","last_became_peered":"2023-10-26T05:47:21.069297+0000","last_unstale":"2023-10-26T05:47:23.596128+0000","last_undegraded":"2023-10-26T05:47:23.596128+0000","last_fullsized":"2023-10-26T05:47:23.596128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:15.639318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596815+0000","last_change":"2023-10-26T05:47:22.073645+0000","last_active":"2023-10-26T05:47:23.596815+0000","last_peered":"2023-10-26T05:47:23.596815+0000","last_clean":"2023-10-26T05:47:23.596815+0000","last_became_active":"2023-10-26T05:47:22.072989+0000","last_became_peered":"2023-10-26T05:47:22.072989+0000","last_unstale":"2023-10-26T05:47:23.596815+0000","last_undegraded":"2023-10-26T05:47:23.596815+0000","last_fullsized":"2023-10-26T05:47:23.596815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:22.636040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298358+0000","last_change":"2023-10-26T05:47:21.086438+0000","last_active":"2023-10-26T05:47:23.298358+0000","last_peered":"2023-10-26T05:47:23.298358+0000","last_clean":"2023-10-26T05:47:23.298358+0000","last_became_active":"2023-10-26T05:47:21.082053+0000","last_became_peered":"2023-10-26T05:47:21.082053+0000","last_unstale":"2023-10-26T05:47:23.298358+0000","last_undegraded":"2023-10-26T05:47:23.298358+0000","last_fullsized":"2023-10-26T05:47:23.298358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:37:21.191528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299687+0000","last_change":"2023-10-26T05:47:22.076444+0000","last_active":"2023-10-26T05:47:23.299687+0000","last_peered":"2023-10-26T05:47:23.299687+0000","last_clean":"2023-10-26T05:47:23.299687+0000","last_became_active":"2023-10-26T05:47:22.074572+0000","last_became_peered":"2023-10-26T05:47:22.074572+0000","last_unstale":"2023-10-26T05:47:23.299687+0000","last_undegraded":"2023-10-26T05:47:23.299687+0000","last_fullsized":"2023-10-26T05:47:23.299687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299516+0000","last_change":"2023-10-26T05:47:21.090350+0000","last_active":"2023-10-26T05:47:23.299516+0000","last_peered":"2023-10-26T05:47:23.299516+0000","last_clean":"2023-10-26T05:47:23.299516+0000","last_became_active":"2023-10-26T05:47:21.087109+0000","last_became_peered":"2023-10-26T05:47:21.087109+0000","last_unstale":"2023-10-26T05:47:23.299516+0000","last_undegraded":"2023-10-26T05:47:23.299516+0000","last_fullsized":"2023-10-26T05:47:23.299516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298687+0000","last_change":"2023-10-26T05:47:22.066147+0000","last_active":"2023-10-26T05:47:23.298687+0000","last_peered":"2023-10-26T05:47:23.298687+0000","last_clean":"2023-10-26T05:47:23.298687+0000","last_became_active":"2023-10-26T05:47:22.065497+0000","last_became_peered":"2023-10-26T05:47:22.065497+0000","last_unstale":"2023-10-26T05:47:23.298687+0000","last_undegraded":"2023-10-26T05:47:23.298687+0000","last_fullsized":"2023-10-26T05:47:23.298687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034102+0000","last_change":"2023-10-26T05:47:21.064585+0000","last_active":"2023-10-26T05:47:23.034102+0000","last_peered":"2023-10-26T05:47:23.034102+0000","last_clean":"2023-10-26T05:47:23.034102+0000","last_became_active":"2023-10-26T05:47:21.064395+0000","last_became_peered":"2023-10-26T05:47:21.064395+0000","last_unstale":"2023-10-26T05:47:23.034102+0000","last_undegraded":"2023-10-26T05:47:23.034102+0000","last_fullsized":"2023-10-26T05:47:23.034102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:30.266251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300524+0000","last_change":"2023-10-26T05:47:22.077148+0000","last_active":"2023-10-26T05:47:23.300524+0000","last_peered":"2023-10-26T05:47:23.300524+0000","last_clean":"2023-10-26T05:47:23.300524+0000","last_became_active":"2023-10-26T05:47:22.075279+0000","last_became_peered":"2023-10-26T05:47:22.075279+0000","last_unstale":"2023-10-26T05:47:23.300524+0000","last_undegraded":"2023-10-26T05:47:23.300524+0000","last_fullsized":"2023-10-26T05:47:23.300524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:12:08.628590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039791+0000","last_change":"2023-10-26T05:47:21.077376+0000","last_active":"2023-10-26T05:47:23.039791+0000","last_peered":"2023-10-26T05:47:23.039791+0000","last_clean":"2023-10-26T05:47:23.039791+0000","last_became_active":"2023-10-26T05:47:21.076766+0000","last_became_peered":"2023-10-26T05:47:21.076766+0000","last_unstale":"2023-10-26T05:47:23.039791+0000","last_undegraded":"2023-10-26T05:47:23.039791+0000","last_fullsized":"2023-10-26T05:47:23.039791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299945+0000","last_change":"2023-10-26T05:47:22.077007+0000","last_active":"2023-10-26T05:47:23.299945+0000","last_peered":"2023-10-26T05:47:23.299945+0000","last_clean":"2023-10-26T05:47:23.299945+0000","last_became_active":"2023-10-26T05:47:22.075297+0000","last_became_peered":"2023-10-26T05:47:22.075297+0000","last_unstale":"2023-10-26T05:47:23.299945+0000","last_undegraded":"2023-10-26T05:47:23.299945+0000","last_fullsized":"2023-10-26T05:47:23.299945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:15.466457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033816+0000","last_change":"2023-10-26T05:47:21.072137+0000","last_active":"2023-10-26T05:47:23.033816+0000","last_peered":"2023-10-26T05:47:23.033816+0000","last_clean":"2023-10-26T05:47:23.033816+0000","last_became_active":"2023-10-26T05:47:21.071720+0000","last_became_peered":"2023-10-26T05:47:21.071720+0000","last_unstale":"2023-10-26T05:47:23.033816+0000","last_undegraded":"2023-10-26T05:47:23.033816+0000","last_fullsized":"2023-10-26T05:47:23.033816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:09.715298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297159+0000","last_change":"2023-10-26T05:47:22.080581+0000","last_active":"2023-10-26T05:47:23.297159+0000","last_peered":"2023-10-26T05:47:23.297159+0000","last_clean":"2023-10-26T05:47:23.297159+0000","last_became_active":"2023-10-26T05:47:22.076913+0000","last_became_peered":"2023-10-26T05:47:22.076913+0000","last_unstale":"2023-10-26T05:47:23.297159+0000","last_undegraded":"2023-10-26T05:47:23.297159+0000","last_fullsized":"2023-10-26T05:47:23.297159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033027+0000","last_change":"2023-10-26T05:47:21.076982+0000","last_active":"2023-10-26T05:47:23.033027+0000","last_peered":"2023-10-26T05:47:23.033027+0000","last_clean":"2023-10-26T05:47:23.033027+0000","last_became_active":"2023-10-26T05:47:21.076695+0000","last_became_peered":"2023-10-26T05:47:21.076695+0000","last_unstale":"2023-10-26T05:47:23.033027+0000","last_undegraded":"2023-10-26T05:47:23.033027+0000","last_fullsized":"2023-10-26T05:47:23.033027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:27:57.904856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300280+0000","last_change":"2023-10-26T05:47:22.063246+0000","last_active":"2023-10-26T05:47:23.300280+0000","last_peered":"2023-10-26T05:47:23.300280+0000","last_clean":"2023-10-26T05:47:23.300280+0000","last_became_active":"2023-10-26T05:47:22.062499+0000","last_became_peered":"2023-10-26T05:47:22.062499+0000","last_unstale":"2023-10-26T05:47:23.300280+0000","last_undegraded":"2023-10-26T05:47:23.300280+0000","last_fullsized":"2023-10-26T05:47:23.300280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:00:18.951486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033647+0000","last_change":"2023-10-26T05:47:21.087999+0000","last_active":"2023-10-26T05:47:23.033647+0000","last_peered":"2023-10-26T05:47:23.033647+0000","last_clean":"2023-10-26T05:47:23.033647+0000","last_became_active":"2023-10-26T05:47:21.087744+0000","last_became_peered":"2023-10-26T05:47:21.087744+0000","last_unstale":"2023-10-26T05:47:23.033647+0000","last_undegraded":"2023-10-26T05:47:23.033647+0000","last_fullsized":"2023-10-26T05:47:23.033647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:46:40.096481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298194+0000","last_change":"2023-10-26T05:47:22.081324+0000","last_active":"2023-10-26T05:47:23.298194+0000","last_peered":"2023-10-26T05:47:23.298194+0000","last_clean":"2023-10-26T05:47:23.298194+0000","last_became_active":"2023-10-26T05:47:22.079073+0000","last_became_peered":"2023-10-26T05:47:22.079073+0000","last_unstale":"2023-10-26T05:47:23.298194+0000","last_undegraded":"2023-10-26T05:47:23.298194+0000","last_fullsized":"2023-10-26T05:47:23.298194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:23.350363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-26T05:47:22.066065+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:47:22.066065+0000","last_peered":"2023-10-26T05:47:22.066065+0000","last_clean":"2023-10-26T05:47:22.066065+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:47:22.066065+0000","last_undegraded":"2023-10-26T05:47:22.066065+0000","last_fullsized":"2023-10-26T05:47:22.066065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298749+0000","last_change":"2023-10-26T05:47:21.087000+0000","last_active":"2023-10-26T05:47:23.298749+0000","last_peered":"2023-10-26T05:47:23.298749+0000","last_clean":"2023-10-26T05:47:23.298749+0000","last_became_active":"2023-10-26T05:47:21.082935+0000","last_became_peered":"2023-10-26T05:47:21.082935+0000","last_unstale":"2023-10-26T05:47:23.298749+0000","last_undegraded":"2023-10-26T05:47:23.298749+0000","last_fullsized":"2023-10-26T05:47:23.298749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291206+0000","last_change":"2023-10-26T05:47:22.055580+0000","last_active":"2023-10-26T05:47:23.291206+0000","last_peered":"2023-10-26T05:47:23.291206+0000","last_clean":"2023-10-26T05:47:23.291206+0000","last_became_active":"2023-10-26T05:47:22.054587+0000","last_became_peered":"2023-10-26T05:47:22.054587+0000","last_unstale":"2023-10-26T05:47:23.291206+0000","last_undegraded":"2023-10-26T05:47:23.291206+0000","last_fullsized":"2023-10-26T05:47:23.291206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297907+0000","last_change":"2023-10-26T05:47:21.100233+0000","last_active":"2023-10-26T05:47:23.297907+0000","last_peered":"2023-10-26T05:47:23.297907+0000","last_clean":"2023-10-26T05:47:23.297907+0000","last_became_active":"2023-10-26T05:47:21.099318+0000","last_became_peered":"2023-10-26T05:47:21.099318+0000","last_unstale":"2023-10-26T05:47:23.297907+0000","last_undegraded":"2023-10-26T05:47:23.297907+0000","last_fullsized":"2023-10-26T05:47:23.297907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596567+0000","last_change":"2023-10-26T05:47:22.057190+0000","last_active":"2023-10-26T05:47:23.596567+0000","last_peered":"2023-10-26T05:47:23.596567+0000","last_clean":"2023-10-26T05:47:23.596567+0000","last_became_active":"2023-10-26T05:47:22.049709+0000","last_became_peered":"2023-10-26T05:47:22.049709+0000","last_unstale":"2023-10-26T05:47:23.596567+0000","last_undegraded":"2023-10-26T05:47:23.596567+0000","last_fullsized":"2023-10-26T05:47:23.596567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.034304+0000","last_change":"2023-10-26T05:47:21.073026+0000","last_active":"2023-10-26T05:47:23.034304+0000","last_peered":"2023-10-26T05:47:23.034304+0000","last_clean":"2023-10-26T05:47:23.034304+0000","last_became_active":"2023-10-26T05:47:21.071577+0000","last_became_peered":"2023-10-26T05:47:21.071577+0000","last_unstale":"2023-10-26T05:47:23.034304+0000","last_undegraded":"2023-10-26T05:47:23.034304+0000","last_fullsized":"2023-10-26T05:47:23.034304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:25.047488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032708+0000","last_change":"2023-10-26T05:47:22.052469+0000","last_active":"2023-10-26T05:47:23.032708+0000","last_peered":"2023-10-26T05:47:23.032708+0000","last_clean":"2023-10-26T05:47:23.032708+0000","last_became_active":"2023-10-26T05:47:22.051614+0000","last_became_peered":"2023-10-26T05:47:22.051614+0000","last_unstale":"2023-10-26T05:47:23.032708+0000","last_undegraded":"2023-10-26T05:47:23.032708+0000","last_fullsized":"2023-10-26T05:47:23.032708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.301108+0000","last_change":"2023-10-26T05:47:21.096632+0000","last_active":"2023-10-26T05:47:23.301108+0000","last_peered":"2023-10-26T05:47:23.301108+0000","last_clean":"2023-10-26T05:47:23.301108+0000","last_became_active":"2023-10-26T05:47:21.093612+0000","last_became_peered":"2023-10-26T05:47:21.093612+0000","last_unstale":"2023-10-26T05:47:23.301108+0000","last_undegraded":"2023-10-26T05:47:23.301108+0000","last_fullsized":"2023-10-26T05:47:23.301108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.039976+0000","last_change":"2023-10-26T05:47:22.060460+0000","last_active":"2023-10-26T05:47:23.039976+0000","last_peered":"2023-10-26T05:47:23.039976+0000","last_clean":"2023-10-26T05:47:23.039976+0000","last_became_active":"2023-10-26T05:47:22.059831+0000","last_became_peered":"2023-10-26T05:47:22.059831+0000","last_unstale":"2023-10-26T05:47:23.039976+0000","last_undegraded":"2023-10-26T05:47:23.039976+0000","last_fullsized":"2023-10-26T05:47:23.039976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:18.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033723+0000","last_change":"2023-10-26T05:47:21.071567+0000","last_active":"2023-10-26T05:47:23.033723+0000","last_peered":"2023-10-26T05:47:23.033723+0000","last_clean":"2023-10-26T05:47:23.033723+0000","last_became_active":"2023-10-26T05:47:21.071275+0000","last_became_peered":"2023-10-26T05:47:21.071275+0000","last_unstale":"2023-10-26T05:47:23.033723+0000","last_undegraded":"2023-10-26T05:47:23.033723+0000","last_fullsized":"2023-10-26T05:47:23.033723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297152+0000","last_change":"2023-10-26T05:47:22.063236+0000","last_active":"2023-10-26T05:47:23.297152+0000","last_peered":"2023-10-26T05:47:23.297152+0000","last_clean":"2023-10-26T05:47:23.297152+0000","last_became_active":"2023-10-26T05:47:22.061746+0000","last_became_peered":"2023-10-26T05:47:22.061746+0000","last_unstale":"2023-10-26T05:47:23.297152+0000","last_undegraded":"2023-10-26T05:47:23.297152+0000","last_fullsized":"2023-10-26T05:47:23.297152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:50.359141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297696+0000","last_change":"2023-10-26T05:47:21.099849+0000","last_active":"2023-10-26T05:47:23.297696+0000","last_peered":"2023-10-26T05:47:23.297696+0000","last_clean":"2023-10-26T05:47:23.297696+0000","last_became_active":"2023-10-26T05:47:21.097871+0000","last_became_peered":"2023-10-26T05:47:21.097871+0000","last_unstale":"2023-10-26T05:47:23.297696+0000","last_undegraded":"2023-10-26T05:47:23.297696+0000","last_fullsized":"2023-10-26T05:47:23.297696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032981+0000","last_change":"2023-10-26T05:47:22.050271+0000","last_active":"2023-10-26T05:47:23.032981+0000","last_peered":"2023-10-26T05:47:23.032981+0000","last_clean":"2023-10-26T05:47:23.032981+0000","last_became_active":"2023-10-26T05:47:22.049654+0000","last_became_peered":"2023-10-26T05:47:22.049654+0000","last_unstale":"2023-10-26T05:47:23.032981+0000","last_undegraded":"2023-10-26T05:47:23.032981+0000","last_fullsized":"2023-10-26T05:47:23.032981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:24.404662+0000","last_change":"2023-10-26T05:47:21.078711+0000","last_active":"2023-10-26T05:47:24.404662+0000","last_peered":"2023-10-26T05:47:24.404662+0000","last_clean":"2023-10-26T05:47:24.404662+0000","last_became_active":"2023-10-26T05:47:21.078403+0000","last_became_peered":"2023-10-26T05:47:21.078403+0000","last_unstale":"2023-10-26T05:47:24.404662+0000","last_undegraded":"2023-10-26T05:47:24.404662+0000","last_fullsized":"2023-10-26T05:47:24.404662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290995+0000","last_change":"2023-10-26T05:47:22.065870+0000","last_active":"2023-10-26T05:47:23.290995+0000","last_peered":"2023-10-26T05:47:23.290995+0000","last_clean":"2023-10-26T05:47:23.290995+0000","last_became_active":"2023-10-26T05:47:22.065412+0000","last_became_peered":"2023-10-26T05:47:22.065412+0000","last_unstale":"2023-10-26T05:47:23.290995+0000","last_undegraded":"2023-10-26T05:47:23.290995+0000","last_fullsized":"2023-10-26T05:47:23.290995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:39.067403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299744+0000","last_change":"2023-10-26T05:47:21.094764+0000","last_active":"2023-10-26T05:47:23.299744+0000","last_peered":"2023-10-26T05:47:23.299744+0000","last_clean":"2023-10-26T05:47:23.299744+0000","last_became_active":"2023-10-26T05:47:21.091991+0000","last_became_peered":"2023-10-26T05:47:21.091991+0000","last_unstale":"2023-10-26T05:47:23.299744+0000","last_undegraded":"2023-10-26T05:47:23.299744+0000","last_fullsized":"2023-10-26T05:47:23.299744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:17.100267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596211+0000","last_change":"2023-10-26T05:47:22.059433+0000","last_active":"2023-10-26T05:47:23.596211+0000","last_peered":"2023-10-26T05:47:23.596211+0000","last_clean":"2023-10-26T05:47:23.596211+0000","last_became_active":"2023-10-26T05:47:22.054466+0000","last_became_peered":"2023-10-26T05:47:22.054466+0000","last_unstale":"2023-10-26T05:47:23.596211+0000","last_undegraded":"2023-10-26T05:47:23.596211+0000","last_fullsized":"2023-10-26T05:47:23.596211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298877+0000","last_change":"2023-10-26T05:47:21.099000+0000","last_active":"2023-10-26T05:47:23.298877+0000","last_peered":"2023-10-26T05:47:23.298877+0000","last_clean":"2023-10-26T05:47:23.298877+0000","last_became_active":"2023-10-26T05:47:21.095552+0000","last_became_peered":"2023-10-26T05:47:21.095552+0000","last_unstale":"2023-10-26T05:47:23.298877+0000","last_undegraded":"2023-10-26T05:47:23.298877+0000","last_fullsized":"2023-10-26T05:47:23.298877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:49.348590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.040667+0000","last_change":"2023-10-26T05:47:22.059642+0000","last_active":"2023-10-26T05:47:23.040667+0000","last_peered":"2023-10-26T05:47:23.040667+0000","last_clean":"2023-10-26T05:47:23.040667+0000","last_became_active":"2023-10-26T05:47:22.054716+0000","last_became_peered":"2023-10-26T05:47:22.054716+0000","last_unstale":"2023-10-26T05:47:23.040667+0000","last_undegraded":"2023-10-26T05:47:23.040667+0000","last_fullsized":"2023-10-26T05:47:23.040667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:10.310768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.033526+0000","last_change":"2023-10-26T05:47:21.090099+0000","last_active":"2023-10-26T05:47:23.033526+0000","last_peered":"2023-10-26T05:47:23.033526+0000","last_clean":"2023-10-26T05:47:23.033526+0000","last_became_active":"2023-10-26T05:47:21.089837+0000","last_became_peered":"2023-10-26T05:47:21.089837+0000","last_unstale":"2023-10-26T05:47:23.033526+0000","last_undegraded":"2023-10-26T05:47:23.033526+0000","last_fullsized":"2023-10-26T05:47:23.033526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:09.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290991+0000","last_change":"2023-10-26T05:47:22.046792+0000","last_active":"2023-10-26T05:47:23.290991+0000","last_peered":"2023-10-26T05:47:23.290991+0000","last_clean":"2023-10-26T05:47:23.290991+0000","last_became_active":"2023-10-26T05:47:22.045151+0000","last_became_peered":"2023-10-26T05:47:22.045151+0000","last_unstale":"2023-10-26T05:47:23.290991+0000","last_undegraded":"2023-10-26T05:47:23.290991+0000","last_fullsized":"2023-10-26T05:47:23.290991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:38:17.409527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.032792+0000","last_change":"2023-10-26T05:47:21.072214+0000","last_active":"2023-10-26T05:47:23.032792+0000","last_peered":"2023-10-26T05:47:23.032792+0000","last_clean":"2023-10-26T05:47:23.032792+0000","last_became_active":"2023-10-26T05:47:21.071876+0000","last_became_peered":"2023-10-26T05:47:21.071876+0000","last_unstale":"2023-10-26T05:47:23.032792+0000","last_undegraded":"2023-10-26T05:47:23.032792+0000","last_fullsized":"2023-10-26T05:47:23.032792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:10:25.797231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.291391+0000","last_change":"2023-10-26T05:47:22.066034+0000","last_active":"2023-10-26T05:47:23.291391+0000","last_peered":"2023-10-26T05:47:23.291391+0000","last_clean":"2023-10-26T05:47:23.291391+0000","last_became_active":"2023-10-26T05:47:22.064885+0000","last_became_peered":"2023-10-26T05:47:22.064885+0000","last_unstale":"2023-10-26T05:47:23.291391+0000","last_undegraded":"2023-10-26T05:47:23.291391+0000","last_fullsized":"2023-10-26T05:47:23.291391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:20:58.807202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.299704+0000","last_change":"2023-10-26T05:47:21.093006+0000","last_active":"2023-10-26T05:47:23.299704+0000","last_peered":"2023-10-26T05:47:23.299704+0000","last_clean":"2023-10-26T05:47:23.299704+0000","last_became_active":"2023-10-26T05:47:21.092342+0000","last_became_peered":"2023-10-26T05:47:21.092342+0000","last_unstale":"2023-10-26T05:47:23.299704+0000","last_undegraded":"2023-10-26T05:47:23.299704+0000","last_fullsized":"2023-10-26T05:47:23.299704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:15:12.988430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.296978+0000","last_change":"2023-10-26T05:47:22.059661+0000","last_active":"2023-10-26T05:47:23.296978+0000","last_peered":"2023-10-26T05:47:23.296978+0000","last_clean":"2023-10-26T05:47:23.296978+0000","last_became_active":"2023-10-26T05:47:22.058379+0000","last_became_peered":"2023-10-26T05:47:22.058379+0000","last_unstale":"2023-10-26T05:47:23.296978+0000","last_undegraded":"2023-10-26T05:47:23.296978+0000","last_fullsized":"2023-10-26T05:47:23.296978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:18.002349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297352+0000","last_change":"2023-10-26T05:47:22.045989+0000","last_active":"2023-10-26T05:47:23.297352+0000","last_peered":"2023-10-26T05:47:23.297352+0000","last_clean":"2023-10-26T05:47:23.297352+0000","last_became_active":"2023-10-26T05:47:22.044537+0000","last_became_peered":"2023-10-26T05:47:22.044537+0000","last_unstale":"2023-10-26T05:47:23.297352+0000","last_undegraded":"2023-10-26T05:47:23.297352+0000","last_fullsized":"2023-10-26T05:47:23.297352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:14.151728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.298638+0000","last_change":"2023-10-26T05:47:21.099538+0000","last_active":"2023-10-26T05:47:23.298638+0000","last_peered":"2023-10-26T05:47:23.298638+0000","last_clean":"2023-10-26T05:47:23.298638+0000","last_became_active":"2023-10-26T05:47:21.096351+0000","last_became_peered":"2023-10-26T05:47:21.096351+0000","last_unstale":"2023-10-26T05:47:23.298638+0000","last_undegraded":"2023-10-26T05:47:23.298638+0000","last_fullsized":"2023-10-26T05:47:23.298638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:54:21.625489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.297831+0000","last_change":"2023-10-26T05:47:22.059627+0000","last_active":"2023-10-26T05:47:23.297831+0000","last_peered":"2023-10-26T05:47:23.297831+0000","last_clean":"2023-10-26T05:47:23.297831+0000","last_became_active":"2023-10-26T05:47:22.054248+0000","last_became_peered":"2023-10-26T05:47:22.054248+0000","last_unstale":"2023-10-26T05:47:23.297831+0000","last_undegraded":"2023-10-26T05:47:23.297831+0000","last_fullsized":"2023-10-26T05:47:23.297831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:34.846169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.300077+0000","last_change":"2023-10-26T05:47:21.097760+0000","last_active":"2023-10-26T05:47:23.300077+0000","last_peered":"2023-10-26T05:47:23.300077+0000","last_clean":"2023-10-26T05:47:23.300077+0000","last_became_active":"2023-10-26T05:47:21.093680+0000","last_became_peered":"2023-10-26T05:47:21.093680+0000","last_unstale":"2023-10-26T05:47:23.300077+0000","last_undegraded":"2023-10-26T05:47:23.300077+0000","last_fullsized":"2023-10-26T05:47:23.300077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:38.056853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596384+0000","last_change":"2023-10-26T05:47:22.038934+0000","last_active":"2023-10-26T05:47:23.596384+0000","last_peered":"2023-10-26T05:47:23.596384+0000","last_clean":"2023-10-26T05:47:23.596384+0000","last_became_active":"2023-10-26T05:47:22.038330+0000","last_became_peered":"2023-10-26T05:47:22.038330+0000","last_unstale":"2023-10-26T05:47:23.596384+0000","last_undegraded":"2023-10-26T05:47:23.596384+0000","last_fullsized":"2023-10-26T05:47:23.596384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:27.557362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290969+0000","last_change":"2023-10-26T05:47:21.070756+0000","last_active":"2023-10-26T05:47:23.290969+0000","last_peered":"2023-10-26T05:47:23.290969+0000","last_clean":"2023-10-26T05:47:23.290969+0000","last_became_active":"2023-10-26T05:47:21.070211+0000","last_became_peered":"2023-10-26T05:47:21.070211+0000","last_unstale":"2023-10-26T05:47:23.290969+0000","last_undegraded":"2023-10-26T05:47:23.290969+0000","last_fullsized":"2023-10-26T05:47:23.290969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:35.930469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.290817+0000","last_change":"2023-10-26T05:47:22.062259+0000","last_active":"2023-10-26T05:47:23.290817+0000","last_peered":"2023-10-26T05:47:23.290817+0000","last_clean":"2023-10-26T05:47:23.290817+0000","last_became_active":"2023-10-26T05:47:22.055535+0000","last_became_peered":"2023-10-26T05:47:22.055535+0000","last_unstale":"2023-10-26T05:47:23.290817+0000","last_undegraded":"2023-10-26T05:47:23.290817+0000","last_fullsized":"2023-10-26T05:47:23.290817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:21.048745+0000","last_clean_scrub_stamp":"2023-10-26T05:47:21.048745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:10:15.249865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-26T05:47:23.596691+0000","last_change":"2023-10-26T05:47:21.084394+0000","last_active":"2023-10-26T05:47:23.596691+0000","last_peered":"2023-10-26T05:47:23.596691+0000","last_clean":"2023-10-26T05:47:23.596691+0000","last_became_active":"2023-10-26T05:47:21.083775+0000","last_became_peered":"2023-10-26T05:47:21.083775+0000","last_unstale":"2023-10-26T05:47:23.596691+0000","last_undegraded":"2023-10-26T05:47:23.596691+0000","last_fullsized":"2023-10-26T05:47:23.596691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:20.038195+0000","last_clean_scrub_stamp":"2023-10-26T05:47:20.038195+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:31.374962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338312,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.478}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:47:32.664 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-26T05:47:32.664 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-26T05:47:32.664 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-26T05:47:32.664 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-26T05:47:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 -- 172.21.15.71:0/3688021392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413d420 msgr2=0x7f92e40ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3688021392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413d420 0x7f92e40ac740 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92e406c690 tx=0x7f92e001c8a0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 -- 172.21.15.71:0/3688021392 shutdown_connections 2023-10-26T05:47:32.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3688021392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92e413d800 0x7f92e415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3688021392 >> v2:172.21.15.86:3300/0 conn(0x7f92e413dc20 0x7f92e415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3688021392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413d420 0x7f92e40ac740 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 -- 172.21.15.71:0/3688021392 >> 172.21.15.71:0/3688021392 conn(0x7f92e4069770 msgr2=0x7f92e4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 -- 172.21.15.71:0/3688021392 shutdown_connections 2023-10-26T05:47:32.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 -- 172.21.15.71:0/3688021392 wait complete. 2023-10-26T05:47:32.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 Processor -- start 2023-10-26T05:47:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.816+0000 7f92ec47d640 1 -- start start 2023-10-26T05:47:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92e413d420 0x7f92e4135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e413d800 0x7f92e41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 0x7f92e4152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e4138600 con 0x7f92e413dc20 2023-10-26T05:47:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e413bc00 con 0x7f92e413d800 2023-10-26T05:47:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e413bee0 con 0x7f92e413d420 2023-10-26T05:47:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 0x7f92e4152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 0x7f92e4152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55706/0 (socket says 172.21.15.71:55706) 2023-10-26T05:47:32.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 -- 172.21.15.71:0/3978592270 learned_addr learned my addr 172.21.15.71:0/3978592270 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 -- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92e413d420 msgr2=0x7f92e4135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92e413d420 0x7f92e4135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 -- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e413d800 msgr2=0x7f92e41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92e9a18640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e413d800 0x7f92e41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e413d800 0x7f92e41524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 -- 172.21.15.71:0/3978592270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009560 con 0x7f92e413dc20 2023-10-26T05:47:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92eaa1a640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 0x7f92e4152e90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00ca60 tx=0x7f92dc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc007d90 con 0x7f92e413dc20 2023-10-26T05:47:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f92dc004b30 con 0x7f92e413dc20 2023-10-26T05:47:32.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e413c1c0 con 0x7f92e413dc20 2023-10-26T05:47:32.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc005020 con 0x7f92e413dc20 2023-10-26T05:47:32.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e413b820 con 0x7f92e413dc20 2023-10-26T05:47:32.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e4066b10 con 0x7f92e413dc20 2023-10-26T05:47:32.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f92dc005180 con 0x7f92e413dc20 2023-10-26T05:47:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92daffd640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92c8066470 0x7f92c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:32.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92dc084690 con 0x7f92e413dc20 2023-10-26T05:47:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.820+0000 7f92ea219640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92c8066470 0x7f92c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.824+0000 7f92ea219640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92c8066470 0x7f92c8068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92e00af000 tx=0x7f92e4158ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:32.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:32.824+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f92dc051b20 con 0x7f92e413dc20 2023-10-26T05:47:33.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f92e4138740 con 0x7f92e413dc20 2023-10-26T05:47:33.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92daffd640 1 -- 172.21.15.71:0/3978592270 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f92e4138740 con 0x7f92e413dc20 2023-10-26T05:47:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:33.006 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-26T05:47:33.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92c8066470 msgr2=0x7f92c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92c8066470 0x7f92c8068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f92e00af000 tx=0x7f92e4158ed0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 msgr2=0x7f92e4152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 0x7f92e4152e90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00ca60 tx=0x7f92dc00cf20 comp rx=0 tx=0).stop 2023-10-26T05:47:33.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 shutdown_connections 2023-10-26T05:47:33.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92c8066470 0x7f92c8068930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92e413dc20 0x7f92e4152e90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92e413d800 0x7f92e41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 --2- 172.21.15.71:0/3978592270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92e413d420 0x7f92e4135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 >> 172.21.15.71:0/3978592270 conn(0x7f92e4069770 msgr2=0x7f92e4159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 shutdown_connections 2023-10-26T05:47:33.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.000+0000 7f92ec47d640 1 -- 172.21.15.71:0/3978592270 wait complete. 2023-10-26T05:47:33.024 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-26T05:47:33.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:33.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/1788179109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 msgr2=0x7fc1340b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- 172.21.15.71:0/1788179109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340b04b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc13406ed70 tx=0x7fc12401cd30 comp rx=0 tx=0).stop 2023-10-26T05:47:33.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/1788179109 shutdown_connections 2023-10-26T05:47:33.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- 172.21.15.71:0/1788179109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc134070fe0 0x7fc134071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- 172.21.15.71:0/1788179109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340b04b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- 172.21.15.71:0/1788179109 >> v2:172.21.15.86:3300/0 conn(0x7fc134140cc0 0x7fc13415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/1788179109 >> 172.21.15.71:0/1788179109 conn(0x7fc134069770 msgr2=0x7fc134069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/1788179109 shutdown_connections 2023-10-26T05:47:33.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/1788179109 wait complete. 2023-10-26T05:47:33.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 Processor -- start 2023-10-26T05:47:33.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- start start 2023-10-26T05:47:33.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc134070fe0 0x7fc134160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc134140cc0 0x7fc134069f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341f3ac0 con 0x7fc1340b0040 2023-10-26T05:47:33.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341f2f40 con 0x7fc134140cc0 2023-10-26T05:47:33.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341f3220 con 0x7fc134070fe0 2023-10-26T05:47:33.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55716/0 (socket says 172.21.15.71:55716) 2023-10-26T05:47:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 -- 172.21.15.71:0/3288992486 learned_addr learned my addr 172.21.15.71:0/3288992486 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc13259c640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc134070fe0 0x7fc134160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc132d9d640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc134140cc0 0x7fc134069f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 -- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc134070fe0 msgr2=0x7fc134160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc134070fe0 0x7fc134160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 -- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc134140cc0 msgr2=0x7fc134069f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc134140cc0 0x7fc134069f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 -- 172.21.15.71:0/3288992486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1240098e0 con 0x7fc1340b0040 2023-10-26T05:47:33.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc131d9b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340acc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00d750 tx=0x7fc11c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:33.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c009c30 con 0x7fc1340b0040 2023-10-26T05:47:33.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc11c009e80 con 0x7fc1340b0040 2023-10-26T05:47:33.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1341f3500 con 0x7fc1340b0040 2023-10-26T05:47:33.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c005280 con 0x7fc1340b0040 2023-10-26T05:47:33.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.140+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc134161410 con 0x7fc1340b0040 2023-10-26T05:47:33.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.144+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc134066b10 con 0x7fc1340b0040 2023-10-26T05:47:33.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.144+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc11c0053e0 con 0x7fc1340b0040 2023-10-26T05:47:33.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.144+0000 7fc12affd640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc114066470 0x7fc114068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.144+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc11c084100 con 0x7fc1340b0040 2023-10-26T05:47:33.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.144+0000 7fc13259c640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc114066470 0x7fc114068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.144+0000 7fc13259c640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc114066470 0x7fc114068930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0058d0 tx=0x7fc12c005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:33.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.148+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc11c0514e0 con 0x7fc1340b0040 2023-10-26T05:47:33.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.292+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1340721e0 con 0x7fc1340b0040 2023-10-26T05:47:33.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.292+0000 7fc12affd640 1 -- 172.21.15.71:0/3288992486 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fc11c020090 con 0x7fc1340b0040 2023-10-26T05:47:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:33.300 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc114066470 msgr2=0x7fc114068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc114066470 0x7fc114068930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0058d0 tx=0x7fc12c005860 comp rx=0 tx=0).stop 2023-10-26T05:47:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 msgr2=0x7fc1340acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340acc70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00d750 tx=0x7fc11c00dc10 comp rx=0 tx=0).stop 2023-10-26T05:47:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 shutdown_connections 2023-10-26T05:47:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc134140cc0 0x7fc134069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc114066470 0x7fc114068930 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc1340b0040 0x7fc1340acc70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 --2- 172.21.15.71:0/3288992486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc134070fe0 0x7fc134160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 >> 172.21.15.71:0/3288992486 conn(0x7fc134069770 msgr2=0x7fc13413ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 shutdown_connections 2023-10-26T05:47:33.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.296+0000 7fc138a0b640 1 -- 172.21.15.71:0/3288992486 wait complete. 2023-10-26T05:47:33.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:33.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:33.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- 172.21.15.71:0/2521318203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 msgr2=0x7f81f8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 --2- 172.21.15.71:0/2521318203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f81f8005190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8208006fd0 tx=0x7f820801cfc0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f8214d6a640 1 -- 172.21.15.71:0/2521318203 reap_dead start 2023-10-26T05:47:33.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- 172.21.15.71:0/2521318203 shutdown_connections 2023-10-26T05:47:33.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- 172.21.15.71:0/2521318203 >> 172.21.15.71:0/2521318203 conn(0x7f82100aec60 msgr2=0x7f82100af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- 172.21.15.71:0/2521318203 shutdown_connections 2023-10-26T05:47:33.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- 172.21.15.71:0/2521318203 wait complete. 2023-10-26T05:47:33.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 Processor -- start 2023-10-26T05:47:33.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- start start 2023-10-26T05:47:33.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f8210144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82101474f0 0x7f8210147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821013fc80 0x7f8210140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8210147eb0 con 0x7f81f8004d10 2023-10-26T05:47:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821014db40 con 0x7f821013fc80 2023-10-26T05:47:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f82167cd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821014de20 con 0x7f82101474f0 2023-10-26T05:47:33.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f8210144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f8210144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55738/0 (socket says 172.21.15.71:55738) 2023-10-26T05:47:33.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 -- 172.21.15.71:0/1289317531 learned_addr learned my addr 172.21.15.71:0/1289317531 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:33.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 -- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82101474f0 msgr2=0x7f8210147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82101474f0 0x7f8210147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 -- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821013fc80 msgr2=0x7f8210140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f8214d6a640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821013fc80 0x7f8210140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821013fc80 0x7f8210140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 -- 172.21.15.71:0/1289317531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82101456d0 con 0x7f81f8004d10 2023-10-26T05:47:33.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.428+0000 7f820ffff640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f8210144d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f820000fa90 tx=0x7f820000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:33.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200011440 con 0x7f81f8004d10 2023-10-26T05:47:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8200011690 con 0x7f81f8004d10 2023-10-26T05:47:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82101459b0 con 0x7f81f8004d10 2023-10-26T05:47:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200011a20 con 0x7f81f8004d10 2023-10-26T05:47:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8210140640 con 0x7f81f8004d10 2023-10-26T05:47:33.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8210066b10 con 0x7f81f8004d10 2023-10-26T05:47:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8200011b80 con 0x7f81f8004d10 2023-10-26T05:47:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820cff9640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81fc0664a0 0x7f81fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8200015070 con 0x7f81f8004d10 2023-10-26T05:47:33.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820f7fe640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81fc0664a0 0x7f81fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.432+0000 7f820f7fe640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81fc0664a0 0x7f81fc068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8204005fd0 tx=0x7f8204005e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:33.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.436+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82000529d0 con 0x7f81f8004d10 2023-10-26T05:47:33.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.576+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8210145530 con 0x7f81f8004d10 2023-10-26T05:47:33.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f820cff9640 1 -- 172.21.15.71:0/1289317531 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8210145530 con 0x7f81f8004d10 2023-10-26T05:47:33.584 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:33.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81fc0664a0 msgr2=0x7f81fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81fc0664a0 0x7f81fc068960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8204005fd0 tx=0x7f8204005e80 comp rx=0 tx=0).stop 2023-10-26T05:47:33.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 msgr2=0x7f8210144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f8210144d40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f820000fa90 tx=0x7f820000ff50 comp rx=0 tx=0).stop 2023-10-26T05:47:33.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 shutdown_connections 2023-10-26T05:47:33.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f821013fc80 0x7f8210140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82101474f0 0x7f8210147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81fc0664a0 0x7f81fc068960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 --2- 172.21.15.71:0/1289317531 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f8004d10 0x7f8210144d40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 >> 172.21.15.71:0/1289317531 conn(0x7f82100aec60 msgr2=0x7f8210159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 shutdown_connections 2023-10-26T05:47:33.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.580+0000 7f82167cd640 1 -- 172.21.15.71:0/1289317531 wait complete. 2023-10-26T05:47:33.602 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:33.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- 172.21.15.71:0/1673315363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0cd880 msgr2=0x7f756c15b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 --2- 172.21.15.71:0/1673315363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0cd880 0x7f756c15b8e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f756c0c4dd0 tx=0x7f756401cab0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- 172.21.15.71:0/1673315363 shutdown_connections 2023-10-26T05:47:33.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 --2- 172.21.15.71:0/1673315363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0cd880 0x7f756c15b8e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 --2- 172.21.15.71:0/1673315363 >> v2:172.21.15.86:3300/0 conn(0x7f756c0d0850 0x7f756c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- 172.21.15.71:0/1673315363 >> 172.21.15.71:0/1673315363 conn(0x7f756c0cdd80 msgr2=0x7f756c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- 172.21.15.71:0/1673315363 shutdown_connections 2023-10-26T05:47:33.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- 172.21.15.71:0/1673315363 wait complete. 2023-10-26T05:47:33.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 Processor -- start 2023-10-26T05:47:33.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- start start 2023-10-26T05:47:33.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f756c0cd880 0x7f756c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f756c0d0470 0x7f756c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 0x7f756c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c159970 con 0x7f756c0d0850 2023-10-26T05:47:33.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c0c80b0 con 0x7f756c0d0470 2023-10-26T05:47:33.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f7573c01640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c1555b0 con 0x7f756c0cd880 2023-10-26T05:47:33.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f757219e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 0x7f756c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f757219e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 0x7f756c15a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55760/0 (socket says 172.21.15.71:55760) 2023-10-26T05:47:33.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.712+0000 7f757219e640 1 -- 172.21.15.71:0/3708055753 learned_addr learned my addr 172.21.15.71:0/3708055753 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:33.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757119c640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f756c0d0470 0x7f756c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757219e640 1 -- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f756c0cd880 msgr2=0x7f756c159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757219e640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f756c0cd880 0x7f756c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757219e640 1 -- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f756c0d0470 msgr2=0x7f756c159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757219e640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f756c0d0470 0x7f756c159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757219e640 1 -- 172.21.15.71:0/3708055753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7564009820 con 0x7f756c0d0850 2023-10-26T05:47:33.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757119c640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f756c0d0470 0x7f756c159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:33.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757219e640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 0x7f756c15a160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f756401a6e0 tx=0x7f75640a5900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:33.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564004e60 con 0x7f756c0d0850 2023-10-26T05:47:33.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75640040f0 con 0x7f756c0d0850 2023-10-26T05:47:33.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c0c73d0 con 0x7f756c0d0850 2023-10-26T05:47:33.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640044c0 con 0x7f756c0d0850 2023-10-26T05:47:33.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c0c7990 con 0x7f756c0d0850 2023-10-26T05:47:33.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c066b10 con 0x7f756c0d0850 2023-10-26T05:47:33.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f75640b4040 con 0x7f756c0d0850 2023-10-26T05:47:33.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f75627fc640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7550066470 0x7f7550068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:33.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f756411e390 con 0x7f756c0d0850 2023-10-26T05:47:33.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757199d640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7550066470 0x7f7550068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:33.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.716+0000 7f757199d640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7550066470 0x7f7550068930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7568007fd0 tx=0x7f7568007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:33.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.720+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f75640eb810 con 0x7f756c0d0850 2023-10-26T05:47:33.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.860+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f756c0b62a0 con 0x7f756c0d0850 2023-10-26T05:47:33.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.860+0000 7f75627fc640 1 -- 172.21.15.71:0/3708055753 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75640eee30 con 0x7f756c0d0850 2023-10-26T05:47:33.867 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7550066470 msgr2=0x7f7550068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7550066470 0x7f7550068930 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7568007fd0 tx=0x7f7568007920 comp rx=0 tx=0).stop 2023-10-26T05:47:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 msgr2=0x7f756c15a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 0x7f756c15a160 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f756401a6e0 tx=0x7f75640a5900 comp rx=0 tx=0).stop 2023-10-26T05:47:33.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 shutdown_connections 2023-10-26T05:47:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7550066470 0x7f7550068930 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f756c0d0850 0x7f756c15a160 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f756c0d0470 0x7f756c159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 --2- 172.21.15.71:0/3708055753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f756c0cd880 0x7f756c159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:33.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 >> 172.21.15.71:0/3708055753 conn(0x7f756c0cdd80 msgr2=0x7f756c158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:33.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 shutdown_connections 2023-10-26T05:47:33.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:33.864+0000 7f7573c01640 1 -- 172.21.15.71:0/3708055753 wait complete. 2023-10-26T05:47:33.883 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- 172.21.15.71:0/3611440349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c0b2390 msgr2=0x7f047c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- 172.21.15.71:0/3611440349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c0b2390 0x7f047c06ff70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f047c06e990 tx=0x7f046c01c5b0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- 172.21.15.71:0/3611440349 shutdown_connections 2023-10-26T05:47:34.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- 172.21.15.71:0/3611440349 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f047c0704b0 0x7f047c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- 172.21.15.71:0/3611440349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c0b2390 0x7f047c06ff70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- 172.21.15.71:0/3611440349 >> v2:172.21.15.86:3300/0 conn(0x7f047c141980 0x7f047c0b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- 172.21.15.71:0/3611440349 >> 172.21.15.71:0/3611440349 conn(0x7f047c0aec60 msgr2=0x7f047c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- 172.21.15.71:0/3611440349 shutdown_connections 2023-10-26T05:47:34.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- 172.21.15.71:0/3611440349 wait complete. 2023-10-26T05:47:34.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 Processor -- start 2023-10-26T05:47:34.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- start start 2023-10-26T05:47:34.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f047c0704b0 0x7f047c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f047c0b2390 0x7f047c1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 0x7f047c162250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c161e30 con 0x7f047c141980 2023-10-26T05:47:34.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047c1538e0 con 0x7f047c0b2390 2023-10-26T05:47:34.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f048385f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047c153b10 con 0x7f047c0704b0 2023-10-26T05:47:34.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 0x7f047c162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 0x7f047c162250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55778/0 (socket says 172.21.15.71:55778) 2023-10-26T05:47:34.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 -- 172.21.15.71:0/2539677956 learned_addr learned my addr 172.21.15.71:0/2539677956 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:34.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0480dfa640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f047c0b2390 0x7f047c1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 -- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f047c0704b0 msgr2=0x7f047c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f047c0704b0 0x7f047c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 -- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f047c0b2390 msgr2=0x7f047c1533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f047c0b2390 0x7f047c1533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 -- 172.21.15.71:0/2539677956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c009560 con 0x7f047c141980 2023-10-26T05:47:34.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0481dfc640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 0x7f047c162250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f047400ca60 tx=0x7f047400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400adc0 con 0x7f047c141980 2023-10-26T05:47:34.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.000+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0474004c40 con 0x7f047c141980 2023-10-26T05:47:34.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c161770 con 0x7f047c141980 2023-10-26T05:47:34.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474005280 con 0x7f047c141980 2023-10-26T05:47:34.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047c13b9d0 con 0x7f047c141980 2023-10-26T05:47:34.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f047c066b10 con 0x7f047c141980 2023-10-26T05:47:34.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f04740053e0 con 0x7f047c141980 2023-10-26T05:47:34.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f0471ffb640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0460066470 0x7f0460068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04740846c0 con 0x7f047c141980 2023-10-26T05:47:34.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f04815fb640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0460066470 0x7f0460068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.004+0000 7f04815fb640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0460066470 0x7f0460068930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f047c0b2360 tx=0x7f0478005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.008+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0474051b40 con 0x7f047c141980 2023-10-26T05:47:34.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f047c161f70 con 0x7f047c141980 2023-10-26T05:47:34.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f0471ffb640 1 -- 172.21.15.71:0/2539677956 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f047c161f70 con 0x7f047c141980 2023-10-26T05:47:34.157 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:34.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0460066470 msgr2=0x7f0460068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0460066470 0x7f0460068930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f047c0b2360 tx=0x7f0478005b60 comp rx=0 tx=0).stop 2023-10-26T05:47:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 msgr2=0x7f047c162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 0x7f047c162250 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f047400ca60 tx=0x7f047400cf20 comp rx=0 tx=0).stop 2023-10-26T05:47:34.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 shutdown_connections 2023-10-26T05:47:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0460066470 0x7f0460068930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f047c141980 0x7f047c162250 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f047c0b2390 0x7f047c1533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 --2- 172.21.15.71:0/2539677956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f047c0704b0 0x7f047c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 >> 172.21.15.71:0/2539677956 conn(0x7f047c0aec60 msgr2=0x7f047c15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 shutdown_connections 2023-10-26T05:47:34.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.152+0000 7f048385f640 1 -- 172.21.15.71:0/2539677956 wait complete. 2023-10-26T05:47:34.175 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:47:34.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.276+0000 7f58c0080640 1 -- 172.21.15.71:0/2064462266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 msgr2=0x7f58b806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.276+0000 7f58c0080640 1 --2- 172.21.15.71:0/2064462266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b806aa70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f58b808a140 tx=0x7f58a801c780 comp rx=0 tx=0).stop 2023-10-26T05:47:34.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.276+0000 7f58c0080640 1 -- 172.21.15.71:0/2064462266 shutdown_connections 2023-10-26T05:47:34.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.276+0000 7f58c0080640 1 --2- 172.21.15.71:0/2064462266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58b80cd880 0x7f58b815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.276+0000 7f58c0080640 1 --2- 172.21.15.71:0/2064462266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b806aa70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.276+0000 7f58c0080640 1 -- 172.21.15.71:0/2064462266 >> 172.21.15.71:0/2064462266 conn(0x7f58b80cdd80 msgr2=0x7f58b80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- 172.21.15.71:0/2064462266 shutdown_connections 2023-10-26T05:47:34.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- 172.21.15.71:0/2064462266 wait complete. 2023-10-26T05:47:34.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 Processor -- start 2023-10-26T05:47:34.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- start start 2023-10-26T05:47:34.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b80cd880 0x7f58b8158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58b80d0850 0x7f58b8159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b814c6f0 con 0x7f58b80d0470 2023-10-26T05:47:34.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8153a20 con 0x7f58b80cd880 2023-10-26T05:47:34.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8153d00 con 0x7f58b80d0850 2023-10-26T05:47:34.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55802/0 (socket says 172.21.15.71:55802) 2023-10-26T05:47:34.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 -- 172.21.15.71:0/700806716 learned_addr learned my addr 172.21.15.71:0/700806716 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:34.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 -- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58b80d0850 msgr2=0x7f58b8159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bde1c640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b80cd880 0x7f58b8158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58b80d0850 0x7f58b8159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 -- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b80cd880 msgr2=0x7f58b8158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b80cd880 0x7f58b8158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 -- 172.21.15.71:0/700806716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8009560 con 0x7f58b80d0470 2023-10-26T05:47:34.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58bd61b640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b815a860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f58b400b410 tx=0x7f58b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4004020 con 0x7f58b80d0470 2023-10-26T05:47:34.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f58b4004270 con 0x7f58b80d0470 2023-10-26T05:47:34.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b8153fe0 con 0x7f58b80d0470 2023-10-26T05:47:34.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4011890 con 0x7f58b80d0470 2023-10-26T05:47:34.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b8159190 con 0x7f58b80d0470 2023-10-26T05:47:34.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.280+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b8066b10 con 0x7f58b80d0470 2023-10-26T05:47:34.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.284+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58b4003600 con 0x7f58b80d0470 2023-10-26T05:47:34.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.284+0000 7f58ae7fc640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f589c066470 0x7f589c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.284+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58b4083f10 con 0x7f58b80d0470 2023-10-26T05:47:34.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.284+0000 7f58bde1c640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f589c066470 0x7f589c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.284+0000 7f58bde1c640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f589c066470 0x7f589c068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58a80af000 tx=0x7f58a801afc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.288+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f58b4051300 con 0x7f58b80d0470 2023-10-26T05:47:34.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.460+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58b80721e0 con 0x7f58b80d0470 2023-10-26T05:47:34.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.460+0000 7f58ae7fc640 1 -- 172.21.15.71:0/700806716 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f58b4015090 con 0x7f58b80d0470 2023-10-26T05:47:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:34.468 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:47:34.469 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:47:34.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f589c066470 msgr2=0x7f589c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f589c066470 0x7f589c068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58a80af000 tx=0x7f58a801afc0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 msgr2=0x7f58b815a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b815a860 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f58b400b410 tx=0x7f58b400b8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 shutdown_connections 2023-10-26T05:47:34.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58b80d0850 0x7f58b8159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f589c066470 0x7f589c068930 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58b80d0470 0x7f58b815a860 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 --2- 172.21.15.71:0/700806716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58b80cd880 0x7f58b8158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 >> 172.21.15.71:0/700806716 conn(0x7f58b80cdd80 msgr2=0x7f58b815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 shutdown_connections 2023-10-26T05:47:34.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.464+0000 7f58c0080640 1 -- 172.21.15.71:0/700806716 wait complete. 2023-10-26T05:47:34.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/2756500305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0850 msgr2=0x7f5c8815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/2756500305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0850 0x7f5c8815b3a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009900 tx=0x7f5c7c01c770 comp rx=0 tx=0).stop 2023-10-26T05:47:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/2756500305 shutdown_connections 2023-10-26T05:47:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/2756500305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0850 0x7f5c8815b3a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/2756500305 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c880d0470 0x7f5c8806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/2756500305 >> 172.21.15.71:0/2756500305 conn(0x7f5c880cdd80 msgr2=0x7f5c880ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/2756500305 shutdown_connections 2023-10-26T05:47:34.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/2756500305 wait complete. 2023-10-26T05:47:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 Processor -- start 2023-10-26T05:47:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- start start 2023-10-26T05:47:34.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 0x7f5c880c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0470 0x7f5c8815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c880d0850 0x7f5c8813f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880c7fa0 con 0x7f5c880cd880 2023-10-26T05:47:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815dfe0 con 0x7f5c880d0470 2023-10-26T05:47:34.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c8d8ba640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880c54c0 con 0x7f5c880d0850 2023-10-26T05:47:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 0x7f5c880c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 0x7f5c880c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55816/0 (socket says 172.21.15.71:55816) 2023-10-26T05:47:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 -- 172.21.15.71:0/3819458197 learned_addr learned my addr 172.21.15.71:0/3819458197 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c867fc640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0470 0x7f5c8815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c877fe640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c880d0850 0x7f5c8813f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 -- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c880d0850 msgr2=0x7f5c8813f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c880d0850 0x7f5c8813f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 -- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0470 msgr2=0x7f5c8815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0470 0x7f5c8815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 -- 172.21.15.71:0/3819458197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c009560 con 0x7f5c880cd880 2023-10-26T05:47:34.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c86ffd640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 0x7f5c880c4e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800e9f0 tx=0x7f5c7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.600+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78009710 con 0x7f5c880cd880 2023-10-26T05:47:34.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5c7800ce70 con 0x7f5c880cd880 2023-10-26T05:47:34.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c880c7600 con 0x7f5c880cd880 2023-10-26T05:47:34.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78010690 con 0x7f5c880cd880 2023-10-26T05:47:34.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c880c7bc0 con 0x7f5c880cd880 2023-10-26T05:47:34.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88066b10 con 0x7f5c880cd880 2023-10-26T05:47:34.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5c780107f0 con 0x7f5c880cd880 2023-10-26T05:47:34.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c6ffff640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5c74066470 0x7f5c74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c78014070 con 0x7f5c880cd880 2023-10-26T05:47:34.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c867fc640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5c74066470 0x7f5c74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.604+0000 7f5c867fc640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5c74066470 0x7f5c74068930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0af000 tx=0x7f5c7c01afb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.608+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5c78050090 con 0x7f5c880cd880 2023-10-26T05:47:34.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.748+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c880721e0 con 0x7f5c880cd880 2023-10-26T05:47:34.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.752+0000 7f5c6ffff640 1 -- 172.21.15.71:0/3819458197 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f5c78055a70 con 0x7f5c880cd880 2023-10-26T05:47:34.759 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:34.759 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.752+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5c74066470 msgr2=0x7f5c74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.752+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5c74066470 0x7f5c74068930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0af000 tx=0x7f5c7c01afb0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.752+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 msgr2=0x7f5c880c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.752+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 0x7f5c880c4e30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800e9f0 tx=0x7f5c7800eeb0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 shutdown_connections 2023-10-26T05:47:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c880d0850 0x7f5c8813f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c880d0470 0x7f5c8815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5c74066470 0x7f5c74068930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 --2- 172.21.15.71:0/3819458197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c880cd880 0x7f5c880c4e30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 >> 172.21.15.71:0/3819458197 conn(0x7f5c880cdd80 msgr2=0x7f5c880c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 shutdown_connections 2023-10-26T05:47:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.756+0000 7f5c8d8ba640 1 -- 172.21.15.71:0/3819458197 wait complete. 2023-10-26T05:47:34.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:34.776 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:34.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/2848159681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 msgr2=0x7f1dfc15cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/2848159681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc15cd00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009840 tx=0x7f1df001c800 comp rx=0 tx=0).stop 2023-10-26T05:47:34.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/2848159681 shutdown_connections 2023-10-26T05:47:34.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/2848159681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dfc15a8d0 0x7f1dfc15ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/2848159681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc15cd00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/2848159681 >> 172.21.15.71:0/2848159681 conn(0x7f1dfc0cbb50 msgr2=0x7f1dfc0cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:34.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/2848159681 shutdown_connections 2023-10-26T05:47:34.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/2848159681 wait complete. 2023-10-26T05:47:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 Processor -- start 2023-10-26T05:47:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- start start 2023-10-26T05:47:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dfc0b4580 0x7f1dfc152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc0b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dfc15a8d0 0x7f1dfc0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c81a0 con 0x7f1dfc0c9720 2023-10-26T05:47:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc157070 con 0x7f1dfc0b4580 2023-10-26T05:47:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0c7840 con 0x7f1dfc15a8d0 2023-10-26T05:47:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc0b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc0b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55834/0 (socket says 172.21.15.71:55834) 2023-10-26T05:47:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 -- 172.21.15.71:0/4013335206 learned_addr learned my addr 172.21.15.71:0/4013335206 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 -- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dfc15a8d0 msgr2=0x7f1dfc0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dfc15a8d0 0x7f1dfc0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 -- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dfc0b4580 msgr2=0x7f1dfc152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dfc0b4580 0x7f1dfc152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 -- 172.21.15.71:0/4013335206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00094a0 con 0x7f1dfc0c9720 2023-10-26T05:47:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1dfaffd640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc0b2b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009790 tx=0x7f1df0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df00044f0 con 0x7f1dfc0c9720 2023-10-26T05:47:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1df0005120 con 0x7f1dfc0c9720 2023-10-26T05:47:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0005510 con 0x7f1dfc0c9720 2023-10-26T05:47:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc0c7ae0 con 0x7f1dfc0c9720 2023-10-26T05:47:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc0b3710 con 0x7f1dfc0c9720 2023-10-26T05:47:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.896+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc066d00 con 0x7f1dfc0c9720 2023-10-26T05:47:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.900+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1df00b4040 con 0x7f1dfc0c9720 2023-10-26T05:47:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.900+0000 7f1de3fff640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1de80664a0 0x7f1de8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:34.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.900+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1df011da30 con 0x7f1dfc0c9720 2023-10-26T05:47:34.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.900+0000 7f1dfb7fe640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1de80664a0 0x7f1de8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.900+0000 7f1dfb7fe640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1de80664a0 0x7f1de8068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000f70 tx=0x7f1dec007a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:34.904+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1df00eae10 con 0x7f1dfc0c9720 2023-10-26T05:47:35.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1dfc0afbe0 con 0x7f1dfc0c9720 2023-10-26T05:47:35.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1de3fff640 1 -- 172.21.15.71:0/4013335206 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1df00b2020 con 0x7f1dfc0c9720 2023-10-26T05:47:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1de80664a0 msgr2=0x7f1de8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1de80664a0 0x7f1de8068960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1dec000f70 tx=0x7f1dec007a40 comp rx=0 tx=0).stop 2023-10-26T05:47:35.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 msgr2=0x7f1dfc0b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc0b2b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009790 tx=0x7f1df0004040 comp rx=0 tx=0).stop 2023-10-26T05:47:35.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 shutdown_connections 2023-10-26T05:47:35.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dfc15a8d0 0x7f1dfc0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1de80664a0 0x7f1de8068960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dfc0c9720 0x7f1dfc0b2b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 --2- 172.21.15.71:0/4013335206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dfc0b4580 0x7f1dfc152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 >> 172.21.15.71:0/4013335206 conn(0x7f1dfc0cbb50 msgr2=0x7f1dfc159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 shutdown_connections 2023-10-26T05:47:35.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.044+0000 7f1e01c5d640 1 -- 172.21.15.71:0/4013335206 wait complete. 2023-10-26T05:47:35.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f0815ad0640 1 -- 172.21.15.71:0/1849102167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 msgr2=0x7f07fc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f0815ad0640 1 --2- 172.21.15.71:0/1849102167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f07fc0047d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07f800c260 tx=0x7f07f801cfb0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f080ffff640 1 -- 172.21.15.71:0/1849102167 reap_dead start 2023-10-26T05:47:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f0815ad0640 1 -- 172.21.15.71:0/1849102167 shutdown_connections 2023-10-26T05:47:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f0815ad0640 1 -- 172.21.15.71:0/1849102167 >> 172.21.15.71:0/1849102167 conn(0x7f08100aec60 msgr2=0x7f08100af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f0815ad0640 1 -- 172.21.15.71:0/1849102167 shutdown_connections 2023-10-26T05:47:35.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.172+0000 7f0815ad0640 1 -- 172.21.15.71:0/1849102167 wait complete. 2023-10-26T05:47:35.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 Processor -- start 2023-10-26T05:47:35.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- start start 2023-10-26T05:47:35.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f0810135ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081015a090 0x7f081013a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f081015a510 0x7f081013aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08101519c0 con 0x7f07fc003fb0 2023-10-26T05:47:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0810151f80 con 0x7f081015a090 2023-10-26T05:47:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0810151790 con 0x7f081015a510 2023-10-26T05:47:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f0810135ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f0810135ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55844/0 (socket says 172.21.15.71:55844) 2023-10-26T05:47:35.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 -- 172.21.15.71:0/2435111609 learned_addr learned my addr 172.21.15.71:0/2435111609 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:35.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080effd640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081015a090 0x7f081013a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080ffff640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f081015a510 0x7f081013aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 -- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f081015a510 msgr2=0x7f081013aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f081015a510 0x7f081013aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 -- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081015a090 msgr2=0x7f081013a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081015a090 0x7f081013a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 -- 172.21.15.71:0/2435111609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081015c070 con 0x7f07fc003fb0 2023-10-26T05:47:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f080f7fe640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f0810135ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f080000d700 tx=0x7f080000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800009c30 con 0x7f07fc003fb0 2023-10-26T05:47:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0800009e80 con 0x7f07fc003fb0 2023-10-26T05:47:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081015c350 con 0x7f07fc003fb0 2023-10-26T05:47:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800010680 con 0x7f07fc003fb0 2023-10-26T05:47:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0810136020 con 0x7f07fc003fb0 2023-10-26T05:47:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.176+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0810066b10 con 0x7f07fc003fb0 2023-10-26T05:47:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.180+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f08000107e0 con 0x7f07fc003fb0 2023-10-26T05:47:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.180+0000 7f0807fff640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07f0066470 0x7f07f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.180+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08000842d0 con 0x7f07fc003fb0 2023-10-26T05:47:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.180+0000 7f080effd640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07f0066470 0x7f07f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.180+0000 7f080effd640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07f0066470 0x7f07f0068930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07f80ab000 tx=0x7f07f8019d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.184+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0800051740 con 0x7f07fc003fb0 2023-10-26T05:47:35.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.328+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08100721e0 con 0x7f07fc003fb0 2023-10-26T05:47:35.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.328+0000 7f0807fff640 1 -- 172.21.15.71:0/2435111609 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0800054d60 con 0x7f07fc003fb0 2023-10-26T05:47:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:35.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07f0066470 msgr2=0x7f07f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07f0066470 0x7f07f0068930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07f80ab000 tx=0x7f07f8019d50 comp rx=0 tx=0).stop 2023-10-26T05:47:35.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 msgr2=0x7f0810135ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f0810135ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f080000d700 tx=0x7f080000dbc0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 shutdown_connections 2023-10-26T05:47:35.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f081015a510 0x7f081013aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f081015a090 0x7f081013a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07f0066470 0x7f07f0068930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 --2- 172.21.15.71:0/2435111609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07fc003fb0 0x7f0810135ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 >> 172.21.15.71:0/2435111609 conn(0x7f08100aec60 msgr2=0x7f08100b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 shutdown_connections 2023-10-26T05:47:35.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.332+0000 7f0815ad0640 1 -- 172.21.15.71:0/2435111609 wait complete. 2023-10-26T05:47:35.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:35.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- 172.21.15.71:0/3614777083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36040cede0 msgr2=0x7f360415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 --2- 172.21.15.71:0/3614777083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36040cede0 0x7f360415ce50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009900 tx=0x7f35f801c7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- 172.21.15.71:0/3614777083 shutdown_connections 2023-10-26T05:47:35.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 --2- 172.21.15.71:0/3614777083 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f360415d390 0x7f360415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 --2- 172.21.15.71:0/3614777083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36040cede0 0x7f360415ce50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- 172.21.15.71:0/3614777083 >> 172.21.15.71:0/3614777083 conn(0x7f36040b0f20 msgr2=0x7f36040b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- 172.21.15.71:0/3614777083 shutdown_connections 2023-10-26T05:47:35.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- 172.21.15.71:0/3614777083 wait complete. 2023-10-26T05:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 Processor -- start 2023-10-26T05:47:35.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- start start 2023-10-26T05:47:35.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040cede0 0x7f360406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 0x7f36041400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f360415d390 0x7f360414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604147f80 con 0x7f360415c720 2023-10-26T05:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360406d140 con 0x7f36040cede0 2023-10-26T05:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360406ce60 con 0x7f360415d390 2023-10-26T05:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 0x7f36041400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 0x7f36041400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55870/0 (socket says 172.21.15.71:55870) 2023-10-26T05:47:35.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 -- 172.21.15.71:0/898925465 learned_addr learned my addr 172.21.15.71:0/898925465 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 -- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f360415d390 msgr2=0x7f360414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f3603fff640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040cede0 0x7f360406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f3608cc4640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f360415d390 0x7f360414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f360415d390 0x7f360414f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 -- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040cede0 msgr2=0x7f360406bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040cede0 0x7f360406bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 -- 172.21.15.71:0/898925465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8009560 con 0x7f360415c720 2023-10-26T05:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f3603fff640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040cede0 0x7f360406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:35.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f36037fe640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 0x7f36041400a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009850 tx=0x7f35f80a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:35.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8003800 con 0x7f360415c720 2023-10-26T05:47:35.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f35f8003960 con 0x7f360415c720 2023-10-26T05:47:35.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.472+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36041478c0 con 0x7f360415c720 2023-10-26T05:47:35.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8003d30 con 0x7f360415c720 2023-10-26T05:47:35.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360406c2e0 con 0x7f360415c720 2023-10-26T05:47:35.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3604147720 con 0x7f360415c720 2023-10-26T05:47:35.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f35f80b4070 con 0x7f360415c720 2023-10-26T05:47:35.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f3600ff9640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f35f0066470 0x7f35f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f811e8d0 con 0x7f360415c720 2023-10-26T05:47:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f3603fff640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f35f0066470 0x7f35f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.476+0000 7f3603fff640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f35f0066470 0x7f35f0068930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36040c52b0 tx=0x7f35f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:35.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.480+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3604147720 con 0x7f360415c720 2023-10-26T05:47:35.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.624+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36041480c0 con 0x7f360415c720 2023-10-26T05:47:35.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.624+0000 7f3600ff9640 1 -- 172.21.15.71:0/898925465 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36041480c0 con 0x7f360415c720 2023-10-26T05:47:35.632 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f35f0066470 msgr2=0x7f35f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f35f0066470 0x7f35f0068930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36040c52b0 tx=0x7f35f4009040 comp rx=0 tx=0).stop 2023-10-26T05:47:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 msgr2=0x7f36041400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 0x7f36041400a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009850 tx=0x7f35f80a56c0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 shutdown_connections 2023-10-26T05:47:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f360415d390 0x7f360414f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f35f0066470 0x7f35f0068930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f360415c720 0x7f36041400a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 --2- 172.21.15.71:0/898925465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36040cede0 0x7f360406bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 >> 172.21.15.71:0/898925465 conn(0x7f36040b0f20 msgr2=0x7f3604158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 shutdown_connections 2023-10-26T05:47:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.628+0000 7f360a727640 1 -- 172.21.15.71:0/898925465 wait complete. 2023-10-26T05:47:35.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:47:35.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7ab010e640 1 -- 172.21.15.71:0/3929260819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 msgr2=0x7f7a94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7ab010e640 1 --2- 172.21.15.71:0/3929260819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7a94003dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40016b0 tx=0x7f7aa401c770 comp rx=0 tx=0).stop 2023-10-26T05:47:35.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7aae6ab640 1 -- 172.21.15.71:0/3929260819 reap_dead start 2023-10-26T05:47:35.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7ab010e640 1 -- 172.21.15.71:0/3929260819 shutdown_connections 2023-10-26T05:47:35.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7ab010e640 1 -- 172.21.15.71:0/3929260819 >> 172.21.15.71:0/3929260819 conn(0x7f7aa80cdd80 msgr2=0x7f7aa80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7ab010e640 1 -- 172.21.15.71:0/3929260819 shutdown_connections 2023-10-26T05:47:35.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.756+0000 7f7ab010e640 1 -- 172.21.15.71:0/3929260819 wait complete. 2023-10-26T05:47:35.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 Processor -- start 2023-10-26T05:47:35.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- start start 2023-10-26T05:47:35.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7aa8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aa81556f0 0x7f7aa80cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa81551a0 0x7f7aa80cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa806a650 con 0x7f7a940039b0 2023-10-26T05:47:35.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80cb260 con 0x7f7aa81551a0 2023-10-26T05:47:35.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80caca0 con 0x7f7aa81556f0 2023-10-26T05:47:35.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7aa8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7aa8154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55900/0 (socket says 172.21.15.71:55900) 2023-10-26T05:47:35.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 -- 172.21.15.71:0/1754830424 learned_addr learned my addr 172.21.15.71:0/1754830424 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:35.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aad6a9640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aa81556f0 0x7f7aa80cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aae6ab640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa81551a0 0x7f7aa80cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 -- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aa81556f0 msgr2=0x7f7aa80cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aa81556f0 0x7f7aa80cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 -- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa81551a0 msgr2=0x7f7aa80cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa81551a0 0x7f7aa80cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 -- 172.21.15.71:0/1754830424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa80caf80 con 0x7f7a940039b0 2023-10-26T05:47:35.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aae6ab640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa81551a0 0x7f7aa80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:35.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7aadeaa640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7aa8154b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005040 tx=0x7f7aa4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:35.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40a9040 con 0x7f7a940039b0 2023-10-26T05:47:35.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7aa4006150 con 0x7f7a940039b0 2023-10-26T05:47:35.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa80c4fd0 con 0x7f7a940039b0 2023-10-26T05:47:35.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40a57b0 con 0x7f7a940039b0 2023-10-26T05:47:35.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa8158450 con 0x7f7a940039b0 2023-10-26T05:47:35.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa8066b10 con 0x7f7a940039b0 2023-10-26T05:47:35.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.760+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7aa40b5020 con 0x7f7a940039b0 2023-10-26T05:47:35.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.764+0000 7f7a9e7fc640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a8c066470 0x7f7a8c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:35.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.764+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa40aa070 con 0x7f7a940039b0 2023-10-26T05:47:35.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.764+0000 7f7aad6a9640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a8c066470 0x7f7a8c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:35.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.764+0000 7f7aad6a9640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a8c066470 0x7f7a8c068930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001850 tx=0x7f7a98000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:35.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.768+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7aa40eb140 con 0x7f7a940039b0 2023-10-26T05:47:35.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.936+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aa80b5120 con 0x7f7a940039b0 2023-10-26T05:47:35.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7a9e7fc640 1 -- 172.21.15.71:0/1754830424 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f7aa40bd030 con 0x7f7a940039b0 2023-10-26T05:47:35.945 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:35.945 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:47:35.945 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:47:35.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a8c066470 msgr2=0x7f7a8c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a8c066470 0x7f7a8c068930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001850 tx=0x7f7a98000ee0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 msgr2=0x7f7aa8154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:35.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7aa8154b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4005040 tx=0x7f7aa4003c20 comp rx=0 tx=0).stop 2023-10-26T05:47:35.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 shutdown_connections 2023-10-26T05:47:35.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7aa81551a0 0x7f7aa80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7aa81556f0 0x7f7aa80cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a8c066470 0x7f7a8c068930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 --2- 172.21.15.71:0/1754830424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a940039b0 0x7f7aa8154b30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:35.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 >> 172.21.15.71:0/1754830424 conn(0x7f7aa80cdd80 msgr2=0x7f7aa815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:35.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 shutdown_connections 2023-10-26T05:47:35.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:35.940+0000 7f7ab010e640 1 -- 172.21.15.71:0/1754830424 wait complete. 2023-10-26T05:47:35.962 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:36.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/2338383149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab40b2a60 msgr2=0x7f4ab40caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/2338383149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab40b2a60 0x7f4ab40caad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009bc0 tx=0x7f4aa401cb40 comp rx=0 tx=0).stop 2023-10-26T05:47:36.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/2338383149 shutdown_connections 2023-10-26T05:47:36.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/2338383149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab40b2a60 0x7f4ab40caad0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/2338383149 >> 172.21.15.71:0/2338383149 conn(0x7f4ab4069770 msgr2=0x7f4ab4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:36.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/2338383149 shutdown_connections 2023-10-26T05:47:36.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/2338383149 wait complete. 2023-10-26T05:47:36.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 Processor -- start 2023-10-26T05:47:36.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- start start 2023-10-26T05:47:36.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab40b2a60 0x7f4ab4133fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ab40cb010 0x7f4ab406a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 0x7f4ab406a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab4152450 con 0x7f4ab413c2f0 2023-10-26T05:47:36.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40ca890 con 0x7f4ab40b2a60 2023-10-26T05:47:36.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab4151ab0 con 0x7f4ab40cb010 2023-10-26T05:47:36.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 0x7f4ab406a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 0x7f4ab406a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55916/0 (socket says 172.21.15.71:55916) 2023-10-26T05:47:36.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 -- 172.21.15.71:0/3014901365 learned_addr learned my addr 172.21.15.71:0/3014901365 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:36.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab8f7e640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab40b2a60 0x7f4ab4133fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 -- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ab40cb010 msgr2=0x7f4ab406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ab40cb010 0x7f4ab406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 -- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab40b2a60 msgr2=0x7f4ab4133fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab40b2a60 0x7f4ab4133fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 -- 172.21.15.71:0/3014901365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa4009820 con 0x7f4ab413c2f0 2023-10-26T05:47:36.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab8f7e640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab40b2a60 0x7f4ab4133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:36.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab977f640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 0x7f4ab406a5a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c410 tx=0x7f4aac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:36.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac003600 con 0x7f4ab413c2f0 2023-10-26T05:47:36.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4aac003760 con 0x7f4ab413c2f0 2023-10-26T05:47:36.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aac003bb0 con 0x7f4ab413c2f0 2023-10-26T05:47:36.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab4151d90 con 0x7f4ab413c2f0 2023-10-26T05:47:36.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.068+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab4134510 con 0x7f4ab413c2f0 2023-10-26T05:47:36.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.072+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab4066b10 con 0x7f4ab413c2f0 2023-10-26T05:47:36.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.072+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4aac003d10 con 0x7f4ab413c2f0 2023-10-26T05:47:36.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.072+0000 7f4ab17fa640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a94066470 0x7f4a94068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.072+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4aac0831d0 con 0x7f4ab413c2f0 2023-10-26T05:47:36.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.072+0000 7f4ab8f7e640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a94066470 0x7f4a94068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.072+0000 7f4ab8f7e640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a94066470 0x7f4a94068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40ae370 tx=0x7f4a9c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:36.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.076+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4aac0506e0 con 0x7f4ab413c2f0 2023-10-26T05:47:36.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.216+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ab4152590 con 0x7f4ab413c2f0 2023-10-26T05:47:36.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4ab17fa640 1 -- 172.21.15.71:0/3014901365 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f4ab4152590 con 0x7f4ab413c2f0 2023-10-26T05:47:36.225 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:36.226 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:36.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a94066470 msgr2=0x7f4a94068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a94066470 0x7f4a94068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40ae370 tx=0x7f4a9c009040 comp rx=0 tx=0).stop 2023-10-26T05:47:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 msgr2=0x7f4ab406a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 0x7f4ab406a5a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4aac00c410 tx=0x7f4aac00c8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 shutdown_connections 2023-10-26T05:47:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a94066470 0x7f4a94068930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ab413c2f0 0x7f4ab406a5a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ab40cb010 0x7f4ab406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 --2- 172.21.15.71:0/3014901365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ab40b2a60 0x7f4ab4133fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 >> 172.21.15.71:0/3014901365 conn(0x7f4ab4069770 msgr2=0x7f4ab415beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 shutdown_connections 2023-10-26T05:47:36.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.220+0000 7f4abb1e2640 1 -- 172.21.15.71:0/3014901365 wait complete. 2023-10-26T05:47:36.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:47:36.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 -- 172.21.15.71:0/3736315635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 msgr2=0x7f3c98070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3736315635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c98070690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000ad90 tx=0x7f3c9001dc50 comp rx=0 tx=0).stop 2023-10-26T05:47:36.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 -- 172.21.15.71:0/3736315635 shutdown_connections 2023-10-26T05:47:36.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3736315635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c98070690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3736315635 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c980ac740 0x7f3c9815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 -- 172.21.15.71:0/3736315635 >> 172.21.15.71:0/3736315635 conn(0x7f3c98069770 msgr2=0x7f3c98069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:36.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 -- 172.21.15.71:0/3736315635 shutdown_connections 2023-10-26T05:47:36.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 -- 172.21.15.71:0/3736315635 wait complete. 2023-10-26T05:47:36.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 Processor -- start 2023-10-26T05:47:36.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.360+0000 7f3c9d398640 1 -- start start 2023-10-26T05:47:36.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c980ac740 0x7f3c98151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c9813d420 0x7f3c980adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c981300f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9814e640 con 0x7f3c9815b9f0 2023-10-26T05:47:36.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98150ca0 con 0x7f3c980ac740 2023-10-26T05:47:36.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c980b0f00 con 0x7f3c9813d420 2023-10-26T05:47:36.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c981300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c981300f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55928/0 (socket says 172.21.15.71:55928) 2023-10-26T05:47:36.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 -- 172.21.15.71:0/3592489397 learned_addr learned my addr 172.21.15.71:0/3592489397 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:36.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c967fc640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c9813d420 0x7f3c980adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c96ffd640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c980ac740 0x7f3c98151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 -- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c9813d420 msgr2=0x7f3c980adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c9813d420 0x7f3c980adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 -- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c980ac740 msgr2=0x7f3c98151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c980ac740 0x7f3c98151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 -- 172.21.15.71:0/3592489397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9000a9f0 con 0x7f3c9815b9f0 2023-10-26T05:47:36.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c96ffd640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c980ac740 0x7f3c98151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:36.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c977fe640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c981300f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3c9001c300 tx=0x7f3c900a6af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:36.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90003d90 con 0x7f3c9815b9f0 2023-10-26T05:47:36.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c90004030 con 0x7f3c9815b9f0 2023-10-26T05:47:36.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c900043e0 con 0x7f3c9815b9f0 2023-10-26T05:47:36.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c980b11e0 con 0x7f3c9815b9f0 2023-10-26T05:47:36.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c980b17a0 con 0x7f3c9815b9f0 2023-10-26T05:47:36.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c98066b10 con 0x7f3c9815b9f0 2023-10-26T05:47:36.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c900b5040 con 0x7f3c9815b9f0 2023-10-26T05:47:36.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c7f7fe640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c84066470 0x7f3c84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:36.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c9011df80 con 0x7f3c9815b9f0 2023-10-26T05:47:36.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.364+0000 7f3c96ffd640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c84066470 0x7f3c84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:36.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.368+0000 7f3c96ffd640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c84066470 0x7f3c84068930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c88003e40 tx=0x7f3c88009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:36.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.368+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3c900eb400 con 0x7f3c9815b9f0 2023-10-26T05:47:36.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c9815be40 con 0x7f3c9815b9f0 2023-10-26T05:47:36.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c7f7fe640 1 -- 172.21.15.71:0/3592489397 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f3c9815be40 con 0x7f3c9815b9f0 2023-10-26T05:47:36.566 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:36.566 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:47:36.566 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:47:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c84066470 msgr2=0x7f3c84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c84066470 0x7f3c84068930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c88003e40 tx=0x7f3c88009040 comp rx=0 tx=0).stop 2023-10-26T05:47:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 msgr2=0x7f3c981300f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c981300f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3c9001c300 tx=0x7f3c900a6af0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 shutdown_connections 2023-10-26T05:47:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c84066470 0x7f3c84068930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c9815b9f0 0x7f3c981300f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c9813d420 0x7f3c980adf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 --2- 172.21.15.71:0/3592489397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c980ac740 0x7f3c98151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 >> 172.21.15.71:0/3592489397 conn(0x7f3c98069770 msgr2=0x7f3c9813d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 shutdown_connections 2023-10-26T05:47:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:36.560+0000 7f3c9d398640 1 -- 172.21.15.71:0/3592489397 wait complete. 2023-10-26T05:47:36.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:47:21.405825+0000', 'modified': '2023-10-26T05:47:23.437258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4369}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4369': {'gid': 4369, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/2733906153', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 2733906153}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2733906153}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:47:36.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:47:36.988 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-26T05:47:37.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 -- 172.21.15.71:0/3792197054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04004d10 msgr2=0x7fdb04005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3792197054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb04004d10 0x7fdb04005190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009f70 tx=0x7fdb1001e6f0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb1eb95640 1 -- 172.21.15.71:0/3792197054 reap_dead start 2023-10-26T05:47:37.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 -- 172.21.15.71:0/3792197054 shutdown_connections 2023-10-26T05:47:37.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 -- 172.21.15.71:0/3792197054 >> 172.21.15.71:0/3792197054 conn(0x7fdb1806b8c0 msgr2=0x7fdb1806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 -- 172.21.15.71:0/3792197054 shutdown_connections 2023-10-26T05:47:37.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 -- 172.21.15.71:0/3792197054 wait complete. 2023-10-26T05:47:37.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 Processor -- start 2023-10-26T05:47:37.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.120+0000 7fdb205f8640 1 -- start start 2023-10-26T05:47:37.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04004d10 0x7fdb18152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb180c86f0 0x7fdb18140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 0x7fdb180cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb180cbed0 con 0x7fdb180cb510 2023-10-26T05:47:37.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18152a50 con 0x7fdb04004d10 2023-10-26T05:47:37.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb180c8b50 con 0x7fdb180c86f0 2023-10-26T05:47:37.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 0x7fdb180cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 0x7fdb180cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55940/0 (socket says 172.21.15.71:55940) 2023-10-26T05:47:37.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 -- 172.21.15.71:0/3317193855 learned_addr learned my addr 172.21.15.71:0/3317193855 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:37.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1e394640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04004d10 0x7fdb18152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1db93640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb180c86f0 0x7fdb18140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 -- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb180c86f0 msgr2=0x7fdb18140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb180c86f0 0x7fdb18140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 -- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04004d10 msgr2=0x7fdb18152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04004d10 0x7fdb18152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 -- 172.21.15.71:0/3317193855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18150100 con 0x7fdb180cb510 2023-10-26T05:47:37.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 0x7fdb180cb990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009f70 tx=0x7fdb100a8f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb0effd640 1 -- 172.21.15.71:0/3317193855 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10004a50 con 0x7fdb180cb510 2023-10-26T05:47:37.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb0effd640 1 -- 172.21.15.71:0/3317193855 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb10004ca0 con 0x7fdb180cb510 2023-10-26T05:47:37.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb181552a0 con 0x7fdb180cb510 2023-10-26T05:47:37.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb0effd640 1 -- 172.21.15.71:0/3317193855 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10003b80 con 0x7fdb180cb510 2023-10-26T05:47:37.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb18150630 con 0x7fdb180cb510 2023-10-26T05:47:37.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb0effd640 1 -- 172.21.15.71:0/3317193855 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdb10003ce0 con 0x7fdb180cb510 2023-10-26T05:47:37.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb0effd640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdafc066470 0x7fdafc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb0effd640 1 -- 172.21.15.71:0/3317193855 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb100ac070 con 0x7fdb180cb510 2023-10-26T05:47:37.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.124+0000 7fdb1e394640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdafc066470 0x7fdafc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 Processor -- start 2023-10-26T05:47:37.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- start start 2023-10-26T05:47:37.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdafc066470 0x7fdafc068930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006fd0 tx=0x7fdb14006b40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e92a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e93360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2c87cc0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2cea920 con 0x55f5c2e92f80 2023-10-26T05:47:37.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2ce7e50 con 0x55f5c2edd2e0 2023-10-26T05:47:37.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55950/0 (socket says 172.21.15.71:55950) 2023-10-26T05:47:37.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 -- 172.21.15.71:0/3672903854 learned_addr learned my addr 172.21.15.71:0/3672903854 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:37.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1db93640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e92a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e93360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 -- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 msgr2=0x55f5c2e92a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e92a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 -- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 msgr2=0x55f5c2e93360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e93360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 -- 172.21.15.71:0/3672903854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c29dc890 con 0x55f5c2edc9c0 2023-10-26T05:47:37.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:37.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1db93640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e92a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:37.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb1e394640 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edcda0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006820 tx=0x7fdb14014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35ec0ccb60f1561 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdafbfff640 1 -- 172.21.15.71:0/3672903854 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb140127d0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdafbfff640 1 -- 172.21.15.71:0/3672903854 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb14012930 con 0x55f5c2edc9c0 2023-10-26T05:47:37.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdafbfff640 1 -- 172.21.15.71:0/3672903854 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb14012ce0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- 172.21.15.71:0/3672903854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 msgr2=0x55f5c2edcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edcda0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006820 tx=0x7fdb14014000 comp rx=0 tx=0).stop 2023-10-26T05:47:37.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- 172.21.15.71:0/3672903854 shutdown_connections 2023-10-26T05:47:37.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e93360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e92a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 --2- 172.21.15.71:0/3672903854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edcda0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- 172.21.15.71:0/3672903854 >> 172.21.15.71:0/3672903854 conn(0x55f5c2e8a2f0 msgr2=0x55f5c2e91320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- 172.21.15.71:0/3672903854 shutdown_connections 2023-10-26T05:47:37.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- 172.21.15.71:0/3672903854 wait complete. 2023-10-26T05:47:37.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 -- start start 2023-10-26T05:47:37.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.128+0000 7fdb23742000 1 Processor -- start 2023-10-26T05:47:37.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2e988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e9ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2ce3740 con 0x55f5c2edc9c0 2023-10-26T05:47:37.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2ca2d10 con 0x55f5c2e92f80 2023-10-26T05:47:37.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2ca29b0 con 0x55f5c2edd2e0 2023-10-26T05:47:37.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 10 client.0 _ll_register_callbacks cb 0x55f5c2e7e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:47:37.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2e988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2e988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55958/0 (socket says 172.21.15.71:55958) 2023-10-26T05:47:37.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 -- 172.21.15.71:0/4247266553 learned_addr learned my addr 172.21.15.71:0/4247266553 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:37.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1e394640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e9ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 -- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 msgr2=0x55f5c2e9ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e9ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 -- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 msgr2=0x55f5c2e983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 -- 172.21.15.71:0/4247266553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1815bcd0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1db93640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2e988f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006af0 tx=0x7fdb08004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd71bd5cf6cb29fd server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:47:37.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 1 -- 172.21.15.71:0/4247266553 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004ea0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 1 -- 172.21.15.71:0/4247266553 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb08017820 con 0x55f5c2edc9c0 2023-10-26T05:47:37.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5c2c97370 con 0x55f5c2edc9c0 2023-10-26T05:47:37.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 1 -- 172.21.15.71:0/4247266553 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb080179e0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 10 client.4686 fetch_fsmap learned FSMap version 6 2023-10-26T05:47:37.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55f5c2ca5460 con 0x55f5c2edc9c0 2023-10-26T05:47:37.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 1 -- 172.21.15.71:0/4247266553 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55f5c2c97370 con 0x55f5c2edc9c0 2023-10-26T05:47:37.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 1 -- 172.21.15.71:0/4247266553 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb08017b40 con 0x55f5c2edc9c0 2023-10-26T05:47:37.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 10 client.4686 fetch_fsmap finished waiting for FSMap version 6 2023-10-26T05:47:37.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 10 client.4686 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-26T05:47:37.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 20 client.4686 populate_metadata read hostname 'smithi071' 2023-10-26T05:47:37.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2e9f5f0 0x55f5c2eeea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 4 client.4686 mds_command: new command op to 4369 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:47:37.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 --> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5c2c7d2d0 con 0x55f5c2e9f5f0 2023-10-26T05:47:37.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1e394640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2e9f5f0 0x55f5c2eeea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 10 client.4686 ms_handle_connect on v2:172.21.15.86:6836/2733906153 2023-10-26T05:47:37.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb1e394640 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2e9f5f0 0x55f5c2eeea70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655a856a9e71c07e server_cookie=1c530c652f59e48c in_seq=0 out_seq=0 2023-10-26T05:47:37.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 1 -- 172.21.15.71:0/4247266553 <== mds.0 v2:172.21.15.86:6836/2733906153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f5c2c7d2d0 con 0x55f5c2e9f5f0 2023-10-26T05:47:37.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 10 client.4686 handle_command_reply: tid=0 2023-10-26T05:47:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 client.4686 shutdown 2023-10-26T05:47:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 msgr2=0x55f5c2e988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2e988f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb08006af0 tx=0x7fdb08004610 comp rx=0 tx=0).stop 2023-10-26T05:47:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 shutdown_connections 2023-10-26T05:47:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2e9f5f0 0x55f5c2eeea70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2edd2e0 0x55f5c2e9ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2e988f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdaf97fa640 0 client.4686 ms_handle_reset on v2:172.21.15.86:6836/2733906153 2023-10-26T05:47:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 --2- 172.21.15.71:0/4247266553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2e92f80 0x55f5c2e983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 >> 172.21.15.71:0/4247266553 conn(0x55f5c2e8a2f0 msgr2=0x55f5c2e91a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 shutdown_connections 2023-10-26T05:47:37.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 1 -- 172.21.15.71:0/4247266553 wait complete. 2023-10-26T05:47:37.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.132+0000 7fdb23742000 20 client.4686 trim_cache size 0 max 16384 2023-10-26T05:47:37.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 Processor -- start 2023-10-26T05:47:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 -- start start 2023-10-26T05:47:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2edb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2eeca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c299d940 con 0x55f5c2edc9c0 2023-10-26T05:47:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2cdda40 con 0x55f5c2edd2e0 2023-10-26T05:47:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2c91880 con 0x55f5c2e92f80 2023-10-26T05:47:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55974/0 (socket says 172.21.15.71:55974) 2023-10-26T05:47:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 -- 172.21.15.71:0/4066632529 learned_addr learned my addr 172.21.15.71:0/4066632529 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2eeca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1e394640 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2edb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 -- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 msgr2=0x55f5c2edb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2edb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 -- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 msgr2=0x55f5c2eeca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.148+0000 7fdb1db93640 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2eeca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 -- 172.21.15.71:0/4066632529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c2c8b760 con 0x55f5c2edc9c0 2023-10-26T05:47:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2eeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edb800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007010 tx=0x7fdb08001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33819affb5c88622 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb0cff9640 1 -- 172.21.15.71:0/4066632529 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004ea0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb0cff9640 1 -- 172.21.15.71:0/4066632529 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb0802d030 con 0x55f5c2edc9c0 2023-10-26T05:47:37.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb0cff9640 1 -- 172.21.15.71:0/4066632529 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08013660 con 0x55f5c2edc9c0 2023-10-26T05:47:37.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/4066632529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 msgr2=0x55f5c2edb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edb800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007010 tx=0x7fdb08001ba0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/4066632529 shutdown_connections 2023-10-26T05:47:37.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2eeca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2edb800 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- 172.21.15.71:0/4066632529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2edb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/4066632529 >> 172.21.15.71:0/4066632529 conn(0x55f5c2e8a2f0 msgr2=0x55f5c2ed9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/4066632529 shutdown_connections 2023-10-26T05:47:37.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/4066632529 wait complete. 2023-10-26T05:47:37.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- start start 2023-10-26T05:47:37.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 Processor -- start 2023-10-26T05:47:37.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2eefae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2f00640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2f00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2cac720 con 0x55f5c2edc9c0 2023-10-26T05:47:37.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c2ce7740 con 0x55f5c2edd2e0 2023-10-26T05:47:37.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5c29dd1b0 con 0x55f5c2e92f80 2023-10-26T05:47:37.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 10 client.0 _ll_register_callbacks cb 0x55f5c2ce1ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:47:37.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2f00640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2f00640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55976/0 (socket says 172.21.15.71:55976) 2023-10-26T05:47:37.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 -- 172.21.15.71:0/2746642820 learned_addr learned my addr 172.21.15.71:0/2746642820 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:37.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1eb95640 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2f00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 -- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 msgr2=0x55f5c2eefae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2eefae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 -- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 msgr2=0x55f5c2f00b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:47:37.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2f00b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 -- 172.21.15.71:0/2746642820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5c2ca5460 con 0x55f5c2edc9c0 2023-10-26T05:47:37.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb1db93640 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2f00640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000f30 tx=0x7fdb08014c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648d0ec9bf591221 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdaf97fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:47:37.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdaf97fa640 1 -- 172.21.15.71:0/2746642820 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08004ea0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdaf97fa640 1 -- 172.21.15.71:0/2746642820 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb08032440 con 0x55f5c2edc9c0 2023-10-26T05:47:37.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5c2ee9e90 con 0x55f5c2edc9c0 2023-10-26T05:47:37.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdaf97fa640 1 -- 172.21.15.71:0/2746642820 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb08001d90 con 0x55f5c2edc9c0 2023-10-26T05:47:37.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 10 client.4692 fetch_fsmap learned FSMap version 6 2023-10-26T05:47:37.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55f5c29dccc0 con 0x55f5c2edc9c0 2023-10-26T05:47:37.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdaf97fa640 1 -- 172.21.15.71:0/2746642820 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55f5c2ee9e90 con 0x55f5c2edc9c0 2023-10-26T05:47:37.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdaf97fa640 1 -- 172.21.15.71:0/2746642820 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdb08032940 con 0x55f5c2edc9c0 2023-10-26T05:47:37.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 10 client.4692 fetch_fsmap finished waiting for FSMap version 6 2023-10-26T05:47:37.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 10 client.4692 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-26T05:47:37.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 20 client.4692 populate_metadata read hostname 'smithi071' 2023-10-26T05:47:37.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2f43a70 0x55f5c2f43e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 4 client.4692 mds_command: new command op to 4369 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:47:37.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.152+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 --> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f5c2c7b6f0 con 0x55f5c2f43a70 2023-10-26T05:47:37.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb1e394640 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2f43a70 0x55f5c2f43e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdaf97fa640 10 client.4692 ms_handle_connect on v2:172.21.15.86:6836/2733906153 2023-10-26T05:47:37.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb1e394640 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2f43a70 0x55f5c2f43e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e0a9bd07ce18fc server_cookie=5a46297c7f8416af in_seq=0 out_seq=0 2023-10-26T05:47:37.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdaf97fa640 1 -- 172.21.15.71:0/2746642820 <== mds.0 v2:172.21.15.86:6836/2733906153 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55f5c2c7b6f0 con 0x55f5c2f43a70 2023-10-26T05:47:37.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdaf97fa640 10 client.4692 handle_command_reply: tid=0 2023-10-26T05:47:37.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 client.4692 shutdown 2023-10-26T05:47:37.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 msgr2=0x55f5c2f00640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2f00640 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb08000f30 tx=0x7fdb08014c60 comp rx=0 tx=0).stop 2023-10-26T05:47:37.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 shutdown_connections 2023-10-26T05:47:37.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:6836/2733906153,v1:172.21.15.86:6837/2733906153] conn(0x55f5c2f43a70 0x55f5c2f43e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5c2edd2e0 0x55f5c2f00b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdaf97fa640 0 client.4692 ms_handle_reset on v2:172.21.15.86:6836/2733906153 2023-10-26T05:47:37.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f5c2edc9c0 0x55f5c2f00640 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 --2- 172.21.15.71:0/2746642820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f5c2e92f80 0x55f5c2eefae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 >> 172.21.15.71:0/2746642820 conn(0x55f5c2e8a2f0 msgr2=0x55f5c2e9e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 shutdown_connections 2023-10-26T05:47:37.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 1 -- 172.21.15.71:0/2746642820 wait complete. 2023-10-26T05:47:37.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb23742000 20 client.4692 trim_cache size 0 max 16384 2023-10-26T05:47:37.183 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:47:37.183 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:47:37.183 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:47:37.183 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4369, 2023-10-26T05:47:37.183 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:47:37.184 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:47:37.184 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:47:37.184 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 15.060658290999999, 2023-10-26T05:47:37.184 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 6, 2023-10-26T05:47:37.184 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 20, 2023-10-26T05:47:37.185 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 20, 2023-10-26T05:47:37.185 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 22.374749436999998 2023-10-26T05:47:37.185 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:47:37.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdafc066470 msgr2=0x7fdafc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdafc066470 0x7fdafc068930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb14006fd0 tx=0x7fdb14006b40 comp rx=0 tx=0).stop 2023-10-26T05:47:37.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 msgr2=0x7fdb180cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 0x7fdb180cb990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009f70 tx=0x7fdb100a8f50 comp rx=0 tx=0).stop 2023-10-26T05:47:37.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 shutdown_connections 2023-10-26T05:47:37.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdafc066470 0x7fdafc068930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb180cb510 0x7fdb180cb990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb180c86f0 0x7fdb18140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 --2- 172.21.15.71:0/3317193855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb04004d10 0x7fdb18152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 >> 172.21.15.71:0/3317193855 conn(0x7fdb1806b8c0 msgr2=0x7fdb18156030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 shutdown_connections 2023-10-26T05:47:37.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.156+0000 7fdb205f8640 1 -- 172.21.15.71:0/3317193855 wait complete. 2023-10-26T05:47:37.189 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-26T05:47:37.211 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-26T05:47:37.211 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-26T05:47:37.212 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-26T05:47:37.212 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T05:47:37.213 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:37.213 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-26T05:47:37.213 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T05:47:37.213 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:37.214 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-26T05:47:37.214 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T05:47:37.214 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:37.214 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-26T05:47:37.215 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-26T05:47:37.215 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:37.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.216 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:47:37.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2023-10-26T05:47:37.276 INFO:teuthology.orchestra.run.smithi071.stderr:Cannot find device "ceph-brx" 2023-10-26T05:47:37.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:47:37.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.278 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:47:37.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2023-10-26T05:47:37.343 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2023-10-26T05:47:37.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:47:37.345 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-26T05:47:37.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:47:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/1573357912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0b2890 msgr2=0x7faf6c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 --2- 172.21.15.71:0/1573357912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0b2890 0x7faf6c0caad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faf6c06f190 tx=0x7faf6001c880 comp rx=0 tx=0).stop 2023-10-26T05:47:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/1573357912 shutdown_connections 2023-10-26T05:47:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 --2- 172.21.15.71:0/1573357912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0b2890 0x7faf6c0caad0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 --2- 172.21.15.71:0/1573357912 >> v2:172.21.15.86:3300/0 conn(0x7faf6c13c2f0 0x7faf6c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/1573357912 >> 172.21.15.71:0/1573357912 conn(0x7faf6c069770 msgr2=0x7faf6c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/1573357912 shutdown_connections 2023-10-26T05:47:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/1573357912 wait complete. 2023-10-26T05:47:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 Processor -- start 2023-10-26T05:47:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- start start 2023-10-26T05:47:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf6c0b2890 0x7faf6c130130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 0x7faf6c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf6c13c2f0 0x7faf6c14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c131300 con 0x7faf6c0cb010 2023-10-26T05:47:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c133470 con 0x7faf6c13c2f0 2023-10-26T05:47:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c130a60 con 0x7faf6c0b2890 2023-10-26T05:47:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 0x7faf6c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 0x7faf6c13d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55982/0 (socket says 172.21.15.71:55982) 2023-10-26T05:47:37.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 -- 172.21.15.71:0/2677564494 learned_addr learned my addr 172.21.15.71:0/2677564494 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:37.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6bfff640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf6c13c2f0 0x7faf6c14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6b7fe640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf6c0b2890 0x7faf6c130130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 -- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf6c0b2890 msgr2=0x7faf6c130130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf6c0b2890 0x7faf6c130130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 -- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf6c13c2f0 msgr2=0x7faf6c14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf6c13c2f0 0x7faf6c14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 -- 172.21.15.71:0/2677564494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009560 con 0x7faf6c0cb010 2023-10-26T05:47:37.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6bfff640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf6c13c2f0 0x7faf6c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:37.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf6affd640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 0x7faf6c13d1e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf60000c10 tx=0x7faf600a6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60004240 con 0x7faf6c0cb010 2023-10-26T05:47:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf600044b0 con 0x7faf6c0cb010 2023-10-26T05:47:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c130d40 con 0x7faf6c0cb010 2023-10-26T05:47:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60005280 con 0x7faf6c0cb010 2023-10-26T05:47:37.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.468+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c130670 con 0x7faf6c0cb010 2023-10-26T05:47:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.472+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c066b10 con 0x7faf6c0cb010 2023-10-26T05:47:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.472+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf600053e0 con 0x7faf6c0cb010 2023-10-26T05:47:37.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.472+0000 7faf53fff640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf58066470 0x7faf58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:37.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.472+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf6011e5a0 con 0x7faf6c0cb010 2023-10-26T05:47:37.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.472+0000 7faf6b7fe640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf58066470 0x7faf58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:37.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.472+0000 7faf6b7fe640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf58066470 0x7faf58068930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00b9a0 tx=0x7faf5c005c30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:37.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.476+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faf600eba20 con 0x7faf6c0cb010 2023-10-26T05:47:37.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.664+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faf6c133240 con 0x7faf6c0cb010 2023-10-26T05:47:37.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.664+0000 7faf53fff640 1 -- 172.21.15.71:0/2677564494 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7faf6c133240 con 0x7faf6c0cb010 2023-10-26T05:47:37.671 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:47:37.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf58066470 msgr2=0x7faf58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf58066470 0x7faf58068930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00b9a0 tx=0x7faf5c005c30 comp rx=0 tx=0).stop 2023-10-26T05:47:37.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 msgr2=0x7faf6c13d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:37.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 0x7faf6c13d1e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faf60000c10 tx=0x7faf600a6580 comp rx=0 tx=0).stop 2023-10-26T05:47:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 shutdown_connections 2023-10-26T05:47:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf6c13c2f0 0x7faf6c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf58066470 0x7faf58068930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf6c0cb010 0x7faf6c13d1e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 --2- 172.21.15.71:0/2677564494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf6c0b2890 0x7faf6c130130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:37.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 >> 172.21.15.71:0/2677564494 conn(0x7faf6c069770 msgr2=0x7faf6c06ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:37.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 shutdown_connections 2023-10-26T05:47:37.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:37.668+0000 7faf71d1f640 1 -- 172.21.15.71:0/2677564494 wait complete. 2023-10-26T05:47:37.689 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:47:37.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:47:37.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:47:37.690 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:47:37.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:37.691 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:47:37.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:47:37.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:47:37.692 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:47:37.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.692 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:47:37.699 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:37.700 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:47:37.701 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:47:37.701 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:37.702 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:47:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:47:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42519sec preferred_lft 42519sec 2023-10-26T05:47:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:47:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:37.703 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:47:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:47:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:37.704 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-26T05:47:37.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.704 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:37.705 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:37.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2023-10-26T05:47:37.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2023-10-26T05:47:37.706 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2023-10-26T05:47:37.706 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-26T05:47:37.706 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:37.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.778 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:47:37.828 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:47:37.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.830 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:47:37.905 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:47:37.905 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:47:37.906 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:47:37.906 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:47:37.906 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:47:37.906 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:47:37.907 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:37.907 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:37.907 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:37.907 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:47:37.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.908 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:37.908 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:37.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:47:37.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:47:37.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:47:37.910 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:37.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:37.985 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:47:38.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:38.033 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2023-10-26T05:47:38.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:38.081 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:38.082 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:38.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:38.082 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-26T05:47:38.083 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:38.146 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-26T05:47:38.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:38.147 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:38.147 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:38.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-26T05:47:38.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-26T05:47:38.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-26T05:47:38.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-26T05:47:38.149 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-26T05:47:38.149 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:38.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:38.261 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:38.261 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:38.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:47:38.262 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2023-10-26T05:47:38.262 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:38.280 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:47:38.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:38.281 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:38.332 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:47:38.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:38.333 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:38.380 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:38.429 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:47:38.430 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:47:38.430 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:47:38.430 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:47:38.430 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:47:38.430 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:47:38.431 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:47:38.431 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:47:38.431 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:47:38.431 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:47:38.431 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:47:38.432 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:47:38.432 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:38.432 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:47:38.432 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:47:38.432 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:47:38.433 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:47:38.433 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:47:38.433 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:38.433 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:47:38.433 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:38.434 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:47:38.434 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:47:38.434 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:47:38.434 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:47:38.434 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:47:38.435 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:47:38.435 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:47:38.435 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:47:38.435 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:38.435 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:38.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:38.437 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:47:38.484 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:47:38.485 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:47:38.529 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:38.535 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:47:38.536 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:47:38.536 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:47:38.536 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:47:38.536 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:47:38.536 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:47:38.537 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:47:38.537 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:47:38.537 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:47:38.537 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:47:38.537 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:47:38.537 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:47:38.538 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:38.538 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:47:38.538 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:47:38.538 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:47:38.538 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:47:38.539 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:47:38.539 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:38.539 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:47:38.539 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:38.539 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:47:38.540 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:47:38.540 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:47:38.540 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:47:38.540 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:47:38.540 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:47:38.540 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:47:38.541 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:47:38.541 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:38.541 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:38.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:38.542 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:47:38.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:47:38.624+0000 7f821e601580 -1 init, newargv = 0x55a962ab5350 newargc=15 2023-10-26T05:47:38.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[35477]: starting ceph client 2023-10-26T05:47:38.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[35477]: starting fuse 2023-10-26T05:47:39.693 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:39.702 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:47:39.702 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:47:39.702 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:47:39.702 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:47:39.702 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:47:39.703 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:47:39.703 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:47:39.703 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:47:39.703 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:47:39.703 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:47:39.704 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:47:39.704 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:47:39.704 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:39.704 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:47:39.704 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:47:39.705 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:47:39.705 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:47:39.705 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:47:39.705 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:39.705 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:47:39.706 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:39.706 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:47:39.706 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:47:39.706 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:47:39.706 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:47:39.707 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:47:39.707 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:47:39.707 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:47:39.707 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:47:39.707 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:39.707 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:39.708 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:39.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:39.709 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:47:39.757 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:47:39.757 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:47:39.757 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:47:39.758 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:47:39.758 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:47:39.758 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:47:39.759 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:47:39.759 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:39.759 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:47:39.760 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:47:39.760 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:47:39.760 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:47:39.760 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:39.761 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:47:39.761 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:47:39.761 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:47:39.762 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:47:39.762 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:39.762 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:47:39.762 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:47:39.763 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:47:39.763 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:47:39.763 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:47:39.763 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:47:39.764 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:47:39.764 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:47:39.764 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:39.765 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:47:39.765 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:47:39.881 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.35477.asok 2023-10-26T05:47:39.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:39.893 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35477.asok 2023-10-26T05:47:39.893 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:39.893 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.35477.asok status 2023-10-26T05:47:39.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:40.083 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:47:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:47:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:47:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:47:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:47:40.084 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:47:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:47:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "35477", 2023-10-26T05:47:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:47:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:40.085 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:47:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:47:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4710, 2023-10-26T05:47:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:47:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:47:40.086 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:47:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 4710 2023-10-26T05:47:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:47:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:47:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:47:40.087 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1759483646 2023-10-26T05:47:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:47:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:47:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:47:40.088 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:47:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1759483646 2023-10-26T05:47:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.4710 192.168.0.1:0/1759483646", 2023-10-26T05:47:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1759483646", 2023-10-26T05:47:40.089 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:47:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 6, 2023-10-26T05:47:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 20, 2023-10-26T05:47:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:47:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:47:40.090 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:47:40.091 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:47:40.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:40.104 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:47:40.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1355330085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 msgr2=0x7ff6b815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:40.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1355330085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b815b3a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6b808a140 tx=0x7ff6a801c810 comp rx=0 tx=0).stop 2023-10-26T05:47:40.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1355330085 shutdown_connections 2023-10-26T05:47:40.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1355330085 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6b80cd880 0x7ff6b815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1355330085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b815b3a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1355330085 >> v2:172.21.15.86:3300/0 conn(0x7ff6b80d0470 0x7ff6b806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1355330085 >> 172.21.15.71:0/1355330085 conn(0x7ff6b80cdd80 msgr2=0x7ff6b80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1355330085 shutdown_connections 2023-10-26T05:47:40.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1355330085 wait complete. 2023-10-26T05:47:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 Processor -- start 2023-10-26T05:47:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- start start 2023-10-26T05:47:40.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6b80cd880 0x7ff6b80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b80d0470 0x7ff6b80c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b80ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81f5690 con 0x7ff6b80d0850 2023-10-26T05:47:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81f4b10 con 0x7ff6b80d0470 2023-10-26T05:47:40.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81f4df0 con 0x7ff6b80cd880 2023-10-26T05:47:40.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b80ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:40.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b80ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55998/0 (socket says 172.21.15.71:55998) 2023-10-26T05:47:40.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 -- 172.21.15.71:0/1802550780 learned_addr learned my addr 172.21.15.71:0/1802550780 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:40.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6b7fff640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6b80cd880 0x7ff6b80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:40.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6b77fe640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b80d0470 0x7ff6b80c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:40.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 -- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6b80cd880 msgr2=0x7ff6b80c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6b80cd880 0x7ff6b80c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 -- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b80d0470 msgr2=0x7ff6b80c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b80d0470 0x7ff6b80c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 -- 172.21.15.71:0/1802550780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a8009560 con 0x7ff6b80d0850 2023-10-26T05:47:40.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6bcb6a640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b80ca160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b970 tx=0x7ff6b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:40.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b000cb40 con 0x7ff6b80d0850 2023-10-26T05:47:40.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff6b000cdb0 con 0x7ff6b80d0850 2023-10-26T05:47:40.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00146a0 con 0x7ff6b80d0850 2023-10-26T05:47:40.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b81f50d0 con 0x7ff6b80d0850 2023-10-26T05:47:40.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.225+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b80c8910 con 0x7ff6b80d0850 2023-10-26T05:47:40.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.229+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b8066b10 con 0x7ff6b80d0850 2023-10-26T05:47:40.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.229+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff6b001d3b0 con 0x7ff6b80d0850 2023-10-26T05:47:40.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.229+0000 7ff6b4ff9640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6a4066470 0x7ff6a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:40.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.229+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6b0084060 con 0x7ff6b80d0850 2023-10-26T05:47:40.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.229+0000 7ff6b7fff640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6a4066470 0x7ff6a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:40.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.229+0000 7ff6b7fff640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6a4066470 0x7ff6a4068930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0058d0 tx=0x7ff6ac005e50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:40.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.233+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff6b00514f0 con 0x7ff6b80d0850 2023-10-26T05:47:40.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.425+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff6b80721e0 con 0x7ff6b80d0850 2023-10-26T05:47:40.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.425+0000 7ff6b4ff9640 1 -- 172.21.15.71:0/1802550780 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff6b0054b10 con 0x7ff6b80d0850 2023-10-26T05:47:40.432 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:47:40.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6a4066470 msgr2=0x7ff6a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:40.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6a4066470 0x7ff6a4068930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0058d0 tx=0x7ff6ac005e50 comp rx=0 tx=0).stop 2023-10-26T05:47:40.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 msgr2=0x7ff6b80ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:40.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b80ca160 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000b970 tx=0x7ff6b000be30 comp rx=0 tx=0).stop 2023-10-26T05:47:40.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 shutdown_connections 2023-10-26T05:47:40.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6a4066470 0x7ff6a4068930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6b80d0850 0x7ff6b80ca160 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6b80d0470 0x7ff6b80c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 --2- 172.21.15.71:0/1802550780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6b80cd880 0x7ff6b80c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:40.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 >> 172.21.15.71:0/1802550780 conn(0x7ff6b80cdd80 msgr2=0x7ff6b8158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:40.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 shutdown_connections 2023-10-26T05:47:40.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:40.429+0000 7ff6be5cd640 1 -- 172.21.15.71:0/1802550780 wait complete. 2023-10-26T05:47:40.450 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:47:40.450 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:47:40.450 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:47:40.451 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-26T05:47:40.451 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:40.451 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:47:40.451 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:47:40.452 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:47:40.452 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-26T05:47:40.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.453 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:47:40.459 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:40.459 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:40.460 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:40.460 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.460 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:40.460 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.460 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:40.461 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:40.461 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:47:40.461 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:40.461 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:40.461 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:47:40.461 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:40.462 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:40.462 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:47:40.462 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:40.462 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:47:40.462 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42516sec preferred_lft 42516sec 2023-10-26T05:47:40.463 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:47:40.463 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.463 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:47:40.463 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:40.463 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:47:40.463 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.464 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:47:40.464 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:40.464 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:47:40.464 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.464 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:47:40.465 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.465 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:47:40.465 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:40.465 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:47:40.465 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.466 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:47:40.510 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:47:40.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.512 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:47:40.563 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:47:40.563 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:47:40.563 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:47:40.563 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:47:40.564 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:47:40.564 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:47:40.564 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:40.564 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:40.564 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:40.565 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:47:40.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.565 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:40.566 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:40.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:47:40.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:47:40.566 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:47:40.567 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:40.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.637 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:47:40.685 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:47:40.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.686 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2023-10-26T05:47:40.733 INFO:teuthology.orchestra.run.smithi071.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-26T05:47:40.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.734 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:40.734 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:40.734 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:40.735 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-26T05:47:40.735 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:40.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.803 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-26T05:47:40.857 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.859 INFO:teuthology.orchestra.run.smithi071.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:47:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether ba:2e:e2:ce:d9:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T05:47:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T05:47:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::b82e:e2ff:fece:d94d/64 scope link 2023-10-26T05:47:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-26T05:47:40.917 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:40.917 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:40.917 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:40.917 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.918 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:40.918 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.918 INFO:teuthology.orchestra.run.smithi071.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:47:40.918 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether ba:2e:e2:ce:d9:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T05:47:40.918 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T05:47:40.919 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.919 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::b82e:e2ff:fece:d94d/64 scope link 2023-10-26T05:47:40.919 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:40.920 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-26T05:47:40.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:40.921 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:40.921 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:40.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-26T05:47:40.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-26T05:47:40.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-26T05:47:40.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-26T05:47:40.923 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-26T05:47:40.923 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:41.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:41.018 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:47:41.018 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:47:41.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.1 up 2023-10-26T05:47:41.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.1 master ceph-brx 2023-10-26T05:47:41.019 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:47:41.038 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-26T05:47:41.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:41.039 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:41.084 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-26T05:47:41.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:41.085 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:41.136 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:41.185 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:47:41.186 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:47:41.186 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:47:41.186 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:47:41.186 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:47:41.186 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:47:41.187 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:47:41.187 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:47:41.187 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:47:41.187 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:47:41.187 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:47:41.188 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:47:41.188 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:41.188 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:47:41.188 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:47:41.188 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:47:41.189 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:47:41.189 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:47:41.189 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:41.189 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:47:41.189 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:41.190 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:47:41.190 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:47:41.190 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:47:41.190 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:47:41.190 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:47:41.191 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:47:41.191 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:47:41.191 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:47:41.191 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:41.191 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:41.192 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:41.192 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:41.192 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:41.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:41.193 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:47:41.240 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:47:41.241 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2023-10-26T05:47:41.241 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-26T05:47:41.285 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:41.291 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:47:41.292 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:47:41.292 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:47:41.292 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:47:41.292 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:47:41.292 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:47:41.293 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:47:41.293 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:47:41.293 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:47:41.293 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:47:41.293 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:47:41.294 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:47:41.294 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:41.294 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:47:41.294 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:47:41.294 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:47:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:47:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:47:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:47:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:41.295 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:47:41.296 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:47:41.296 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:47:41.296 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:47:41.296 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:47:41.296 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:47:41.297 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:47:41.297 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:47:41.297 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:41.297 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:41.297 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:41.297 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:41.298 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:41.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:41.299 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:47:41.344 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:47:41.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi071.stderr:2023-10-26T05:47:41.361+0000 7ff901aa1580 -1 init, newargv = 0x5565f1fdc240 newargc=15 2023-10-26T05:47:41.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi071.stderr:ceph-fuse[35613]: starting ceph client 2023-10-26T05:47:41.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi071.stderr:ceph-fuse[35613]: starting fuse 2023-10-26T05:47:42.447 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:42.456 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:47:42.456 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:47:42.456 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:47:42.457 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:47:42.457 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:47:42.457 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:47:42.457 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:47:42.457 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:47:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:47:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:47:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:47:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:47:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:47:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:47:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:47:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:47:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:47:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:47:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:47:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:47:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:47:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:47:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:47:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:47:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:47:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:47:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:47:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:47:42.463 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:42.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:42.465 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:47:42.513 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:47:42.513 INFO:teuthology.orchestra.run.smithi071.stdout:45 2023-10-26T05:47:42.514 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 45] 2023-10-26T05:47:42.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:47:42.515 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:47:42.515 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:47:42.515 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:47:42.515 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:47:42.516 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:42.516 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:47:42.516 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-26T05:47:42.517 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:47:42.517 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-26T05:47:42.517 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:42.517 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:47:42.518 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:47:42.518 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:47:42.518 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:47:42.519 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:42.519 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:47:42.519 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:47:42.520 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:47:42.520 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:47:42.520 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:47:42.521 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:47:42.521 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:47:42.521 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:47:42.522 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:47:42.522 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.2")) 2023-10-26T05:47:42.522 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:47:42.637 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.2.35613.asok 2023-10-26T05:47:42.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:42.648 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35613.asok 2023-10-26T05:47:42.649 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:42.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.2.35613.asok status 2023-10-26T05:47:42.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:42.840 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:47:42.840 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:47:42.840 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:47:42.840 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:47:42.841 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "2", 2023-10-26T05:47:42.841 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:47:42.841 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-26T05:47:42.841 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "35613", 2023-10-26T05:47:42.842 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:47:42.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:42.842 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:47:42.842 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:47:42.842 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4725, 2023-10-26T05:47:42.843 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:47:42.843 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:47:42.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:47:42.843 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 4725 2023-10-26T05:47:42.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:42.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:47:42.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:47:42.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.2:0", 2023-10-26T05:47:42.844 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3928909759 2023-10-26T05:47:42.844 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:47:42.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:42.845 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:47:42.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:47:42.845 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.2:0", 2023-10-26T05:47:42.845 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3928909759 2023-10-26T05:47:42.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:47:42.846 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.4725 192.168.0.2:0/3928909759", 2023-10-26T05:47:42.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.2:0/3928909759", 2023-10-26T05:47:42.846 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:47:42.846 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 6, 2023-10-26T05:47:42.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 20, 2023-10-26T05:47:42.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:47:42.847 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:47:42.847 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:47:42.847 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:42.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:47:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 -- 172.21.15.71:0/768271931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 msgr2=0x7fb52415b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 --2- 172.21.15.71:0/768271931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb52415b8e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb5240732b0 tx=0x7fb51c01cc20 comp rx=0 tx=0).stop 2023-10-26T05:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 -- 172.21.15.71:0/768271931 shutdown_connections 2023-10-26T05:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 --2- 172.21.15.71:0/768271931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb52415b8e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 --2- 172.21.15.71:0/768271931 >> v2:172.21.15.86:3300/0 conn(0x7fb5240d0850 0x7fb52415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 --2- 172.21.15.71:0/768271931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0470 0x7fb52406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 -- 172.21.15.71:0/768271931 >> 172.21.15.71:0/768271931 conn(0x7fb5240cdd80 msgr2=0x7fb5240ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 -- 172.21.15.71:0/768271931 shutdown_connections 2023-10-26T05:47:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 -- 172.21.15.71:0/768271931 wait complete. 2023-10-26T05:47:42.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 Processor -- start 2023-10-26T05:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.981+0000 7fb52b37d640 1 -- start start 2023-10-26T05:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb524159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5240d0470 0x7fb5240c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0850 0x7fb5241550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524156120 con 0x7fb5240cd880 2023-10-26T05:47:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524156400 con 0x7fb5240d0470 2023-10-26T05:47:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5241566e0 con 0x7fb5240d0850 2023-10-26T05:47:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb524159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb524159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35412/0 (socket says 172.21.15.71:35412) 2023-10-26T05:47:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 -- 172.21.15.71:0/812137148 learned_addr learned my addr 172.21.15.71:0/812137148 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb528918640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5240d0470 0x7fb5240c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52991a640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0850 0x7fb5241550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 -- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0850 msgr2=0x7fb5241550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0850 0x7fb5241550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:42.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 -- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5240d0470 msgr2=0x7fb5240c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5240d0470 0x7fb5240c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 -- 172.21.15.71:0/812137148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0098e0 con 0x7fb5240cd880 2023-10-26T05:47:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52991a640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0850 0x7fb5241550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb529119640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb524159cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d750 tx=0x7fb52000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:42.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520009c30 con 0x7fb5240cd880 2023-10-26T05:47:42.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb520009e80 con 0x7fb5240cd880 2023-10-26T05:47:42.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5241569c0 con 0x7fb5240cd880 2023-10-26T05:47:42.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520005280 con 0x7fb5240cd880 2023-10-26T05:47:42.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52415a200 con 0x7fb5240cd880 2023-10-26T05:47:42.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524066b10 con 0x7fb5240cd880 2023-10-26T05:47:42.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb5200053e0 con 0x7fb5240cd880 2023-10-26T05:47:42.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb519ffb640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb508066470 0x7fb508068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:42.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb520083fd0 con 0x7fb5240cd880 2023-10-26T05:47:42.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.985+0000 7fb528918640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb508066470 0x7fb508068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:42.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.989+0000 7fb528918640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb508066470 0x7fb508068930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5140058d0 tx=0x7fb514005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:42.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:42.989+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb5200514e0 con 0x7fb5240cd880 2023-10-26T05:47:43.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.173+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb52415a420 con 0x7fb5240cd880 2023-10-26T05:47:43.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.173+0000 7fb519ffb640 1 -- 172.21.15.71:0/812137148 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb520017070 con 0x7fb5240cd880 2023-10-26T05:47:43.179 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:47:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb508066470 msgr2=0x7fb508068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:43.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb508066470 0x7fb508068930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5140058d0 tx=0x7fb514005860 comp rx=0 tx=0).stop 2023-10-26T05:47:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 msgr2=0x7fb524159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb524159cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d750 tx=0x7fb52000dc10 comp rx=0 tx=0).stop 2023-10-26T05:47:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 shutdown_connections 2023-10-26T05:47:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5240d0850 0x7fb5241550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:43.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5240d0470 0x7fb5240c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:43.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb508066470 0x7fb508068930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:43.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 --2- 172.21.15.71:0/812137148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5240cd880 0x7fb524159cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:43.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 >> 172.21.15.71:0/812137148 conn(0x7fb5240cdd80 msgr2=0x7fb524158490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:43.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 shutdown_connections 2023-10-26T05:47:43.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:43.177+0000 7fb52b37d640 1 -- 172.21.15.71:0/812137148 wait complete. 2023-10-26T05:47:43.196 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:47:43.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:47:43.197 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:47:43.197 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:47:43.197 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:43.197 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:47:43.198 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:47:43.198 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:47:43.198 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:47:43.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.199 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:47:43.206 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:43.206 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:43.206 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:43.207 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:43.207 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:43.207 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:43.207 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:43.207 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:43.208 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:47:43.208 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:43.208 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:43.208 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:47:43.208 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:43.209 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:43.209 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:47:43.209 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:43.209 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:47:43.209 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42508sec preferred_lft 42508sec 2023-10-26T05:47:43.209 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:47:43.210 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:43.210 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:47:43.210 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:43.210 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:47:43.210 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:43.211 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-26T05:47:43.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.211 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:43.212 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:43.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2023-10-26T05:47:43.212 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2023-10-26T05:47:43.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2023-10-26T05:47:43.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-26T05:47:43.213 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:43.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.291 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:47:43.342 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:47:43.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.343 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:47:43.413 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:47:43.414 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:47:43.414 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:47:43.414 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:47:43.414 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:47:43.415 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:47:43.415 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:43.415 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:43.415 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:43.415 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:47:43.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.416 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:43.417 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:43.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:47:43.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:47:43.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:47:43.418 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:43.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.490 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:47:43.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.542 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2023-10-26T05:47:43.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.591 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:43.591 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:43.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:43.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-26T05:47:43.592 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:43.657 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-26T05:47:43.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.658 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:43.658 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:43.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-26T05:47:43.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-26T05:47:43.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-26T05:47:43.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-26T05:47:43.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-26T05:47:43.660 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:43.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.764 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:43.764 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:43.764 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:47:43.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2023-10-26T05:47:43.765 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:43.784 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:47:43.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:43.785 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:43.829 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:47:43.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:43.830 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:43.877 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:43.925 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:47:43.926 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:47:43.926 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:47:43.926 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:47:43.926 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:47:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:47:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:47:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:47:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:47:43.927 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:47:43.928 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:47:43.928 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:47:43.928 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:43.928 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:47:43.928 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:47:43.929 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:47:43.929 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:47:43.929 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:47:43.930 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:43.930 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:47:43.930 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:43.930 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:47:43.930 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:47:43.931 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:47:43.931 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:47:43.931 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:47:43.931 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:47:43.931 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:47:43.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:47:43.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:43.932 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:43.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:43.934 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:47:43.981 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:47:43.982 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:47:44.027 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:44.033 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:47:44.034 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:47:44.034 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:47:44.034 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:47:44.034 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:47:44.035 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:47:44.035 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:47:44.035 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:47:44.035 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:47:44.035 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:47:44.036 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:47:44.036 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:47:44.036 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:44.036 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:47:44.036 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:47:44.036 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:47:44.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:47:44.037 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:47:44.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:44.037 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:47:44.037 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:44.038 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:47:44.038 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:47:44.038 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:47:44.038 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:47:44.038 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:47:44.039 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:47:44.039 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:47:44.039 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:47:44.039 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:44.039 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:44.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:44.041 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:47:44.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:47:44.115+0000 7f1088325580 -1 init, newargv = 0x55e6e769fae0 newargc=15 2023-10-26T05:47:44.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[33767]: starting ceph client 2023-10-26T05:47:44.125 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[33767]: starting fuse 2023-10-26T05:47:44.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:47:44.424 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:47:45.190 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:45.199 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:47:45.199 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:47:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:47:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:47:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:47:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:47:45.200 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:47:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:47:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:47:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:47:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:47:45.201 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:47:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:47:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:47:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:47:45.202 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:47:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:47:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:47:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:45.203 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:47:45.204 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:47:45.204 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:47:45.204 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:47:45.204 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:47:45.204 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:47:45.204 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:47:45.205 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:47:45.205 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:45.205 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:45.205 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:45.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:45.206 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:47:45.254 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:47:45.254 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:47:45.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:47:45.255 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:47:45.255 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:47:45.256 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:47:45.256 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:47:45.256 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:45.257 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:47:45.257 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:47:45.257 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:47:45.258 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:47:45.258 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:45.258 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:47:45.258 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:47:45.259 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:47:45.259 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:47:45.259 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:45.260 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:47:45.260 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:47:45.260 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:47:45.260 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:47:45.261 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:47:45.261 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:47:45.262 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:47:45.262 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:47:45.263 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:45.263 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:47:45.263 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:47:45.376 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.33767.asok 2023-10-26T05:47:45.380 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:45.388 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33767.asok 2023-10-26T05:47:45.389 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:45.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.33767.asok status 2023-10-26T05:47:45.475 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:45.577 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:47:45.577 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:47:45.578 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:47:45.578 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:47:45.578 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:47:45.578 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:47:45.578 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:47:45.579 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "33767", 2023-10-26T05:47:45.579 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:47:45.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:45.579 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:47:45.580 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:47:45.580 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4594, 2023-10-26T05:47:45.580 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:47:45.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:47:45.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:47:45.581 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4594 2023-10-26T05:47:45.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:45.581 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:47:45.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:47:45.581 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:47:45.581 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 558592866 2023-10-26T05:47:45.582 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:47:45.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:45.582 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:47:45.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:47:45.583 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:47:45.583 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 558592866 2023-10-26T05:47:45.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:45.583 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4594 192.168.0.1:0/558592866", 2023-10-26T05:47:45.583 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/558592866", 2023-10-26T05:47:45.583 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:47:45.584 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 6, 2023-10-26T05:47:45.584 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 20, 2023-10-26T05:47:45.584 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:47:45.584 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:47:45.584 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:47:45.585 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:47:45.589 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:45.597 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:47:45.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 -- 172.21.15.71:0/1873310505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6140ce1c0 msgr2=0x7fa6140b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:45.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 --2- 172.21.15.71:0/1873310505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6140ce1c0 0x7fa6140b2300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa604009930 tx=0x7fa60401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 -- 172.21.15.71:0/1873310505 shutdown_connections 2023-10-26T05:47:45.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 --2- 172.21.15.71:0/1873310505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6140ceef0 0x7fa6140cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 --2- 172.21.15.71:0/1873310505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6140ce1c0 0x7fa6140b2300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 -- 172.21.15.71:0/1873310505 >> 172.21.15.71:0/1873310505 conn(0x7fa61406f440 msgr2=0x7fa61406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:45.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 -- 172.21.15.71:0/1873310505 shutdown_connections 2023-10-26T05:47:45.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.729+0000 7fa619e97640 1 -- 172.21.15.71:0/1873310505 wait complete. 2023-10-26T05:47:45.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 Processor -- start 2023-10-26T05:47:45.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- start start 2023-10-26T05:47:45.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6140ceef0 0x7fa61413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:45.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6141538a0 0x7fa614146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:45.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 0x7fa6140b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614146c20 con 0x7fa6141466c0 2023-10-26T05:47:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61414da50 con 0x7fa6140ceef0 2023-10-26T05:47:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61414dd30 con 0x7fa6141538a0 2023-10-26T05:47:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 0x7fa6140b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:45.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 0x7fa6140b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35438/0 (socket says 172.21.15.71:35438) 2023-10-26T05:47:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 -- 172.21.15.71:0/2460612924 learned_addr learned my addr 172.21.15.71:0/2460612924 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa612ffd640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6141538a0 0x7fa614146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa6137fe640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6140ceef0 0x7fa61413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 -- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6141538a0 msgr2=0x7fa614146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:45.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6141538a0 0x7fa614146180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 -- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6140ceef0 msgr2=0x7fa61413eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:45.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6140ceef0 0x7fa61413eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 -- 172.21.15.71:0/2460612924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604009560 con 0x7fa6141466c0 2023-10-26T05:47:45.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa613fff640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 0x7fa6140b0cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00ea20 tx=0x7fa60c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:45.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00cde0 con 0x7fa6141466c0 2023-10-26T05:47:45.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa60c004c40 con 0x7fa6141466c0 2023-10-26T05:47:45.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61414e010 con 0x7fa6141466c0 2023-10-26T05:47:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c005020 con 0x7fa6141466c0 2023-10-26T05:47:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61413f070 con 0x7fa6141466c0 2023-10-26T05:47:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa614066b10 con 0x7fa6141466c0 2023-10-26T05:47:45.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.733+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa60c005180 con 0x7fa6141466c0 2023-10-26T05:47:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.737+0000 7fa5fbfff640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa600066470 0x7fa600068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.737+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa60c084840 con 0x7fa6141466c0 2023-10-26T05:47:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.737+0000 7fa6137fe640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa600066470 0x7fa600068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.737+0000 7fa6137fe640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa600066470 0x7fa600068930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa60800b810 tx=0x7fa608008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.741+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa60c051d50 con 0x7fa6141466c0 2023-10-26T05:47:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.925+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa614066d00 con 0x7fa6141466c0 2023-10-26T05:47:45.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa5fbfff640 1 -- 172.21.15.71:0/2460612924 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa60c01f070 con 0x7fa6141466c0 2023-10-26T05:47:45.933 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:47:45.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa600066470 msgr2=0x7fa600068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:45.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa600066470 0x7fa600068930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa60800b810 tx=0x7fa608008040 comp rx=0 tx=0).stop 2023-10-26T05:47:45.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 msgr2=0x7fa6140b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:45.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 0x7fa6140b0cd0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00ea20 tx=0x7fa60c00eee0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 shutdown_connections 2023-10-26T05:47:45.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa600066470 0x7fa600068930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6141466c0 0x7fa6140b0cd0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6141538a0 0x7fa614146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 --2- 172.21.15.71:0/2460612924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6140ceef0 0x7fa61413eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:45.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 >> 172.21.15.71:0/2460612924 conn(0x7fa61406f440 msgr2=0x7fa61406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:45.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 shutdown_connections 2023-10-26T05:47:45.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:45.929+0000 7fa619e97640 1 -- 172.21.15.71:0/2460612924 wait complete. 2023-10-26T05:47:45.950 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:47:45.951 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:47:45.951 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:47:45.951 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-26T05:47:45.951 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:45.952 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:47:45.952 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:47:45.952 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:47:45.952 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-26T05:47:45.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:45.953 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:47:45.960 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:45.961 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:45.961 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:45.961 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.961 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:45.961 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.962 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:45.962 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:45.962 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:47:45.962 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:45.962 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:45.963 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:47:45.963 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:47:45.963 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:45.963 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:47:45.963 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:45.964 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:47:45.964 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42505sec preferred_lft 42505sec 2023-10-26T05:47:45.964 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:47:45.964 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.964 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:47:45.965 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:45.965 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:47:45.965 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.965 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:47:45.965 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:47:45.965 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:47:45.966 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.966 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:47:45.966 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.966 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:47:45.966 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:45.967 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:47:45.967 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:45.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:45.968 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:47:46.015 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:47:46.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.016 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:47:46.064 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:47:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:47:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:47:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:47:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:47:46.065 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:47:46.066 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:46.066 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:46.066 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:47:46.066 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:47:46.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.067 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:46.067 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:46.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:47:46.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:47:46.068 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:47:46.069 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:46.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.138 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:47:46.185 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:47:46.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.186 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2023-10-26T05:47:46.233 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-26T05:47:46.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.234 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:46.234 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:46.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:46.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-26T05:47:46.235 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:46.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-26T05:47:46.358 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:46.359 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:46.359 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:46.359 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.359 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:46.360 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.360 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:47:46.360 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 3e:3b:59:fa:a3:23 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T05:47:46.360 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T05:47:46.360 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.361 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::3c3b:59ff:fefa:a323/64 scope link 2023-10-26T05:47:46.361 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.362 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-26T05:47:46.417 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:47:46.417 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:47:46.417 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:47:46.418 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.418 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:47:46.418 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.418 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:47:46.418 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 3e:3b:59:fa:a3:23 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-26T05:47:46.419 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-26T05:47:46.419 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.419 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::3c3b:59ff:fefa:a323/64 scope link 2023-10-26T05:47:46.419 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:47:46.420 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-26T05:47:46.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.421 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:46.421 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:46.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-26T05:47:46.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-26T05:47:46.422 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-26T05:47:46.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-26T05:47:46.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-26T05:47:46.423 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:46.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.515 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:47:46.515 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:47:46.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2023-10-26T05:47:46.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2023-10-26T05:47:46.516 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:47:46.535 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-26T05:47:46.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:46.536 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:46.581 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-26T05:47:46.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:47:46.582 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:46.629 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:46.678 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:47:46.679 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:47:46.679 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:47:46.679 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:47:46.679 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:47:46.679 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:47:46.680 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:47:46.680 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:47:46.680 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:47:46.680 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:47:46.680 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:47:46.681 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:47:46.681 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:46.681 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:47:46.681 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:47:46.681 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:47:46.682 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:47:46.682 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:47:46.682 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:46.682 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:47:46.682 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:46.683 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:47:46.683 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:47:46.683 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:47:46.683 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:47:46.683 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:47:46.684 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:47:46.684 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:47:46.684 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:47:46.684 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:46.684 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:46.684 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:46.685 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:46.685 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:46.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.686 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:47:46.734 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:47:46.734 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-10-26T05:47:46.735 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-26T05:47:46.778 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:46.784 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:47:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:47:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:47:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:47:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:47:46.785 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:47:46.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:47:46.786 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:47:46.786 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:47:46.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:47:46.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:47:46.787 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:47:46.787 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:46.787 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:47:46.787 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:47:46.787 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:47:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:47:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:47:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:47:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:46.788 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:47:46.789 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:47:46.789 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:47:46.789 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:47:46.789 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:47:46.789 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:47:46.790 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:47:46.790 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:47:46.790 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:46.790 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:46.790 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:46.791 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:46.791 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:46.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:46.792 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:47:46.837 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:47:46.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:2023-10-26T05:47:46.855+0000 7f0ca67b9580 -1 init, newargv = 0x55d20b7ba4b0 newargc=15 2023-10-26T05:47:46.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:ceph-fuse[33879]: starting ceph client 2023-10-26T05:47:46.862 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:ceph-fuse[33879]: starting fuse 2023-10-26T05:47:47.939 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:47:47.948 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:47:47.948 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:47:47.949 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:47:47.949 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:47:47.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:47:47.949 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:47:47.949 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:47:47.950 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:47:47.950 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:47:47.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:47:47.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:47:47.951 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:47:47.951 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:47:47.951 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:47:47.951 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:47:47.951 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:47:47.952 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:47:47.952 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:47:47.952 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:47:47.952 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:47:47.953 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:47:47.953 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:47:47.953 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:47:47.953 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:47:47.953 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:47:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:47:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:47:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:47:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:47:47.954 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:47:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:47.955 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:47:47.956 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:47.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:47:47.957 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:47:48.006 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:47:48.006 INFO:teuthology.orchestra.run.smithi086.stdout:46 2023-10-26T05:47:48.007 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-10-26T05:47:48.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:47:48.007 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:47:48.008 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:47:48.008 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:47:48.008 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:47:48.009 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:48.009 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:47:48.009 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-26T05:47:48.009 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:47:48.010 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-26T05:47:48.010 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:48.010 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:47:48.011 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:47:48.011 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:47:48.011 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:47:48.012 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:48.012 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:47:48.012 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:47:48.013 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:47:48.013 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:47:48.013 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:47:48.013 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:47:48.014 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:47:48.014 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:47:48.015 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:47:48.015 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.3")) 2023-10-26T05:47:48.015 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:47:48.130 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.3.33879.asok 2023-10-26T05:47:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.142 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33879.asok 2023-10-26T05:47:48.142 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.3.33879.asok status 2023-10-26T05:47:48.233 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.330 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:47:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:47:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:47:48.330 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:47:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "3", 2023-10-26T05:47:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:47:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-26T05:47:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "33879", 2023-10-26T05:47:48.331 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:47:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:47:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:47:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4475, 2023-10-26T05:47:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:47:48.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:47:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:47:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4475 2023-10-26T05:47:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:47:48.333 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:47:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.2:0", 2023-10-26T05:47:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2239231057 2023-10-26T05:47:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:47:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:48.334 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:47:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:47:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.2:0", 2023-10-26T05:47:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2239231057 2023-10-26T05:47:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:47:48.335 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4475 192.168.0.2:0/2239231057", 2023-10-26T05:47:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.2:0/2239231057", 2023-10-26T05:47:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:47:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 6, 2023-10-26T05:47:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 20, 2023-10-26T05:47:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:47:48.336 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:47:48.337 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:47:48.337 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:47:48.342 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.353 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:47:48.354 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.410 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:47:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.418 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:48.418 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:47:48.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.477 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.478 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:47:48.478 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:48.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.522 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:47:48.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.529 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:48.529 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.529 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-26T05:47:48.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.634 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:47:48.635 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.691 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:47:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:48.699 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:47:48.745 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.753 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:47:48.753 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:48.802 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.803 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:47:48.803 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.810 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:48.811 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:47:48.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-26T05:47:48.860 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:48.860 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:47:48.868 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-26T05:47:48.883 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:49.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 -- 172.21.15.71:0/2112164724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 msgr2=0x7fa050004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 --2- 172.21.15.71:0/2112164724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa050004c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa04c001680 tx=0x7fa04c01c760 comp rx=0 tx=0).stop 2023-10-26T05:47:49.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa062d9d640 1 -- 172.21.15.71:0/2112164724 reap_dead start 2023-10-26T05:47:49.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 -- 172.21.15.71:0/2112164724 shutdown_connections 2023-10-26T05:47:49.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 -- 172.21.15.71:0/2112164724 >> 172.21.15.71:0/2112164724 conn(0x7fa064069770 msgr2=0x7fa064069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 -- 172.21.15.71:0/2112164724 shutdown_connections 2023-10-26T05:47:49.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 -- 172.21.15.71:0/2112164724 wait complete. 2023-10-26T05:47:49.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 Processor -- start 2023-10-26T05:47:49.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.017+0000 7fa068e1c640 1 -- start start 2023-10-26T05:47:49.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa06413c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0640a7440 0x7fa064159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa06415a490 0x7fa0640a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06415a910 con 0x7fa0500047d0 2023-10-26T05:47:49.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641586d0 con 0x7fa0640a7440 2023-10-26T05:47:49.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa064154a70 con 0x7fa06415a490 2023-10-26T05:47:49.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa06413c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa06413c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35462/0 (socket says 172.21.15.71:35462) 2023-10-26T05:47:49.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 -- 172.21.15.71:0/1368472158 learned_addr learned my addr 172.21.15.71:0/1368472158 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:49.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa059d9b640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0640a7440 0x7fa064159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa062d9d640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa06415a490 0x7fa0640a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 -- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa06415a490 msgr2=0x7fa0640a8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa06415a490 0x7fa0640a8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 -- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0640a7440 msgr2=0x7fa064159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0640a7440 0x7fa064159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 -- 172.21.15.71:0/1368472158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa064154d50 con 0x7fa0500047d0 2023-10-26T05:47:49.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa062d9d640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa06415a490 0x7fa0640a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:49.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa06259c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa06413c940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e390 tx=0x7fa05400e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054012910 con 0x7fa0500047d0 2023-10-26T05:47:49.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa054012b60 con 0x7fa0500047d0 2023-10-26T05:47:49.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa064155030 con 0x7fa0500047d0 2023-10-26T05:47:49.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05401a400 con 0x7fa0500047d0 2023-10-26T05:47:49.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0641555f0 con 0x7fa0500047d0 2023-10-26T05:47:49.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa064066b10 con 0x7fa0500047d0 2023-10-26T05:47:49.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa05401a560 con 0x7fa0500047d0 2023-10-26T05:47:49.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa05b7fe640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0440664a0 0x7fa044068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.021+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa054087930 con 0x7fa0500047d0 2023-10-26T05:47:49.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.025+0000 7fa059d9b640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0440664a0 0x7fa044068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.025+0000 7fa059d9b640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0440664a0 0x7fa044068960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa0500057b0 tx=0x7fa04c019500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.025+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa054054f20 con 0x7fa0500047d0 2023-10-26T05:47:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.181+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa064155810 con 0x7fa0500047d0 2023-10-26T05:47:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa05b7fe640 1 -- 172.21.15.71:0/1368472158 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fa054058540 con 0x7fa0500047d0 2023-10-26T05:47:49.188 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:49.189 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:49.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0440664a0 msgr2=0x7fa044068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0440664a0 0x7fa044068960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa0500057b0 tx=0x7fa04c019500 comp rx=0 tx=0).stop 2023-10-26T05:47:49.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 msgr2=0x7fa06413c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa06413c940 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa05400e390 tx=0x7fa05400e850 comp rx=0 tx=0).stop 2023-10-26T05:47:49.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 shutdown_connections 2023-10-26T05:47:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa06415a490 0x7fa0640a8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0640a7440 0x7fa064159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0440664a0 0x7fa044068960 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 --2- 172.21.15.71:0/1368472158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa0500047d0 0x7fa06413c940 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 >> 172.21.15.71:0/1368472158 conn(0x7fa064069770 msgr2=0x7fa064155fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 shutdown_connections 2023-10-26T05:47:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.185+0000 7fa068e1c640 1 -- 172.21.15.71:0/1368472158 wait complete. 2023-10-26T05:47:49.207 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:49.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:49.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/980219581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d3813ee00 msgr2=0x7f8d3815cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- 172.21.15.71:0/980219581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d3813ee00 0x7f8d3815cc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8d380cdfb0 tx=0x7f8d2801ca70 comp rx=0 tx=0).stop 2023-10-26T05:47:49.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/980219581 shutdown_connections 2023-10-26T05:47:49.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- 172.21.15.71:0/980219581 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d380cbd60 0x7f8d3813e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- 172.21.15.71:0/980219581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d3813ee00 0x7f8d3815cc90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- 172.21.15.71:0/980219581 >> v2:172.21.15.86:3300/0 conn(0x7f8d3813c190 0x7f8d380cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/980219581 >> 172.21.15.71:0/980219581 conn(0x7f8d380aec60 msgr2=0x7f8d380af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/980219581 shutdown_connections 2023-10-26T05:47:49.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/980219581 wait complete. 2023-10-26T05:47:49.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 Processor -- start 2023-10-26T05:47:49.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- start start 2023-10-26T05:47:49.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 0x7f8d38134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d3813c190 0x7f8d380b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d3813ee00 0x7f8d381540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380add30 con 0x7f8d380cbd60 2023-10-26T05:47:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380ae540 con 0x7f8d3813ee00 2023-10-26T05:47:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380b29a0 con 0x7f8d3813c190 2023-10-26T05:47:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 0x7f8d38134510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 0x7f8d38134510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35484/0 (socket says 172.21.15.71:35484) 2023-10-26T05:47:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 -- 172.21.15.71:0/3891615815 learned_addr learned my addr 172.21.15.71:0/3891615815 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3d5bf640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d3813ee00 0x7f8d381540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d37fff640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d3813c190 0x7f8d380b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 -- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d3813c190 msgr2=0x7f8d380b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d3813c190 0x7f8d380b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 -- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d3813ee00 msgr2=0x7f8d381540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d3813ee00 0x7f8d381540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 -- 172.21.15.71:0/3891615815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d280098e0 con 0x7f8d380cbd60 2023-10-26T05:47:49.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d37fff640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d3813c190 0x7f8d380b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:49.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3cdbe640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 0x7f8d38134510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000c970 tx=0x7f8d3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30007bb0 con 0x7f8d380cbd60 2023-10-26T05:47:49.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d30007e00 con 0x7f8d380cbd60 2023-10-26T05:47:49.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d380adf80 con 0x7f8d380cbd60 2023-10-26T05:47:49.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30005280 con 0x7f8d380cbd60 2023-10-26T05:47:49.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.337+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d38134ba0 con 0x7f8d380cbd60 2023-10-26T05:47:49.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.341+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d38066b10 con 0x7f8d380cbd60 2023-10-26T05:47:49.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.341+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d300053e0 con 0x7f8d380cbd60 2023-10-26T05:47:49.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.341+0000 7f8d357fa640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d180664a0 0x7f8d18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.341+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d30084360 con 0x7f8d380cbd60 2023-10-26T05:47:49.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.341+0000 7f8d37fff640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d180664a0 0x7f8d18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.341+0000 7f8d37fff640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d180664a0 0x7f8d18068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d20005fd0 tx=0x7f8d20005d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.345+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d30051870 con 0x7f8d380cbd60 2023-10-26T05:47:49.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d380721e0 con 0x7f8d380cbd60 2023-10-26T05:47:49.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d357fa640 1 -- 172.21.15.71:0/3891615815 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d3001d0a0 con 0x7f8d380cbd60 2023-10-26T05:47:49.505 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:49.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d180664a0 msgr2=0x7f8d18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d180664a0 0x7f8d18068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d20005fd0 tx=0x7f8d20005d40 comp rx=0 tx=0).stop 2023-10-26T05:47:49.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 msgr2=0x7f8d38134510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 0x7f8d38134510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000c970 tx=0x7f8d3000ce30 comp rx=0 tx=0).stop 2023-10-26T05:47:49.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 shutdown_connections 2023-10-26T05:47:49.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d3813ee00 0x7f8d381540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d3813c190 0x7f8d380b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d180664a0 0x7f8d18068960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 --2- 172.21.15.71:0/3891615815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d380cbd60 0x7f8d38134510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 >> 172.21.15.71:0/3891615815 conn(0x7f8d380aec60 msgr2=0x7f8d381538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 shutdown_connections 2023-10-26T05:47:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.501+0000 7f8d3f022640 1 -- 172.21.15.71:0/3891615815 wait complete. 2023-10-26T05:47:49.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:49.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.653+0000 7ff3582b4640 1 -- 172.21.15.71:0/3542700006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350141bc0 msgr2=0x7ff35006f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.653+0000 7ff3582b4640 1 --2- 172.21.15.71:0/3542700006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350141bc0 0x7ff35006f540 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff3500cdfb0 tx=0x7ff34c01c840 comp rx=0 tx=0).stop 2023-10-26T05:47:49.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- 172.21.15.71:0/3542700006 shutdown_connections 2023-10-26T05:47:49.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 --2- 172.21.15.71:0/3542700006 >> v2:172.21.15.86:3300/0 conn(0x7ff3500b0ec0 0x7ff350070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 --2- 172.21.15.71:0/3542700006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350141bc0 0x7ff35006f540 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- 172.21.15.71:0/3542700006 >> 172.21.15.71:0/3542700006 conn(0x7ff3500aec60 msgr2=0x7ff3500af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- 172.21.15.71:0/3542700006 shutdown_connections 2023-10-26T05:47:49.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- 172.21.15.71:0/3542700006 wait complete. 2023-10-26T05:47:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 Processor -- start 2023-10-26T05:47:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- start start 2023-10-26T05:47:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 0x7ff35015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff350141bc0 0x7ff35015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff35015bc10 0x7ff3501579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cc7b0 con 0x7ff3500b0ec0 2023-10-26T05:47:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cca00 con 0x7ff350141bc0 2023-10-26T05:47:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500ccce0 con 0x7ff35015bc10 2023-10-26T05:47:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 0x7ff35015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 0x7ff35015ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35506/0 (socket says 172.21.15.71:35506) 2023-10-26T05:47:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 -- 172.21.15.71:0/4070558770 learned_addr learned my addr 172.21.15.71:0/4070558770 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356851640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff35015bc10 0x7ff3501579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff35584f640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff350141bc0 0x7ff35015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 -- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff35015bc10 msgr2=0x7ff3501579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff35015bc10 0x7ff3501579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 -- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff350141bc0 msgr2=0x7ff35015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff350141bc0 0x7ff35015ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 -- 172.21.15.71:0/4070558770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c009560 con 0x7ff3500b0ec0 2023-10-26T05:47:49.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff356050640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 0x7ff35015ed50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0b0040 tx=0x7ff34c0a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c005220 con 0x7ff3500b0ec0 2023-10-26T05:47:49.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff34c0053a0 con 0x7ff3500b0ec0 2023-10-26T05:47:49.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3500ccfc0 con 0x7ff3500b0ec0 2023-10-26T05:47:49.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c004930 con 0x7ff3500b0ec0 2023-10-26T05:47:49.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.657+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3500cd580 con 0x7ff3500b0ec0 2023-10-26T05:47:49.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.661+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff350066b10 con 0x7ff3500b0ec0 2023-10-26T05:47:49.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.661+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff34c0b4070 con 0x7ff3500b0ec0 2023-10-26T05:47:49.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.661+0000 7ff346ffd640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff334066470 0x7ff334068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.661+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff34c11e340 con 0x7ff3500b0ec0 2023-10-26T05:47:49.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.661+0000 7ff35584f640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff334066470 0x7ff334068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.661+0000 7ff35584f640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff334066470 0x7ff334068930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff350162f70 tx=0x7ff340005e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.665+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff34c0eb730 con 0x7ff3500b0ec0 2023-10-26T05:47:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.817+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3500721e0 con 0x7ff3500b0ec0 2023-10-26T05:47:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff346ffd640 1 -- 172.21.15.71:0/4070558770 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff34c0eed50 con 0x7ff3500b0ec0 2023-10-26T05:47:49.825 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff334066470 msgr2=0x7ff334068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff334066470 0x7ff334068930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff350162f70 tx=0x7ff340005e30 comp rx=0 tx=0).stop 2023-10-26T05:47:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 msgr2=0x7ff35015ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 0x7ff35015ed50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0b0040 tx=0x7ff34c0a5780 comp rx=0 tx=0).stop 2023-10-26T05:47:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 shutdown_connections 2023-10-26T05:47:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff35015bc10 0x7ff3501579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff350141bc0 0x7ff35015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff334066470 0x7ff334068930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 --2- 172.21.15.71:0/4070558770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3500b0ec0 0x7ff35015ed50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 >> 172.21.15.71:0/4070558770 conn(0x7ff3500aec60 msgr2=0x7ff3501402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 shutdown_connections 2023-10-26T05:47:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.821+0000 7ff3582b4640 1 -- 172.21.15.71:0/4070558770 wait complete. 2023-10-26T05:47:49.843 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:49.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/365256513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 msgr2=0x7f7544004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 --2- 172.21.15.71:0/365256513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f7544004c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7550009b30 tx=0x7f755001d1f0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f7560ac8640 1 -- 172.21.15.71:0/365256513 reap_dead start 2023-10-26T05:47:49.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/365256513 shutdown_connections 2023-10-26T05:47:49.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/365256513 >> 172.21.15.71:0/365256513 conn(0x7f755c0aec60 msgr2=0x7f755c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:49.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/365256513 shutdown_connections 2023-10-26T05:47:49.978 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/365256513 wait complete. 2023-10-26T05:47:49.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 Processor -- start 2023-10-26T05:47:49.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- start start 2023-10-26T05:47:49.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f755c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f755c138e10 0x7f755c13c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755c139290 0x7f755c13c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c0ae720 con 0x7f75440047d0 2023-10-26T05:47:49.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755c1586f0 con 0x7f755c139290 2023-10-26T05:47:49.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c158040 con 0x7f755c138e10 2023-10-26T05:47:49.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f755c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f755c157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35524/0 (socket says 172.21.15.71:35524) 2023-10-26T05:47:49.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 -- 172.21.15.71:0/3555303940 learned_addr learned my addr 172.21.15.71:0/3555303940 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:49.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f7560ac8640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755c139290 0x7f755c13c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755b7fe640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f755c138e10 0x7f755c13c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 -- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f755c138e10 msgr2=0x7f755c13c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f755c138e10 0x7f755c13c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 -- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755c139290 msgr2=0x7f755c13c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:49.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755c139290 0x7f755c13c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:49.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 -- 172.21.15.71:0/3555303940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c1ed3d0 con 0x7f75440047d0 2023-10-26T05:47:49.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f7560ac8640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755c139290 0x7f755c13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:49.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f755bfff640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f755c157b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f754c000f70 tx=0x7f754c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c011870 con 0x7f75440047d0 2023-10-26T05:47:49.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f754c011ac0 con 0x7f75440047d0 2023-10-26T05:47:49.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f755c1ed690 con 0x7f75440047d0 2023-10-26T05:47:49.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c001d50 con 0x7f75440047d0 2023-10-26T05:47:49.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755c1edc50 con 0x7f75440047d0 2023-10-26T05:47:49.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.973+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755c066b10 con 0x7f75440047d0 2023-10-26T05:47:49.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.977+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f754c01a400 con 0x7f75440047d0 2023-10-26T05:47:49.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.977+0000 7f7558ff9640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7548066470 0x7f7548068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:49.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.977+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754c084bd0 con 0x7f75440047d0 2023-10-26T05:47:49.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.977+0000 7f755b7fe640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7548066470 0x7f7548068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:49.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.977+0000 7f755b7fe640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7548066470 0x7f7548068930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75500ae000 tx=0x7f7550019f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:49.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:49.981+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f754c052050 con 0x7f75440047d0 2023-10-26T05:47:50.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.133+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f755c0721e0 con 0x7f75440047d0 2023-10-26T05:47:50.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f7558ff9640 1 -- 172.21.15.71:0/3555303940 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f754c055670 con 0x7f75440047d0 2023-10-26T05:47:50.140 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:50.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7548066470 msgr2=0x7f7548068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7548066470 0x7f7548068930 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f75500ae000 tx=0x7f7550019f90 comp rx=0 tx=0).stop 2023-10-26T05:47:50.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 msgr2=0x7f755c157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f755c157b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f754c000f70 tx=0x7f754c001640 comp rx=0 tx=0).stop 2023-10-26T05:47:50.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 shutdown_connections 2023-10-26T05:47:50.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f755c139290 0x7f755c13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f755c138e10 0x7f755c13c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7548066470 0x7f7548068930 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 --2- 172.21.15.71:0/3555303940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75440047d0 0x7f755c157b00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 >> 172.21.15.71:0/3555303940 conn(0x7f755c0aec60 msgr2=0x7f755c0afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 shutdown_connections 2023-10-26T05:47:50.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.137+0000 7f756252b640 1 -- 172.21.15.71:0/3555303940 wait complete. 2023-10-26T05:47:50.159 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:50.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 -- 172.21.15.71:0/929418792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6828005e20 msgr2=0x7f68280062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 --2- 172.21.15.71:0/929418792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6828005e20 0x7f68280062a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f683400c5c0 tx=0x7f683401d340 comp rx=0 tx=0).stop 2023-10-26T05:47:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6842333640 1 -- 172.21.15.71:0/929418792 reap_dead start 2023-10-26T05:47:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 -- 172.21.15.71:0/929418792 shutdown_connections 2023-10-26T05:47:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 -- 172.21.15.71:0/929418792 >> 172.21.15.71:0/929418792 conn(0x7f683c069770 msgr2=0x7f683c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 -- 172.21.15.71:0/929418792 shutdown_connections 2023-10-26T05:47:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 -- 172.21.15.71:0/929418792 wait complete. 2023-10-26T05:47:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 Processor -- start 2023-10-26T05:47:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.277+0000 7f6843d96640 1 -- start start 2023-10-26T05:47:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 0x7f683c152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c152890 0x7f683c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c134a90 0x7f683c1ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c1532d0 con 0x7f683c1346b0 2023-10-26T05:47:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c150910 con 0x7f683c152890 2023-10-26T05:47:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c152d10 con 0x7f683c134a90 2023-10-26T05:47:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 0x7f683c152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 0x7f683c152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35550/0 (socket says 172.21.15.71:35550) 2023-10-26T05:47:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 -- 172.21.15.71:0/2110612795 learned_addr learned my addr 172.21.15.71:0/2110612795 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 -- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c134a90 msgr2=0x7f683c1ef8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6842333640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c134a90 0x7f683c1ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841331640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c152890 0x7f683c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c134a90 0x7f683c1ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 -- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c152890 msgr2=0x7f683c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c152890 0x7f683c151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 -- 172.21.15.71:0/2110612795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c152ff0 con 0x7f683c1346b0 2023-10-26T05:47:50.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841b32640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 0x7f683c152350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f683800c4d0 tx=0x7f683800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:50.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68380052d0 con 0x7f683c1346b0 2023-10-26T05:47:50.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6838005450 con 0x7f683c1346b0 2023-10-26T05:47:50.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c150b20 con 0x7f683c1346b0 2023-10-26T05:47:50.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838012890 con 0x7f683c1346b0 2023-10-26T05:47:50.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c13c8a0 con 0x7f683c1346b0 2023-10-26T05:47:50.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c066b10 con 0x7f683c1346b0 2023-10-26T05:47:50.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6838003600 con 0x7f683c1346b0 2023-10-26T05:47:50.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f68327fc640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6820066470 0x7f6820068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6838083b50 con 0x7f683c1346b0 2023-10-26T05:47:50.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.281+0000 7f6841331640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6820066470 0x7f6820068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.285+0000 7f6841331640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6820066470 0x7f6820068930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f683c0719d0 tx=0x7f682c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:50.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.285+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6838050fd0 con 0x7f683c1346b0 2023-10-26T05:47:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.441+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f683c0721e0 con 0x7f683c1346b0 2023-10-26T05:47:50.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.445+0000 7f68327fc640 1 -- 172.21.15.71:0/2110612795 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f68380545f0 con 0x7f683c1346b0 2023-10-26T05:47:50.451 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:50.451 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:50.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6820066470 msgr2=0x7f6820068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6820066470 0x7f6820068930 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f683c0719d0 tx=0x7f682c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 msgr2=0x7f683c152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 0x7f683c152350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f683800c4d0 tx=0x7f683800c990 comp rx=0 tx=0).stop 2023-10-26T05:47:50.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 shutdown_connections 2023-10-26T05:47:50.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f683c134a90 0x7f683c1ef8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f683c152890 0x7f683c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6820066470 0x7f6820068930 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 --2- 172.21.15.71:0/2110612795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683c1346b0 0x7f683c152350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 >> 172.21.15.71:0/2110612795 conn(0x7f683c069770 msgr2=0x7f683c158580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:50.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 shutdown_connections 2023-10-26T05:47:50.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.449+0000 7f6843d96640 1 -- 172.21.15.71:0/2110612795 wait complete. 2023-10-26T05:47:50.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:50.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 -- 172.21.15.71:0/3640562459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 msgr2=0x7f7c6000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/3640562459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c6000b990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009eb0 tx=0x7f7c6c01d5d0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 -- 172.21.15.71:0/3640562459 shutdown_connections 2023-10-26T05:47:50.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/3640562459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c6000b990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 -- 172.21.15.71:0/3640562459 >> 172.21.15.71:0/3640562459 conn(0x7f7c740aec60 msgr2=0x7f7c740af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 -- 172.21.15.71:0/3640562459 shutdown_connections 2023-10-26T05:47:50.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.597+0000 7f7c7bccb640 1 -- 172.21.15.71:0/3640562459 wait complete. 2023-10-26T05:47:50.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 Processor -- start 2023-10-26T05:47:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- start start 2023-10-26T05:47:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c741ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c740abd80 0x7f7c7413abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c7413b110 0x7f7c741f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7413b5c0 con 0x7f7c60005190 2023-10-26T05:47:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741ef9c0 con 0x7f7c740abd80 2023-10-26T05:47:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741f2df0 con 0x7f7c7413b110 2023-10-26T05:47:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c741ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c741ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35570/0 (socket says 172.21.15.71:35570) 2023-10-26T05:47:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 -- 172.21.15.71:0/1153244853 learned_addr learned my addr 172.21.15.71:0/1153244853 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79266640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c740abd80 0x7f7c7413abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7a268640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c7413b110 0x7f7c741f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 -- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c7413b110 msgr2=0x7f7c741f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c7413b110 0x7f7c741f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 -- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c740abd80 msgr2=0x7f7c7413abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c740abd80 0x7f7c7413abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 -- 172.21.15.71:0/1153244853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740aafc0 con 0x7f7c60005190 2023-10-26T05:47:50.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79266640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c740abd80 0x7f7c7413abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:50.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7a268640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c7413b110 0x7f7c741f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c79a67640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c741ef480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c70004fd0 tx=0x7f7c7000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70010890 con 0x7f7c60005190 2023-10-26T05:47:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c70010b00 con 0x7f7c60005190 2023-10-26T05:47:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740ab2a0 con 0x7f7c60005190 2023-10-26T05:47:50.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70011460 con 0x7f7c60005190 2023-10-26T05:47:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c740ab860 con 0x7f7c60005190 2023-10-26T05:47:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.601+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c74066b10 con 0x7f7c60005190 2023-10-26T05:47:50.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.605+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c700115c0 con 0x7f7c60005190 2023-10-26T05:47:50.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.605+0000 7f7c6a7fc640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c58066470 0x7f7c58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.605+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c70084580 con 0x7f7c60005190 2023-10-26T05:47:50.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.605+0000 7f7c79266640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c58066470 0x7f7c58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.605+0000 7f7c79266640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c58066470 0x7f7c58068930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7c741396c0 tx=0x7f7c64006180 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:50.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.609+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c70051a90 con 0x7f7c60005190 2023-10-26T05:47:50.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.761+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c740aba50 con 0x7f7c60005190 2023-10-26T05:47:50.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c6a7fc640 1 -- 172.21.15.71:0/1153244853 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c700150a0 con 0x7f7c60005190 2023-10-26T05:47:50.768 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:50.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c58066470 msgr2=0x7f7c58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c58066470 0x7f7c58068930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7c741396c0 tx=0x7f7c64006180 comp rx=0 tx=0).stop 2023-10-26T05:47:50.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 msgr2=0x7f7c741ef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c741ef480 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7c70004fd0 tx=0x7f7c7000e960 comp rx=0 tx=0).stop 2023-10-26T05:47:50.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 shutdown_connections 2023-10-26T05:47:50.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c7413b110 0x7f7c741f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c740abd80 0x7f7c7413abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c58066470 0x7f7c58068930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 --2- 172.21.15.71:0/1153244853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c60005190 0x7f7c741ef480 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 >> 172.21.15.71:0/1153244853 conn(0x7f7c740aec60 msgr2=0x7f7c7406ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:50.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 shutdown_connections 2023-10-26T05:47:50.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.765+0000 7f7c7bccb640 1 -- 172.21.15.71:0/1153244853 wait complete. 2023-10-26T05:47:50.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:50.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- 172.21.15.71:0/571265320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680cd880 msgr2=0x7f4d6815b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/571265320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680cd880 0x7f4d6815b8e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4d680c4dd0 tx=0x7f4d6001cc70 comp rx=0 tx=0).stop 2023-10-26T05:47:50.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- 172.21.15.71:0/571265320 shutdown_connections 2023-10-26T05:47:50.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/571265320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680cd880 0x7f4d6815b8e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/571265320 >> v2:172.21.15.86:3300/0 conn(0x7f4d680d0850 0x7f4d6815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- 172.21.15.71:0/571265320 >> 172.21.15.71:0/571265320 conn(0x7f4d680cdd80 msgr2=0x7f4d680ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:50.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- 172.21.15.71:0/571265320 shutdown_connections 2023-10-26T05:47:50.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- 172.21.15.71:0/571265320 wait complete. 2023-10-26T05:47:50.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 Processor -- start 2023-10-26T05:47:50.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- start start 2023-10-26T05:47:50.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d680cd880 0x7f4d68152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d680d0470 0x7f4d680c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 0x7f4d680c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681f95c0 con 0x7f4d680d0850 2023-10-26T05:47:50.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681f8940 con 0x7f4d680d0470 2023-10-26T05:47:50.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6cc12640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681f8c20 con 0x7f4d680cd880 2023-10-26T05:47:50.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 0x7f4d680c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 0x7f4d680c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35588/0 (socket says 172.21.15.71:35588) 2023-10-26T05:47:50.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 -- 172.21.15.71:0/2951222914 learned_addr learned my addr 172.21.15.71:0/2951222914 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:50.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 -- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d680cd880 msgr2=0x7f4d68152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d6659c640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d680cd880 0x7f4d68152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d680cd880 0x7f4d68152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d65d9b640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d680d0470 0x7f4d680c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 -- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d680d0470 msgr2=0x7f4d680c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:50.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d680d0470 0x7f4d680c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:50.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 -- 172.21.15.71:0/2951222914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600098e0 con 0x7f4d680d0850 2023-10-26T05:47:50.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d66d9d640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 0x7f4d680c7070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d600a8020 tx=0x7f4d60005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:50.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60003e50 con 0x7f4d680d0850 2023-10-26T05:47:50.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d60005320 con 0x7f4d680d0850 2023-10-26T05:47:50.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.909+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60005710 con 0x7f4d680d0850 2023-10-26T05:47:50.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d681f8f00 con 0x7f4d680d0850 2023-10-26T05:47:50.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d68152fd0 con 0x7f4d680d0850 2023-10-26T05:47:50.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68066b10 con 0x7f4d680d0850 2023-10-26T05:47:50.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4d600b4040 con 0x7f4d680d0850 2023-10-26T05:47:50.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d5effd640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d48066470 0x7f4d48068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:50.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d6011ee20 con 0x7f4d680d0850 2023-10-26T05:47:50.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d6659c640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d48066470 0x7f4d48068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:50.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.913+0000 7f4d6659c640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d48066470 0x7f4d48068930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d58001600 tx=0x7f4d58009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:50.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:50.917+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4d600ec450 con 0x7f4d680d0850 2023-10-26T05:47:51.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.069+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d68066b10 con 0x7f4d680d0850 2023-10-26T05:47:51.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d5effd640 1 -- 172.21.15.71:0/2951222914 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d600b20a0 con 0x7f4d680d0850 2023-10-26T05:47:51.076 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:51.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d48066470 msgr2=0x7f4d48068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d48066470 0x7f4d48068930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d58001600 tx=0x7f4d58009040 comp rx=0 tx=0).stop 2023-10-26T05:47:51.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 msgr2=0x7f4d680c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 0x7f4d680c7070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d600a8020 tx=0x7f4d60005040 comp rx=0 tx=0).stop 2023-10-26T05:47:51.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 shutdown_connections 2023-10-26T05:47:51.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d48066470 0x7f4d48068930 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d680d0850 0x7f4d680c7070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d680d0470 0x7f4d680c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 --2- 172.21.15.71:0/2951222914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d680cd880 0x7f4d68152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 >> 172.21.15.71:0/2951222914 conn(0x7f4d680cdd80 msgr2=0x7f4d6815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 shutdown_connections 2023-10-26T05:47:51.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.073+0000 7f4d6cc12640 1 -- 172.21.15.71:0/2951222914 wait complete. 2023-10-26T05:47:51.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:51.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/128374966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 msgr2=0x7f1c3c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 --2- 172.21.15.71:0/128374966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c15b3a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009840 tx=0x7f1c2c01c6b0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/128374966 shutdown_connections 2023-10-26T05:47:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 --2- 172.21.15.71:0/128374966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c15b3a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 --2- 172.21.15.71:0/128374966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c3c0d0470 0x7f1c3c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/128374966 >> 172.21.15.71:0/128374966 conn(0x7f1c3c0cdd80 msgr2=0x7f1c3c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/128374966 shutdown_connections 2023-10-26T05:47:51.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/128374966 wait complete. 2023-10-26T05:47:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 Processor -- start 2023-10-26T05:47:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- start start 2023-10-26T05:47:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c3c0cd880 0x7f1c3c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c0d0470 0x7f1c3c1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c151ea0 con 0x7f1c3c0d0850 2023-10-26T05:47:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c14a900 con 0x7f1c3c0d0470 2023-10-26T05:47:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c154eb0 con 0x7f1c3c0cd880 2023-10-26T05:47:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35610/0 (socket says 172.21.15.71:35610) 2023-10-26T05:47:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 -- 172.21.15.71:0/3797137275 learned_addr learned my addr 172.21.15.71:0/3797137275 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c41e0b640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c3c0cd880 0x7f1c3c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 -- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c3c0cd880 msgr2=0x7f1c3c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c3c0cd880 0x7f1c3c1561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4160a640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c0d0470 0x7f1c3c1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 -- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c0d0470 msgr2=0x7f1c3c1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c0d0470 0x7f1c3c1599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 -- 172.21.15.71:0/3797137275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0094a0 con 0x7f1c3c0d0850 2023-10-26T05:47:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4260c640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c159f30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c780 tx=0x7f1c3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34003db0 con 0x7f1c3c0d0850 2023-10-26T05:47:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c3400ce90 con 0x7f1c3c0d0850 2023-10-26T05:47:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c14ac30 con 0x7f1c3c0d0850 2023-10-26T05:47:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340052d0 con 0x7f1c3c0d0850 2023-10-26T05:47:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.217+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c154800 con 0x7f1c3c0d0850 2023-10-26T05:47:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.221+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c066b10 con 0x7f1c3c0d0850 2023-10-26T05:47:51.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.221+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c34005430 con 0x7f1c3c0d0850 2023-10-26T05:47:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.221+0000 7f1c327fc640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c200664a0 0x7f1c20068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.221+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c34083e60 con 0x7f1c3c0d0850 2023-10-26T05:47:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.221+0000 7f1c41e0b640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c200664a0 0x7f1c20068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.221+0000 7f1c41e0b640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c200664a0 0x7f1c20068960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c380058d0 tx=0x7f1c38005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.225+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1c340512e0 con 0x7f1c3c0d0850 2023-10-26T05:47:51.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c3c154a50 con 0x7f1c3c0d0850 2023-10-26T05:47:51.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c327fc640 1 -- 172.21.15.71:0/3797137275 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c34054900 con 0x7f1c3c0d0850 2023-10-26T05:47:51.385 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:51.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c200664a0 msgr2=0x7f1c20068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c200664a0 0x7f1c20068960 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c380058d0 tx=0x7f1c38005860 comp rx=0 tx=0).stop 2023-10-26T05:47:51.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 msgr2=0x7f1c3c159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c159f30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400c780 tx=0x7f1c3400cc40 comp rx=0 tx=0).stop 2023-10-26T05:47:51.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 shutdown_connections 2023-10-26T05:47:51.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c200664a0 0x7f1c20068960 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.381+0000 7f1c4406f640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c3c0d0850 0x7f1c3c159f30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.385+0000 7f1c4406f640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c3c0d0470 0x7f1c3c1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.385+0000 7f1c4406f640 1 --2- 172.21.15.71:0/3797137275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c3c0cd880 0x7f1c3c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.385+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 >> 172.21.15.71:0/3797137275 conn(0x7f1c3c0cdd80 msgr2=0x7f1c3c0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.385+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 shutdown_connections 2023-10-26T05:47:51.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.385+0000 7f1c4406f640 1 -- 172.21.15.71:0/3797137275 wait complete. 2023-10-26T05:47:51.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:51.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/2390111385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4280ac740 msgr2=0x7fd42815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 --2- 172.21.15.71:0/2390111385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4280ac740 0x7fd42815b4b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd4280caa50 tx=0x7fd41c01d9a0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/2390111385 shutdown_connections 2023-10-26T05:47:51.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 --2- 172.21.15.71:0/2390111385 >> v2:172.21.15.86:3300/0 conn(0x7fd42815b9f0 0x7fd428070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 --2- 172.21.15.71:0/2390111385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd4280ac740 0x7fd42815b4b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/2390111385 >> 172.21.15.71:0/2390111385 conn(0x7fd428069770 msgr2=0x7fd428069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/2390111385 shutdown_connections 2023-10-26T05:47:51.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/2390111385 wait complete. 2023-10-26T05:47:51.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 Processor -- start 2023-10-26T05:47:51.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- start start 2023-10-26T05:47:51.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4280ac740 0x7fd4281f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42813d420 0x7fd4280b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 0x7fd428135170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4280b2040 con 0x7fd42815b9f0 2023-10-26T05:47:51.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4280b2700 con 0x7fd42813d420 2023-10-26T05:47:51.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4281356b0 con 0x7fd4280ac740 2023-10-26T05:47:51.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 0x7fd428135170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 0x7fd428135170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35630/0 (socket says 172.21.15.71:35630) 2023-10-26T05:47:51.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 -- 172.21.15.71:0/239508101 learned_addr learned my addr 172.21.15.71:0/239508101 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:51.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42d9fe640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4280ac740 0x7fd4281f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42d1fd640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42813d420 0x7fd4280b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 -- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4280ac740 msgr2=0x7fd4281f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4280ac740 0x7fd4281f4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 -- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42813d420 msgr2=0x7fd4280b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42813d420 0x7fd4280b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 -- 172.21.15.71:0/239508101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c00a5b0 con 0x7fd42815b9f0 2023-10-26T05:47:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42e1ff640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 0x7fd428135170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd41800b970 tx=0x7fd41800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41800cb40 con 0x7fd42815b9f0 2023-10-26T05:47:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd41800cdb0 con 0x7fd42815b9f0 2023-10-26T05:47:51.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4280b1830 con 0x7fd42815b9f0 2023-10-26T05:47:51.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4180146a0 con 0x7fd42815b9f0 2023-10-26T05:47:51.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.521+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4280b1a80 con 0x7fd42815b9f0 2023-10-26T05:47:51.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.525+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd428066b10 con 0x7fd42815b9f0 2023-10-26T05:47:51.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.525+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd41801d3b0 con 0x7fd42815b9f0 2023-10-26T05:47:51.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.525+0000 7fd4267fc640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd414066470 0x7fd414068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.525+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd418083f80 con 0x7fd42815b9f0 2023-10-26T05:47:51.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.525+0000 7fd42d9fe640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd414066470 0x7fd414068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.525+0000 7fd42d9fe640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd414066470 0x7fd414068930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd420000f70 tx=0x7fd420009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:51.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.529+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd4180b5090 con 0x7fd42815b9f0 2023-10-26T05:47:51.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.669+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd428072230 con 0x7fd42815b9f0 2023-10-26T05:47:51.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.673+0000 7fd4267fc640 1 -- 172.21.15.71:0/239508101 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fd418019090 con 0x7fd42815b9f0 2023-10-26T05:47:51.678 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:51.678 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:51.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.673+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd414066470 msgr2=0x7fd414068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.673+0000 7fd42fc62640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd414066470 0x7fd414068930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd420000f70 tx=0x7fd420009040 comp rx=0 tx=0).stop 2023-10-26T05:47:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.673+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 msgr2=0x7fd428135170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 0x7fd428135170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd41800b970 tx=0x7fd41800be30 comp rx=0 tx=0).stop 2023-10-26T05:47:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 shutdown_connections 2023-10-26T05:47:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd414066470 0x7fd414068930 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd42815b9f0 0x7fd428135170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd42813d420 0x7fd4280b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 --2- 172.21.15.71:0/239508101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd4280ac740 0x7fd4281f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 >> 172.21.15.71:0/239508101 conn(0x7fd428069770 msgr2=0x7fd428161280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 shutdown_connections 2023-10-26T05:47:51.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.677+0000 7fd42fc62640 1 -- 172.21.15.71:0/239508101 wait complete. 2023-10-26T05:47:51.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:51.698 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/223639615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 msgr2=0x7f118c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 --2- 172.21.15.71:0/223639615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c06aa70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f118c0c4dd0 tx=0x7f117c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/223639615 shutdown_connections 2023-10-26T05:47:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 --2- 172.21.15.71:0/223639615 >> v2:172.21.15.86:3300/0 conn(0x7f118c0d0850 0x7f118c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 --2- 172.21.15.71:0/223639615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c06aa70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/223639615 >> 172.21.15.71:0/223639615 conn(0x7f118c0cdd80 msgr2=0x7f118c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/223639615 shutdown_connections 2023-10-26T05:47:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/223639615 wait complete. 2023-10-26T05:47:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 Processor -- start 2023-10-26T05:47:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- start start 2023-10-26T05:47:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f118c0cd880 0x7f118c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f118c0d0850 0x7f118c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c14fb50 con 0x7f118c0d0470 2023-10-26T05:47:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c15e060 con 0x7f118c0d0850 2023-10-26T05:47:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118c15e340 con 0x7f118c0cd880 2023-10-26T05:47:51.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55380/0 (socket says 172.21.15.71:55380) 2023-10-26T05:47:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 -- 172.21.15.71:0/1126954320 learned_addr learned my addr 172.21.15.71:0/1126954320 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:51.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1190cd1640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f118c0d0850 0x7f118c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f118bfff640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f118c0cd880 0x7f118c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 -- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f118c0cd880 msgr2=0x7f118c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f118c0cd880 0x7f118c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 -- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f118c0d0850 msgr2=0x7f118c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f118c0d0850 0x7f118c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 -- 172.21.15.71:0/1126954320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009560 con 0x7f118c0d0470 2023-10-26T05:47:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f118bfff640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f118c0cd880 0x7f118c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:51.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11837fe640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c15c010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f117400b410 tx=0x7f117400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:51.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174011830 con 0x7f118c0d0470 2023-10-26T05:47:51.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1174011a80 con 0x7f118c0d0470 2023-10-26T05:47:51.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c15e620 con 0x7f118c0d0470 2023-10-26T05:47:51.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1174011e90 con 0x7f118c0d0470 2023-10-26T05:47:51.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c15ebe0 con 0x7f118c0d0470 2023-10-26T05:47:51.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.813+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c066b10 con 0x7f118c0d0470 2023-10-26T05:47:51.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.817+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f117401a400 con 0x7f118c0d0470 2023-10-26T05:47:51.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.817+0000 7f11897fa640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f116c066470 0x7f116c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:51.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.817+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1174084270 con 0x7f118c0d0470 2023-10-26T05:47:51.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.817+0000 7f118bfff640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f116c066470 0x7f116c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:51.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.817+0000 7f118bfff640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f116c066470 0x7f116c068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f117c0af000 tx=0x7f117c01afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:51.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.821+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11740516f0 con 0x7f118c0d0470 2023-10-26T05:47:51.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f118c0721e0 con 0x7f118c0d0470 2023-10-26T05:47:51.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f11897fa640 1 -- 172.21.15.71:0/1126954320 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1174054d10 con 0x7f118c0d0470 2023-10-26T05:47:51.966 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:51.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f116c066470 msgr2=0x7f116c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f116c066470 0x7f116c068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f117c0af000 tx=0x7f117c01afe0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 msgr2=0x7f118c15c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:51.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c15c010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f117400b410 tx=0x7f117400b8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 shutdown_connections 2023-10-26T05:47:51.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f118c0d0850 0x7f118c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f116c066470 0x7f116c068930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.961+0000 7f1192734640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f118c0d0470 0x7f118c15c010 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.965+0000 7f1192734640 1 --2- 172.21.15.71:0/1126954320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f118c0cd880 0x7f118c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:51.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.965+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 >> 172.21.15.71:0/1126954320 conn(0x7f118c0cdd80 msgr2=0x7f118c154530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:51.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.965+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 shutdown_connections 2023-10-26T05:47:51.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:51.965+0000 7f1192734640 1 -- 172.21.15.71:0/1126954320 wait complete. 2023-10-26T05:47:51.983 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:52.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 -- 172.21.15.71:0/3891005003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 msgr2=0x7fc8140b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 --2- 172.21.15.71:0/3891005003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc8140b1250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00a950 tx=0x7fc80c01dae0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 -- 172.21.15.71:0/3891005003 shutdown_connections 2023-10-26T05:47:52.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 --2- 172.21.15.71:0/3891005003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc8140b1250 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 -- 172.21.15.71:0/3891005003 >> 172.21.15.71:0/3891005003 conn(0x7fc8140aec60 msgr2=0x7fc8140af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:52.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 -- 172.21.15.71:0/3891005003 shutdown_connections 2023-10-26T05:47:52.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 -- 172.21.15.71:0/3891005003 wait complete. 2023-10-26T05:47:52.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.097+0000 7fc81c39b640 1 Processor -- start 2023-10-26T05:47:52.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- start start 2023-10-26T05:47:52.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc814069770 0x7fc814140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc814070de0 0x7fc814142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc814153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814141400 con 0x7fc8140714c0 2023-10-26T05:47:52.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141431e0 con 0x7fc814070de0 2023-10-26T05:47:52.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141434c0 con 0x7fc814069770 2023-10-26T05:47:52.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc814153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc814153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55398/0 (socket says 172.21.15.71:55398) 2023-10-26T05:47:52.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 -- 172.21.15.71:0/807081732 learned_addr learned my addr 172.21.15.71:0/807081732 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:52.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc819936640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc814070de0 0x7fc814142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a137640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc814069770 0x7fc814140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 -- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc814069770 msgr2=0x7fc814140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc814069770 0x7fc814140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 -- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc814070de0 msgr2=0x7fc814142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc814070de0 0x7fc814142720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 -- 172.21.15.71:0/807081732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c00a5b0 con 0x7fc8140714c0 2023-10-26T05:47:52.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a938640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc814153cd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc80c01a6f0 tx=0x7fc80c0a69a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:52.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003a10 con 0x7fc8140714c0 2023-10-26T05:47:52.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc80c0a6b90 con 0x7fc8140714c0 2023-10-26T05:47:52.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8141437a0 con 0x7fc8140714c0 2023-10-26T05:47:52.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003ec0 con 0x7fc8140714c0 2023-10-26T05:47:52.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81414fa60 con 0x7fc8140714c0 2023-10-26T05:47:52.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc814066b10 con 0x7fc8140714c0 2023-10-26T05:47:52.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc80c0b5040 con 0x7fc8140714c0 2023-10-26T05:47:52.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc80affd640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc7f8066470 0x7fc7f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc80c11f030 con 0x7fc8140714c0 2023-10-26T05:47:52.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.101+0000 7fc81a137640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc7f8066470 0x7fc7f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.105+0000 7fc81a137640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc7f8066470 0x7fc7f8068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc810001cf0 tx=0x7fc810009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:52.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.105+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc80c0ec4c0 con 0x7fc8140714c0 2023-10-26T05:47:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.261+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc814066b10 con 0x7fc8140714c0 2023-10-26T05:47:52.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.261+0000 7fc80affd640 1 -- 172.21.15.71:0/807081732 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc80c0efae0 con 0x7fc8140714c0 2023-10-26T05:47:52.267 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:52.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc7f8066470 msgr2=0x7fc7f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc7f8066470 0x7fc7f8068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc810001cf0 tx=0x7fc810009040 comp rx=0 tx=0).stop 2023-10-26T05:47:52.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 msgr2=0x7fc814153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc814153cd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc80c01a6f0 tx=0x7fc80c0a69a0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 shutdown_connections 2023-10-26T05:47:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc7f8066470 0x7fc7f8068930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8140714c0 0x7fc814153cd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc814070de0 0x7fc814142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 --2- 172.21.15.71:0/807081732 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc814069770 0x7fc814140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 >> 172.21.15.71:0/807081732 conn(0x7fc8140aec60 msgr2=0x7fc814156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:52.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 shutdown_connections 2023-10-26T05:47:52.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.265+0000 7fc81c39b640 1 -- 172.21.15.71:0/807081732 wait complete. 2023-10-26T05:47:52.286 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:52.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/3923318173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 msgr2=0x7fae980b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 --2- 172.21.15.71:0/3923318173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae980b0fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae88009840 tx=0x7fae8801c790 comp rx=0 tx=0).stop 2023-10-26T05:47:52.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/3923318173 shutdown_connections 2023-10-26T05:47:52.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 --2- 172.21.15.71:0/3923318173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae980b0fa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/3923318173 >> 172.21.15.71:0/3923318173 conn(0x7fae980aec60 msgr2=0x7fae980af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/3923318173 shutdown_connections 2023-10-26T05:47:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/3923318173 wait complete. 2023-10-26T05:47:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 Processor -- start 2023-10-26T05:47:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- start start 2023-10-26T05:47:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae98070dc0 0x7fae9813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae9813b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9815bf70 0x7fae98140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980ac2e0 con 0x7fae9815bb90 2023-10-26T05:47:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9813b860 con 0x7fae9815bf70 2023-10-26T05:47:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9813bb30 con 0x7fae98070dc0 2023-10-26T05:47:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae9813b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae9813b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55428/0 (socket says 172.21.15.71:55428) 2023-10-26T05:47:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 -- 172.21.15.71:0/4122883601 learned_addr learned my addr 172.21.15.71:0/4122883601 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9659c640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae98070dc0 0x7fae9813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 -- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae98070dc0 msgr2=0x7fae9813ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae96d9d640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9815bf70 0x7fae98140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae98070dc0 0x7fae9813ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 -- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9815bf70 msgr2=0x7fae98140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9815bf70 0x7fae98140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 -- 172.21.15.71:0/4122883601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae880094a0 con 0x7fae9815bb90 2023-10-26T05:47:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8dd9b640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae9813b320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c410 tx=0x7fae8000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80011790 con 0x7fae9815bb90 2023-10-26T05:47:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae800119e0 con 0x7fae9815bb90 2023-10-26T05:47:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9813be10 con 0x7fae9815bb90 2023-10-26T05:47:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae80011e30 con 0x7fae9815bb90 2023-10-26T05:47:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.413+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae98138b00 con 0x7fae9815bb90 2023-10-26T05:47:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.417+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae98066b10 con 0x7fae9815bb90 2023-10-26T05:47:52.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.417+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fae80007020 con 0x7fae9815bb90 2023-10-26T05:47:52.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.417+0000 7fae8f7fe640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae780664a0 0x7fae78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.417+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae800846f0 con 0x7fae9815bb90 2023-10-26T05:47:52.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.417+0000 7fae9659c640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae780664a0 0x7fae78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.417+0000 7fae9659c640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae780664a0 0x7fae78068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae880af000 tx=0x7fae8801afd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:52.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.421+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fae80051ce0 con 0x7fae9815bb90 2023-10-26T05:47:52.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae98138d20 con 0x7fae9815bb90 2023-10-26T05:47:52.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae8f7fe640 1 -- 172.21.15.71:0/4122883601 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae80055300 con 0x7fae9815bb90 2023-10-26T05:47:52.566 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae780664a0 msgr2=0x7fae78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae780664a0 0x7fae78068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fae880af000 tx=0x7fae8801afd0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 msgr2=0x7fae9813b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae9813b320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae8000c410 tx=0x7fae8000c8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 shutdown_connections 2023-10-26T05:47:52.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae9815bf70 0x7fae98140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae780664a0 0x7fae78068960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9815bb90 0x7fae9813b320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 --2- 172.21.15.71:0/4122883601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae98070dc0 0x7fae9813ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 >> 172.21.15.71:0/4122883601 conn(0x7fae980aec60 msgr2=0x7fae980cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 shutdown_connections 2023-10-26T05:47:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.561+0000 7fae9cff8640 1 -- 172.21.15.71:0/4122883601 wait complete. 2023-10-26T05:47:52.583 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:52.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- 172.21.15.71:0/3477738707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 msgr2=0x7f2a040af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/3477738707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040af5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009bc0 tx=0x7f29fc01cab0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- 172.21.15.71:0/3477738707 shutdown_connections 2023-10-26T05:47:52.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/3477738707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040af5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- 172.21.15.71:0/3477738707 >> 172.21.15.71:0/3477738707 conn(0x7f2a040b0f20 msgr2=0x7f2a040b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:52.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- 172.21.15.71:0/3477738707 shutdown_connections 2023-10-26T05:47:52.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- 172.21.15.71:0/3477738707 wait complete. 2023-10-26T05:47:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 Processor -- start 2023-10-26T05:47:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- start start 2023-10-26T05:47:52.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a040cc3a0 0x7f2a040c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a04144c30 0x7f2a0415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040c9310 con 0x7f2a04145010 2023-10-26T05:47:52.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0415ac60 con 0x7f2a04144c30 2023-10-26T05:47:52.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0c45c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a040c7d30 con 0x7f2a040cc3a0 2023-10-26T05:47:52.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0a9f9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.697+0000 7f2a0a9f9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55446/0 (socket says 172.21.15.71:55446) 2023-10-26T05:47:52.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 -- 172.21.15.71:0/1815662238 learned_addr learned my addr 172.21.15.71:0/1815662238 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:52.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a099f7640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a04144c30 0x7f2a0415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a1f8640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a040cc3a0 0x7f2a040c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 -- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a040cc3a0 msgr2=0x7f2a040c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a040cc3a0 0x7f2a040c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 -- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a04144c30 msgr2=0x7f2a0415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a04144c30 0x7f2a0415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 -- 172.21.15.71:0/1815662238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc009820 con 0x7f2a04145010 2023-10-26T05:47:52.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a9f9640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040c6d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f29fc01c520 tx=0x7f29fc0a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:52.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc003b80 con 0x7f2a04145010 2023-10-26T05:47:52.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29fc003dd0 con 0x7f2a04145010 2023-10-26T05:47:52.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a040c8010 con 0x7f2a04145010 2023-10-26T05:47:52.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc004230 con 0x7f2a04145010 2023-10-26T05:47:52.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a040c85d0 con 0x7f2a04145010 2023-10-26T05:47:52.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a04066d00 con 0x7f2a04145010 2023-10-26T05:47:52.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29fc004390 con 0x7f2a04145010 2023-10-26T05:47:52.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f29faffd640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29e8066470 0x7f29e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:52.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29fc11efb0 con 0x7f2a04145010 2023-10-26T05:47:52.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a1f8640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29e8066470 0x7f29e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:52.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.701+0000 7f2a0a1f8640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29e8066470 0x7f29e8068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a040ccbf0 tx=0x7f2a00009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:52.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.705+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29fc150090 con 0x7f2a04145010 2023-10-26T05:47:52.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a04066d00 con 0x7f2a04145010 2023-10-26T05:47:52.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f29faffd640 1 -- 172.21.15.71:0/1815662238 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f29fc0b2020 con 0x7f2a04145010 2023-10-26T05:47:52.866 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:52.866 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29e8066470 msgr2=0x7f29e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29e8066470 0x7f29e8068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a040ccbf0 tx=0x7f2a00009040 comp rx=0 tx=0).stop 2023-10-26T05:47:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 msgr2=0x7f2a040c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:52.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040c6d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f29fc01c520 tx=0x7f29fc0a5910 comp rx=0 tx=0).stop 2023-10-26T05:47:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 shutdown_connections 2023-10-26T05:47:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29e8066470 0x7f29e8068930 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a04145010 0x7f2a040c6d80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a04144c30 0x7f2a0415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 --2- 172.21.15.71:0/1815662238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a040cc3a0 0x7f2a040c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 >> 172.21.15.71:0/1815662238 conn(0x7f2a040b0f20 msgr2=0x7f2a040c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 shutdown_connections 2023-10-26T05:47:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:52.861+0000 7f2a0c45c640 1 -- 172.21.15.71:0/1815662238 wait complete. 2023-10-26T05:47:52.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:52.885 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:53.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff7929d6640 1 -- 172.21.15.71:0/548294744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff774005190 msgr2=0x7ff77400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff7929d6640 1 --2- 172.21.15.71:0/548294744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff774005190 0x7ff77400b990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff784001da0 tx=0x7ff78401cb20 comp rx=0 tx=0).stop 2023-10-26T05:47:53.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff790f73640 1 -- 172.21.15.71:0/548294744 reap_dead start 2023-10-26T05:47:53.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff7929d6640 1 -- 172.21.15.71:0/548294744 shutdown_connections 2023-10-26T05:47:53.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff7929d6640 1 -- 172.21.15.71:0/548294744 >> 172.21.15.71:0/548294744 conn(0x7ff78c0aec60 msgr2=0x7ff78c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff7929d6640 1 -- 172.21.15.71:0/548294744 shutdown_connections 2023-10-26T05:47:53.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.005+0000 7ff7929d6640 1 -- 172.21.15.71:0/548294744 wait complete. 2023-10-26T05:47:53.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 Processor -- start 2023-10-26T05:47:53.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 -- start start 2023-10-26T05:47:53.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff78c1eef50 0x7ff78c13f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 0x7ff78c0ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c1410e0 0x7ff78c0ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c140c50 con 0x7ff78c12fff0 2023-10-26T05:47:53.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0abee0 con 0x7ff78c1410e0 2023-10-26T05:47:53.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c140040 con 0x7ff78c1eef50 2023-10-26T05:47:53.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 0x7ff78c0ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 0x7ff78c0ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55468/0 (socket says 172.21.15.71:55468) 2023-10-26T05:47:53.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 -- 172.21.15.71:0/544574815 learned_addr learned my addr 172.21.15.71:0/544574815 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:53.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff790f73640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c1410e0 0x7ff78c0ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78bfff640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff78c1eef50 0x7ff78c13f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 -- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff78c1eef50 msgr2=0x7ff78c13f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff78c1eef50 0x7ff78c13f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 -- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c1410e0 msgr2=0x7ff78c0ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c1410e0 0x7ff78c0ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 -- 172.21.15.71:0/544574815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c140320 con 0x7ff78c12fff0 2023-10-26T05:47:53.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff78b7fe640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 0x7ff78c0ab460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff780006e30 tx=0x7ff780004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7800108c0 con 0x7ff78c12fff0 2023-10-26T05:47:53.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff780010b10 con 0x7ff78c12fff0 2023-10-26T05:47:53.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780011460 con 0x7ff78c12fff0 2023-10-26T05:47:53.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c140600 con 0x7ff78c12fff0 2023-10-26T05:47:53.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.009+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c13fc80 con 0x7ff78c12fff0 2023-10-26T05:47:53.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.013+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c066b10 con 0x7ff78c12fff0 2023-10-26T05:47:53.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.013+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7800115c0 con 0x7ff78c12fff0 2023-10-26T05:47:53.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.013+0000 7ff788ff9640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7780664a0 0x7ff778068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.013+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7800848b0 con 0x7ff78c12fff0 2023-10-26T05:47:53.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.013+0000 7ff78bfff640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7780664a0 0x7ff778068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.013+0000 7ff78bfff640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7780664a0 0x7ff778068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff77c005fd0 tx=0x7ff77c005f60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.017+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff780051d40 con 0x7ff78c12fff0 2023-10-26T05:47:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff78c071220 con 0x7ff78c12fff0 2023-10-26T05:47:53.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff788ff9640 1 -- 172.21.15.71:0/544574815 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff78c071220 con 0x7ff78c12fff0 2023-10-26T05:47:53.173 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7780664a0 msgr2=0x7ff778068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7780664a0 0x7ff778068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff77c005fd0 tx=0x7ff77c005f60 comp rx=0 tx=0).stop 2023-10-26T05:47:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 msgr2=0x7ff78c0ab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 0x7ff78c0ab460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff780006e30 tx=0x7ff780004ac0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 shutdown_connections 2023-10-26T05:47:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff78c1410e0 0x7ff78c0ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7780664a0 0x7ff778068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff78c12fff0 0x7ff78c0ab460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 --2- 172.21.15.71:0/544574815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff78c1eef50 0x7ff78c13f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 >> 172.21.15.71:0/544574815 conn(0x7ff78c0aec60 msgr2=0x7ff78c13d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 shutdown_connections 2023-10-26T05:47:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.169+0000 7ff7929d6640 1 -- 172.21.15.71:0/544574815 wait complete. 2023-10-26T05:47:53.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:53.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- 172.21.15.71:0/2314226030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 msgr2=0x7f9538004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 --2- 172.21.15.71:0/2314226030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f9538004c50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f953400c700 tx=0x7f953401c680 comp rx=0 tx=0).stop 2023-10-26T05:47:53.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- 172.21.15.71:0/2314226030 shutdown_connections 2023-10-26T05:47:53.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 --2- 172.21.15.71:0/2314226030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f9538004c50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- 172.21.15.71:0/2314226030 >> 172.21.15.71:0/2314226030 conn(0x7f954c0aec60 msgr2=0x7f954c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- 172.21.15.71:0/2314226030 shutdown_connections 2023-10-26T05:47:53.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- 172.21.15.71:0/2314226030 wait complete. 2023-10-26T05:47:53.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 Processor -- start 2023-10-26T05:47:53.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- start start 2023-10-26T05:47:53.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f954c13b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f954c140440 0x7f954c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f954c159e60 0x7f954c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c156950 con 0x7f95380047d0 2023-10-26T05:47:53.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c13bcc0 con 0x7f954c140440 2023-10-26T05:47:53.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9550843640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c157900 con 0x7f954c159e60 2023-10-26T05:47:53.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f954c13b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f954c13b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55480/0 (socket says 172.21.15.71:55480) 2023-10-26T05:47:53.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 -- 172.21.15.71:0/3690717129 learned_addr learned my addr 172.21.15.71:0/3690717129 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:53.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954ad9d640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f954c159e60 0x7f954c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f9541d9b640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f954c140440 0x7f954c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 -- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f954c159e60 msgr2=0x7f954c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f954c159e60 0x7f954c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 -- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f954c140440 msgr2=0x7f954c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f954c140440 0x7f954c159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 -- 172.21.15.71:0/3690717129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c156b40 con 0x7f95380047d0 2023-10-26T05:47:53.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f954a59c640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f954c13b780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f953c00f9b0 tx=0x7f953c00fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.301+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00bc50 con 0x7f95380047d0 2023-10-26T05:47:53.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f953c011440 con 0x7f95380047d0 2023-10-26T05:47:53.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c156e20 con 0x7f95380047d0 2023-10-26T05:47:53.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c00bea0 con 0x7f95380047d0 2023-10-26T05:47:53.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c1573e0 con 0x7f95380047d0 2023-10-26T05:47:53.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c066b10 con 0x7f95380047d0 2023-10-26T05:47:53.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f953c011850 con 0x7f95380047d0 2023-10-26T05:47:53.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f95437fe640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f952c066470 0x7f952c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f953c084a60 con 0x7f95380047d0 2023-10-26T05:47:53.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f9541d9b640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f952c066470 0x7f952c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.305+0000 7f9541d9b640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f952c066470 0x7f952c068930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95380057b0 tx=0x7f9534019420 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.309+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f953c051ee0 con 0x7f95380047d0 2023-10-26T05:47:53.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f954c0721e0 con 0x7f95380047d0 2023-10-26T05:47:53.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f95437fe640 1 -- 172.21.15.71:0/3690717129 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f953c0200a0 con 0x7f95380047d0 2023-10-26T05:47:53.457 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f952c066470 msgr2=0x7f952c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f952c066470 0x7f952c068930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f95380057b0 tx=0x7f9534019420 comp rx=0 tx=0).stop 2023-10-26T05:47:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 msgr2=0x7f954c13b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f954c13b780 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f953c00f9b0 tx=0x7f953c00fe70 comp rx=0 tx=0).stop 2023-10-26T05:47:53.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 shutdown_connections 2023-10-26T05:47:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f954c159e60 0x7f954c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f954c140440 0x7f954c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f952c066470 0x7f952c068930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 --2- 172.21.15.71:0/3690717129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95380047d0 0x7f954c13b780 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 >> 172.21.15.71:0/3690717129 conn(0x7f954c0aec60 msgr2=0x7f954c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 shutdown_connections 2023-10-26T05:47:53.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.453+0000 7f9550843640 1 -- 172.21.15.71:0/3690717129 wait complete. 2023-10-26T05:47:53.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:53.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/3519642779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47bc0052b0 msgr2=0x7f47bc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 --2- 172.21.15.71:0/3519642779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47bc0052b0 0x7f47bc00b990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c590 tx=0x7f47c801fa80 comp rx=0 tx=0).stop 2023-10-26T05:47:53.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/3519642779 shutdown_connections 2023-10-26T05:47:53.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 --2- 172.21.15.71:0/3519642779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47bc0052b0 0x7f47bc00b990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/3519642779 >> 172.21.15.71:0/3519642779 conn(0x7f47d006f440 msgr2=0x7f47d006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/3519642779 shutdown_connections 2023-10-26T05:47:53.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/3519642779 wait complete. 2023-10-26T05:47:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 Processor -- start 2023-10-26T05:47:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- start start 2023-10-26T05:47:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc0052b0 0x7f47d0138bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47d00cebc0 0x7f47d013f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 0x7f47d015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0154e10 con 0x7f47d0141b70 2023-10-26T05:47:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0141420 con 0x7f47bc0052b0 2023-10-26T05:47:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00cf960 con 0x7f47d00cebc0 2023-10-26T05:47:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 0x7f47d015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 0x7f47d015c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55504/0 (socket says 172.21.15.71:55504) 2023-10-26T05:47:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 -- 172.21.15.71:0/2084032512 learned_addr learned my addr 172.21.15.71:0/2084032512 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47ceffd640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc0052b0 0x7f47d0138bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47c7fff640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47d00cebc0 0x7f47d013f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 -- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47d00cebc0 msgr2=0x7f47d013f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47d00cebc0 0x7f47d013f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 -- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc0052b0 msgr2=0x7f47d0138bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc0052b0 0x7f47d0138bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 -- 172.21.15.71:0/2084032512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00cf0c0 con 0x7f47d0141b70 2023-10-26T05:47:53.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47ceffd640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc0052b0 0x7f47d0138bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:47:53.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47cf7fe640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 0x7f47d015c9b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47c801e300 tx=0x7f47c8002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80b6070 con 0x7f47d0141b70 2023-10-26T05:47:53.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47c8001800 con 0x7f47d0141b70 2023-10-26T05:47:53.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80a95b0 con 0x7f47d0141b70 2023-10-26T05:47:53.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d00cf3a0 con 0x7f47d0141b70 2023-10-26T05:47:53.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.585+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d0141650 con 0x7f47d0141b70 2023-10-26T05:47:53.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.589+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d0066b10 con 0x7f47d0141b70 2023-10-26T05:47:53.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.589+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f47c80af070 con 0x7f47d0141b70 2023-10-26T05:47:53.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.589+0000 7f47ccff9640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47b00664a0 0x7f47b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.589+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47c8120a00 con 0x7f47d0141b70 2023-10-26T05:47:53.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.589+0000 7f47ceffd640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47b00664a0 0x7f47b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.589+0000 7f47ceffd640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47b00664a0 0x7f47b0068960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47c00017f0 tx=0x7f47c0006b90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.593+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f47c80edde0 con 0x7f47d0141b70 2023-10-26T05:47:53.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.741+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47d0141840 con 0x7f47d0141b70 2023-10-26T05:47:53.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47ccff9640 1 -- 172.21.15.71:0/2084032512 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47c80f1400 con 0x7f47d0141b70 2023-10-26T05:47:53.748 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:53.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47b00664a0 msgr2=0x7f47b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47b00664a0 0x7f47b0068960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47c00017f0 tx=0x7f47c0006b90 comp rx=0 tx=0).stop 2023-10-26T05:47:53.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 msgr2=0x7f47d015c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 0x7f47d015c9b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47c801e300 tx=0x7f47c8002000 comp rx=0 tx=0).stop 2023-10-26T05:47:53.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 shutdown_connections 2023-10-26T05:47:53.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47b00664a0 0x7f47b0068960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d0141b70 0x7f47d015c9b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47d00cebc0 0x7f47d013f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 --2- 172.21.15.71:0/2084032512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47bc0052b0 0x7f47d0138bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 >> 172.21.15.71:0/2084032512 conn(0x7f47d006f440 msgr2=0x7f47d0158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 shutdown_connections 2023-10-26T05:47:53.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.745+0000 7f47d581f640 1 -- 172.21.15.71:0/2084032512 wait complete. 2023-10-26T05:47:53.801 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-26T05:47:53.802 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T05:47:53.803 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:53.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 -- 172.21.15.71:0/4181362805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 msgr2=0x7f9bec071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 --2- 172.21.15.71:0/4181362805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec071440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009c80 tx=0x7f9be401cd50 comp rx=0 tx=0).stop 2023-10-26T05:47:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 -- 172.21.15.71:0/4181362805 shutdown_connections 2023-10-26T05:47:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 --2- 172.21.15.71:0/4181362805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec071440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 --2- 172.21.15.71:0/4181362805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec140cc0 0x7f9bec15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 -- 172.21.15.71:0/4181362805 >> 172.21.15.71:0/4181362805 conn(0x7f9bec069770 msgr2=0x7f9bec069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 -- 172.21.15.71:0/4181362805 shutdown_connections 2023-10-26T05:47:53.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 -- 172.21.15.71:0/4181362805 wait complete. 2023-10-26T05:47:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 Processor -- start 2023-10-26T05:47:53.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.913+0000 7f9bf2338640 1 -- start start 2023-10-26T05:47:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec070fe0 0x7f9bec1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec0a9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bec140cc0 0x7f9bec06fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0a6d30 con 0x7f9bec0b2370 2023-10-26T05:47:53.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec157720 con 0x7f9bec140cc0 2023-10-26T05:47:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0a98b0 con 0x7f9bec070fe0 2023-10-26T05:47:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec0a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec0a9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55526/0 (socket says 172.21.15.71:55526) 2023-10-26T05:47:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 -- 172.21.15.71:0/2522636908 learned_addr learned my addr 172.21.15.71:0/2522636908 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:53.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf08d5640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bec140cc0 0x7f9bec06fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bebfff640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec070fe0 0x7f9bec1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 -- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec070fe0 msgr2=0x7f9bec1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec070fe0 0x7f9bec1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 -- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bec140cc0 msgr2=0x7f9bec06fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bec140cc0 0x7f9bec06fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 -- 172.21.15.71:0/2522636908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40098e0 con 0x7f9bec0b2370 2023-10-26T05:47:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf08d5640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bec140cc0 0x7f9bec06fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:53.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bebfff640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec070fe0 0x7f9bec1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9beb7fe640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec0a9370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c500 tx=0x7f9be000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0005220 con 0x7f9bec0b2370 2023-10-26T05:47:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9be0005470 con 0x7f9bec0b2370 2023-10-26T05:47:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec0ae780 con 0x7f9bec0b2370 2023-10-26T05:47:53.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0007290 con 0x7f9bec0b2370 2023-10-26T05:47:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec0a6950 con 0x7f9bec0b2370 2023-10-26T05:47:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec066b10 con 0x7f9bec0b2370 2023-10-26T05:47:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9be00073f0 con 0x7f9bec0b2370 2023-10-26T05:47:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9be8ff9640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9bd8066470 0x7f9bd8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9be0083a10 con 0x7f9bec0b2370 2023-10-26T05:47:53.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.917+0000 7f9bebfff640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9bd8066470 0x7f9bd8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:53.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.921+0000 7f9bebfff640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9bd8066470 0x7f9bd8068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bec1f5c20 tx=0x7f9bdc0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:53.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:53.921+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9be0050ea0 con 0x7f9bec0b2370 2023-10-26T05:47:54.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.069+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bec0721e0 con 0x7f9bec0b2370 2023-10-26T05:47:54.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.069+0000 7f9be8ff9640 1 -- 172.21.15.71:0/2522636908 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f9be00544c0 con 0x7f9bec0b2370 2023-10-26T05:47:54.075 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:54.075 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9bd8066470 msgr2=0x7f9bd8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9bd8066470 0x7f9bd8068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9bec1f5c20 tx=0x7f9bdc0091e0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 msgr2=0x7f9bec0a9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec0a9370 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c500 tx=0x7f9be000c9c0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 shutdown_connections 2023-10-26T05:47:54.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bec140cc0 0x7f9bec06fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9bd8066470 0x7f9bd8068930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9bec0b2370 0x7f9bec0a9370 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 --2- 172.21.15.71:0/2522636908 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bec070fe0 0x7f9bec1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 >> 172.21.15.71:0/2522636908 conn(0x7f9bec069770 msgr2=0x7f9bec069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 shutdown_connections 2023-10-26T05:47:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.073+0000 7f9bf2338640 1 -- 172.21.15.71:0/2522636908 wait complete. 2023-10-26T05:47:54.093 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:54.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:54.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/42376710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 msgr2=0x7faf740b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 --2- 172.21.15.71:0/42376710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf740b0fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faf64009840 tx=0x7faf6401c790 comp rx=0 tx=0).stop 2023-10-26T05:47:54.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/42376710 shutdown_connections 2023-10-26T05:47:54.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 --2- 172.21.15.71:0/42376710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf740b0fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/42376710 >> 172.21.15.71:0/42376710 conn(0x7faf740aec60 msgr2=0x7faf740af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/42376710 shutdown_connections 2023-10-26T05:47:54.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/42376710 wait complete. 2023-10-26T05:47:54.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 Processor -- start 2023-10-26T05:47:54.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- start start 2023-10-26T05:47:54.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf74070dc0 0x7faf741548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf7413f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf7415bf70 0x7faf740b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74138310 con 0x7faf7415bb90 2023-10-26T05:47:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7415a0d0 con 0x7faf74070dc0 2023-10-26T05:47:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7413f0e0 con 0x7faf7415bf70 2023-10-26T05:47:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf7413f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf7413f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55544/0 (socket says 172.21.15.71:55544) 2023-10-26T05:47:54.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 -- 172.21.15.71:0/3844524657 learned_addr learned my addr 172.21.15.71:0/3844524657 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf794c6640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf7415bf70 0x7faf740b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 -- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf7415bf70 msgr2=0x7faf740b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf7415bf70 0x7faf740b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 -- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf74070dc0 msgr2=0x7faf741548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf78cc5640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf74070dc0 0x7faf741548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf74070dc0 0x7faf741548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 -- 172.21.15.71:0/3844524657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640094a0 con 0x7faf7415bb90 2023-10-26T05:47:54.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf73fff640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf7413f9a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c410 tx=0x7faf6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:54.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68003600 con 0x7faf7415bb90 2023-10-26T05:47:54.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf68003780 con 0x7faf7415bb90 2023-10-26T05:47:54.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7413f3c0 con 0x7faf7415bb90 2023-10-26T05:47:54.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68003b90 con 0x7faf7415bb90 2023-10-26T05:47:54.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.213+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7413fee0 con 0x7faf7415bb90 2023-10-26T05:47:54.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.217+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf74066b10 con 0x7faf7415bb90 2023-10-26T05:47:54.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.217+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf68003cf0 con 0x7faf7415bb90 2023-10-26T05:47:54.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.217+0000 7faf717fa640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf60066470 0x7faf60068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.217+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf68083330 con 0x7faf7415bb90 2023-10-26T05:47:54.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.217+0000 7faf78cc5640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf60066470 0x7faf60068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.217+0000 7faf78cc5640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf60066470 0x7faf60068930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faf640af000 tx=0x7faf6401afd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:54.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.221+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faf68050730 con 0x7faf7415bb90 2023-10-26T05:47:54.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.361+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf7415a210 con 0x7faf7415bb90 2023-10-26T05:47:54.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf717fa640 1 -- 172.21.15.71:0/3844524657 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf68053d50 con 0x7faf7415bb90 2023-10-26T05:47:54.369 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf60066470 msgr2=0x7faf60068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf60066470 0x7faf60068930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faf640af000 tx=0x7faf6401afd0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 msgr2=0x7faf7413f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf7413f9a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c410 tx=0x7faf6800c8d0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 shutdown_connections 2023-10-26T05:47:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf7415bf70 0x7faf740b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf60066470 0x7faf60068930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf7415bb90 0x7faf7413f9a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 --2- 172.21.15.71:0/3844524657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf74070dc0 0x7faf741548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 >> 172.21.15.71:0/3844524657 conn(0x7faf740aec60 msgr2=0x7faf740cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 shutdown_connections 2023-10-26T05:47:54.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.365+0000 7faf7af29640 1 -- 172.21.15.71:0/3844524657 wait complete. 2023-10-26T05:47:54.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:54.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- 172.21.15.71:0/1956223291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 msgr2=0x7ff5b006a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 --2- 172.21.15.71:0/1956223291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 0x7ff5b006a450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009c80 tx=0x7ff5a801cd80 comp rx=0 tx=0).stop 2023-10-26T05:47:54.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- 172.21.15.71:0/1956223291 shutdown_connections 2023-10-26T05:47:54.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 --2- 172.21.15.71:0/1956223291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 0x7ff5b006a450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 --2- 172.21.15.71:0/1956223291 >> v2:172.21.15.86:3300/0 conn(0x7ff5b014e7b0 0x7ff5b0157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- 172.21.15.71:0/1956223291 >> 172.21.15.71:0/1956223291 conn(0x7ff5b00aec60 msgr2=0x7ff5b00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- 172.21.15.71:0/1956223291 shutdown_connections 2023-10-26T05:47:54.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- 172.21.15.71:0/1956223291 wait complete. 2023-10-26T05:47:54.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 Processor -- start 2023-10-26T05:47:54.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- start start 2023-10-26T05:47:54.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 0x7ff5b0136730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b0071270 0x7ff5b0149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 0x7ff5b0145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b014b930 con 0x7ff5b014e7b0 2023-10-26T05:47:54.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b014bc10 con 0x7ff5b0071270 2023-10-26T05:47:54.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5b6090640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b014bef0 con 0x7ff5b006a000 2023-10-26T05:47:54.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5affff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 0x7ff5b0145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5affff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 0x7ff5b0145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55554/0 (socket says 172.21.15.71:55554) 2023-10-26T05:47:54.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5affff640 1 -- 172.21.15.71:0/3848818966 learned_addr learned my addr 172.21.15.71:0/3848818966 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:54.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.497+0000 7ff5af7fe640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 0x7ff5b0136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5aeffd640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b0071270 0x7ff5b0149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5affff640 1 -- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 msgr2=0x7ff5b0136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5affff640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 0x7ff5b0136730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5affff640 1 -- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b0071270 msgr2=0x7ff5b0149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5affff640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b0071270 0x7ff5b0149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5affff640 1 -- 172.21.15.71:0/3848818966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80098e0 con 0x7ff5b014e7b0 2023-10-26T05:47:54.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5affff640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 0x7ff5b0145530 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801a9b0 tx=0x7ff5a80ae6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:54.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80bc070 con 0x7ff5b014e7b0 2023-10-26T05:47:54.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b014c1d0 con 0x7ff5b014e7b0 2023-10-26T05:47:54.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5a8003c80 con 0x7ff5b014e7b0 2023-10-26T05:47:54.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a80aeb00 con 0x7ff5b014e7b0 2023-10-26T05:47:54.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b0136c70 con 0x7ff5b014e7b0 2023-10-26T05:47:54.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b0066b10 con 0x7ff5b014e7b0 2023-10-26T05:47:54.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5a80b4040 con 0x7ff5b014e7b0 2023-10-26T05:47:54.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff597fff640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff59c066470 0x7ff59c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5a811ec20 con 0x7ff5b014e7b0 2023-10-26T05:47:54.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5af7fe640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff59c066470 0x7ff59c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.501+0000 7ff5af7fe640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff59c066470 0x7ff59c068930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009b40 tx=0x7ff5a00091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:54.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.505+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5a80ec020 con 0x7ff5b014e7b0 2023-10-26T05:47:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.645+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5b00721e0 con 0x7ff5b014e7b0 2023-10-26T05:47:54.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff597fff640 1 -- 172.21.15.71:0/3848818966 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5a80ef640 con 0x7ff5b014e7b0 2023-10-26T05:47:54.653 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff59c066470 msgr2=0x7ff59c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff59c066470 0x7ff59c068930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009b40 tx=0x7ff5a00091e0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 msgr2=0x7ff5b0145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 0x7ff5b0145530 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5a801a9b0 tx=0x7ff5a80ae6b0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 shutdown_connections 2023-10-26T05:47:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff59c066470 0x7ff59c068930 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5b014e7b0 0x7ff5b0145530 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b0071270 0x7ff5b0149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 --2- 172.21.15.71:0/3848818966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5b006a000 0x7ff5b0136730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 >> 172.21.15.71:0/3848818966 conn(0x7ff5b00aec60 msgr2=0x7ff5b0157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 shutdown_connections 2023-10-26T05:47:54.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.649+0000 7ff5b6090640 1 -- 172.21.15.71:0/3848818966 wait complete. 2023-10-26T05:47:54.669 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:54.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/3007644646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107015d3f0 msgr2=0x7f10700cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 --2- 172.21.15.71:0/3007644646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107015d3f0 0x7f10700cb820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f10700afff0 tx=0x7f106401c7a0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/3007644646 shutdown_connections 2023-10-26T05:47:54.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 --2- 172.21.15.71:0/3007644646 >> v2:172.21.15.86:3300/0 conn(0x7f10700cbd60 0x7f1070069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 --2- 172.21.15.71:0/3007644646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107015d3f0 0x7f10700cb820 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/3007644646 >> 172.21.15.71:0/3007644646 conn(0x7f10700aec60 msgr2=0x7f10700af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/3007644646 shutdown_connections 2023-10-26T05:47:54.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/3007644646 wait complete. 2023-10-26T05:47:54.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 Processor -- start 2023-10-26T05:47:54.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- start start 2023-10-26T05:47:54.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10700cbd60 0x7f107014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 0x7f1070158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107015d3f0 0x7f1070137cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10701363d0 con 0x7f107013c220 2023-10-26T05:47:54.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070070be0 con 0x7f107015d3f0 2023-10-26T05:47:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1070070ec0 con 0x7f10700cbd60 2023-10-26T05:47:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 0x7f1070158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 0x7f1070158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55576/0 (socket says 172.21.15.71:55576) 2023-10-26T05:47:54.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 -- 172.21.15.71:0/2695875224 learned_addr learned my addr 172.21.15.71:0/2695875224 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:54.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106ed9d640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107015d3f0 0x7f1070137cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 -- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10700cbd60 msgr2=0x7f107014e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106e59c640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10700cbd60 0x7f107014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10700cbd60 0x7f107014e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 -- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107015d3f0 msgr2=0x7f1070137cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107015d3f0 0x7f1070137cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 -- 172.21.15.71:0/2695875224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009560 con 0x7f107013c220 2023-10-26T05:47:54.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106ed9d640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107015d3f0 0x7f1070137cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:54.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106e59c640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10700cbd60 0x7f107014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:54.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f106dd9b640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 0x7f1070158390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f106401a3d0 tx=0x7f10640a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:54.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064003800 con 0x7f107013c220 2023-10-26T05:47:54.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1064003960 con 0x7f107013c220 2023-10-26T05:47:54.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10700711a0 con 0x7f107013c220 2023-10-26T05:47:54.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064003d70 con 0x7f107013c220 2023-10-26T05:47:54.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.789+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1070070820 con 0x7f107013c220 2023-10-26T05:47:54.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.793+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070066b10 con 0x7f107013c220 2023-10-26T05:47:54.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.793+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10640b4040 con 0x7f107013c220 2023-10-26T05:47:54.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.793+0000 7f105effd640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1050066470 0x7f1050068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:54.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.793+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f106411ec90 con 0x7f107013c220 2023-10-26T05:47:54.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.793+0000 7f106e59c640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1050066470 0x7f1050068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:54.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.793+0000 7f106e59c640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1050066470 0x7f1050068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10600016e0 tx=0x7f1060009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:54.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.797+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10640ec150 con 0x7f107013c220 2023-10-26T05:47:54.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10700721e0 con 0x7f107013c220 2023-10-26T05:47:54.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f105effd640 1 -- 172.21.15.71:0/2695875224 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10640ef770 con 0x7f107013c220 2023-10-26T05:47:54.942 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:54.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1050066470 msgr2=0x7f1050068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1050066470 0x7f1050068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10600016e0 tx=0x7f1060009040 comp rx=0 tx=0).stop 2023-10-26T05:47:54.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 msgr2=0x7f1070158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:54.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 0x7f1070158390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f106401a3d0 tx=0x7f10640a56e0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 shutdown_connections 2023-10-26T05:47:54.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f107015d3f0 0x7f1070137cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1050066470 0x7f1050068930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f107013c220 0x7f1070158390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 --2- 172.21.15.71:0/2695875224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10700cbd60 0x7f107014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:54.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 >> 172.21.15.71:0/2695875224 conn(0x7f10700aec60 msgr2=0x7f10700b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:54.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 shutdown_connections 2023-10-26T05:47:54.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:54.937+0000 7f1074bba640 1 -- 172.21.15.71:0/2695875224 wait complete. 2023-10-26T05:47:54.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-26T05:47:55.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/1549150740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c015bbc0 msgr2=0x7fb2c0162840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:55.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/1549150740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c015bbc0 0x7fb2c0162840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00aa10 tx=0x7fb2bc01dbd0 comp rx=0 tx=0).stop 2023-10-26T05:47:55.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/1549150740 shutdown_connections 2023-10-26T05:47:55.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/1549150740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c015bbc0 0x7fb2c0162840 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:55.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/1549150740 >> 172.21.15.71:0/1549150740 conn(0x7fb2c0069770 msgr2=0x7fb2c0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:55.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/1549150740 shutdown_connections 2023-10-26T05:47:55.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/1549150740 wait complete. 2023-10-26T05:47:55.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 Processor -- start 2023-10-26T05:47:55.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- start start 2023-10-26T05:47:55.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 0x7fb2c015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:55.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c013d420 0x7fb2c014db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:55.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2c015bbc0 0x7fb2c0159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:55.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c015a930 con 0x7fb2c00ac740 2023-10-26T05:47:55.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0159cb0 con 0x7fb2c013d420 2023-10-26T05:47:55.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0159f90 con 0x7fb2c015bbc0 2023-10-26T05:47:55.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 0x7fb2c015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:55.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 0x7fb2c015ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55592/0 (socket says 172.21.15.71:55592) 2023-10-26T05:47:55.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 -- 172.21.15.71:0/3730473082 learned_addr learned my addr 172.21.15.71:0/3730473082 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:55.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 -- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2c015bbc0 msgr2=0x7fb2c0159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:55.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2c015bbc0 0x7fb2c0159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:55.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 -- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c013d420 msgr2=0x7fb2c014db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:55.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c013d420 0x7fb2c014db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:55.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 -- 172.21.15.71:0/3730473082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc00a670 con 0x7fb2c00ac740 2023-10-26T05:47:55.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c5f7e640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 0x7fb2c015ad30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c7c0 tx=0x7fb2b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:55.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b800aa50 con 0x7fb2c00ac740 2023-10-26T05:47:55.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2b800aca0 con 0x7fb2c00ac740 2023-10-26T05:47:55.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c015a270 con 0x7fb2c00ac740 2023-10-26T05:47:55.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8012680 con 0x7fb2c00ac740 2023-10-26T05:47:55.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c00abca0 con 0x7fb2c00ac740 2023-10-26T05:47:55.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.073+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c0066b10 con 0x7fb2c00ac740 2023-10-26T05:47:55.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.077+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2b801b400 con 0x7fb2c00ac740 2023-10-26T05:47:55.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.077+0000 7fb2b67fc640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2a4066470 0x7fb2a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:55.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.077+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2b80846a0 con 0x7fb2c00ac740 2023-10-26T05:47:55.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.077+0000 7fb2c577d640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2a4066470 0x7fb2a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:55.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.077+0000 7fb2c577d640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2a4066470 0x7fb2a4068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0007fd0 tx=0x7fb2b0007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:55.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.081+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2b8051b20 con 0x7fb2c00ac740 2023-10-26T05:47:55.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:55.273+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fb2c013bc40 con 0x7fb2c00ac740 2023-10-26T05:47:56.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.273+0000 7fb2b67fc640 1 -- 172.21.15.71:0/3730473082 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb2b8055140 con 0x7fb2c00ac740 2023-10-26T05:47:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2a4066470 msgr2=0x7fb2a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2a4066470 0x7fb2a4068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0007fd0 tx=0x7fb2b0007920 comp rx=0 tx=0).stop 2023-10-26T05:47:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 msgr2=0x7fb2c015ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 0x7fb2c015ad30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800c7c0 tx=0x7fb2b800cc80 comp rx=0 tx=0).stop 2023-10-26T05:47:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 shutdown_connections 2023-10-26T05:47:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2c015bbc0 0x7fb2c0159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c013d420 0x7fb2c014db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2a4066470 0x7fb2a4068930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 --2- 172.21.15.71:0/3730473082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2c00ac740 0x7fb2c015ad30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 >> 172.21.15.71:0/3730473082 conn(0x7fb2c0069770 msgr2=0x7fb2c015b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:56.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 shutdown_connections 2023-10-26T05:47:56.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.277+0000 7fb2c81e2640 1 -- 172.21.15.71:0/3730473082 wait complete. 2023-10-26T05:47:56.305 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T05:47:56.441 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_objectstore":"memstore"} 2023-10-26T05:47:56.441 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T05:47:56.442 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:47:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/3193169835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 msgr2=0x7fcde4071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/3193169835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde4071470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009900 tx=0x7fcdd801c5d0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/3193169835 shutdown_connections 2023-10-26T05:47:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/3193169835 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde40b1360 0x7fcde40b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/3193169835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde4071470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/3193169835 >> 172.21.15.71:0/3193169835 conn(0x7fcde4069770 msgr2=0x7fcde4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/3193169835 shutdown_connections 2023-10-26T05:47:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.561+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/3193169835 wait complete. 2023-10-26T05:47:56.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 Processor -- start 2023-10-26T05:47:56.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- start start 2023-10-26T05:47:56.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde41651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde40b1360 0x7fcde4138580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde4140cc0 0x7fcde4156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40ab2c0 con 0x7fcde4070fe0 2023-10-26T05:47:56.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4165710 con 0x7fcde40b1360 2023-10-26T05:47:56.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40ab540 con 0x7fcde4140cc0 2023-10-26T05:47:56.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde41651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde41651d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55624/0 (socket says 172.21.15.71:55624) 2023-10-26T05:47:56.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 -- 172.21.15.71:0/1475273047 learned_addr learned my addr 172.21.15.71:0/1475273047 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:56.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9844640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde4140cc0 0x7fcde4156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde8842640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde40b1360 0x7fcde4138580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 -- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde4140cc0 msgr2=0x7fcde4156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde4140cc0 0x7fcde4156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 -- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde40b1360 msgr2=0x7fcde4138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde40b1360 0x7fcde4138580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 -- 172.21.15.71:0/1475273047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8009560 con 0x7fcde4070fe0 2023-10-26T05:47:56.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9844640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde4140cc0 0x7fcde4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:56.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde8842640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde40b1360 0x7fcde4138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:56.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde9043640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde41651d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c7a0 tx=0x7fcdd400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:56.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4003db0 con 0x7fcde4070fe0 2023-10-26T05:47:56.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdd4005070 con 0x7fcde4070fe0 2023-10-26T05:47:56.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde40ab820 con 0x7fcde4070fe0 2023-10-26T05:47:56.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4005480 con 0x7fcde4070fe0 2023-10-26T05:47:56.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40abde0 con 0x7fcde4070fe0 2023-10-26T05:47:56.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde4066b10 con 0x7fcde4070fe0 2023-10-26T05:47:56.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcdd40055e0 con 0x7fcde4070fe0 2023-10-26T05:47:56.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde1ffb640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdd0066470 0x7fcdd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.565+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdd4083940 con 0x7fcde4070fe0 2023-10-26T05:47:56.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.569+0000 7fcde8842640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdd0066470 0x7fcdd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.569+0000 7fcde8842640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdd0066470 0x7fcdd0068930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80ae000 tx=0x7fcdd801ae10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:56.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.573+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcdd4050e50 con 0x7fcde4070fe0 2023-10-26T05:47:56.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.725+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fcde40721e0 con 0x7fcde4070fe0 2023-10-26T05:47:56.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.725+0000 7fcde1ffb640 1 -- 172.21.15.71:0/1475273047 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v18) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fcdd4017070 con 0x7fcde4070fe0 2023-10-26T05:47:56.732 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:47:56.732 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:47:56.733 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:47:56.733 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:47:56.733 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:47:56.734 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:47:56.734 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:47:56.735 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:47:56.735 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:47:56.735 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:47:56.735 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:47:56.736 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:47:56.736 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:47:56.736 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:47:56.736 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:47:56.737 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:47:56.737 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:47:56.737 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:47:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdd0066470 msgr2=0x7fcdd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdd0066470 0x7fcdd0068930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80ae000 tx=0x7fcdd801ae10 comp rx=0 tx=0).stop 2023-10-26T05:47:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 msgr2=0x7fcde41651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde41651d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400c7a0 tx=0x7fcdd400cc60 comp rx=0 tx=0).stop 2023-10-26T05:47:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 shutdown_connections 2023-10-26T05:47:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcde4140cc0 0x7fcde4156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcde40b1360 0x7fcde4138580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdd0066470 0x7fcdd0068930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 --2- 172.21.15.71:0/1475273047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcde4070fe0 0x7fcde41651d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 >> 172.21.15.71:0/1475273047 conn(0x7fcde4069770 msgr2=0x7fcde40b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:56.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 shutdown_connections 2023-10-26T05:47:56.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.729+0000 7fcdeb2a7640 1 -- 172.21.15.71:0/1475273047 wait complete. 2023-10-26T05:47:56.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:47:56.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- 172.21.15.71:0/3629805217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 msgr2=0x7f0f7406a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 --2- 172.21.15.71:0/3629805217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f7406a090 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009bc0 tx=0x7f0f6c01cb50 comp rx=0 tx=0).stop 2023-10-26T05:47:56.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- 172.21.15.71:0/3629805217 shutdown_connections 2023-10-26T05:47:56.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 --2- 172.21.15.71:0/3629805217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f7406a090 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 --2- 172.21.15.71:0/3629805217 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f74070de0 0x7f0f74069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- 172.21.15.71:0/3629805217 >> 172.21.15.71:0/3629805217 conn(0x7f0f740aec60 msgr2=0x7f0f740af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:56.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- 172.21.15.71:0/3629805217 shutdown_connections 2023-10-26T05:47:56.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- 172.21.15.71:0/3629805217 wait complete. 2023-10-26T05:47:56.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 Processor -- start 2023-10-26T05:47:56.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- start start 2023-10-26T05:47:56.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f74069770 0x7f0f7413f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f74070de0 0x7f0f74146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f741f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7413ff10 con 0x7f0f740714c0 2023-10-26T05:47:56.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74145520 con 0x7f0f74070de0 2023-10-26T05:47:56.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f7b738640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f741457c0 con 0x7f0f74069770 2023-10-26T05:47:56.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f741f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f741f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55642/0 (socket says 172.21.15.71:55642) 2023-10-26T05:47:56.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 -- 172.21.15.71:0/2592571643 learned_addr learned my addr 172.21.15.71:0/2592571643 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:56.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f794d4640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f74069770 0x7f0f7413f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f78cd3640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f74070de0 0x7f0f74146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 -- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f74069770 msgr2=0x7f0f7413f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f74069770 0x7f0f7413f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 -- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f74070de0 msgr2=0x7f0f74146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:56.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f74070de0 0x7f0f74146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:56.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 -- 172.21.15.71:0/2592571643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c009820 con 0x7f0f740714c0 2023-10-26T05:47:56.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f78cd3640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f74070de0 0x7f0f74146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:56.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.877+0000 7f0f79cd5640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f741f3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009b10 tx=0x7f0f6c0a5aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:56.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c003b80 con 0x7f0f740714c0 2023-10-26T05:47:56.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f6c003ce0 con 0x7f0f740714c0 2023-10-26T05:47:56.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f74145aa0 con 0x7f0f740714c0 2023-10-26T05:47:56.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c004290 con 0x7f0f740714c0 2023-10-26T05:47:56.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f74146060 con 0x7f0f740714c0 2023-10-26T05:47:56.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f74066b10 con 0x7f0f740714c0 2023-10-26T05:47:56.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f6c0b4040 con 0x7f0f740714c0 2023-10-26T05:47:56.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f69ffb640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f580664a0 0x7f0f58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:56.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f6c11f620 con 0x7f0f740714c0 2023-10-26T05:47:56.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f794d4640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f580664a0 0x7f0f58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:56.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.881+0000 7f0f794d4640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f580664a0 0x7f0f58068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f70001650 tx=0x7f0f70009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:56.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:56.885+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f6c0ecaf0 con 0x7f0f740714c0 2023-10-26T05:47:57.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.029+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0f740721e0 con 0x7f0f740714c0 2023-10-26T05:47:57.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f69ffb640 1 -- 172.21.15.71:0/2592571643 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0f6c0b2070 con 0x7f0f740714c0 2023-10-26T05:47:57.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f580664a0 msgr2=0x7f0f58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f580664a0 0x7f0f58068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f70001650 tx=0x7f0f70009040 comp rx=0 tx=0).stop 2023-10-26T05:47:57.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 msgr2=0x7f0f741f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f741f3680 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009b10 tx=0x7f0f6c0a5aa0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 shutdown_connections 2023-10-26T05:47:57.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f580664a0 0x7f0f58068960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f740714c0 0x7f0f741f3680 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f74070de0 0x7f0f74146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 --2- 172.21.15.71:0/2592571643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f74069770 0x7f0f7413f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 >> 172.21.15.71:0/2592571643 conn(0x7f0f740aec60 msgr2=0x7f0f740af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 shutdown_connections 2023-10-26T05:47:57.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.037+0000 7f0f7b738640 1 -- 172.21.15.71:0/2592571643 wait complete. 2023-10-26T05:47:57.059 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:47:57.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:47:57.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:47:57.155 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:47:57.156 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:57.156 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:47:57.162 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:57.162 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:47:57.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:57.254 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:47:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:47:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 -- 172.21.15.71:0/2778637298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c0b2370 msgr2=0x7f878c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 --2- 172.21.15.71:0/2778637298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c0b2370 0x7f878c071470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8784009c80 tx=0x7f878401cd50 comp rx=0 tx=0).stop 2023-10-26T05:47:57.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 -- 172.21.15.71:0/2778637298 shutdown_connections 2023-10-26T05:47:57.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 --2- 172.21.15.71:0/2778637298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c0b2370 0x7f878c071470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 --2- 172.21.15.71:0/2778637298 >> v2:172.21.15.86:3300/0 conn(0x7f878c140cc0 0x7f878c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 -- 172.21.15.71:0/2778637298 >> 172.21.15.71:0/2778637298 conn(0x7f878c069770 msgr2=0x7f878c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 -- 172.21.15.71:0/2778637298 shutdown_connections 2023-10-26T05:47:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 -- 172.21.15.71:0/2778637298 wait complete. 2023-10-26T05:47:57.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 Processor -- start 2023-10-26T05:47:57.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.381+0000 7f87927d7640 1 -- start start 2023-10-26T05:47:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c070fe0 0x7f878c156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0b2370 0x7f878c136140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 0x7f878c136680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c156880 con 0x7f878c140cc0 2023-10-26T05:47:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878c136c70 con 0x7f878c0b2370 2023-10-26T05:47:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878c136f50 con 0x7f878c070fe0 2023-10-26T05:47:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 0x7f878c136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 0x7f878c136680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55644/0 (socket says 172.21.15.71:55644) 2023-10-26T05:47:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 -- 172.21.15.71:0/1040126673 learned_addr learned my addr 172.21.15.71:0/1040126673 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f878bfff640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c070fe0 0x7f878c156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f878b7fe640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0b2370 0x7f878c136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 -- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c070fe0 msgr2=0x7f878c156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c070fe0 0x7f878c156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 -- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0b2370 msgr2=0x7f878c136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0b2370 0x7f878c136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 -- 172.21.15.71:0/1040126673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840098e0 con 0x7f878c140cc0 2023-10-26T05:47:57.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8790d74640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 0x7f878c136680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8784009bd0 tx=0x7f8784003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:57.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784004c50 con 0x7f878c140cc0 2023-10-26T05:47:57.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8784004000 con 0x7f878c140cc0 2023-10-26T05:47:57.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878c137230 con 0x7f878c140cc0 2023-10-26T05:47:57.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784004ea0 con 0x7f878c140cc0 2023-10-26T05:47:57.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878c156b30 con 0x7f878c140cc0 2023-10-26T05:47:57.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878c066b10 con 0x7f878c140cc0 2023-10-26T05:47:57.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87840b4040 con 0x7f878c140cc0 2023-10-26T05:47:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8788ff9640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8778066470 0x7f8778068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f878411e1d0 con 0x7f878c140cc0 2023-10-26T05:47:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.385+0000 7f878bfff640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8778066470 0x7f8778068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.389+0000 7f878bfff640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8778066470 0x7f8778068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f877c009ad0 tx=0x7f877c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.389+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f87840eb630 con 0x7f878c140cc0 2023-10-26T05:47:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.545+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f878c0aca10 con 0x7f878c140cc0 2023-10-26T05:47:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f8788ff9640 1 -- 172.21.15.71:0/1040126673 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f878c0aca10 con 0x7f878c140cc0 2023-10-26T05:47:57.553 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:57.553 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8778066470 msgr2=0x7f8778068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8778066470 0x7f8778068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f877c009ad0 tx=0x7f877c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 msgr2=0x7f878c136680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 0x7f878c136680 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8784009bd0 tx=0x7f8784003f80 comp rx=0 tx=0).stop 2023-10-26T05:47:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 shutdown_connections 2023-10-26T05:47:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8778066470 0x7f8778068930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878c140cc0 0x7f878c136680 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f878c0b2370 0x7f878c136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 --2- 172.21.15.71:0/1040126673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f878c070fe0 0x7f878c156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 >> 172.21.15.71:0/1040126673 conn(0x7f878c069770 msgr2=0x7f878c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:57.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 shutdown_connections 2023-10-26T05:47:57.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.549+0000 7f87927d7640 1 -- 172.21.15.71:0/1040126673 wait complete. 2023-10-26T05:47:57.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:57.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:57.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- 172.21.15.71:0/4133128722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3415b280 msgr2=0x7f5e34070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 --2- 172.21.15.71:0/4133128722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3415b280 0x7f5e34070430 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5e3406f680 tx=0x7f5e1c01cc40 comp rx=0 tx=0).stop 2023-10-26T05:47:57.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- 172.21.15.71:0/4133128722 shutdown_connections 2023-10-26T05:47:57.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 --2- 172.21.15.71:0/4133128722 >> v2:172.21.15.86:3300/0 conn(0x7f5e340ab080 0x7f5e34070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 --2- 172.21.15.71:0/4133128722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3415b280 0x7f5e34070430 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- 172.21.15.71:0/4133128722 >> 172.21.15.71:0/4133128722 conn(0x7f5e340aec60 msgr2=0x7f5e340af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:57.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- 172.21.15.71:0/4133128722 shutdown_connections 2023-10-26T05:47:57.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- 172.21.15.71:0/4133128722 wait complete. 2023-10-26T05:47:57.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 Processor -- start 2023-10-26T05:47:57.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- start start 2023-10-26T05:47:57.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e340ab080 0x7f5e3414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 0x7f5e3415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3415b280 0x7f5e34149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3414a980 con 0x7f5e340b2d00 2023-10-26T05:47:57.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34156b80 con 0x7f5e3415b280 2023-10-26T05:47:57.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e39977640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34156e60 con 0x7f5e340ab080 2023-10-26T05:47:57.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 0x7f5e3415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 0x7f5e3415c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55664/0 (socket says 172.21.15.71:55664) 2023-10-26T05:47:57.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 -- 172.21.15.71:0/2848156212 learned_addr learned my addr 172.21.15.71:0/2848156212 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:57.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 -- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e340ab080 msgr2=0x7f5e3414abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e340ab080 0x7f5e3414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 -- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3415b280 msgr2=0x7f5e34149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3415b280 0x7f5e34149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 -- 172.21.15.71:0/2848156212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c0098e0 con 0x7f5e340b2d00 2023-10-26T05:47:57.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.689+0000 7f5e327fc640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 0x7f5e3415c910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5e24005d10 tx=0x7f5e2400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:57.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24009c30 con 0x7f5e340b2d00 2023-10-26T05:47:57.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e24009e80 con 0x7f5e340b2d00 2023-10-26T05:47:57.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e34157140 con 0x7f5e340b2d00 2023-10-26T05:47:57.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24005280 con 0x7f5e340b2d00 2023-10-26T05:47:57.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34151d00 con 0x7f5e340b2d00 2023-10-26T05:47:57.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e34066b10 con 0x7f5e340b2d00 2023-10-26T05:47:57.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5e240053e0 con 0x7f5e340b2d00 2023-10-26T05:47:57.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e2bfff640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e18066470 0x7f5e18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e24084090 con 0x7f5e340b2d00 2023-10-26T05:47:57.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e32ffd640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e18066470 0x7f5e18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.693+0000 7f5e32ffd640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e18066470 0x7f5e18068930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0af000 tx=0x7f5e1c0199e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:57.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.697+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5e24051510 con 0x7f5e340b2d00 2023-10-26T05:47:57.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.837+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e340721e0 con 0x7f5e340b2d00 2023-10-26T05:47:57.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.837+0000 7f5e2bfff640 1 -- 172.21.15.71:0/2848156212 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e24017020 con 0x7f5e340b2d00 2023-10-26T05:47:57.843 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:57.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e18066470 msgr2=0x7f5e18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e18066470 0x7f5e18068930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0af000 tx=0x7f5e1c0199e0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 msgr2=0x7f5e3415c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 0x7f5e3415c910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5e24005d10 tx=0x7f5e2400dde0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 shutdown_connections 2023-10-26T05:47:57.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e3415b280 0x7f5e34149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e18066470 0x7f5e18068930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e340b2d00 0x7f5e3415c910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 --2- 172.21.15.71:0/2848156212 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e340ab080 0x7f5e3414abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 >> 172.21.15.71:0/2848156212 conn(0x7f5e340aec60 msgr2=0x7f5e340cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:57.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 shutdown_connections 2023-10-26T05:47:57.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.841+0000 7f5e39977640 1 -- 172.21.15.71:0/2848156212 wait complete. 2023-10-26T05:47:57.860 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:57.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- 172.21.15.71:0/2798672755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0850 msgr2=0x7f850415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 --2- 172.21.15.71:0/2798672755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0850 0x7f850415b3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009900 tx=0x7f84f801c830 comp rx=0 tx=0).stop 2023-10-26T05:47:57.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- 172.21.15.71:0/2798672755 shutdown_connections 2023-10-26T05:47:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 --2- 172.21.15.71:0/2798672755 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85040cd880 0x7f850415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 --2- 172.21.15.71:0/2798672755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0850 0x7f850415b3a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- 172.21.15.71:0/2798672755 >> 172.21.15.71:0/2798672755 conn(0x7f85040cdd80 msgr2=0x7f85040ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- 172.21.15.71:0/2798672755 shutdown_connections 2023-10-26T05:47:57.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- 172.21.15.71:0/2798672755 wait complete. 2023-10-26T05:47:57.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 Processor -- start 2023-10-26T05:47:57.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- start start 2023-10-26T05:47:57.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 0x7f850414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0470 0x7f850414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85040d0850 0x7f85040cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850415d090 con 0x7f85040cd880 2023-10-26T05:47:57.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85041575a0 con 0x7f85040d0470 2023-10-26T05:47:57.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f85088d3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8504156990 con 0x7f85040d0850 2023-10-26T05:47:57.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.981+0000 7f850259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 0x7f850414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 0x7f850414f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55680/0 (socket says 172.21.15.71:55680) 2023-10-26T05:47:57.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 -- 172.21.15.71:0/3062647680 learned_addr learned my addr 172.21.15.71:0/3062647680 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:57.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f8502d9d640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85040d0850 0x7f85040cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 -- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85040d0850 msgr2=0x7f85040cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85040d0850 0x7f85040cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 -- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0470 msgr2=0x7f850414d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:57.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f8501d9b640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0470 0x7f850414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0470 0x7f850414d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:57.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 -- 172.21.15.71:0/3062647680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009560 con 0x7f85040cd880 2023-10-26T05:47:57.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f850259c640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 0x7f850414f180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c480 tx=0x7f84f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:57.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40052d0 con 0x7f85040cd880 2023-10-26T05:47:57.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84f4005450 con 0x7f85040cd880 2023-10-26T05:47:57.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8504156c70 con 0x7f85040cd880 2023-10-26T05:47:57.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4012890 con 0x7f85040cd880 2023-10-26T05:47:57.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8504157230 con 0x7f85040cd880 2023-10-26T05:47:57.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8504066b10 con 0x7f85040cd880 2023-10-26T05:47:57.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f84f4003620 con 0x7f85040cd880 2023-10-26T05:47:57.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f84f2ffd640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84e4066470 0x7f84e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:57.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84f4083b40 con 0x7f85040cd880 2023-10-26T05:47:57.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f8501d9b640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84e4066470 0x7f84e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:57.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.985+0000 7f8501d9b640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84e4066470 0x7f84e4068930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84f80af000 tx=0x7f84f801b070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:57.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:57.989+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f84f4051050 con 0x7f85040cd880 2023-10-26T05:47:58.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.129+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85040b5130 con 0x7f85040cd880 2023-10-26T05:47:58.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.129+0000 7f84f2ffd640 1 -- 172.21.15.71:0/3062647680 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84f4054670 con 0x7f85040cd880 2023-10-26T05:47:58.136 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:58.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84e4066470 msgr2=0x7f84e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84e4066470 0x7f84e4068930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84f80af000 tx=0x7f84f801b070 comp rx=0 tx=0).stop 2023-10-26T05:47:58.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 msgr2=0x7f850414f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 0x7f850414f180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c480 tx=0x7f84f400c940 comp rx=0 tx=0).stop 2023-10-26T05:47:58.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 shutdown_connections 2023-10-26T05:47:58.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85040d0850 0x7f85040cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85040d0470 0x7f850414d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84e4066470 0x7f84e4068930 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 --2- 172.21.15.71:0/3062647680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85040cd880 0x7f850414f180 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 >> 172.21.15.71:0/3062647680 conn(0x7f85040cdd80 msgr2=0x7f85040b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 shutdown_connections 2023-10-26T05:47:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.133+0000 7f85088d3640 1 -- 172.21.15.71:0/3062647680 wait complete. 2023-10-26T05:47:58.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:58.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/57159907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580b2390 msgr2=0x7faf5806ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/57159907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580b2390 0x7faf5806ff70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faf5806e990 tx=0x7faf4c01c5b0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/57159907 shutdown_connections 2023-10-26T05:47:58.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/57159907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf580704b0 0x7faf580aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/57159907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580b2390 0x7faf5806ff70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/57159907 >> v2:172.21.15.86:3300/0 conn(0x7faf58141980 0x7faf580b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/57159907 >> 172.21.15.71:0/57159907 conn(0x7faf580aec60 msgr2=0x7faf580af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:58.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/57159907 shutdown_connections 2023-10-26T05:47:58.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/57159907 wait complete. 2023-10-26T05:47:58.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 Processor -- start 2023-10-26T05:47:58.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- start start 2023-10-26T05:47:58.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 0x7faf581f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf580b2390 0x7faf5813d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf58141980 0x7faf58160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf581582a0 con 0x7faf580704b0 2023-10-26T05:47:58.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf58163cf0 con 0x7faf580b2390 2023-10-26T05:47:58.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf58163fd0 con 0x7faf58141980 2023-10-26T05:47:58.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 0x7faf581f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 0x7faf581f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55694/0 (socket says 172.21.15.71:55694) 2023-10-26T05:47:58.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 -- 172.21.15.71:0/1455117824 learned_addr learned my addr 172.21.15.71:0/1455117824 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:58.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf56d9d640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf58141980 0x7faf58160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf55d9b640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf580b2390 0x7faf5813d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 -- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf58141980 msgr2=0x7faf58160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf58141980 0x7faf58160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 -- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf580b2390 msgr2=0x7faf5813d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf580b2390 0x7faf5813d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 -- 172.21.15.71:0/1455117824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c009560 con 0x7faf580704b0 2023-10-26T05:47:58.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf56d9d640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf58141980 0x7faf58160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:58.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5659c640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 0x7faf581f7430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faf480058d0 tx=0x7faf4800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:58.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf48009c30 con 0x7faf580704b0 2023-10-26T05:47:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf48009e80 con 0x7faf580704b0 2023-10-26T05:47:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf581642b0 con 0x7faf580704b0 2023-10-26T05:47:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480106a0 con 0x7faf580704b0 2023-10-26T05:47:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf58164870 con 0x7faf580704b0 2023-10-26T05:47:58.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.273+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf58066b10 con 0x7faf580704b0 2023-10-26T05:47:58.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.277+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf48010800 con 0x7faf580704b0 2023-10-26T05:47:58.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.277+0000 7faf46ffd640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf38066470 0x7faf38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.277+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf48084410 con 0x7faf580704b0 2023-10-26T05:47:58.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.277+0000 7faf55d9b640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf38066470 0x7faf38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.277+0000 7faf55d9b640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf38066470 0x7faf38068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0ae000 tx=0x7faf4c01adf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:58.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.281+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faf48051810 con 0x7faf580704b0 2023-10-26T05:47:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf580721e0 con 0x7faf580704b0 2023-10-26T05:47:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf46ffd640 1 -- 172.21.15.71:0/1455117824 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf48054e30 con 0x7faf580704b0 2023-10-26T05:47:58.441 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:47:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf38066470 msgr2=0x7faf38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf38066470 0x7faf38068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0ae000 tx=0x7faf4c01adf0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 msgr2=0x7faf581f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 0x7faf581f7430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faf480058d0 tx=0x7faf4800dde0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 shutdown_connections 2023-10-26T05:47:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf58141980 0x7faf58160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf580b2390 0x7faf5813d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf38066470 0x7faf38068930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 --2- 172.21.15.71:0/1455117824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf580704b0 0x7faf581f7430 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 >> 172.21.15.71:0/1455117824 conn(0x7faf580aec60 msgr2=0x7faf580af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 shutdown_connections 2023-10-26T05:47:58.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.437+0000 7faf5cbbf640 1 -- 172.21.15.71:0/1455117824 wait complete. 2023-10-26T05:47:58.459 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:47:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 -- 172.21.15.71:0/2115334310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813ae90 msgr2=0x7fb1a80cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 --2- 172.21.15.71:0/2115334310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813ae90 0x7fb1a80cb8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80cad90 tx=0x7fb19801c720 comp rx=0 tx=0).stop 2023-10-26T05:47:58.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 -- 172.21.15.71:0/2115334310 shutdown_connections 2023-10-26T05:47:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 --2- 172.21.15.71:0/2115334310 >> v2:172.21.15.86:3300/0 conn(0x7fb1a8159060 0x7fb1a813d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 --2- 172.21.15.71:0/2115334310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813ae90 0x7fb1a80cb8c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 -- 172.21.15.71:0/2115334310 >> 172.21.15.71:0/2115334310 conn(0x7fb1a806b8c0 msgr2=0x7fb1a806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:58.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 -- 172.21.15.71:0/2115334310 shutdown_connections 2023-10-26T05:47:58.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.581+0000 7fb1ae223640 1 -- 172.21.15.71:0/2115334310 wait complete. 2023-10-26T05:47:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 Processor -- start 2023-10-26T05:47:58.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- start start 2023-10-26T05:47:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a813ae90 0x7fb1a814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 0x7fb1a80aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1a8159060 0x7fb1a80aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a80adf50 con 0x7fb1a813dcb0 2023-10-26T05:47:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a80ab410 con 0x7fb1a813ae90 2023-10-26T05:47:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a80ab6f0 con 0x7fb1a8159060 2023-10-26T05:47:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 0x7fb1a80aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 0x7fb1a80aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55712/0 (socket says 172.21.15.71:55712) 2023-10-26T05:47:58.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 -- 172.21.15.71:0/1269038992 learned_addr learned my addr 172.21.15.71:0/1269038992 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a7fff640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1a8159060 0x7fb1a80aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a77fe640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a813ae90 0x7fb1a814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 -- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1a8159060 msgr2=0x7fb1a80aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1a8159060 0x7fb1a80aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 -- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a813ae90 msgr2=0x7fb1a814ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a813ae90 0x7fb1a814ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 -- 172.21.15.71:0/1269038992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980094a0 con 0x7fb1a813dcb0 2023-10-26T05:47:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a7fff640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1a8159060 0x7fb1a80aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:58.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb19effd640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 0x7fb1a80aa8e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb19000c970 tx=0x7fb19000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb190007c70 con 0x7fb1a813dcb0 2023-10-26T05:47:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb19000f450 con 0x7fb1a813dcb0 2023-10-26T05:47:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a80ab9d0 con 0x7fb1a813dcb0 2023-10-26T05:47:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19000f860 con 0x7fb1a813dcb0 2023-10-26T05:47:58.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a80abf90 con 0x7fb1a813dcb0 2023-10-26T05:47:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a8066b10 con 0x7fb1a813dcb0 2023-10-26T05:47:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb19000f9c0 con 0x7fb1a813dcb0 2023-10-26T05:47:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a4ff9640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb188066470 0x7fb188068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.585+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb190084730 con 0x7fb1a813dcb0 2023-10-26T05:47:58.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.589+0000 7fb1a77fe640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb188066470 0x7fb188068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.589+0000 7fb1a77fe640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb188066470 0x7fb188068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1980af000 tx=0x7fb19801af60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:58.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.589+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb190051bb0 con 0x7fb1a813dcb0 2023-10-26T05:47:58.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb1a806c7d0 con 0x7fb1a813dcb0 2023-10-26T05:47:58.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1a4ff9640 1 -- 172.21.15.71:0/1269038992 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb1900551d0 con 0x7fb1a813dcb0 2023-10-26T05:47:58.761 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:47:58.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb188066470 msgr2=0x7fb188068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb188066470 0x7fb188068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1980af000 tx=0x7fb19801af60 comp rx=0 tx=0).stop 2023-10-26T05:47:58.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 msgr2=0x7fb1a80aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 0x7fb1a80aa8e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb19000c970 tx=0x7fb19000ce30 comp rx=0 tx=0).stop 2023-10-26T05:47:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 shutdown_connections 2023-10-26T05:47:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1a8159060 0x7fb1a80aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb188066470 0x7fb188068930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1a813dcb0 0x7fb1a80aa8e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 --2- 172.21.15.71:0/1269038992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1a813ae90 0x7fb1a814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 >> 172.21.15.71:0/1269038992 conn(0x7fb1a806b8c0 msgr2=0x7fb1a8156ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:58.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 shutdown_connections 2023-10-26T05:47:58.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.757+0000 7fb1ae223640 1 -- 172.21.15.71:0/1269038992 wait complete. 2023-10-26T05:47:58.779 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:47:58.780 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:47:58.786 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:47:58.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- 172.21.15.71:0/2863019351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0cd880 msgr2=0x7f27dc15bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 --2- 172.21.15.71:0/2863019351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0cd880 0x7f27dc15bab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009bc0 tx=0x7f27d401cac0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- 172.21.15.71:0/2863019351 shutdown_connections 2023-10-26T05:47:58.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 --2- 172.21.15.71:0/2863019351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0cd880 0x7f27dc15bab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- 172.21.15.71:0/2863019351 >> 172.21.15.71:0/2863019351 conn(0x7f27dc0cdd80 msgr2=0x7f27dc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- 172.21.15.71:0/2863019351 shutdown_connections 2023-10-26T05:47:58.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- 172.21.15.71:0/2863019351 wait complete. 2023-10-26T05:47:58.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 Processor -- start 2023-10-26T05:47:58.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- start start 2023-10-26T05:47:58.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27dc0cd880 0x7f27dc0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27dc0d0470 0x7f27dc150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 0x7f27dc151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc06a650 con 0x7f27dc0d0850 2023-10-26T05:47:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc15cf70 con 0x7f27dc0d0470 2023-10-26T05:47:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27e172b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0c5580 con 0x7f27dc0cd880 2023-10-26T05:47:58.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 0x7f27dc151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 0x7f27dc151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55740/0 (socket says 172.21.15.71:55740) 2023-10-26T05:47:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 -- 172.21.15.71:0/3450827354 learned_addr learned my addr 172.21.15.71:0/3450827354 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27daffd640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27dc0cd880 0x7f27dc0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 -- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27dc0cd880 msgr2=0x7f27dc0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27dc0cd880 0x7f27dc0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 -- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27dc0d0470 msgr2=0x7f27dc150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27da7fc640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27dc0d0470 0x7f27dc150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27dc0d0470 0x7f27dc150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 -- 172.21.15.71:0/3450827354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d4009820 con 0x7f27dc0d0850 2023-10-26T05:47:58.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27daffd640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27dc0cd880 0x7f27dc0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27db7fe640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 0x7f27dc151360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f27d401c530 tx=0x7f27d40a5990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4003b80 con 0x7f27dc0d0850 2023-10-26T05:47:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27d40a5b80 con 0x7f27dc0d0850 2023-10-26T05:47:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.953+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4004030 con 0x7f27dc0d0850 2023-10-26T05:47:58.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc0c5860 con 0x7f27dc0d0850 2023-10-26T05:47:58.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc15cc00 con 0x7f27dc0d0850 2023-10-26T05:47:58.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc066b10 con 0x7f27dc0d0850 2023-10-26T05:47:58.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27d4004190 con 0x7f27dc0d0850 2023-10-26T05:47:58.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27c37fe640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f27c80664a0 0x7f27c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:58.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27d411eff0 con 0x7f27dc0d0850 2023-10-26T05:47:58.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27daffd640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f27c80664a0 0x7f27c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:58.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.957+0000 7f27daffd640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f27c80664a0 0x7f27c8068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0016f0 tx=0x7f27cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:58.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:58.961+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f27d40ec590 con 0x7f27dc0d0850 2023-10-26T05:47:59.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.117+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27dc0c5d80 con 0x7f27dc0d0850 2023-10-26T05:47:59.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.117+0000 7f27c37fe640 1 -- 172.21.15.71:0/3450827354 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f27dc0c5d80 con 0x7f27dc0d0850 2023-10-26T05:47:59.123 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:47:59.124 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:47:59.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f27c80664a0 msgr2=0x7f27c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f27c80664a0 0x7f27c8068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0016f0 tx=0x7f27cc009040 comp rx=0 tx=0).stop 2023-10-26T05:47:59.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 msgr2=0x7f27dc151360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 0x7f27dc151360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f27d401c530 tx=0x7f27d40a5990 comp rx=0 tx=0).stop 2023-10-26T05:47:59.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 shutdown_connections 2023-10-26T05:47:59.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f27c80664a0 0x7f27c8068960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27dc0d0850 0x7f27dc151360 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27dc0d0470 0x7f27dc150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 --2- 172.21.15.71:0/3450827354 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27dc0cd880 0x7f27dc0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 >> 172.21.15.71:0/3450827354 conn(0x7f27dc0cdd80 msgr2=0x7f27dc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:59.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 shutdown_connections 2023-10-26T05:47:59.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.121+0000 7f27e172b640 1 -- 172.21.15.71:0/3450827354 wait complete. 2023-10-26T05:47:59.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:47:59.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:47:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- 172.21.15.71:0/1512626653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 msgr2=0x7f6d5400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 --2- 172.21.15.71:0/1512626653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d5400b990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6d64001da0 tx=0x7f6d6401cb20 comp rx=0 tx=0).stop 2023-10-26T05:47:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- 172.21.15.71:0/1512626653 shutdown_connections 2023-10-26T05:47:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 --2- 172.21.15.71:0/1512626653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d5400b990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- 172.21.15.71:0/1512626653 >> 172.21.15.71:0/1512626653 conn(0x7f6d6c0cdd80 msgr2=0x7f6d6c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:59.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- 172.21.15.71:0/1512626653 shutdown_connections 2023-10-26T05:47:59.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- 172.21.15.71:0/1512626653 wait complete. 2023-10-26T05:47:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 Processor -- start 2023-10-26T05:47:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- start start 2023-10-26T05:47:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d6c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c0cb170 0x7f6d6c153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c1535a0 0x7f6d6c145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c06a650 con 0x7f6d54005190 2023-10-26T05:47:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c152c40 con 0x7f6d6c1535a0 2023-10-26T05:47:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d71b76640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c151fc0 con 0x7f6d6c0cb170 2023-10-26T05:47:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d6c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d6c0cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55762/0 (socket says 172.21.15.71:55762) 2023-10-26T05:47:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 -- 172.21.15.71:0/2028956518 learned_addr learned my addr 172.21.15.71:0/2028956518 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6affd640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c0cb170 0x7f6d6c153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6bfff640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c1535a0 0x7f6d6c145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 -- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c0cb170 msgr2=0x7f6d6c153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c0cb170 0x7f6d6c153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 -- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c1535a0 msgr2=0x7f6d6c145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c1535a0 0x7f6d6c145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6b7fe640 1 -- 172.21.15.71:0/2028956518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c1522a0 con 0x7f6d54005190 2023-10-26T05:47:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.261+0000 7f6d6bfff640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c1535a0 0x7f6d6c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:47:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d6b7fe640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d6c0cac30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c010a40 tx=0x7f6d5c010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0096b0 con 0x7f6d54005190 2023-10-26T05:47:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6d5c00eda0 con 0x7f6d54005190 2023-10-26T05:47:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c005280 con 0x7f6d54005190 2023-10-26T05:47:59.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c152580 con 0x7f6d54005190 2023-10-26T05:47:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c0c7e30 con 0x7f6d54005190 2023-10-26T05:47:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c066b10 con 0x7f6d54005190 2023-10-26T05:47:59.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6d5c0053e0 con 0x7f6d54005190 2023-10-26T05:47:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d53fff640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6d580664a0 0x7f6d58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d5c016070 con 0x7f6d54005190 2023-10-26T05:47:59.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d6affd640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6d580664a0 0x7f6d58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.265+0000 7f6d6affd640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6d580664a0 0x7f6d58068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0b5710 tx=0x7f6d60001440 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:59.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.269+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6d5c0520c0 con 0x7f6d54005190 2023-10-26T05:47:59.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.413+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d6c071e20 con 0x7f6d54005190 2023-10-26T05:47:59.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.413+0000 7f6d53fff640 1 -- 172.21.15.71:0/2028956518 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d5c0556e0 con 0x7f6d54005190 2023-10-26T05:47:59.419 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:47:59.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6d580664a0 msgr2=0x7f6d58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6d580664a0 0x7f6d58068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0b5710 tx=0x7f6d60001440 comp rx=0 tx=0).stop 2023-10-26T05:47:59.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 msgr2=0x7f6d6c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d6c0cac30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c010a40 tx=0x7f6d5c010f00 comp rx=0 tx=0).stop 2023-10-26T05:47:59.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 shutdown_connections 2023-10-26T05:47:59.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d6c1535a0 0x7f6d6c145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d6c0cb170 0x7f6d6c153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6d580664a0 0x7f6d58068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 --2- 172.21.15.71:0/2028956518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d54005190 0x7f6d6c0cac30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 >> 172.21.15.71:0/2028956518 conn(0x7f6d6c0cdd80 msgr2=0x7f6d6c15e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:59.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 shutdown_connections 2023-10-26T05:47:59.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.417+0000 7f6d71b76640 1 -- 172.21.15.71:0/2028956518 wait complete. 2023-10-26T05:47:59.436 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:47:59.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- 172.21.15.71:0/1473312876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 msgr2=0x7f98a00cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1473312876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a00cbb10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9894009900 tx=0x7f989401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- 172.21.15.71:0/1473312876 shutdown_connections 2023-10-26T05:47:59.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1473312876 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a00cc050 0x7f98a00cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1473312876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a00cbb10 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- 172.21.15.71:0/1473312876 >> 172.21.15.71:0/1473312876 conn(0x7f98a006f440 msgr2=0x7f98a006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:59.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- 172.21.15.71:0/1473312876 shutdown_connections 2023-10-26T05:47:59.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- 172.21.15.71:0/1473312876 wait complete. 2023-10-26T05:47:59.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 Processor -- start 2023-10-26T05:47:59.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- start start 2023-10-26T05:47:59.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a01408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a00cc050 0x7f98a0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a0142b30 0x7f98a0141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0141870 con 0x7f98a00af990 2023-10-26T05:47:59.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0153f40 con 0x7f98a0142b30 2023-10-26T05:47:59.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a7d65640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00cbe40 con 0x7f98a00cc050 2023-10-26T05:47:59.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a01408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a01408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55782/0 (socket says 172.21.15.71:55782) 2023-10-26T05:47:59.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 -- 172.21.15.71:0/1687055700 learned_addr learned my addr 172.21.15.71:0/1687055700 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:59.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5300640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a00cc050 0x7f98a0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a6302640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a0142b30 0x7f98a0141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 -- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a00cc050 msgr2=0x7f98a0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a00cc050 0x7f98a0140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 -- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a0142b30 msgr2=0x7f98a0141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a0142b30 0x7f98a0141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.545+0000 7f98a5b01640 1 -- 172.21.15.71:0/1687055700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894009560 con 0x7f98a00af990 2023-10-26T05:47:59.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f98a5b01640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a01408b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f989800ba50 tx=0x7f989800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:59.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800cc10 con 0x7f98a00af990 2023-10-26T05:47:59.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f989800ce60 con 0x7f98a00af990 2023-10-26T05:47:59.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a0158170 con 0x7f98a00af990 2023-10-26T05:47:59.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898014680 con 0x7f98a00af990 2023-10-26T05:47:59.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a0158710 con 0x7f98a00af990 2023-10-26T05:47:59.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a0066b10 con 0x7f98a00af990 2023-10-26T05:47:59.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9898003980 con 0x7f98a00af990 2023-10-26T05:47:59.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f989e7fc640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f988c0664a0 0x7f988c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9898083e60 con 0x7f98a00af990 2023-10-26T05:47:59.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f98a5300640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f988c0664a0 0x7f988c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.549+0000 7f98a5300640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f988c0664a0 0x7f988c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98940af000 tx=0x7f989401b000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:59.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.553+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98980511c0 con 0x7f98a00af990 2023-10-26T05:47:59.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98a00721e0 con 0x7f98a00af990 2023-10-26T05:47:59.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f989e7fc640 1 -- 172.21.15.71:0/1687055700 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98980547e0 con 0x7f98a00af990 2023-10-26T05:47:59.710 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:47:59.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f988c0664a0 msgr2=0x7f988c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f988c0664a0 0x7f988c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98940af000 tx=0x7f989401b000 comp rx=0 tx=0).stop 2023-10-26T05:47:59.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 msgr2=0x7f98a01408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a01408b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f989800ba50 tx=0x7f989800bf10 comp rx=0 tx=0).stop 2023-10-26T05:47:59.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 shutdown_connections 2023-10-26T05:47:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a0142b30 0x7f98a0141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a00cc050 0x7f98a0140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f988c0664a0 0x7f988c068960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 --2- 172.21.15.71:0/1687055700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a00af990 0x7f98a01408b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.705+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 >> 172.21.15.71:0/1687055700 conn(0x7f98a006f440 msgr2=0x7f98a00b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.709+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 shutdown_connections 2023-10-26T05:47:59.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.709+0000 7f98a7d65640 1 -- 172.21.15.71:0/1687055700 wait complete. 2023-10-26T05:47:59.727 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:47:59.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 -- 172.21.15.71:0/1948341295 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153bf0 msgr2=0x7fce900c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 --2- 172.21.15.71:0/1948341295 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153bf0 0x7fce900c9800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce84009900 tx=0x7fce8401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 -- 172.21.15.71:0/1948341295 shutdown_connections 2023-10-26T05:47:59.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 --2- 172.21.15.71:0/1948341295 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153bf0 0x7fce900c9800 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 --2- 172.21.15.71:0/1948341295 >> v2:172.21.15.86:3300/0 conn(0x7fce90153810 0x7fce9015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 -- 172.21.15.71:0/1948341295 >> 172.21.15.71:0/1948341295 conn(0x7fce900cbc30 msgr2=0x7fce900cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:47:59.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 -- 172.21.15.71:0/1948341295 shutdown_connections 2023-10-26T05:47:59.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 -- 172.21.15.71:0/1948341295 wait complete. 2023-10-26T05:47:59.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 Processor -- start 2023-10-26T05:47:59.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.849+0000 7fce96a34640 1 -- start start 2023-10-26T05:47:59.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce900ce360 0x7fce90155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153810 0x7fce900c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 0x7fce900ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90155c80 con 0x7fce90153bf0 2023-10-26T05:47:59.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9014ed70 con 0x7fce900ce360 2023-10-26T05:47:59.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9014f050 con 0x7fce90153810 2023-10-26T05:47:59.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 0x7fce900ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 0x7fce900ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55794/0 (socket says 172.21.15.71:55794) 2023-10-26T05:47:59.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 -- 172.21.15.71:0/3527681561 learned_addr learned my addr 172.21.15.71:0/3527681561 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:47:59.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8ffff640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce900ce360 0x7fce90155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8f7fe640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153810 0x7fce900c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 -- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153810 msgr2=0x7fce900c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153810 0x7fce900c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 -- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce900ce360 msgr2=0x7fce90155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:47:59.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce900ce360 0x7fce90155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:47:59.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 -- 172.21.15.71:0/3527681561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800094a0 con 0x7fce90153bf0 2023-10-26T05:47:59.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce94fd1640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 0x7fce900ccd00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c770 tx=0x7fce8800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:59.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88003b80 con 0x7fce90153bf0 2023-10-26T05:47:59.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce88003ce0 con 0x7fce90153bf0 2023-10-26T05:47:59.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce84009560 con 0x7fce90153bf0 2023-10-26T05:47:59.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800aab0 con 0x7fce90153bf0 2023-10-26T05:47:59.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9014f610 con 0x7fce90153bf0 2023-10-26T05:47:59.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90066b10 con 0x7fce90153bf0 2023-10-26T05:47:59.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce8800ac10 con 0x7fce90153bf0 2023-10-26T05:47:59.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8cff9640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce7c066470 0x7fce7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:47:59.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce88083bf0 con 0x7fce90153bf0 2023-10-26T05:47:59.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8ffff640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce7c066470 0x7fce7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:47:59.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.853+0000 7fce8ffff640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce7c066470 0x7fce7c068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce80009d70 tx=0x7fce800091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:47:59.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.857+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce880511b0 con 0x7fce90153bf0 2023-10-26T05:48:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:47:59.997+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce900721e0 con 0x7fce90153bf0 2023-10-26T05:48:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce8cff9640 1 -- 172.21.15.71:0/3527681561 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce880547d0 con 0x7fce90153bf0 2023-10-26T05:48:00.004 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:00.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce7c066470 msgr2=0x7fce7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce7c066470 0x7fce7c068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce80009d70 tx=0x7fce800091e0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 msgr2=0x7fce900ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 0x7fce900ccd00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce8800c770 tx=0x7fce8800cc30 comp rx=0 tx=0).stop 2023-10-26T05:48:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 shutdown_connections 2023-10-26T05:48:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce7c066470 0x7fce7c068930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce90153bf0 0x7fce900ccd00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce90153810 0x7fce900c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 --2- 172.21.15.71:0/3527681561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce900ce360 0x7fce90155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 >> 172.21.15.71:0/3527681561 conn(0x7fce900cbc30 msgr2=0x7fce900cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:00.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 shutdown_connections 2023-10-26T05:48:00.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.001+0000 7fce96a34640 1 -- 172.21.15.71:0/3527681561 wait complete. 2023-10-26T05:48:00.022 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- 172.21.15.71:0/3207764696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ee0005190 msgr2=0x7f1ee000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3207764696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ee0005190 0x7f1ee000b990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1eec006fd0 tx=0x7f1eec01ca00 comp rx=0 tx=0).stop 2023-10-26T05:48:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- 172.21.15.71:0/3207764696 shutdown_connections 2023-10-26T05:48:00.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3207764696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ee0005190 0x7f1ee000b990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- 172.21.15.71:0/3207764696 >> 172.21.15.71:0/3207764696 conn(0x7f1ef40afeb0 msgr2=0x7f1ef40b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- 172.21.15.71:0/3207764696 shutdown_connections 2023-10-26T05:48:00.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- 172.21.15.71:0/3207764696 wait complete. 2023-10-26T05:48:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 Processor -- start 2023-10-26T05:48:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- start start 2023-10-26T05:48:00.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ee0005190 0x7f1ef4152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef414c990 0x7f1ef4152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 0x7f1ef40aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4152da0 con 0x7f1ef41f2430 2023-10-26T05:48:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef414beb0 con 0x7f1ef414c990 2023-10-26T05:48:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1efba0c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef414c190 con 0x7f1ee0005190 2023-10-26T05:48:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1ef9fa9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 0x7f1ef40aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1ef9fa9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 0x7f1ef40aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55806/0 (socket says 172.21.15.71:55806) 2023-10-26T05:48:00.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.129+0000 7f1ef9fa9640 1 -- 172.21.15.71:0/3488611947 learned_addr learned my addr 172.21.15.71:0/3488611947 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef97a8640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ee0005190 0x7f1ef4152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef8fa7640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef414c990 0x7f1ef4152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef9fa9640 1 -- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ee0005190 msgr2=0x7f1ef4152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef9fa9640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ee0005190 0x7f1ef4152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef9fa9640 1 -- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef414c990 msgr2=0x7f1ef4152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef9fa9640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef414c990 0x7f1ef4152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef9fa9640 1 -- 172.21.15.71:0/3488611947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef414c470 con 0x7f1ef41f2430 2023-10-26T05:48:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef9fa9640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 0x7f1ef40aeba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1eec006040 tx=0x7f1eec0a6e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec004bb0 con 0x7f1ef41f2430 2023-10-26T05:48:00.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1eec0b3070 con 0x7f1ef41f2430 2023-10-26T05:48:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef4152ab0 con 0x7f1ef41f2430 2023-10-26T05:48:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec006330 con 0x7f1ef41f2430 2023-10-26T05:48:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef41f28b0 con 0x7f1ef41f2430 2023-10-26T05:48:00.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef4069770 con 0x7f1ef41f2430 2023-10-26T05:48:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1eec0b5030 con 0x7f1ef41f2430 2023-10-26T05:48:00.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ee9ffb640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1ed8066470 0x7f1ed8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eec11e490 con 0x7f1ef41f2430 2023-10-26T05:48:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.133+0000 7f1ef97a8640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1ed8066470 0x7f1ed8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.137+0000 7f1ef97a8640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1ed8066470 0x7f1ed8068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001210 tx=0x7f1ef0000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:00.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.137+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1eec0eb9a0 con 0x7f1ef41f2430 2023-10-26T05:48:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.289+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1ef404db60 con 0x7f1ef41f2430 2023-10-26T05:48:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.289+0000 7f1ee9ffb640 1 -- 172.21.15.71:0/3488611947 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1eec0be020 con 0x7f1ef41f2430 2023-10-26T05:48:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1ed8066470 msgr2=0x7f1ed8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1ed8066470 0x7f1ed8068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001210 tx=0x7f1ef0000ee0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 msgr2=0x7f1ef40aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 0x7f1ef40aeba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1eec006040 tx=0x7f1eec0a6e90 comp rx=0 tx=0).stop 2023-10-26T05:48:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 shutdown_connections 2023-10-26T05:48:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1ed8066470 0x7f1ed8068930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ef41f2430 0x7f1ef40aeba0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ef414c990 0x7f1ef4152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 --2- 172.21.15.71:0/3488611947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ee0005190 0x7f1ef4152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 >> 172.21.15.71:0/3488611947 conn(0x7f1ef40afeb0 msgr2=0x7f1ef414f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 shutdown_connections 2023-10-26T05:48:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.293+0000 7f1efba0c640 1 -- 172.21.15.71:0/3488611947 wait complete. 2023-10-26T05:48:00.312 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:48:00.312 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:00.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:00.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:00.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[35477]: fuse finished with error 0 and tester_r 0 2023-10-26T05:48:00.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:00.388 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:00.389 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:48:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:00.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:48:00.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:00.397 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:48:00.487 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:00.488 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:00.494 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:00.495 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:00.543 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:00.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:48:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:00.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2897098863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00d0850 msgr2=0x7fb0e015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2897098863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00d0850 0x7fb0e015b3a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8009840 tx=0x7fb0c801c700 comp rx=0 tx=0).stop 2023-10-26T05:48:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2897098863 shutdown_connections 2023-10-26T05:48:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2897098863 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00d0850 0x7fb0e015b3a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2897098863 >> 172.21.15.71:0/2897098863 conn(0x7fb0e00cdd80 msgr2=0x7fb0e00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:00.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2897098863 shutdown_connections 2023-10-26T05:48:00.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2897098863 wait complete. 2023-10-26T05:48:00.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 Processor -- start 2023-10-26T05:48:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- start start 2023-10-26T05:48:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 0x7fb0e0150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e00d0470 0x7fb0e0158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0e00d0850 0x7fb0e0152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e006a650 con 0x7fb0e00cd880 2023-10-26T05:48:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015cb00 con 0x7fb0e00d0470 2023-10-26T05:48:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e01510c0 con 0x7fb0e00d0850 2023-10-26T05:48:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 0x7fb0e0150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 0x7fb0e0150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55836/0 (socket says 172.21.15.71:55836) 2023-10-26T05:48:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 -- 172.21.15.71:0/2784789764 learned_addr learned my addr 172.21.15.71:0/2784789764 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 -- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0e00d0850 msgr2=0x7fb0e0152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0e00d0850 0x7fb0e0152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 -- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e00d0470 msgr2=0x7fb0e0158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e00d0470 0x7fb0e0158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 -- 172.21.15.71:0/2784789764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80094a0 con 0x7fb0e00cd880 2023-10-26T05:48:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0deffd640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 0x7fb0e0150b80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c7c0 tx=0x7fb0d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800aa50 con 0x7fb0e00cd880 2023-10-26T05:48:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb0d800aca0 con 0x7fb0e00cd880 2023-10-26T05:48:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e015e900 con 0x7fb0e00cd880 2023-10-26T05:48:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8012680 con 0x7fb0e00cd880 2023-10-26T05:48:00.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.669+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e015c4b0 con 0x7fb0e00cd880 2023-10-26T05:48:00.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.673+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0066b10 con 0x7fb0e00cd880 2023-10-26T05:48:00.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.673+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb0d801b400 con 0x7fb0e00cd880 2023-10-26T05:48:00.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.673+0000 7fb0d77fe640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0c0066470 0x7fb0c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.673+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0d80843c0 con 0x7fb0e00cd880 2023-10-26T05:48:00.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.673+0000 7fb0de7fc640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0c0066470 0x7fb0c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.673+0000 7fb0de7fc640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0c0066470 0x7fb0c0068930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80af000 tx=0x7fb0c80194a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:00.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.677+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0d80517a0 con 0x7fb0e00cd880 2023-10-26T05:48:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.829+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0e00721e0 con 0x7fb0e00cd880 2023-10-26T05:48:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.833+0000 7fb0d77fe640 1 -- 172.21.15.71:0/2784789764 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fb0d8054dc0 con 0x7fb0e00cd880 2023-10-26T05:48:00.838 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:00.838 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0c0066470 msgr2=0x7fb0c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0c0066470 0x7fb0c0068930 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80af000 tx=0x7fb0c80194a0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 msgr2=0x7fb0e0150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 0x7fb0e0150b80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c7c0 tx=0x7fb0d800cc80 comp rx=0 tx=0).stop 2023-10-26T05:48:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 shutdown_connections 2023-10-26T05:48:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0e00d0850 0x7fb0e0152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0e00d0470 0x7fb0e0158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0c0066470 0x7fb0c0068930 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 --2- 172.21.15.71:0/2784789764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e00cd880 0x7fb0e0150b80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 >> 172.21.15.71:0/2784789764 conn(0x7fb0e00cdd80 msgr2=0x7fb0e00b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 shutdown_connections 2023-10-26T05:48:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.837+0000 7fb0e5415640 1 -- 172.21.15.71:0/2784789764 wait complete. 2023-10-26T05:48:00.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:00.857 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:00.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 -- 172.21.15.71:0/1239428922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740cd880 msgr2=0x7f157415b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 --2- 172.21.15.71:0/1239428922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740cd880 0x7f157415b8e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15740c4dd0 tx=0x7f156c01cab0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 -- 172.21.15.71:0/1239428922 shutdown_connections 2023-10-26T05:48:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 --2- 172.21.15.71:0/1239428922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740cd880 0x7f157415b8e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 --2- 172.21.15.71:0/1239428922 >> v2:172.21.15.86:3300/0 conn(0x7f15740d0850 0x7f157415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 -- 172.21.15.71:0/1239428922 >> 172.21.15.71:0/1239428922 conn(0x7f15740cdd80 msgr2=0x7f15740ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:00.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 -- 172.21.15.71:0/1239428922 shutdown_connections 2023-10-26T05:48:00.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.977+0000 7f157958b640 1 -- 172.21.15.71:0/1239428922 wait complete. 2023-10-26T05:48:00.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 Processor -- start 2023-10-26T05:48:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- start start 2023-10-26T05:48:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15740cd880 0x7f15740c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15740d0470 0x7f157415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 0x7f157414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1574158ad0 con 0x7f15740d0850 2023-10-26T05:48:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15741587a0 con 0x7f15740cd880 2023-10-26T05:48:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157415adb0 con 0x7f15740d0470 2023-10-26T05:48:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 0x7f157414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 0x7f157414bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55856/0 (socket says 172.21.15.71:55856) 2023-10-26T05:48:00.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 -- 172.21.15.71:0/2935992825 learned_addr learned my addr 172.21.15.71:0/2935992825 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15727fc640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15740d0470 0x7f157415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f1572ffd640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15740cd880 0x7f15740c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 -- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15740d0470 msgr2=0x7f157415a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15740d0470 0x7f157415a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 -- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15740cd880 msgr2=0x7f15740c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15740cd880 0x7f15740c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 -- 172.21.15.71:0/2935992825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c009820 con 0x7f15740d0850 2023-10-26T05:48:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f15737fe640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 0x7f157414bc10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f156c0196c0 tx=0x7f156c0a5900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004e60 con 0x7f15740d0850 2023-10-26T05:48:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f156c0040c0 con 0x7f15740d0850 2023-10-26T05:48:00.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157415e4b0 con 0x7f15740d0850 2023-10-26T05:48:00.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c004490 con 0x7f15740d0850 2023-10-26T05:48:00.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157415ea70 con 0x7f15740d0850 2023-10-26T05:48:00.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1574066b10 con 0x7f15740d0850 2023-10-26T05:48:00.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.981+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f156c0b4040 con 0x7f15740d0850 2023-10-26T05:48:00.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.985+0000 7f155b7fe640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1560066470 0x7f1560068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:00.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.985+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f156c11e1a0 con 0x7f15740d0850 2023-10-26T05:48:00.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.985+0000 7f1572ffd640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1560066470 0x7f1560068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:00.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.985+0000 7f1572ffd640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1560066470 0x7f1560068930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1564000f70 tx=0x7f1564009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:00.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:00.989+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f156c0eb620 con 0x7f15740d0850 2023-10-26T05:48:01.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.141+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1574158c80 con 0x7f15740d0850 2023-10-26T05:48:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f155b7fe640 1 -- 172.21.15.71:0/2935992825 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f156c0b2020 con 0x7f15740d0850 2023-10-26T05:48:01.148 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1560066470 msgr2=0x7f1560068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1560066470 0x7f1560068930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1564000f70 tx=0x7f1564009040 comp rx=0 tx=0).stop 2023-10-26T05:48:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 msgr2=0x7f157414bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 0x7f157414bc10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f156c0196c0 tx=0x7f156c0a5900 comp rx=0 tx=0).stop 2023-10-26T05:48:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 shutdown_connections 2023-10-26T05:48:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1560066470 0x7f1560068930 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15740d0850 0x7f157414bc10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15740d0470 0x7f157415a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 --2- 172.21.15.71:0/2935992825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15740cd880 0x7f15740c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 >> 172.21.15.71:0/2935992825 conn(0x7f15740cdd80 msgr2=0x7f15740b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:01.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 shutdown_connections 2023-10-26T05:48:01.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.145+0000 7f157958b640 1 -- 172.21.15.71:0/2935992825 wait complete. 2023-10-26T05:48:01.166 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2738716836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 msgr2=0x7f52e40cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2738716836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e40cb820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52e40aebf0 tx=0x7f52d401c830 comp rx=0 tx=0).stop 2023-10-26T05:48:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2738716836 shutdown_connections 2023-10-26T05:48:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2738716836 >> v2:172.21.15.86:3300/0 conn(0x7f52e413efd0 0x7f52e4069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2738716836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e40cb820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2738716836 >> 172.21.15.71:0/2738716836 conn(0x7f52e40aec60 msgr2=0x7f52e40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:01.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2738716836 shutdown_connections 2023-10-26T05:48:01.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2738716836 wait complete. 2023-10-26T05:48:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 Processor -- start 2023-10-26T05:48:01.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.293+0000 7f52ebfbe640 1 -- start start 2023-10-26T05:48:01.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40cbd60 0x7f52e415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e4133d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52e413efd0 0x7f52e4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e4132d10 con 0x7f52e413c190 2023-10-26T05:48:01.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e4132fb0 con 0x7f52e40cbd60 2023-10-26T05:48:01.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e4133290 con 0x7f52e413efd0 2023-10-26T05:48:01.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e4133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e4133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55862/0 (socket says 172.21.15.71:55862) 2023-10-26T05:48:01.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 -- 172.21.15.71:0/2527710282 learned_addr learned my addr 172.21.15.71:0/2527710282 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:01.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9d5a640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40cbd60 0x7f52e415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ea55b640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52e413efd0 0x7f52e4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 -- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52e413efd0 msgr2=0x7f52e4159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52e413efd0 0x7f52e4159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 -- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40cbd60 msgr2=0x7f52e415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40cbd60 0x7f52e415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 -- 172.21.15.71:0/2527710282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009560 con 0x7f52e413c190 2023-10-26T05:48:01.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52e9559640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e4133d70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c450 tx=0x7f52e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:01.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0012830 con 0x7f52e413c190 2023-10-26T05:48:01.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f52e0012aa0 con 0x7f52e413c190 2023-10-26T05:48:01.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0011380 con 0x7f52e413c190 2023-10-26T05:48:01.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e4133570 con 0x7f52e413c190 2023-10-26T05:48:01.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e415c5c0 con 0x7f52e413c190 2023-10-26T05:48:01.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e4066b10 con 0x7f52e413c190 2023-10-26T05:48:01.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f52e00114e0 con 0x7f52e413c190 2023-10-26T05:48:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52da7fc640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f52c8066470 0x7f52c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.297+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52e0083a70 con 0x7f52e413c190 2023-10-26T05:48:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.301+0000 7f52e9d5a640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f52c8066470 0x7f52c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.301+0000 7f52e9d5a640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f52c8066470 0x7f52c8068930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52d40af000 tx=0x7f52d401b070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:01.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.305+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f52e0050de0 con 0x7f52e413c190 2023-10-26T05:48:01.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52e40721e0 con 0x7f52e413c190 2023-10-26T05:48:01.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52da7fc640 1 -- 172.21.15.71:0/2527710282 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52e0054400 con 0x7f52e413c190 2023-10-26T05:48:01.462 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f52c8066470 msgr2=0x7f52c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f52c8066470 0x7f52c8068930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f52d40af000 tx=0x7f52d401b070 comp rx=0 tx=0).stop 2023-10-26T05:48:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 msgr2=0x7f52e4133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e4133d70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c450 tx=0x7f52e000c910 comp rx=0 tx=0).stop 2023-10-26T05:48:01.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 shutdown_connections 2023-10-26T05:48:01.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52e413efd0 0x7f52e4159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f52c8066470 0x7f52c8068930 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52e413c190 0x7f52e4133d70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.457+0000 7f52ebfbe640 1 --2- 172.21.15.71:0/2527710282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52e40cbd60 0x7f52e415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.461+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 >> 172.21.15.71:0/2527710282 conn(0x7f52e40aec60 msgr2=0x7f52e4137c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:01.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.461+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 shutdown_connections 2023-10-26T05:48:01.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.461+0000 7f52ebfbe640 1 -- 172.21.15.71:0/2527710282 wait complete. 2023-10-26T05:48:01.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:01.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/101231770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb8003980 msgr2=0x7f0fb8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/101231770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb8003980 0x7f0fb8003da0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009b60 tx=0x7f0fbc01d200 comp rx=0 tx=0).stop 2023-10-26T05:48:01.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fcbfff640 1 -- 172.21.15.71:0/101231770 reap_dead start 2023-10-26T05:48:01.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/101231770 shutdown_connections 2023-10-26T05:48:01.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/101231770 >> 172.21.15.71:0/101231770 conn(0x7f0fcc0aec60 msgr2=0x7f0fcc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:01.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/101231770 shutdown_connections 2023-10-26T05:48:01.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.601+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/101231770 wait complete. 2023-10-26T05:48:01.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 Processor -- start 2023-10-26T05:48:01.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- start start 2023-10-26T05:48:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb8003980 0x7f0fcc0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 0x7f0fcc14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fcc0adda0 0x7f0fcc0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc160e40 con 0x7f0fcc0abe30 2023-10-26T05:48:01.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc15ff50 con 0x7f0fb8003980 2023-10-26T05:48:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc160230 con 0x7f0fcc0adda0 2023-10-26T05:48:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 0x7f0fcc14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 0x7f0fcc14b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55868/0 (socket says 172.21.15.71:55868) 2023-10-26T05:48:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 -- 172.21.15.71:0/2978077924 learned_addr learned my addr 172.21.15.71:0/2978077924 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcbfff640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fcc0adda0 0x7f0fcc0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcb7fe640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb8003980 0x7f0fcc0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 -- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fcc0adda0 msgr2=0x7f0fcc0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fcc0adda0 0x7f0fcc0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 -- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb8003980 msgr2=0x7f0fcc0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb8003980 0x7f0fcc0b01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 -- 172.21.15.71:0/2978077924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc160510 con 0x7f0fcc0abe30 2023-10-26T05:48:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcb7fe640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb8003980 0x7f0fcc0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcaffd640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 0x7f0fcc14b3d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4010a10 tx=0x7f0fb4010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400eda0 con 0x7f0fcc0abe30 2023-10-26T05:48:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0fb4004030 con 0x7f0fcc0abe30 2023-10-26T05:48:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc1607f0 con 0x7f0fcc0abe30 2023-10-26T05:48:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4004420 con 0x7f0fcc0abe30 2023-10-26T05:48:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc0ae760 con 0x7f0fcc0abe30 2023-10-26T05:48:01.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc066b10 con 0x7f0fcc0abe30 2023-10-26T05:48:01.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0fb4003600 con 0x7f0fcc0abe30 2023-10-26T05:48:01.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fc3fff640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fac066470 0x7f0fac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fb4016070 con 0x7f0fcc0abe30 2023-10-26T05:48:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.605+0000 7f0fcb7fe640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fac066470 0x7f0fac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.609+0000 7f0fcb7fe640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fac066470 0x7f0fac068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0ae000 tx=0x7f0fbc01a000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.609+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0fb4052110 con 0x7f0fcc0abe30 2023-10-26T05:48:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.761+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fcc0721e0 con 0x7f0fcc0abe30 2023-10-26T05:48:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fc3fff640 1 -- 172.21.15.71:0/2978077924 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fb4055730 con 0x7f0fcc0abe30 2023-10-26T05:48:01.768 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fac066470 msgr2=0x7f0fac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fac066470 0x7f0fac068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0ae000 tx=0x7f0fbc01a000 comp rx=0 tx=0).stop 2023-10-26T05:48:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 msgr2=0x7f0fcc14b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 0x7f0fcc14b3d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4010a10 tx=0x7f0fb4010ed0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 shutdown_connections 2023-10-26T05:48:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fcc0adda0 0x7f0fcc0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fac066470 0x7f0fac068930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fcc0abe30 0x7f0fcc14b3d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 --2- 172.21.15.71:0/2978077924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb8003980 0x7f0fcc0b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 >> 172.21.15.71:0/2978077924 conn(0x7f0fcc0aec60 msgr2=0x7f0fcc162580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 shutdown_connections 2023-10-26T05:48:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.765+0000 7f0fd1cd5640 1 -- 172.21.15.71:0/2978077924 wait complete. 2023-10-26T05:48:01.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:01.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 -- 172.21.15.71:0/2787759891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 msgr2=0x7fbd28003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 --2- 172.21.15.71:0/2787759891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd28003da0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009b60 tx=0x7fbd3001d260 comp rx=0 tx=0).stop 2023-10-26T05:48:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd3f7fe640 1 -- 172.21.15.71:0/2787759891 reap_dead start 2023-10-26T05:48:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 -- 172.21.15.71:0/2787759891 shutdown_connections 2023-10-26T05:48:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 -- 172.21.15.71:0/2787759891 >> 172.21.15.71:0/2787759891 conn(0x7fbd400addc0 msgr2=0x7fbd400b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 -- 172.21.15.71:0/2787759891 shutdown_connections 2023-10-26T05:48:01.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 -- 172.21.15.71:0/2787759891 wait complete. 2023-10-26T05:48:01.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.905+0000 7fbd456a3640 1 Processor -- start 2023-10-26T05:48:01.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- start start 2023-10-26T05:48:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd40149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd40153cc0 0x7fbd40154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd4014eae0 0x7fbd4014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4014f4d0 con 0x7fbd28003980 2023-10-26T05:48:01.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4014a970 con 0x7fbd40153cc0 2023-10-26T05:48:01.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd40154780 con 0x7fbd4014eae0 2023-10-26T05:48:01.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd40149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd40149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37816/0 (socket says 172.21.15.71:37816) 2023-10-26T05:48:01.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 -- 172.21.15.71:0/660063208 learned_addr learned my addr 172.21.15.71:0/660063208 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:01.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3e7fc640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd40153cc0 0x7fbd40154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3f7fe640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd4014eae0 0x7fbd4014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 -- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd4014eae0 msgr2=0x7fbd4014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd4014eae0 0x7fbd4014ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 -- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd40153cc0 msgr2=0x7fbd40154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:01.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd40153cc0 0x7fbd40154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:01.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 -- 172.21.15.71:0/660063208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd40154a60 con 0x7fbd28003980 2023-10-26T05:48:01.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3effd640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd40149370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd3001d090 tx=0x7fbd30005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:01.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd30004a10 con 0x7fbd28003980 2023-10-26T05:48:01.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd30004b70 con 0x7fbd28003980 2023-10-26T05:48:01.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd40154d40 con 0x7fbd28003980 2023-10-26T05:48:01.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd300b6410 con 0x7fbd28003980 2023-10-26T05:48:01.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd401498b0 con 0x7fbd28003980 2023-10-26T05:48:01.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd400cdea0 con 0x7fbd28003980 2023-10-26T05:48:01.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbd300b3070 con 0x7fbd28003980 2023-10-26T05:48:01.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd277fe640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd2c0664a0 0x7fbd2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:01.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd300a5e40 con 0x7fbd28003980 2023-10-26T05:48:01.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.909+0000 7fbd3e7fc640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd2c0664a0 0x7fbd2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:01.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.913+0000 7fbd3e7fc640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd2c0664a0 0x7fbd2c068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbd34000fc0 tx=0x7fbd34001c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:01.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:01.913+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbd300ec160 con 0x7fbd28003980 2023-10-26T05:48:02.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd40154f30 con 0x7fbd28003980 2023-10-26T05:48:02.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd277fe640 1 -- 172.21.15.71:0/660063208 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd300b1080 con 0x7fbd28003980 2023-10-26T05:48:02.090 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:02.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd2c0664a0 msgr2=0x7fbd2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd2c0664a0 0x7fbd2c068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbd34000fc0 tx=0x7fbd34001c00 comp rx=0 tx=0).stop 2023-10-26T05:48:02.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 msgr2=0x7fbd40149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd40149370 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd3001d090 tx=0x7fbd30005640 comp rx=0 tx=0).stop 2023-10-26T05:48:02.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 shutdown_connections 2023-10-26T05:48:02.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd4014eae0 0x7fbd4014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.085+0000 7fbd456a3640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd40153cc0 0x7fbd40154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.089+0000 7fbd456a3640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd2c0664a0 0x7fbd2c068960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.089+0000 7fbd456a3640 1 --2- 172.21.15.71:0/660063208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd28003980 0x7fbd40149370 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.089+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 >> 172.21.15.71:0/660063208 conn(0x7fbd400addc0 msgr2=0x7fbd40159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.089+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 shutdown_connections 2023-10-26T05:48:02.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.089+0000 7fbd456a3640 1 -- 172.21.15.71:0/660063208 wait complete. 2023-10-26T05:48:02.108 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:02.109 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:02.115 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:02.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- 172.21.15.71:0/3821161472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 msgr2=0x7f2a4400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 --2- 172.21.15.71:0/3821161472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a4400b990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009ee0 tx=0x7f2a5001d610 comp rx=0 tx=0).stop 2023-10-26T05:48:02.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5dbed640 1 -- 172.21.15.71:0/3821161472 reap_dead start 2023-10-26T05:48:02.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- 172.21.15.71:0/3821161472 shutdown_connections 2023-10-26T05:48:02.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- 172.21.15.71:0/3821161472 >> 172.21.15.71:0/3821161472 conn(0x7f2a580cdd80 msgr2=0x7f2a580ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- 172.21.15.71:0/3821161472 shutdown_connections 2023-10-26T05:48:02.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- 172.21.15.71:0/3821161472 wait complete. 2023-10-26T05:48:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 Processor -- start 2023-10-26T05:48:02.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- start start 2023-10-26T05:48:02.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a580c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a58156d30 0x7f2a5815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a581571b0 0x7f2a5815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580cbe50 con 0x7f2a44005190 2023-10-26T05:48:02.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58157920 con 0x7f2a581571b0 2023-10-26T05:48:02.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5f650640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580cc2e0 con 0x7f2a58156d30 2023-10-26T05:48:02.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a580c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a580c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37832/0 (socket says 172.21.15.71:37832) 2023-10-26T05:48:02.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 -- 172.21.15.71:0/4076243720 learned_addr learned my addr 172.21.15.71:0/4076243720 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5cbeb640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a58156d30 0x7f2a5815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5dbed640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a581571b0 0x7f2a5815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 -- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a58156d30 msgr2=0x7f2a5815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a58156d30 0x7f2a5815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 -- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a581571b0 msgr2=0x7f2a5815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a581571b0 0x7f2a5815e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 -- 172.21.15.71:0/4076243720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580cb240 con 0x7f2a44005190 2023-10-26T05:48:02.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5cbeb640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a58156d30 0x7f2a5815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:02.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5dbed640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a581571b0 0x7f2a5815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:02.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a5d3ec640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a580c83d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c760 tx=0x7f2a4800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:02.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48005480 con 0x7f2a44005190 2023-10-26T05:48:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.229+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a4800ce10 con 0x7f2a44005190 2023-10-26T05:48:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a580cb520 con 0x7f2a44005190 2023-10-26T05:48:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800a8a0 con 0x7f2a44005190 2023-10-26T05:48:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a580cbae0 con 0x7f2a44005190 2023-10-26T05:48:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58066b10 con 0x7f2a44005190 2023-10-26T05:48:02.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2a4800aa00 con 0x7f2a44005190 2023-10-26T05:48:02.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a4dffb640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a3c066470 0x7f2a3c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a48083c00 con 0x7f2a44005190 2023-10-26T05:48:02.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a5cbeb640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a3c066470 0x7f2a3c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.233+0000 7f2a5cbeb640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a3c066470 0x7f2a3c068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a54005d10 tx=0x7f2a54005f60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:02.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.237+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2a48051160 con 0x7f2a44005190 2023-10-26T05:48:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.381+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a580721e0 con 0x7f2a44005190 2023-10-26T05:48:02.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a4dffb640 1 -- 172.21.15.71:0/4076243720 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f2a4800acb0 con 0x7f2a44005190 2023-10-26T05:48:02.388 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:02.388 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:02.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a3c066470 msgr2=0x7f2a3c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a3c066470 0x7f2a3c068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2a54005d10 tx=0x7f2a54005f60 comp rx=0 tx=0).stop 2023-10-26T05:48:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 msgr2=0x7f2a580c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a580c83d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c760 tx=0x7f2a4800cc20 comp rx=0 tx=0).stop 2023-10-26T05:48:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 shutdown_connections 2023-10-26T05:48:02.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a581571b0 0x7f2a5815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a58156d30 0x7f2a5815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a3c066470 0x7f2a3c068930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 --2- 172.21.15.71:0/4076243720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a44005190 0x7f2a580c83d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 >> 172.21.15.71:0/4076243720 conn(0x7f2a580cdd80 msgr2=0x7f2a580b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 shutdown_connections 2023-10-26T05:48:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.385+0000 7f2a5f650640 1 -- 172.21.15.71:0/4076243720 wait complete. 2023-10-26T05:48:02.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:02.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f843396c640 1 -- 172.21.15.71:0/3781057569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84180047d0 msgr2=0x7f8418004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f843396c640 1 --2- 172.21.15.71:0/3781057569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84180047d0 0x7f8418004c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8420005fd0 tx=0x7f842001cd40 comp rx=0 tx=0).stop 2023-10-26T05:48:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f8431f09640 1 -- 172.21.15.71:0/3781057569 reap_dead start 2023-10-26T05:48:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f843396c640 1 -- 172.21.15.71:0/3781057569 shutdown_connections 2023-10-26T05:48:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f843396c640 1 -- 172.21.15.71:0/3781057569 >> 172.21.15.71:0/3781057569 conn(0x7f842c0aec60 msgr2=0x7f842c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f843396c640 1 -- 172.21.15.71:0/3781057569 shutdown_connections 2023-10-26T05:48:02.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.529+0000 7f843396c640 1 -- 172.21.15.71:0/3781057569 wait complete. 2023-10-26T05:48:02.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 Processor -- start 2023-10-26T05:48:02.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- start start 2023-10-26T05:48:02.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84180047d0 0x7f842c13c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f842c0b0790 0x7f842c1354c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 0x7f842c135a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c13c990 con 0x7f842c0b0c10 2023-10-26T05:48:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c156700 con 0x7f84180047d0 2023-10-26T05:48:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c1569e0 con 0x7f842c0b0790 2023-10-26T05:48:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 0x7f842c135a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 0x7f842c135a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37846/0 (socket says 172.21.15.71:37846) 2023-10-26T05:48:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 -- 172.21.15.71:0/752812282 learned_addr learned my addr 172.21.15.71:0/752812282 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431708640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84180047d0 0x7f842c13c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8430f07640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f842c0b0790 0x7f842c1354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 -- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f842c0b0790 msgr2=0x7f842c1354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f842c0b0790 0x7f842c1354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 -- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84180047d0 msgr2=0x7f842c13c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84180047d0 0x7f842c13c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 -- 172.21.15.71:0/752812282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c156cc0 con 0x7f842c0b0c10 2023-10-26T05:48:02.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431708640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84180047d0 0x7f842c13c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:02.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f8431f09640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 0x7f842c135a00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f842c06b5c0 tx=0x7f842400fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:02.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424003d90 con 0x7f842c0b0c10 2023-10-26T05:48:02.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f842400e8a0 con 0x7f842c0b0c10 2023-10-26T05:48:02.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c156fa0 con 0x7f842c0b0c10 2023-10-26T05:48:02.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400ecb0 con 0x7f842c0b0c10 2023-10-26T05:48:02.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c157ad0 con 0x7f842c0b0c10 2023-10-26T05:48:02.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c066b10 con 0x7f842c0b0c10 2023-10-26T05:48:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.533+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8424004030 con 0x7f842c0b0c10 2023-10-26T05:48:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.537+0000 7f841dffb640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8410066470 0x7f8410068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.537+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84240849a0 con 0x7f842c0b0c10 2023-10-26T05:48:02.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.537+0000 7f8431708640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8410066470 0x7f8410068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.537+0000 7f8431708640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8410066470 0x7f8410068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8428001830 tx=0x7f8428000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:02.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.541+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8424051e10 con 0x7f842c0b0c10 2023-10-26T05:48:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f842c1574c0 con 0x7f842c0b0c10 2023-10-26T05:48:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f841dffb640 1 -- 172.21.15.71:0/752812282 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f842c1574c0 con 0x7f842c0b0c10 2023-10-26T05:48:02.689 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:02.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8410066470 msgr2=0x7f8410068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8410066470 0x7f8410068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8428001830 tx=0x7f8428000ee0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 msgr2=0x7f842c135a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 0x7f842c135a00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f842c06b5c0 tx=0x7f842400fd90 comp rx=0 tx=0).stop 2023-10-26T05:48:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 shutdown_connections 2023-10-26T05:48:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8410066470 0x7f8410068930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842c0b0c10 0x7f842c135a00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f842c0b0790 0x7f842c1354c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 --2- 172.21.15.71:0/752812282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84180047d0 0x7f842c13c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 >> 172.21.15.71:0/752812282 conn(0x7f842c0aec60 msgr2=0x7f842c15a900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 shutdown_connections 2023-10-26T05:48:02.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.685+0000 7f843396c640 1 -- 172.21.15.71:0/752812282 wait complete. 2023-10-26T05:48:02.706 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:02.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/3061500990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc8003980 msgr2=0x7f1dc8003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 --2- 172.21.15.71:0/3061500990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc8003980 0x7f1dc8003d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800c700 tx=0x7f1dd801c630 comp rx=0 tx=0).stop 2023-10-26T05:48:02.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/3061500990 shutdown_connections 2023-10-26T05:48:02.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 --2- 172.21.15.71:0/3061500990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc8003980 0x7f1dc8003d80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/3061500990 >> 172.21.15.71:0/3061500990 conn(0x7f1ddc0aec60 msgr2=0x7f1ddc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/3061500990 shutdown_connections 2023-10-26T05:48:02.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/3061500990 wait complete. 2023-10-26T05:48:02.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 Processor -- start 2023-10-26T05:48:02.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- start start 2023-10-26T05:48:02.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dc8003980 0x7f1ddc139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 0x7f1ddc1e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ddc1f6750 0x7f1ddc1f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1577f0 con 0x7f1ddc1e6770 2023-10-26T05:48:02.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1f7110 con 0x7f1ddc1f6750 2023-10-26T05:48:02.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc1f3030 con 0x7f1dc8003980 2023-10-26T05:48:02.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 0x7f1ddc1e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 0x7f1ddc1e6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37868/0 (socket says 172.21.15.71:37868) 2023-10-26T05:48:02.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 -- 172.21.15.71:0/844626756 learned_addr learned my addr 172.21.15.71:0/844626756 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:02.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de12a8640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dc8003980 0x7f1ddc139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de1aa9640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ddc1f6750 0x7f1ddc1f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 -- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dc8003980 msgr2=0x7f1ddc139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dc8003980 0x7f1ddc139440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 -- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ddc1f6750 msgr2=0x7f1ddc1f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ddc1f6750 0x7f1ddc1f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 -- 172.21.15.71:0/844626756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc1e7230 con 0x7f1ddc1e6770 2023-10-26T05:48:02.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de0aa7640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 0x7f1ddc1e6bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e6a0 tx=0x7f1dcc00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:02.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc00ca50 con 0x7f1ddc1e6770 2023-10-26T05:48:02.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1dcc00cca0 con 0x7f1ddc1e6770 2023-10-26T05:48:02.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc1e7510 con 0x7f1ddc1e6770 2023-10-26T05:48:02.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0116d0 con 0x7f1ddc1e6770 2023-10-26T05:48:02.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.829+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc157410 con 0x7f1ddc1e6770 2023-10-26T05:48:02.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.833+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc066b10 con 0x7f1ddc1e6770 2023-10-26T05:48:02.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.833+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1dcc011830 con 0x7f1ddc1e6770 2023-10-26T05:48:02.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.833+0000 7f1dd1ffb640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1dc00664a0 0x7f1dc0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:02.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.833+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dcc084c50 con 0x7f1ddc1e6770 2023-10-26T05:48:02.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.833+0000 7f1de12a8640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1dc00664a0 0x7f1dc0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:02.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.833+0000 7f1de12a8640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1dc00664a0 0x7f1dc0068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80057b0 tx=0x7f1dd80193d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:02.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.837+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1dcc0521c0 con 0x7f1ddc1e6770 2023-10-26T05:48:02.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ddc0721e0 con 0x7f1ddc1e6770 2023-10-26T05:48:02.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1dd1ffb640 1 -- 172.21.15.71:0/844626756 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1dcc0557e0 con 0x7f1ddc1e6770 2023-10-26T05:48:02.981 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:02.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1dc00664a0 msgr2=0x7f1dc0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1dc00664a0 0x7f1dc0068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1dc80057b0 tx=0x7f1dd80193d0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 msgr2=0x7f1ddc1e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:02.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 0x7f1ddc1e6bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00e6a0 tx=0x7f1dcc00eb60 comp rx=0 tx=0).stop 2023-10-26T05:48:02.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 shutdown_connections 2023-10-26T05:48:02.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ddc1f6750 0x7f1ddc1f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1dc00664a0 0x7f1dc0068960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ddc1e6770 0x7f1ddc1e6bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 --2- 172.21.15.71:0/844626756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1dc8003980 0x7f1ddc139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:02.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 >> 172.21.15.71:0/844626756 conn(0x7f1ddc0aec60 msgr2=0x7f1ddc0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:02.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 shutdown_connections 2023-10-26T05:48:02.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:02.977+0000 7f1de350c640 1 -- 172.21.15.71:0/844626756 wait complete. 2023-10-26T05:48:02.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:03.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/1838320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e0070c60 msgr2=0x7fa3e00b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/1838320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e0070c60 0x7fa3e00b1dd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009840 tx=0x7fa3d001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/1838320619 shutdown_connections 2023-10-26T05:48:03.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/1838320619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e0070c60 0x7fa3e00b1dd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/1838320619 >> 172.21.15.71:0/1838320619 conn(0x7fa3e00afe40 msgr2=0x7fa3e00b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:03.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/1838320619 shutdown_connections 2023-10-26T05:48:03.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/1838320619 wait complete. 2023-10-26T05:48:03.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 Processor -- start 2023-10-26T05:48:03.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- start start 2023-10-26T05:48:03.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3e0070c60 0x7fa3e006ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 0x7fa3e0141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01429e0 0x7fa3e0138b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e006b270 con 0x7fa3e00c86f0 2023-10-26T05:48:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e006cb80 con 0x7fa3e01429e0 2023-10-26T05:48:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b2880 con 0x7fa3e0070c60 2023-10-26T05:48:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 0x7fa3e0141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 0x7fa3e0141d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37878/0 (socket says 172.21.15.71:37878) 2023-10-26T05:48:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 -- 172.21.15.71:0/2327090578 learned_addr learned my addr 172.21.15.71:0/2327090578 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:03.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 -- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3e0070c60 msgr2=0x7fa3e006ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e612d640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01429e0 0x7fa3e0138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3e0070c60 0x7fa3e006ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 -- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01429e0 msgr2=0x7fa3e0138b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01429e0 0x7fa3e0138b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 -- 172.21.15.71:0/2327090578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00094a0 con 0x7fa3e00c86f0 2023-10-26T05:48:03.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e512b640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 0x7fa3e0141d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa3d001ccb0 tx=0x7fa3d00056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:03.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00a5890 con 0x7fa3e00c86f0 2023-10-26T05:48:03.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3d00047c0 con 0x7fa3e00c86f0 2023-10-26T05:48:03.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e0142240 con 0x7fa3e00c86f0 2023-10-26T05:48:03.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004bb0 con 0x7fa3e00c86f0 2023-10-26T05:48:03.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.117+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e00b2af0 con 0x7fa3e00c86f0 2023-10-26T05:48:03.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.121+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e0066d00 con 0x7fa3e00c86f0 2023-10-26T05:48:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.121+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa3d0004d10 con 0x7fa3e00c86f0 2023-10-26T05:48:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.121+0000 7fa3d67fc640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa3c4066470 0x7fa3c4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.121+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3d011f040 con 0x7fa3e00c86f0 2023-10-26T05:48:03.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.121+0000 7fa3e592c640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa3c4066470 0x7fa3c4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.121+0000 7fa3e592c640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa3c4066470 0x7fa3c4068930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc000f70 tx=0x7fa3dc0079e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:03.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.125+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa3d00ec5e0 con 0x7fa3e00c86f0 2023-10-26T05:48:03.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.273+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3e00c9e10 con 0x7fa3e00c86f0 2023-10-26T05:48:03.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.273+0000 7fa3d67fc640 1 -- 172.21.15.71:0/2327090578 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa3d00b2020 con 0x7fa3e00c86f0 2023-10-26T05:48:03.278 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:03.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa3c4066470 msgr2=0x7fa3c4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa3c4066470 0x7fa3c4068930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc000f70 tx=0x7fa3dc0079e0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 msgr2=0x7fa3e0141d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 0x7fa3e0141d00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa3d001ccb0 tx=0x7fa3d00056b0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 shutdown_connections 2023-10-26T05:48:03.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01429e0 0x7fa3e0138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa3c4066470 0x7fa3c4068930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3e00c86f0 0x7fa3e0141d00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 --2- 172.21.15.71:0/2327090578 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3e0070c60 0x7fa3e006ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 >> 172.21.15.71:0/2327090578 conn(0x7fa3e00afe40 msgr2=0x7fa3e00cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:03.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 shutdown_connections 2023-10-26T05:48:03.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.277+0000 7fa3e7b90640 1 -- 172.21.15.71:0/2327090578 wait complete. 2023-10-26T05:48:03.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 -- 172.21.15.71:0/3664324959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813b0f0 msgr2=0x7fd1180cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3664324959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813b0f0 0x7fd1180cb8c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd108009840 tx=0x7fd10801c790 comp rx=0 tx=0).stop 2023-10-26T05:48:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 -- 172.21.15.71:0/3664324959 shutdown_connections 2023-10-26T05:48:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3664324959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813b0f0 0x7fd1180cb8c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 -- 172.21.15.71:0/3664324959 >> 172.21.15.71:0/3664324959 conn(0x7fd11806b8c0 msgr2=0x7fd11806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 -- 172.21.15.71:0/3664324959 shutdown_connections 2023-10-26T05:48:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.401+0000 7fd11d83b640 1 -- 172.21.15.71:0/3664324959 wait complete. 2023-10-26T05:48:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 Processor -- start 2023-10-26T05:48:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- start start 2023-10-26T05:48:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11813b0f0 0x7fd118132b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 0x7fd1181539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1181590b0 0x7fd11815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11815c460 con 0x7fd11813def0 2023-10-26T05:48:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd118152ee0 con 0x7fd11813b0f0 2023-10-26T05:48:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1181531c0 con 0x7fd1181590b0 2023-10-26T05:48:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 0x7fd1181539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 0x7fd1181539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37898/0 (socket says 172.21.15.71:37898) 2023-10-26T05:48:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 -- 172.21.15.71:0/3637885425 learned_addr learned my addr 172.21.15.71:0/3637885425 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd116ffd640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11813b0f0 0x7fd118132b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd1177fe640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1181590b0 0x7fd11815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 -- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1181590b0 msgr2=0x7fd11815ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1181590b0 0x7fd11815ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 -- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11813b0f0 msgr2=0x7fd118132b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11813b0f0 0x7fd118132b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 -- 172.21.15.71:0/3637885425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080094a0 con 0x7fd11813def0 2023-10-26T05:48:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd1177fe640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1181590b0 0x7fd11815ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd10ffff640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 0x7fd1181539c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd10000c410 tx=0x7fd10000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011790 con 0x7fd11813def0 2023-10-26T05:48:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1000119e0 con 0x7fd11813def0 2023-10-26T05:48:03.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd118070bf0 con 0x7fd11813def0 2023-10-26T05:48:03.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011dd0 con 0x7fd11813def0 2023-10-26T05:48:03.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd118133040 con 0x7fd11813def0 2023-10-26T05:48:03.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd118066b10 con 0x7fd11813def0 2023-10-26T05:48:03.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd10001c460 con 0x7fd11813def0 2023-10-26T05:48:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd114ff9640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0f8066470 0x7fd0f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd100084100 con 0x7fd11813def0 2023-10-26T05:48:03.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.405+0000 7fd116ffd640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0f8066470 0x7fd0f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.409+0000 7fd116ffd640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0f8066470 0x7fd0f8068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1080af000 tx=0x7fd10801afd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:03.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.409+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd1000515b0 con 0x7fd11813def0 2023-10-26T05:48:03.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.565+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd11806c7d0 con 0x7fd11813def0 2023-10-26T05:48:03.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.565+0000 7fd114ff9640 1 -- 172.21.15.71:0/3637885425 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd100054bd0 con 0x7fd11813def0 2023-10-26T05:48:03.572 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:03.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0f8066470 msgr2=0x7fd0f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0f8066470 0x7fd0f8068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd1080af000 tx=0x7fd10801afd0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 msgr2=0x7fd1181539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 0x7fd1181539c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd10000c410 tx=0x7fd10000c8d0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 shutdown_connections 2023-10-26T05:48:03.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1181590b0 0x7fd11815ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0f8066470 0x7fd0f8068930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd11813def0 0x7fd1181539c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 --2- 172.21.15.71:0/3637885425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd11813b0f0 0x7fd118132b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 >> 172.21.15.71:0/3637885425 conn(0x7fd11806b8c0 msgr2=0x7fd1180ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:03.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 shutdown_connections 2023-10-26T05:48:03.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.569+0000 7fd11d83b640 1 -- 172.21.15.71:0/3637885425 wait complete. 2023-10-26T05:48:03.588 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:48:03.589 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:03.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:03.656 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[33767]: fuse finished with error 0 and tester_r 0 2023-10-26T05:48:03.656 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:03.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:48:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:03.664 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:03.672 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:48:03.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:03.673 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:48:03.728 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:03.728 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:03.734 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:03.735 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:03.780 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:03.824 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi071.stderr:2023-10-26T05:48:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:03.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 -- 172.21.15.71:0/3352498184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 msgr2=0x7f7a7c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 --2- 172.21.15.71:0/3352498184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a7c004c50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009eb0 tx=0x7f7a8001d120 comp rx=0 tx=0).stop 2023-10-26T05:48:03.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 -- 172.21.15.71:0/3352498184 shutdown_connections 2023-10-26T05:48:03.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 --2- 172.21.15.71:0/3352498184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a7c004c50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 -- 172.21.15.71:0/3352498184 >> 172.21.15.71:0/3352498184 conn(0x7f7a900cdd80 msgr2=0x7f7a900ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:03.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 -- 172.21.15.71:0/3352498184 shutdown_connections 2023-10-26T05:48:03.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.945+0000 7f7a96258640 1 -- 172.21.15.71:0/3352498184 wait complete. 2023-10-26T05:48:03.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 Processor -- start 2023-10-26T05:48:03.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- start start 2023-10-26T05:48:03.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a901f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a900cac20 0x7f7a900cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a900c64f0 0x7f7a900c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9006a650 con 0x7f7a7c0047d0 2023-10-26T05:48:03.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900cb5e0 con 0x7f7a900cac20 2023-10-26T05:48:03.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a900c6eb0 con 0x7f7a900c64f0 2023-10-26T05:48:03.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a901f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a901f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37918/0 (socket says 172.21.15.71:37918) 2023-10-26T05:48:03.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 -- 172.21.15.71:0/2556249725 learned_addr learned my addr 172.21.15.71:0/2556249725 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:03.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8effd640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a900cac20 0x7f7a900cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 -- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a900c64f0 msgr2=0x7f7a900c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8ffff640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a900c64f0 0x7f7a900c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a900c64f0 0x7f7a900c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 -- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a900cac20 msgr2=0x7f7a900cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:03.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a900cac20 0x7f7a900cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:03.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 -- 172.21.15.71:0/2556249725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a900c4bb0 con 0x7f7a7c0047d0 2023-10-26T05:48:03.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a8f7fe640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a901f4ce0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a78001060 tx=0x7f7a7800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:03.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800c970 con 0x7f7a7c0047d0 2023-10-26T05:48:03.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a7800cbe0 con 0x7f7a7c0047d0 2023-10-26T05:48:03.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78011460 con 0x7f7a7c0047d0 2023-10-26T05:48:03.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a900c7140 con 0x7f7a7c0047d0 2023-10-26T05:48:03.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a900c7700 con 0x7f7a7c0047d0 2023-10-26T05:48:03.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.949+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a90066b10 con 0x7f7a7c0047d0 2023-10-26T05:48:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.953+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7a780115e0 con 0x7f7a7c0047d0 2023-10-26T05:48:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.953+0000 7f7a87fff640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a70066470 0x7f7a70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.953+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a780846e0 con 0x7f7a7c0047d0 2023-10-26T05:48:03.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.953+0000 7f7a8effd640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a70066470 0x7f7a70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:03.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.953+0000 7f7a8effd640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a70066470 0x7f7a70068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a800ae000 tx=0x7f7a80019ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:03.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:03.957+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7a78051b60 con 0x7f7a7c0047d0 2023-10-26T05:48:04.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.097+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a900721e0 con 0x7f7a7c0047d0 2023-10-26T05:48:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a87fff640 1 -- 172.21.15.71:0/2556249725 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f7a7801f090 con 0x7f7a7c0047d0 2023-10-26T05:48:04.105 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:04.105 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a70066470 msgr2=0x7f7a70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a70066470 0x7f7a70068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a800ae000 tx=0x7f7a80019ec0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 msgr2=0x7f7a901f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a901f4ce0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7a78001060 tx=0x7f7a7800ea90 comp rx=0 tx=0).stop 2023-10-26T05:48:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 shutdown_connections 2023-10-26T05:48:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a900c64f0 0x7f7a900c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a900cac20 0x7f7a900cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7a70066470 0x7f7a70068930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 --2- 172.21.15.71:0/2556249725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a7c0047d0 0x7f7a901f4ce0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 >> 172.21.15.71:0/2556249725 conn(0x7f7a900cdd80 msgr2=0x7f7a9015a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 shutdown_connections 2023-10-26T05:48:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.101+0000 7f7a96258640 1 -- 172.21.15.71:0/2556249725 wait complete. 2023-10-26T05:48:04.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:04.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:04.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/51023026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84480047d0 msgr2=0x7f8448004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 --2- 172.21.15.71:0/51023026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84480047d0 0x7f8448004c50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f845400c700 tx=0x7f845401c650 comp rx=0 tx=0).stop 2023-10-26T05:48:04.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/51023026 shutdown_connections 2023-10-26T05:48:04.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 --2- 172.21.15.71:0/51023026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84480047d0 0x7f8448004c50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/51023026 >> 172.21.15.71:0/51023026 conn(0x7f84600cdd80 msgr2=0x7f84600ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/51023026 shutdown_connections 2023-10-26T05:48:04.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/51023026 wait complete. 2023-10-26T05:48:04.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 Processor -- start 2023-10-26T05:48:04.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- start start 2023-10-26T05:48:04.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84480047d0 0x7f846015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846014f5b0 0x7f84600c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 0x7f84600c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846006a650 con 0x7f84600c8d70 2023-10-26T05:48:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846015bbd0 con 0x7f846014f5b0 2023-10-26T05:48:04.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846015e280 con 0x7f84480047d0 2023-10-26T05:48:04.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 0x7f84600c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 0x7f84600c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37940/0 (socket says 172.21.15.71:37940) 2023-10-26T05:48:04.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 -- 172.21.15.71:0/3052907965 learned_addr learned my addr 172.21.15.71:0/3052907965 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:04.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845f7fe640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84480047d0 0x7f846015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845effd640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846014f5b0 0x7f84600c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 -- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84480047d0 msgr2=0x7f846015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84480047d0 0x7f846015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 -- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846014f5b0 msgr2=0x7f84600c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846014f5b0 0x7f84600c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 -- 172.21.15.71:0/3052907965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84600c7740 con 0x7f84600c8d70 2023-10-26T05:48:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845effd640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846014f5b0 0x7f84600c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f845ffff640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 0x7f84600c9220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8458002090 tx=0x7f845800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:04.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845800ca20 con 0x7f84600c8d70 2023-10-26T05:48:04.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f845800cc90 con 0x7f84600c8d70 2023-10-26T05:48:04.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458011460 con 0x7f84600c8d70 2023-10-26T05:48:04.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846015b230 con 0x7f84600c8d70 2023-10-26T05:48:04.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.245+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846015b7f0 con 0x7f84600c8d70 2023-10-26T05:48:04.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.249+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8460066b10 con 0x7f84600c8d70 2023-10-26T05:48:04.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.249+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f84580115c0 con 0x7f84600c8d70 2023-10-26T05:48:04.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.249+0000 7f8447fff640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f844c0664a0 0x7f844c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.249+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8458084690 con 0x7f84600c8d70 2023-10-26T05:48:04.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.249+0000 7f845f7fe640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f844c0664a0 0x7f844c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.249+0000 7f845f7fe640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f844c0664a0 0x7f844c068960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f84500058d0 tx=0x7f8450005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:04.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.253+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8458055500 con 0x7f84600c8d70 2023-10-26T05:48:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.393+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f846015bd10 con 0x7f84600c8d70 2023-10-26T05:48:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8447fff640 1 -- 172.21.15.71:0/3052907965 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f846015bd10 con 0x7f84600c8d70 2023-10-26T05:48:04.400 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f844c0664a0 msgr2=0x7f844c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f844c0664a0 0x7f844c068960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f84500058d0 tx=0x7f8450005860 comp rx=0 tx=0).stop 2023-10-26T05:48:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 msgr2=0x7f84600c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 0x7f84600c9220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8458002090 tx=0x7f845800ed10 comp rx=0 tx=0).stop 2023-10-26T05:48:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 shutdown_connections 2023-10-26T05:48:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f844c0664a0 0x7f844c068960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84600c8d70 0x7f84600c9220 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f846014f5b0 0x7f84600c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 --2- 172.21.15.71:0/3052907965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84480047d0 0x7f846015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 >> 172.21.15.71:0/3052907965 conn(0x7f84600cdd80 msgr2=0x7f84600b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 shutdown_connections 2023-10-26T05:48:04.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.397+0000 7f8465b64640 1 -- 172.21.15.71:0/3052907965 wait complete. 2023-10-26T05:48:04.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:04.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/964587912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 msgr2=0x7f45b40c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/964587912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b40c9970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45a400a950 tx=0x7f45a401dae0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/964587912 shutdown_connections 2023-10-26T05:48:04.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/964587912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b40c9970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/964587912 >> 172.21.15.71:0/964587912 conn(0x7f45b40aec60 msgr2=0x7f45b40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/964587912 shutdown_connections 2023-10-26T05:48:04.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/964587912 wait complete. 2023-10-26T05:48:04.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 Processor -- start 2023-10-26T05:48:04.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- start start 2023-10-26T05:48:04.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b414d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b4070de0 0x7f45b41f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45b40c9eb0 0x7f45b4144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4145650 con 0x7f45b4069770 2023-10-26T05:48:04.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4145930 con 0x7f45b4070de0 2023-10-26T05:48:04.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b4145c10 con 0x7f45b40c9eb0 2023-10-26T05:48:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b414d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b414d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37956/0 (socket says 172.21.15.71:37956) 2023-10-26T05:48:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 -- 172.21.15.71:0/2597006030 learned_addr learned my addr 172.21.15.71:0/2597006030 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba94d640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45b40c9eb0 0x7f45b4144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45b994b640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b4070de0 0x7f45b41f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 -- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45b40c9eb0 msgr2=0x7f45b4144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45b40c9eb0 0x7f45b4144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 -- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b4070de0 msgr2=0x7f45b41f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b4070de0 0x7f45b41f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 -- 172.21.15.71:0/2597006030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a400a5b0 con 0x7f45b4069770 2023-10-26T05:48:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba94d640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45b40c9eb0 0x7f45b4144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45ba14c640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b414d5f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f45a4000c10 tx=0x7f45a40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40048d0 con 0x7f45b4069770 2023-10-26T05:48:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45a4004b20 con 0x7f45b4069770 2023-10-26T05:48:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b4145ef0 con 0x7f45b4069770 2023-10-26T05:48:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40ae440 con 0x7f45b4069770 2023-10-26T05:48:04.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.537+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b4143e60 con 0x7f45b4069770 2023-10-26T05:48:04.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.541+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b4066b10 con 0x7f45b4069770 2023-10-26T05:48:04.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.541+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45a40b5040 con 0x7f45b4069770 2023-10-26T05:48:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.541+0000 7f45aaffd640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4598066470 0x7f4598068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.541+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45a411e280 con 0x7f45b4069770 2023-10-26T05:48:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.541+0000 7f45b994b640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4598066470 0x7f4598068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.541+0000 7f45b994b640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4598066470 0x7f4598068930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45b4144210 tx=0x7f45b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.545+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f45a40eb700 con 0x7f45b4069770 2023-10-26T05:48:04.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.685+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45b4144050 con 0x7f45b4069770 2023-10-26T05:48:04.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.685+0000 7f45aaffd640 1 -- 172.21.15.71:0/2597006030 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45b4144050 con 0x7f45b4069770 2023-10-26T05:48:04.690 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:04.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4598066470 msgr2=0x7f4598068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4598066470 0x7f4598068930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f45b4144210 tx=0x7f45b0009040 comp rx=0 tx=0).stop 2023-10-26T05:48:04.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 msgr2=0x7f45b414d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b414d5f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f45a4000c10 tx=0x7f45a40056d0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 shutdown_connections 2023-10-26T05:48:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45b40c9eb0 0x7f45b4144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b4070de0 0x7f45b41f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4598066470 0x7f4598068930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 --2- 172.21.15.71:0/2597006030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45b4069770 0x7f45b414d5f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 >> 172.21.15.71:0/2597006030 conn(0x7f45b40aec60 msgr2=0x7f45b4156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 shutdown_connections 2023-10-26T05:48:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.689+0000 7f45bc3b0640 1 -- 172.21.15.71:0/2597006030 wait complete. 2023-10-26T05:48:04.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:04.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- 172.21.15.71:0/2049503273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bb90 msgr2=0x7f79780b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 --2- 172.21.15.71:0/2049503273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bb90 0x7f79780b0fa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7968009840 tx=0x7f796801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- 172.21.15.71:0/2049503273 shutdown_connections 2023-10-26T05:48:04.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 --2- 172.21.15.71:0/2049503273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7978070dc0 0x7f797806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 --2- 172.21.15.71:0/2049503273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bb90 0x7f79780b0fa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- 172.21.15.71:0/2049503273 >> 172.21.15.71:0/2049503273 conn(0x7f79780aec60 msgr2=0x7f79780af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- 172.21.15.71:0/2049503273 shutdown_connections 2023-10-26T05:48:04.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- 172.21.15.71:0/2049503273 wait complete. 2023-10-26T05:48:04.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 Processor -- start 2023-10-26T05:48:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- start start 2023-10-26T05:48:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7978070dc0 0x7f7978140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 0x7f79781f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bf70 0x7f79781f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797813e1f0 con 0x7f797815bb90 2023-10-26T05:48:04.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797813e4d0 con 0x7f797815bf70 2023-10-26T05:48:04.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.829+0000 7f797d50c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797813e7b0 con 0x7f7978070dc0 2023-10-26T05:48:04.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 0x7f79781f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 0x7f79781f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37962/0 (socket says 172.21.15.71:37962) 2023-10-26T05:48:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 -- 172.21.15.71:0/1386636408 learned_addr learned my addr 172.21.15.71:0/1386636408 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f7976ffd640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7978070dc0 0x7f7978140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79777fe640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bf70 0x7f79781f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 -- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7978070dc0 msgr2=0x7f7978140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7978070dc0 0x7f7978140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 -- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bf70 msgr2=0x7f79781f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bf70 0x7f79781f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 -- 172.21.15.71:0/1386636408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680094a0 con 0x7f797815bb90 2023-10-26T05:48:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f79767fc640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 0x7f79781f2590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f796c00e9f0 tx=0x7f796c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c009710 con 0x7f797815bb90 2023-10-26T05:48:04.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f796c00ce70 con 0x7f797815bb90 2023-10-26T05:48:04.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797813ea90 con 0x7f797815bb90 2023-10-26T05:48:04.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c010690 con 0x7f797815bb90 2023-10-26T05:48:04.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7978140740 con 0x7f797815bb90 2023-10-26T05:48:04.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7978066b10 con 0x7f797815bb90 2023-10-26T05:48:04.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f796c0107f0 con 0x7f797815bb90 2023-10-26T05:48:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f795f7fe640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7964066470 0x7f7964068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f796c014070 con 0x7f797815bb90 2023-10-26T05:48:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f7976ffd640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7964066470 0x7f7964068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.833+0000 7f7976ffd640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7964066470 0x7f7964068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79680af000 tx=0x7f796801b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:04.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.837+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f796c050090 con 0x7f797815bb90 2023-10-26T05:48:04.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79780721e0 con 0x7f797815bb90 2023-10-26T05:48:04.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f795f7fe640 1 -- 172.21.15.71:0/1386636408 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f796c055a50 con 0x7f797815bb90 2023-10-26T05:48:04.985 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:04.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7964066470 msgr2=0x7f7964068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7964066470 0x7f7964068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f79680af000 tx=0x7f796801b010 comp rx=0 tx=0).stop 2023-10-26T05:48:04.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 msgr2=0x7f79781f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 0x7f79781f2590 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f796c00e9f0 tx=0x7f796c00eeb0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 shutdown_connections 2023-10-26T05:48:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f797815bf70 0x7f79781f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7964066470 0x7f7964068930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797815bb90 0x7f79781f2590 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 --2- 172.21.15.71:0/1386636408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7978070dc0 0x7f7978140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:04.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 >> 172.21.15.71:0/1386636408 conn(0x7f79780aec60 msgr2=0x7f79780adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:04.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 shutdown_connections 2023-10-26T05:48:04.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:04.981+0000 7f797d50c640 1 -- 172.21.15.71:0/1386636408 wait complete. 2023-10-26T05:48:05.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 -- 172.21.15.71:0/1740159382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e4003980 msgr2=0x7f11e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 --2- 172.21.15.71:0/1740159382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e4003980 0x7f11e4003da0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11f400c260 tx=0x7f11f401cf10 comp rx=0 tx=0).stop 2023-10-26T05:48:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 -- 172.21.15.71:0/1740159382 shutdown_connections 2023-10-26T05:48:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 --2- 172.21.15.71:0/1740159382 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11e4003980 0x7f11e4003da0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 -- 172.21.15.71:0/1740159382 >> 172.21.15.71:0/1740159382 conn(0x7f11f806b8c0 msgr2=0x7f11f806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:05.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 -- 172.21.15.71:0/1740159382 shutdown_connections 2023-10-26T05:48:05.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 -- 172.21.15.71:0/1740159382 wait complete. 2023-10-26T05:48:05.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 Processor -- start 2023-10-26T05:48:05.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.121+0000 7f11ff63c640 1 -- start start 2023-10-26T05:48:05.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e4003980 0x7f11f813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f8149050 0x7f11f8153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 0x7f11f8155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8152f20 con 0x7f11f8150e00 2023-10-26T05:48:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8153200 con 0x7f11e4003980 2023-10-26T05:48:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81534e0 con 0x7f11f8149050 2023-10-26T05:48:05.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 0x7f11f8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 0x7f11f8155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37976/0 (socket says 172.21.15.71:37976) 2023-10-26T05:48:05.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 -- 172.21.15.71:0/2138054610 learned_addr learned my addr 172.21.15.71:0/2138054610 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:05.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fcbd7640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f8149050 0x7f11f8153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fd3d8640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e4003980 0x7f11f813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 -- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f8149050 msgr2=0x7f11f8153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f8149050 0x7f11f8153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 -- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e4003980 msgr2=0x7f11f813d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e4003980 0x7f11f813d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 -- 172.21.15.71:0/2138054610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f81537c0 con 0x7f11f8150e00 2023-10-26T05:48:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fdbd9640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 0x7f11f8155d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11f80b2630 tx=0x7f11f000fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0003d90 con 0x7f11f8150e00 2023-10-26T05:48:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11f000d8d0 con 0x7f11f8150e00 2023-10-26T05:48:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f8153aa0 con 0x7f11f8150e00 2023-10-26T05:48:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000dce0 con 0x7f11f8150e00 2023-10-26T05:48:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f8151480 con 0x7f11f8150e00 2023-10-26T05:48:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f8066b10 con 0x7f11f8150e00 2023-10-26T05:48:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11f0004030 con 0x7f11f8150e00 2023-10-26T05:48:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11edffb640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11dc066470 0x7f11dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11f00848d0 con 0x7f11f8150e00 2023-10-26T05:48:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.125+0000 7f11fd3d8640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11dc066470 0x7f11dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.129+0000 7f11fd3d8640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11dc066470 0x7f11dc068930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ac000 tx=0x7f11f40a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.129+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11f0051e00 con 0x7f11f8150e00 2023-10-26T05:48:05.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.281+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11f806c7d0 con 0x7f11f8150e00 2023-10-26T05:48:05.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.281+0000 7f11edffb640 1 -- 172.21.15.71:0/2138054610 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11f0021080 con 0x7f11f8150e00 2023-10-26T05:48:05.287 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:05.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11dc066470 msgr2=0x7f11dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11dc066470 0x7f11dc068930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f11f40ac000 tx=0x7f11f40a7040 comp rx=0 tx=0).stop 2023-10-26T05:48:05.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 msgr2=0x7f11f8155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 0x7f11f8155d50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11f80b2630 tx=0x7f11f000fe20 comp rx=0 tx=0).stop 2023-10-26T05:48:05.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 shutdown_connections 2023-10-26T05:48:05.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11dc066470 0x7f11dc068930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11f8150e00 0x7f11f8155d50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f8149050 0x7f11f8153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 --2- 172.21.15.71:0/2138054610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11e4003980 0x7f11f813d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 >> 172.21.15.71:0/2138054610 conn(0x7f11f806b8c0 msgr2=0x7f11f806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 shutdown_connections 2023-10-26T05:48:05.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.285+0000 7f11ff63c640 1 -- 172.21.15.71:0/2138054610 wait complete. 2023-10-26T05:48:05.305 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:05.306 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:05.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:05.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2692991474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6e40039b0 msgr2=0x7fa6e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2692991474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6e40039b0 0x7fa6e4003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001680 tx=0x7fa6e801c760 comp rx=0 tx=0).stop 2023-10-26T05:48:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6f7fff640 1 -- 172.21.15.71:0/2692991474 reap_dead start 2023-10-26T05:48:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2692991474 shutdown_connections 2023-10-26T05:48:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2692991474 >> 172.21.15.71:0/2692991474 conn(0x7fa6f8069770 msgr2=0x7fa6f8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:05.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2692991474 shutdown_connections 2023-10-26T05:48:05.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2692991474 wait complete. 2023-10-26T05:48:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 Processor -- start 2023-10-26T05:48:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- start start 2023-10-26T05:48:05.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6e40039b0 0x7fa6f81368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f8150930 0x7fa6f813fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 0x7fa6f81588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8158e00 con 0x7fa6f8158440 2023-10-26T05:48:05.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8150e60 con 0x7fa6f8150930 2023-10-26T05:48:05.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.473+0000 7fa6fdf10640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8151140 con 0x7fa6e40039b0 2023-10-26T05:48:05.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 0x7fa6f81588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 0x7fa6f81588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38000/0 (socket says 172.21.15.71:38000) 2023-10-26T05:48:05.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 -- 172.21.15.71:0/2294777198 learned_addr learned my addr 172.21.15.71:0/2294777198 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:05.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f77fe640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6e40039b0 0x7fa6f81368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6effff640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f8150930 0x7fa6f813fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 -- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6e40039b0 msgr2=0x7fa6f81368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6e40039b0 0x7fa6f81368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 -- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f8150930 msgr2=0x7fa6f813fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f8150930 0x7fa6f813fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 -- 172.21.15.71:0/2294777198 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8151420 con 0x7fa6f8158440 2023-10-26T05:48:05.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f77fe640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6e40039b0 0x7fa6f81368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:05.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6effff640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f8150930 0x7fa6f813fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:05.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f7fff640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 0x7fa6f81588c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80afd40 tx=0x7fa6f000ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:05.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0018070 con 0x7fa6f8158440 2023-10-26T05:48:05.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa6f0004c20 con 0x7fa6f8158440 2023-10-26T05:48:05.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f8151700 con 0x7fa6f8158440 2023-10-26T05:48:05.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000cad0 con 0x7fa6f8158440 2023-10-26T05:48:05.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f8136e20 con 0x7fa6f8158440 2023-10-26T05:48:05.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8066b10 con 0x7fa6f8158440 2023-10-26T05:48:05.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa6f000cc30 con 0x7fa6f8158440 2023-10-26T05:48:05.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f4ff9640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa6d8066470 0x7fa6d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6f0083b50 con 0x7fa6f8158440 2023-10-26T05:48:05.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f77fe640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa6d8066470 0x7fa6d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.477+0000 7fa6f77fe640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa6d8066470 0x7fa6d8068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40057b0 tx=0x7fa6e8019500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:05.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.481+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa6f0050fd0 con 0x7fa6f8158440 2023-10-26T05:48:05.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.625+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6f80721e0 con 0x7fa6f8158440 2023-10-26T05:48:05.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.625+0000 7fa6f4ff9640 1 -- 172.21.15.71:0/2294777198 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fa6f00545f0 con 0x7fa6f8158440 2023-10-26T05:48:05.632 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:05.632 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:05.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa6d8066470 msgr2=0x7fa6d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa6d8066470 0x7fa6d8068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40057b0 tx=0x7fa6e8019500 comp rx=0 tx=0).stop 2023-10-26T05:48:05.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 msgr2=0x7fa6f81588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 0x7fa6f81588c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80afd40 tx=0x7fa6f000ee10 comp rx=0 tx=0).stop 2023-10-26T05:48:05.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 shutdown_connections 2023-10-26T05:48:05.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa6d8066470 0x7fa6d8068930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6f8158440 0x7fa6f81588c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6f8150930 0x7fa6f813fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 --2- 172.21.15.71:0/2294777198 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa6e40039b0 0x7fa6f81368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 >> 172.21.15.71:0/2294777198 conn(0x7fa6f8069770 msgr2=0x7fa6f80acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:05.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 shutdown_connections 2023-10-26T05:48:05.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.629+0000 7fa6fdf10640 1 -- 172.21.15.71:0/2294777198 wait complete. 2023-10-26T05:48:05.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:05.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:05.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/1971538616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0470 msgr2=0x7fbe9406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 --2- 172.21.15.71:0/1971538616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0470 0x7fbe9406aa70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbe9408a140 tx=0x7fbe9001c780 comp rx=0 tx=0).stop 2023-10-26T05:48:05.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/1971538616 shutdown_connections 2023-10-26T05:48:05.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 --2- 172.21.15.71:0/1971538616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe940cd880 0x7fbe9415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 --2- 172.21.15.71:0/1971538616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0470 0x7fbe9406aa70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/1971538616 >> 172.21.15.71:0/1971538616 conn(0x7fbe940cdd80 msgr2=0x7fbe940ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:05.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/1971538616 shutdown_connections 2023-10-26T05:48:05.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/1971538616 wait complete. 2023-10-26T05:48:05.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 Processor -- start 2023-10-26T05:48:05.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- start start 2023-10-26T05:48:05.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd880 0x7fbe94151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe940d0470 0x7fbe94153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 0x7fbe9415d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94153d00 con 0x7fbe940d0850 2023-10-26T05:48:05.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94153fe0 con 0x7fbe940cd880 2023-10-26T05:48:05.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941542c0 con 0x7fbe940d0470 2023-10-26T05:48:05.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 0x7fbe9415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 0x7fbe9415d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38010/0 (socket says 172.21.15.71:38010) 2023-10-26T05:48:05.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 -- 172.21.15.71:0/3974930709 learned_addr learned my addr 172.21.15.71:0/3974930709 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:05.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe996e3640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd880 0x7fbe94151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 -- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe940d0470 msgr2=0x7fbe94153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe98ee2640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe940d0470 0x7fbe94153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe940d0470 0x7fbe94153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 -- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd880 msgr2=0x7fbe94151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd880 0x7fbe94151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 -- 172.21.15.71:0/3974930709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90009560 con 0x7fbe940d0850 2023-10-26T05:48:05.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe996e3640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd880 0x7fbe94151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:05.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe99ee4640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 0x7fbe9415d600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00ea20 tx=0x7fbe8c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:05.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00cde0 con 0x7fbe940d0850 2023-10-26T05:48:05.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe8c004c40 con 0x7fbe940d0850 2023-10-26T05:48:05.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c005020 con 0x7fbe940d0850 2023-10-26T05:48:05.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe941545a0 con 0x7fbe940d0850 2023-10-26T05:48:05.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.769+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe94154da0 con 0x7fbe940d0850 2023-10-26T05:48:05.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.773+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe94066b10 con 0x7fbe940d0850 2023-10-26T05:48:05.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.773+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe8c005180 con 0x7fbe940d0850 2023-10-26T05:48:05.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.773+0000 7fbe89ffb640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe78066470 0x7fbe78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:05.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.773+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe8c0848b0 con 0x7fbe940d0850 2023-10-26T05:48:05.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.773+0000 7fbe996e3640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe78066470 0x7fbe78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.773+0000 7fbe996e3640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe78066470 0x7fbe78068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe900af000 tx=0x7fbe9001afc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:05.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.777+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe8c051d30 con 0x7fbe940d0850 2023-10-26T05:48:05.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.925+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbe940721e0 con 0x7fbe940d0850 2023-10-26T05:48:05.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe89ffb640 1 -- 172.21.15.71:0/3974930709 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe8c01f070 con 0x7fbe940d0850 2023-10-26T05:48:05.932 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:05.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe78066470 msgr2=0x7fbe78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe78066470 0x7fbe78068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe900af000 tx=0x7fbe9001afc0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 msgr2=0x7fbe9415d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:05.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 0x7fbe9415d600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00ea20 tx=0x7fbe8c00eee0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 shutdown_connections 2023-10-26T05:48:05.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe78066470 0x7fbe78068930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe940d0850 0x7fbe9415d600 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe940d0470 0x7fbe94153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 --2- 172.21.15.71:0/3974930709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd880 0x7fbe94151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:05.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 >> 172.21.15.71:0/3974930709 conn(0x7fbe940cdd80 msgr2=0x7fbe940cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:05.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 shutdown_connections 2023-10-26T05:48:05.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:05.929+0000 7fbe9b947640 1 -- 172.21.15.71:0/3974930709 wait complete. 2023-10-26T05:48:05.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:06.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- 172.21.15.71:0/3882939034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1300047d0 msgr2=0x7fd130004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 --2- 172.21.15.71:0/3882939034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1300047d0 0x7fd130004c50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd13400bbb0 tx=0x7fd13401cdc0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- 172.21.15.71:0/3882939034 shutdown_connections 2023-10-26T05:48:06.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 --2- 172.21.15.71:0/3882939034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1300047d0 0x7fd130004c50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- 172.21.15.71:0/3882939034 >> 172.21.15.71:0/3882939034 conn(0x7fd1440aec60 msgr2=0x7fd1440af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:06.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- 172.21.15.71:0/3882939034 shutdown_connections 2023-10-26T05:48:06.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- 172.21.15.71:0/3882939034 wait complete. 2023-10-26T05:48:06.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 Processor -- start 2023-10-26T05:48:06.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- start start 2023-10-26T05:48:06.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1300047d0 0x7fd1441f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd144140230 0x7fd14413dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 0x7fd14413e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd144140c90 con 0x7fd144138130 2023-10-26T05:48:06.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1440b0ee0 con 0x7fd1300047d0 2023-10-26T05:48:06.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14c104640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441406d0 con 0x7fd144140230 2023-10-26T05:48:06.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 0x7fd14413e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 0x7fd14413e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38032/0 (socket says 172.21.15.71:38032) 2023-10-26T05:48:06.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 -- 172.21.15.71:0/3778617587 learned_addr learned my addr 172.21.15.71:0/3778617587 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:06.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14969f640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd144140230 0x7fd14413dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd149ea0640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1300047d0 0x7fd1441f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 -- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd144140230 msgr2=0x7fd14413dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd144140230 0x7fd14413dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 -- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1300047d0 msgr2=0x7fd1441f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1300047d0 0x7fd1441f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14a6a1640 1 -- 172.21.15.71:0/3778617587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1441409b0 con 0x7fd144138130 2023-10-26T05:48:06.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd14969f640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd144140230 0x7fd14413dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:06.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.069+0000 7fd149ea0640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1300047d0 0x7fd1441f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:06.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd14a6a1640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 0x7fd14413e1f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd14415a7d0 tx=0x7fd13c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:06.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c011870 con 0x7fd144138130 2023-10-26T05:48:06.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd13c002100 con 0x7fd144138130 2023-10-26T05:48:06.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1440b0b70 con 0x7fd144138130 2023-10-26T05:48:06.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c011d80 con 0x7fd144138130 2023-10-26T05:48:06.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14413f340 con 0x7fd144138130 2023-10-26T05:48:06.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd144066b10 con 0x7fd144138130 2023-10-26T05:48:06.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd13c01a400 con 0x7fd144138130 2023-10-26T05:48:06.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd13a7fc640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd128066470 0x7fd128068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd13c084ac0 con 0x7fd144138130 2023-10-26T05:48:06.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd149ea0640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd128066470 0x7fd128068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.073+0000 7fd149ea0640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd128066470 0x7fd128068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1400046e0 tx=0x7fd140004610 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:06.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.077+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd13c051f50 con 0x7fd144138130 2023-10-26T05:48:06.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.217+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1440b1070 con 0x7fd144138130 2023-10-26T05:48:06.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.217+0000 7fd13a7fc640 1 -- 172.21.15.71:0/3778617587 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd13c015020 con 0x7fd144138130 2023-10-26T05:48:06.222 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:06.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd128066470 msgr2=0x7fd128068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd128066470 0x7fd128068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1400046e0 tx=0x7fd140004610 comp rx=0 tx=0).stop 2023-10-26T05:48:06.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 msgr2=0x7fd14413e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 0x7fd14413e1f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd14415a7d0 tx=0x7fd13c0019c0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 shutdown_connections 2023-10-26T05:48:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd128066470 0x7fd128068930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd144138130 0x7fd14413e1f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd144140230 0x7fd14413dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 --2- 172.21.15.71:0/3778617587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1300047d0 0x7fd1441f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 >> 172.21.15.71:0/3778617587 conn(0x7fd1440aec60 msgr2=0x7fd14415b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 shutdown_connections 2023-10-26T05:48:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.221+0000 7fd14c104640 1 -- 172.21.15.71:0/3778617587 wait complete. 2023-10-26T05:48:06.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:06.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- 172.21.15.71:0/392824624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf2c004d10 msgr2=0x7fcf2c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 --2- 172.21.15.71:0/392824624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf2c004d10 0x7fcf2c005190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009ee0 tx=0x7fcf3801d5b0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 -- 172.21.15.71:0/392824624 reap_dead start 2023-10-26T05:48:06.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- 172.21.15.71:0/392824624 shutdown_connections 2023-10-26T05:48:06.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- 172.21.15.71:0/392824624 >> 172.21.15.71:0/392824624 conn(0x7fcf400aec60 msgr2=0x7fcf400af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:06.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- 172.21.15.71:0/392824624 shutdown_connections 2023-10-26T05:48:06.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- 172.21.15.71:0/392824624 wait complete. 2023-10-26T05:48:06.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 Processor -- start 2023-10-26T05:48:06.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- start start 2023-10-26T05:48:06.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf40157a20 0x7fcf4013d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf4013d910 0x7fcf401e5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 0x7fcf401585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4013dd90 con 0x7fcf401e5790 2023-10-26T05:48:06.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401e5c10 con 0x7fcf40157a20 2023-10-26T05:48:06.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf480e3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40140650 con 0x7fcf4013d910 2023-10-26T05:48:06.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 0x7fcf401585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 0x7fcf401585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38046/0 (socket says 172.21.15.71:38046) 2023-10-26T05:48:06.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 -- 172.21.15.71:0/2779757666 learned_addr learned my addr 172.21.15.71:0/2779757666 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:06.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf45e7f640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf40157a20 0x7fcf4013d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf4567e640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf4013d910 0x7fcf401e5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 -- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf4013d910 msgr2=0x7fcf401e5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf4013d910 0x7fcf401e5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 -- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf40157a20 msgr2=0x7fcf4013d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf40157a20 0x7fcf4013d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf46680640 1 -- 172.21.15.71:0/2779757666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf401e32a0 con 0x7fcf401e5790 2023-10-26T05:48:06.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.361+0000 7fcf45e7f640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf40157a20 0x7fcf4013d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:06.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf46680640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 0x7fcf401585f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf3801d020 tx=0x7fcf38004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:06.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38003b80 con 0x7fcf401e5790 2023-10-26T05:48:06.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf38003dd0 con 0x7fcf401e5790 2023-10-26T05:48:06.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf401e3580 con 0x7fcf401e5790 2023-10-26T05:48:06.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf380a7bc0 con 0x7fcf401e5790 2023-10-26T05:48:06.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401e3b40 con 0x7fcf401e5790 2023-10-26T05:48:06.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf40066b10 con 0x7fcf401e5790 2023-10-26T05:48:06.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf380a7d20 con 0x7fcf401e5790 2023-10-26T05:48:06.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf367fc640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf240664a0 0x7fcf24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf3811ea80 con 0x7fcf401e5790 2023-10-26T05:48:06.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf45e7f640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf240664a0 0x7fcf24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.365+0000 7fcf45e7f640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf240664a0 0x7fcf24068960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0017b0 tx=0x7fcf3c001120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:06.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.369+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf380ebf00 con 0x7fcf401e5790 2023-10-26T05:48:06.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.525+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf401e3d30 con 0x7fcf401e5790 2023-10-26T05:48:06.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.525+0000 7fcf367fc640 1 -- 172.21.15.71:0/2779757666 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf380ad020 con 0x7fcf401e5790 2023-10-26T05:48:06.530 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.525+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf240664a0 msgr2=0x7fcf24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf240664a0 0x7fcf24068960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0017b0 tx=0x7fcf3c001120 comp rx=0 tx=0).stop 2023-10-26T05:48:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 msgr2=0x7fcf401585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 0x7fcf401585f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf3801d020 tx=0x7fcf38004f80 comp rx=0 tx=0).stop 2023-10-26T05:48:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 shutdown_connections 2023-10-26T05:48:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf240664a0 0x7fcf24068960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf401e5790 0x7fcf401585f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf4013d910 0x7fcf401e5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 --2- 172.21.15.71:0/2779757666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf40157a20 0x7fcf4013d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 >> 172.21.15.71:0/2779757666 conn(0x7fcf400aec60 msgr2=0x7fcf400b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 shutdown_connections 2023-10-26T05:48:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.529+0000 7fcf480e3640 1 -- 172.21.15.71:0/2779757666 wait complete. 2023-10-26T05:48:06.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:06.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- 172.21.15.71:0/2003508361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 msgr2=0x7feb2015bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 --2- 172.21.15.71:0/2003508361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2015bf00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb18009c80 tx=0x7feb1801ccc0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- 172.21.15.71:0/2003508361 shutdown_connections 2023-10-26T05:48:06.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 --2- 172.21.15.71:0/2003508361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2015bf00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 --2- 172.21.15.71:0/2003508361 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 0x7feb200ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- 172.21.15.71:0/2003508361 >> 172.21.15.71:0/2003508361 conn(0x7feb200d1330 msgr2=0x7feb200d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:06.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- 172.21.15.71:0/2003508361 shutdown_connections 2023-10-26T05:48:06.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- 172.21.15.71:0/2003508361 wait complete. 2023-10-26T05:48:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 Processor -- start 2023-10-26T05:48:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- start start 2023-10-26T05:48:06.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb200cd400 0x7feb200ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 0x7feb201569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2014b730 con 0x7feb20069980 2023-10-26T05:48:06.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c87c0 con 0x7feb200cd400 2023-10-26T05:48:06.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb25846640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c8aa0 con 0x7feb2015b410 2023-10-26T05:48:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb1effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb1effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2014b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38074/0 (socket says 172.21.15.71:38074) 2023-10-26T05:48:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.665+0000 7feb1effd640 1 -- 172.21.15.71:0/3985345965 learned_addr learned my addr 172.21.15.71:0/3985345965 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1e7fc640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb200cd400 0x7feb200ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1f7fe640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 0x7feb201569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1effd640 1 -- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 msgr2=0x7feb201569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1effd640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 0x7feb201569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1effd640 1 -- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb200cd400 msgr2=0x7feb200ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1effd640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb200cd400 0x7feb200ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1effd640 1 -- 172.21.15.71:0/3985345965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb180098e0 con 0x7feb20069980 2023-10-26T05:48:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1e7fc640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb200cd400 0x7feb200ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1f7fe640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 0x7feb201569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:06.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1effd640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2014b1f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feb1000c9f0 tx=0x7feb1000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10007bb0 con 0x7feb20069980 2023-10-26T05:48:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb10007e00 con 0x7feb20069980 2023-10-26T05:48:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200c8d80 con 0x7feb20069980 2023-10-26T05:48:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000f6b0 con 0x7feb20069980 2023-10-26T05:48:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb200c9340 con 0x7feb20069980 2023-10-26T05:48:06.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb20066b10 con 0x7feb20069980 2023-10-26T05:48:06.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb1000f810 con 0x7feb20069980 2023-10-26T05:48:06.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb07fff640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb0c066470 0x7feb0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:06.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb10084660 con 0x7feb20069980 2023-10-26T05:48:06.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1e7fc640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb0c066470 0x7feb0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:06.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.669+0000 7feb1e7fc640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb0c066470 0x7feb0c068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb200c9b20 tx=0x7feb14009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:06.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.673+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb10051b70 con 0x7feb20069980 2023-10-26T05:48:06.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb2006c2c0 con 0x7feb20069980 2023-10-26T05:48:06.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb07fff640 1 -- 172.21.15.71:0/3985345965 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb10055190 con 0x7feb20069980 2023-10-26T05:48:06.833 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb0c066470 msgr2=0x7feb0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb0c066470 0x7feb0c068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb200c9b20 tx=0x7feb14009040 comp rx=0 tx=0).stop 2023-10-26T05:48:06.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 msgr2=0x7feb2014b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2014b1f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7feb1000c9f0 tx=0x7feb1000ceb0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 shutdown_connections 2023-10-26T05:48:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb2015b410 0x7feb201569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb200cd400 0x7feb200ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb0c066470 0x7feb0c068930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 --2- 172.21.15.71:0/3985345965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb20069980 0x7feb2014b1f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:06.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 >> 172.21.15.71:0/3985345965 conn(0x7feb200d1330 msgr2=0x7feb2015d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:06.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 shutdown_connections 2023-10-26T05:48:06.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06.829+0000 7feb25846640 1 -- 172.21.15.71:0/3985345965 wait complete. 2023-10-26T05:48:06.850 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:48:06.850 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:06.851 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-26T05:48:06.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:06.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi071.stderr:ceph-fuse[35613]: fuse finished with error 0 and tester_r 0 2023-10-26T05:48:06.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:06.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi071.stderr:2023-10-26T05:48:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:06.906 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:06.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:48:06.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:06.913 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:48:06.964 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:06.964 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:06.971 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:06.972 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:07.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:07.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:2023-10-26T05:48:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:07.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 -- 172.21.15.71:0/1604117782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 msgr2=0x7f829015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 --2- 172.21.15.71:0/1604117782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f829015bab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f829008a140 tx=0x7f828801cca0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 -- 172.21.15.71:0/1604117782 shutdown_connections 2023-10-26T05:48:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 --2- 172.21.15.71:0/1604117782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f829015bab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 --2- 172.21.15.71:0/1604117782 >> v2:172.21.15.86:3300/0 conn(0x7f82900d0470 0x7f829006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 -- 172.21.15.71:0/1604117782 >> 172.21.15.71:0/1604117782 conn(0x7f82900cdd80 msgr2=0x7f82900ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:07.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 -- 172.21.15.71:0/1604117782 shutdown_connections 2023-10-26T05:48:07.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.133+0000 7f8297500640 1 -- 172.21.15.71:0/1604117782 wait complete. 2023-10-26T05:48:07.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 Processor -- start 2023-10-26T05:48:07.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- start start 2023-10-26T05:48:07.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f8290152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82900d0470 0x7f8290154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82900d0850 0x7f82900cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829015d150 con 0x7f82900cd880 2023-10-26T05:48:07.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829015d3f0 con 0x7f82900d0850 2023-10-26T05:48:07.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829015d6d0 con 0x7f82900d0470 2023-10-26T05:48:07.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f8290152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f8290152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38094/0 (socket says 172.21.15.71:38094) 2023-10-26T05:48:07.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 -- 172.21.15.71:0/4237543894 learned_addr learned my addr 172.21.15.71:0/4237543894 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:07.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8294a9b640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82900d0470 0x7f8290154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8295a9d640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82900d0850 0x7f82900cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 -- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82900d0470 msgr2=0x7f8290154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82900d0470 0x7f8290154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 -- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82900d0850 msgr2=0x7f82900cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82900d0850 0x7f82900cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 -- 172.21.15.71:0/4237543894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880098e0 con 0x7f82900cd880 2023-10-26T05:48:07.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f829529c640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f8290152630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c970 tx=0x7f828c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:07.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c007bb0 con 0x7f82900cd880 2023-10-26T05:48:07.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f828c007e00 con 0x7f82900cd880 2023-10-26T05:48:07.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c005280 con 0x7f82900cd880 2023-10-26T05:48:07.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829015d9b0 con 0x7f82900cd880 2023-10-26T05:48:07.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8290152b70 con 0x7f82900cd880 2023-10-26T05:48:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.137+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8290066b10 con 0x7f82900cd880 2023-10-26T05:48:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.141+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f828c0053e0 con 0x7f82900cd880 2023-10-26T05:48:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.141+0000 7f8285ffb640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82740664a0 0x7f8274068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.141+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f828c084430 con 0x7f82900cd880 2023-10-26T05:48:07.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.141+0000 7f8294a9b640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82740664a0 0x7f8274068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.141+0000 7f8294a9b640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82740664a0 0x7f8274068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82900b5770 tx=0x7f8280009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:07.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.145+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f828c0553d0 con 0x7f82900cd880 2023-10-26T05:48:07.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.285+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82900721e0 con 0x7f82900cd880 2023-10-26T05:48:07.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8285ffb640 1 -- 172.21.15.71:0/4237543894 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f828c01d090 con 0x7f82900cd880 2023-10-26T05:48:07.292 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:07.293 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:47:23.027634+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:07.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82740664a0 msgr2=0x7f8274068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82740664a0 0x7f8274068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82900b5770 tx=0x7f8280009040 comp rx=0 tx=0).stop 2023-10-26T05:48:07.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 msgr2=0x7f8290152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f8290152630 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c970 tx=0x7f828c00ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:07.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 shutdown_connections 2023-10-26T05:48:07.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82900d0850 0x7f82900cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82900d0470 0x7f8290154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82740664a0 0x7f8274068960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 --2- 172.21.15.71:0/4237543894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82900cd880 0x7f8290152630 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 >> 172.21.15.71:0/4237543894 conn(0x7f82900cdd80 msgr2=0x7f82900b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:07.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 shutdown_connections 2023-10-26T05:48:07.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.289+0000 7f8297500640 1 -- 172.21.15.71:0/4237543894 wait complete. 2023-10-26T05:48:07.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:07.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:07.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3763194220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b814e7b0 msgr2=0x7f61b8157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3763194220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b814e7b0 0x7f61b8157db0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f61b8071700 tx=0x7f61b001cb60 comp rx=0 tx=0).stop 2023-10-26T05:48:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3763194220 shutdown_connections 2023-10-26T05:48:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3763194220 >> v2:172.21.15.86:3300/0 conn(0x7f61b806a000 0x7f61b806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3763194220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b814e7b0 0x7f61b8157db0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3763194220 >> 172.21.15.71:0/3763194220 conn(0x7f61b80aec60 msgr2=0x7f61b80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3763194220 shutdown_connections 2023-10-26T05:48:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3763194220 wait complete. 2023-10-26T05:48:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 Processor -- start 2023-10-26T05:48:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- start start 2023-10-26T05:48:07.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 0x7f61b814c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b8071270 0x7f61b8143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b814e7b0 0x7f61b80adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b814c990 con 0x7f61b806a000 2023-10-26T05:48:07.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b812f3f0 con 0x7f61b8071270 2023-10-26T05:48:07.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b812f6d0 con 0x7f61b814e7b0 2023-10-26T05:48:07.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 0x7f61b814c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 0x7f61b814c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38098/0 (socket says 172.21.15.71:38098) 2023-10-26T05:48:07.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 -- 172.21.15.71:0/3761332485 learned_addr learned my addr 172.21.15.71:0/3761332485 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:07.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b77fe640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b8071270 0x7f61b8143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61bcf66640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b814e7b0 0x7f61b80adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 -- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b814e7b0 msgr2=0x7f61b80adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b814e7b0 0x7f61b80adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 -- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b8071270 msgr2=0x7f61b8143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b8071270 0x7f61b8143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 -- 172.21.15.71:0/3761332485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0009820 con 0x7f61b806a000 2023-10-26T05:48:07.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61bcf66640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b814e7b0 0x7f61b80adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:07.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b7fff640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 0x7f61b814c450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000f90 tx=0x7f61b00a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:07.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004c50 con 0x7f61b806a000 2023-10-26T05:48:07.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61b0005030 con 0x7f61b806a000 2023-10-26T05:48:07.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b812f9b0 con 0x7f61b806a000 2023-10-26T05:48:07.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004ea0 con 0x7f61b806a000 2023-10-26T05:48:07.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b812ff70 con 0x7f61b806a000 2023-10-26T05:48:07.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.433+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b8066b10 con 0x7f61b806a000 2023-10-26T05:48:07.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.437+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61b00b4040 con 0x7f61b806a000 2023-10-26T05:48:07.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.437+0000 7f61b4ff9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6198066470 0x7f6198068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.437+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b011e1f0 con 0x7f61b806a000 2023-10-26T05:48:07.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.437+0000 7f61b77fe640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6198066470 0x7f6198068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.437+0000 7f61b77fe640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6198066470 0x7f6198068930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f61b806b080 tx=0x7f61a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:07.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.441+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61b00eb700 con 0x7f61b806a000 2023-10-26T05:48:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61b80721e0 con 0x7f61b806a000 2023-10-26T05:48:07.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61b4ff9640 1 -- 172.21.15.71:0/3761332485 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61b00b20a0 con 0x7f61b806a000 2023-10-26T05:48:07.597 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:07.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6198066470 msgr2=0x7f6198068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6198066470 0x7f6198068930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f61b806b080 tx=0x7f61a0009040 comp rx=0 tx=0).stop 2023-10-26T05:48:07.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 msgr2=0x7f61b814c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 0x7f61b814c450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000f90 tx=0x7f61b00a5a80 comp rx=0 tx=0).stop 2023-10-26T05:48:07.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 shutdown_connections 2023-10-26T05:48:07.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61b814e7b0 0x7f61b80adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b8071270 0x7f61b8143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6198066470 0x7f6198068930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 --2- 172.21.15.71:0/3761332485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b806a000 0x7f61b814c450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 >> 172.21.15.71:0/3761332485 conn(0x7f61b80aec60 msgr2=0x7f61b815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:07.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 shutdown_connections 2023-10-26T05:48:07.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.593+0000 7f61be9c9640 1 -- 172.21.15.71:0/3761332485 wait complete. 2023-10-26T05:48:07.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:07.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/182556749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 msgr2=0x7f45d013e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/182556749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d013e850 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f45d006c780 tx=0x7f45c401c870 comp rx=0 tx=0).stop 2023-10-26T05:48:07.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/182556749 shutdown_connections 2023-10-26T05:48:07.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/182556749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d013e850 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/182556749 >> v2:172.21.15.86:3300/0 conn(0x7f45d013c120 0x7f45d00cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/182556749 >> 172.21.15.71:0/182556749 conn(0x7f45d00aec60 msgr2=0x7f45d00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/182556749 shutdown_connections 2023-10-26T05:48:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/182556749 wait complete. 2023-10-26T05:48:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 Processor -- start 2023-10-26T05:48:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- start start 2023-10-26T05:48:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d013c120 0x7f45d01377f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d015a3f0 0x7f45d00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0071610 con 0x7f45d00cbe00 2023-10-26T05:48:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0138220 con 0x7f45d013c120 2023-10-26T05:48:07.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d006f540 con 0x7f45d015a3f0 2023-10-26T05:48:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d00710d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38114/0 (socket says 172.21.15.71:38114) 2023-10-26T05:48:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 -- 172.21.15.71:0/2988046308 learned_addr learned my addr 172.21.15.71:0/2988046308 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 -- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d015a3f0 msgr2=0x7f45d00adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45cdd9b640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d013c120 0x7f45d01377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ced9d640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d015a3f0 0x7f45d00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d015a3f0 0x7f45d00adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 -- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d013c120 msgr2=0x7f45d01377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d013c120 0x7f45d01377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 -- 172.21.15.71:0/2988046308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4009560 con 0x7f45d00cbe00 2023-10-26T05:48:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45ce59c640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d00710d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e940 tx=0x7f45c000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0009730 con 0x7f45d00cbe00 2023-10-26T05:48:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45c000cd80 con 0x7f45d00cbe00 2023-10-26T05:48:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d006f820 con 0x7f45d00cbe00 2023-10-26T05:48:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0005280 con 0x7f45d00cbe00 2023-10-26T05:48:07.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.741+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d006fde0 con 0x7f45d00cbe00 2023-10-26T05:48:07.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.745+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d0066b10 con 0x7f45d00cbe00 2023-10-26T05:48:07.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.745+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45c00053e0 con 0x7f45d00cbe00 2023-10-26T05:48:07.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.745+0000 7f45beffd640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b0066470 0x7f45b0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:07.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.745+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45c0014070 con 0x7f45d00cbe00 2023-10-26T05:48:07.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.745+0000 7f45cdd9b640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b0066470 0x7f45b0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:07.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.745+0000 7f45cdd9b640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b0066470 0x7f45b0068930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45c40af000 tx=0x7f45c401b0b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:07.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.749+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f45c0051eb0 con 0x7f45d00cbe00 2023-10-26T05:48:07.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45d00721e0 con 0x7f45d00cbe00 2023-10-26T05:48:07.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45beffd640 1 -- 172.21.15.71:0/2988046308 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45c00554d0 con 0x7f45d00cbe00 2023-10-26T05:48:07.913 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b0066470 msgr2=0x7f45b0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b0066470 0x7f45b0068930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45c40af000 tx=0x7f45c401b0b0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 msgr2=0x7f45d00710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:07.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d00710d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e940 tx=0x7f45c000ee00 comp rx=0 tx=0).stop 2023-10-26T05:48:07.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 shutdown_connections 2023-10-26T05:48:07.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d015a3f0 0x7f45d00adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d013c120 0x7f45d01377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b0066470 0x7f45b0068930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 --2- 172.21.15.71:0/2988046308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d00cbe00 0x7f45d00710d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:07.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 >> 172.21.15.71:0/2988046308 conn(0x7f45d00aec60 msgr2=0x7f45d0139fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:07.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 shutdown_connections 2023-10-26T05:48:07.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:07.909+0000 7f45d4fb0640 1 -- 172.21.15.71:0/2988046308 wait complete. 2023-10-26T05:48:07.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:08.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 -- 172.21.15.71:0/4259203779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b0003fe0 msgr2=0x7f95b00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4259203779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95b0003fe0 0x7f95b00047d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f95b400bb10 tx=0x7f95b401cd60 comp rx=0 tx=0).stop 2023-10-26T05:48:08.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95c993a640 1 -- 172.21.15.71:0/4259203779 reap_dead start 2023-10-26T05:48:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 -- 172.21.15.71:0/4259203779 shutdown_connections 2023-10-26T05:48:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 -- 172.21.15.71:0/4259203779 >> 172.21.15.71:0/4259203779 conn(0x7f95c40cdd80 msgr2=0x7f95c40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 -- 172.21.15.71:0/4259203779 shutdown_connections 2023-10-26T05:48:08.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 -- 172.21.15.71:0/4259203779 wait complete. 2023-10-26T05:48:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 Processor -- start 2023-10-26T05:48:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.041+0000 7f95cb39d640 1 -- start start 2023-10-26T05:48:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0003fe0 0x7f95c4161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 0x7f95c40b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c414e800 0x7f95c40b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c415e580 con 0x7f95c414e3a0 2023-10-26T05:48:08.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c415f7e0 con 0x7f95b0003fe0 2023-10-26T05:48:08.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c415d4e0 con 0x7f95c414e800 2023-10-26T05:48:08.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 0x7f95c40b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 0x7f95c40b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38132/0 (socket says 172.21.15.71:38132) 2023-10-26T05:48:08.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 -- 172.21.15.71:0/4127018544 learned_addr learned my addr 172.21.15.71:0/4127018544 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:08.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c9139640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0003fe0 0x7f95c4161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c993a640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c414e800 0x7f95c40b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 -- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c414e800 msgr2=0x7f95c40b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c414e800 0x7f95c40b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 -- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0003fe0 msgr2=0x7f95c4161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0003fe0 0x7f95c4161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 -- 172.21.15.71:0/4127018544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c415d7c0 con 0x7f95c414e3a0 2023-10-26T05:48:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c993a640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c414e800 0x7f95c40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c8938640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 0x7f95c40b1280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95b400bb10 tx=0x7f95b40050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4005420 con 0x7f95c414e3a0 2023-10-26T05:48:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95b4005670 con 0x7f95c414e3a0 2023-10-26T05:48:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c415daa0 con 0x7f95c414e3a0 2023-10-26T05:48:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4004910 con 0x7f95c414e3a0 2023-10-26T05:48:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c415e060 con 0x7f95c414e3a0 2023-10-26T05:48:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95c4066b10 con 0x7f95c414e3a0 2023-10-26T05:48:08.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f95b4004a70 con 0x7f95c414e3a0 2023-10-26T05:48:08.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95b9ffb640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95a8066470 0x7f95a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95b411eda0 con 0x7f95c414e3a0 2023-10-26T05:48:08.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c9139640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95a8066470 0x7f95a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.045+0000 7f95c9139640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95a8066470 0x7f95a8068930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f95bc001b50 tx=0x7f95bc0012c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.049+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f95b40ec190 con 0x7f95c414e3a0 2023-10-26T05:48:08.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.193+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95c40721e0 con 0x7f95c414e3a0 2023-10-26T05:48:08.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.193+0000 7f95b9ffb640 1 -- 172.21.15.71:0/4127018544 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95b40ab030 con 0x7f95c414e3a0 2023-10-26T05:48:08.199 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:08.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95a8066470 msgr2=0x7f95a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95a8066470 0x7f95a8068930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f95bc001b50 tx=0x7f95bc0012c0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 msgr2=0x7f95c40b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 0x7f95c40b1280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95b400bb10 tx=0x7f95b40050b0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 shutdown_connections 2023-10-26T05:48:08.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95c414e800 0x7f95c40b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95a8066470 0x7f95a8068930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95c414e3a0 0x7f95c40b1280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 --2- 172.21.15.71:0/4127018544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95b0003fe0 0x7f95c4161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 >> 172.21.15.71:0/4127018544 conn(0x7f95c40cdd80 msgr2=0x7f95c40cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 shutdown_connections 2023-10-26T05:48:08.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.197+0000 7f95cb39d640 1 -- 172.21.15.71:0/4127018544 wait complete. 2023-10-26T05:48:08.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:08.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- 172.21.15.71:0/694116612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 msgr2=0x7f8fec13d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/694116612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec13d770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8fec069f80 tx=0x7f8fe001c520 comp rx=0 tx=0).stop 2023-10-26T05:48:08.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- 172.21.15.71:0/694116612 shutdown_connections 2023-10-26T05:48:08.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/694116612 >> v2:172.21.15.86:3300/0 conn(0x7f8fec13dcb0 0x7f8fec15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/694116612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec13d770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/694116612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fec13ae90 0x7f8fec0cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- 172.21.15.71:0/694116612 >> 172.21.15.71:0/694116612 conn(0x7f8fec06b8c0 msgr2=0x7f8fec06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- 172.21.15.71:0/694116612 shutdown_connections 2023-10-26T05:48:08.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- 172.21.15.71:0/694116612 wait complete. 2023-10-26T05:48:08.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 Processor -- start 2023-10-26T05:48:08.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- start start 2023-10-26T05:48:08.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fec13ae90 0x7f8fec15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec13dcb0 0x7f8fec15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec15bfd0 con 0x7f8fec159060 2023-10-26T05:48:08.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec15b1d0 con 0x7f8fec13dcb0 2023-10-26T05:48:08.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8ff1c16640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec15b450 con 0x7f8fec13ae90 2023-10-26T05:48:08.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8febfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8febfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38154/0 (socket says 172.21.15.71:38154) 2023-10-26T05:48:08.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.333+0000 7f8febfff640 1 -- 172.21.15.71:0/3198595662 learned_addr learned my addr 172.21.15.71:0/3198595662 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8feb7fe640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fec13ae90 0x7f8fec15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8feaffd640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec13dcb0 0x7f8fec15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8febfff640 1 -- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fec13ae90 msgr2=0x7f8fec15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8febfff640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fec13ae90 0x7f8fec15a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8febfff640 1 -- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec13dcb0 msgr2=0x7f8fec15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8febfff640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec13dcb0 0x7f8fec15ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8febfff640 1 -- 172.21.15.71:0/3198595662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0009560 con 0x7f8fec159060 2023-10-26T05:48:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8feaffd640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec13dcb0 0x7f8fec15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8febfff640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec154830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400ea20 tx=0x7f8fe400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400cde0 con 0x7f8fec159060 2023-10-26T05:48:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8fe4004c40 con 0x7f8fec159060 2023-10-26T05:48:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec15b730 con 0x7f8fec159060 2023-10-26T05:48:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4005020 con 0x7f8fec159060 2023-10-26T05:48:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec15bcf0 con 0x7f8fec159060 2023-10-26T05:48:08.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec066b10 con 0x7f8fec159060 2023-10-26T05:48:08.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8fe4005180 con 0x7f8fec159060 2023-10-26T05:48:08.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8fd3fff640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd80664a0 0x7f8fd8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe40848d0 con 0x7f8fec159060 2023-10-26T05:48:08.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8feb7fe640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd80664a0 0x7f8fd8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.337+0000 7f8feb7fe640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd80664a0 0x7f8fd8068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005fd0 tx=0x7f8fdc005d80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.341+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8fe4051d50 con 0x7f8fec159060 2023-10-26T05:48:08.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.501+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8fec06c7d0 con 0x7f8fec159060 2023-10-26T05:48:08.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.501+0000 7f8fd3fff640 1 -- 172.21.15.71:0/3198595662 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8fe401f070 con 0x7f8fec159060 2023-10-26T05:48:08.506 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:08.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.501+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd80664a0 msgr2=0x7f8fd8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.501+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd80664a0 0x7f8fd8068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc005fd0 tx=0x7f8fdc005d80 comp rx=0 tx=0).stop 2023-10-26T05:48:08.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.501+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 msgr2=0x7f8fec154830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.501+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec154830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400ea20 tx=0x7f8fe400eee0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 shutdown_connections 2023-10-26T05:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd80664a0 0x7f8fd8068960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec159060 0x7f8fec154830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fec13dcb0 0x7f8fec15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 --2- 172.21.15.71:0/3198595662 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8fec13ae90 0x7f8fec15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 >> 172.21.15.71:0/3198595662 conn(0x7f8fec06b8c0 msgr2=0x7f8fec0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 shutdown_connections 2023-10-26T05:48:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.505+0000 7f8ff1c16640 1 -- 172.21.15.71:0/3198595662 wait complete. 2023-10-26T05:48:08.523 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:08.523 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:08.530 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:08.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 -- 172.21.15.71:0/3530083781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878158500 msgr2=0x7ff8780b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 --2- 172.21.15.71:0/3530083781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878158500 0x7ff8780b1c70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff860009840 tx=0x7ff86001c700 comp rx=0 tx=0).stop 2023-10-26T05:48:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 -- 172.21.15.71:0/3530083781 shutdown_connections 2023-10-26T05:48:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 --2- 172.21.15.71:0/3530083781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff878158500 0x7ff8780b1c70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 -- 172.21.15.71:0/3530083781 >> 172.21.15.71:0/3530083781 conn(0x7ff8780ce270 msgr2=0x7ff8780ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 -- 172.21.15.71:0/3530083781 shutdown_connections 2023-10-26T05:48:08.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.645+0000 7ff87cee3640 1 -- 172.21.15.71:0/3530083781 wait complete. 2023-10-26T05:48:08.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 Processor -- start 2023-10-26T05:48:08.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- start start 2023-10-26T05:48:08.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8780b21b0 0x7ff8781549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 0x7ff87814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8780ce860 0x7ff878147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878148e00 con 0x7ff8781588f0 2023-10-26T05:48:08.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780ced60 con 0x7ff8780ce860 2023-10-26T05:48:08.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8781540a0 con 0x7ff8780b21b0 2023-10-26T05:48:08.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 0x7ff87814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 0x7ff87814d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38182/0 (socket says 172.21.15.71:38182) 2023-10-26T05:48:08.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 -- 172.21.15.71:0/824389513 learned_addr learned my addr 172.21.15.71:0/824389513 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:08.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 -- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8780b21b0 msgr2=0x7ff8781549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff876d9d640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8780ce860 0x7ff878147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87659c640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8780b21b0 0x7ff8781549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8780b21b0 0x7ff8781549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 -- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8780ce860 msgr2=0x7ff878147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8780ce860 0x7ff878147320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 -- 172.21.15.71:0/824389513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8600094a0 con 0x7ff8781588f0 2023-10-26T05:48:08.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff876d9d640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8780ce860 0x7ff878147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:08.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff875d9b640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 0x7ff87814d100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff860009790 tx=0x7ff860004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8600044f0 con 0x7ff8781588f0 2023-10-26T05:48:08.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff860005120 con 0x7ff8781588f0 2023-10-26T05:48:08.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff878154380 con 0x7ff8781588f0 2023-10-26T05:48:08.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff860005510 con 0x7ff8781588f0 2023-10-26T05:48:08.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff878148130 con 0x7ff8781588f0 2023-10-26T05:48:08.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.649+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff878066d00 con 0x7ff8781588f0 2023-10-26T05:48:08.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.653+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff8600b4040 con 0x7ff8781588f0 2023-10-26T05:48:08.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.653+0000 7ff86effd640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff858066470 0x7ff858068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.653+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff86011dd90 con 0x7ff8781588f0 2023-10-26T05:48:08.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.653+0000 7ff87659c640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff858066470 0x7ff858068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.653+0000 7ff87659c640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff858066470 0x7ff858068930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff87806da20 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.657+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff8600eb190 con 0x7ff8781588f0 2023-10-26T05:48:08.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.813+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8780cf0a0 con 0x7ff8781588f0 2023-10-26T05:48:08.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.813+0000 7ff86effd640 1 -- 172.21.15.71:0/824389513 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16670 (secure 0 0 0) 0x7ff8600b2090 con 0x7ff8781588f0 2023-10-26T05:48:08.819 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:08.820 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":21,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:08.356888+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:08.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff858066470 msgr2=0x7ff858068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff858066470 0x7ff858068930 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff87806da20 tx=0x7ff870008040 comp rx=0 tx=0).stop 2023-10-26T05:48:08.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 msgr2=0x7ff87814d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 0x7ff87814d100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff860009790 tx=0x7ff860004040 comp rx=0 tx=0).stop 2023-10-26T05:48:08.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 shutdown_connections 2023-10-26T05:48:08.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8780ce860 0x7ff878147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff858066470 0x7ff858068930 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8781588f0 0x7ff87814d100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 --2- 172.21.15.71:0/824389513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8780b21b0 0x7ff8781549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 >> 172.21.15.71:0/824389513 conn(0x7ff8780ce270 msgr2=0x7ff87815c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 shutdown_connections 2023-10-26T05:48:08.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.817+0000 7ff87cee3640 1 -- 172.21.15.71:0/824389513 wait complete. 2023-10-26T05:48:08.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:08.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:08.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 -- 172.21.15.71:0/1027708319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c15bb90 msgr2=0x7fee6c0b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 --2- 172.21.15.71:0/1027708319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c15bb90 0x7fee6c0b0fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fee6c06c660 tx=0x7fee5c01c5b0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 -- 172.21.15.71:0/1027708319 shutdown_connections 2023-10-26T05:48:08.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 --2- 172.21.15.71:0/1027708319 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee6c15bf70 0x7fee6c0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 --2- 172.21.15.71:0/1027708319 >> v2:172.21.15.86:3300/0 conn(0x7fee6c070dc0 0x7fee6c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 --2- 172.21.15.71:0/1027708319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c15bb90 0x7fee6c0b0fa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 -- 172.21.15.71:0/1027708319 >> 172.21.15.71:0/1027708319 conn(0x7fee6c0aec60 msgr2=0x7fee6c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:08.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 -- 172.21.15.71:0/1027708319 shutdown_connections 2023-10-26T05:48:08.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 -- 172.21.15.71:0/1027708319 wait complete. 2023-10-26T05:48:08.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 Processor -- start 2023-10-26T05:48:08.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.957+0000 7fee72004640 1 -- start start 2023-10-26T05:48:08.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 0x7fee6c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c15bb90 0x7fee6c152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee6c15bf70 0x7fee6c159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c140000 con 0x7fee6c070dc0 2023-10-26T05:48:08.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c140fa0 con 0x7fee6c15bb90 2023-10-26T05:48:08.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1401e0 con 0x7fee6c15bf70 2023-10-26T05:48:08.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 0x7fee6c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 0x7fee6c13f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38196/0 (socket says 172.21.15.71:38196) 2023-10-26T05:48:08.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 -- 172.21.15.71:0/1980058272 learned_addr learned my addr 172.21.15.71:0/1980058272 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:08.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6bfff640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee6c15bf70 0x7fee6c159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6affd640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c15bb90 0x7fee6c152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 -- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee6c15bf70 msgr2=0x7fee6c159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee6c15bf70 0x7fee6c159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 -- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c15bb90 msgr2=0x7fee6c152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:08.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c15bb90 0x7fee6c152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:08.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 -- 172.21.15.71:0/1980058272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009560 con 0x7fee6c070dc0 2023-10-26T05:48:08.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6b7fe640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 0x7fee6c13f6a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009850 tx=0x7fee5c0a6a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0b6070 con 0x7fee6c070dc0 2023-10-26T05:48:08.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee5c0a6c80 con 0x7fee6c070dc0 2023-10-26T05:48:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c1404c0 con 0x7fee6c070dc0 2023-10-26T05:48:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004030 con 0x7fee6c070dc0 2023-10-26T05:48:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c140a80 con 0x7fee6c070dc0 2023-10-26T05:48:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c066b10 con 0x7fee6c070dc0 2023-10-26T05:48:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fee5c004190 con 0x7fee6c070dc0 2023-10-26T05:48:08.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee53fff640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066490 0x7fee58068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:08.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fee5c11de70 con 0x7fee6c070dc0 2023-10-26T05:48:08.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6affd640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066490 0x7fee58068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:08.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.961+0000 7fee6affd640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066490 0x7fee58068950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee600058d0 tx=0x7fee60005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:08.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:08.965+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fee5c0eb1d0 con 0x7fee6c070dc0 2023-10-26T05:48:09.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.117+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee6c0721e0 con 0x7fee6c070dc0 2023-10-26T05:48:09.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.117+0000 7fee53fff640 1 -- 172.21.15.71:0/1980058272 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee5c0c0020 con 0x7fee6c070dc0 2023-10-26T05:48:09.123 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:09.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066490 msgr2=0x7fee58068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066490 0x7fee58068950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fee600058d0 tx=0x7fee60005860 comp rx=0 tx=0).stop 2023-10-26T05:48:09.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 msgr2=0x7fee6c13f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 0x7fee6c13f6a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009850 tx=0x7fee5c0a6a90 comp rx=0 tx=0).stop 2023-10-26T05:48:09.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 shutdown_connections 2023-10-26T05:48:09.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee6c15bf70 0x7fee6c159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee6c15bb90 0x7fee6c152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066490 0x7fee58068950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 --2- 172.21.15.71:0/1980058272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee6c070dc0 0x7fee6c13f6a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 >> 172.21.15.71:0/1980058272 conn(0x7fee6c0aec60 msgr2=0x7fee6c0ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:09.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 shutdown_connections 2023-10-26T05:48:09.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.121+0000 7fee72004640 1 -- 172.21.15.71:0/1980058272 wait complete. 2023-10-26T05:48:09.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:09.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 -- 172.21.15.71:0/2809697048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd80047d0 msgr2=0x7fbcd8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2809697048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd80047d0 0x7fbcd8004c50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009b60 tx=0x7fbcd401d240 comp rx=0 tx=0).stop 2023-10-26T05:48:09.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 -- 172.21.15.71:0/2809697048 shutdown_connections 2023-10-26T05:48:09.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2809697048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcd80047d0 0x7fbcd8004c50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 -- 172.21.15.71:0/2809697048 >> 172.21.15.71:0/2809697048 conn(0x7fbcec0ce270 msgr2=0x7fbcec0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:09.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 -- 172.21.15.71:0/2809697048 shutdown_connections 2023-10-26T05:48:09.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 -- 172.21.15.71:0/2809697048 wait complete. 2023-10-26T05:48:09.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 Processor -- start 2023-10-26T05:48:09.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 -- start start 2023-10-26T05:48:09.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.253+0000 7fbcf1227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd80047d0 0x7fbcec0ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcec14dcc0 0x7fbcec0ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 0x7fbcec06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec06ca90 con 0x7fbcec0acd20 2023-10-26T05:48:09.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0ab4e0 con 0x7fbcd80047d0 2023-10-26T05:48:09.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0ab7c0 con 0x7fbcec14dcc0 2023-10-26T05:48:09.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 0x7fbcec06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 0x7fbcec06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38216/0 (socket says 172.21.15.71:38216) 2023-10-26T05:48:09.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 -- 172.21.15.71:0/2215094160 learned_addr learned my addr 172.21.15.71:0/2215094160 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:09.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcea59c640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcec14dcc0 0x7fbcec0ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcead9d640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd80047d0 0x7fbcec0ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 -- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcec14dcc0 msgr2=0x7fbcec0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcec14dcc0 0x7fbcec0ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 -- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd80047d0 msgr2=0x7fbcec0ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd80047d0 0x7fbcec0ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 -- 172.21.15.71:0/2215094160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0abaa0 con 0x7fbcec0acd20 2023-10-26T05:48:09.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbceb59e640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 0x7fbcec06b110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0afef0 tx=0x7fbcdc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc003b80 con 0x7fbcec0acd20 2023-10-26T05:48:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbcdc003ce0 con 0x7fbcec0acd20 2023-10-26T05:48:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec0abd80 con 0x7fbcec0acd20 2023-10-26T05:48:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc005330 con 0x7fbcec0acd20 2023-10-26T05:48:09.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec14b2b0 con 0x7fbcec0acd20 2023-10-26T05:48:09.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0abfc0 con 0x7fbcec0acd20 2023-10-26T05:48:09.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbcdc005490 con 0x7fbcec0acd20 2023-10-26T05:48:09.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbce37fe640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbccc066470 0x7fbccc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbcdc083ab0 con 0x7fbcec0acd20 2023-10-26T05:48:09.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcead9d640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbccc066470 0x7fbccc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.257+0000 7fbcead9d640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbccc066470 0x7fbccc068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbce4001060 tx=0x7fbce4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:09.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.261+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbcec0abfc0 con 0x7fbcec0acd20 2023-10-26T05:48:09.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.405+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcec0abfc0 con 0x7fbcec0acd20 2023-10-26T05:48:09.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbce37fe640 1 -- 172.21.15.71:0/2215094160 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbcec0abfc0 con 0x7fbcec0acd20 2023-10-26T05:48:09.412 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:09.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbccc066470 msgr2=0x7fbccc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbccc066470 0x7fbccc068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbce4001060 tx=0x7fbce4009040 comp rx=0 tx=0).stop 2023-10-26T05:48:09.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 msgr2=0x7fbcec06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 0x7fbcec06b110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0afef0 tx=0x7fbcdc00cb10 comp rx=0 tx=0).stop 2023-10-26T05:48:09.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 shutdown_connections 2023-10-26T05:48:09.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbccc066470 0x7fbccc068930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbcec0acd20 0x7fbcec06b110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbcec14dcc0 0x7fbcec0ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 --2- 172.21.15.71:0/2215094160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbcd80047d0 0x7fbcec0ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 >> 172.21.15.71:0/2215094160 conn(0x7fbcec0ce270 msgr2=0x7fbcec159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 shutdown_connections 2023-10-26T05:48:09.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.409+0000 7fbcf1227640 1 -- 172.21.15.71:0/2215094160 wait complete. 2023-10-26T05:48:09.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:09.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/3940859025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9840004d10 msgr2=0x7f9840005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 --2- 172.21.15.71:0/3940859025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9840004d10 0x7f9840005190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f985000c610 tx=0x7f985001d330 comp rx=0 tx=0).stop 2023-10-26T05:48:09.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9857fff640 1 -- 172.21.15.71:0/3940859025 reap_dead start 2023-10-26T05:48:09.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/3940859025 shutdown_connections 2023-10-26T05:48:09.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/3940859025 >> 172.21.15.71:0/3940859025 conn(0x7f98580cdd80 msgr2=0x7f98580ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:09.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/3940859025 shutdown_connections 2023-10-26T05:48:09.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/3940859025 wait complete. 2023-10-26T05:48:09.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 Processor -- start 2023-10-26T05:48:09.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- start start 2023-10-26T05:48:09.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9840004d10 0x7f9858159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 0x7f9858163f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580cac80 0x7f98580cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98580cb670 con 0x7f98580c8710 2023-10-26T05:48:09.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858164810 con 0x7f9840004d10 2023-10-26T05:48:09.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9858164af0 con 0x7f98580cac80 2023-10-26T05:48:09.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 0x7f9858163f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 0x7f9858163f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38242/0 (socket says 172.21.15.71:38242) 2023-10-26T05:48:09.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 -- 172.21.15.71:0/4292113209 learned_addr learned my addr 172.21.15.71:0/4292113209 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:09.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9857fff640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580cac80 0x7f98580cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f98577fe640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9840004d10 0x7f9858159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 -- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580cac80 msgr2=0x7f98580cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580cac80 0x7f98580cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 -- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9840004d10 msgr2=0x7f9858159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9840004d10 0x7f9858159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 -- 172.21.15.71:0/4292113209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858164dd0 con 0x7f98580c8710 2023-10-26T05:48:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f9856ffd640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 0x7f9858163f90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f984c00e8e0 tx=0x7f984c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00cc60 con 0x7f98580c8710 2023-10-26T05:48:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f984c00ceb0 con 0x7f98580c8710 2023-10-26T05:48:09.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98580a85d0 con 0x7f98580c8710 2023-10-26T05:48:09.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c005260 con 0x7f98580c8710 2023-10-26T05:48:09.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.541+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98581644d0 con 0x7f98580c8710 2023-10-26T05:48:09.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.545+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9858066b10 con 0x7f98580c8710 2023-10-26T05:48:09.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.545+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f984c0053c0 con 0x7f98580c8710 2023-10-26T05:48:09.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.545+0000 7f983ffff640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9844066470 0x7f9844068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.545+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f984c084a30 con 0x7f98580c8710 2023-10-26T05:48:09.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.545+0000 7f98577fe640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9844066470 0x7f9844068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.545+0000 7f98577fe640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9844066470 0x7f9844068930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98580afe90 tx=0x7f984800c500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:09.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.549+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f984c0558c0 con 0x7f98580c8710 2023-10-26T05:48:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98580a8790 con 0x7f98580c8710 2023-10-26T05:48:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f983ffff640 1 -- 172.21.15.71:0/4292113209 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f984c01f070 con 0x7f98580c8710 2023-10-26T05:48:09.693 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:09.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9844066470 msgr2=0x7f9844068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9844066470 0x7f9844068930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98580afe90 tx=0x7f984800c500 comp rx=0 tx=0).stop 2023-10-26T05:48:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 msgr2=0x7f9858163f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 0x7f9858163f90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f984c00e8e0 tx=0x7f984c00eda0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 shutdown_connections 2023-10-26T05:48:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580cac80 0x7f98580cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9844066470 0x7f9844068930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98580c8710 0x7f9858163f90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 --2- 172.21.15.71:0/4292113209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9840004d10 0x7f9858159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 >> 172.21.15.71:0/4292113209 conn(0x7f98580cdd80 msgr2=0x7f985815c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 shutdown_connections 2023-10-26T05:48:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.689+0000 7f985e10c640 1 -- 172.21.15.71:0/4292113209 wait complete. 2023-10-26T05:48:09.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:09.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/463364858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 msgr2=0x7f732815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 --2- 172.21.15.71:0/463364858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f732815efd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7328073000 tx=0x7f731801c830 comp rx=0 tx=0).stop 2023-10-26T05:48:09.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/463364858 shutdown_connections 2023-10-26T05:48:09.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 --2- 172.21.15.71:0/463364858 >> v2:172.21.15.86:3300/0 conn(0x7f732815f9c0 0x7f732806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 --2- 172.21.15.71:0/463364858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f732815efd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/463364858 >> 172.21.15.71:0/463364858 conn(0x7f73280d1350 msgr2=0x7f73280d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:09.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/463364858 shutdown_connections 2023-10-26T05:48:09.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/463364858 wait complete. 2023-10-26T05:48:09.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 Processor -- start 2023-10-26T05:48:09.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- start start 2023-10-26T05:48:09.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f73280c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732815f510 0x7f73280caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f732815f9c0 0x7f73280b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b0cd0 con 0x7f732815b4c0 2023-10-26T05:48:09.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73280c96c0 con 0x7f732815f510 2023-10-26T05:48:09.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73280b17e0 con 0x7f732815f9c0 2023-10-26T05:48:09.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f73280c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f73280c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38268/0 (socket says 172.21.15.71:38268) 2023-10-26T05:48:09.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 -- 172.21.15.71:0/2816929840 learned_addr learned my addr 172.21.15.71:0/2816929840 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:09.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 -- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f732815f9c0 msgr2=0x7f73280b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f73267fc640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732815f510 0x7f73280caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f73277fe640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f732815f9c0 0x7f73280b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f732815f9c0 0x7f73280b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 -- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732815f510 msgr2=0x7f73280caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732815f510 0x7f73280caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:09.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 -- 172.21.15.71:0/2816929840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318009560 con 0x7f732815b4c0 2023-10-26T05:48:09.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f7326ffd640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f73280c9180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f731801a460 tx=0x7f73180a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:09.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318005220 con 0x7f732815b4c0 2023-10-26T05:48:09.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7318005380 con 0x7f732815b4c0 2023-10-26T05:48:09.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73280b0eb0 con 0x7f732815b4c0 2023-10-26T05:48:09.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180046d0 con 0x7f732815b4c0 2023-10-26T05:48:09.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73280b1470 con 0x7f732815b4c0 2023-10-26T05:48:09.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.829+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7328166140 con 0x7f732815b4c0 2023-10-26T05:48:09.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.833+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f73180b4040 con 0x7f732815b4c0 2023-10-26T05:48:09.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.833+0000 7f730f7fe640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7314066470 0x7f7314068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:09.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.833+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f731811e390 con 0x7f732815b4c0 2023-10-26T05:48:09.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.833+0000 7f73267fc640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7314066470 0x7f7314068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:09.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.833+0000 7f73267fc640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7314066470 0x7f7314068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7328067260 tx=0x7f731c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:09.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.837+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f73180eb6f0 con 0x7f732815b4c0 2023-10-26T05:48:09.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f732804db60 con 0x7f732815b4c0 2023-10-26T05:48:09.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f730f7fe640 1 -- 172.21.15.71:0/2816929840 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f73180eed10 con 0x7f732815b4c0 2023-10-26T05:48:09.997 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:09.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7314066470 msgr2=0x7f7314068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:09.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7314066470 0x7f7314068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7328067260 tx=0x7f731c009040 comp rx=0 tx=0).stop 2023-10-26T05:48:10.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 msgr2=0x7f73280c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f73280c9180 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f731801a460 tx=0x7f73180a5770 comp rx=0 tx=0).stop 2023-10-26T05:48:10.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 shutdown_connections 2023-10-26T05:48:10.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f732815f9c0 0x7f73280b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f732815f510 0x7f73280caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7314066470 0x7f7314068930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 --2- 172.21.15.71:0/2816929840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f732815b4c0 0x7f73280c9180 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 >> 172.21.15.71:0/2816929840 conn(0x7f73280d1350 msgr2=0x7f73281585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:10.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 shutdown_connections 2023-10-26T05:48:10.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:09.993+0000 7f732d531640 1 -- 172.21.15.71:0/2816929840 wait complete. 2023-10-26T05:48:10.015 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:48:10.015 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:10.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-26T05:48:10.073 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:10.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:ceph-fuse[33879]: fuse finished with error 0 and tester_r 0 2023-10-26T05:48:10.074 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:10.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:2023-10-26T05:48:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:10.083 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:10.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:48:10.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:10.091 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:48:10.139 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:48:10.139 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/2345545066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 msgr2=0x7f3e94070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/2345545066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e94070690 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009c80 tx=0x7f3e8c01cd80 comp rx=0 tx=0).stop 2023-10-26T05:48:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/2345545066 shutdown_connections 2023-10-26T05:48:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/2345545066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e94070690 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/2345545066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e940ac740 0x7f3e9415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/2345545066 >> 172.21.15.71:0/2345545066 conn(0x7f3e94069770 msgr2=0x7f3e94069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/2345545066 shutdown_connections 2023-10-26T05:48:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/2345545066 wait complete. 2023-10-26T05:48:10.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 Processor -- start 2023-10-26T05:48:10.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- start start 2023-10-26T05:48:10.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 0x7f3e940b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e9413d420 0x7f3e940ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e940a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ad240 con 0x7f3e940ac740 2023-10-26T05:48:10.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ad520 con 0x7f3e9415b9f0 2023-10-26T05:48:10.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940ad800 con 0x7f3e9413d420 2023-10-26T05:48:10.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 0x7f3e940b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 0x7f3e940b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38274/0 (socket says 172.21.15.71:38274) 2023-10-26T05:48:10.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 -- 172.21.15.71:0/109703150 learned_addr learned my addr 172.21.15.71:0/109703150 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:10.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9acd8640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e940a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e99cd6640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e9413d420 0x7f3e940ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 -- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e9413d420 msgr2=0x7f3e940ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e9413d420 0x7f3e940ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 -- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 msgr2=0x7f3e940a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e940a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 -- 172.21.15.71:0/109703150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840094a0 con 0x7f3e940ac740 2023-10-26T05:48:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9acd8640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e940a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9a4d7640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 0x7f3e940b1ea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c420 tx=0x7f3e9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e900052d0 con 0x7f3e940ac740 2023-10-26T05:48:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3e90007020 con 0x7f3e940ac740 2023-10-26T05:48:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c0098e0 con 0x7f3e940ac740 2023-10-26T05:48:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90007430 con 0x7f3e940ac740 2023-10-26T05:48:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.261+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e940addc0 con 0x7f3e940ac740 2023-10-26T05:48:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.265+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e94066b10 con 0x7f3e940ac740 2023-10-26T05:48:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.265+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3e900067e0 con 0x7f3e940ac740 2023-10-26T05:48:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.265+0000 7f3e8affd640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e78066470 0x7f3e78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.265+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3e90083360 con 0x7f3e940ac740 2023-10-26T05:48:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.265+0000 7f3e99cd6640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e78066470 0x7f3e78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.265+0000 7f3e99cd6640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e78066470 0x7f3e78068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e9414e9e0 tx=0x7f3e840091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.269+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3e900507d0 con 0x7f3e940ac740 2023-10-26T05:48:10.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e9415be40 con 0x7f3e940ac740 2023-10-26T05:48:10.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e8affd640 1 -- 172.21.15.71:0/109703150 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f3e9415be40 con 0x7f3e940ac740 2023-10-26T05:48:10.465 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:10.465 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:48:10.466 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:48:10.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e78066470 msgr2=0x7f3e78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e78066470 0x7f3e78068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e9414e9e0 tx=0x7f3e840091e0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 msgr2=0x7f3e940b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 0x7f3e940b1ea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c420 tx=0x7f3e9000c8e0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 shutdown_connections 2023-10-26T05:48:10.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e9415b9f0 0x7f3e940a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3e9413d420 0x7f3e940ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e78066470 0x7f3e78068930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 --2- 172.21.15.71:0/109703150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e940ac740 0x7f3e940b1ea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.461+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 >> 172.21.15.71:0/109703150 conn(0x7f3e94069770 msgr2=0x7f3e94069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:10.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.465+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 shutdown_connections 2023-10-26T05:48:10.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.465+0000 7f3e9c73b640 1 -- 172.21.15.71:0/109703150 wait complete. 2023-10-26T05:48:10.485 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/3092605721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4690003f80 msgr2=0x7f46900047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 --2- 172.21.15.71:0/3092605721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4690003f80 0x7f46900047d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4694006ee0 tx=0x7f469401ce70 comp rx=0 tx=0).stop 2023-10-26T05:48:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 -- 172.21.15.71:0/3092605721 reap_dead start 2023-10-26T05:48:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/3092605721 shutdown_connections 2023-10-26T05:48:10.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/3092605721 >> 172.21.15.71:0/3092605721 conn(0x7f46a40cdd80 msgr2=0x7f46a40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:10.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/3092605721 shutdown_connections 2023-10-26T05:48:10.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/3092605721 wait complete. 2023-10-26T05:48:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 Processor -- start 2023-10-26T05:48:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- start start 2023-10-26T05:48:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4690003f80 0x7f46a4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46a40c66a0 0x7f46a4156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 0x7f46a40ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a406a650 con 0x7f46a40c74b0 2023-10-26T05:48:10.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a40c6210 con 0x7f46a40c66a0 2023-10-26T05:48:10.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4157140 con 0x7f4690003f80 2023-10-26T05:48:10.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 0x7f46a40ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 0x7f46a40ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38300/0 (socket says 172.21.15.71:38300) 2023-10-26T05:48:10.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 -- 172.21.15.71:0/1942030030 learned_addr learned my addr 172.21.15.71:0/1942030030 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:10.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46a9a07640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46a40c66a0 0x7f46a4156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aa208640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4690003f80 0x7f46a4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 -- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4690003f80 msgr2=0x7f46a4157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4690003f80 0x7f46a4157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 -- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46a40c66a0 msgr2=0x7f46a4156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46a40c66a0 0x7f46a4156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 -- 172.21.15.71:0/1942030030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a40c58e0 con 0x7f46a40c74b0 2023-10-26T05:48:10.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46a9a07640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46a40c66a0 0x7f46a4156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:10.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46aaa09640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 0x7f46a40ca020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46a40c8360 tx=0x7f469c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:10.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c00eeb0 con 0x7f46a40c74b0 2023-10-26T05:48:10.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f469c003ab0 con 0x7f46a40c74b0 2023-10-26T05:48:10.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c004030 con 0x7f46a40c74b0 2023-10-26T05:48:10.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a40c5bc0 con 0x7f46a40c74b0 2023-10-26T05:48:10.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.601+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a40c7a70 con 0x7f46a40c74b0 2023-10-26T05:48:10.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.605+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a4066b10 con 0x7f46a40c74b0 2023-10-26T05:48:10.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.605+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f469c019030 con 0x7f46a40c74b0 2023-10-26T05:48:10.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.605+0000 7f469affd640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46880664a0 0x7f4688068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.605+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f469c017070 con 0x7f46a40c74b0 2023-10-26T05:48:10.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.605+0000 7f46aa208640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46880664a0 0x7f4688068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.605+0000 7f46aa208640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46880664a0 0x7f4688068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46a0000c10 tx=0x7f46a0006b40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:10.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.609+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f469c051dc0 con 0x7f46a40c74b0 2023-10-26T05:48:10.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.757+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46a40c63a0 con 0x7f46a40c74b0 2023-10-26T05:48:10.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.761+0000 7f469affd640 1 -- 172.21.15.71:0/1942030030 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16670 (secure 0 0 0) 0x7f469c0553e0 con 0x7f46a40c74b0 2023-10-26T05:48:10.766 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:10.766 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:09.368261+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:10.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46880664a0 msgr2=0x7f4688068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46880664a0 0x7f4688068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46a0000c10 tx=0x7f46a0006b40 comp rx=0 tx=0).stop 2023-10-26T05:48:10.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 msgr2=0x7f46a40ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 0x7f46a40ca020 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f46a40c8360 tx=0x7f469c010f40 comp rx=0 tx=0).stop 2023-10-26T05:48:10.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 shutdown_connections 2023-10-26T05:48:10.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46880664a0 0x7f4688068960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46a40c74b0 0x7f46a40ca020 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46a40c66a0 0x7f46a4156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 --2- 172.21.15.71:0/1942030030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4690003f80 0x7f46a4157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 >> 172.21.15.71:0/1942030030 conn(0x7f46a40cdd80 msgr2=0x7f46a40cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:10.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 shutdown_connections 2023-10-26T05:48:10.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.765+0000 7f46ac46c640 1 -- 172.21.15.71:0/1942030030 wait complete. 2023-10-26T05:48:10.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:47:19.148575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:47:20.403929+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:10.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:10.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- 172.21.15.71:0/2886777834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400cd880 msgr2=0x7f3b4015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 --2- 172.21.15.71:0/2886777834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400cd880 0x7f3b4015bab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3b4008a140 tx=0x7f3b3801caa0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- 172.21.15.71:0/2886777834 shutdown_connections 2023-10-26T05:48:10.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 --2- 172.21.15.71:0/2886777834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400cd880 0x7f3b4015bab0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 --2- 172.21.15.71:0/2886777834 >> v2:172.21.15.86:3300/0 conn(0x7f3b400d0470 0x7f3b4006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- 172.21.15.71:0/2886777834 >> 172.21.15.71:0/2886777834 conn(0x7f3b400cdd80 msgr2=0x7f3b400ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:10.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- 172.21.15.71:0/2886777834 shutdown_connections 2023-10-26T05:48:10.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- 172.21.15.71:0/2886777834 wait complete. 2023-10-26T05:48:10.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 Processor -- start 2023-10-26T05:48:10.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- start start 2023-10-26T05:48:10.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b400cd880 0x7f3b401529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b400d0470 0x7f3b400c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 0x7f3b400c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b401588a0 con 0x7f3b400d0850 2023-10-26T05:48:10.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40158af0 con 0x7f3b400d0470 2023-10-26T05:48:10.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b47f39640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40158dd0 con 0x7f3b400cd880 2023-10-26T05:48:10.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b464d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 0x7f3b400c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b464d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 0x7f3b400c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38322/0 (socket says 172.21.15.71:38322) 2023-10-26T05:48:10.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.901+0000 7f3b464d6640 1 -- 172.21.15.71:0/958340790 learned_addr learned my addr 172.21.15.71:0/958340790 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:10.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b464d6640 1 -- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b400cd880 msgr2=0x7f3b401529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b464d6640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b400cd880 0x7f3b401529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b464d6640 1 -- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b400d0470 msgr2=0x7f3b400c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:10.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b464d6640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b400d0470 0x7f3b400c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:10.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b464d6640 1 -- 172.21.15.71:0/958340790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b38009820 con 0x7f3b400d0850 2023-10-26T05:48:10.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b464d6640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 0x7f3b400c4eb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b380196b0 tx=0x7f3b380a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:10.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38003b80 con 0x7f3b400d0850 2023-10-26T05:48:10.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b38003dd0 con 0x7f3b400d0850 2023-10-26T05:48:10.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b401590b0 con 0x7f3b400d0850 2023-10-26T05:48:10.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38004290 con 0x7f3b400d0850 2023-10-26T05:48:10.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b400c7510 con 0x7f3b400d0850 2023-10-26T05:48:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b40066b10 con 0x7f3b400d0850 2023-10-26T05:48:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b380043f0 con 0x7f3b400d0850 2023-10-26T05:48:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b367fc640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b24066470 0x7f3b24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3b3811ea50 con 0x7f3b400d0850 2023-10-26T05:48:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b45cd5640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b24066470 0x7f3b24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:10.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.905+0000 7f3b45cd5640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b24066470 0x7f3b24068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0058d0 tx=0x7f3b3c005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:10.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:10.909+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b380ebe50 con 0x7f3b400d0850 2023-10-26T05:48:11.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b400721e0 con 0x7f3b400d0850 2023-10-26T05:48:11.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b367fc640 1 -- 172.21.15.71:0/958340790 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b380b2020 con 0x7f3b400d0850 2023-10-26T05:48:11.073 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:11.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b24066470 msgr2=0x7f3b24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b24066470 0x7f3b24068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0058d0 tx=0x7f3b3c005860 comp rx=0 tx=0).stop 2023-10-26T05:48:11.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 msgr2=0x7f3b400c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 0x7f3b400c4eb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3b380196b0 tx=0x7f3b380a5a80 comp rx=0 tx=0).stop 2023-10-26T05:48:11.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 shutdown_connections 2023-10-26T05:48:11.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b24066470 0x7f3b24068930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b400d0850 0x7f3b400c4eb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b400d0470 0x7f3b400c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 --2- 172.21.15.71:0/958340790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b400cd880 0x7f3b401529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 >> 172.21.15.71:0/958340790 conn(0x7f3b400cdd80 msgr2=0x7f3b400b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 shutdown_connections 2023-10-26T05:48:11.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.069+0000 7f3b47f39640 1 -- 172.21.15.71:0/958340790 wait complete. 2023-10-26T05:48:11.091 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:11.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 -- 172.21.15.71:0/1312302767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c158b80 msgr2=0x7fad2c06ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 --2- 172.21.15.71:0/1312302767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c158b80 0x7fad2c06ad00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad2c065ec0 tx=0x7fad2801c830 comp rx=0 tx=0).stop 2023-10-26T05:48:11.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 -- 172.21.15.71:0/1312302767 shutdown_connections 2023-10-26T05:48:11.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 --2- 172.21.15.71:0/1312302767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c158b80 0x7fad2c06ad00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 --2- 172.21.15.71:0/1312302767 >> v2:172.21.15.86:3300/0 conn(0x7fad2c0b1520 0x7fad2c0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 -- 172.21.15.71:0/1312302767 >> 172.21.15.71:0/1312302767 conn(0x7fad2c0ca230 msgr2=0x7fad2c0ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 -- 172.21.15.71:0/1312302767 shutdown_connections 2023-10-26T05:48:11.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 -- 172.21.15.71:0/1312302767 wait complete. 2023-10-26T05:48:11.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 Processor -- start 2023-10-26T05:48:11.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.213+0000 7fad3410d640 1 -- start start 2023-10-26T05:48:11.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad2c070e10 0x7fad2c14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 0x7fad2c14cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad2c158b80 0x7fad2c156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c14b3e0 con 0x7fad2c0b1520 2023-10-26T05:48:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c15bbb0 con 0x7fad2c070e10 2023-10-26T05:48:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c15ca10 con 0x7fad2c158b80 2023-10-26T05:48:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 0x7fad2c14cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 0x7fad2c14cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38340/0 (socket says 172.21.15.71:38340) 2023-10-26T05:48:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 -- 172.21.15.71:0/341663362 learned_addr learned my addr 172.21.15.71:0/341663362 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad31ea9640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad2c070e10 0x7fad2c14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad326aa640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad2c158b80 0x7fad2c156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 -- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad2c158b80 msgr2=0x7fad2c156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad2c158b80 0x7fad2c156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 -- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad2c070e10 msgr2=0x7fad2c14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad2c070e10 0x7fad2c14aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 -- 172.21.15.71:0/341663362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009560 con 0x7fad2c0b1520 2023-10-26T05:48:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad326aa640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad2c158b80 0x7fad2c156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:11.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad316a8640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 0x7fad2c14cc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00c450 tx=0x7fad1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:11.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c012830 con 0x7fad2c0b1520 2023-10-26T05:48:11.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad1c012aa0 con 0x7fad2c0b1520 2023-10-26T05:48:11.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c14b810 con 0x7fad2c0b1520 2023-10-26T05:48:11.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c011380 con 0x7fad2c0b1520 2023-10-26T05:48:11.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c15c0e0 con 0x7fad2c0b1520 2023-10-26T05:48:11.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c066d00 con 0x7fad2c0b1520 2023-10-26T05:48:11.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad1c0114e0 con 0x7fad2c0b1520 2023-10-26T05:48:11.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad227fc640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad10066470 0x7fad10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fad1c083f00 con 0x7fad2c0b1520 2023-10-26T05:48:11.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.217+0000 7fad31ea9640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad10066470 0x7fad10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.221+0000 7fad31ea9640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad10066470 0x7fad10068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad24005c50 tx=0x7fad24005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:11.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.221+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad1c051300 con 0x7fad2c0b1520 2023-10-26T05:48:11.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.373+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad2c152d60 con 0x7fad2c0b1520 2023-10-26T05:48:11.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad227fc640 1 -- 172.21.15.71:0/341663362 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad1c054920 con 0x7fad2c0b1520 2023-10-26T05:48:11.380 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad10066470 msgr2=0x7fad10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad10066470 0x7fad10068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fad24005c50 tx=0x7fad24005be0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 msgr2=0x7fad2c14cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 0x7fad2c14cc00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00c450 tx=0x7fad1c00c910 comp rx=0 tx=0).stop 2023-10-26T05:48:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 shutdown_connections 2023-10-26T05:48:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad2c158b80 0x7fad2c156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad10066470 0x7fad10068930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad2c0b1520 0x7fad2c14cc00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 --2- 172.21.15.71:0/341663362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad2c070e10 0x7fad2c14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 >> 172.21.15.71:0/341663362 conn(0x7fad2c0ca230 msgr2=0x7fad2c14f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 shutdown_connections 2023-10-26T05:48:11.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.377+0000 7fad3410d640 1 -- 172.21.15.71:0/341663362 wait complete. 2023-10-26T05:48:11.399 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:11.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 -- 172.21.15.71:0/901298489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900d0850 msgr2=0x7fcc9015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 --2- 172.21.15.71:0/901298489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900d0850 0x7fcc9015b3a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcc9008a3b0 tx=0x7fcc8401c7f0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 -- 172.21.15.71:0/901298489 shutdown_connections 2023-10-26T05:48:11.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 --2- 172.21.15.71:0/901298489 >> v2:172.21.15.86:3300/0 conn(0x7fcc900cd880 0x7fcc9015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 --2- 172.21.15.71:0/901298489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900d0850 0x7fcc9015b3a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 -- 172.21.15.71:0/901298489 >> 172.21.15.71:0/901298489 conn(0x7fcc900cdd80 msgr2=0x7fcc900ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 -- 172.21.15.71:0/901298489 shutdown_connections 2023-10-26T05:48:11.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.513+0000 7fcc96a58640 1 -- 172.21.15.71:0/901298489 wait complete. 2023-10-26T05:48:11.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 Processor -- start 2023-10-26T05:48:11.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- start start 2023-10-26T05:48:11.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 0x7fcc9015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc900d0470 0x7fcc9015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc900d0850 0x7fcc90151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9015c580 con 0x7fcc900cd880 2023-10-26T05:48:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900c92e0 con 0x7fcc900d0470 2023-10-26T05:48:11.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9015d310 con 0x7fcc900d0850 2023-10-26T05:48:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 0x7fcc9015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 0x7fcc9015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38358/0 (socket says 172.21.15.71:38358) 2023-10-26T05:48:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 -- 172.21.15.71:0/3427957501 learned_addr learned my addr 172.21.15.71:0/3427957501 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8f7fe640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc900d0470 0x7fcc9015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 -- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc900d0850 msgr2=0x7fcc90151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc900d0850 0x7fcc90151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 -- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc900d0470 msgr2=0x7fcc9015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc900d0470 0x7fcc9015c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 -- 172.21.15.71:0/3427957501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84009560 con 0x7fcc900cd880 2023-10-26T05:48:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8ffff640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 0x7fcc9015a9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c820 tx=0x7fcc8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80003b80 con 0x7fcc900cd880 2023-10-26T05:48:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc80003ce0 con 0x7fcc900cd880 2023-10-26T05:48:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9015c790 con 0x7fcc900cd880 2023-10-26T05:48:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000aab0 con 0x7fcc900cd880 2023-10-26T05:48:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9015cd50 con 0x7fcc900cd880 2023-10-26T05:48:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc90066b10 con 0x7fcc900cd880 2023-10-26T05:48:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.517+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcc8000ac10 con 0x7fcc900cd880 2023-10-26T05:48:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.521+0000 7fcc8cff9640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc7c066470 0x7fcc7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.521+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcc80083f90 con 0x7fcc900cd880 2023-10-26T05:48:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.521+0000 7fcc8f7fe640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc7c066470 0x7fcc7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.521+0000 7fcc8f7fe640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc7c066470 0x7fcc7c068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc840af000 tx=0x7fcc8401b030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:11.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.525+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcc800b5090 con 0x7fcc900cd880 2023-10-26T05:48:11.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.669+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc900b58e0 con 0x7fcc900cd880 2023-10-26T05:48:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.669+0000 7fcc8cff9640 1 -- 172.21.15.71:0/3427957501 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc80051420 con 0x7fcc900cd880 2023-10-26T05:48:11.675 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc7c066470 msgr2=0x7fcc7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc7c066470 0x7fcc7c068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc840af000 tx=0x7fcc8401b030 comp rx=0 tx=0).stop 2023-10-26T05:48:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 msgr2=0x7fcc9015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 0x7fcc9015a9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c820 tx=0x7fcc8000cce0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 shutdown_connections 2023-10-26T05:48:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc900d0850 0x7fcc90151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc900d0470 0x7fcc9015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc7c066470 0x7fcc7c068930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 --2- 172.21.15.71:0/3427957501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc900cd880 0x7fcc9015a9f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 >> 172.21.15.71:0/3427957501 conn(0x7fcc900cdd80 msgr2=0x7fcc90154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 shutdown_connections 2023-10-26T05:48:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.673+0000 7fcc96a58640 1 -- 172.21.15.71:0/3427957501 wait complete. 2023-10-26T05:48:11.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:11.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/359066109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d015bbc0 msgr2=0x7fb0d0070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/359066109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d015bbc0 0x7fb0d0070690 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb0d006db30 tx=0x7fb0c801cd60 comp rx=0 tx=0).stop 2023-10-26T05:48:11.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/359066109 shutdown_connections 2023-10-26T05:48:11.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/359066109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d015bbc0 0x7fb0d0070690 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/359066109 >> v2:172.21.15.86:3300/0 conn(0x7fb0d013d420 0x7fb0d00ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/359066109 >> 172.21.15.71:0/359066109 conn(0x7fb0d0069770 msgr2=0x7fb0d0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/359066109 shutdown_connections 2023-10-26T05:48:11.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/359066109 wait complete. 2023-10-26T05:48:11.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 Processor -- start 2023-10-26T05:48:11.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- start start 2023-10-26T05:48:11.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 0x7fb0d00ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0d013d800 0x7fb0d01f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d015bbc0 0x7fb0d00abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01374c0 con 0x7fb0d013d420 2023-10-26T05:48:11.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00adab0 con 0x7fb0d015bbc0 2023-10-26T05:48:11.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0155660 con 0x7fb0d013d800 2023-10-26T05:48:11.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 0x7fb0d00ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 0x7fb0d00ac950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39028/0 (socket says 172.21.15.71:39028) 2023-10-26T05:48:11.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 -- 172.21.15.71:0/2485481979 learned_addr learned my addr 172.21.15.71:0/2485481979 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:11.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 -- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0d013d800 msgr2=0x7fb0d01f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0d013d800 0x7fb0d01f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d64c9640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d015bbc0 0x7fb0d00abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 -- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d015bbc0 msgr2=0x7fb0d00abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d015bbc0 0x7fb0d00abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 -- 172.21.15.71:0/2485481979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80098e0 con 0x7fb0d013d420 2023-10-26T05:48:11.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d5cc8640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 0x7fb0d00ac950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c970 tx=0x7fb0cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:11.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc007bb0 con 0x7fb0d013d420 2023-10-26T05:48:11.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb0cc007e00 con 0x7fb0d013d420 2023-10-26T05:48:11.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d00ac240 con 0x7fb0d013d420 2023-10-26T05:48:11.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc005280 con 0x7fb0d013d420 2023-10-26T05:48:11.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d00ad210 con 0x7fb0d013d420 2023-10-26T05:48:11.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.801+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d0066b10 con 0x7fb0d013d420 2023-10-26T05:48:11.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.805+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb0cc0053e0 con 0x7fb0d013d420 2023-10-26T05:48:11.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.805+0000 7fb0c67fc640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0b4066470 0x7fb0b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:11.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.805+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb0cc084600 con 0x7fb0d013d420 2023-10-26T05:48:11.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.805+0000 7fb0d54c7640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0b4066470 0x7fb0b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:11.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.805+0000 7fb0d54c7640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0b4066470 0x7fb0b4068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0007fd0 tx=0x7fb0c0007960 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:11.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.809+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0cc051980 con 0x7fb0d013d420 2023-10-26T05:48:11.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.977+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0d00721e0 con 0x7fb0d013d420 2023-10-26T05:48:11.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0c67fc640 1 -- 172.21.15.71:0/2485481979 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7fb0cc054fa0 con 0x7fb0d013d420 2023-10-26T05:48:11.984 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:11.984 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:48:11.984 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:48:11.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0b4066470 msgr2=0x7fb0b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0b4066470 0x7fb0b4068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0007fd0 tx=0x7fb0c0007960 comp rx=0 tx=0).stop 2023-10-26T05:48:11.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 msgr2=0x7fb0d00ac950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:11.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 0x7fb0d00ac950 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c970 tx=0x7fb0cc00ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:11.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 shutdown_connections 2023-10-26T05:48:11.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0d015bbc0 0x7fb0d00abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0d013d800 0x7fb0d01f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0b4066470 0x7fb0b4068930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 --2- 172.21.15.71:0/2485481979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0d013d420 0x7fb0d00ac950 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:11.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 >> 172.21.15.71:0/2485481979 conn(0x7fb0d0069770 msgr2=0x7fb0d013c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:11.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 shutdown_connections 2023-10-26T05:48:11.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:11.981+0000 7fb0d7f2c640 1 -- 172.21.15.71:0/2485481979 wait complete. 2023-10-26T05:48:12.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:12.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1090045770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 msgr2=0x7fa89815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1090045770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa89815ce50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa898157430 tx=0x7fa88401c720 comp rx=0 tx=0).stop 2023-10-26T05:48:12.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1090045770 shutdown_connections 2023-10-26T05:48:12.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1090045770 >> v2:172.21.15.86:3300/0 conn(0x7fa89815d390 0x7fa89815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1090045770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa89815ce50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1090045770 >> 172.21.15.71:0/1090045770 conn(0x7fa8980b0f20 msgr2=0x7fa8980b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:12.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1090045770 shutdown_connections 2023-10-26T05:48:12.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1090045770 wait complete. 2023-10-26T05:48:12.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 Processor -- start 2023-10-26T05:48:12.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.117+0000 7fa89ee2e640 1 -- start start 2023-10-26T05:48:12.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa898155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa89815c720 0x7fa898149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa89815d390 0x7fa8981486f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898155720 con 0x7fa8980cede0 2023-10-26T05:48:12.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898155a00 con 0x7fa89815d390 2023-10-26T05:48:12.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa898155ce0 con 0x7fa89815c720 2023-10-26T05:48:12.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa898155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa898155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39044/0 (socket says 172.21.15.71:39044) 2023-10-26T05:48:12.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 -- 172.21.15.71:0/1872357966 learned_addr learned my addr 172.21.15.71:0/1872357966 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:12.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 -- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa89815c720 msgr2=0x7fa898149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa89815c720 0x7fa898149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 -- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa89815d390 msgr2=0x7fa8981486f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa89815d390 0x7fa8981486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 -- 172.21.15.71:0/1872357966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840094a0 con 0x7fa8980cede0 2023-10-26T05:48:12.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89cbca640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa898155140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c550 tx=0x7fa88c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:12.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c012830 con 0x7fa8980cede0 2023-10-26T05:48:12.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa88c012a80 con 0x7fa8980cede0 2023-10-26T05:48:12.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa898155fc0 con 0x7fa8980cede0 2023-10-26T05:48:12.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c012e90 con 0x7fa8980cede0 2023-10-26T05:48:12.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89814a4c0 con 0x7fa8980cede0 2023-10-26T05:48:12.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa898066d00 con 0x7fa8980cede0 2023-10-26T05:48:12.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa88c01a3b0 con 0x7fa8980cede0 2023-10-26T05:48:12.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa8957fa640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa87c066470 0x7fa87c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa88c084070 con 0x7fa8980cede0 2023-10-26T05:48:12.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.121+0000 7fa897fff640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa87c066470 0x7fa87c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.125+0000 7fa897fff640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa87c066470 0x7fa87c068930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa8840af000 tx=0x7fa8840194c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:12.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.125+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa88c054f10 con 0x7fa8980cede0 2023-10-26T05:48:12.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.269+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa898156470 con 0x7fa8980cede0 2023-10-26T05:48:12.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.269+0000 7fa8957fa640 1 -- 172.21.15.71:0/1872357966 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16670 (secure 0 0 0) 0x7fa88c054aa0 con 0x7fa8980cede0 2023-10-26T05:48:12.276 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:12.276 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:09.368261+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa87c066470 msgr2=0x7fa87c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa87c066470 0x7fa87c068930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa8840af000 tx=0x7fa8840194c0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 msgr2=0x7fa898155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa898155140 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c550 tx=0x7fa88c00ca10 comp rx=0 tx=0).stop 2023-10-26T05:48:12.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 shutdown_connections 2023-10-26T05:48:12.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa89815d390 0x7fa8981486f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa89815c720 0x7fa898149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa87c066470 0x7fa87c068930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 --2- 172.21.15.71:0/1872357966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8980cede0 0x7fa898155140 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 >> 172.21.15.71:0/1872357966 conn(0x7fa8980b0f20 msgr2=0x7fa8980b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:12.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 shutdown_connections 2023-10-26T05:48:12.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.273+0000 7fa89ee2e640 1 -- 172.21.15.71:0/1872357966 wait complete. 2023-10-26T05:48:12.294 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:48:12.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:12.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 -- 172.21.15.71:0/3572601731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b340c9e90 msgr2=0x7f7b3406b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/3572601731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b340c9e90 0x7f7b3406b010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009840 tx=0x7f7b2401c740 comp rx=0 tx=0).stop 2023-10-26T05:48:12.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 -- 172.21.15.71:0/3572601731 shutdown_connections 2023-10-26T05:48:12.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/3572601731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b341585f0 0x7f7b3406b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/3572601731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b340c9e90 0x7f7b3406b010 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 -- 172.21.15.71:0/3572601731 >> 172.21.15.71:0/3572601731 conn(0x7f7b34070170 msgr2=0x7f7b34070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:12.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 -- 172.21.15.71:0/3572601731 shutdown_connections 2023-10-26T05:48:12.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.413+0000 7f7b39ccb640 1 -- 172.21.15.71:0/3572601731 wait complete. 2023-10-26T05:48:12.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 Processor -- start 2023-10-26T05:48:12.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- start start 2023-10-26T05:48:12.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 0x7f7b341549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b341585f0 0x7f7b34158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b34138eb0 0x7f7b3414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34154540 con 0x7f7b340ca270 2023-10-26T05:48:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34154f10 con 0x7f7b34138eb0 2023-10-26T05:48:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34153930 con 0x7f7b341585f0 2023-10-26T05:48:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 0x7f7b341549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 0x7f7b341549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39056/0 (socket says 172.21.15.71:39056) 2023-10-26T05:48:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 -- 172.21.15.71:0/1078059 learned_addr learned my addr 172.21.15.71:0/1078059 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b33fff640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b34138eb0 0x7f7b3414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b2bfff640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b341585f0 0x7f7b34158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 -- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b341585f0 msgr2=0x7f7b34158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b341585f0 0x7f7b34158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 -- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b34138eb0 msgr2=0x7f7b3414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b34138eb0 0x7f7b3414c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 -- 172.21.15.71:0/1078059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c009820 con 0x7f7b340ca270 2023-10-26T05:48:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b337fe640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 0x7f7b341549d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009790 tx=0x7f7b240a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:12.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b240bb070 con 0x7f7b340ca270 2023-10-26T05:48:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7b240051c0 con 0x7f7b340ca270 2023-10-26T05:48:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b240094a0 con 0x7f7b340ca270 2023-10-26T05:48:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b24005590 con 0x7f7b340ca270 2023-10-26T05:48:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b34153ef0 con 0x7f7b340ca270 2023-10-26T05:48:12.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b34066d00 con 0x7f7b340ca270 2023-10-26T05:48:12.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7b240b4040 con 0x7f7b340ca270 2023-10-26T05:48:12.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b30ff9640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b14066470 0x7f7b14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.417+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7b2411dde0 con 0x7f7b340ca270 2023-10-26T05:48:12.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.421+0000 7f7b2bfff640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b14066470 0x7f7b14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.421+0000 7f7b2bfff640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b14066470 0x7f7b14068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7b34158470 tx=0x7f7b1c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:12.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.421+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7b240eb1e0 con 0x7f7b340ca270 2023-10-26T05:48:12.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b341581c0 con 0x7f7b340ca270 2023-10-26T05:48:12.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b30ff9640 1 -- 172.21.15.71:0/1078059 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f7b240b2090 con 0x7f7b340ca270 2023-10-26T05:48:12.606 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:12.606 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:48:12.606 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:48:12.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b14066470 msgr2=0x7f7b14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b14066470 0x7f7b14068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7b34158470 tx=0x7f7b1c009040 comp rx=0 tx=0).stop 2023-10-26T05:48:12.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 msgr2=0x7f7b341549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 0x7f7b341549d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009790 tx=0x7f7b240a5810 comp rx=0 tx=0).stop 2023-10-26T05:48:12.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 shutdown_connections 2023-10-26T05:48:12.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.601+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b34138eb0 0x7f7b3414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.605+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7b341585f0 0x7f7b34158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.605+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b14066470 0x7f7b14068930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.605+0000 7f7b39ccb640 1 --2- 172.21.15.71:0/1078059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b340ca270 0x7f7b341549d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.605+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 >> 172.21.15.71:0/1078059 conn(0x7f7b34070170 msgr2=0x7f7b340b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:12.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.605+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 shutdown_connections 2023-10-26T05:48:12.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.605+0000 7f7b39ccb640 1 -- 172.21.15.71:0/1078059 wait complete. 2023-10-26T05:48:12.623 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:12.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/3494446991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 msgr2=0x7f3588004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 --2- 172.21.15.71:0/3494446991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f3588004c50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f35840016b0 tx=0x7f358401c780 comp rx=0 tx=0).stop 2023-10-26T05:48:12.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/3494446991 shutdown_connections 2023-10-26T05:48:12.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 --2- 172.21.15.71:0/3494446991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f3588004c50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/3494446991 >> 172.21.15.71:0/3494446991 conn(0x7f359c070800 msgr2=0x7f359c070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:12.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/3494446991 shutdown_connections 2023-10-26T05:48:12.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/3494446991 wait complete. 2023-10-26T05:48:12.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 Processor -- start 2023-10-26T05:48:12.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- start start 2023-10-26T05:48:12.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f359c0ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f359c0abe70 0x7f359c0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359c0c9a60 0x7f359c1337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0ac2f0 con 0x7f35880047d0 2023-10-26T05:48:12.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1316a0 con 0x7f359c0c9a60 2023-10-26T05:48:12.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c131910 con 0x7f359c0abe70 2023-10-26T05:48:12.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f359c0ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f359c0ab930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39070/0 (socket says 172.21.15.71:39070) 2023-10-26T05:48:12.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 -- 172.21.15.71:0/416482003 learned_addr learned my addr 172.21.15.71:0/416482003 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:12.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 -- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f359c0abe70 msgr2=0x7f359c0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f359c0abe70 0x7f359c0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 -- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359c0c9a60 msgr2=0x7f359c1337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359c0c9a60 0x7f359c1337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 -- 172.21.15.71:0/416482003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c131bf0 con 0x7f35880047d0 2023-10-26T05:48:12.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f359b7fe640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f359c0ab930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f358c00f710 tx=0x7f358c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:12.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00abd0 con 0x7f35880047d0 2023-10-26T05:48:12.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f358c00ae20 con 0x7f35880047d0 2023-10-26T05:48:12.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0126d0 con 0x7f35880047d0 2023-10-26T05:48:12.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c131ed0 con 0x7f35880047d0 2023-10-26T05:48:12.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.737+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c0c2fc0 con 0x7f35880047d0 2023-10-26T05:48:12.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.741+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c066d00 con 0x7f35880047d0 2023-10-26T05:48:12.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.741+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f358c012830 con 0x7f35880047d0 2023-10-26T05:48:12.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.741+0000 7f3598ff9640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f357c0664a0 0x7f357c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:12.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.741+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f358c087620 con 0x7f35880047d0 2023-10-26T05:48:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.741+0000 7f3593fff640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f357c0664a0 0x7f357c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.741+0000 7f3593fff640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f357c0664a0 0x7f357c068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3584001680 tx=0x7f3588005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:12.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.745+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f358c054a20 con 0x7f35880047d0 2023-10-26T05:48:12.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.921+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f359c140e90 con 0x7f35880047d0 2023-10-26T05:48:12.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.921+0000 7f3598ff9640 1 -- 172.21.15.71:0/416482003 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4901 (secure 0 0 0) 0x7f358c058040 con 0x7f35880047d0 2023-10-26T05:48:12.926 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:12.927 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4280,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/84936742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":84936742},{"type":"v1","addr":"172.21.15.86:6835","nonce":84936742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4302,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/2166267402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2166267402},{"type":"v1","addr":"172.21.15.71:6833","nonce":2166267402}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4305,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2003875059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2003875059},{"type":"v1","addr":"172.21.15.71:6835","nonce":2003875059}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:47:21.405825+0000","modified":"2023-10-26T05:47:23.437258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4369},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4369":{"gid":4369,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/2733906153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2733906153},{"type":"v1","addr":"172.21.15.86:6837","nonce":2733906153}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-26T05:48:12.927 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2023-10-26T05:48:12.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f357c0664a0 msgr2=0x7f357c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f357c0664a0 0x7f357c068960 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3584001680 tx=0x7f3588005800 comp rx=0 tx=0).stop 2023-10-26T05:48:12.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 msgr2=0x7f359c0ab930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:12.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f359c0ab930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f358c00f710 tx=0x7f358c00fbd0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 shutdown_connections 2023-10-26T05:48:12.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f359c0c9a60 0x7f359c1337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f359c0abe70 0x7f359c0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f357c0664a0 0x7f357c068960 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 --2- 172.21.15.71:0/416482003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35880047d0 0x7f359c0ab930 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:12.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 >> 172.21.15.71:0/416482003 conn(0x7f359c070800 msgr2=0x7f359c06bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:12.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 shutdown_connections 2023-10-26T05:48:12.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:12.925+0000 7f35a206d640 1 -- 172.21.15.71:0/416482003 wait complete. 2023-10-26T05:48:12.943 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5c3be640 1 -- 172.21.15.71:0/12839352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c40004d10 msgr2=0x7f7c40005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/12839352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c40004d10 0x7f7c40005190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c009ee0 tx=0x7f7c4c01d1d0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5a95b640 1 -- 172.21.15.71:0/12839352 reap_dead start 2023-10-26T05:48:13.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5c3be640 1 -- 172.21.15.71:0/12839352 shutdown_connections 2023-10-26T05:48:13.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5c3be640 1 -- 172.21.15.71:0/12839352 >> 172.21.15.71:0/12839352 conn(0x7f7c54069770 msgr2=0x7f7c54069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:13.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5c3be640 1 -- 172.21.15.71:0/12839352 shutdown_connections 2023-10-26T05:48:13.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.057+0000 7f7c5c3be640 1 -- 172.21.15.71:0/12839352 wait complete. 2023-10-26T05:48:13.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 Processor -- start 2023-10-26T05:48:13.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- start start 2023-10-26T05:48:13.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c40004d10 0x7f7c540ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 0x7f7c5406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c541f2930 0x7f7c541f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5406ad20 con 0x7f7c5406a360 2023-10-26T05:48:13.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540a66e0 con 0x7f7c541f2930 2023-10-26T05:48:13.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c541f3820 con 0x7f7c40004d10 2023-10-26T05:48:13.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 0x7f7c5406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 0x7f7c5406a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39096/0 (socket says 172.21.15.71:39096) 2023-10-26T05:48:13.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 -- 172.21.15.71:0/4077210285 learned_addr learned my addr 172.21.15.71:0/4077210285 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:13.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5a15a640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c40004d10 0x7f7c540ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5a95b640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c541f2930 0x7f7c541f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 -- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c40004d10 msgr2=0x7f7c540ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c40004d10 0x7f7c540ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 -- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c541f2930 msgr2=0x7f7c541f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c541f2930 0x7f7c541f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 -- 172.21.15.71:0/4077210285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c541f32f0 con 0x7f7c5406a360 2023-10-26T05:48:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c59959640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 0x7f7c5406a7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400e760 tx=0x7f7c4400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400cc60 con 0x7f7c5406a360 2023-10-26T05:48:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c4400ee70 con 0x7f7c5406a360 2023-10-26T05:48:13.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c54157d10 con 0x7f7c5406a360 2023-10-26T05:48:13.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c440116b0 con 0x7f7c5406a360 2023-10-26T05:48:13.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c540c5b60 con 0x7f7c5406a360 2023-10-26T05:48:13.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c54066b10 con 0x7f7c5406a360 2023-10-26T05:48:13.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c44011810 con 0x7f7c5406a360 2023-10-26T05:48:13.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.061+0000 7f7c4affd640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c38066470 0x7f7c38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.065+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7c44084ac0 con 0x7f7c5406a360 2023-10-26T05:48:13.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.065+0000 7f7c5a15a640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c38066470 0x7f7c38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.065+0000 7f7c5a15a640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c38066470 0x7f7c38068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c50001ec0 tx=0x7f7c50001d90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:13.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.069+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c44051ec0 con 0x7f7c5406a360 2023-10-26T05:48:13.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.213+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c540721e0 con 0x7f7c5406a360 2023-10-26T05:48:13.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.213+0000 7f7c4affd640 1 -- 172.21.15.71:0/4077210285 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16670 (secure 0 0 0) 0x7f7c4401f020 con 0x7f7c5406a360 2023-10-26T05:48:13.219 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:13.219 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":22,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:09.368261+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:47:19.148575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-26T05:47:20.403929+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:13.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c38066470 msgr2=0x7f7c38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c38066470 0x7f7c38068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c50001ec0 tx=0x7f7c50001d90 comp rx=0 tx=0).stop 2023-10-26T05:48:13.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 msgr2=0x7f7c5406a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 0x7f7c5406a7e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400e760 tx=0x7f7c4400ec20 comp rx=0 tx=0).stop 2023-10-26T05:48:13.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 shutdown_connections 2023-10-26T05:48:13.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c541f2930 0x7f7c541f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c38066470 0x7f7c38068930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c5406a360 0x7f7c5406a7e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 --2- 172.21.15.71:0/4077210285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c40004d10 0x7f7c540ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 >> 172.21.15.71:0/4077210285 conn(0x7f7c54069770 msgr2=0x7f7c5413c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 shutdown_connections 2023-10-26T05:48:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.217+0000 7f7c5c3be640 1 -- 172.21.15.71:0/4077210285 wait complete. 2023-10-26T05:48:13.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:48:13.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 -- 172.21.15.71:0/3841194458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf540cabd0 msgr2=0x7fdf540b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/3841194458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf540cabd0 0x7fdf540b1300 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdf540b0820 tx=0x7fdf4801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 -- 172.21.15.71:0/3841194458 shutdown_connections 2023-10-26T05:48:13.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/3841194458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf5415ccf0 0x7fdf5415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/3841194458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf540cabd0 0x7fdf540b1300 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/3841194458 >> v2:172.21.15.86:3300/0 conn(0x7fdf540ca7f0 0x7fdf54159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 -- 172.21.15.71:0/3841194458 >> 172.21.15.71:0/3841194458 conn(0x7fdf5406b8c0 msgr2=0x7fdf5406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:13.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 -- 172.21.15.71:0/3841194458 shutdown_connections 2023-10-26T05:48:13.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 -- 172.21.15.71:0/3841194458 wait complete. 2023-10-26T05:48:13.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 Processor -- start 2023-10-26T05:48:13.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.345+0000 7fdf5aae7640 1 -- start start 2023-10-26T05:48:13.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf540ca7f0 0x7fdf54137d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf540cabd0 0x7fdf5413bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 0x7fdf5413a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54138bf0 con 0x7fdf5415ccf0 2023-10-26T05:48:13.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5413ab60 con 0x7fdf540ca7f0 2023-10-26T05:48:13.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5413ae40 con 0x7fdf540cabd0 2023-10-26T05:48:13.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 0x7fdf5413a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 0x7fdf5413a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39116/0 (socket says 172.21.15.71:39116) 2023-10-26T05:48:13.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 -- 172.21.15.71:0/1110751761 learned_addr learned my addr 172.21.15.71:0/1110751761 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:13.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf53fff640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf540cabd0 0x7fdf5413bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 -- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf540cabd0 msgr2=0x7fdf5413bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf540cabd0 0x7fdf5413bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 -- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf540ca7f0 msgr2=0x7fdf54137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf58883640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf540ca7f0 0x7fdf54137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf540ca7f0 0x7fdf54137d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 -- 172.21.15.71:0/1110751761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48009560 con 0x7fdf5415ccf0 2023-10-26T05:48:13.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf59084640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 0x7fdf5413a460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00b390 tx=0x7fdf4c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:13.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c011830 con 0x7fdf5415ccf0 2023-10-26T05:48:13.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf4c011a80 con 0x7fdf5415ccf0 2023-10-26T05:48:13.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf540cb190 con 0x7fdf5415ccf0 2023-10-26T05:48:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c011e90 con 0x7fdf5415ccf0 2023-10-26T05:48:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5413b400 con 0x7fdf5415ccf0 2023-10-26T05:48:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf54066b10 con 0x7fdf5415ccf0 2023-10-26T05:48:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf4c01a400 con 0x7fdf5415ccf0 2023-10-26T05:48:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf517fa640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf400664a0 0x7fdf40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdf4c084230 con 0x7fdf5415ccf0 2023-10-26T05:48:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf58883640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf400664a0 0x7fdf40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.349+0000 7fdf58883640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf400664a0 0x7fdf40068960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf44005fd0 tx=0x7fdf440082b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:13.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.353+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdf4c0516c0 con 0x7fdf5415ccf0 2023-10-26T05:48:13.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.533+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdf540ca660 con 0x7fdf5415ccf0 2023-10-26T05:48:13.537 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:48:13.533+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:48:13.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.537+0000 7fdf517fa640 1 -- 172.21.15.71:0/1110751761 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdf4c054ce0 con 0x7fdf5415ccf0 2023-10-26T05:48:13.543 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:48:13.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf400664a0 msgr2=0x7fdf40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf400664a0 0x7fdf40068960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdf44005fd0 tx=0x7fdf440082b0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 msgr2=0x7fdf5413a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 0x7fdf5413a460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00b390 tx=0x7fdf4c00b850 comp rx=0 tx=0).stop 2023-10-26T05:48:13.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 shutdown_connections 2023-10-26T05:48:13.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf400664a0 0x7fdf40068960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf5415ccf0 0x7fdf5413a460 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf540cabd0 0x7fdf5413bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 --2- 172.21.15.71:0/1110751761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf540ca7f0 0x7fdf54137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 >> 172.21.15.71:0/1110751761 conn(0x7fdf5406b8c0 msgr2=0x7fdf5413efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:13.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 shutdown_connections 2023-10-26T05:48:13.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.541+0000 7fdf5aae7640 1 -- 172.21.15.71:0/1110751761 wait complete. 2023-10-26T05:48:13.556 INFO:tasks.ceph.mds.d.smithi086.stderr: -3293> 2023-10-26T05:47:24.793+0000 7f06e55a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:48:13.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:48:13.621 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:48:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- 172.21.15.71:0/3357664516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 msgr2=0x7fde240ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3357664516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde240ca780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fde2406e940 tx=0x7fde1c01f510 comp rx=0 tx=0).stop 2023-10-26T05:48:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- 172.21.15.71:0/3357664516 shutdown_connections 2023-10-26T05:48:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3357664516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde240ca780 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3357664516 >> v2:172.21.15.86:3300/0 conn(0x7fde24141a40 0x7fde240c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- 172.21.15.71:0/3357664516 >> 172.21.15.71:0/3357664516 conn(0x7fde240aec60 msgr2=0x7fde240af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- 172.21.15.71:0/3357664516 shutdown_connections 2023-10-26T05:48:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- 172.21.15.71:0/3357664516 wait complete. 2023-10-26T05:48:13.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 Processor -- start 2023-10-26T05:48:13.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- start start 2023-10-26T05:48:13.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde240b1230 0x7fde24134a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde241f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde24141a40 0x7fde24149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde241365d0 con 0x7fde240b2370 2023-10-26T05:48:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2414a530 con 0x7fde24141a40 2023-10-26T05:48:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.677+0000 7fde2afbd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde241f4220 con 0x7fde240b1230 2023-10-26T05:48:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde241f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde241f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39136/0 (socket says 172.21.15.71:39136) 2023-10-26T05:48:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 -- 172.21.15.71:0/3472089364 learned_addr learned my addr 172.21.15.71:0/3472089364 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde2955a640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde24141a40 0x7fde24149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde28d59640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde240b1230 0x7fde24134a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 -- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde240b1230 msgr2=0x7fde24134a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde240b1230 0x7fde24134a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 -- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde24141a40 msgr2=0x7fde24149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde24141a40 0x7fde24149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 -- 172.21.15.71:0/3472089364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c0098e0 con 0x7fde240b2370 2023-10-26T05:48:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde2955a640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde24141a40 0x7fde24149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde23fff640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde241f32c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c490 tx=0x7fde10001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10003600 con 0x7fde240b2370 2023-10-26T05:48:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde241f3980 con 0x7fde240b2370 2023-10-26T05:48:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fde10003780 con 0x7fde240b2370 2023-10-26T05:48:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10003bd0 con 0x7fde240b2370 2023-10-26T05:48:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde241f3f40 con 0x7fde240b2370 2023-10-26T05:48:13.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde24066b10 con 0x7fde240b2370 2023-10-26T05:48:13.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fde10003d30 con 0x7fde240b2370 2023-10-26T05:48:13.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde217fa640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fde08066470 0x7fde08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:13.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fde10083660 con 0x7fde240b2370 2023-10-26T05:48:13.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde28d59640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fde08066470 0x7fde08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:13.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.681+0000 7fde28d59640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fde08066470 0x7fde08068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde18006fd0 tx=0x7fde18008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:13.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.685+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fde10050aa0 con 0x7fde240b2370 2023-10-26T05:48:13.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:13.857+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fde240721e0 con 0x7fde240b2370 2023-10-26T05:48:14.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.541+0000 7fde217fa640 1 -- 172.21.15.71:0/3472089364 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7fde100540c0 con 0x7fde240b2370 2023-10-26T05:48:14.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fde08066470 msgr2=0x7fde08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:14.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fde08066470 0x7fde08068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde18006fd0 tx=0x7fde18008040 comp rx=0 tx=0).stop 2023-10-26T05:48:14.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 msgr2=0x7fde241f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:14.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde241f32c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fde1000c490 tx=0x7fde10001e80 comp rx=0 tx=0).stop 2023-10-26T05:48:14.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 shutdown_connections 2023-10-26T05:48:14.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde24141a40 0x7fde24149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fde08066470 0x7fde08068930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde240b2370 0x7fde241f32c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 --2- 172.21.15.71:0/3472089364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fde240b1230 0x7fde24134a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 >> 172.21.15.71:0/3472089364 conn(0x7fde240aec60 msgr2=0x7fde24140050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:14.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 shutdown_connections 2023-10-26T05:48:14.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.545+0000 7fde2afbd640 1 -- 172.21.15.71:0/3472089364 wait complete. 2023-10-26T05:48:14.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:48:14.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:48:14.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:48:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1750840778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d80088ab0 msgr2=0x7f1d8006fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1750840778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d80088ab0 0x7f1d8006fd30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009900 tx=0x7f1d7401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1750840778 shutdown_connections 2023-10-26T05:48:14.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1750840778 >> v2:172.21.15.86:3300/0 conn(0x7f1d801684e0 0x7f1d800b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1750840778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d80088ab0 0x7f1d8006fd30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1750840778 >> 172.21.15.71:0/1750840778 conn(0x7f1d800cceb0 msgr2=0x7f1d800cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:14.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1750840778 shutdown_connections 2023-10-26T05:48:14.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.709+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1750840778 wait complete. 2023-10-26T05:48:14.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 Processor -- start 2023-10-26T05:48:14.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- start start 2023-10-26T05:48:14.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80088ab0 0x7f1d8015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:14.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d800b1220 0x7f1d80166c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:14.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 0x7f1d801604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:14.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8015c980 con 0x7f1d801684e0 2023-10-26T05:48:14.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d801674a0 con 0x7f1d80088ab0 2023-10-26T05:48:14.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d80167780 con 0x7f1d800b1220 2023-10-26T05:48:14.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 0x7f1d801604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:14.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 0x7f1d801604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39152/0 (socket says 172.21.15.71:39152) 2023-10-26T05:48:14.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 -- 172.21.15.71:0/1276503225 learned_addr learned my addr 172.21.15.71:0/1276503225 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:14.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d84ffa640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d800b1220 0x7f1d80166c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:14.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d857fb640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80088ab0 0x7f1d8015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:14.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 -- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d800b1220 msgr2=0x7f1d80166c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:14.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d800b1220 0x7f1d80166c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 -- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80088ab0 msgr2=0x7f1d8015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:14.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80088ab0 0x7f1d8015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:14.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 -- 172.21.15.71:0/1276503225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009560 con 0x7f1d801684e0 2023-10-26T05:48:14.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d85ffc640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 0x7f1d801604d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1d78001570 tx=0x7f1d7800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:14.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78003db0 con 0x7f1d801684e0 2023-10-26T05:48:14.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d78005070 con 0x7f1d801684e0 2023-10-26T05:48:14.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8015b960 con 0x7f1d801684e0 2023-10-26T05:48:14.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78005480 con 0x7f1d801684e0 2023-10-26T05:48:14.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8015bf20 con 0x7f1d801684e0 2023-10-26T05:48:14.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d80066b10 con 0x7f1d801684e0 2023-10-26T05:48:14.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.713+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1d780055e0 con 0x7f1d801684e0 2023-10-26T05:48:14.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.717+0000 7f1d71ffb640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:14.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.717+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f1d78083a90 con 0x7f1d801684e0 2023-10-26T05:48:14.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.717+0000 7f1d857fb640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:14.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.717+0000 7f1d857fb640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d740af000 tx=0x7f1d7401b100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:14.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.721+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1d78050e70 con 0x7f1d801684e0 2023-10-26T05:48:14.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:14.865+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1d80065c40 con 0x7f1d801684e0 2023-10-26T05:48:15.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.545+0000 7f1d71ffb640 1 -- 172.21.15.71:0/1276503225 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1d78017020 con 0x7f1d801684e0 2023-10-26T05:48:15.549 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:48:15.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 msgr2=0x7f1d64068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:15.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1d740af000 tx=0x7f1d7401b100 comp rx=0 tx=0).stop 2023-10-26T05:48:15.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 msgr2=0x7f1d801604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:15.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 0x7f1d801604d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1d78001570 tx=0x7f1d7800cd20 comp rx=0 tx=0).stop 2023-10-26T05:48:15.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 shutdown_connections 2023-10-26T05:48:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d801684e0 0x7f1d801604d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d800b1220 0x7f1d80166c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 --2- 172.21.15.71:0/1276503225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80088ab0 0x7f1d8015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 >> 172.21.15.71:0/1276503225 conn(0x7f1d800cceb0 msgr2=0x7f1d8016abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:15.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 shutdown_connections 2023-10-26T05:48:15.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.549+0000 7f1d87a5f640 1 -- 172.21.15.71:0/1276503225 wait complete. 2023-10-26T05:48:15.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:48:15.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d9f79640 1 -- 172.21.15.71:0/1313307965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02bc0039b0 msgr2=0x7f02bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:15.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d9f79640 1 --2- 172.21.15.71:0/1313307965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02bc0039b0 0x7f02bc003dd0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f02c4001ec0 tx=0x7f02c401cf10 comp rx=0 tx=0).stop 2023-10-26T05:48:15.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d3fff640 1 -- 172.21.15.71:0/1313307965 reap_dead start 2023-10-26T05:48:15.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d9f79640 1 -- 172.21.15.71:0/1313307965 shutdown_connections 2023-10-26T05:48:15.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d9f79640 1 -- 172.21.15.71:0/1313307965 >> 172.21.15.71:0/1313307965 conn(0x7f02d40d13f0 msgr2=0x7f02d40d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:15.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d9f79640 1 -- 172.21.15.71:0/1313307965 shutdown_connections 2023-10-26T05:48:15.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.697+0000 7f02d9f79640 1 -- 172.21.15.71:0/1313307965 wait complete. 2023-10-26T05:48:15.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 Processor -- start 2023-10-26T05:48:15.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- start start 2023-10-26T05:48:15.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 0x7f02d4153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d40c4ad0 0x7f02d41520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02d4152610 0x7f02d40c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4153710 con 0x7f02d40c46f0 2023-10-26T05:48:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4152a90 con 0x7f02d40c4ad0 2023-10-26T05:48:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d4152d70 con 0x7f02d4152610 2023-10-26T05:48:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 0x7f02d4153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 0x7f02d4153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39170/0 (socket says 172.21.15.71:39170) 2023-10-26T05:48:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 -- 172.21.15.71:0/3984742815 learned_addr learned my addr 172.21.15.71:0/3984742815 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d2ffd640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d40c4ad0 0x7f02d41520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 -- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02d4152610 msgr2=0x7f02d40c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02d4152610 0x7f02d40c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 -- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d40c4ad0 msgr2=0x7f02d41520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d40c4ad0 0x7f02d41520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 -- 172.21.15.71:0/3984742815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d4153050 con 0x7f02d40c46f0 2023-10-26T05:48:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d2ffd640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d40c4ad0 0x7f02d41520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d37fe640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 0x7f02d4153b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f02c401cd40 tx=0x7f02c4000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c40b3070 con 0x7f02d40c46f0 2023-10-26T05:48:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f02c40a7e20 con 0x7f02d40c46f0 2023-10-26T05:48:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d4153330 con 0x7f02d40c46f0 2023-10-26T05:48:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c40016a0 con 0x7f02d40c46f0 2023-10-26T05:48:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d4150f00 con 0x7f02d40c46f0 2023-10-26T05:48:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d4066b10 con 0x7f02d40c46f0 2023-10-26T05:48:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f02c40ab020 con 0x7f02d40c46f0 2023-10-26T05:48:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02bbfff640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f02c0066470 0x7f02c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f02c411e050 con 0x7f02d40c46f0 2023-10-26T05:48:15.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.701+0000 7f02d2ffd640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f02c0066470 0x7f02c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:15.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.705+0000 7f02d2ffd640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f02c0066470 0x7f02c0068930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005b70 tx=0x7f02c80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:15.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.705+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f02c40eb640 con 0x7f02d40c46f0 2023-10-26T05:48:15.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:15.861+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f02d4152eb0 con 0x7f02d40c46f0 2023-10-26T05:48:16.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.549+0000 7f02bbfff640 1 -- 172.21.15.71:0/3984742815 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f02d4152eb0 con 0x7f02d40c46f0 2023-10-26T05:48:16.553 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:48:16.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.553+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f02c0066470 msgr2=0x7f02c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:16.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.553+0000 7f02d9f79640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f02c0066470 0x7f02c0068930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f02c8005b70 tx=0x7f02c80091e0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.553+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 msgr2=0x7f02d4153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:16.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.553+0000 7f02d9f79640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 0x7f02d4153b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f02c401cd40 tx=0x7f02c4000f70 comp rx=0 tx=0).stop 2023-10-26T05:48:16.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 shutdown_connections 2023-10-26T05:48:16.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f02d4152610 0x7f02d40c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d40c4ad0 0x7f02d41520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f02c0066470 0x7f02c0068930 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 --2- 172.21.15.71:0/3984742815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02d40c46f0 0x7f02d4153b30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 >> 172.21.15.71:0/3984742815 conn(0x7f02d40d13f0 msgr2=0x7f02d40b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:16.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 shutdown_connections 2023-10-26T05:48:16.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.557+0000 7f02d9f79640 1 -- 172.21.15.71:0/3984742815 wait complete. 2023-10-26T05:48:16.580 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:48:16.580 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:48:16.581 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:48:16.581 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:16.581 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:48:16.581 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:48:16.582 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:48:16.582 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:16.582 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:48:16.582 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:48:16.583 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:48:16.583 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:16.584 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:48:16.584 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:48:16.584 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:48:16.584 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:16.590 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:48:16.591 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:48:16.592 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:48:16.592 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:48:16.596 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:48:16.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:48:16.597 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:48:16.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:48:16.636 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:48:16.637 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:16.639 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:48:16.639 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:16.640 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:48:16.641 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:16.641 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:48:16.642 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:16.643 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:48:16.700 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:48:16.702 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:48:16.710 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:48:16.711 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:48:16.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 -- 172.21.15.71:0/1203660309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 msgr2=0x7f89b015a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:16.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/1203660309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b015a280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f89b006e620 tx=0x7f89a001cd30 comp rx=0 tx=0).stop 2023-10-26T05:48:16.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 -- 172.21.15.71:0/1203660309 shutdown_connections 2023-10-26T05:48:16.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/1203660309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89b00ce250 0x7f89b00ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/1203660309 >> v2:172.21.15.86:3300/0 conn(0x7f89b00c97d0 0x7f89b00cb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/1203660309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b015a280 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 -- 172.21.15.71:0/1203660309 >> 172.21.15.71:0/1203660309 conn(0x7f89b006fd90 msgr2=0x7f89b00b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 -- 172.21.15.71:0/1203660309 shutdown_connections 2023-10-26T05:48:16.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 -- 172.21.15.71:0/1203660309 wait complete. 2023-10-26T05:48:16.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 Processor -- start 2023-10-26T05:48:16.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 -- start start 2023-10-26T05:48:16.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.769+0000 7f89b6ba2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b00c97d0 0x7f89b00c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:16.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89b00ce250 0x7f89b0154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:16.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b00c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:16.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b0155500 con 0x7f89b0159160 2023-10-26T05:48:16.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00c66b0 con 0x7f89b00c97d0 2023-10-26T05:48:16.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b00c2bd0 con 0x7f89b00ce250 2023-10-26T05:48:16.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b00c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:16.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b00c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39238/0 (socket says 172.21.15.71:39238) 2023-10-26T05:48:16.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 -- 172.21.15.71:0/2708488888 learned_addr learned my addr 172.21.15.71:0/2708488888 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:16.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b493e640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b00c97d0 0x7f89b00c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:16.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89affff640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89b00ce250 0x7f89b0154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:16.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 -- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89b00ce250 msgr2=0x7f89b0154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:16.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89b00ce250 0x7f89b0154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 -- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b00c97d0 msgr2=0x7f89b00c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:16.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b00c97d0 0x7f89b00c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:16.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 -- 172.21.15.71:0/2708488888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00098e0 con 0x7f89b0159160 2023-10-26T05:48:16.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b513f640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b00c3620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f89a800d750 tx=0x7f89a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:16.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8009c30 con 0x7f89b0159160 2023-10-26T05:48:16.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f89a8009e80 con 0x7f89b0159160 2023-10-26T05:48:16.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b00c68e0 con 0x7f89b0159160 2023-10-26T05:48:16.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8004280 con 0x7f89b0159160 2023-10-26T05:48:16.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b00c2e20 con 0x7f89b0159160 2023-10-26T05:48:16.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b00c5fc0 con 0x7f89b0159160 2023-10-26T05:48:16.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89a8003600 con 0x7f89b0159160 2023-10-26T05:48:16.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89ad7fa640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f899c066470 0x7f899c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:16.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f89a8083bc0 con 0x7f89b0159160 2023-10-26T05:48:16.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b493e640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f899c066470 0x7f899c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:16.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.773+0000 7f89b493e640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f899c066470 0x7f899c068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89a00af000 tx=0x7f89a001b570 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:16.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.777+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89a8051480 con 0x7f89b0159160 2023-10-26T05:48:16.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:16.937+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f89b006a7b0 con 0x7f89b0159160 2023-10-26T05:48:18.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.085+0000 7f89ad7fa640 1 -- 172.21.15.71:0/2708488888 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f89a8017080 con 0x7f89b0159160 2023-10-26T05:48:18.089 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:48:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f899c066470 msgr2=0x7f899c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f899c066470 0x7f899c068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89a00af000 tx=0x7f89a001b570 comp rx=0 tx=0).stop 2023-10-26T05:48:18.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 msgr2=0x7f89b00c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b00c3620 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f89a800d750 tx=0x7f89a800dc10 comp rx=0 tx=0).stop 2023-10-26T05:48:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 shutdown_connections 2023-10-26T05:48:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f899c066470 0x7f899c068930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89b0159160 0x7f89b00c3620 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f89b00ce250 0x7f89b0154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 --2- 172.21.15.71:0/2708488888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f89b00c97d0 0x7f89b00c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 >> 172.21.15.71:0/2708488888 conn(0x7f89b006fd90 msgr2=0x7f89b00ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 shutdown_connections 2023-10-26T05:48:18.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.089+0000 7f89b6ba2640 1 -- 172.21.15.71:0/2708488888 wait complete. 2023-10-26T05:48:18.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 -- 172.21.15.71:0/713839625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0850 msgr2=0x7ff52c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 --2- 172.21.15.71:0/713839625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0850 0x7ff52c15b3a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009840 tx=0x7ff51c01c700 comp rx=0 tx=0).stop 2023-10-26T05:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 -- 172.21.15.71:0/713839625 shutdown_connections 2023-10-26T05:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 --2- 172.21.15.71:0/713839625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0850 0x7ff52c15b3a0 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009840 tx=0x7ff51c01c700 comp rx=0 tx=0).stop 2023-10-26T05:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 -- 172.21.15.71:0/713839625 >> 172.21.15.71:0/713839625 conn(0x7ff52c0cdd80 msgr2=0x7ff52c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 -- 172.21.15.71:0/713839625 shutdown_connections 2023-10-26T05:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.237+0000 7ff53117b640 1 -- 172.21.15.71:0/713839625 wait complete. 2023-10-26T05:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 Processor -- start 2023-10-26T05:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- start start 2023-10-26T05:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52c0cd880 0x7ff52c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 0x7ff52c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52c15cdd0 0x7ff52c147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c06a650 con 0x7ff52c0d0470 2023-10-26T05:48:18.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0cb4e0 con 0x7ff52c0cd880 2023-10-26T05:48:18.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c9730 con 0x7ff52c15cdd0 2023-10-26T05:48:18.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 0x7ff52c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 0x7ff52c0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39250/0 (socket says 172.21.15.71:39250) 2023-10-26T05:48:18.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 -- 172.21.15.71:0/2717065435 learned_addr learned my addr 172.21.15.71:0/2717065435 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:18.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52ad9d640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52c0cd880 0x7ff52c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52b59e640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52c15cdd0 0x7ff52c147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 -- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52c15cdd0 msgr2=0x7ff52c147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52c15cdd0 0x7ff52c147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 -- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52c0cd880 msgr2=0x7ff52c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52c0cd880 0x7ff52c0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 -- 172.21.15.71:0/2717065435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0094a0 con 0x7ff52c0d0470 2023-10-26T05:48:18.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52ad9d640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52c0cd880 0x7ff52c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:18.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff52a59c640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 0x7ff52c0c91f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff51c01cbe0 tx=0x7ff51c0a5830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:18.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c004340 con 0x7ff52c0d0470 2023-10-26T05:48:18.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff51c003600 con 0x7ff52c0d0470 2023-10-26T05:48:18.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c15d290 con 0x7ff52c0d0470 2023-10-26T05:48:18.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0039b0 con 0x7ff52c0d0470 2023-10-26T05:48:18.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c15d850 con 0x7ff52c0d0470 2023-10-26T05:48:18.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c066b10 con 0x7ff52c0d0470 2023-10-26T05:48:18.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff51c003b10 con 0x7ff52c0d0470 2023-10-26T05:48:18.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.241+0000 7ff5137fe640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff518066470 0x7ff518068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.245+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff51c11e950 con 0x7ff52c0d0470 2023-10-26T05:48:18.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.245+0000 7ff52ad9d640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff518066470 0x7ff518068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.245+0000 7ff52ad9d640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff518066470 0x7ff518068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff520000f70 tx=0x7ff520009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:18.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.249+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff51c0ec1b0 con 0x7ff52c0d0470 2023-10-26T05:48:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.437+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff52c0721e0 con 0x7ff52c0d0470 2023-10-26T05:48:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff5137fe640 1 -- 172.21.15.71:0/2717065435 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff51c0bb090 con 0x7ff52c0d0470 2023-10-26T05:48:18.445 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:18.445 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:18.445 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:48:18.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.446 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:48:18.446 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:48:18.446 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.446 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.446 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:18.447 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:18.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.448 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:48:18.448 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:48:18.448 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.448 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.448 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:18.449 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:18.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.450 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:48:18.450 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:48:18.450 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.450 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.450 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:18.451 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:18.451 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.452 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:48:18.452 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:48:18.452 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.452 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.452 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:18.453 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:18.453 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.454 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:48:18.454 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:48:18.454 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.454 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.455 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.456 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:48:18.456 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:48:18.456 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.456 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.456 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.457 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.457 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.457 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:48:18.458 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:48:18.458 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.458 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.458 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.458 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.459 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:48:18.459 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:48:18.460 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.460 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.460 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.460 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.461 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:48:18.461 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:48:18.461 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.462 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.462 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.462 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.463 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:48:18.463 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:48:18.463 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.464 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.464 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.464 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.464 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.465 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:48:18.465 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:48:18.465 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.466 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.466 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.466 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.467 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:48:18.467 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:48:18.467 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.467 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:18.468 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:18.468 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.469 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:48:18.469 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:48:18.469 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.469 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.470 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:18.470 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:18.470 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:18.470 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.471 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:48:18.471 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:48:18.471 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.471 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.472 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:18.472 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:18.472 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:18.472 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.473 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:48:18.473 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:48:18.473 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.473 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.474 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:18.474 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:18.474 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:18.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.475 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:48:18.475 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:48:18.475 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.475 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:18.476 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:18.476 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:18.476 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:18.476 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.477 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:48:18.477 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:48:18.477 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.477 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:48:18.478 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:18.478 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:18.478 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.479 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-mds", 2023-10-26T05:48:18.479 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBW/Tll0BILFhAAEo4wBiaY+6kfIDs3nV57Xg==", 2023-10-26T05:48:18.479 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.480 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-mds" 2023-10-26T05:48:18.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.481 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-mgr", 2023-10-26T05:48:18.481 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBW/TlleSwLFhAANzdBlexyISLZfkxLiURMCA==", 2023-10-26T05:48:18.481 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.481 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-26T05:48:18.482 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.482 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-osd", 2023-10-26T05:48:18.483 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBW/TllRUILFhAAO3EyRjEWUQTeJsrm8ePJoQ==", 2023-10-26T05:48:18.483 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.483 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-osd" 2023-10-26T05:48:18.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.485 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rbd", 2023-10-26T05:48:18.485 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBW/Tllk1gLFhAAqCAMNH86rMCIvGFAVZvcyg==", 2023-10-26T05:48:18.486 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.486 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-26T05:48:18.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.487 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-26T05:48:18.487 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBW/TllAm8LFhAAFO+jyGu63fQV058aGQ6u2g==", 2023-10-26T05:48:18.487 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.487 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-26T05:48:18.488 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.488 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.bootstrap-rgw", 2023-10-26T05:48:18.489 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBW/TllloULFhAAsI8g9HYNbE704dgRxinEhQ==", 2023-10-26T05:48:18.489 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.489 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-26T05:48:18.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.490 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.490 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:48:18.490 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:48:18.490 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.490 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:48:18.491 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:48:18.491 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.491 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:18.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:18.492 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:48:18.492 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:48:18.493 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:18.493 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:48:18.493 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:48:18.493 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:18.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.494 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:18.494 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:48:18.494 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:18.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff518066470 msgr2=0x7ff518068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff518066470 0x7ff518068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff520000f70 tx=0x7ff520009040 comp rx=0 tx=0).stop 2023-10-26T05:48:18.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 msgr2=0x7ff52c0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 0x7ff52c0c91f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff51c01cbe0 tx=0x7ff51c0a5830 comp rx=0 tx=0).stop 2023-10-26T05:48:18.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 shutdown_connections 2023-10-26T05:48:18.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff52c15cdd0 0x7ff52c147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff518066470 0x7ff518068930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff52c0d0470 0x7ff52c0c91f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 --2- 172.21.15.71:0/2717065435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff52c0cd880 0x7ff52c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 >> 172.21.15.71:0/2717065435 conn(0x7ff52c0cdd80 msgr2=0x7ff52c0b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:18.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 shutdown_connections 2023-10-26T05:48:18.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.441+0000 7ff53117b640 1 -- 172.21.15.71:0/2717065435 wait complete. 2023-10-26T05:48:18.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-26T05:48:18.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3415094771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 msgr2=0x7fae7806a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3415094771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 0x7fae7806a450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fae70009900 tx=0x7fae7001c890 comp rx=0 tx=0).stop 2023-10-26T05:48:18.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3415094771 shutdown_connections 2023-10-26T05:48:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3415094771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 0x7fae7806a450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3415094771 >> v2:172.21.15.86:3300/0 conn(0x7fae780b15f0 0x7fae781586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3415094771 >> 172.21.15.71:0/3415094771 conn(0x7fae7806b8c0 msgr2=0x7fae7806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3415094771 shutdown_connections 2023-10-26T05:48:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3415094771 wait complete. 2023-10-26T05:48:18.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 Processor -- start 2023-10-26T05:48:18.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.625+0000 7fae7e5d1640 1 -- start start 2023-10-26T05:48:18.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 0x7fae7814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 0x7fae7814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae780cc730 0x7fae7814ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae78152dd0 con 0x7fae780b15f0 2023-10-26T05:48:18.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7814e330 con 0x7fae780cc730 2023-10-26T05:48:18.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7814e610 con 0x7fae7806a000 2023-10-26T05:48:18.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 0x7fae7814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 0x7fae7814d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39256/0 (socket says 172.21.15.71:39256) 2023-10-26T05:48:18.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 -- 172.21.15.71:0/3748662338 learned_addr learned my addr 172.21.15.71:0/3748662338 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:18.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae77fff640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 0x7fae7814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7cb6e640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae780cc730 0x7fae7814ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 -- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 msgr2=0x7fae7814c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 0x7fae7814c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 -- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae780cc730 msgr2=0x7fae7814ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae780cc730 0x7fae7814ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 -- 172.21.15.71:0/3748662338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae600094a0 con 0x7fae780b15f0 2023-10-26T05:48:18.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7cb6e640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae780cc730 0x7fae7814ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:18.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae777fe640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 0x7fae7814d9e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae6000e9c0 tx=0x7fae6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:18.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6000cde0 con 0x7fae780b15f0 2023-10-26T05:48:18.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70009560 con 0x7fae780b15f0 2023-10-26T05:48:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae60010520 con 0x7fae780b15f0 2023-10-26T05:48:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60010930 con 0x7fae780b15f0 2023-10-26T05:48:18.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae78153090 con 0x7fae780b15f0 2023-10-26T05:48:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78066b10 con 0x7fae780b15f0 2023-10-26T05:48:18.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fae60010a90 con 0x7fae780b15f0 2023-10-26T05:48:18.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae74ff9640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae58066470 0x7fae58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:18.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae60014070 con 0x7fae780b15f0 2023-10-26T05:48:18.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.629+0000 7fae77fff640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae58066470 0x7fae58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:18.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.633+0000 7fae77fff640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae58066470 0x7fae58068930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae680012f0 tx=0x7fae68009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:18.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.633+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fae600520b0 con 0x7fae780b15f0 2023-10-26T05:48:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.841+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fae7806c7d0 con 0x7fae780b15f0 2023-10-26T05:48:18.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.845+0000 7fae74ff9640 1 -- 172.21.15.71:0/3748662338 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v23) v1 ==== 90+0+0 (secure 0 0 0) 0x7fae600556d0 con 0x7fae780b15f0 2023-10-26T05:48:18.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae58066470 msgr2=0x7fae58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae58066470 0x7fae58068930 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fae680012f0 tx=0x7fae68009040 comp rx=0 tx=0).stop 2023-10-26T05:48:18.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 msgr2=0x7fae7814d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 0x7fae7814d9e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae6000e9c0 tx=0x7fae6000ee80 comp rx=0 tx=0).stop 2023-10-26T05:48:18.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 shutdown_connections 2023-10-26T05:48:18.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae780cc730 0x7fae7814ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae58066470 0x7fae58068930 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae780b15f0 0x7fae7814d9e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 --2- 172.21.15.71:0/3748662338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae7806a000 0x7fae7814c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 >> 172.21.15.71:0/3748662338 conn(0x7fae7806b8c0 msgr2=0x7fae7814f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 shutdown_connections 2023-10-26T05:48:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.849+0000 7fae7e5d1640 1 -- 172.21.15.71:0/3748662338 wait complete. 2023-10-26T05:48:18.868 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-26T05:48:18.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/3558349463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f68159060 msgr2=0x7f3f6806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:18.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 --2- 172.21.15.71:0/3558349463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f68159060 0x7f3f6806f6f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009c80 tx=0x7f3f6001d6e0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/3558349463 shutdown_connections 2023-10-26T05:48:18.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 --2- 172.21.15.71:0/3558349463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f68159060 0x7f3f6806f6f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 --2- 172.21.15.71:0/3558349463 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f6813de80 0x7f3f680cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:18.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/3558349463 >> 172.21.15.71:0/3558349463 conn(0x7f3f6806b8c0 msgr2=0x7f3f6806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:18.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/3558349463 shutdown_connections 2023-10-26T05:48:18.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/3558349463 wait complete. 2023-10-26T05:48:19.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 Processor -- start 2023-10-26T05:48:19.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- start start 2023-10-26T05:48:19.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 0x7f3f6814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f6813de80 0x7f3f68132de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f68159060 0x7f3f68154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f681303d0 con 0x7f3f6813ae90 2023-10-26T05:48:19.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68153d70 con 0x7f3f6813de80 2023-10-26T05:48:19.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68153160 con 0x7f3f68159060 2023-10-26T05:48:19.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 0x7f3f6814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 0x7f3f6814eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39268/0 (socket says 172.21.15.71:39268) 2023-10-26T05:48:19.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 -- 172.21.15.71:0/297552403 learned_addr learned my addr 172.21.15.71:0/297552403 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:19.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6cca4640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f6813de80 0x7f3f68132de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6dca6640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f68159060 0x7f3f68154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 -- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f68159060 msgr2=0x7f3f68154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f68159060 0x7f3f68154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 -- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f6813de80 msgr2=0x7f3f68132de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f6813de80 0x7f3f68132de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 -- 172.21.15.71:0/297552403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600098e0 con 0x7f3f6813ae90 2023-10-26T05:48:19.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6dca6640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f68159060 0x7f3f68154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:19.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6d4a5640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 0x7f3f6814eb10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b3e0 tx=0x7f3f6400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:19.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64011830 con 0x7f3f6813ae90 2023-10-26T05:48:19.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f64011aa0 con 0x7f3f6813ae90 2023-10-26T05:48:19.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68153440 con 0x7f3f6813ae90 2023-10-26T05:48:19.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64010380 con 0x7f3f6813ae90 2023-10-26T05:48:19.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.989+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f68153a00 con 0x7f3f6813ae90 2023-10-26T05:48:19.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.993+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f68066b10 con 0x7f3f6813ae90 2023-10-26T05:48:19.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.993+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f640104e0 con 0x7f3f6813ae90 2023-10-26T05:48:19.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.993+0000 7f3f5dffb640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f4c066470 0x7f3f4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.993+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f64083fa0 con 0x7f3f6813ae90 2023-10-26T05:48:19.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.993+0000 7f3f6cca4640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f4c066470 0x7f3f4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.993+0000 7f3f6cca4640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f4c066470 0x7f3f4c068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006fd0 tx=0x7f3f58008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:19.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:18.997+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f640b5090 con 0x7f3f6813ae90 2023-10-26T05:48:19.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.209+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3f680afb90 con 0x7f3f6813ae90 2023-10-26T05:48:19.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f5dffb640 1 -- 172.21.15.71:0/297552403 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v24) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3f640150a0 con 0x7f3f6813ae90 2023-10-26T05:48:19.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f4c066470 msgr2=0x7f3f4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f4c066470 0x7f3f4c068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006fd0 tx=0x7f3f58008040 comp rx=0 tx=0).stop 2023-10-26T05:48:19.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 msgr2=0x7f3f6814eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 0x7f3f6814eb10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b3e0 tx=0x7f3f6400b8a0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 shutdown_connections 2023-10-26T05:48:19.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f68159060 0x7f3f68154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f6813de80 0x7f3f68132de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f4c066470 0x7f3f4c068930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 --2- 172.21.15.71:0/297552403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f6813ae90 0x7f3f6814eb10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 >> 172.21.15.71:0/297552403 conn(0x7f3f6806b8c0 msgr2=0x7f3f68156230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:19.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 shutdown_connections 2023-10-26T05:48:19.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.213+0000 7f3f6f709640 1 -- 172.21.15.71:0/297552403 wait complete. 2023-10-26T05:48:19.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-26T05:48:19.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 -- 172.21.15.71:0/4246666172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde815b3b0 msgr2=0x7fbde80b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 --2- 172.21.15.71:0/4246666172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde815b3b0 0x7fbde80b43e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbde808a600 tx=0x7fbde001f560 comp rx=0 tx=0).stop 2023-10-26T05:48:19.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 -- 172.21.15.71:0/4246666172 shutdown_connections 2023-10-26T05:48:19.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 --2- 172.21.15.71:0/4246666172 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde806a390 0x7fbde806a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 --2- 172.21.15.71:0/4246666172 >> v2:172.21.15.86:3300/0 conn(0x7fbde80cc7c0 0x7fbde80ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 --2- 172.21.15.71:0/4246666172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde815b3b0 0x7fbde80b43e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 -- 172.21.15.71:0/4246666172 >> 172.21.15.71:0/4246666172 conn(0x7fbde80d12d0 msgr2=0x7fbde80d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:19.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 -- 172.21.15.71:0/4246666172 shutdown_connections 2023-10-26T05:48:19.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 -- 172.21.15.71:0/4246666172 wait complete. 2023-10-26T05:48:19.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 Processor -- start 2023-10-26T05:48:19.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.349+0000 7fbded5b1640 1 -- start start 2023-10-26T05:48:19.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 0x7fbde80c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde80cc7c0 0x7fbde80c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815b3b0 0x7fbde80aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80c99b0 con 0x7fbde806a390 2023-10-26T05:48:19.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80b0030 con 0x7fbde815b3b0 2023-10-26T05:48:19.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80c8da0 con 0x7fbde80cc7c0 2023-10-26T05:48:19.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 0x7fbde80c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 0x7fbde80c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39290/0 (socket says 172.21.15.71:39290) 2023-10-26T05:48:19.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 -- 172.21.15.71:0/3019391982 learned_addr learned my addr 172.21.15.71:0/3019391982 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:19.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde77fe640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815b3b0 0x7fbde80aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde67fc640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde80cc7c0 0x7fbde80c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 -- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde80cc7c0 msgr2=0x7fbde80c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde80cc7c0 0x7fbde80c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 -- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815b3b0 msgr2=0x7fbde80aa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815b3b0 0x7fbde80aa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 -- 172.21.15.71:0/3019391982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00098e0 con 0x7fbde806a390 2023-10-26T05:48:19.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde77fe640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815b3b0 0x7fbde80aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:19.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde6ffd640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 0x7fbde80c87c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009bd0 tx=0x7fbde00b1550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:19.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00b1e10 con 0x7fbde806a390 2023-10-26T05:48:19.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbde00aa070 con 0x7fbde806a390 2023-10-26T05:48:19.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde80c9080 con 0x7fbde806a390 2023-10-26T05:48:19.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde80c9640 con 0x7fbde806a390 2023-10-26T05:48:19.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0003cf0 con 0x7fbde806a390 2023-10-26T05:48:19.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc005260 con 0x7fbde806a390 2023-10-26T05:48:19.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbde000a040 con 0x7fbde806a390 2023-10-26T05:48:19.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbddf7fe640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbdc8066470 0x7fbdc8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbde0121b20 con 0x7fbde806a390 2023-10-26T05:48:19.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.353+0000 7fbde67fc640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbdc8066470 0x7fbdc8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.357+0000 7fbde67fc640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbdc8066470 0x7fbdc8068930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbde815fc00 tx=0x7fbdd0005c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:19.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.361+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbde00ef2f0 con 0x7fbde806a390 2023-10-26T05:48:19.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.553+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fbdcc005a90 con 0x7fbde806a390 2023-10-26T05:48:19.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbddf7fe640 1 -- 172.21.15.71:0/3019391982 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v25) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbde00c10a0 con 0x7fbde806a390 2023-10-26T05:48:19.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbdc8066470 msgr2=0x7fbdc8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbdc8066470 0x7fbdc8068930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbde815fc00 tx=0x7fbdd0005c60 comp rx=0 tx=0).stop 2023-10-26T05:48:19.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 msgr2=0x7fbde80c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 0x7fbde80c87c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009bd0 tx=0x7fbde00b1550 comp rx=0 tx=0).stop 2023-10-26T05:48:19.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 shutdown_connections 2023-10-26T05:48:19.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde815b3b0 0x7fbde80aa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbde80cc7c0 0x7fbde80c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbdc8066470 0x7fbdc8068930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 --2- 172.21.15.71:0/3019391982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde806a390 0x7fbde80c87c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 >> 172.21.15.71:0/3019391982 conn(0x7fbde80d12d0 msgr2=0x7fbde80b5290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:19.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 shutdown_connections 2023-10-26T05:48:19.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.557+0000 7fbded5b1640 1 -- 172.21.15.71:0/3019391982 wait complete. 2023-10-26T05:48:19.579 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-26T05:48:19.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- 172.21.15.71:0/3699451811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda006a390 msgr2=0x7feda006a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 --2- 172.21.15.71:0/3699451811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda006a390 0x7feda006a7b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fed94009c80 tx=0x7fed9401f540 comp rx=0 tx=0).stop 2023-10-26T05:48:19.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- 172.21.15.71:0/3699451811 shutdown_connections 2023-10-26T05:48:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 --2- 172.21.15.71:0/3699451811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda006a390 0x7feda006a7b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 --2- 172.21.15.71:0/3699451811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feda015b3b0 0x7feda00b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- 172.21.15.71:0/3699451811 >> 172.21.15.71:0/3699451811 conn(0x7feda00d12d0 msgr2=0x7feda00d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:19.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- 172.21.15.71:0/3699451811 shutdown_connections 2023-10-26T05:48:19.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- 172.21.15.71:0/3699451811 wait complete. 2023-10-26T05:48:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 Processor -- start 2023-10-26T05:48:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- start start 2023-10-26T05:48:19.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda006a390 0x7feda00cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feda00cc7c0 0x7feda00b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 0x7feda0154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00c83a0 con 0x7feda015b3b0 2023-10-26T05:48:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00c8680 con 0x7feda006a390 2023-10-26T05:48:19.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda7de6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda00c8960 con 0x7feda00cc7c0 2023-10-26T05:48:19.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 0x7feda0154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 0x7feda0154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39310/0 (socket says 172.21.15.71:39310) 2023-10-26T05:48:19.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 -- 172.21.15.71:0/51063901 learned_addr learned my addr 172.21.15.71:0/51063901 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:19.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda5381640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feda00cc7c0 0x7feda00b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda5b82640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda006a390 0x7feda00cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 -- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feda00cc7c0 msgr2=0x7feda00b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feda00cc7c0 0x7feda00b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 -- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda006a390 msgr2=0x7feda00cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda006a390 0x7feda00cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda6383640 1 -- 172.21.15.71:0/51063901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940098e0 con 0x7feda015b3b0 2023-10-26T05:48:19.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.705+0000 7feda5b82640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda006a390 0x7feda00cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7feda6383640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 0x7feda0154390 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c3c0 tx=0x7fed9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90011790 con 0x7feda015b3b0 2023-10-26T05:48:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed900119e0 con 0x7feda015b3b0 2023-10-26T05:48:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90011df0 con 0x7feda015b3b0 2023-10-26T05:48:19.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda01f1e80 con 0x7feda015b3b0 2023-10-26T05:48:19.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda00cb940 con 0x7feda015b3b0 2023-10-26T05:48:19.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda0066b10 con 0x7feda015b3b0 2023-10-26T05:48:19.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed9001c460 con 0x7feda015b3b0 2023-10-26T05:48:19.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7fed9e7fc640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed8c0664a0 0x7fed8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:19.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed90083b40 con 0x7feda015b3b0 2023-10-26T05:48:19.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7feda5b82640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed8c0664a0 0x7fed8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:19.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.709+0000 7feda5b82640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed8c0664a0 0x7fed8c068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed98005fd0 tx=0x7fed98005dc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:19.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.713+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed900513a0 con 0x7feda015b3b0 2023-10-26T05:48:19.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.921+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7feda0069e50 con 0x7feda015b3b0 2023-10-26T05:48:19.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.925+0000 7fed9e7fc640 1 -- 172.21.15.71:0/51063901 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v26) v1 ==== 90+0+0 (secure 0 0 0) 0x7fed90003810 con 0x7feda015b3b0 2023-10-26T05:48:19.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed8c0664a0 msgr2=0x7fed8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed8c0664a0 0x7fed8c068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fed98005fd0 tx=0x7fed98005dc0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 msgr2=0x7feda0154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:19.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 0x7feda0154390 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c3c0 tx=0x7fed9000c880 comp rx=0 tx=0).stop 2023-10-26T05:48:19.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 shutdown_connections 2023-10-26T05:48:19.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed8c0664a0 0x7fed8c068960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feda015b3b0 0x7feda0154390 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feda00cc7c0 0x7feda00b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 --2- 172.21.15.71:0/51063901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feda006a390 0x7feda00cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 >> 172.21.15.71:0/51063901 conn(0x7feda00d12d0 msgr2=0x7feda00cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 shutdown_connections 2023-10-26T05:48:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:19.929+0000 7feda7de6640 1 -- 172.21.15.71:0/51063901 wait complete. 2023-10-26T05:48:19.948 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-26T05:48:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61f127640 1 -- 172.21.15.71:0/2791056540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5fc0039b0 msgr2=0x7ff5fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61f127640 1 --2- 172.21.15.71:0/2791056540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5fc0039b0 0x7ff5fc003dd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff60800c260 tx=0x7ff60801cf70 comp rx=0 tx=0).stop 2023-10-26T05:48:20.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61d6c4640 1 -- 172.21.15.71:0/2791056540 reap_dead start 2023-10-26T05:48:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61f127640 1 -- 172.21.15.71:0/2791056540 shutdown_connections 2023-10-26T05:48:20.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61f127640 1 -- 172.21.15.71:0/2791056540 >> 172.21.15.71:0/2791056540 conn(0x7ff61806b8c0 msgr2=0x7ff61806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:20.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61f127640 1 -- 172.21.15.71:0/2791056540 shutdown_connections 2023-10-26T05:48:20.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.061+0000 7ff61f127640 1 -- 172.21.15.71:0/2791056540 wait complete. 2023-10-26T05:48:20.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 Processor -- start 2023-10-26T05:48:20.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- start start 2023-10-26T05:48:20.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5fc0039b0 0x7ff618152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181564c0 0x7ff618156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 0x7ff618153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff618156e80 con 0x7ff618152cb0 2023-10-26T05:48:20.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61812ef60 con 0x7ff6181564c0 2023-10-26T05:48:20.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618153c30 con 0x7ff5fc0039b0 2023-10-26T05:48:20.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 0x7ff618153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 0x7ff618153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39328/0 (socket says 172.21.15.71:39328) 2023-10-26T05:48:20.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 -- 172.21.15.71:0/1894117921 learned_addr learned my addr 172.21.15.71:0/1894117921 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:20.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61cec3640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5fc0039b0 0x7ff618152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff617fff640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181564c0 0x7ff618156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 -- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5fc0039b0 msgr2=0x7ff618152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5fc0039b0 0x7ff618152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 -- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181564c0 msgr2=0x7ff618156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181564c0 0x7ff618156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 -- 172.21.15.71:0/1894117921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618153950 con 0x7ff618152cb0 2023-10-26T05:48:20.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff617fff640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181564c0 0x7ff618156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:20.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61d6c4640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 0x7ff618153130 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff61806b110 tx=0x7ff61000fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:20.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610004d30 con 0x7ff618152cb0 2023-10-26T05:48:20.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff61000dad0 con 0x7ff618152cb0 2023-10-26T05:48:20.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff618153ff0 con 0x7ff618152cb0 2023-10-26T05:48:20.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61000dee0 con 0x7ff618152cb0 2023-10-26T05:48:20.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6181545b0 con 0x7ff618152cb0 2023-10-26T05:48:20.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff618066b10 con 0x7ff618152cb0 2023-10-26T05:48:20.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.065+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff610011440 con 0x7ff618152cb0 2023-10-26T05:48:20.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.069+0000 7ff6157fa640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6000664a0 0x7ff600068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.069+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff610015070 con 0x7ff618152cb0 2023-10-26T05:48:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.069+0000 7ff61cec3640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6000664a0 0x7ff600068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.069+0000 7ff61cec3640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6000664a0 0x7ff600068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6080ac000 tx=0x7ff6080a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.069+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff610051c60 con 0x7ff618152cb0 2023-10-26T05:48:20.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.269+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ff618153e50 con 0x7ff618152cb0 2023-10-26T05:48:20.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff6157fa640 1 -- 172.21.15.71:0/1894117921 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff618153e50 con 0x7ff618152cb0 2023-10-26T05:48:20.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6000664a0 msgr2=0x7ff600068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6000664a0 0x7ff600068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff6080ac000 tx=0x7ff6080a7040 comp rx=0 tx=0).stop 2023-10-26T05:48:20.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 msgr2=0x7ff618153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 0x7ff618153130 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff61806b110 tx=0x7ff61000fe10 comp rx=0 tx=0).stop 2023-10-26T05:48:20.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 shutdown_connections 2023-10-26T05:48:20.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff6000664a0 0x7ff600068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff618152cb0 0x7ff618153130 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6181564c0 0x7ff618156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 --2- 172.21.15.71:0/1894117921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5fc0039b0 0x7ff618152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 >> 172.21.15.71:0/1894117921 conn(0x7ff61806b8c0 msgr2=0x7ff6180ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:20.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 shutdown_connections 2023-10-26T05:48:20.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.273+0000 7ff61f127640 1 -- 172.21.15.71:0/1894117921 wait complete. 2023-10-26T05:48:20.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-26T05:48:20.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 -- 172.21.15.71:0/3288760362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0480069d30 msgr2=0x7f048006a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 --2- 172.21.15.71:0/3288760362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0480069d30 0x7f048006a1c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0478009c80 tx=0x7f047801ccf0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 -- 172.21.15.71:0/3288760362 shutdown_connections 2023-10-26T05:48:20.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 --2- 172.21.15.71:0/3288760362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0480069d30 0x7f048006a1c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 --2- 172.21.15.71:0/3288760362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04800ca7f0 0x7f0480159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 -- 172.21.15.71:0/3288760362 >> 172.21.15.71:0/3288760362 conn(0x7f048006b8c0 msgr2=0x7f048006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:20.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 -- 172.21.15.71:0/3288760362 shutdown_connections 2023-10-26T05:48:20.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.413+0000 7f048599f640 1 -- 172.21.15.71:0/3288760362 wait complete. 2023-10-26T05:48:20.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 Processor -- start 2023-10-26T05:48:20.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- start start 2023-10-26T05:48:20.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 0x7f048013af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04800ca7f0 0x7f048013e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04800cabd0 0x7f048013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0480156bd0 con 0x7f0480069d30 2023-10-26T05:48:20.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048013b480 con 0x7f04800cabd0 2023-10-26T05:48:20.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0480155f50 con 0x7f04800ca7f0 2023-10-26T05:48:20.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 0x7f048013af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 0x7f048013af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39334/0 (socket says 172.21.15.71:39334) 2023-10-26T05:48:20.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 -- 172.21.15.71:0/527665125 learned_addr learned my addr 172.21.15.71:0/527665125 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:20.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047f7fe640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04800cabd0 0x7f048013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047e7fc640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04800ca7f0 0x7f048013e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 -- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04800ca7f0 msgr2=0x7f048013e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04800ca7f0 0x7f048013e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 -- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04800cabd0 msgr2=0x7f048013e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04800cabd0 0x7f048013e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 -- 172.21.15.71:0/527665125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04700094a0 con 0x7f0480069d30 2023-10-26T05:48:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047f7fe640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04800cabd0 0x7f048013e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f047effd640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 0x7f048013af40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f047000ea10 tx=0x7f047000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047000cde0 con 0x7f0480069d30 2023-10-26T05:48:20.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0470004020 con 0x7f0480069d30 2023-10-26T05:48:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04780098e0 con 0x7f0480069d30 2023-10-26T05:48:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470004430 con 0x7f0480069d30 2023-10-26T05:48:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0480156510 con 0x7f0480069d30 2023-10-26T05:48:20.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0480066b10 con 0x7f0480069d30 2023-10-26T05:48:20.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.417+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0470003600 con 0x7f0480069d30 2023-10-26T05:48:20.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.421+0000 7f0467fff640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f046c066470 0x7f046c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.421+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0470014070 con 0x7f0480069d30 2023-10-26T05:48:20.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.421+0000 7f047e7fc640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f046c066470 0x7f046c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.421+0000 7f047e7fc640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f046c066470 0x7f046c068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0474007fd0 tx=0x7f0474009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:20.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.425+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0470051e80 con 0x7f0480069d30 2023-10-26T05:48:20.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.637+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f048006c7d0 con 0x7f0480069d30 2023-10-26T05:48:20.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f0467fff640 1 -- 172.21.15.71:0/527665125 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v28) v1 ==== 90+0+0 (secure 0 0 0) 0x7f04700554a0 con 0x7f0480069d30 2023-10-26T05:48:20.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f046c066470 msgr2=0x7f046c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f046c066470 0x7f046c068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0474007fd0 tx=0x7f0474009040 comp rx=0 tx=0).stop 2023-10-26T05:48:20.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 msgr2=0x7f048013af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 0x7f048013af40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f047000ea10 tx=0x7f047000eed0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 shutdown_connections 2023-10-26T05:48:20.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04800cabd0 0x7f048013e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04800ca7f0 0x7f048013e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f046c066470 0x7f046c068930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 --2- 172.21.15.71:0/527665125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0480069d30 0x7f048013af40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 >> 172.21.15.71:0/527665125 conn(0x7f048006b8c0 msgr2=0x7f048013efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:20.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 shutdown_connections 2023-10-26T05:48:20.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.641+0000 7f048599f640 1 -- 172.21.15.71:0/527665125 wait complete. 2023-10-26T05:48:20.664 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:20.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 -- 172.21.15.71:0/824659563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33200047d0 msgr2=0x7f3320004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 --2- 172.21.15.71:0/824659563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33200047d0 0x7f3320004c50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f331c0016b0 tx=0x7f331c01c780 comp rx=0 tx=0).stop 2023-10-26T05:48:20.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 -- 172.21.15.71:0/824659563 shutdown_connections 2023-10-26T05:48:20.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 --2- 172.21.15.71:0/824659563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33200047d0 0x7f3320004c50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 -- 172.21.15.71:0/824659563 >> 172.21.15.71:0/824659563 conn(0x7f33340ce270 msgr2=0x7f33340ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:20.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 -- 172.21.15.71:0/824659563 shutdown_connections 2023-10-26T05:48:20.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.777+0000 7f3339a4d640 1 -- 172.21.15.71:0/824659563 wait complete. 2023-10-26T05:48:20.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 Processor -- start 2023-10-26T05:48:20.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- start start 2023-10-26T05:48:20.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33200047d0 0x7f333413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 0x7f3334140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f333413dd40 0x7f3334140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333413e9c0 con 0x7f333413d8c0 2023-10-26T05:48:20.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b2140 con 0x7f33200047d0 2023-10-26T05:48:20.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333413e300 con 0x7f333413dd40 2023-10-26T05:48:20.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 0x7f3334140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 0x7f3334140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39350/0 (socket says 172.21.15.71:39350) 2023-10-26T05:48:20.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 -- 172.21.15.71:0/2150452336 learned_addr learned my addr 172.21.15.71:0/2150452336 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:20.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3332ffd640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33200047d0 0x7f333413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f33337fe640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f333413dd40 0x7f3334140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 -- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f333413dd40 msgr2=0x7f3334140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f333413dd40 0x7f3334140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 -- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33200047d0 msgr2=0x7f333413ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33200047d0 0x7f333413ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 -- 172.21.15.71:0/2150452336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340cc450 con 0x7f333413d8c0 2023-10-26T05:48:20.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f33337fe640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f333413dd40 0x7f3334140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:20.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f332a7fc640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 0x7f3334140730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ab040 tx=0x7f331c003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:20.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0a7660 con 0x7f333413d8c0 2023-10-26T05:48:20.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f331c0a78b0 con 0x7f333413d8c0 2023-10-26T05:48:20.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333413e5e0 con 0x7f333413d8c0 2023-10-26T05:48:20.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0a7ca0 con 0x7f333413d8c0 2023-10-26T05:48:20.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33340b13f0 con 0x7f333413d8c0 2023-10-26T05:48:20.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.781+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333413e160 con 0x7f333413d8c0 2023-10-26T05:48:20.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.785+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f331c0af030 con 0x7f333413d8c0 2023-10-26T05:48:20.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.785+0000 7f3330ff9640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3314066470 0x7f3314068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:20.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.785+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f331c11cdf0 con 0x7f333413d8c0 2023-10-26T05:48:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.785+0000 7f3332ffd640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3314066470 0x7f3314068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.785+0000 7f3332ffd640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3314066470 0x7f3314068930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3324000c10 tx=0x7f3324000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.789+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f333413e160 con 0x7f333413d8c0 2023-10-26T05:48:20.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.933+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f333413eb00 con 0x7f333413d8c0 2023-10-26T05:48:20.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3330ff9640 1 -- 172.21.15.71:0/2150452336 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f333413eb00 con 0x7f333413d8c0 2023-10-26T05:48:20.940 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:20.941 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:18.077565+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:20.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3314066470 msgr2=0x7f3314068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3314066470 0x7f3314068930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3324000c10 tx=0x7f3324000ee0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 msgr2=0x7f3334140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:20.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 0x7f3334140730 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f331c0ab040 tx=0x7f331c003c40 comp rx=0 tx=0).stop 2023-10-26T05:48:20.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 shutdown_connections 2023-10-26T05:48:20.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f333413dd40 0x7f3334140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3314066470 0x7f3314068930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f333413d8c0 0x7f3334140730 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 --2- 172.21.15.71:0/2150452336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33200047d0 0x7f333413ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:20.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 >> 172.21.15.71:0/2150452336 conn(0x7f33340ce270 msgr2=0x7f333406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:20.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 shutdown_connections 2023-10-26T05:48:20.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:20.937+0000 7f3339a4d640 1 -- 172.21.15.71:0/2150452336 wait complete. 2023-10-26T05:48:20.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:20.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:21.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/2537103698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900cd880 msgr2=0x7f0a9015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- 172.21.15.71:0/2537103698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900cd880 0x7f0a9015b8e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0a9008a140 tx=0x7f0a8801cc40 comp rx=0 tx=0).stop 2023-10-26T05:48:21.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/2537103698 shutdown_connections 2023-10-26T05:48:21.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- 172.21.15.71:0/2537103698 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a900d0850 0x7f0a9015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- 172.21.15.71:0/2537103698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900cd880 0x7f0a9015b8e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- 172.21.15.71:0/2537103698 >> v2:172.21.15.86:3300/0 conn(0x7f0a900d0470 0x7f0a9006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/2537103698 >> 172.21.15.71:0/2537103698 conn(0x7f0a900cdd80 msgr2=0x7f0a900ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:21.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/2537103698 shutdown_connections 2023-10-26T05:48:21.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/2537103698 wait complete. 2023-10-26T05:48:21.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 Processor -- start 2023-10-26T05:48:21.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- start start 2023-10-26T05:48:21.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a900cd880 0x7f0a901506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 0x7f0a9015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a900d0850 0x7f0a90153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9015a800 con 0x7f0a900d0470 2023-10-26T05:48:21.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a90151140 con 0x7f0a900cd880 2023-10-26T05:48:21.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a90150c30 con 0x7f0a900d0850 2023-10-26T05:48:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 0x7f0a9015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 0x7f0a9015e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39378/0 (socket says 172.21.15.71:39378) 2023-10-26T05:48:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 -- 172.21.15.71:0/1794051750 learned_addr learned my addr 172.21.15.71:0/1794051750 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a95e0c640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a900d0850 0x7f0a90153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9560b640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a900cd880 0x7f0a901506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 -- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a900d0850 msgr2=0x7f0a90153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a900d0850 0x7f0a90153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 -- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a900cd880 msgr2=0x7f0a901506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a900cd880 0x7f0a901506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 -- 172.21.15.71:0/1794051750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880098e0 con 0x7f0a900d0470 2023-10-26T05:48:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9560b640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a900cd880 0x7f0a901506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a94e0a640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 0x7f0a9015e290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000c940 tx=0x7f0a8000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004220 con 0x7f0a900d0470 2023-10-26T05:48:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0a80004470 con 0x7f0a900d0470 2023-10-26T05:48:21.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80007bb0 con 0x7f0a900d0470 2023-10-26T05:48:21.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9015e7d0 con 0x7f0a900d0470 2023-10-26T05:48:21.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.081+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9015a520 con 0x7f0a900d0470 2023-10-26T05:48:21.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.085+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a90066b10 con 0x7f0a900d0470 2023-10-26T05:48:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.085+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0a80007d10 con 0x7f0a900d0470 2023-10-26T05:48:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.085+0000 7f0a85ffb640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0a74066470 0x7f0a74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.085+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a80084040 con 0x7f0a900d0470 2023-10-26T05:48:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.085+0000 7f0a9560b640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0a74066470 0x7f0a74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.085+0000 7f0a9560b640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0a74066470 0x7f0a74068930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005950 tx=0x7f0a8c005ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.089+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0a80051810 con 0x7f0a900d0470 2023-10-26T05:48:21.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.229+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a900721e0 con 0x7f0a900d0470 2023-10-26T05:48:21.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.229+0000 7f0a85ffb640 1 -- 172.21.15.71:0/1794051750 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a8001d020 con 0x7f0a900d0470 2023-10-26T05:48:21.235 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:21.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0a74066470 msgr2=0x7f0a74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0a74066470 0x7f0a74068930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005950 tx=0x7f0a8c005ca0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 msgr2=0x7f0a9015e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 0x7f0a9015e290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000c940 tx=0x7f0a8000ce00 comp rx=0 tx=0).stop 2023-10-26T05:48:21.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 shutdown_connections 2023-10-26T05:48:21.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a900d0850 0x7f0a90153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0a74066470 0x7f0a74068930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a900d0470 0x7f0a9015e290 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 --2- 172.21.15.71:0/1794051750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a900cd880 0x7f0a901506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 >> 172.21.15.71:0/1794051750 conn(0x7f0a900cdd80 msgr2=0x7f0a900b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:21.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 shutdown_connections 2023-10-26T05:48:21.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.233+0000 7f0a9786f640 1 -- 172.21.15.71:0/1794051750 wait complete. 2023-10-26T05:48:21.252 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:48:21.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 -- 172.21.15.71:0/2798485823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27600b2890 msgr2=0x7f27600caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 --2- 172.21.15.71:0/2798485823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27600b2890 0x7f27600caad0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2748009900 tx=0x7f274801c8c0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 -- 172.21.15.71:0/2798485823 shutdown_connections 2023-10-26T05:48:21.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 --2- 172.21.15.71:0/2798485823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27600b2890 0x7f27600caad0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 --2- 172.21.15.71:0/2798485823 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f276013c2f0 0x7f27600b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 -- 172.21.15.71:0/2798485823 >> 172.21.15.71:0/2798485823 conn(0x7f2760069770 msgr2=0x7f2760069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:21.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.365+0000 7f2767064640 1 -- 172.21.15.71:0/2798485823 shutdown_connections 2023-10-26T05:48:21.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- 172.21.15.71:0/2798485823 wait complete. 2023-10-26T05:48:21.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 Processor -- start 2023-10-26T05:48:21.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- start start 2023-10-26T05:48:21.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27600b2890 0x7f2760135760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27600cb010 0x7f2760151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 0x7f276013d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760152a10 con 0x7f276013c2f0 2023-10-26T05:48:21.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276013cce0 con 0x7f27600cb010 2023-10-26T05:48:21.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2760151e00 con 0x7f27600b2890 2023-10-26T05:48:21.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 0x7f276013d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 0x7f276013d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39392/0 (socket says 172.21.15.71:39392) 2023-10-26T05:48:21.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 -- 172.21.15.71:0/1314405804 learned_addr learned my addr 172.21.15.71:0/1314405804 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:21.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f275ffff640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27600cb010 0x7f2760151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 -- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27600b2890 msgr2=0x7f2760135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2764e00640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27600b2890 0x7f2760135760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27600b2890 0x7f2760135760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 -- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27600cb010 msgr2=0x7f2760151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27600cb010 0x7f2760151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 -- 172.21.15.71:0/1314405804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748009560 con 0x7f276013c2f0 2023-10-26T05:48:21.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f275ffff640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27600cb010 0x7f2760151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:21.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2765601640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 0x7f276013d1c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f275800c780 tx=0x7f275800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:21.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758003db0 con 0x7f276013c2f0 2023-10-26T05:48:21.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f275800ce90 con 0x7f276013c2f0 2023-10-26T05:48:21.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27580052d0 con 0x7f276013c2f0 2023-10-26T05:48:21.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27601520e0 con 0x7f276013c2f0 2023-10-26T05:48:21.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27601526a0 con 0x7f276013c2f0 2023-10-26T05:48:21.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.369+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2760066b10 con 0x7f276013c2f0 2023-10-26T05:48:21.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.373+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2758005430 con 0x7f276013c2f0 2023-10-26T05:48:21.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.373+0000 7f275d7fa640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2740066470 0x7f2740068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.373+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2758083ae0 con 0x7f276013c2f0 2023-10-26T05:48:21.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.373+0000 7f2764e00640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2740066470 0x7f2740068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.373+0000 7f2764e00640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2740066470 0x7f2740068930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f27500058d0 tx=0x7f2750005bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:21.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.377+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2758051340 con 0x7f276013c2f0 2023-10-26T05:48:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.517+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2760070ca0 con 0x7f276013c2f0 2023-10-26T05:48:21.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.517+0000 7f275d7fa640 1 -- 172.21.15.71:0/1314405804 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2758054960 con 0x7f276013c2f0 2023-10-26T05:48:21.523 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:21.523 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:48:21.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:21.523 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:48:21.524 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:48:21.524 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:21.524 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:48:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2740066470 msgr2=0x7f2740068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2740066470 0x7f2740068930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f27500058d0 tx=0x7f2750005bd0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 msgr2=0x7f276013d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 0x7f276013d1c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f275800c780 tx=0x7f275800cc40 comp rx=0 tx=0).stop 2023-10-26T05:48:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 shutdown_connections 2023-10-26T05:48:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2740066470 0x7f2740068930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f276013c2f0 0x7f276013d1c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27600cb010 0x7f2760151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 --2- 172.21.15.71:0/1314405804 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27600b2890 0x7f2760135760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 >> 172.21.15.71:0/1314405804 conn(0x7f2760069770 msgr2=0x7f27600ad570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 shutdown_connections 2023-10-26T05:48:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.521+0000 7f2767064640 1 -- 172.21.15.71:0/1314405804 wait complete. 2023-10-26T05:48:21.540 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:48:21.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:48:21.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 -- 172.21.15.71:0/2458385113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 msgr2=0x7f59200cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 --2- 172.21.15.71:0/2458385113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200cb850 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5910009900 tx=0x7f591001d220 comp rx=0 tx=0).stop 2023-10-26T05:48:21.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 -- 172.21.15.71:0/2458385113 shutdown_connections 2023-10-26T05:48:21.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 --2- 172.21.15.71:0/2458385113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200cb850 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 --2- 172.21.15.71:0/2458385113 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 0x7f59200cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 -- 172.21.15.71:0/2458385113 >> 172.21.15.71:0/2458385113 conn(0x7f592006fd90 msgr2=0x7f59200b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:21.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 -- 172.21.15.71:0/2458385113 shutdown_connections 2023-10-26T05:48:21.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 -- 172.21.15.71:0/2458385113 wait complete. 2023-10-26T05:48:21.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 Processor -- start 2023-10-26T05:48:21.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.657+0000 7f5925ec7640 1 -- start start 2023-10-26T05:48:21.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 0x7f592014eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920159230 0x7f5920151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59200c62c0 con 0x7f592006edf0 2023-10-26T05:48:21.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592014f060 con 0x7f5920159230 2023-10-26T05:48:21.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59200b31c0 con 0x7f59200cd3a0 2023-10-26T05:48:21.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39412/0 (socket says 172.21.15.71:39412) 2023-10-26T05:48:21.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 -- 172.21.15.71:0/3981877579 learned_addr learned my addr 172.21.15.71:0/3981877579 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:21.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591ffff640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920159230 0x7f5920151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591effd640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 0x7f592014eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 -- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 msgr2=0x7f592014eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 0x7f592014eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 -- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920159230 msgr2=0x7f5920151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:21.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920159230 0x7f5920151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:21.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 -- 172.21.15.71:0/3981877579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910009560 con 0x7f592006edf0 2023-10-26T05:48:21.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591effd640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 0x7f592014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:21.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591f7fe640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200b2be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f591800b410 tx=0x7f591800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:21.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918004020 con 0x7f592006edf0 2023-10-26T05:48:21.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5918004270 con 0x7f592006edf0 2023-10-26T05:48:21.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59200b34a0 con 0x7f592006edf0 2023-10-26T05:48:21.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918011890 con 0x7f592006edf0 2023-10-26T05:48:21.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59200b3a60 con 0x7f592006edf0 2023-10-26T05:48:21.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5920066d00 con 0x7f592006edf0 2023-10-26T05:48:21.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5918003600 con 0x7f592006edf0 2023-10-26T05:48:21.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5917fff640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5900066470 0x7f5900068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:21.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5918083ac0 con 0x7f592006edf0 2023-10-26T05:48:21.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591effd640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5900066470 0x7f5900068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:21.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.661+0000 7f591effd640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5900066470 0x7f5900068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59201528f0 tx=0x7f59080093c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:21.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.665+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5918051320 con 0x7f592006edf0 2023-10-26T05:48:21.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:21.809+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f592006ebd0 con 0x7f592006edf0 2023-10-26T05:48:22.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.657+0000 7f5917fff640 1 -- 172.21.15.71:0/3981877579 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5918054940 con 0x7f592006edf0 2023-10-26T05:48:22.663 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:48:22.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5900066470 msgr2=0x7f5900068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:22.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5900066470 0x7f5900068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59201528f0 tx=0x7f59080093c0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 msgr2=0x7f59200b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:22.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200b2be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f591800b410 tx=0x7f591800b8d0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 shutdown_connections 2023-10-26T05:48:22.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5920159230 0x7f5920151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59200cd3a0 0x7f592014eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5900066470 0x7f5900068930 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 --2- 172.21.15.71:0/3981877579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f592006edf0 0x7f59200b2be0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 >> 172.21.15.71:0/3981877579 conn(0x7f592006fd90 msgr2=0x7f59200ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:22.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 shutdown_connections 2023-10-26T05:48:22.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.661+0000 7f5925ec7640 1 -- 172.21.15.71:0/3981877579 wait complete. 2023-10-26T05:48:22.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:48:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/379984299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b1230 msgr2=0x7fcb340b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 --2- 172.21.15.71:0/379984299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b1230 0x7fcb340b1c60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcb34069920 tx=0x7fcb1c01f050 comp rx=0 tx=0).stop 2023-10-26T05:48:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/379984299 shutdown_connections 2023-10-26T05:48:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 --2- 172.21.15.71:0/379984299 >> v2:172.21.15.86:3300/0 conn(0x7fcb340b21a0 0x7fcb340ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 --2- 172.21.15.71:0/379984299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b1230 0x7fcb340b1c60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/379984299 >> 172.21.15.71:0/379984299 conn(0x7fcb340aec60 msgr2=0x7fcb340af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/379984299 shutdown_connections 2023-10-26T05:48:22.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/379984299 wait complete. 2023-10-26T05:48:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 Processor -- start 2023-10-26T05:48:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- start start 2023-10-26T05:48:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb340b1230 0x7fcb3413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 0x7fcb34156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb34141a40 0x7fcb3413d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34157ae0 con 0x7fcb340b21a0 2023-10-26T05:48:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3413cc70 con 0x7fcb340b1230 2023-10-26T05:48:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34158e00 con 0x7fcb34141a40 2023-10-26T05:48:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 0x7fcb34156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 0x7fcb34156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58252/0 (socket says 172.21.15.71:58252) 2023-10-26T05:48:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 -- 172.21.15.71:0/1092766466 learned_addr learned my addr 172.21.15.71:0/1092766466 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb32ffd640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb340b1230 0x7fcb3413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb337fe640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb34141a40 0x7fcb3413d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 -- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb34141a40 msgr2=0x7fcb3413d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb34141a40 0x7fcb3413d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 -- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb340b1230 msgr2=0x7fcb3413e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb340b1230 0x7fcb3413e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 -- 172.21.15.71:0/1092766466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c009560 con 0x7fcb340b21a0 2023-10-26T05:48:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb337fe640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb34141a40 0x7fcb3413d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:22.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb327fc640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 0x7fcb34156a40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c01cc80 tx=0x7fcb1c005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:22.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0052f0 con 0x7fcb340b21a0 2023-10-26T05:48:22.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcb1c005540 con 0x7fcb340b21a0 2023-10-26T05:48:22.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb34158840 con 0x7fcb340b21a0 2023-10-26T05:48:22.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0b1a90 con 0x7fcb340b21a0 2023-10-26T05:48:22.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.825+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3413cf10 con 0x7fcb340b21a0 2023-10-26T05:48:22.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.829+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb34066b10 con 0x7fcb340b21a0 2023-10-26T05:48:22.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.829+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcb1c0b7040 con 0x7fcb340b21a0 2023-10-26T05:48:22.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.829+0000 7fcb2bfff640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcb14066470 0x7fcb14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:22.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.829+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcb1c122840 con 0x7fcb340b21a0 2023-10-26T05:48:22.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.829+0000 7fcb32ffd640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcb14066470 0x7fcb14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:22.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.829+0000 7fcb32ffd640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcb14066470 0x7fcb14068930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b810 tx=0x7fcb24008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:22.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.833+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcb1c0eff80 con 0x7fcb340b21a0 2023-10-26T05:48:22.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:22.985+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcb3413cad0 con 0x7fcb340b21a0 2023-10-26T05:48:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.673+0000 7fcb2bfff640 1 -- 172.21.15.71:0/1092766466 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcb3413cad0 con 0x7fcb340b21a0 2023-10-26T05:48:23.678 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:48:23.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcb14066470 msgr2=0x7fcb14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:23.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcb14066470 0x7fcb14068930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b810 tx=0x7fcb24008040 comp rx=0 tx=0).stop 2023-10-26T05:48:23.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 msgr2=0x7fcb34156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:23.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 0x7fcb34156a40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c01cc80 tx=0x7fcb1c005e90 comp rx=0 tx=0).stop 2023-10-26T05:48:23.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 shutdown_connections 2023-10-26T05:48:23.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcb34141a40 0x7fcb3413d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:23.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcb14066470 0x7fcb14068930 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:23.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb340b21a0 0x7fcb34156a40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:23.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 --2- 172.21.15.71:0/1092766466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb340b1230 0x7fcb3413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:23.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 >> 172.21.15.71:0/1092766466 conn(0x7fcb340aec60 msgr2=0x7fcb3413c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:23.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 shutdown_connections 2023-10-26T05:48:23.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.677+0000 7fcb39a39640 1 -- 172.21.15.71:0/1092766466 wait complete. 2023-10-26T05:48:23.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:48:23.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/2309126779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b54003980 msgr2=0x7f6b54003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:23.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 --2- 172.21.15.71:0/2309126779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b54003980 0x7f6b54003da0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009b90 tx=0x7f6b6401d260 comp rx=0 tx=0).stop 2023-10-26T05:48:23.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6e989640 1 -- 172.21.15.71:0/2309126779 reap_dead start 2023-10-26T05:48:23.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/2309126779 shutdown_connections 2023-10-26T05:48:23.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/2309126779 >> 172.21.15.71:0/2309126779 conn(0x7f6b680cd770 msgr2=0x7f6b680cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:23.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/2309126779 shutdown_connections 2023-10-26T05:48:23.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/2309126779 wait complete. 2023-10-26T05:48:23.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 Processor -- start 2023-10-26T05:48:23.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- start start 2023-10-26T05:48:23.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b54003980 0x7f6b68152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:23.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 0x7f6b6815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:23.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b681573b0 0x7f6b68157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:23.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68157d70 con 0x7f6b6815ef30 2023-10-26T05:48:23.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68153800 con 0x7f6b54003980 2023-10-26T05:48:23.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68153300 con 0x7f6b681573b0 2023-10-26T05:48:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 0x7f6b6815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 0x7f6b6815f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58272/0 (socket says 172.21.15.71:58272) 2023-10-26T05:48:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 -- 172.21.15.71:0/648201526 learned_addr learned my addr 172.21.15.71:0/648201526 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6e989640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b681573b0 0x7f6b68157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6e188640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b54003980 0x7f6b68152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 -- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b681573b0 msgr2=0x7f6b68157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b681573b0 0x7f6b68157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 -- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b54003980 msgr2=0x7f6b68152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b54003980 0x7f6b68152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 -- 172.21.15.71:0/648201526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6815faa0 con 0x7f6b6815ef30 2023-10-26T05:48:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b6d987640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 0x7f6b6815f3b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6b58000c10 tx=0x7f6b5800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b580062e0 con 0x7f6b6815ef30 2023-10-26T05:48:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b58006530 con 0x7f6b6815ef30 2023-10-26T05:48:23.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6815fd80 con 0x7f6b6815ef30 2023-10-26T05:48:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800ba60 con 0x7f6b6815ef30 2023-10-26T05:48:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.833+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b68160340 con 0x7f6b6815ef30 2023-10-26T05:48:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.837+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b68066b10 con 0x7f6b6815ef30 2023-10-26T05:48:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.837+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6b5800bbc0 con 0x7f6b6815ef30 2023-10-26T05:48:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.837+0000 7f6b5effd640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b4c066470 0x7f6b4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.837+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6b58084b30 con 0x7f6b6815ef30 2023-10-26T05:48:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.837+0000 7f6b6e188640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b4c066470 0x7f6b4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.837+0000 7f6b6e188640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b4c066470 0x7f6b4c068930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ae000 tx=0x7f6b6401a000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:23.841+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6b58052090 con 0x7f6b6815ef30 2023-10-26T05:48:24.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.029+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6b680ae780 con 0x7f6b6815ef30 2023-10-26T05:48:24.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.697+0000 7f6b5effd640 1 -- 172.21.15.71:0/648201526 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v10) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6b580556b0 con 0x7f6b6815ef30 2023-10-26T05:48:24.705 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 4 and data pool 5 2023-10-26T05:48:24.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b4c066470 msgr2=0x7f6b4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:24.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b4c066470 0x7f6b4c068930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ae000 tx=0x7f6b6401a000 comp rx=0 tx=0).stop 2023-10-26T05:48:24.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 msgr2=0x7f6b6815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:24.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 0x7f6b6815f3b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6b58000c10 tx=0x7f6b5800dd80 comp rx=0 tx=0).stop 2023-10-26T05:48:24.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 shutdown_connections 2023-10-26T05:48:24.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b681573b0 0x7f6b68157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b4c066470 0x7f6b4c068930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b6815ef30 0x7f6b6815f3b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 --2- 172.21.15.71:0/648201526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b54003980 0x7f6b68152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 >> 172.21.15.71:0/648201526 conn(0x7f6b680cd770 msgr2=0x7f6b680c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:24.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.705+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 shutdown_connections 2023-10-26T05:48:24.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.709+0000 7f6b703ec640 1 -- 172.21.15.71:0/648201526 wait complete. 2023-10-26T05:48:24.734 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:24.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 -- 172.21.15.71:0/1445519266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0470 msgr2=0x7f9b4006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:24.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 --2- 172.21.15.71:0/1445519266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0470 0x7f9b4006aa70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9b400c4dd0 tx=0x7f9b3001c720 comp rx=0 tx=0).stop 2023-10-26T05:48:24.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 -- 172.21.15.71:0/1445519266 shutdown_connections 2023-10-26T05:48:24.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 --2- 172.21.15.71:0/1445519266 >> v2:172.21.15.86:3300/0 conn(0x7f9b400d0850 0x7f9b4015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 --2- 172.21.15.71:0/1445519266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0470 0x7f9b4006aa70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 -- 172.21.15.71:0/1445519266 >> 172.21.15.71:0/1445519266 conn(0x7f9b400cdd80 msgr2=0x7f9b400ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:24.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 -- 172.21.15.71:0/1445519266 shutdown_connections 2023-10-26T05:48:24.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.841+0000 7f9b4557d640 1 -- 172.21.15.71:0/1445519266 wait complete. 2023-10-26T05:48:24.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 Processor -- start 2023-10-26T05:48:24.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- start start 2023-10-26T05:48:24.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b400cd880 0x7f9b40165f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:24.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b400d0470 0x7f9b401566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:24.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 0x7f9b4015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:24.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4015bde0 con 0x7f9b400d0850 2023-10-26T05:48:24.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4015b210 con 0x7f9b400cd880 2023-10-26T05:48:24.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4015b4b0 con 0x7f9b400d0470 2023-10-26T05:48:24.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 0x7f9b4015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:24.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 0x7f9b4015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58294/0 (socket says 172.21.15.71:58294) 2023-10-26T05:48:24.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 -- 172.21.15.71:0/935917793 learned_addr learned my addr 172.21.15.71:0/935917793 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:24.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 -- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b400d0470 msgr2=0x7f9b401566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:24.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b400d0470 0x7f9b401566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 -- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b400cd880 msgr2=0x7f9b40165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:24.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b400cd880 0x7f9b40165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:24.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 -- 172.21.15.71:0/935917793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b300094a0 con 0x7f9b400d0850 2023-10-26T05:48:24.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b3f7fe640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 0x7f9b4015f280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800cae0 tx=0x7f9b3800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:24.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38003b80 con 0x7f9b400d0850 2023-10-26T05:48:24.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b38003dd0 con 0x7f9b400d0850 2023-10-26T05:48:24.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800aab0 con 0x7f9b400d0850 2023-10-26T05:48:24.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4015b790 con 0x7f9b400d0850 2023-10-26T05:48:24.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b40158250 con 0x7f9b400d0850 2023-10-26T05:48:24.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b40066b10 con 0x7f9b400d0850 2023-10-26T05:48:24.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.845+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b3800ac10 con 0x7f9b400d0850 2023-10-26T05:48:24.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.849+0000 7f9b277fe640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b2c066470 0x7f9b2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:24.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.849+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b380845e0 con 0x7f9b400d0850 2023-10-26T05:48:24.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.849+0000 7f9b3effd640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b2c066470 0x7f9b2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:24.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.849+0000 7f9b3effd640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b2c066470 0x7f9b2c068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b300af000 tx=0x7f9b300194c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:24.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:24.853+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b380519e0 con 0x7f9b400d0850 2023-10-26T05:48:25.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.009+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b400721e0 con 0x7f9b400d0850 2023-10-26T05:48:25.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.013+0000 7f9b277fe640 1 -- 172.21.15.71:0/935917793 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16498 (secure 0 0 0) 0x7f9b38073070 con 0x7f9b400d0850 2023-10-26T05:48:25.018 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:25.018 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":33,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:24.032633+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:25.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.013+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b2c066470 msgr2=0x7f9b2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.013+0000 7f9b4557d640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b2c066470 0x7f9b2c068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b300af000 tx=0x7f9b300194c0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.013+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 msgr2=0x7f9b4015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.013+0000 7f9b4557d640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 0x7f9b4015f280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800cae0 tx=0x7f9b3800cbd0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 shutdown_connections 2023-10-26T05:48:25.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b2c066470 0x7f9b2c068930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b400d0850 0x7f9b4015f280 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b400d0470 0x7f9b401566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 --2- 172.21.15.71:0/935917793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b400cd880 0x7f9b40165f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 >> 172.21.15.71:0/935917793 conn(0x7f9b400cdd80 msgr2=0x7f9b400b5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:25.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 shutdown_connections 2023-10-26T05:48:25.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.017+0000 7f9b4557d640 1 -- 172.21.15.71:0/935917793 wait complete. 2023-10-26T05:48:25.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:25.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/2053729269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2938070fe0 msgr2=0x7f2938071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 --2- 172.21.15.71:0/2053729269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2938070fe0 0x7f2938071470 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2920009900 tx=0x7f292001c810 comp rx=0 tx=0).stop 2023-10-26T05:48:25.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/2053729269 shutdown_connections 2023-10-26T05:48:25.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 --2- 172.21.15.71:0/2053729269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2938070fe0 0x7f2938071470 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 --2- 172.21.15.71:0/2053729269 >> v2:172.21.15.86:3300/0 conn(0x7f2938140cc0 0x7f293815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/2053729269 >> 172.21.15.71:0/2053729269 conn(0x7f2938069770 msgr2=0x7f2938069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:25.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/2053729269 shutdown_connections 2023-10-26T05:48:25.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/2053729269 wait complete. 2023-10-26T05:48:25.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 Processor -- start 2023-10-26T05:48:25.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- start start 2023-10-26T05:48:25.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2938070fe0 0x7f2938134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 0x7f293813ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2938140cc0 0x7f2938139100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293813a790 con 0x7f29380b0040 2023-10-26T05:48:25.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29381346a0 con 0x7f2938070fe0 2023-10-26T05:48:25.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2938139b80 con 0x7f2938140cc0 2023-10-26T05:48:25.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 0x7f293813ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 0x7f293813ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58306/0 (socket says 172.21.15.71:58306) 2023-10-26T05:48:25.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 -- 172.21.15.71:0/1058504192 learned_addr learned my addr 172.21.15.71:0/1058504192 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:25.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293659c640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2938070fe0 0x7f2938134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 -- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2938140cc0 msgr2=0x7f2938139100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2938140cc0 0x7f2938139100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 -- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2938070fe0 msgr2=0x7f2938134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2938070fe0 0x7f2938134160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 -- 172.21.15.71:0/1058504192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009560 con 0x7f29380b0040 2023-10-26T05:48:25.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293659c640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2938070fe0 0x7f2938134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:25.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f2935d9b640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 0x7f293813ac20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2920009850 tx=0x7f29200a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:25.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920005230 con 0x7f29380b0040 2023-10-26T05:48:25.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2920005480 con 0x7f29380b0040 2023-10-26T05:48:25.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2938139e60 con 0x7f29380b0040 2023-10-26T05:48:25.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920004930 con 0x7f29380b0040 2023-10-26T05:48:25.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.161+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293813a420 con 0x7f29380b0040 2023-10-26T05:48:25.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.165+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2938066b10 con 0x7f29380b0040 2023-10-26T05:48:25.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.165+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29200b4070 con 0x7f29380b0040 2023-10-26T05:48:25.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.165+0000 7f292effd640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2918066470 0x7f2918068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.165+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f292011e4d0 con 0x7f29380b0040 2023-10-26T05:48:25.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.165+0000 7f293659c640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2918066470 0x7f2918068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.165+0000 7f293659c640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2918066470 0x7f2918068930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29380705c0 tx=0x7f2928009340 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:25.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.169+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29200ebac0 con 0x7f29380b0040 2023-10-26T05:48:25.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.321+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2938066b10 con 0x7f29380b0040 2023-10-26T05:48:25.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.321+0000 7f292effd640 1 -- 172.21.15.71:0/1058504192 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16498 (secure 0 0 0) 0x7f29200a5980 con 0x7f29380b0040 2023-10-26T05:48:25.328 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:25.328 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":33,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:24.032633+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:25.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2918066470 msgr2=0x7f2918068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2918066470 0x7f2918068930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29380705c0 tx=0x7f2928009340 comp rx=0 tx=0).stop 2023-10-26T05:48:25.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 msgr2=0x7f293813ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 0x7f293813ac20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2920009850 tx=0x7f29200a56d0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 shutdown_connections 2023-10-26T05:48:25.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2938140cc0 0x7f2938139100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2918066470 0x7f2918068930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29380b0040 0x7f293813ac20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 --2- 172.21.15.71:0/1058504192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2938070fe0 0x7f2938134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 >> 172.21.15.71:0/1058504192 conn(0x7f2938069770 msgr2=0x7f2938070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:25.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 shutdown_connections 2023-10-26T05:48:25.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.325+0000 7f293ca81640 1 -- 172.21.15.71:0/1058504192 wait complete. 2023-10-26T05:48:25.345 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:48:25.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 -- 172.21.15.71:0/3249958285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280141a40 msgr2=0x7f12800c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 --2- 172.21.15.71:0/3249958285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280141a40 0x7f12800c9060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1270009840 tx=0x7f127001eff0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 -- 172.21.15.71:0/3249958285 shutdown_connections 2023-10-26T05:48:25.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 --2- 172.21.15.71:0/3249958285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12800b1230 0x7f12800b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 --2- 172.21.15.71:0/3249958285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1280141a40 0x7f12800c9060 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 -- 172.21.15.71:0/3249958285 >> 172.21.15.71:0/3249958285 conn(0x7f12800aec60 msgr2=0x7f12800af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:25.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 -- 172.21.15.71:0/3249958285 shutdown_connections 2023-10-26T05:48:25.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 -- 172.21.15.71:0/3249958285 wait complete. 2023-10-26T05:48:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 Processor -- start 2023-10-26T05:48:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 -- start start 2023-10-26T05:48:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12800b1230 0x7f128013a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.453+0000 7f1286fbb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 0x7f12801f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280141a40 0x7f128015a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128013a150 con 0x7f12800b21a0 2023-10-26T05:48:25.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128013f290 con 0x7f1280141a40 2023-10-26T05:48:25.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128015af50 con 0x7f12800b1230 2023-10-26T05:48:25.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 0x7f12801f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 0x7f12801f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58328/0 (socket says 172.21.15.71:58328) 2023-10-26T05:48:25.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 -- 172.21.15.71:0/1125733885 learned_addr learned my addr 172.21.15.71:0/1125733885 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:25.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1284d57640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12800b1230 0x7f128013a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1285558640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280141a40 0x7f128015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 -- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12800b1230 msgr2=0x7f128013a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12800b1230 0x7f128013a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 -- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280141a40 msgr2=0x7f128015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280141a40 0x7f128015a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 -- 172.21.15.71:0/1125733885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700094a0 con 0x7f12800b21a0 2023-10-26T05:48:25.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127ffff640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 0x7f12801f1040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f127400c970 tx=0x7f127400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:25.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274007bb0 con 0x7f12800b21a0 2023-10-26T05:48:25.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1274007e00 con 0x7f12800b21a0 2023-10-26T05:48:25.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128015b230 con 0x7f12800b21a0 2023-10-26T05:48:25.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274005280 con 0x7f12800b21a0 2023-10-26T05:48:25.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128013a850 con 0x7f12800b21a0 2023-10-26T05:48:25.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1280066b10 con 0x7f12800b21a0 2023-10-26T05:48:25.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f12740053e0 con 0x7f12800b21a0 2023-10-26T05:48:25.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127d7fa640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f126c0664a0 0x7f126c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12740848e0 con 0x7f12800b21a0 2023-10-26T05:48:25.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1284d57640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f126c0664a0 0x7f126c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.457+0000 7f1284d57640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f126c0664a0 0x7f126c068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12700b2000 tx=0x7f127001d830 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:25.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.461+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1274051dc0 con 0x7f12800b21a0 2023-10-26T05:48:25.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.633+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f128015b420 con 0x7f12800b21a0 2023-10-26T05:48:25.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.709+0000 7f127d7fa640 1 -- 172.21.15.71:0/1125733885 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7f127401d0b0 con 0x7f12800b21a0 2023-10-26T05:48:25.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f126c0664a0 msgr2=0x7f126c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f126c0664a0 0x7f126c068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12700b2000 tx=0x7f127001d830 comp rx=0 tx=0).stop 2023-10-26T05:48:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 msgr2=0x7f12801f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 0x7f12801f1040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f127400c970 tx=0x7f127400ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 shutdown_connections 2023-10-26T05:48:25.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1280141a40 0x7f128015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f126c0664a0 0x7f126c068960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12800b21a0 0x7f12801f1040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 --2- 172.21.15.71:0/1125733885 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f12800b1230 0x7f128013a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 >> 172.21.15.71:0/1125733885 conn(0x7f12800aec60 msgr2=0x7f128013e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:25.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 shutdown_connections 2023-10-26T05:48:25.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.713+0000 7f1286fbb640 1 -- 172.21.15.71:0/1125733885 wait complete. 2023-10-26T05:48:25.734 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:48:25.735 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:48:25.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- 172.21.15.71:0/1464472433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c14e7b0 msgr2=0x7f779c157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 --2- 172.21.15.71:0/1464472433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c14e7b0 0x7f779c157db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f778c009840 tx=0x7f778c01c720 comp rx=0 tx=0).stop 2023-10-26T05:48:25.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- 172.21.15.71:0/1464472433 shutdown_connections 2023-10-26T05:48:25.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 --2- 172.21.15.71:0/1464472433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f779c071270 0x7f779c15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 --2- 172.21.15.71:0/1464472433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c14e7b0 0x7f779c157db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- 172.21.15.71:0/1464472433 >> 172.21.15.71:0/1464472433 conn(0x7f779c0aec60 msgr2=0x7f779c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:25.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- 172.21.15.71:0/1464472433 shutdown_connections 2023-10-26T05:48:25.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- 172.21.15.71:0/1464472433 wait complete. 2023-10-26T05:48:25.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 Processor -- start 2023-10-26T05:48:25.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- start start 2023-10-26T05:48:25.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 0x7f779c0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f779c14e7b0 0x7f779c13f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779c15c0a0 0x7f779c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1515c0 con 0x7f779c071270 2023-10-26T05:48:25.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1518a0 con 0x7f779c15c0a0 2023-10-26T05:48:25.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f77a1aed640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c151b80 con 0x7f779c14e7b0 2023-10-26T05:48:25.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 0x7f779c0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 0x7f779c0cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58338/0 (socket says 172.21.15.71:58338) 2023-10-26T05:48:25.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 -- 172.21.15.71:0/3827377026 learned_addr learned my addr 172.21.15.71:0/3827377026 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:25.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779affd640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f779c14e7b0 0x7f779c13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779bfff640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779c15c0a0 0x7f779c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 -- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f779c14e7b0 msgr2=0x7f779c13f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f779c14e7b0 0x7f779c13f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 -- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779c15c0a0 msgr2=0x7f779c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:25.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779c15c0a0 0x7f779c15a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:25.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779b7fe640 1 -- 172.21.15.71:0/3827377026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0094a0 con 0x7f779c071270 2023-10-26T05:48:25.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.857+0000 7f779bfff640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779c15c0a0 0x7f779c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:25.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f779b7fe640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 0x7f779c0cc980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f778c01c550 tx=0x7f778c0a7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:25.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c0a7c30 con 0x7f779c071270 2023-10-26T05:48:25.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f778c0a7e80 con 0x7f779c071270 2023-10-26T05:48:25.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c151e60 con 0x7f779c071270 2023-10-26T05:48:25.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c005240 con 0x7f779c071270 2023-10-26T05:48:25.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c15a850 con 0x7f779c071270 2023-10-26T05:48:25.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c15aaa0 con 0x7f779c071270 2023-10-26T05:48:25.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f778c0053a0 con 0x7f779c071270 2023-10-26T05:48:25.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f7783fff640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f77880664a0 0x7f7788068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:25.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f778c0aa070 con 0x7f779c071270 2023-10-26T05:48:25.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f779affd640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f77880664a0 0x7f7788068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:25.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.861+0000 7f779affd640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f77880664a0 0x7f7788068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f77900058d0 tx=0x7f7790005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:25.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:25.865+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f778c0ebea0 con 0x7f779c071270 2023-10-26T05:48:26.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.049+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f779c066b10 con 0x7f779c071270 2023-10-26T05:48:26.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f7783fff640 1 -- 172.21.15.71:0/3827377026 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f778c0ef4c0 con 0x7f779c071270 2023-10-26T05:48:26.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f77880664a0 msgr2=0x7f7788068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:26.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f77880664a0 0x7f7788068960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f77900058d0 tx=0x7f7790005860 comp rx=0 tx=0).stop 2023-10-26T05:48:26.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 msgr2=0x7f779c0cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:26.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 0x7f779c0cc980 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f778c01c550 tx=0x7f778c0a7780 comp rx=0 tx=0).stop 2023-10-26T05:48:26.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 shutdown_connections 2023-10-26T05:48:26.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f779c15c0a0 0x7f779c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f779c14e7b0 0x7f779c13f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f77880664a0 0x7f7788068960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 --2- 172.21.15.71:0/3827377026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f779c071270 0x7f779c0cc980 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 >> 172.21.15.71:0/3827377026 conn(0x7f779c0aec60 msgr2=0x7f779c0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:26.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 shutdown_connections 2023-10-26T05:48:26.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.713+0000 7f77a1aed640 1 -- 172.21.15.71:0/3827377026 wait complete. 2023-10-26T05:48:26.740 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:48:26.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:26.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/1593039963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 msgr2=0x7fad240b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:26.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 --2- 172.21.15.71:0/1593039963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240b17b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009c80 tx=0x7fad1c01ca50 comp rx=0 tx=0).stop 2023-10-26T05:48:26.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/1593039963 shutdown_connections 2023-10-26T05:48:26.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 --2- 172.21.15.71:0/1593039963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240b17b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 --2- 172.21.15.71:0/1593039963 >> v2:172.21.15.86:3300/0 conn(0x7fad24070fe0 0x7fad24071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/1593039963 >> 172.21.15.71:0/1593039963 conn(0x7fad24069770 msgr2=0x7fad24069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:26.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/1593039963 shutdown_connections 2023-10-26T05:48:26.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/1593039963 wait complete. 2023-10-26T05:48:26.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 Processor -- start 2023-10-26T05:48:26.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- start start 2023-10-26T05:48:26.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad24070fe0 0x7fad24157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:26.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:26.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24140cc0 0x7fad2413b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:26.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad24157940 con 0x7fad240b1360 2023-10-26T05:48:26.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad240ad8d0 con 0x7fad24140cc0 2023-10-26T05:48:26.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad240ad310 con 0x7fad24070fe0 2023-10-26T05:48:26.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:26.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58348/0 (socket says 172.21.15.71:58348) 2023-10-26T05:48:26.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 -- 172.21.15.71:0/2053264074 learned_addr learned my addr 172.21.15.71:0/2053264074 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:26.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 -- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad24070fe0 msgr2=0x7fad24157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:26.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad24070fe0 0x7fad24157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 -- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24140cc0 msgr2=0x7fad2413b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:26.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24140cc0 0x7fad2413b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:26.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 -- 172.21.15.71:0/2053264074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0098e0 con 0x7fad240b1360 2023-10-26T05:48:26.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad1bfff640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240aca10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b550 tx=0x7fad0c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:26.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c010870 con 0x7fad240b1360 2023-10-26T05:48:26.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad0c010ae0 con 0x7fad240b1360 2023-10-26T05:48:26.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c00f3c0 con 0x7fad240b1360 2023-10-26T05:48:26.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad240ad5f0 con 0x7fad240b1360 2023-10-26T05:48:26.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.861+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2413b390 con 0x7fad240b1360 2023-10-26T05:48:26.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.865+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad24066b10 con 0x7fad240b1360 2023-10-26T05:48:26.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.865+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad0c00f520 con 0x7fad240b1360 2023-10-26T05:48:26.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.865+0000 7fad217fa640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad08066470 0x7fad08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:26.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.865+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad0c084650 con 0x7fad240b1360 2023-10-26T05:48:26.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.865+0000 7fad23fff640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad08066470 0x7fad08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:26.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.865+0000 7fad23fff640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad08066470 0x7fad08068930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad14000f70 tx=0x7fad140079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:26.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:26.869+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad0c051a20 con 0x7fad240b1360 2023-10-26T05:48:27.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.057+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad2413b5b0 con 0x7fad240b1360 2023-10-26T05:48:27.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.057+0000 7fad217fa640 1 -- 172.21.15.71:0/2053264074 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fad0c01e090 con 0x7fad240b1360 2023-10-26T05:48:27.063 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:27.063 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:27.064 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:27.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad08066470 msgr2=0x7fad08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad08066470 0x7fad08068930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fad14000f70 tx=0x7fad140079c0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 msgr2=0x7fad240aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240aca10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fad0c00b550 tx=0x7fad0c00ba10 comp rx=0 tx=0).stop 2023-10-26T05:48:27.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 shutdown_connections 2023-10-26T05:48:27.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad24140cc0 0x7fad2413b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad08066470 0x7fad08068930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad240b1360 0x7fad240aca10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 --2- 172.21.15.71:0/2053264074 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fad24070fe0 0x7fad24157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 >> 172.21.15.71:0/2053264074 conn(0x7fad24069770 msgr2=0x7fad2413db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:27.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 shutdown_connections 2023-10-26T05:48:27.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.061+0000 7fad2a99c640 1 -- 172.21.15.71:0/2053264074 wait complete. 2023-10-26T05:48:27.081 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:27.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 -- 172.21.15.71:0/853042896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 msgr2=0x7f7c0c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 --2- 172.21.15.71:0/853042896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c06aa70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009900 tx=0x7f7c0801c770 comp rx=0 tx=0).stop 2023-10-26T05:48:27.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 -- 172.21.15.71:0/853042896 shutdown_connections 2023-10-26T05:48:27.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 --2- 172.21.15.71:0/853042896 >> v2:172.21.15.86:3300/0 conn(0x7f7c0c0cd880 0x7f7c0c15bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 --2- 172.21.15.71:0/853042896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c06aa70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 -- 172.21.15.71:0/853042896 >> 172.21.15.71:0/853042896 conn(0x7f7c0c0cdd80 msgr2=0x7f7c0c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:27.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 -- 172.21.15.71:0/853042896 shutdown_connections 2023-10-26T05:48:27.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.197+0000 7f7c1423c640 1 -- 172.21.15.71:0/853042896 wait complete. 2023-10-26T05:48:27.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 Processor -- start 2023-10-26T05:48:27.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- start start 2023-10-26T05:48:27.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c0cd880 0x7f7c0c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 0x7f7c0c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c151860 con 0x7f7c0c0d0850 2023-10-26T05:48:27.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c159d50 con 0x7f7c0c0cd880 2023-10-26T05:48:27.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15a030 con 0x7f7c0c0d0470 2023-10-26T05:48:27.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 0x7f7c0c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 0x7f7c0c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58360/0 (socket says 172.21.15.71:58360) 2023-10-26T05:48:27.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 -- 172.21.15.71:0/4078915318 learned_addr learned my addr 172.21.15.71:0/4078915318 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:27.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c11fd8640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c0cd880 0x7f7c0c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c117d7640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 -- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 msgr2=0x7f7c0c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 -- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c0cd880 msgr2=0x7f7c0c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c0cd880 0x7f7c0c151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 -- 172.21.15.71:0/4078915318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c04009820 con 0x7f7c0c0d0850 2023-10-26T05:48:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c117d7640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c11fd8640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c0cd880 0x7f7c0c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c127d9640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 0x7f7c0c152660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400efd0 tx=0x7f7c0400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04009b80 con 0x7f7c0c0d0850 2023-10-26T05:48:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c04003b80 con 0x7f7c0c0d0850 2023-10-26T05:48:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c08009560 con 0x7f7c0c0d0850 2023-10-26T05:48:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004020 con 0x7f7c0c0d0850 2023-10-26T05:48:27.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c15a5f0 con 0x7f7c0c0d0850 2023-10-26T05:48:27.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.201+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c066b10 con 0x7f7c0c0d0850 2023-10-26T05:48:27.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.205+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c04004180 con 0x7f7c0c0d0850 2023-10-26T05:48:27.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.205+0000 7f7c027fc640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7bf00664a0 0x7f7bf0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.205+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c04014070 con 0x7f7c0c0d0850 2023-10-26T05:48:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.205+0000 7f7c11fd8640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7bf00664a0 0x7f7bf0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.205+0000 7f7c11fd8640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7bf00664a0 0x7f7bf0068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c080af000 tx=0x7f7c0801afb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.209+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c04052220 con 0x7f7c0c0d0850 2023-10-26T05:48:27.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.349+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c0c0721e0 con 0x7f7c0c0d0850 2023-10-26T05:48:27.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c027fc640 1 -- 172.21.15.71:0/4078915318 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f7c04055840 con 0x7f7c0c0d0850 2023-10-26T05:48:27.358 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:27.358 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":34,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:25.668304+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7bf00664a0 msgr2=0x7f7bf0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7bf00664a0 0x7f7bf0068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c080af000 tx=0x7f7c0801afb0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 msgr2=0x7f7c0c152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 0x7f7c0c152660 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400efd0 tx=0x7f7c0400c6a0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 shutdown_connections 2023-10-26T05:48:27.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7bf00664a0 0x7f7bf0068960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c0c0d0850 0x7f7c0c152660 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c0c0d0470 0x7f7c0c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 --2- 172.21.15.71:0/4078915318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c0c0cd880 0x7f7c0c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.353+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 >> 172.21.15.71:0/4078915318 conn(0x7f7c0c0cdd80 msgr2=0x7f7c0c0ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:27.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.357+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 shutdown_connections 2023-10-26T05:48:27.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.357+0000 7f7c1423c640 1 -- 172.21.15.71:0/4078915318 wait complete. 2023-10-26T05:48:27.375 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:48:27.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:48:27.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 -- 172.21.15.71:0/3349470721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40b6910 msgr2=0x7fdbf40b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 --2- 172.21.15.71:0/3349470721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40b6910 0x7fdbf40b6d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40d0e20 tx=0x7fdbdc01c740 comp rx=0 tx=0).stop 2023-10-26T05:48:27.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 -- 172.21.15.71:0/3349470721 shutdown_connections 2023-10-26T05:48:27.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 --2- 172.21.15.71:0/3349470721 >> v2:172.21.15.86:3300/0 conn(0x7fdbf40ceea0 0x7fdbf40cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 --2- 172.21.15.71:0/3349470721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40b6910 0x7fdbf40b6d80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 -- 172.21.15.71:0/3349470721 >> 172.21.15.71:0/3349470721 conn(0x7fdbf40d12d0 msgr2=0x7fdbf40d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:27.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 -- 172.21.15.71:0/3349470721 shutdown_connections 2023-10-26T05:48:27.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.481+0000 7fdbfa199640 1 -- 172.21.15.71:0/3349470721 wait complete. 2023-10-26T05:48:27.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 Processor -- start 2023-10-26T05:48:27.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- start start 2023-10-26T05:48:27.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbf40b6910 0x7fdbf414f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 0x7fdbf415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf415b5c0 0x7fdbf41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf414fa40 con 0x7fdbf40ceea0 2023-10-26T05:48:27.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4150c40 con 0x7fdbf415b5c0 2023-10-26T05:48:27.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4150030 con 0x7fdbf40b6910 2023-10-26T05:48:27.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 0x7fdbf415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 0x7fdbf415c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58372/0 (socket says 172.21.15.71:58372) 2023-10-26T05:48:27.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 -- 172.21.15.71:0/2342359104 learned_addr learned my addr 172.21.15.71:0/2342359104 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:27.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbf3fff640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf415b5c0 0x7fdbf41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 -- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbf40b6910 msgr2=0x7fdbf414f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbf37fe640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbf40b6910 0x7fdbf414f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbf40b6910 0x7fdbf414f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 -- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf415b5c0 msgr2=0x7fdbf41548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf415b5c0 0x7fdbf41548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 -- 172.21.15.71:0/2342359104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc009560 con 0x7fdbf40ceea0 2023-10-26T05:48:27.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbf3fff640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf415b5c0 0x7fdbf41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:27.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbebfff640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 0x7fdbf415c4c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b0040 tx=0x7fdbdc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:27.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0044b0 con 0x7fdbf40ceea0 2023-10-26T05:48:27.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdbdc005120 con 0x7fdbf40ceea0 2023-10-26T05:48:27.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf415ff70 con 0x7fdbf40ceea0 2023-10-26T05:48:27.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc005530 con 0x7fdbf40ceea0 2023-10-26T05:48:27.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf41505f0 con 0x7fdbf40ceea0 2023-10-26T05:48:27.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.485+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf4066b10 con 0x7fdbf40ceea0 2023-10-26T05:48:27.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.489+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdbdc0b4070 con 0x7fdbf40ceea0 2023-10-26T05:48:27.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.489+0000 7fdbf0ff9640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdbd40664a0 0x7fdbd4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.489+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbdc11df70 con 0x7fdbf40ceea0 2023-10-26T05:48:27.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.489+0000 7fdbf37fe640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdbd40664a0 0x7fdbd4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.489+0000 7fdbf37fe640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdbd40664a0 0x7fdbd4068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400a790 tx=0x7fdbe4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:27.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.493+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdbdc0eb3e0 con 0x7fdbf40ceea0 2023-10-26T05:48:27.568 INFO:tasks.ceph.mds.a.smithi071.stderr:2023-10-26T05:48:27.565+0000 7fbcc234f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:48:27.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.633+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbf4150df0 con 0x7fdbd40664a0 2023-10-26T05:48:27.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbf0ff9640 1 -- 172.21.15.71:0/2342359104 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278482 (secure 0 0 0) 0x7fdbf4150df0 con 0x7fdbd40664a0 2023-10-26T05:48:27.662 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:27.664 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:48:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdbd40664a0 msgr2=0x7fdbd4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdbd40664a0 0x7fdbd4068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400a790 tx=0x7fdbe4008040 comp rx=0 tx=0).stop 2023-10-26T05:48:27.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 msgr2=0x7fdbf415c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 0x7fdbf415c4c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b0040 tx=0x7fdbdc0040c0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 shutdown_connections 2023-10-26T05:48:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdbf415b5c0 0x7fdbf41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdbd40664a0 0x7fdbd4068960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdbf40ceea0 0x7fdbf415c4c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 --2- 172.21.15.71:0/2342359104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdbf40b6910 0x7fdbf414f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 >> 172.21.15.71:0/2342359104 conn(0x7fdbf40d12d0 msgr2=0x7fdbf41586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:27.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 shutdown_connections 2023-10-26T05:48:27.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.657+0000 7fdbfa199640 1 -- 172.21.15.71:0/2342359104 wait complete. 2023-10-26T05:48:27.678 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-26T05:48:27.433648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560291,"statfs":{"total":1600000000,"available":1597738663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700234+0000","last_change":"2023-10-26T05:48:23.700234+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700234+0000","last_undegraded":"2023-10-26T05:48:23.700234+0000","last_fullsized":"2023-10-26T05:48:23.700234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694918+0000","last_change":"2023-10-26T05:48:23.694918+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694918+0000","last_undegraded":"2023-10-26T05:48:23.694918+0000","last_fullsized":"2023-10-26T05:48:23.694918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695709+0000","last_change":"2023-10-26T05:48:23.695709+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695709+0000","last_undegraded":"2023-10-26T05:48:23.695709+0000","last_fullsized":"2023-10-26T05:48:23.695709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688038+0000","last_change":"2023-10-26T05:48:23.688038+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688038+0000","last_undegraded":"2023-10-26T05:48:23.688038+0000","last_fullsized":"2023-10-26T05:48:23.688038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694048+0000","last_change":"2023-10-26T05:48:23.694048+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694048+0000","last_undegraded":"2023-10-26T05:48:23.694048+0000","last_fullsized":"2023-10-26T05:48:23.694048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694220+0000","last_change":"2023-10-26T05:48:23.694220+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694220+0000","last_undegraded":"2023-10-26T05:48:23.694220+0000","last_fullsized":"2023-10-26T05:48:23.694220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696282+0000","last_change":"2023-10-26T05:48:23.696282+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696282+0000","last_undegraded":"2023-10-26T05:48:23.696282+0000","last_fullsized":"2023-10-26T05:48:23.696282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700861+0000","last_change":"2023-10-26T05:48:23.700861+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700861+0000","last_undegraded":"2023-10-26T05:48:23.700861+0000","last_fullsized":"2023-10-26T05:48:23.700861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687341+0000","last_change":"2023-10-26T05:48:23.687341+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687341+0000","last_undegraded":"2023-10-26T05:48:23.687341+0000","last_fullsized":"2023-10-26T05:48:23.687341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688537+0000","last_change":"2023-10-26T05:48:23.688537+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688537+0000","last_undegraded":"2023-10-26T05:48:23.688537+0000","last_fullsized":"2023-10-26T05:48:23.688537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.697056+0000","last_change":"2023-10-26T05:48:23.697056+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.697056+0000","last_undegraded":"2023-10-26T05:48:23.697056+0000","last_fullsized":"2023-10-26T05:48:23.697056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694710+0000","last_change":"2023-10-26T05:48:23.694710+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694710+0000","last_undegraded":"2023-10-26T05:48:23.694710+0000","last_fullsized":"2023-10-26T05:48:23.694710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695640+0000","last_change":"2023-10-26T05:48:23.695640+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695640+0000","last_undegraded":"2023-10-26T05:48:23.695640+0000","last_fullsized":"2023-10-26T05:48:23.695640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692044+0000","last_change":"2023-10-26T05:48:23.692044+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692044+0000","last_undegraded":"2023-10-26T05:48:23.692044+0000","last_fullsized":"2023-10-26T05:48:23.692044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692542+0000","last_change":"2023-10-26T05:48:23.692542+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692542+0000","last_undegraded":"2023-10-26T05:48:23.692542+0000","last_fullsized":"2023-10-26T05:48:23.692542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696259+0000","last_change":"2023-10-26T05:48:23.696259+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696259+0000","last_undegraded":"2023-10-26T05:48:23.696259+0000","last_fullsized":"2023-10-26T05:48:23.696259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.706851+0000","last_change":"2023-10-26T05:48:23.706851+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.706851+0000","last_undegraded":"2023-10-26T05:48:23.706851+0000","last_fullsized":"2023-10-26T05:48:23.706851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695326+0000","last_change":"2023-10-26T05:48:23.695326+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695326+0000","last_undegraded":"2023-10-26T05:48:23.695326+0000","last_fullsized":"2023-10-26T05:48:23.695326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691624+0000","last_change":"2023-10-26T05:48:23.691624+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691624+0000","last_undegraded":"2023-10-26T05:48:23.691624+0000","last_fullsized":"2023-10-26T05:48:23.691624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693579+0000","last_change":"2023-10-26T05:48:23.693579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693579+0000","last_undegraded":"2023-10-26T05:48:23.693579+0000","last_fullsized":"2023-10-26T05:48:23.693579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685254+0000","last_change":"2023-10-26T05:48:23.685254+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685254+0000","last_undegraded":"2023-10-26T05:48:23.685254+0000","last_fullsized":"2023-10-26T05:48:23.685254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685503+0000","last_change":"2023-10-26T05:48:23.685503+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685503+0000","last_undegraded":"2023-10-26T05:48:23.685503+0000","last_fullsized":"2023-10-26T05:48:23.685503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691108+0000","last_change":"2023-10-26T05:48:23.691108+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691108+0000","last_undegraded":"2023-10-26T05:48:23.691108+0000","last_fullsized":"2023-10-26T05:48:23.691108+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690897+0000","last_change":"2023-10-26T05:48:23.690897+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690897+0000","last_undegraded":"2023-10-26T05:48:23.690897+0000","last_fullsized":"2023-10-26T05:48:23.690897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688241+0000","last_change":"2023-10-26T05:48:23.688241+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688241+0000","last_undegraded":"2023-10-26T05:48:23.688241+0000","last_fullsized":"2023-10-26T05:48:23.688241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692899+0000","last_change":"2023-10-26T05:48:23.692899+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692899+0000","last_undegraded":"2023-10-26T05:48:23.692899+0000","last_fullsized":"2023-10-26T05:48:23.692899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687469+0000","last_change":"2023-10-26T05:48:23.687469+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687469+0000","last_undegraded":"2023-10-26T05:48:23.687469+0000","last_fullsized":"2023-10-26T05:48:23.687469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693430+0000","last_change":"2023-10-26T05:48:23.693430+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693430+0000","last_undegraded":"2023-10-26T05:48:23.693430+0000","last_fullsized":"2023-10-26T05:48:23.693430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.686546+0000","last_change":"2023-10-26T05:48:23.686546+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.686546+0000","last_undegraded":"2023-10-26T05:48:23.686546+0000","last_fullsized":"2023-10-26T05:48:23.686546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694718+0000","last_change":"2023-10-26T05:48:23.694718+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694718+0000","last_undegraded":"2023-10-26T05:48:23.694718+0000","last_fullsized":"2023-10-26T05:48:23.694718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.699413+0000","last_change":"2023-10-26T05:48:23.699413+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.699413+0000","last_undegraded":"2023-10-26T05:48:23.699413+0000","last_fullsized":"2023-10-26T05:48:23.699413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.703535+0000","last_change":"2023-10-26T05:48:23.703535+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.703535+0000","last_undegraded":"2023-10-26T05:48:23.703535+0000","last_fullsized":"2023-10-26T05:48:23.703535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688409+0000","last_change":"2023-10-26T05:48:23.688409+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688409+0000","last_undegraded":"2023-10-26T05:48:23.688409+0000","last_fullsized":"2023-10-26T05:48:23.688409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.711284+0000","last_change":"2023-10-26T05:48:23.711070+0000","last_active":"2023-10-26T05:48:23.711284+0000","last_peered":"2023-10-26T05:48:23.711284+0000","last_clean":"2023-10-26T05:48:23.711284+0000","last_became_active":"2023-10-26T05:48:23.710421+0000","last_became_peered":"2023-10-26T05:48:23.710421+0000","last_unstale":"2023-10-26T05:48:23.711284+0000","last_undegraded":"2023-10-26T05:48:23.711284+0000","last_fullsized":"2023-10-26T05:48:23.711284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695820+0000","last_change":"2023-10-26T05:48:23.695820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695820+0000","last_undegraded":"2023-10-26T05:48:23.695820+0000","last_fullsized":"2023-10-26T05:48:23.695820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703916+0000","last_change":"2023-10-26T05:48:23.703810+0000","last_active":"2023-10-26T05:48:23.703916+0000","last_peered":"2023-10-26T05:48:23.703916+0000","last_clean":"2023-10-26T05:48:23.703916+0000","last_became_active":"2023-10-26T05:48:23.703500+0000","last_became_peered":"2023-10-26T05:48:23.703500+0000","last_unstale":"2023-10-26T05:48:23.703916+0000","last_undegraded":"2023-10-26T05:48:23.703916+0000","last_fullsized":"2023-10-26T05:48:23.703916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.683637+0000","last_change":"2023-10-26T05:48:23.683637+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.683637+0000","last_undegraded":"2023-10-26T05:48:23.683637+0000","last_fullsized":"2023-10-26T05:48:23.683637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690187+0000","last_change":"2023-10-26T05:48:23.690022+0000","last_active":"2023-10-26T05:48:23.690187+0000","last_peered":"2023-10-26T05:48:23.690187+0000","last_clean":"2023-10-26T05:48:23.690187+0000","last_became_active":"2023-10-26T05:48:23.689605+0000","last_became_peered":"2023-10-26T05:48:23.689605+0000","last_unstale":"2023-10-26T05:48:23.690187+0000","last_undegraded":"2023-10-26T05:48:23.690187+0000","last_fullsized":"2023-10-26T05:48:23.690187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687820+0000","last_change":"2023-10-26T05:48:23.687820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687820+0000","last_undegraded":"2023-10-26T05:48:23.687820+0000","last_fullsized":"2023-10-26T05:48:23.687820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701416+0000","last_change":"2023-10-26T05:48:23.701287+0000","last_active":"2023-10-26T05:48:23.701416+0000","last_peered":"2023-10-26T05:48:23.701416+0000","last_clean":"2023-10-26T05:48:23.701416+0000","last_became_active":"2023-10-26T05:48:23.700949+0000","last_became_peered":"2023-10-26T05:48:23.700949+0000","last_unstale":"2023-10-26T05:48:23.701416+0000","last_undegraded":"2023-10-26T05:48:23.701416+0000","last_fullsized":"2023-10-26T05:48:23.701416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700105+0000","last_change":"2023-10-26T05:48:23.700105+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700105+0000","last_undegraded":"2023-10-26T05:48:23.700105+0000","last_fullsized":"2023-10-26T05:48:23.700105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691137+0000","last_change":"2023-10-26T05:48:23.690572+0000","last_active":"2023-10-26T05:48:23.691137+0000","last_peered":"2023-10-26T05:48:23.691137+0000","last_clean":"2023-10-26T05:48:23.691137+0000","last_became_active":"2023-10-26T05:48:23.689616+0000","last_became_peered":"2023-10-26T05:48:23.689616+0000","last_unstale":"2023-10-26T05:48:23.691137+0000","last_undegraded":"2023-10-26T05:48:23.691137+0000","last_fullsized":"2023-10-26T05:48:23.691137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:36:17.063916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693976+0000","last_change":"2023-10-26T05:48:23.693976+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693976+0000","last_undegraded":"2023-10-26T05:48:23.693976+0000","last_fullsized":"2023-10-26T05:48:23.693976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701389+0000","last_change":"2023-10-26T05:48:23.701241+0000","last_active":"2023-10-26T05:48:23.701389+0000","last_peered":"2023-10-26T05:48:23.701389+0000","last_clean":"2023-10-26T05:48:23.701389+0000","last_became_active":"2023-10-26T05:48:23.699629+0000","last_became_peered":"2023-10-26T05:48:23.699629+0000","last_unstale":"2023-10-26T05:48:23.701389+0000","last_undegraded":"2023-10-26T05:48:23.701389+0000","last_fullsized":"2023-10-26T05:48:23.701389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:24:45.082594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688715+0000","last_change":"2023-10-26T05:48:23.688715+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688715+0000","last_undegraded":"2023-10-26T05:48:23.688715+0000","last_fullsized":"2023-10-26T05:48:23.688715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.700492+0000","last_change":"2023-10-26T05:48:23.700412+0000","last_active":"2023-10-26T05:48:23.700492+0000","last_peered":"2023-10-26T05:48:23.700492+0000","last_clean":"2023-10-26T05:48:23.700492+0000","last_became_active":"2023-10-26T05:48:23.698966+0000","last_became_peered":"2023-10-26T05:48:23.698966+0000","last_unstale":"2023-10-26T05:48:23.700492+0000","last_undegraded":"2023-10-26T05:48:23.700492+0000","last_fullsized":"2023-10-26T05:48:23.700492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687623+0000","last_change":"2023-10-26T05:48:23.687623+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687623+0000","last_undegraded":"2023-10-26T05:48:23.687623+0000","last_fullsized":"2023-10-26T05:48:23.687623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715611+0000","last_change":"2023-10-26T05:48:23.715497+0000","last_active":"2023-10-26T05:48:23.715611+0000","last_peered":"2023-10-26T05:48:23.715611+0000","last_clean":"2023-10-26T05:48:23.715611+0000","last_became_active":"2023-10-26T05:48:23.714277+0000","last_became_peered":"2023-10-26T05:48:23.714277+0000","last_unstale":"2023-10-26T05:48:23.715611+0000","last_undegraded":"2023-10-26T05:48:23.715611+0000","last_fullsized":"2023-10-26T05:48:23.715611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:16:54.930577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691328+0000","last_change":"2023-10-26T05:48:23.691328+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691328+0000","last_undegraded":"2023-10-26T05:48:23.691328+0000","last_fullsized":"2023-10-26T05:48:23.691328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705835+0000","last_change":"2023-10-26T05:48:23.705748+0000","last_active":"2023-10-26T05:48:23.705835+0000","last_peered":"2023-10-26T05:48:23.705835+0000","last_clean":"2023-10-26T05:48:23.705835+0000","last_became_active":"2023-10-26T05:48:23.705389+0000","last_became_peered":"2023-10-26T05:48:23.705389+0000","last_unstale":"2023-10-26T05:48:23.705835+0000","last_undegraded":"2023-10-26T05:48:23.705835+0000","last_fullsized":"2023-10-26T05:48:23.705835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692498+0000","last_change":"2023-10-26T05:48:23.692498+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692498+0000","last_undegraded":"2023-10-26T05:48:23.692498+0000","last_fullsized":"2023-10-26T05:48:23.692498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.696945+0000","last_change":"2023-10-26T05:48:23.696857+0000","last_active":"2023-10-26T05:48:23.696945+0000","last_peered":"2023-10-26T05:48:23.696945+0000","last_clean":"2023-10-26T05:48:23.696945+0000","last_became_active":"2023-10-26T05:48:23.696471+0000","last_became_peered":"2023-10-26T05:48:23.696471+0000","last_unstale":"2023-10-26T05:48:23.696945+0000","last_undegraded":"2023-10-26T05:48:23.696945+0000","last_fullsized":"2023-10-26T05:48:23.696945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691173+0000","last_change":"2023-10-26T05:48:23.691173+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691173+0000","last_undegraded":"2023-10-26T05:48:23.691173+0000","last_fullsized":"2023-10-26T05:48:23.691173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.709783+0000","last_change":"2023-10-26T05:48:23.709670+0000","last_active":"2023-10-26T05:48:23.709783+0000","last_peered":"2023-10-26T05:48:23.709783+0000","last_clean":"2023-10-26T05:48:23.709783+0000","last_became_active":"2023-10-26T05:48:23.709170+0000","last_became_peered":"2023-10-26T05:48:23.709170+0000","last_unstale":"2023-10-26T05:48:23.709783+0000","last_undegraded":"2023-10-26T05:48:23.709783+0000","last_fullsized":"2023-10-26T05:48:23.709783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:03.464765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692408+0000","last_change":"2023-10-26T05:48:23.692408+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692408+0000","last_undegraded":"2023-10-26T05:48:23.692408+0000","last_fullsized":"2023-10-26T05:48:23.692408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690158+0000","last_change":"2023-10-26T05:48:23.689973+0000","last_active":"2023-10-26T05:48:23.690158+0000","last_peered":"2023-10-26T05:48:23.690158+0000","last_clean":"2023-10-26T05:48:23.690158+0000","last_became_active":"2023-10-26T05:48:23.689550+0000","last_became_peered":"2023-10-26T05:48:23.689550+0000","last_unstale":"2023-10-26T05:48:23.690158+0000","last_undegraded":"2023-10-26T05:48:23.690158+0000","last_fullsized":"2023-10-26T05:48:23.690158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.768450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692771+0000","last_change":"2023-10-26T05:48:23.692771+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692771+0000","last_undegraded":"2023-10-26T05:48:23.692771+0000","last_fullsized":"2023-10-26T05:48:23.692771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715379+0000","last_change":"2023-10-26T05:48:23.715244+0000","last_active":"2023-10-26T05:48:23.715379+0000","last_peered":"2023-10-26T05:48:23.715379+0000","last_clean":"2023-10-26T05:48:23.715379+0000","last_became_active":"2023-10-26T05:48:23.713621+0000","last_became_peered":"2023-10-26T05:48:23.713621+0000","last_unstale":"2023-10-26T05:48:23.715379+0000","last_undegraded":"2023-10-26T05:48:23.715379+0000","last_fullsized":"2023-10-26T05:48:23.715379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:52.471566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691733+0000","last_change":"2023-10-26T05:48:23.691733+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691733+0000","last_undegraded":"2023-10-26T05:48:23.691733+0000","last_fullsized":"2023-10-26T05:48:23.691733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.704334+0000","last_change":"2023-10-26T05:48:23.704259+0000","last_active":"2023-10-26T05:48:23.704334+0000","last_peered":"2023-10-26T05:48:23.704334+0000","last_clean":"2023-10-26T05:48:23.704334+0000","last_became_active":"2023-10-26T05:48:23.703997+0000","last_became_peered":"2023-10-26T05:48:23.703997+0000","last_unstale":"2023-10-26T05:48:23.704334+0000","last_undegraded":"2023-10-26T05:48:23.704334+0000","last_fullsized":"2023-10-26T05:48:23.704334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694411+0000","last_change":"2023-10-26T05:48:23.694411+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694411+0000","last_undegraded":"2023-10-26T05:48:23.694411+0000","last_fullsized":"2023-10-26T05:48:23.694411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.706547+0000","last_change":"2023-10-26T05:48:23.706487+0000","last_active":"2023-10-26T05:48:23.706547+0000","last_peered":"2023-10-26T05:48:23.706547+0000","last_clean":"2023-10-26T05:48:23.706547+0000","last_became_active":"2023-10-26T05:48:23.706232+0000","last_became_peered":"2023-10-26T05:48:23.706232+0000","last_unstale":"2023-10-26T05:48:23.706547+0000","last_undegraded":"2023-10-26T05:48:23.706547+0000","last_fullsized":"2023-10-26T05:48:23.706547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696971+0000","last_change":"2023-10-26T05:48:23.696971+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696971+0000","last_undegraded":"2023-10-26T05:48:23.696971+0000","last_fullsized":"2023-10-26T05:48:23.696971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.689358+0000","last_change":"2023-10-26T05:48:23.688956+0000","last_active":"2023-10-26T05:48:23.689358+0000","last_peered":"2023-10-26T05:48:23.689358+0000","last_clean":"2023-10-26T05:48:23.689358+0000","last_became_active":"2023-10-26T05:48:23.688311+0000","last_became_peered":"2023-10-26T05:48:23.688311+0000","last_unstale":"2023-10-26T05:48:23.689358+0000","last_undegraded":"2023-10-26T05:48:23.689358+0000","last_fullsized":"2023-10-26T05:48:23.689358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:52.984869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687957+0000","last_change":"2023-10-26T05:48:23.687957+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687957+0000","last_undegraded":"2023-10-26T05:48:23.687957+0000","last_fullsized":"2023-10-26T05:48:23.687957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.688670+0000","last_change":"2023-10-26T05:48:23.688351+0000","last_active":"2023-10-26T05:48:23.688670+0000","last_peered":"2023-10-26T05:48:23.688670+0000","last_clean":"2023-10-26T05:48:23.688670+0000","last_became_active":"2023-10-26T05:48:23.687015+0000","last_became_peered":"2023-10-26T05:48:23.687015+0000","last_unstale":"2023-10-26T05:48:23.688670+0000","last_undegraded":"2023-10-26T05:48:23.688670+0000","last_fullsized":"2023-10-26T05:48:23.688670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.689977+0000","last_change":"2023-10-26T05:48:23.689977+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.689977+0000","last_undegraded":"2023-10-26T05:48:23.689977+0000","last_fullsized":"2023-10-26T05:48:23.689977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715055+0000","last_change":"2023-10-26T05:48:23.714853+0000","last_active":"2023-10-26T05:48:23.715055+0000","last_peered":"2023-10-26T05:48:23.715055+0000","last_clean":"2023-10-26T05:48:23.715055+0000","last_became_active":"2023-10-26T05:48:23.712919+0000","last_became_peered":"2023-10-26T05:48:23.712919+0000","last_unstale":"2023-10-26T05:48:23.715055+0000","last_undegraded":"2023-10-26T05:48:23.715055+0000","last_fullsized":"2023-10-26T05:48:23.715055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:48.260866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687100+0000","last_change":"2023-10-26T05:48:23.687100+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687100+0000","last_undegraded":"2023-10-26T05:48:23.687100+0000","last_fullsized":"2023-10-26T05:48:23.687100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703330+0000","last_change":"2023-10-26T05:48:23.703179+0000","last_active":"2023-10-26T05:48:23.703330+0000","last_peered":"2023-10-26T05:48:23.703330+0000","last_clean":"2023-10-26T05:48:23.703330+0000","last_became_active":"2023-10-26T05:48:23.702603+0000","last_became_peered":"2023-10-26T05:48:23.702603+0000","last_unstale":"2023-10-26T05:48:23.703330+0000","last_undegraded":"2023-10-26T05:48:23.703330+0000","last_fullsized":"2023-10-26T05:48:23.703330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:10.226049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694579+0000","last_change":"2023-10-26T05:48:23.694579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694579+0000","last_undegraded":"2023-10-26T05:48:23.694579+0000","last_fullsized":"2023-10-26T05:48:23.694579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691086+0000","last_change":"2023-10-26T05:48:23.690214+0000","last_active":"2023-10-26T05:48:23.691086+0000","last_peered":"2023-10-26T05:48:23.691086+0000","last_clean":"2023-10-26T05:48:23.691086+0000","last_became_active":"2023-10-26T05:48:23.688325+0000","last_became_peered":"2023-10-26T05:48:23.688325+0000","last_unstale":"2023-10-26T05:48:23.691086+0000","last_undegraded":"2023-10-26T05:48:23.691086+0000","last_fullsized":"2023-10-26T05:48:23.691086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:55.310630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694075+0000","last_change":"2023-10-26T05:48:23.694075+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694075+0000","last_undegraded":"2023-10-26T05:48:23.694075+0000","last_fullsized":"2023-10-26T05:48:23.694075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692061+0000","last_change":"2023-10-26T05:48:23.691997+0000","last_active":"2023-10-26T05:48:23.692061+0000","last_peered":"2023-10-26T05:48:23.692061+0000","last_clean":"2023-10-26T05:48:23.692061+0000","last_became_active":"2023-10-26T05:48:23.691414+0000","last_became_peered":"2023-10-26T05:48:23.691414+0000","last_unstale":"2023-10-26T05:48:23.692061+0000","last_undegraded":"2023-10-26T05:48:23.692061+0000","last_fullsized":"2023-10-26T05:48:23.692061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695252+0000","last_change":"2023-10-26T05:48:23.695252+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695252+0000","last_undegraded":"2023-10-26T05:48:23.695252+0000","last_fullsized":"2023-10-26T05:48:23.695252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705798+0000","last_change":"2023-10-26T05:48:23.705656+0000","last_active":"2023-10-26T05:48:23.705798+0000","last_peered":"2023-10-26T05:48:23.705798+0000","last_clean":"2023-10-26T05:48:23.705798+0000","last_became_active":"2023-10-26T05:48:23.705077+0000","last_became_peered":"2023-10-26T05:48:23.705077+0000","last_unstale":"2023-10-26T05:48:23.705798+0000","last_undegraded":"2023-10-26T05:48:23.705798+0000","last_fullsized":"2023-10-26T05:48:23.705798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694168+0000","last_change":"2023-10-26T05:48:23.694168+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694168+0000","last_undegraded":"2023-10-26T05:48:23.694168+0000","last_fullsized":"2023-10-26T05:48:23.694168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712295+0000","last_change":"2023-10-26T05:48:23.712097+0000","last_active":"2023-10-26T05:48:23.712295+0000","last_peered":"2023-10-26T05:48:23.712295+0000","last_clean":"2023-10-26T05:48:23.712295+0000","last_became_active":"2023-10-26T05:48:23.704476+0000","last_became_peered":"2023-10-26T05:48:23.704476+0000","last_unstale":"2023-10-26T05:48:23.712295+0000","last_undegraded":"2023-10-26T05:48:23.712295+0000","last_fullsized":"2023-10-26T05:48:23.712295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:04.497346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693873+0000","last_change":"2023-10-26T05:48:23.693873+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693873+0000","last_undegraded":"2023-10-26T05:48:23.693873+0000","last_fullsized":"2023-10-26T05:48:23.693873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.708766+0000","last_change":"2023-10-26T05:48:23.708587+0000","last_active":"2023-10-26T05:48:23.708766+0000","last_peered":"2023-10-26T05:48:23.708766+0000","last_clean":"2023-10-26T05:48:23.708766+0000","last_became_active":"2023-10-26T05:48:23.707969+0000","last_became_peered":"2023-10-26T05:48:23.707969+0000","last_unstale":"2023-10-26T05:48:23.708766+0000","last_undegraded":"2023-10-26T05:48:23.708766+0000","last_fullsized":"2023-10-26T05:48:23.708766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694854+0000","last_change":"2023-10-26T05:48:23.694854+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694854+0000","last_undegraded":"2023-10-26T05:48:23.694854+0000","last_fullsized":"2023-10-26T05:48:23.694854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705560+0000","last_change":"2023-10-26T05:48:23.705491+0000","last_active":"2023-10-26T05:48:23.705560+0000","last_peered":"2023-10-26T05:48:23.705560+0000","last_clean":"2023-10-26T05:48:23.705560+0000","last_became_active":"2023-10-26T05:48:23.705095+0000","last_became_peered":"2023-10-26T05:48:23.705095+0000","last_unstale":"2023-10-26T05:48:23.705560+0000","last_undegraded":"2023-10-26T05:48:23.705560+0000","last_fullsized":"2023-10-26T05:48:23.705560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693231+0000","last_change":"2023-10-26T05:48:23.693231+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693231+0000","last_undegraded":"2023-10-26T05:48:23.693231+0000","last_fullsized":"2023-10-26T05:48:23.693231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692208+0000","last_change":"2023-10-26T05:48:23.692126+0000","last_active":"2023-10-26T05:48:23.692208+0000","last_peered":"2023-10-26T05:48:23.692208+0000","last_clean":"2023-10-26T05:48:23.692208+0000","last_became_active":"2023-10-26T05:48:23.691722+0000","last_became_peered":"2023-10-26T05:48:23.691722+0000","last_unstale":"2023-10-26T05:48:23.692208+0000","last_undegraded":"2023-10-26T05:48:23.692208+0000","last_fullsized":"2023-10-26T05:48:23.692208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688037+0000","last_change":"2023-10-26T05:48:23.688037+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688037+0000","last_undegraded":"2023-10-26T05:48:23.688037+0000","last_fullsized":"2023-10-26T05:48:23.688037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.699555+0000","last_change":"2023-10-26T05:48:23.699301+0000","last_active":"2023-10-26T05:48:23.699555+0000","last_peered":"2023-10-26T05:48:23.699555+0000","last_clean":"2023-10-26T05:48:23.699555+0000","last_became_active":"2023-10-26T05:48:23.698926+0000","last_became_peered":"2023-10-26T05:48:23.698926+0000","last_unstale":"2023-10-26T05:48:23.699555+0000","last_undegraded":"2023-10-26T05:48:23.699555+0000","last_fullsized":"2023-10-26T05:48:23.699555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696097+0000","last_change":"2023-10-26T05:48:23.696097+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696097+0000","last_undegraded":"2023-10-26T05:48:23.696097+0000","last_fullsized":"2023-10-26T05:48:23.696097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.697196+0000","last_change":"2023-10-26T05:48:23.697103+0000","last_active":"2023-10-26T05:48:23.697196+0000","last_peered":"2023-10-26T05:48:23.697196+0000","last_clean":"2023-10-26T05:48:23.697196+0000","last_became_active":"2023-10-26T05:48:23.696770+0000","last_became_peered":"2023-10-26T05:48:23.696770+0000","last_unstale":"2023-10-26T05:48:23.697196+0000","last_undegraded":"2023-10-26T05:48:23.697196+0000","last_fullsized":"2023-10-26T05:48:23.697196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690554+0000","last_change":"2023-10-26T05:48:23.690554+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690554+0000","last_undegraded":"2023-10-26T05:48:23.690554+0000","last_fullsized":"2023-10-26T05:48:23.690554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705457+0000","last_change":"2023-10-26T05:48:23.705357+0000","last_active":"2023-10-26T05:48:23.705457+0000","last_peered":"2023-10-26T05:48:23.705457+0000","last_clean":"2023-10-26T05:48:23.705457+0000","last_became_active":"2023-10-26T05:48:23.704629+0000","last_became_peered":"2023-10-26T05:48:23.704629+0000","last_unstale":"2023-10-26T05:48:23.705457+0000","last_undegraded":"2023-10-26T05:48:23.705457+0000","last_fullsized":"2023-10-26T05:48:23.705457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690693+0000","last_change":"2023-10-26T05:48:23.690693+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690693+0000","last_undegraded":"2023-10-26T05:48:23.690693+0000","last_fullsized":"2023-10-26T05:48:23.690693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701672+0000","last_change":"2023-10-26T05:48:23.701546+0000","last_active":"2023-10-26T05:48:23.701672+0000","last_peered":"2023-10-26T05:48:23.701672+0000","last_clean":"2023-10-26T05:48:23.701672+0000","last_became_active":"2023-10-26T05:48:23.700654+0000","last_became_peered":"2023-10-26T05:48:23.700654+0000","last_unstale":"2023-10-26T05:48:23.701672+0000","last_undegraded":"2023-10-26T05:48:23.701672+0000","last_fullsized":"2023-10-26T05:48:23.701672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.684207+0000","last_change":"2023-10-26T05:48:23.684207+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.684207+0000","last_undegraded":"2023-10-26T05:48:23.684207+0000","last_fullsized":"2023-10-26T05:48:23.684207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703737+0000","last_change":"2023-10-26T05:48:23.703600+0000","last_active":"2023-10-26T05:48:23.703737+0000","last_peered":"2023-10-26T05:48:23.703737+0000","last_clean":"2023-10-26T05:48:23.703737+0000","last_became_active":"2023-10-26T05:48:23.703254+0000","last_became_peered":"2023-10-26T05:48:23.703254+0000","last_unstale":"2023-10-26T05:48:23.703737+0000","last_undegraded":"2023-10-26T05:48:23.703737+0000","last_fullsized":"2023-10-26T05:48:23.703737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:35.361883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692904+0000","last_change":"2023-10-26T05:48:23.692904+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692904+0000","last_undegraded":"2023-10-26T05:48:23.692904+0000","last_fullsized":"2023-10-26T05:48:23.692904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712540+0000","last_change":"2023-10-26T05:48:23.712299+0000","last_active":"2023-10-26T05:48:23.712540+0000","last_peered":"2023-10-26T05:48:23.712540+0000","last_clean":"2023-10-26T05:48:23.712540+0000","last_became_active":"2023-10-26T05:48:23.711619+0000","last_became_peered":"2023-10-26T05:48:23.711619+0000","last_unstale":"2023-10-26T05:48:23.712540+0000","last_undegraded":"2023-10-26T05:48:23.712540+0000","last_fullsized":"2023-10-26T05:48:23.712540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-26T05:48:22.672770+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:48:22.672770+0000","last_peered":"2023-10-26T05:48:22.672770+0000","last_clean":"2023-10-26T05:48:22.672770+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:48:22.672770+0000","last_undegraded":"2023-10-26T05:48:22.672770+0000","last_fullsized":"2023-10-26T05:48:22.672770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194606,"statfs":{"total":200000000,"available":199276907,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.022},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.05}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.147},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.292},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2390000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.879}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.0409999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.0700000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.622},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.196}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.087}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.012},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.159}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.751},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.24}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.752},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.036}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.77800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.99099999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.083},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.871}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270640,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.73799999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:48:27.685 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:48:27.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/224978884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6be8003980 msgr2=0x7f6be8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 --2- 172.21.15.71:0/224978884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6be8003980 0x7f6be8003da0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005fd0 tx=0x7f6bec01cd10 comp rx=0 tx=0).stop 2023-10-26T05:48:27.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfb59e640 1 -- 172.21.15.71:0/224978884 reap_dead start 2023-10-26T05:48:27.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/224978884 shutdown_connections 2023-10-26T05:48:27.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/224978884 >> 172.21.15.71:0/224978884 conn(0x7f6bfc069770 msgr2=0x7f6bfc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:27.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/224978884 shutdown_connections 2023-10-26T05:48:27.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/224978884 wait complete. 2023-10-26T05:48:27.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 Processor -- start 2023-10-26T05:48:27.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- start start 2023-10-26T05:48:27.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 0x7f6bfc13be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bfc15bca0 0x7f6bfc159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bfc15a1b0 0x7f6bfc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc15a630 con 0x7f6bfc157c80 2023-10-26T05:48:27.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc132150 con 0x7f6bfc15a1b0 2023-10-26T05:48:27.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc0ad750 con 0x7f6bfc15bca0 2023-10-26T05:48:27.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 0x7f6bfc13be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 0x7f6bfc13be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58388/0 (socket says 172.21.15.71:58388) 2023-10-26T05:48:27.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 -- 172.21.15.71:0/663636488 learned_addr learned my addr 172.21.15.71:0/663636488 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:27.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfb59e640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bfc15a1b0 0x7f6bfc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bf259c640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bfc15bca0 0x7f6bfc159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 -- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bfc15bca0 msgr2=0x7f6bfc159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bfc15bca0 0x7f6bfc159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 -- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bfc15a1b0 msgr2=0x7f6bfc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bfc15a1b0 0x7f6bfc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 -- 172.21.15.71:0/663636488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc0ada30 con 0x7f6bfc157c80 2023-10-26T05:48:27.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bfad9d640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 0x7f6bfc13be50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0b1040 tx=0x7f6bec005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0be070 con 0x7f6bfc157c80 2023-10-26T05:48:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6bec0053e0 con 0x7f6bfc157c80 2023-10-26T05:48:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc0ae180 con 0x7f6bfc157c80 2023-10-26T05:48:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec004020 con 0x7f6bfc157c80 2023-10-26T05:48:27.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.841+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc0add10 con 0x7f6bfc157c80 2023-10-26T05:48:27.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.845+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc066b10 con 0x7f6bfc157c80 2023-10-26T05:48:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.845+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6bec0041a0 con 0x7f6bfc157c80 2023-10-26T05:48:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.845+0000 7f6bf3fff640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bdc0664a0 0x7f6bdc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.845+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bec11e8a0 con 0x7f6bfc157c80 2023-10-26T05:48:27.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.845+0000 7f6bf259c640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bdc0664a0 0x7f6bdc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:27.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.845+0000 7f6bf259c640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bdc0664a0 0x7f6bdc068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005bd0 tx=0x7f6be4005a80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:27.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.849+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6bec0ebd00 con 0x7f6bfc157c80 2023-10-26T05:48:27.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:27.989+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6bfc141260 con 0x7f6bdc0664a0 2023-10-26T05:48:28.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.009+0000 7f6bf3fff640 1 -- 172.21.15.71:0/663636488 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278482 (secure 0 0 0) 0x7f6be4005e80 con 0x7f6bdc0664a0 2023-10-26T05:48:28.017 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:28.019 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:48:28.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bdc0664a0 msgr2=0x7f6bdc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bdc0664a0 0x7f6bdc068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005bd0 tx=0x7f6be4005a80 comp rx=0 tx=0).stop 2023-10-26T05:48:28.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 msgr2=0x7f6bfc13be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 0x7f6bfc13be50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0b1040 tx=0x7f6bec005040 comp rx=0 tx=0).stop 2023-10-26T05:48:28.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 shutdown_connections 2023-10-26T05:48:28.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bfc15a1b0 0x7f6bfc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6bfc15bca0 0x7f6bfc159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bdc0664a0 0x7f6bdc068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 --2- 172.21.15.71:0/663636488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc157c80 0x7f6bfc13be50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 >> 172.21.15.71:0/663636488 conn(0x7f6bfc069770 msgr2=0x7f6bfc071440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:28.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 shutdown_connections 2023-10-26T05:48:28.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.013+0000 7f6c01009640 1 -- 172.21.15.71:0/663636488 wait complete. 2023-10-26T05:48:28.034 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-26T05:48:27.433648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560291,"statfs":{"total":1600000000,"available":1597738663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700234+0000","last_change":"2023-10-26T05:48:23.700234+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700234+0000","last_undegraded":"2023-10-26T05:48:23.700234+0000","last_fullsized":"2023-10-26T05:48:23.700234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694918+0000","last_change":"2023-10-26T05:48:23.694918+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694918+0000","last_undegraded":"2023-10-26T05:48:23.694918+0000","last_fullsized":"2023-10-26T05:48:23.694918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695709+0000","last_change":"2023-10-26T05:48:23.695709+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695709+0000","last_undegraded":"2023-10-26T05:48:23.695709+0000","last_fullsized":"2023-10-26T05:48:23.695709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688038+0000","last_change":"2023-10-26T05:48:23.688038+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688038+0000","last_undegraded":"2023-10-26T05:48:23.688038+0000","last_fullsized":"2023-10-26T05:48:23.688038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694048+0000","last_change":"2023-10-26T05:48:23.694048+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694048+0000","last_undegraded":"2023-10-26T05:48:23.694048+0000","last_fullsized":"2023-10-26T05:48:23.694048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694220+0000","last_change":"2023-10-26T05:48:23.694220+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694220+0000","last_undegraded":"2023-10-26T05:48:23.694220+0000","last_fullsized":"2023-10-26T05:48:23.694220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696282+0000","last_change":"2023-10-26T05:48:23.696282+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696282+0000","last_undegraded":"2023-10-26T05:48:23.696282+0000","last_fullsized":"2023-10-26T05:48:23.696282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700861+0000","last_change":"2023-10-26T05:48:23.700861+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700861+0000","last_undegraded":"2023-10-26T05:48:23.700861+0000","last_fullsized":"2023-10-26T05:48:23.700861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687341+0000","last_change":"2023-10-26T05:48:23.687341+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687341+0000","last_undegraded":"2023-10-26T05:48:23.687341+0000","last_fullsized":"2023-10-26T05:48:23.687341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688537+0000","last_change":"2023-10-26T05:48:23.688537+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688537+0000","last_undegraded":"2023-10-26T05:48:23.688537+0000","last_fullsized":"2023-10-26T05:48:23.688537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.697056+0000","last_change":"2023-10-26T05:48:23.697056+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.697056+0000","last_undegraded":"2023-10-26T05:48:23.697056+0000","last_fullsized":"2023-10-26T05:48:23.697056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694710+0000","last_change":"2023-10-26T05:48:23.694710+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694710+0000","last_undegraded":"2023-10-26T05:48:23.694710+0000","last_fullsized":"2023-10-26T05:48:23.694710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695640+0000","last_change":"2023-10-26T05:48:23.695640+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695640+0000","last_undegraded":"2023-10-26T05:48:23.695640+0000","last_fullsized":"2023-10-26T05:48:23.695640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692044+0000","last_change":"2023-10-26T05:48:23.692044+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692044+0000","last_undegraded":"2023-10-26T05:48:23.692044+0000","last_fullsized":"2023-10-26T05:48:23.692044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692542+0000","last_change":"2023-10-26T05:48:23.692542+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692542+0000","last_undegraded":"2023-10-26T05:48:23.692542+0000","last_fullsized":"2023-10-26T05:48:23.692542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696259+0000","last_change":"2023-10-26T05:48:23.696259+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696259+0000","last_undegraded":"2023-10-26T05:48:23.696259+0000","last_fullsized":"2023-10-26T05:48:23.696259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.706851+0000","last_change":"2023-10-26T05:48:23.706851+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.706851+0000","last_undegraded":"2023-10-26T05:48:23.706851+0000","last_fullsized":"2023-10-26T05:48:23.706851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695326+0000","last_change":"2023-10-26T05:48:23.695326+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695326+0000","last_undegraded":"2023-10-26T05:48:23.695326+0000","last_fullsized":"2023-10-26T05:48:23.695326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691624+0000","last_change":"2023-10-26T05:48:23.691624+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691624+0000","last_undegraded":"2023-10-26T05:48:23.691624+0000","last_fullsized":"2023-10-26T05:48:23.691624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693579+0000","last_change":"2023-10-26T05:48:23.693579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693579+0000","last_undegraded":"2023-10-26T05:48:23.693579+0000","last_fullsized":"2023-10-26T05:48:23.693579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685254+0000","last_change":"2023-10-26T05:48:23.685254+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685254+0000","last_undegraded":"2023-10-26T05:48:23.685254+0000","last_fullsized":"2023-10-26T05:48:23.685254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685503+0000","last_change":"2023-10-26T05:48:23.685503+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685503+0000","last_undegraded":"2023-10-26T05:48:23.685503+0000","last_fullsized":"2023-10-26T05:48:23.685503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691108+0000","last_change":"2023-10-26T05:48:23.691108+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691108+0000","last_undegraded":"2023-10-26T05:48:23.691108+0000","last_fullsized":"2023-10-26T05:48:23.691108+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690897+0000","last_change":"2023-10-26T05:48:23.690897+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690897+0000","last_undegraded":"2023-10-26T05:48:23.690897+0000","last_fullsized":"2023-10-26T05:48:23.690897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688241+0000","last_change":"2023-10-26T05:48:23.688241+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688241+0000","last_undegraded":"2023-10-26T05:48:23.688241+0000","last_fullsized":"2023-10-26T05:48:23.688241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692899+0000","last_change":"2023-10-26T05:48:23.692899+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692899+0000","last_undegraded":"2023-10-26T05:48:23.692899+0000","last_fullsized":"2023-10-26T05:48:23.692899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687469+0000","last_change":"2023-10-26T05:48:23.687469+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687469+0000","last_undegraded":"2023-10-26T05:48:23.687469+0000","last_fullsized":"2023-10-26T05:48:23.687469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693430+0000","last_change":"2023-10-26T05:48:23.693430+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693430+0000","last_undegraded":"2023-10-26T05:48:23.693430+0000","last_fullsized":"2023-10-26T05:48:23.693430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.686546+0000","last_change":"2023-10-26T05:48:23.686546+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.686546+0000","last_undegraded":"2023-10-26T05:48:23.686546+0000","last_fullsized":"2023-10-26T05:48:23.686546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694718+0000","last_change":"2023-10-26T05:48:23.694718+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694718+0000","last_undegraded":"2023-10-26T05:48:23.694718+0000","last_fullsized":"2023-10-26T05:48:23.694718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.699413+0000","last_change":"2023-10-26T05:48:23.699413+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.699413+0000","last_undegraded":"2023-10-26T05:48:23.699413+0000","last_fullsized":"2023-10-26T05:48:23.699413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.703535+0000","last_change":"2023-10-26T05:48:23.703535+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.703535+0000","last_undegraded":"2023-10-26T05:48:23.703535+0000","last_fullsized":"2023-10-26T05:48:23.703535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688409+0000","last_change":"2023-10-26T05:48:23.688409+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688409+0000","last_undegraded":"2023-10-26T05:48:23.688409+0000","last_fullsized":"2023-10-26T05:48:23.688409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.711284+0000","last_change":"2023-10-26T05:48:23.711070+0000","last_active":"2023-10-26T05:48:23.711284+0000","last_peered":"2023-10-26T05:48:23.711284+0000","last_clean":"2023-10-26T05:48:23.711284+0000","last_became_active":"2023-10-26T05:48:23.710421+0000","last_became_peered":"2023-10-26T05:48:23.710421+0000","last_unstale":"2023-10-26T05:48:23.711284+0000","last_undegraded":"2023-10-26T05:48:23.711284+0000","last_fullsized":"2023-10-26T05:48:23.711284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695820+0000","last_change":"2023-10-26T05:48:23.695820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695820+0000","last_undegraded":"2023-10-26T05:48:23.695820+0000","last_fullsized":"2023-10-26T05:48:23.695820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703916+0000","last_change":"2023-10-26T05:48:23.703810+0000","last_active":"2023-10-26T05:48:23.703916+0000","last_peered":"2023-10-26T05:48:23.703916+0000","last_clean":"2023-10-26T05:48:23.703916+0000","last_became_active":"2023-10-26T05:48:23.703500+0000","last_became_peered":"2023-10-26T05:48:23.703500+0000","last_unstale":"2023-10-26T05:48:23.703916+0000","last_undegraded":"2023-10-26T05:48:23.703916+0000","last_fullsized":"2023-10-26T05:48:23.703916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.683637+0000","last_change":"2023-10-26T05:48:23.683637+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.683637+0000","last_undegraded":"2023-10-26T05:48:23.683637+0000","last_fullsized":"2023-10-26T05:48:23.683637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690187+0000","last_change":"2023-10-26T05:48:23.690022+0000","last_active":"2023-10-26T05:48:23.690187+0000","last_peered":"2023-10-26T05:48:23.690187+0000","last_clean":"2023-10-26T05:48:23.690187+0000","last_became_active":"2023-10-26T05:48:23.689605+0000","last_became_peered":"2023-10-26T05:48:23.689605+0000","last_unstale":"2023-10-26T05:48:23.690187+0000","last_undegraded":"2023-10-26T05:48:23.690187+0000","last_fullsized":"2023-10-26T05:48:23.690187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687820+0000","last_change":"2023-10-26T05:48:23.687820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687820+0000","last_undegraded":"2023-10-26T05:48:23.687820+0000","last_fullsized":"2023-10-26T05:48:23.687820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701416+0000","last_change":"2023-10-26T05:48:23.701287+0000","last_active":"2023-10-26T05:48:23.701416+0000","last_peered":"2023-10-26T05:48:23.701416+0000","last_clean":"2023-10-26T05:48:23.701416+0000","last_became_active":"2023-10-26T05:48:23.700949+0000","last_became_peered":"2023-10-26T05:48:23.700949+0000","last_unstale":"2023-10-26T05:48:23.701416+0000","last_undegraded":"2023-10-26T05:48:23.701416+0000","last_fullsized":"2023-10-26T05:48:23.701416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700105+0000","last_change":"2023-10-26T05:48:23.700105+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700105+0000","last_undegraded":"2023-10-26T05:48:23.700105+0000","last_fullsized":"2023-10-26T05:48:23.700105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691137+0000","last_change":"2023-10-26T05:48:23.690572+0000","last_active":"2023-10-26T05:48:23.691137+0000","last_peered":"2023-10-26T05:48:23.691137+0000","last_clean":"2023-10-26T05:48:23.691137+0000","last_became_active":"2023-10-26T05:48:23.689616+0000","last_became_peered":"2023-10-26T05:48:23.689616+0000","last_unstale":"2023-10-26T05:48:23.691137+0000","last_undegraded":"2023-10-26T05:48:23.691137+0000","last_fullsized":"2023-10-26T05:48:23.691137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:36:17.063916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693976+0000","last_change":"2023-10-26T05:48:23.693976+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693976+0000","last_undegraded":"2023-10-26T05:48:23.693976+0000","last_fullsized":"2023-10-26T05:48:23.693976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701389+0000","last_change":"2023-10-26T05:48:23.701241+0000","last_active":"2023-10-26T05:48:23.701389+0000","last_peered":"2023-10-26T05:48:23.701389+0000","last_clean":"2023-10-26T05:48:23.701389+0000","last_became_active":"2023-10-26T05:48:23.699629+0000","last_became_peered":"2023-10-26T05:48:23.699629+0000","last_unstale":"2023-10-26T05:48:23.701389+0000","last_undegraded":"2023-10-26T05:48:23.701389+0000","last_fullsized":"2023-10-26T05:48:23.701389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:24:45.082594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688715+0000","last_change":"2023-10-26T05:48:23.688715+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688715+0000","last_undegraded":"2023-10-26T05:48:23.688715+0000","last_fullsized":"2023-10-26T05:48:23.688715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.700492+0000","last_change":"2023-10-26T05:48:23.700412+0000","last_active":"2023-10-26T05:48:23.700492+0000","last_peered":"2023-10-26T05:48:23.700492+0000","last_clean":"2023-10-26T05:48:23.700492+0000","last_became_active":"2023-10-26T05:48:23.698966+0000","last_became_peered":"2023-10-26T05:48:23.698966+0000","last_unstale":"2023-10-26T05:48:23.700492+0000","last_undegraded":"2023-10-26T05:48:23.700492+0000","last_fullsized":"2023-10-26T05:48:23.700492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687623+0000","last_change":"2023-10-26T05:48:23.687623+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687623+0000","last_undegraded":"2023-10-26T05:48:23.687623+0000","last_fullsized":"2023-10-26T05:48:23.687623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715611+0000","last_change":"2023-10-26T05:48:23.715497+0000","last_active":"2023-10-26T05:48:23.715611+0000","last_peered":"2023-10-26T05:48:23.715611+0000","last_clean":"2023-10-26T05:48:23.715611+0000","last_became_active":"2023-10-26T05:48:23.714277+0000","last_became_peered":"2023-10-26T05:48:23.714277+0000","last_unstale":"2023-10-26T05:48:23.715611+0000","last_undegraded":"2023-10-26T05:48:23.715611+0000","last_fullsized":"2023-10-26T05:48:23.715611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:16:54.930577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691328+0000","last_change":"2023-10-26T05:48:23.691328+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691328+0000","last_undegraded":"2023-10-26T05:48:23.691328+0000","last_fullsized":"2023-10-26T05:48:23.691328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705835+0000","last_change":"2023-10-26T05:48:23.705748+0000","last_active":"2023-10-26T05:48:23.705835+0000","last_peered":"2023-10-26T05:48:23.705835+0000","last_clean":"2023-10-26T05:48:23.705835+0000","last_became_active":"2023-10-26T05:48:23.705389+0000","last_became_peered":"2023-10-26T05:48:23.705389+0000","last_unstale":"2023-10-26T05:48:23.705835+0000","last_undegraded":"2023-10-26T05:48:23.705835+0000","last_fullsized":"2023-10-26T05:48:23.705835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692498+0000","last_change":"2023-10-26T05:48:23.692498+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692498+0000","last_undegraded":"2023-10-26T05:48:23.692498+0000","last_fullsized":"2023-10-26T05:48:23.692498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.696945+0000","last_change":"2023-10-26T05:48:23.696857+0000","last_active":"2023-10-26T05:48:23.696945+0000","last_peered":"2023-10-26T05:48:23.696945+0000","last_clean":"2023-10-26T05:48:23.696945+0000","last_became_active":"2023-10-26T05:48:23.696471+0000","last_became_peered":"2023-10-26T05:48:23.696471+0000","last_unstale":"2023-10-26T05:48:23.696945+0000","last_undegraded":"2023-10-26T05:48:23.696945+0000","last_fullsized":"2023-10-26T05:48:23.696945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691173+0000","last_change":"2023-10-26T05:48:23.691173+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691173+0000","last_undegraded":"2023-10-26T05:48:23.691173+0000","last_fullsized":"2023-10-26T05:48:23.691173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.709783+0000","last_change":"2023-10-26T05:48:23.709670+0000","last_active":"2023-10-26T05:48:23.709783+0000","last_peered":"2023-10-26T05:48:23.709783+0000","last_clean":"2023-10-26T05:48:23.709783+0000","last_became_active":"2023-10-26T05:48:23.709170+0000","last_became_peered":"2023-10-26T05:48:23.709170+0000","last_unstale":"2023-10-26T05:48:23.709783+0000","last_undegraded":"2023-10-26T05:48:23.709783+0000","last_fullsized":"2023-10-26T05:48:23.709783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:03.464765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692408+0000","last_change":"2023-10-26T05:48:23.692408+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692408+0000","last_undegraded":"2023-10-26T05:48:23.692408+0000","last_fullsized":"2023-10-26T05:48:23.692408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690158+0000","last_change":"2023-10-26T05:48:23.689973+0000","last_active":"2023-10-26T05:48:23.690158+0000","last_peered":"2023-10-26T05:48:23.690158+0000","last_clean":"2023-10-26T05:48:23.690158+0000","last_became_active":"2023-10-26T05:48:23.689550+0000","last_became_peered":"2023-10-26T05:48:23.689550+0000","last_unstale":"2023-10-26T05:48:23.690158+0000","last_undegraded":"2023-10-26T05:48:23.690158+0000","last_fullsized":"2023-10-26T05:48:23.690158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.768450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692771+0000","last_change":"2023-10-26T05:48:23.692771+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692771+0000","last_undegraded":"2023-10-26T05:48:23.692771+0000","last_fullsized":"2023-10-26T05:48:23.692771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715379+0000","last_change":"2023-10-26T05:48:23.715244+0000","last_active":"2023-10-26T05:48:23.715379+0000","last_peered":"2023-10-26T05:48:23.715379+0000","last_clean":"2023-10-26T05:48:23.715379+0000","last_became_active":"2023-10-26T05:48:23.713621+0000","last_became_peered":"2023-10-26T05:48:23.713621+0000","last_unstale":"2023-10-26T05:48:23.715379+0000","last_undegraded":"2023-10-26T05:48:23.715379+0000","last_fullsized":"2023-10-26T05:48:23.715379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:52.471566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691733+0000","last_change":"2023-10-26T05:48:23.691733+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691733+0000","last_undegraded":"2023-10-26T05:48:23.691733+0000","last_fullsized":"2023-10-26T05:48:23.691733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.704334+0000","last_change":"2023-10-26T05:48:23.704259+0000","last_active":"2023-10-26T05:48:23.704334+0000","last_peered":"2023-10-26T05:48:23.704334+0000","last_clean":"2023-10-26T05:48:23.704334+0000","last_became_active":"2023-10-26T05:48:23.703997+0000","last_became_peered":"2023-10-26T05:48:23.703997+0000","last_unstale":"2023-10-26T05:48:23.704334+0000","last_undegraded":"2023-10-26T05:48:23.704334+0000","last_fullsized":"2023-10-26T05:48:23.704334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694411+0000","last_change":"2023-10-26T05:48:23.694411+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694411+0000","last_undegraded":"2023-10-26T05:48:23.694411+0000","last_fullsized":"2023-10-26T05:48:23.694411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.706547+0000","last_change":"2023-10-26T05:48:23.706487+0000","last_active":"2023-10-26T05:48:23.706547+0000","last_peered":"2023-10-26T05:48:23.706547+0000","last_clean":"2023-10-26T05:48:23.706547+0000","last_became_active":"2023-10-26T05:48:23.706232+0000","last_became_peered":"2023-10-26T05:48:23.706232+0000","last_unstale":"2023-10-26T05:48:23.706547+0000","last_undegraded":"2023-10-26T05:48:23.706547+0000","last_fullsized":"2023-10-26T05:48:23.706547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696971+0000","last_change":"2023-10-26T05:48:23.696971+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696971+0000","last_undegraded":"2023-10-26T05:48:23.696971+0000","last_fullsized":"2023-10-26T05:48:23.696971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.689358+0000","last_change":"2023-10-26T05:48:23.688956+0000","last_active":"2023-10-26T05:48:23.689358+0000","last_peered":"2023-10-26T05:48:23.689358+0000","last_clean":"2023-10-26T05:48:23.689358+0000","last_became_active":"2023-10-26T05:48:23.688311+0000","last_became_peered":"2023-10-26T05:48:23.688311+0000","last_unstale":"2023-10-26T05:48:23.689358+0000","last_undegraded":"2023-10-26T05:48:23.689358+0000","last_fullsized":"2023-10-26T05:48:23.689358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:52.984869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687957+0000","last_change":"2023-10-26T05:48:23.687957+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687957+0000","last_undegraded":"2023-10-26T05:48:23.687957+0000","last_fullsized":"2023-10-26T05:48:23.687957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.688670+0000","last_change":"2023-10-26T05:48:23.688351+0000","last_active":"2023-10-26T05:48:23.688670+0000","last_peered":"2023-10-26T05:48:23.688670+0000","last_clean":"2023-10-26T05:48:23.688670+0000","last_became_active":"2023-10-26T05:48:23.687015+0000","last_became_peered":"2023-10-26T05:48:23.687015+0000","last_unstale":"2023-10-26T05:48:23.688670+0000","last_undegraded":"2023-10-26T05:48:23.688670+0000","last_fullsized":"2023-10-26T05:48:23.688670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.689977+0000","last_change":"2023-10-26T05:48:23.689977+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.689977+0000","last_undegraded":"2023-10-26T05:48:23.689977+0000","last_fullsized":"2023-10-26T05:48:23.689977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715055+0000","last_change":"2023-10-26T05:48:23.714853+0000","last_active":"2023-10-26T05:48:23.715055+0000","last_peered":"2023-10-26T05:48:23.715055+0000","last_clean":"2023-10-26T05:48:23.715055+0000","last_became_active":"2023-10-26T05:48:23.712919+0000","last_became_peered":"2023-10-26T05:48:23.712919+0000","last_unstale":"2023-10-26T05:48:23.715055+0000","last_undegraded":"2023-10-26T05:48:23.715055+0000","last_fullsized":"2023-10-26T05:48:23.715055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:48.260866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687100+0000","last_change":"2023-10-26T05:48:23.687100+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687100+0000","last_undegraded":"2023-10-26T05:48:23.687100+0000","last_fullsized":"2023-10-26T05:48:23.687100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703330+0000","last_change":"2023-10-26T05:48:23.703179+0000","last_active":"2023-10-26T05:48:23.703330+0000","last_peered":"2023-10-26T05:48:23.703330+0000","last_clean":"2023-10-26T05:48:23.703330+0000","last_became_active":"2023-10-26T05:48:23.702603+0000","last_became_peered":"2023-10-26T05:48:23.702603+0000","last_unstale":"2023-10-26T05:48:23.703330+0000","last_undegraded":"2023-10-26T05:48:23.703330+0000","last_fullsized":"2023-10-26T05:48:23.703330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:10.226049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694579+0000","last_change":"2023-10-26T05:48:23.694579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694579+0000","last_undegraded":"2023-10-26T05:48:23.694579+0000","last_fullsized":"2023-10-26T05:48:23.694579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691086+0000","last_change":"2023-10-26T05:48:23.690214+0000","last_active":"2023-10-26T05:48:23.691086+0000","last_peered":"2023-10-26T05:48:23.691086+0000","last_clean":"2023-10-26T05:48:23.691086+0000","last_became_active":"2023-10-26T05:48:23.688325+0000","last_became_peered":"2023-10-26T05:48:23.688325+0000","last_unstale":"2023-10-26T05:48:23.691086+0000","last_undegraded":"2023-10-26T05:48:23.691086+0000","last_fullsized":"2023-10-26T05:48:23.691086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:55.310630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694075+0000","last_change":"2023-10-26T05:48:23.694075+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694075+0000","last_undegraded":"2023-10-26T05:48:23.694075+0000","last_fullsized":"2023-10-26T05:48:23.694075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692061+0000","last_change":"2023-10-26T05:48:23.691997+0000","last_active":"2023-10-26T05:48:23.692061+0000","last_peered":"2023-10-26T05:48:23.692061+0000","last_clean":"2023-10-26T05:48:23.692061+0000","last_became_active":"2023-10-26T05:48:23.691414+0000","last_became_peered":"2023-10-26T05:48:23.691414+0000","last_unstale":"2023-10-26T05:48:23.692061+0000","last_undegraded":"2023-10-26T05:48:23.692061+0000","last_fullsized":"2023-10-26T05:48:23.692061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695252+0000","last_change":"2023-10-26T05:48:23.695252+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695252+0000","last_undegraded":"2023-10-26T05:48:23.695252+0000","last_fullsized":"2023-10-26T05:48:23.695252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705798+0000","last_change":"2023-10-26T05:48:23.705656+0000","last_active":"2023-10-26T05:48:23.705798+0000","last_peered":"2023-10-26T05:48:23.705798+0000","last_clean":"2023-10-26T05:48:23.705798+0000","last_became_active":"2023-10-26T05:48:23.705077+0000","last_became_peered":"2023-10-26T05:48:23.705077+0000","last_unstale":"2023-10-26T05:48:23.705798+0000","last_undegraded":"2023-10-26T05:48:23.705798+0000","last_fullsized":"2023-10-26T05:48:23.705798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694168+0000","last_change":"2023-10-26T05:48:23.694168+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694168+0000","last_undegraded":"2023-10-26T05:48:23.694168+0000","last_fullsized":"2023-10-26T05:48:23.694168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712295+0000","last_change":"2023-10-26T05:48:23.712097+0000","last_active":"2023-10-26T05:48:23.712295+0000","last_peered":"2023-10-26T05:48:23.712295+0000","last_clean":"2023-10-26T05:48:23.712295+0000","last_became_active":"2023-10-26T05:48:23.704476+0000","last_became_peered":"2023-10-26T05:48:23.704476+0000","last_unstale":"2023-10-26T05:48:23.712295+0000","last_undegraded":"2023-10-26T05:48:23.712295+0000","last_fullsized":"2023-10-26T05:48:23.712295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:04.497346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693873+0000","last_change":"2023-10-26T05:48:23.693873+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693873+0000","last_undegraded":"2023-10-26T05:48:23.693873+0000","last_fullsized":"2023-10-26T05:48:23.693873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.708766+0000","last_change":"2023-10-26T05:48:23.708587+0000","last_active":"2023-10-26T05:48:23.708766+0000","last_peered":"2023-10-26T05:48:23.708766+0000","last_clean":"2023-10-26T05:48:23.708766+0000","last_became_active":"2023-10-26T05:48:23.707969+0000","last_became_peered":"2023-10-26T05:48:23.707969+0000","last_unstale":"2023-10-26T05:48:23.708766+0000","last_undegraded":"2023-10-26T05:48:23.708766+0000","last_fullsized":"2023-10-26T05:48:23.708766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694854+0000","last_change":"2023-10-26T05:48:23.694854+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694854+0000","last_undegraded":"2023-10-26T05:48:23.694854+0000","last_fullsized":"2023-10-26T05:48:23.694854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705560+0000","last_change":"2023-10-26T05:48:23.705491+0000","last_active":"2023-10-26T05:48:23.705560+0000","last_peered":"2023-10-26T05:48:23.705560+0000","last_clean":"2023-10-26T05:48:23.705560+0000","last_became_active":"2023-10-26T05:48:23.705095+0000","last_became_peered":"2023-10-26T05:48:23.705095+0000","last_unstale":"2023-10-26T05:48:23.705560+0000","last_undegraded":"2023-10-26T05:48:23.705560+0000","last_fullsized":"2023-10-26T05:48:23.705560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693231+0000","last_change":"2023-10-26T05:48:23.693231+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693231+0000","last_undegraded":"2023-10-26T05:48:23.693231+0000","last_fullsized":"2023-10-26T05:48:23.693231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692208+0000","last_change":"2023-10-26T05:48:23.692126+0000","last_active":"2023-10-26T05:48:23.692208+0000","last_peered":"2023-10-26T05:48:23.692208+0000","last_clean":"2023-10-26T05:48:23.692208+0000","last_became_active":"2023-10-26T05:48:23.691722+0000","last_became_peered":"2023-10-26T05:48:23.691722+0000","last_unstale":"2023-10-26T05:48:23.692208+0000","last_undegraded":"2023-10-26T05:48:23.692208+0000","last_fullsized":"2023-10-26T05:48:23.692208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688037+0000","last_change":"2023-10-26T05:48:23.688037+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688037+0000","last_undegraded":"2023-10-26T05:48:23.688037+0000","last_fullsized":"2023-10-26T05:48:23.688037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.699555+0000","last_change":"2023-10-26T05:48:23.699301+0000","last_active":"2023-10-26T05:48:23.699555+0000","last_peered":"2023-10-26T05:48:23.699555+0000","last_clean":"2023-10-26T05:48:23.699555+0000","last_became_active":"2023-10-26T05:48:23.698926+0000","last_became_peered":"2023-10-26T05:48:23.698926+0000","last_unstale":"2023-10-26T05:48:23.699555+0000","last_undegraded":"2023-10-26T05:48:23.699555+0000","last_fullsized":"2023-10-26T05:48:23.699555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696097+0000","last_change":"2023-10-26T05:48:23.696097+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696097+0000","last_undegraded":"2023-10-26T05:48:23.696097+0000","last_fullsized":"2023-10-26T05:48:23.696097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.697196+0000","last_change":"2023-10-26T05:48:23.697103+0000","last_active":"2023-10-26T05:48:23.697196+0000","last_peered":"2023-10-26T05:48:23.697196+0000","last_clean":"2023-10-26T05:48:23.697196+0000","last_became_active":"2023-10-26T05:48:23.696770+0000","last_became_peered":"2023-10-26T05:48:23.696770+0000","last_unstale":"2023-10-26T05:48:23.697196+0000","last_undegraded":"2023-10-26T05:48:23.697196+0000","last_fullsized":"2023-10-26T05:48:23.697196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690554+0000","last_change":"2023-10-26T05:48:23.690554+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690554+0000","last_undegraded":"2023-10-26T05:48:23.690554+0000","last_fullsized":"2023-10-26T05:48:23.690554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705457+0000","last_change":"2023-10-26T05:48:23.705357+0000","last_active":"2023-10-26T05:48:23.705457+0000","last_peered":"2023-10-26T05:48:23.705457+0000","last_clean":"2023-10-26T05:48:23.705457+0000","last_became_active":"2023-10-26T05:48:23.704629+0000","last_became_peered":"2023-10-26T05:48:23.704629+0000","last_unstale":"2023-10-26T05:48:23.705457+0000","last_undegraded":"2023-10-26T05:48:23.705457+0000","last_fullsized":"2023-10-26T05:48:23.705457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690693+0000","last_change":"2023-10-26T05:48:23.690693+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690693+0000","last_undegraded":"2023-10-26T05:48:23.690693+0000","last_fullsized":"2023-10-26T05:48:23.690693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701672+0000","last_change":"2023-10-26T05:48:23.701546+0000","last_active":"2023-10-26T05:48:23.701672+0000","last_peered":"2023-10-26T05:48:23.701672+0000","last_clean":"2023-10-26T05:48:23.701672+0000","last_became_active":"2023-10-26T05:48:23.700654+0000","last_became_peered":"2023-10-26T05:48:23.700654+0000","last_unstale":"2023-10-26T05:48:23.701672+0000","last_undegraded":"2023-10-26T05:48:23.701672+0000","last_fullsized":"2023-10-26T05:48:23.701672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.684207+0000","last_change":"2023-10-26T05:48:23.684207+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.684207+0000","last_undegraded":"2023-10-26T05:48:23.684207+0000","last_fullsized":"2023-10-26T05:48:23.684207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703737+0000","last_change":"2023-10-26T05:48:23.703600+0000","last_active":"2023-10-26T05:48:23.703737+0000","last_peered":"2023-10-26T05:48:23.703737+0000","last_clean":"2023-10-26T05:48:23.703737+0000","last_became_active":"2023-10-26T05:48:23.703254+0000","last_became_peered":"2023-10-26T05:48:23.703254+0000","last_unstale":"2023-10-26T05:48:23.703737+0000","last_undegraded":"2023-10-26T05:48:23.703737+0000","last_fullsized":"2023-10-26T05:48:23.703737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:35.361883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692904+0000","last_change":"2023-10-26T05:48:23.692904+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692904+0000","last_undegraded":"2023-10-26T05:48:23.692904+0000","last_fullsized":"2023-10-26T05:48:23.692904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712540+0000","last_change":"2023-10-26T05:48:23.712299+0000","last_active":"2023-10-26T05:48:23.712540+0000","last_peered":"2023-10-26T05:48:23.712540+0000","last_clean":"2023-10-26T05:48:23.712540+0000","last_became_active":"2023-10-26T05:48:23.711619+0000","last_became_peered":"2023-10-26T05:48:23.711619+0000","last_unstale":"2023-10-26T05:48:23.712540+0000","last_undegraded":"2023-10-26T05:48:23.712540+0000","last_fullsized":"2023-10-26T05:48:23.712540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-26T05:48:22.672770+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:48:22.672770+0000","last_peered":"2023-10-26T05:48:22.672770+0000","last_clean":"2023-10-26T05:48:22.672770+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:48:22.672770+0000","last_undegraded":"2023-10-26T05:48:22.672770+0000","last_fullsized":"2023-10-26T05:48:22.672770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194606,"statfs":{"total":200000000,"available":199276907,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.022},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.05}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.147},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.292},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2390000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.879}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.0409999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.0700000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.622},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.196}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.087}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.012},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.159}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.751},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.24}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.752},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.036}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.77800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.99099999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.083},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.871}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270640,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.73799999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:48:28.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:48:28.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- 172.21.15.71:0/4040349948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac80b1f10 msgr2=0x7f8ac8141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 --2- 172.21.15.71:0/4040349948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac80b1f10 0x7f8ac8141890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009bc0 tx=0x7f8ac001cb80 comp rx=0 tx=0).stop 2023-10-26T05:48:28.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- 172.21.15.71:0/4040349948 shutdown_connections 2023-10-26T05:48:28.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 --2- 172.21.15.71:0/4040349948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac80b1f10 0x7f8ac8141890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 --2- 172.21.15.71:0/4040349948 >> v2:172.21.15.86:3300/0 conn(0x7f8ac815bc50 0x7f8ac815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- 172.21.15.71:0/4040349948 >> 172.21.15.71:0/4040349948 conn(0x7f8ac80caa90 msgr2=0x7f8ac80caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:28.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- 172.21.15.71:0/4040349948 shutdown_connections 2023-10-26T05:48:28.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- 172.21.15.71:0/4040349948 wait complete. 2023-10-26T05:48:28.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 Processor -- start 2023-10-26T05:48:28.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- start start 2023-10-26T05:48:28.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 0x7f8ac81331a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac80b1f10 0x7f8ac8153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac815bc50 0x7f8ac812e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac8136570 con 0x7f8ac80b1a60 2023-10-26T05:48:28.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac813f460 con 0x7f8ac80b1f10 2023-10-26T05:48:28.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ace664640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac81336e0 con 0x7f8ac815bc50 2023-10-26T05:48:28.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 0x7f8ac81331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 0x7f8ac81331a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58394/0 (socket says 172.21.15.71:58394) 2023-10-26T05:48:28.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.185+0000 7f8ac7fff640 1 -- 172.21.15.71:0/417434689 learned_addr learned my addr 172.21.15.71:0/417434689 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:28.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8abffff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac80b1f10 0x7f8ac8153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8accc01640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac815bc50 0x7f8ac812e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac7fff640 1 -- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac815bc50 msgr2=0x7f8ac812e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac7fff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac815bc50 0x7f8ac812e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac7fff640 1 -- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac80b1f10 msgr2=0x7f8ac8153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac7fff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac80b1f10 0x7f8ac8153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac7fff640 1 -- 172.21.15.71:0/417434689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0009820 con 0x7f8ac80b1a60 2023-10-26T05:48:28.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8abffff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac80b1f10 0x7f8ac8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:28.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac7fff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 0x7f8ac81331a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:28.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8009710 con 0x7f8ac80b1a60 2023-10-26T05:48:28.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ab800ce70 con 0x7f8ac80b1a60 2023-10-26T05:48:28.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac8135fb0 con 0x7f8ac80b1a60 2023-10-26T05:48:28.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8010690 con 0x7f8ac80b1a60 2023-10-26T05:48:28.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac8136340 con 0x7f8ac80b1a60 2023-10-26T05:48:28.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac8066d00 con 0x7f8ac80b1a60 2023-10-26T05:48:28.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8ab80107f0 con 0x7f8ac80b1a60 2023-10-26T05:48:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac57fa640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8aa80664a0 0x7f8aa8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ab8014070 con 0x7f8ac80b1a60 2023-10-26T05:48:28.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8abffff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8aa80664a0 0x7f8aa8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.189+0000 7f8abffff640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8aa80664a0 0x7f8aa8068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ac813bfd0 tx=0x7f8ab0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:28.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.193+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8ab8050090 con 0x7f8ac80b1a60 2023-10-26T05:48:28.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.333+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ac81645b0 con 0x7f8aa80664a0 2023-10-26T05:48:28.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.353+0000 7f8ac57fa640 1 -- 172.21.15.71:0/417434689 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278482 (secure 0 0 0) 0x7f8ac81645b0 con 0x7f8aa80664a0 2023-10-26T05:48:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:28.362 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:48:28.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8aa80664a0 msgr2=0x7f8aa8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8aa80664a0 0x7f8aa8068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ac813bfd0 tx=0x7f8ab0009040 comp rx=0 tx=0).stop 2023-10-26T05:48:28.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 msgr2=0x7f8ac81331a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 0x7f8ac81331a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800eeb0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 shutdown_connections 2023-10-26T05:48:28.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ac815bc50 0x7f8ac812e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ac80b1f10 0x7f8ac8153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8aa80664a0 0x7f8aa8068960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 --2- 172.21.15.71:0/417434689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ac80b1a60 0x7f8ac81331a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 >> 172.21.15.71:0/417434689 conn(0x7f8ac80caa90 msgr2=0x7f8ac80c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:28.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 shutdown_connections 2023-10-26T05:48:28.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.357+0000 7f8ace664640 1 -- 172.21.15.71:0/417434689 wait complete. 2023-10-26T05:48:28.377 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-26T05:48:27.433648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560291,"statfs":{"total":1600000000,"available":1597738663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700234+0000","last_change":"2023-10-26T05:48:23.700234+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700234+0000","last_undegraded":"2023-10-26T05:48:23.700234+0000","last_fullsized":"2023-10-26T05:48:23.700234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694918+0000","last_change":"2023-10-26T05:48:23.694918+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694918+0000","last_undegraded":"2023-10-26T05:48:23.694918+0000","last_fullsized":"2023-10-26T05:48:23.694918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695709+0000","last_change":"2023-10-26T05:48:23.695709+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695709+0000","last_undegraded":"2023-10-26T05:48:23.695709+0000","last_fullsized":"2023-10-26T05:48:23.695709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688038+0000","last_change":"2023-10-26T05:48:23.688038+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688038+0000","last_undegraded":"2023-10-26T05:48:23.688038+0000","last_fullsized":"2023-10-26T05:48:23.688038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694048+0000","last_change":"2023-10-26T05:48:23.694048+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694048+0000","last_undegraded":"2023-10-26T05:48:23.694048+0000","last_fullsized":"2023-10-26T05:48:23.694048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694220+0000","last_change":"2023-10-26T05:48:23.694220+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694220+0000","last_undegraded":"2023-10-26T05:48:23.694220+0000","last_fullsized":"2023-10-26T05:48:23.694220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696282+0000","last_change":"2023-10-26T05:48:23.696282+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696282+0000","last_undegraded":"2023-10-26T05:48:23.696282+0000","last_fullsized":"2023-10-26T05:48:23.696282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700861+0000","last_change":"2023-10-26T05:48:23.700861+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700861+0000","last_undegraded":"2023-10-26T05:48:23.700861+0000","last_fullsized":"2023-10-26T05:48:23.700861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687341+0000","last_change":"2023-10-26T05:48:23.687341+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687341+0000","last_undegraded":"2023-10-26T05:48:23.687341+0000","last_fullsized":"2023-10-26T05:48:23.687341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688537+0000","last_change":"2023-10-26T05:48:23.688537+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688537+0000","last_undegraded":"2023-10-26T05:48:23.688537+0000","last_fullsized":"2023-10-26T05:48:23.688537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.697056+0000","last_change":"2023-10-26T05:48:23.697056+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.697056+0000","last_undegraded":"2023-10-26T05:48:23.697056+0000","last_fullsized":"2023-10-26T05:48:23.697056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694710+0000","last_change":"2023-10-26T05:48:23.694710+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694710+0000","last_undegraded":"2023-10-26T05:48:23.694710+0000","last_fullsized":"2023-10-26T05:48:23.694710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695640+0000","last_change":"2023-10-26T05:48:23.695640+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695640+0000","last_undegraded":"2023-10-26T05:48:23.695640+0000","last_fullsized":"2023-10-26T05:48:23.695640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692044+0000","last_change":"2023-10-26T05:48:23.692044+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692044+0000","last_undegraded":"2023-10-26T05:48:23.692044+0000","last_fullsized":"2023-10-26T05:48:23.692044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692542+0000","last_change":"2023-10-26T05:48:23.692542+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692542+0000","last_undegraded":"2023-10-26T05:48:23.692542+0000","last_fullsized":"2023-10-26T05:48:23.692542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696259+0000","last_change":"2023-10-26T05:48:23.696259+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696259+0000","last_undegraded":"2023-10-26T05:48:23.696259+0000","last_fullsized":"2023-10-26T05:48:23.696259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.706851+0000","last_change":"2023-10-26T05:48:23.706851+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.706851+0000","last_undegraded":"2023-10-26T05:48:23.706851+0000","last_fullsized":"2023-10-26T05:48:23.706851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695326+0000","last_change":"2023-10-26T05:48:23.695326+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695326+0000","last_undegraded":"2023-10-26T05:48:23.695326+0000","last_fullsized":"2023-10-26T05:48:23.695326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691624+0000","last_change":"2023-10-26T05:48:23.691624+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691624+0000","last_undegraded":"2023-10-26T05:48:23.691624+0000","last_fullsized":"2023-10-26T05:48:23.691624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693579+0000","last_change":"2023-10-26T05:48:23.693579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693579+0000","last_undegraded":"2023-10-26T05:48:23.693579+0000","last_fullsized":"2023-10-26T05:48:23.693579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685254+0000","last_change":"2023-10-26T05:48:23.685254+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685254+0000","last_undegraded":"2023-10-26T05:48:23.685254+0000","last_fullsized":"2023-10-26T05:48:23.685254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685503+0000","last_change":"2023-10-26T05:48:23.685503+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685503+0000","last_undegraded":"2023-10-26T05:48:23.685503+0000","last_fullsized":"2023-10-26T05:48:23.685503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691108+0000","last_change":"2023-10-26T05:48:23.691108+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691108+0000","last_undegraded":"2023-10-26T05:48:23.691108+0000","last_fullsized":"2023-10-26T05:48:23.691108+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690897+0000","last_change":"2023-10-26T05:48:23.690897+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690897+0000","last_undegraded":"2023-10-26T05:48:23.690897+0000","last_fullsized":"2023-10-26T05:48:23.690897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688241+0000","last_change":"2023-10-26T05:48:23.688241+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688241+0000","last_undegraded":"2023-10-26T05:48:23.688241+0000","last_fullsized":"2023-10-26T05:48:23.688241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692899+0000","last_change":"2023-10-26T05:48:23.692899+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692899+0000","last_undegraded":"2023-10-26T05:48:23.692899+0000","last_fullsized":"2023-10-26T05:48:23.692899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687469+0000","last_change":"2023-10-26T05:48:23.687469+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687469+0000","last_undegraded":"2023-10-26T05:48:23.687469+0000","last_fullsized":"2023-10-26T05:48:23.687469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693430+0000","last_change":"2023-10-26T05:48:23.693430+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693430+0000","last_undegraded":"2023-10-26T05:48:23.693430+0000","last_fullsized":"2023-10-26T05:48:23.693430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.686546+0000","last_change":"2023-10-26T05:48:23.686546+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.686546+0000","last_undegraded":"2023-10-26T05:48:23.686546+0000","last_fullsized":"2023-10-26T05:48:23.686546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694718+0000","last_change":"2023-10-26T05:48:23.694718+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694718+0000","last_undegraded":"2023-10-26T05:48:23.694718+0000","last_fullsized":"2023-10-26T05:48:23.694718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.699413+0000","last_change":"2023-10-26T05:48:23.699413+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.699413+0000","last_undegraded":"2023-10-26T05:48:23.699413+0000","last_fullsized":"2023-10-26T05:48:23.699413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.703535+0000","last_change":"2023-10-26T05:48:23.703535+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.703535+0000","last_undegraded":"2023-10-26T05:48:23.703535+0000","last_fullsized":"2023-10-26T05:48:23.703535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688409+0000","last_change":"2023-10-26T05:48:23.688409+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688409+0000","last_undegraded":"2023-10-26T05:48:23.688409+0000","last_fullsized":"2023-10-26T05:48:23.688409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.711284+0000","last_change":"2023-10-26T05:48:23.711070+0000","last_active":"2023-10-26T05:48:23.711284+0000","last_peered":"2023-10-26T05:48:23.711284+0000","last_clean":"2023-10-26T05:48:23.711284+0000","last_became_active":"2023-10-26T05:48:23.710421+0000","last_became_peered":"2023-10-26T05:48:23.710421+0000","last_unstale":"2023-10-26T05:48:23.711284+0000","last_undegraded":"2023-10-26T05:48:23.711284+0000","last_fullsized":"2023-10-26T05:48:23.711284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695820+0000","last_change":"2023-10-26T05:48:23.695820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695820+0000","last_undegraded":"2023-10-26T05:48:23.695820+0000","last_fullsized":"2023-10-26T05:48:23.695820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703916+0000","last_change":"2023-10-26T05:48:23.703810+0000","last_active":"2023-10-26T05:48:23.703916+0000","last_peered":"2023-10-26T05:48:23.703916+0000","last_clean":"2023-10-26T05:48:23.703916+0000","last_became_active":"2023-10-26T05:48:23.703500+0000","last_became_peered":"2023-10-26T05:48:23.703500+0000","last_unstale":"2023-10-26T05:48:23.703916+0000","last_undegraded":"2023-10-26T05:48:23.703916+0000","last_fullsized":"2023-10-26T05:48:23.703916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.683637+0000","last_change":"2023-10-26T05:48:23.683637+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.683637+0000","last_undegraded":"2023-10-26T05:48:23.683637+0000","last_fullsized":"2023-10-26T05:48:23.683637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690187+0000","last_change":"2023-10-26T05:48:23.690022+0000","last_active":"2023-10-26T05:48:23.690187+0000","last_peered":"2023-10-26T05:48:23.690187+0000","last_clean":"2023-10-26T05:48:23.690187+0000","last_became_active":"2023-10-26T05:48:23.689605+0000","last_became_peered":"2023-10-26T05:48:23.689605+0000","last_unstale":"2023-10-26T05:48:23.690187+0000","last_undegraded":"2023-10-26T05:48:23.690187+0000","last_fullsized":"2023-10-26T05:48:23.690187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687820+0000","last_change":"2023-10-26T05:48:23.687820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687820+0000","last_undegraded":"2023-10-26T05:48:23.687820+0000","last_fullsized":"2023-10-26T05:48:23.687820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701416+0000","last_change":"2023-10-26T05:48:23.701287+0000","last_active":"2023-10-26T05:48:23.701416+0000","last_peered":"2023-10-26T05:48:23.701416+0000","last_clean":"2023-10-26T05:48:23.701416+0000","last_became_active":"2023-10-26T05:48:23.700949+0000","last_became_peered":"2023-10-26T05:48:23.700949+0000","last_unstale":"2023-10-26T05:48:23.701416+0000","last_undegraded":"2023-10-26T05:48:23.701416+0000","last_fullsized":"2023-10-26T05:48:23.701416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700105+0000","last_change":"2023-10-26T05:48:23.700105+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700105+0000","last_undegraded":"2023-10-26T05:48:23.700105+0000","last_fullsized":"2023-10-26T05:48:23.700105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691137+0000","last_change":"2023-10-26T05:48:23.690572+0000","last_active":"2023-10-26T05:48:23.691137+0000","last_peered":"2023-10-26T05:48:23.691137+0000","last_clean":"2023-10-26T05:48:23.691137+0000","last_became_active":"2023-10-26T05:48:23.689616+0000","last_became_peered":"2023-10-26T05:48:23.689616+0000","last_unstale":"2023-10-26T05:48:23.691137+0000","last_undegraded":"2023-10-26T05:48:23.691137+0000","last_fullsized":"2023-10-26T05:48:23.691137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:36:17.063916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693976+0000","last_change":"2023-10-26T05:48:23.693976+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693976+0000","last_undegraded":"2023-10-26T05:48:23.693976+0000","last_fullsized":"2023-10-26T05:48:23.693976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701389+0000","last_change":"2023-10-26T05:48:23.701241+0000","last_active":"2023-10-26T05:48:23.701389+0000","last_peered":"2023-10-26T05:48:23.701389+0000","last_clean":"2023-10-26T05:48:23.701389+0000","last_became_active":"2023-10-26T05:48:23.699629+0000","last_became_peered":"2023-10-26T05:48:23.699629+0000","last_unstale":"2023-10-26T05:48:23.701389+0000","last_undegraded":"2023-10-26T05:48:23.701389+0000","last_fullsized":"2023-10-26T05:48:23.701389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:24:45.082594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688715+0000","last_change":"2023-10-26T05:48:23.688715+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688715+0000","last_undegraded":"2023-10-26T05:48:23.688715+0000","last_fullsized":"2023-10-26T05:48:23.688715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.700492+0000","last_change":"2023-10-26T05:48:23.700412+0000","last_active":"2023-10-26T05:48:23.700492+0000","last_peered":"2023-10-26T05:48:23.700492+0000","last_clean":"2023-10-26T05:48:23.700492+0000","last_became_active":"2023-10-26T05:48:23.698966+0000","last_became_peered":"2023-10-26T05:48:23.698966+0000","last_unstale":"2023-10-26T05:48:23.700492+0000","last_undegraded":"2023-10-26T05:48:23.700492+0000","last_fullsized":"2023-10-26T05:48:23.700492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687623+0000","last_change":"2023-10-26T05:48:23.687623+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687623+0000","last_undegraded":"2023-10-26T05:48:23.687623+0000","last_fullsized":"2023-10-26T05:48:23.687623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715611+0000","last_change":"2023-10-26T05:48:23.715497+0000","last_active":"2023-10-26T05:48:23.715611+0000","last_peered":"2023-10-26T05:48:23.715611+0000","last_clean":"2023-10-26T05:48:23.715611+0000","last_became_active":"2023-10-26T05:48:23.714277+0000","last_became_peered":"2023-10-26T05:48:23.714277+0000","last_unstale":"2023-10-26T05:48:23.715611+0000","last_undegraded":"2023-10-26T05:48:23.715611+0000","last_fullsized":"2023-10-26T05:48:23.715611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:16:54.930577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691328+0000","last_change":"2023-10-26T05:48:23.691328+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691328+0000","last_undegraded":"2023-10-26T05:48:23.691328+0000","last_fullsized":"2023-10-26T05:48:23.691328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705835+0000","last_change":"2023-10-26T05:48:23.705748+0000","last_active":"2023-10-26T05:48:23.705835+0000","last_peered":"2023-10-26T05:48:23.705835+0000","last_clean":"2023-10-26T05:48:23.705835+0000","last_became_active":"2023-10-26T05:48:23.705389+0000","last_became_peered":"2023-10-26T05:48:23.705389+0000","last_unstale":"2023-10-26T05:48:23.705835+0000","last_undegraded":"2023-10-26T05:48:23.705835+0000","last_fullsized":"2023-10-26T05:48:23.705835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692498+0000","last_change":"2023-10-26T05:48:23.692498+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692498+0000","last_undegraded":"2023-10-26T05:48:23.692498+0000","last_fullsized":"2023-10-26T05:48:23.692498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.696945+0000","last_change":"2023-10-26T05:48:23.696857+0000","last_active":"2023-10-26T05:48:23.696945+0000","last_peered":"2023-10-26T05:48:23.696945+0000","last_clean":"2023-10-26T05:48:23.696945+0000","last_became_active":"2023-10-26T05:48:23.696471+0000","last_became_peered":"2023-10-26T05:48:23.696471+0000","last_unstale":"2023-10-26T05:48:23.696945+0000","last_undegraded":"2023-10-26T05:48:23.696945+0000","last_fullsized":"2023-10-26T05:48:23.696945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691173+0000","last_change":"2023-10-26T05:48:23.691173+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691173+0000","last_undegraded":"2023-10-26T05:48:23.691173+0000","last_fullsized":"2023-10-26T05:48:23.691173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.709783+0000","last_change":"2023-10-26T05:48:23.709670+0000","last_active":"2023-10-26T05:48:23.709783+0000","last_peered":"2023-10-26T05:48:23.709783+0000","last_clean":"2023-10-26T05:48:23.709783+0000","last_became_active":"2023-10-26T05:48:23.709170+0000","last_became_peered":"2023-10-26T05:48:23.709170+0000","last_unstale":"2023-10-26T05:48:23.709783+0000","last_undegraded":"2023-10-26T05:48:23.709783+0000","last_fullsized":"2023-10-26T05:48:23.709783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:03.464765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692408+0000","last_change":"2023-10-26T05:48:23.692408+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692408+0000","last_undegraded":"2023-10-26T05:48:23.692408+0000","last_fullsized":"2023-10-26T05:48:23.692408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690158+0000","last_change":"2023-10-26T05:48:23.689973+0000","last_active":"2023-10-26T05:48:23.690158+0000","last_peered":"2023-10-26T05:48:23.690158+0000","last_clean":"2023-10-26T05:48:23.690158+0000","last_became_active":"2023-10-26T05:48:23.689550+0000","last_became_peered":"2023-10-26T05:48:23.689550+0000","last_unstale":"2023-10-26T05:48:23.690158+0000","last_undegraded":"2023-10-26T05:48:23.690158+0000","last_fullsized":"2023-10-26T05:48:23.690158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.768450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692771+0000","last_change":"2023-10-26T05:48:23.692771+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692771+0000","last_undegraded":"2023-10-26T05:48:23.692771+0000","last_fullsized":"2023-10-26T05:48:23.692771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715379+0000","last_change":"2023-10-26T05:48:23.715244+0000","last_active":"2023-10-26T05:48:23.715379+0000","last_peered":"2023-10-26T05:48:23.715379+0000","last_clean":"2023-10-26T05:48:23.715379+0000","last_became_active":"2023-10-26T05:48:23.713621+0000","last_became_peered":"2023-10-26T05:48:23.713621+0000","last_unstale":"2023-10-26T05:48:23.715379+0000","last_undegraded":"2023-10-26T05:48:23.715379+0000","last_fullsized":"2023-10-26T05:48:23.715379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:52.471566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691733+0000","last_change":"2023-10-26T05:48:23.691733+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691733+0000","last_undegraded":"2023-10-26T05:48:23.691733+0000","last_fullsized":"2023-10-26T05:48:23.691733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.704334+0000","last_change":"2023-10-26T05:48:23.704259+0000","last_active":"2023-10-26T05:48:23.704334+0000","last_peered":"2023-10-26T05:48:23.704334+0000","last_clean":"2023-10-26T05:48:23.704334+0000","last_became_active":"2023-10-26T05:48:23.703997+0000","last_became_peered":"2023-10-26T05:48:23.703997+0000","last_unstale":"2023-10-26T05:48:23.704334+0000","last_undegraded":"2023-10-26T05:48:23.704334+0000","last_fullsized":"2023-10-26T05:48:23.704334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694411+0000","last_change":"2023-10-26T05:48:23.694411+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694411+0000","last_undegraded":"2023-10-26T05:48:23.694411+0000","last_fullsized":"2023-10-26T05:48:23.694411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.706547+0000","last_change":"2023-10-26T05:48:23.706487+0000","last_active":"2023-10-26T05:48:23.706547+0000","last_peered":"2023-10-26T05:48:23.706547+0000","last_clean":"2023-10-26T05:48:23.706547+0000","last_became_active":"2023-10-26T05:48:23.706232+0000","last_became_peered":"2023-10-26T05:48:23.706232+0000","last_unstale":"2023-10-26T05:48:23.706547+0000","last_undegraded":"2023-10-26T05:48:23.706547+0000","last_fullsized":"2023-10-26T05:48:23.706547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696971+0000","last_change":"2023-10-26T05:48:23.696971+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696971+0000","last_undegraded":"2023-10-26T05:48:23.696971+0000","last_fullsized":"2023-10-26T05:48:23.696971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.689358+0000","last_change":"2023-10-26T05:48:23.688956+0000","last_active":"2023-10-26T05:48:23.689358+0000","last_peered":"2023-10-26T05:48:23.689358+0000","last_clean":"2023-10-26T05:48:23.689358+0000","last_became_active":"2023-10-26T05:48:23.688311+0000","last_became_peered":"2023-10-26T05:48:23.688311+0000","last_unstale":"2023-10-26T05:48:23.689358+0000","last_undegraded":"2023-10-26T05:48:23.689358+0000","last_fullsized":"2023-10-26T05:48:23.689358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:52.984869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687957+0000","last_change":"2023-10-26T05:48:23.687957+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687957+0000","last_undegraded":"2023-10-26T05:48:23.687957+0000","last_fullsized":"2023-10-26T05:48:23.687957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.688670+0000","last_change":"2023-10-26T05:48:23.688351+0000","last_active":"2023-10-26T05:48:23.688670+0000","last_peered":"2023-10-26T05:48:23.688670+0000","last_clean":"2023-10-26T05:48:23.688670+0000","last_became_active":"2023-10-26T05:48:23.687015+0000","last_became_peered":"2023-10-26T05:48:23.687015+0000","last_unstale":"2023-10-26T05:48:23.688670+0000","last_undegraded":"2023-10-26T05:48:23.688670+0000","last_fullsized":"2023-10-26T05:48:23.688670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.689977+0000","last_change":"2023-10-26T05:48:23.689977+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.689977+0000","last_undegraded":"2023-10-26T05:48:23.689977+0000","last_fullsized":"2023-10-26T05:48:23.689977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715055+0000","last_change":"2023-10-26T05:48:23.714853+0000","last_active":"2023-10-26T05:48:23.715055+0000","last_peered":"2023-10-26T05:48:23.715055+0000","last_clean":"2023-10-26T05:48:23.715055+0000","last_became_active":"2023-10-26T05:48:23.712919+0000","last_became_peered":"2023-10-26T05:48:23.712919+0000","last_unstale":"2023-10-26T05:48:23.715055+0000","last_undegraded":"2023-10-26T05:48:23.715055+0000","last_fullsized":"2023-10-26T05:48:23.715055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:48.260866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687100+0000","last_change":"2023-10-26T05:48:23.687100+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687100+0000","last_undegraded":"2023-10-26T05:48:23.687100+0000","last_fullsized":"2023-10-26T05:48:23.687100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703330+0000","last_change":"2023-10-26T05:48:23.703179+0000","last_active":"2023-10-26T05:48:23.703330+0000","last_peered":"2023-10-26T05:48:23.703330+0000","last_clean":"2023-10-26T05:48:23.703330+0000","last_became_active":"2023-10-26T05:48:23.702603+0000","last_became_peered":"2023-10-26T05:48:23.702603+0000","last_unstale":"2023-10-26T05:48:23.703330+0000","last_undegraded":"2023-10-26T05:48:23.703330+0000","last_fullsized":"2023-10-26T05:48:23.703330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:10.226049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694579+0000","last_change":"2023-10-26T05:48:23.694579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694579+0000","last_undegraded":"2023-10-26T05:48:23.694579+0000","last_fullsized":"2023-10-26T05:48:23.694579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691086+0000","last_change":"2023-10-26T05:48:23.690214+0000","last_active":"2023-10-26T05:48:23.691086+0000","last_peered":"2023-10-26T05:48:23.691086+0000","last_clean":"2023-10-26T05:48:23.691086+0000","last_became_active":"2023-10-26T05:48:23.688325+0000","last_became_peered":"2023-10-26T05:48:23.688325+0000","last_unstale":"2023-10-26T05:48:23.691086+0000","last_undegraded":"2023-10-26T05:48:23.691086+0000","last_fullsized":"2023-10-26T05:48:23.691086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:55.310630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694075+0000","last_change":"2023-10-26T05:48:23.694075+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694075+0000","last_undegraded":"2023-10-26T05:48:23.694075+0000","last_fullsized":"2023-10-26T05:48:23.694075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692061+0000","last_change":"2023-10-26T05:48:23.691997+0000","last_active":"2023-10-26T05:48:23.692061+0000","last_peered":"2023-10-26T05:48:23.692061+0000","last_clean":"2023-10-26T05:48:23.692061+0000","last_became_active":"2023-10-26T05:48:23.691414+0000","last_became_peered":"2023-10-26T05:48:23.691414+0000","last_unstale":"2023-10-26T05:48:23.692061+0000","last_undegraded":"2023-10-26T05:48:23.692061+0000","last_fullsized":"2023-10-26T05:48:23.692061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695252+0000","last_change":"2023-10-26T05:48:23.695252+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695252+0000","last_undegraded":"2023-10-26T05:48:23.695252+0000","last_fullsized":"2023-10-26T05:48:23.695252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705798+0000","last_change":"2023-10-26T05:48:23.705656+0000","last_active":"2023-10-26T05:48:23.705798+0000","last_peered":"2023-10-26T05:48:23.705798+0000","last_clean":"2023-10-26T05:48:23.705798+0000","last_became_active":"2023-10-26T05:48:23.705077+0000","last_became_peered":"2023-10-26T05:48:23.705077+0000","last_unstale":"2023-10-26T05:48:23.705798+0000","last_undegraded":"2023-10-26T05:48:23.705798+0000","last_fullsized":"2023-10-26T05:48:23.705798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694168+0000","last_change":"2023-10-26T05:48:23.694168+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694168+0000","last_undegraded":"2023-10-26T05:48:23.694168+0000","last_fullsized":"2023-10-26T05:48:23.694168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712295+0000","last_change":"2023-10-26T05:48:23.712097+0000","last_active":"2023-10-26T05:48:23.712295+0000","last_peered":"2023-10-26T05:48:23.712295+0000","last_clean":"2023-10-26T05:48:23.712295+0000","last_became_active":"2023-10-26T05:48:23.704476+0000","last_became_peered":"2023-10-26T05:48:23.704476+0000","last_unstale":"2023-10-26T05:48:23.712295+0000","last_undegraded":"2023-10-26T05:48:23.712295+0000","last_fullsized":"2023-10-26T05:48:23.712295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:04.497346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693873+0000","last_change":"2023-10-26T05:48:23.693873+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693873+0000","last_undegraded":"2023-10-26T05:48:23.693873+0000","last_fullsized":"2023-10-26T05:48:23.693873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.708766+0000","last_change":"2023-10-26T05:48:23.708587+0000","last_active":"2023-10-26T05:48:23.708766+0000","last_peered":"2023-10-26T05:48:23.708766+0000","last_clean":"2023-10-26T05:48:23.708766+0000","last_became_active":"2023-10-26T05:48:23.707969+0000","last_became_peered":"2023-10-26T05:48:23.707969+0000","last_unstale":"2023-10-26T05:48:23.708766+0000","last_undegraded":"2023-10-26T05:48:23.708766+0000","last_fullsized":"2023-10-26T05:48:23.708766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694854+0000","last_change":"2023-10-26T05:48:23.694854+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694854+0000","last_undegraded":"2023-10-26T05:48:23.694854+0000","last_fullsized":"2023-10-26T05:48:23.694854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705560+0000","last_change":"2023-10-26T05:48:23.705491+0000","last_active":"2023-10-26T05:48:23.705560+0000","last_peered":"2023-10-26T05:48:23.705560+0000","last_clean":"2023-10-26T05:48:23.705560+0000","last_became_active":"2023-10-26T05:48:23.705095+0000","last_became_peered":"2023-10-26T05:48:23.705095+0000","last_unstale":"2023-10-26T05:48:23.705560+0000","last_undegraded":"2023-10-26T05:48:23.705560+0000","last_fullsized":"2023-10-26T05:48:23.705560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693231+0000","last_change":"2023-10-26T05:48:23.693231+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693231+0000","last_undegraded":"2023-10-26T05:48:23.693231+0000","last_fullsized":"2023-10-26T05:48:23.693231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692208+0000","last_change":"2023-10-26T05:48:23.692126+0000","last_active":"2023-10-26T05:48:23.692208+0000","last_peered":"2023-10-26T05:48:23.692208+0000","last_clean":"2023-10-26T05:48:23.692208+0000","last_became_active":"2023-10-26T05:48:23.691722+0000","last_became_peered":"2023-10-26T05:48:23.691722+0000","last_unstale":"2023-10-26T05:48:23.692208+0000","last_undegraded":"2023-10-26T05:48:23.692208+0000","last_fullsized":"2023-10-26T05:48:23.692208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688037+0000","last_change":"2023-10-26T05:48:23.688037+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688037+0000","last_undegraded":"2023-10-26T05:48:23.688037+0000","last_fullsized":"2023-10-26T05:48:23.688037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.699555+0000","last_change":"2023-10-26T05:48:23.699301+0000","last_active":"2023-10-26T05:48:23.699555+0000","last_peered":"2023-10-26T05:48:23.699555+0000","last_clean":"2023-10-26T05:48:23.699555+0000","last_became_active":"2023-10-26T05:48:23.698926+0000","last_became_peered":"2023-10-26T05:48:23.698926+0000","last_unstale":"2023-10-26T05:48:23.699555+0000","last_undegraded":"2023-10-26T05:48:23.699555+0000","last_fullsized":"2023-10-26T05:48:23.699555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696097+0000","last_change":"2023-10-26T05:48:23.696097+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696097+0000","last_undegraded":"2023-10-26T05:48:23.696097+0000","last_fullsized":"2023-10-26T05:48:23.696097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.697196+0000","last_change":"2023-10-26T05:48:23.697103+0000","last_active":"2023-10-26T05:48:23.697196+0000","last_peered":"2023-10-26T05:48:23.697196+0000","last_clean":"2023-10-26T05:48:23.697196+0000","last_became_active":"2023-10-26T05:48:23.696770+0000","last_became_peered":"2023-10-26T05:48:23.696770+0000","last_unstale":"2023-10-26T05:48:23.697196+0000","last_undegraded":"2023-10-26T05:48:23.697196+0000","last_fullsized":"2023-10-26T05:48:23.697196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690554+0000","last_change":"2023-10-26T05:48:23.690554+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690554+0000","last_undegraded":"2023-10-26T05:48:23.690554+0000","last_fullsized":"2023-10-26T05:48:23.690554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705457+0000","last_change":"2023-10-26T05:48:23.705357+0000","last_active":"2023-10-26T05:48:23.705457+0000","last_peered":"2023-10-26T05:48:23.705457+0000","last_clean":"2023-10-26T05:48:23.705457+0000","last_became_active":"2023-10-26T05:48:23.704629+0000","last_became_peered":"2023-10-26T05:48:23.704629+0000","last_unstale":"2023-10-26T05:48:23.705457+0000","last_undegraded":"2023-10-26T05:48:23.705457+0000","last_fullsized":"2023-10-26T05:48:23.705457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690693+0000","last_change":"2023-10-26T05:48:23.690693+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690693+0000","last_undegraded":"2023-10-26T05:48:23.690693+0000","last_fullsized":"2023-10-26T05:48:23.690693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701672+0000","last_change":"2023-10-26T05:48:23.701546+0000","last_active":"2023-10-26T05:48:23.701672+0000","last_peered":"2023-10-26T05:48:23.701672+0000","last_clean":"2023-10-26T05:48:23.701672+0000","last_became_active":"2023-10-26T05:48:23.700654+0000","last_became_peered":"2023-10-26T05:48:23.700654+0000","last_unstale":"2023-10-26T05:48:23.701672+0000","last_undegraded":"2023-10-26T05:48:23.701672+0000","last_fullsized":"2023-10-26T05:48:23.701672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.684207+0000","last_change":"2023-10-26T05:48:23.684207+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.684207+0000","last_undegraded":"2023-10-26T05:48:23.684207+0000","last_fullsized":"2023-10-26T05:48:23.684207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703737+0000","last_change":"2023-10-26T05:48:23.703600+0000","last_active":"2023-10-26T05:48:23.703737+0000","last_peered":"2023-10-26T05:48:23.703737+0000","last_clean":"2023-10-26T05:48:23.703737+0000","last_became_active":"2023-10-26T05:48:23.703254+0000","last_became_peered":"2023-10-26T05:48:23.703254+0000","last_unstale":"2023-10-26T05:48:23.703737+0000","last_undegraded":"2023-10-26T05:48:23.703737+0000","last_fullsized":"2023-10-26T05:48:23.703737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:35.361883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692904+0000","last_change":"2023-10-26T05:48:23.692904+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692904+0000","last_undegraded":"2023-10-26T05:48:23.692904+0000","last_fullsized":"2023-10-26T05:48:23.692904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712540+0000","last_change":"2023-10-26T05:48:23.712299+0000","last_active":"2023-10-26T05:48:23.712540+0000","last_peered":"2023-10-26T05:48:23.712540+0000","last_clean":"2023-10-26T05:48:23.712540+0000","last_became_active":"2023-10-26T05:48:23.711619+0000","last_became_peered":"2023-10-26T05:48:23.711619+0000","last_unstale":"2023-10-26T05:48:23.712540+0000","last_undegraded":"2023-10-26T05:48:23.712540+0000","last_fullsized":"2023-10-26T05:48:23.712540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-26T05:48:22.672770+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:48:22.672770+0000","last_peered":"2023-10-26T05:48:22.672770+0000","last_clean":"2023-10-26T05:48:22.672770+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:48:22.672770+0000","last_undegraded":"2023-10-26T05:48:22.672770+0000","last_fullsized":"2023-10-26T05:48:22.672770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194606,"statfs":{"total":200000000,"available":199276907,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.022},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.05}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.147},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.292},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2390000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.879}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.0409999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.0700000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.622},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.196}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.087}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.012},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.159}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.751},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.24}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.752},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.036}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.77800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.99099999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.083},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.871}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270640,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.73799999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:48:28.383 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-10-26T05:48:28.384 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.700234+0000', 'last_change': '2023-10-26T05:48:23.700234+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.700234+0000', 'last_undegraded': '2023-10-26T05:48:23.700234+0000', 'last_fullsized': '2023-10-26T05:48:23.700234+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:48:28.384 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-10-26T05:48:28.384 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694918+0000', 'last_change': '2023-10-26T05:48:23.694918+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694918+0000', 'last_undegraded': '2023-10-26T05:48:23.694918+0000', 'last_fullsized': '2023-10-26T05:48:23.694918+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.385 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-10-26T05:48:28.385 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.695709+0000', 'last_change': '2023-10-26T05:48:23.695709+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.695709+0000', 'last_undegraded': '2023-10-26T05:48:23.695709+0000', 'last_fullsized': '2023-10-26T05:48:23.695709+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.386 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-10-26T05:48:28.386 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.688038+0000', 'last_change': '2023-10-26T05:48:23.688038+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.688038+0000', 'last_undegraded': '2023-10-26T05:48:23.688038+0000', 'last_fullsized': '2023-10-26T05:48:23.688038+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-10-26T05:48:28.387 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694048+0000', 'last_change': '2023-10-26T05:48:23.694048+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694048+0000', 'last_undegraded': '2023-10-26T05:48:23.694048+0000', 'last_fullsized': '2023-10-26T05:48:23.694048+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.387 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-10-26T05:48:28.388 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694220+0000', 'last_change': '2023-10-26T05:48:23.694220+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694220+0000', 'last_undegraded': '2023-10-26T05:48:23.694220+0000', 'last_fullsized': '2023-10-26T05:48:23.694220+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.388 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-10-26T05:48:28.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.696282+0000', 'last_change': '2023-10-26T05:48:23.696282+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.696282+0000', 'last_undegraded': '2023-10-26T05:48:23.696282+0000', 'last_fullsized': '2023-10-26T05:48:23.696282+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.389 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-10-26T05:48:28.389 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.700861+0000', 'last_change': '2023-10-26T05:48:23.700861+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.700861+0000', 'last_undegraded': '2023-10-26T05:48:23.700861+0000', 'last_fullsized': '2023-10-26T05:48:23.700861+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.390 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-10-26T05:48:28.390 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.687341+0000', 'last_change': '2023-10-26T05:48:23.687341+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.687341+0000', 'last_undegraded': '2023-10-26T05:48:23.687341+0000', 'last_fullsized': '2023-10-26T05:48:23.687341+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-10-26T05:48:28.391 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.688537+0000', 'last_change': '2023-10-26T05:48:23.688537+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.688537+0000', 'last_undegraded': '2023-10-26T05:48:23.688537+0000', 'last_fullsized': '2023-10-26T05:48:23.688537+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.391 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-10-26T05:48:28.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.697056+0000', 'last_change': '2023-10-26T05:48:23.697056+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.697056+0000', 'last_undegraded': '2023-10-26T05:48:23.697056+0000', 'last_fullsized': '2023-10-26T05:48:23.697056+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-10-26T05:48:28.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694710+0000', 'last_change': '2023-10-26T05:48:23.694710+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694710+0000', 'last_undegraded': '2023-10-26T05:48:23.694710+0000', 'last_fullsized': '2023-10-26T05:48:23.694710+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-10-26T05:48:28.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.695640+0000', 'last_change': '2023-10-26T05:48:23.695640+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.695640+0000', 'last_undegraded': '2023-10-26T05:48:23.695640+0000', 'last_fullsized': '2023-10-26T05:48:23.695640+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-10-26T05:48:28.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692044+0000', 'last_change': '2023-10-26T05:48:23.692044+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692044+0000', 'last_undegraded': '2023-10-26T05:48:23.692044+0000', 'last_fullsized': '2023-10-26T05:48:23.692044+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-10-26T05:48:28.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692542+0000', 'last_change': '2023-10-26T05:48:23.692542+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692542+0000', 'last_undegraded': '2023-10-26T05:48:23.692542+0000', 'last_fullsized': '2023-10-26T05:48:23.692542+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-10-26T05:48:28.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.696259+0000', 'last_change': '2023-10-26T05:48:23.696259+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.696259+0000', 'last_undegraded': '2023-10-26T05:48:23.696259+0000', 'last_fullsized': '2023-10-26T05:48:23.696259+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-10-26T05:48:28.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.706851+0000', 'last_change': '2023-10-26T05:48:23.706851+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.706851+0000', 'last_undegraded': '2023-10-26T05:48:23.706851+0000', 'last_fullsized': '2023-10-26T05:48:23.706851+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-10-26T05:48:28.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.695326+0000', 'last_change': '2023-10-26T05:48:23.695326+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.695326+0000', 'last_undegraded': '2023-10-26T05:48:23.695326+0000', 'last_fullsized': '2023-10-26T05:48:23.695326+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-10-26T05:48:28.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.691624+0000', 'last_change': '2023-10-26T05:48:23.691624+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.691624+0000', 'last_undegraded': '2023-10-26T05:48:23.691624+0000', 'last_fullsized': '2023-10-26T05:48:23.691624+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-10-26T05:48:28.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.693579+0000', 'last_change': '2023-10-26T05:48:23.693579+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.693579+0000', 'last_undegraded': '2023-10-26T05:48:23.693579+0000', 'last_fullsized': '2023-10-26T05:48:23.693579+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-10-26T05:48:28.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.685254+0000', 'last_change': '2023-10-26T05:48:23.685254+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.685254+0000', 'last_undegraded': '2023-10-26T05:48:23.685254+0000', 'last_fullsized': '2023-10-26T05:48:23.685254+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.400 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-10-26T05:48:28.400 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.685503+0000', 'last_change': '2023-10-26T05:48:23.685503+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.685503+0000', 'last_undegraded': '2023-10-26T05:48:23.685503+0000', 'last_fullsized': '2023-10-26T05:48:23.685503+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-10-26T05:48:28.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.691108+0000', 'last_change': '2023-10-26T05:48:23.691108+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.691108+0000', 'last_undegraded': '2023-10-26T05:48:23.691108+0000', 'last_fullsized': '2023-10-26T05:48:23.691108+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:48:28.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-10-26T05:48:28.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.690897+0000', 'last_change': '2023-10-26T05:48:23.690897+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.690897+0000', 'last_undegraded': '2023-10-26T05:48:23.690897+0000', 'last_fullsized': '2023-10-26T05:48:23.690897+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-10-26T05:48:28.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.688241+0000', 'last_change': '2023-10-26T05:48:23.688241+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.688241+0000', 'last_undegraded': '2023-10-26T05:48:23.688241+0000', 'last_fullsized': '2023-10-26T05:48:23.688241+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-10-26T05:48:28.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692899+0000', 'last_change': '2023-10-26T05:48:23.692899+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692899+0000', 'last_undegraded': '2023-10-26T05:48:23.692899+0000', 'last_fullsized': '2023-10-26T05:48:23.692899+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-10-26T05:48:28.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.687469+0000', 'last_change': '2023-10-26T05:48:23.687469+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.687469+0000', 'last_undegraded': '2023-10-26T05:48:23.687469+0000', 'last_fullsized': '2023-10-26T05:48:23.687469+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-10-26T05:48:28.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.693430+0000', 'last_change': '2023-10-26T05:48:23.693430+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.693430+0000', 'last_undegraded': '2023-10-26T05:48:23.693430+0000', 'last_fullsized': '2023-10-26T05:48:23.693430+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-10-26T05:48:28.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.686546+0000', 'last_change': '2023-10-26T05:48:23.686546+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.686546+0000', 'last_undegraded': '2023-10-26T05:48:23.686546+0000', 'last_fullsized': '2023-10-26T05:48:23.686546+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-10-26T05:48:28.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694718+0000', 'last_change': '2023-10-26T05:48:23.694718+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694718+0000', 'last_undegraded': '2023-10-26T05:48:23.694718+0000', 'last_fullsized': '2023-10-26T05:48:23.694718+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-10-26T05:48:28.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.699413+0000', 'last_change': '2023-10-26T05:48:23.699413+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.699413+0000', 'last_undegraded': '2023-10-26T05:48:23.699413+0000', 'last_fullsized': '2023-10-26T05:48:23.699413+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:48:28.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-10-26T05:48:28.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.703535+0000', 'last_change': '2023-10-26T05:48:23.703535+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.703535+0000', 'last_undegraded': '2023-10-26T05:48:23.703535+0000', 'last_fullsized': '2023-10-26T05:48:23.703535+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-10-26T05:48:28.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.688409+0000', 'last_change': '2023-10-26T05:48:23.688409+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.688409+0000', 'last_undegraded': '2023-10-26T05:48:23.688409+0000', 'last_fullsized': '2023-10-26T05:48:23.688409+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:48:28.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-10-26T05:48:28.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.695820+0000', 'last_change': '2023-10-26T05:48:23.695820+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.695820+0000', 'last_undegraded': '2023-10-26T05:48:23.695820+0000', 'last_fullsized': '2023-10-26T05:48:23.695820+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:48:28.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-10-26T05:48:28.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.683637+0000', 'last_change': '2023-10-26T05:48:23.683637+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.683637+0000', 'last_undegraded': '2023-10-26T05:48:23.683637+0000', 'last_fullsized': '2023-10-26T05:48:23.683637+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-10-26T05:48:28.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.687820+0000', 'last_change': '2023-10-26T05:48:23.687820+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.687820+0000', 'last_undegraded': '2023-10-26T05:48:23.687820+0000', 'last_fullsized': '2023-10-26T05:48:23.687820+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-10-26T05:48:28.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.700105+0000', 'last_change': '2023-10-26T05:48:23.700105+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.700105+0000', 'last_undegraded': '2023-10-26T05:48:23.700105+0000', 'last_fullsized': '2023-10-26T05:48:23.700105+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-10-26T05:48:28.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.693976+0000', 'last_change': '2023-10-26T05:48:23.693976+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.693976+0000', 'last_undegraded': '2023-10-26T05:48:23.693976+0000', 'last_fullsized': '2023-10-26T05:48:23.693976+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-10-26T05:48:28.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.688715+0000', 'last_change': '2023-10-26T05:48:23.688715+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.688715+0000', 'last_undegraded': '2023-10-26T05:48:23.688715+0000', 'last_fullsized': '2023-10-26T05:48:23.688715+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-10-26T05:48:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.687623+0000', 'last_change': '2023-10-26T05:48:23.687623+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.687623+0000', 'last_undegraded': '2023-10-26T05:48:23.687623+0000', 'last_fullsized': '2023-10-26T05:48:23.687623+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-10-26T05:48:28.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.691328+0000', 'last_change': '2023-10-26T05:48:23.691328+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.691328+0000', 'last_undegraded': '2023-10-26T05:48:23.691328+0000', 'last_fullsized': '2023-10-26T05:48:23.691328+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-10-26T05:48:28.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692498+0000', 'last_change': '2023-10-26T05:48:23.692498+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692498+0000', 'last_undegraded': '2023-10-26T05:48:23.692498+0000', 'last_fullsized': '2023-10-26T05:48:23.692498+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-10-26T05:48:28.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.691173+0000', 'last_change': '2023-10-26T05:48:23.691173+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.691173+0000', 'last_undegraded': '2023-10-26T05:48:23.691173+0000', 'last_fullsized': '2023-10-26T05:48:23.691173+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-10-26T05:48:28.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692408+0000', 'last_change': '2023-10-26T05:48:23.692408+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692408+0000', 'last_undegraded': '2023-10-26T05:48:23.692408+0000', 'last_fullsized': '2023-10-26T05:48:23.692408+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-10-26T05:48:28.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692771+0000', 'last_change': '2023-10-26T05:48:23.692771+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692771+0000', 'last_undegraded': '2023-10-26T05:48:23.692771+0000', 'last_fullsized': '2023-10-26T05:48:23.692771+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-10-26T05:48:28.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.691733+0000', 'last_change': '2023-10-26T05:48:23.691733+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.691733+0000', 'last_undegraded': '2023-10-26T05:48:23.691733+0000', 'last_fullsized': '2023-10-26T05:48:23.691733+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-10-26T05:48:28.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694411+0000', 'last_change': '2023-10-26T05:48:23.694411+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694411+0000', 'last_undegraded': '2023-10-26T05:48:23.694411+0000', 'last_fullsized': '2023-10-26T05:48:23.694411+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-10-26T05:48:28.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.696971+0000', 'last_change': '2023-10-26T05:48:23.696971+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.696971+0000', 'last_undegraded': '2023-10-26T05:48:23.696971+0000', 'last_fullsized': '2023-10-26T05:48:23.696971+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:48:28.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-10-26T05:48:28.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.687957+0000', 'last_change': '2023-10-26T05:48:23.687957+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.687957+0000', 'last_undegraded': '2023-10-26T05:48:23.687957+0000', 'last_fullsized': '2023-10-26T05:48:23.687957+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-10-26T05:48:28.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.689977+0000', 'last_change': '2023-10-26T05:48:23.689977+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.689977+0000', 'last_undegraded': '2023-10-26T05:48:23.689977+0000', 'last_fullsized': '2023-10-26T05:48:23.689977+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-10-26T05:48:28.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.687100+0000', 'last_change': '2023-10-26T05:48:23.687100+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.687100+0000', 'last_undegraded': '2023-10-26T05:48:23.687100+0000', 'last_fullsized': '2023-10-26T05:48:23.687100+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:48:28.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-10-26T05:48:28.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694579+0000', 'last_change': '2023-10-26T05:48:23.694579+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694579+0000', 'last_undegraded': '2023-10-26T05:48:23.694579+0000', 'last_fullsized': '2023-10-26T05:48:23.694579+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-10-26T05:48:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694075+0000', 'last_change': '2023-10-26T05:48:23.694075+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694075+0000', 'last_undegraded': '2023-10-26T05:48:23.694075+0000', 'last_fullsized': '2023-10-26T05:48:23.694075+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-10-26T05:48:28.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.695252+0000', 'last_change': '2023-10-26T05:48:23.695252+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.695252+0000', 'last_undegraded': '2023-10-26T05:48:23.695252+0000', 'last_fullsized': '2023-10-26T05:48:23.695252+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-10-26T05:48:28.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694168+0000', 'last_change': '2023-10-26T05:48:23.694168+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694168+0000', 'last_undegraded': '2023-10-26T05:48:23.694168+0000', 'last_fullsized': '2023-10-26T05:48:23.694168+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-10-26T05:48:28.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.693873+0000', 'last_change': '2023-10-26T05:48:23.693873+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.693873+0000', 'last_undegraded': '2023-10-26T05:48:23.693873+0000', 'last_fullsized': '2023-10-26T05:48:23.693873+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-10-26T05:48:28.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.694854+0000', 'last_change': '2023-10-26T05:48:23.694854+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.694854+0000', 'last_undegraded': '2023-10-26T05:48:23.694854+0000', 'last_fullsized': '2023-10-26T05:48:23.694854+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:48:28.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-10-26T05:48:28.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.693231+0000', 'last_change': '2023-10-26T05:48:23.693231+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.693231+0000', 'last_undegraded': '2023-10-26T05:48:23.693231+0000', 'last_fullsized': '2023-10-26T05:48:23.693231+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:48:28.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-10-26T05:48:28.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.688037+0000', 'last_change': '2023-10-26T05:48:23.688037+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.688037+0000', 'last_undegraded': '2023-10-26T05:48:23.688037+0000', 'last_fullsized': '2023-10-26T05:48:23.688037+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:48:28.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-10-26T05:48:28.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.696097+0000', 'last_change': '2023-10-26T05:48:23.696097+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.696097+0000', 'last_undegraded': '2023-10-26T05:48:23.696097+0000', 'last_fullsized': '2023-10-26T05:48:23.696097+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:48:28.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-10-26T05:48:28.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.690554+0000', 'last_change': '2023-10-26T05:48:23.690554+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.690554+0000', 'last_undegraded': '2023-10-26T05:48:23.690554+0000', 'last_fullsized': '2023-10-26T05:48:23.690554+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:48:28.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-10-26T05:48:28.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.690693+0000', 'last_change': '2023-10-26T05:48:23.690693+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.690693+0000', 'last_undegraded': '2023-10-26T05:48:23.690693+0000', 'last_fullsized': '2023-10-26T05:48:23.690693+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:48:28.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-10-26T05:48:28.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.684207+0000', 'last_change': '2023-10-26T05:48:23.684207+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.684207+0000', 'last_undegraded': '2023-10-26T05:48:23.684207+0000', 'last_fullsized': '2023-10-26T05:48:23.684207+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:48:28.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-10-26T05:48:28.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:48:23.692904+0000', 'last_change': '2023-10-26T05:48:23.692904+0000', 'last_active': '2023-10-26T05:48:23.662302+0000', 'last_peered': '2023-10-26T05:48:23.662302+0000', 'last_clean': '2023-10-26T05:48:23.662302+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:48:23.692904+0000', 'last_undegraded': '2023-10-26T05:48:23.692904+0000', 'last_fullsized': '2023-10-26T05:48:23.692904+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'last_clean_scrub_stamp': '2023-10-26T05:48:23.662302+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:48:28.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T05:48:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 -- 172.21.15.71:0/1400873653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d8413c2f0 msgr2=0x7f1d840b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/1400873653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d8413c2f0 0x7f1d840b2350 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1d840ac6e0 tx=0x7f1d7401c880 comp rx=0 tx=0).stop 2023-10-26T05:48:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 -- 172.21.15.71:0/1400873653 shutdown_connections 2023-10-26T05:48:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/1400873653 >> v2:172.21.15.86:3300/0 conn(0x7f1d840cb1e0 0x7f1d84159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/1400873653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d8413c2f0 0x7f1d840b2350 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 -- 172.21.15.71:0/1400873653 >> 172.21.15.71:0/1400873653 conn(0x7f1d84069770 msgr2=0x7f1d84069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 -- 172.21.15.71:0/1400873653 shutdown_connections 2023-10-26T05:48:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.553+0000 7f1d88e0b640 1 -- 172.21.15.71:0/1400873653 wait complete. 2023-10-26T05:48:28.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 Processor -- start 2023-10-26T05:48:28.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- start start 2023-10-26T05:48:28.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d840b2890 0x7f1d840b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 0x7f1d841328c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d8413c2f0 0x7f1d841352a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840aa8e0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d84132e00 con 0x7f1d840b2890 2023-10-26T05:48:28.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d840aaaf0 con 0x7f1d8413c2f0 2023-10-26T05:48:28.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 0x7f1d841328c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 0x7f1d841328c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58412/0 (socket says 172.21.15.71:58412) 2023-10-26T05:48:28.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 -- 172.21.15.71:0/754032339 learned_addr learned my addr 172.21.15.71:0/754032339 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:28.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 -- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d8413c2f0 msgr2=0x7f1d841352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d8413c2f0 0x7f1d841352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 -- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d840b2890 msgr2=0x7f1d840b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d840b2890 0x7f1d840b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 -- 172.21.15.71:0/754032339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009560 con 0x7f1d840cb1e0 2023-10-26T05:48:28.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d79d9b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 0x7f1d841328c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c450 tx=0x7f1d6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:28.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0052d0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d6c005450 con 0x7f1d840cb1e0 2023-10-26T05:48:28.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d840aadd0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012890 con 0x7f1d840cb1e0 2023-10-26T05:48:28.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d840ab390 con 0x7f1d840cb1e0 2023-10-26T05:48:28.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d84066b10 con 0x7f1d840cb1e0 2023-10-26T05:48:28.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1d6c003620 con 0x7f1d840cb1e0 2023-10-26T05:48:28.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d7b7fe640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.557+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d6c083ab0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.561+0000 7f1d8259c640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.561+0000 7f1d8259c640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d740af000 tx=0x7f1d74019620 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:28.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.561+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1d6c050fc0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.769+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1d840721e0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d7b7fe640 1 -- 172.21.15.71:0/754032339 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1187 (secure 0 0 0) 0x7f1d6c0545e0 con 0x7f1d840cb1e0 2023-10-26T05:48:28.776 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:28.776 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1698299229,"num_in_osds":8,"osd_in_since":1698299226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2261337,"bytes_avail":1597738663,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":12,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5280}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-10-26T05:48:17.430955+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-26T05:48:28.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 msgr2=0x7f1d64068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d740af000 tx=0x7f1d74019620 comp rx=0 tx=0).stop 2023-10-26T05:48:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 msgr2=0x7f1d841328c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 0x7f1d841328c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c450 tx=0x7f1d6c00c910 comp rx=0 tx=0).stop 2023-10-26T05:48:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 shutdown_connections 2023-10-26T05:48:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d8413c2f0 0x7f1d841352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d64066470 0x7f1d64068930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d840cb1e0 0x7f1d841328c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 --2- 172.21.15.71:0/754032339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d840b2890 0x7f1d840b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 >> 172.21.15.71:0/754032339 conn(0x7f1d84069770 msgr2=0x7f1d840ad9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:28.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 shutdown_connections 2023-10-26T05:48:28.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.773+0000 7f1d88e0b640 1 -- 172.21.15.71:0/754032339 wait complete. 2023-10-26T05:48:28.794 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T05:48:28.795 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:48:28.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- 172.21.15.71:0/2649144221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e8015a5a0 msgr2=0x7f6e8006c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 --2- 172.21.15.71:0/2649144221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e8015a5a0 0x7f6e8006c550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009bc0 tx=0x7f6e7801d490 comp rx=0 tx=0).stop 2023-10-26T05:48:28.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- 172.21.15.71:0/2649144221 shutdown_connections 2023-10-26T05:48:28.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 --2- 172.21.15.71:0/2649144221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e8015a5a0 0x7f6e8006c550 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 --2- 172.21.15.71:0/2649144221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e80150520 0x7f6e8006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- 172.21.15.71:0/2649144221 >> 172.21.15.71:0/2649144221 conn(0x7f6e800cd830 msgr2=0x7f6e800cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:28.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- 172.21.15.71:0/2649144221 shutdown_connections 2023-10-26T05:48:28.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- 172.21.15.71:0/2649144221 wait complete. 2023-10-26T05:48:28.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 Processor -- start 2023-10-26T05:48:28.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- start start 2023-10-26T05:48:28.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e80150140 0x7f6e801424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 0x7f6e80140110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e8015a5a0 0x7f6e80140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8014f4b0 con 0x7f6e80150520 2023-10-26T05:48:28.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8014f790 con 0x7f6e80150140 2023-10-26T05:48:28.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e88273640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80151f20 con 0x7f6e8015a5a0 2023-10-26T05:48:28.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 0x7f6e80140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 0x7f6e80140110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58438/0 (socket says 172.21.15.71:58438) 2023-10-26T05:48:28.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 -- 172.21.15.71:0/201352562 learned_addr learned my addr 172.21.15.71:0/201352562 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:28.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 -- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e8015a5a0 msgr2=0x7f6e80140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e8015a5a0 0x7f6e80140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 -- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e80150140 msgr2=0x7f6e801424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:28.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e80150140 0x7f6e801424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:28.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 -- 172.21.15.71:0/201352562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78009820 con 0x7f6e80150520 2023-10-26T05:48:28.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e8580e640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 0x7f6e80140110 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c970 tx=0x7f6e7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:28.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74007c70 con 0x7f6e80150520 2023-10-26T05:48:28.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e7400f450 con 0x7f6e80150520 2023-10-26T05:48:28.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.913+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400f860 con 0x7f6e80150520 2023-10-26T05:48:28.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e80152200 con 0x7f6e80150520 2023-10-26T05:48:28.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e801429f0 con 0x7f6e80150520 2023-10-26T05:48:28.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e80066b10 con 0x7f6e80150520 2023-10-26T05:48:28.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e7400f9c0 con 0x7f6e80150520 2023-10-26T05:48:28.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e7effd640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e6c066470 0x7f6e6c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:28.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e74084700 con 0x7f6e80150520 2023-10-26T05:48:28.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e8600f640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e6c066470 0x7f6e6c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:28.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.917+0000 7f6e8600f640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e6c066470 0x7f6e6c068930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e70007fd0 tx=0x7f6e70007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:28.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:28.921+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e74051c10 con 0x7f6e80150520 2023-10-26T05:48:29.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.069+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e80151da0 con 0x7f6e6c066470 2023-10-26T05:48:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.093+0000 7f6e7effd640 1 -- 172.21.15.71:0/201352562 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278482 (secure 0 0 0) 0x7f6e700050d0 con 0x7f6e6c066470 2023-10-26T05:48:29.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:29.100 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:48:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e6c066470 msgr2=0x7f6e6c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e6c066470 0x7f6e6c068930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6e70007fd0 tx=0x7f6e70007920 comp rx=0 tx=0).stop 2023-10-26T05:48:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 msgr2=0x7f6e80140110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 0x7f6e80140110 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c970 tx=0x7f6e7400ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 shutdown_connections 2023-10-26T05:48:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e8015a5a0 0x7f6e80140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e6c066470 0x7f6e6c068930 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e80150520 0x7f6e80140110 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 --2- 172.21.15.71:0/201352562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e80150140 0x7f6e801424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 >> 172.21.15.71:0/201352562 conn(0x7f6e800cd830 msgr2=0x7f6e8015d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 shutdown_connections 2023-10-26T05:48:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:29.097+0000 7f6e88273640 1 -- 172.21.15.71:0/201352562 wait complete. 2023-10-26T05:48:29.118 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-10-26T05:48:27.433648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560291,"statfs":{"total":1600000000,"available":1597738663,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700234+0000","last_change":"2023-10-26T05:48:23.700234+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700234+0000","last_undegraded":"2023-10-26T05:48:23.700234+0000","last_fullsized":"2023-10-26T05:48:23.700234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694918+0000","last_change":"2023-10-26T05:48:23.694918+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694918+0000","last_undegraded":"2023-10-26T05:48:23.694918+0000","last_fullsized":"2023-10-26T05:48:23.694918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695709+0000","last_change":"2023-10-26T05:48:23.695709+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695709+0000","last_undegraded":"2023-10-26T05:48:23.695709+0000","last_fullsized":"2023-10-26T05:48:23.695709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688038+0000","last_change":"2023-10-26T05:48:23.688038+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688038+0000","last_undegraded":"2023-10-26T05:48:23.688038+0000","last_fullsized":"2023-10-26T05:48:23.688038+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694048+0000","last_change":"2023-10-26T05:48:23.694048+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694048+0000","last_undegraded":"2023-10-26T05:48:23.694048+0000","last_fullsized":"2023-10-26T05:48:23.694048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694220+0000","last_change":"2023-10-26T05:48:23.694220+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694220+0000","last_undegraded":"2023-10-26T05:48:23.694220+0000","last_fullsized":"2023-10-26T05:48:23.694220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696282+0000","last_change":"2023-10-26T05:48:23.696282+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696282+0000","last_undegraded":"2023-10-26T05:48:23.696282+0000","last_fullsized":"2023-10-26T05:48:23.696282+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700861+0000","last_change":"2023-10-26T05:48:23.700861+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700861+0000","last_undegraded":"2023-10-26T05:48:23.700861+0000","last_fullsized":"2023-10-26T05:48:23.700861+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687341+0000","last_change":"2023-10-26T05:48:23.687341+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687341+0000","last_undegraded":"2023-10-26T05:48:23.687341+0000","last_fullsized":"2023-10-26T05:48:23.687341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688537+0000","last_change":"2023-10-26T05:48:23.688537+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688537+0000","last_undegraded":"2023-10-26T05:48:23.688537+0000","last_fullsized":"2023-10-26T05:48:23.688537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.697056+0000","last_change":"2023-10-26T05:48:23.697056+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.697056+0000","last_undegraded":"2023-10-26T05:48:23.697056+0000","last_fullsized":"2023-10-26T05:48:23.697056+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694710+0000","last_change":"2023-10-26T05:48:23.694710+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694710+0000","last_undegraded":"2023-10-26T05:48:23.694710+0000","last_fullsized":"2023-10-26T05:48:23.694710+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695640+0000","last_change":"2023-10-26T05:48:23.695640+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695640+0000","last_undegraded":"2023-10-26T05:48:23.695640+0000","last_fullsized":"2023-10-26T05:48:23.695640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692044+0000","last_change":"2023-10-26T05:48:23.692044+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692044+0000","last_undegraded":"2023-10-26T05:48:23.692044+0000","last_fullsized":"2023-10-26T05:48:23.692044+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692542+0000","last_change":"2023-10-26T05:48:23.692542+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692542+0000","last_undegraded":"2023-10-26T05:48:23.692542+0000","last_fullsized":"2023-10-26T05:48:23.692542+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696259+0000","last_change":"2023-10-26T05:48:23.696259+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696259+0000","last_undegraded":"2023-10-26T05:48:23.696259+0000","last_fullsized":"2023-10-26T05:48:23.696259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.706851+0000","last_change":"2023-10-26T05:48:23.706851+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.706851+0000","last_undegraded":"2023-10-26T05:48:23.706851+0000","last_fullsized":"2023-10-26T05:48:23.706851+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695326+0000","last_change":"2023-10-26T05:48:23.695326+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695326+0000","last_undegraded":"2023-10-26T05:48:23.695326+0000","last_fullsized":"2023-10-26T05:48:23.695326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691624+0000","last_change":"2023-10-26T05:48:23.691624+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691624+0000","last_undegraded":"2023-10-26T05:48:23.691624+0000","last_fullsized":"2023-10-26T05:48:23.691624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693579+0000","last_change":"2023-10-26T05:48:23.693579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693579+0000","last_undegraded":"2023-10-26T05:48:23.693579+0000","last_fullsized":"2023-10-26T05:48:23.693579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685254+0000","last_change":"2023-10-26T05:48:23.685254+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685254+0000","last_undegraded":"2023-10-26T05:48:23.685254+0000","last_fullsized":"2023-10-26T05:48:23.685254+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.685503+0000","last_change":"2023-10-26T05:48:23.685503+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.685503+0000","last_undegraded":"2023-10-26T05:48:23.685503+0000","last_fullsized":"2023-10-26T05:48:23.685503+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691108+0000","last_change":"2023-10-26T05:48:23.691108+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691108+0000","last_undegraded":"2023-10-26T05:48:23.691108+0000","last_fullsized":"2023-10-26T05:48:23.691108+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690897+0000","last_change":"2023-10-26T05:48:23.690897+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690897+0000","last_undegraded":"2023-10-26T05:48:23.690897+0000","last_fullsized":"2023-10-26T05:48:23.690897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688241+0000","last_change":"2023-10-26T05:48:23.688241+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688241+0000","last_undegraded":"2023-10-26T05:48:23.688241+0000","last_fullsized":"2023-10-26T05:48:23.688241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692899+0000","last_change":"2023-10-26T05:48:23.692899+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692899+0000","last_undegraded":"2023-10-26T05:48:23.692899+0000","last_fullsized":"2023-10-26T05:48:23.692899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687469+0000","last_change":"2023-10-26T05:48:23.687469+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687469+0000","last_undegraded":"2023-10-26T05:48:23.687469+0000","last_fullsized":"2023-10-26T05:48:23.687469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693430+0000","last_change":"2023-10-26T05:48:23.693430+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693430+0000","last_undegraded":"2023-10-26T05:48:23.693430+0000","last_fullsized":"2023-10-26T05:48:23.693430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.686546+0000","last_change":"2023-10-26T05:48:23.686546+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.686546+0000","last_undegraded":"2023-10-26T05:48:23.686546+0000","last_fullsized":"2023-10-26T05:48:23.686546+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694718+0000","last_change":"2023-10-26T05:48:23.694718+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694718+0000","last_undegraded":"2023-10-26T05:48:23.694718+0000","last_fullsized":"2023-10-26T05:48:23.694718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.699413+0000","last_change":"2023-10-26T05:48:23.699413+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.699413+0000","last_undegraded":"2023-10-26T05:48:23.699413+0000","last_fullsized":"2023-10-26T05:48:23.699413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.703535+0000","last_change":"2023-10-26T05:48:23.703535+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.703535+0000","last_undegraded":"2023-10-26T05:48:23.703535+0000","last_fullsized":"2023-10-26T05:48:23.703535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688409+0000","last_change":"2023-10-26T05:48:23.688409+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688409+0000","last_undegraded":"2023-10-26T05:48:23.688409+0000","last_fullsized":"2023-10-26T05:48:23.688409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.711284+0000","last_change":"2023-10-26T05:48:23.711070+0000","last_active":"2023-10-26T05:48:23.711284+0000","last_peered":"2023-10-26T05:48:23.711284+0000","last_clean":"2023-10-26T05:48:23.711284+0000","last_became_active":"2023-10-26T05:48:23.710421+0000","last_became_peered":"2023-10-26T05:48:23.710421+0000","last_unstale":"2023-10-26T05:48:23.711284+0000","last_undegraded":"2023-10-26T05:48:23.711284+0000","last_fullsized":"2023-10-26T05:48:23.711284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695820+0000","last_change":"2023-10-26T05:48:23.695820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695820+0000","last_undegraded":"2023-10-26T05:48:23.695820+0000","last_fullsized":"2023-10-26T05:48:23.695820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703916+0000","last_change":"2023-10-26T05:48:23.703810+0000","last_active":"2023-10-26T05:48:23.703916+0000","last_peered":"2023-10-26T05:48:23.703916+0000","last_clean":"2023-10-26T05:48:23.703916+0000","last_became_active":"2023-10-26T05:48:23.703500+0000","last_became_peered":"2023-10-26T05:48:23.703500+0000","last_unstale":"2023-10-26T05:48:23.703916+0000","last_undegraded":"2023-10-26T05:48:23.703916+0000","last_fullsized":"2023-10-26T05:48:23.703916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.683637+0000","last_change":"2023-10-26T05:48:23.683637+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.683637+0000","last_undegraded":"2023-10-26T05:48:23.683637+0000","last_fullsized":"2023-10-26T05:48:23.683637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690187+0000","last_change":"2023-10-26T05:48:23.690022+0000","last_active":"2023-10-26T05:48:23.690187+0000","last_peered":"2023-10-26T05:48:23.690187+0000","last_clean":"2023-10-26T05:48:23.690187+0000","last_became_active":"2023-10-26T05:48:23.689605+0000","last_became_peered":"2023-10-26T05:48:23.689605+0000","last_unstale":"2023-10-26T05:48:23.690187+0000","last_undegraded":"2023-10-26T05:48:23.690187+0000","last_fullsized":"2023-10-26T05:48:23.690187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687820+0000","last_change":"2023-10-26T05:48:23.687820+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687820+0000","last_undegraded":"2023-10-26T05:48:23.687820+0000","last_fullsized":"2023-10-26T05:48:23.687820+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701416+0000","last_change":"2023-10-26T05:48:23.701287+0000","last_active":"2023-10-26T05:48:23.701416+0000","last_peered":"2023-10-26T05:48:23.701416+0000","last_clean":"2023-10-26T05:48:23.701416+0000","last_became_active":"2023-10-26T05:48:23.700949+0000","last_became_peered":"2023-10-26T05:48:23.700949+0000","last_unstale":"2023-10-26T05:48:23.701416+0000","last_undegraded":"2023-10-26T05:48:23.701416+0000","last_fullsized":"2023-10-26T05:48:23.701416+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.700105+0000","last_change":"2023-10-26T05:48:23.700105+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.700105+0000","last_undegraded":"2023-10-26T05:48:23.700105+0000","last_fullsized":"2023-10-26T05:48:23.700105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691137+0000","last_change":"2023-10-26T05:48:23.690572+0000","last_active":"2023-10-26T05:48:23.691137+0000","last_peered":"2023-10-26T05:48:23.691137+0000","last_clean":"2023-10-26T05:48:23.691137+0000","last_became_active":"2023-10-26T05:48:23.689616+0000","last_became_peered":"2023-10-26T05:48:23.689616+0000","last_unstale":"2023-10-26T05:48:23.691137+0000","last_undegraded":"2023-10-26T05:48:23.691137+0000","last_fullsized":"2023-10-26T05:48:23.691137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:36:17.063916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693976+0000","last_change":"2023-10-26T05:48:23.693976+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693976+0000","last_undegraded":"2023-10-26T05:48:23.693976+0000","last_fullsized":"2023-10-26T05:48:23.693976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701389+0000","last_change":"2023-10-26T05:48:23.701241+0000","last_active":"2023-10-26T05:48:23.701389+0000","last_peered":"2023-10-26T05:48:23.701389+0000","last_clean":"2023-10-26T05:48:23.701389+0000","last_became_active":"2023-10-26T05:48:23.699629+0000","last_became_peered":"2023-10-26T05:48:23.699629+0000","last_unstale":"2023-10-26T05:48:23.701389+0000","last_undegraded":"2023-10-26T05:48:23.701389+0000","last_fullsized":"2023-10-26T05:48:23.701389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:24:45.082594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688715+0000","last_change":"2023-10-26T05:48:23.688715+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688715+0000","last_undegraded":"2023-10-26T05:48:23.688715+0000","last_fullsized":"2023-10-26T05:48:23.688715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.700492+0000","last_change":"2023-10-26T05:48:23.700412+0000","last_active":"2023-10-26T05:48:23.700492+0000","last_peered":"2023-10-26T05:48:23.700492+0000","last_clean":"2023-10-26T05:48:23.700492+0000","last_became_active":"2023-10-26T05:48:23.698966+0000","last_became_peered":"2023-10-26T05:48:23.698966+0000","last_unstale":"2023-10-26T05:48:23.700492+0000","last_undegraded":"2023-10-26T05:48:23.700492+0000","last_fullsized":"2023-10-26T05:48:23.700492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687623+0000","last_change":"2023-10-26T05:48:23.687623+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687623+0000","last_undegraded":"2023-10-26T05:48:23.687623+0000","last_fullsized":"2023-10-26T05:48:23.687623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715611+0000","last_change":"2023-10-26T05:48:23.715497+0000","last_active":"2023-10-26T05:48:23.715611+0000","last_peered":"2023-10-26T05:48:23.715611+0000","last_clean":"2023-10-26T05:48:23.715611+0000","last_became_active":"2023-10-26T05:48:23.714277+0000","last_became_peered":"2023-10-26T05:48:23.714277+0000","last_unstale":"2023-10-26T05:48:23.715611+0000","last_undegraded":"2023-10-26T05:48:23.715611+0000","last_fullsized":"2023-10-26T05:48:23.715611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:16:54.930577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691328+0000","last_change":"2023-10-26T05:48:23.691328+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691328+0000","last_undegraded":"2023-10-26T05:48:23.691328+0000","last_fullsized":"2023-10-26T05:48:23.691328+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705835+0000","last_change":"2023-10-26T05:48:23.705748+0000","last_active":"2023-10-26T05:48:23.705835+0000","last_peered":"2023-10-26T05:48:23.705835+0000","last_clean":"2023-10-26T05:48:23.705835+0000","last_became_active":"2023-10-26T05:48:23.705389+0000","last_became_peered":"2023-10-26T05:48:23.705389+0000","last_unstale":"2023-10-26T05:48:23.705835+0000","last_undegraded":"2023-10-26T05:48:23.705835+0000","last_fullsized":"2023-10-26T05:48:23.705835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692498+0000","last_change":"2023-10-26T05:48:23.692498+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692498+0000","last_undegraded":"2023-10-26T05:48:23.692498+0000","last_fullsized":"2023-10-26T05:48:23.692498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.696945+0000","last_change":"2023-10-26T05:48:23.696857+0000","last_active":"2023-10-26T05:48:23.696945+0000","last_peered":"2023-10-26T05:48:23.696945+0000","last_clean":"2023-10-26T05:48:23.696945+0000","last_became_active":"2023-10-26T05:48:23.696471+0000","last_became_peered":"2023-10-26T05:48:23.696471+0000","last_unstale":"2023-10-26T05:48:23.696945+0000","last_undegraded":"2023-10-26T05:48:23.696945+0000","last_fullsized":"2023-10-26T05:48:23.696945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691173+0000","last_change":"2023-10-26T05:48:23.691173+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691173+0000","last_undegraded":"2023-10-26T05:48:23.691173+0000","last_fullsized":"2023-10-26T05:48:23.691173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.709783+0000","last_change":"2023-10-26T05:48:23.709670+0000","last_active":"2023-10-26T05:48:23.709783+0000","last_peered":"2023-10-26T05:48:23.709783+0000","last_clean":"2023-10-26T05:48:23.709783+0000","last_became_active":"2023-10-26T05:48:23.709170+0000","last_became_peered":"2023-10-26T05:48:23.709170+0000","last_unstale":"2023-10-26T05:48:23.709783+0000","last_undegraded":"2023-10-26T05:48:23.709783+0000","last_fullsized":"2023-10-26T05:48:23.709783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:03.464765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692408+0000","last_change":"2023-10-26T05:48:23.692408+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692408+0000","last_undegraded":"2023-10-26T05:48:23.692408+0000","last_fullsized":"2023-10-26T05:48:23.692408+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.690158+0000","last_change":"2023-10-26T05:48:23.689973+0000","last_active":"2023-10-26T05:48:23.690158+0000","last_peered":"2023-10-26T05:48:23.690158+0000","last_clean":"2023-10-26T05:48:23.690158+0000","last_became_active":"2023-10-26T05:48:23.689550+0000","last_became_peered":"2023-10-26T05:48:23.689550+0000","last_unstale":"2023-10-26T05:48:23.690158+0000","last_undegraded":"2023-10-26T05:48:23.690158+0000","last_fullsized":"2023-10-26T05:48:23.690158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.768450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692771+0000","last_change":"2023-10-26T05:48:23.692771+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692771+0000","last_undegraded":"2023-10-26T05:48:23.692771+0000","last_fullsized":"2023-10-26T05:48:23.692771+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715379+0000","last_change":"2023-10-26T05:48:23.715244+0000","last_active":"2023-10-26T05:48:23.715379+0000","last_peered":"2023-10-26T05:48:23.715379+0000","last_clean":"2023-10-26T05:48:23.715379+0000","last_became_active":"2023-10-26T05:48:23.713621+0000","last_became_peered":"2023-10-26T05:48:23.713621+0000","last_unstale":"2023-10-26T05:48:23.715379+0000","last_undegraded":"2023-10-26T05:48:23.715379+0000","last_fullsized":"2023-10-26T05:48:23.715379+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:52.471566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.691733+0000","last_change":"2023-10-26T05:48:23.691733+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.691733+0000","last_undegraded":"2023-10-26T05:48:23.691733+0000","last_fullsized":"2023-10-26T05:48:23.691733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.704334+0000","last_change":"2023-10-26T05:48:23.704259+0000","last_active":"2023-10-26T05:48:23.704334+0000","last_peered":"2023-10-26T05:48:23.704334+0000","last_clean":"2023-10-26T05:48:23.704334+0000","last_became_active":"2023-10-26T05:48:23.703997+0000","last_became_peered":"2023-10-26T05:48:23.703997+0000","last_unstale":"2023-10-26T05:48:23.704334+0000","last_undegraded":"2023-10-26T05:48:23.704334+0000","last_fullsized":"2023-10-26T05:48:23.704334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694411+0000","last_change":"2023-10-26T05:48:23.694411+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694411+0000","last_undegraded":"2023-10-26T05:48:23.694411+0000","last_fullsized":"2023-10-26T05:48:23.694411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.706547+0000","last_change":"2023-10-26T05:48:23.706487+0000","last_active":"2023-10-26T05:48:23.706547+0000","last_peered":"2023-10-26T05:48:23.706547+0000","last_clean":"2023-10-26T05:48:23.706547+0000","last_became_active":"2023-10-26T05:48:23.706232+0000","last_became_peered":"2023-10-26T05:48:23.706232+0000","last_unstale":"2023-10-26T05:48:23.706547+0000","last_undegraded":"2023-10-26T05:48:23.706547+0000","last_fullsized":"2023-10-26T05:48:23.706547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696971+0000","last_change":"2023-10-26T05:48:23.696971+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696971+0000","last_undegraded":"2023-10-26T05:48:23.696971+0000","last_fullsized":"2023-10-26T05:48:23.696971+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.689358+0000","last_change":"2023-10-26T05:48:23.688956+0000","last_active":"2023-10-26T05:48:23.689358+0000","last_peered":"2023-10-26T05:48:23.689358+0000","last_clean":"2023-10-26T05:48:23.689358+0000","last_became_active":"2023-10-26T05:48:23.688311+0000","last_became_peered":"2023-10-26T05:48:23.688311+0000","last_unstale":"2023-10-26T05:48:23.689358+0000","last_undegraded":"2023-10-26T05:48:23.689358+0000","last_fullsized":"2023-10-26T05:48:23.689358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:52.984869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687957+0000","last_change":"2023-10-26T05:48:23.687957+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687957+0000","last_undegraded":"2023-10-26T05:48:23.687957+0000","last_fullsized":"2023-10-26T05:48:23.687957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.688670+0000","last_change":"2023-10-26T05:48:23.688351+0000","last_active":"2023-10-26T05:48:23.688670+0000","last_peered":"2023-10-26T05:48:23.688670+0000","last_clean":"2023-10-26T05:48:23.688670+0000","last_became_active":"2023-10-26T05:48:23.687015+0000","last_became_peered":"2023-10-26T05:48:23.687015+0000","last_unstale":"2023-10-26T05:48:23.688670+0000","last_undegraded":"2023-10-26T05:48:23.688670+0000","last_fullsized":"2023-10-26T05:48:23.688670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.689977+0000","last_change":"2023-10-26T05:48:23.689977+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.689977+0000","last_undegraded":"2023-10-26T05:48:23.689977+0000","last_fullsized":"2023-10-26T05:48:23.689977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.715055+0000","last_change":"2023-10-26T05:48:23.714853+0000","last_active":"2023-10-26T05:48:23.715055+0000","last_peered":"2023-10-26T05:48:23.715055+0000","last_clean":"2023-10-26T05:48:23.715055+0000","last_became_active":"2023-10-26T05:48:23.712919+0000","last_became_peered":"2023-10-26T05:48:23.712919+0000","last_unstale":"2023-10-26T05:48:23.715055+0000","last_undegraded":"2023-10-26T05:48:23.715055+0000","last_fullsized":"2023-10-26T05:48:23.715055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:48.260866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.687100+0000","last_change":"2023-10-26T05:48:23.687100+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.687100+0000","last_undegraded":"2023-10-26T05:48:23.687100+0000","last_fullsized":"2023-10-26T05:48:23.687100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703330+0000","last_change":"2023-10-26T05:48:23.703179+0000","last_active":"2023-10-26T05:48:23.703330+0000","last_peered":"2023-10-26T05:48:23.703330+0000","last_clean":"2023-10-26T05:48:23.703330+0000","last_became_active":"2023-10-26T05:48:23.702603+0000","last_became_peered":"2023-10-26T05:48:23.702603+0000","last_unstale":"2023-10-26T05:48:23.703330+0000","last_undegraded":"2023-10-26T05:48:23.703330+0000","last_fullsized":"2023-10-26T05:48:23.703330+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:10.226049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694579+0000","last_change":"2023-10-26T05:48:23.694579+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694579+0000","last_undegraded":"2023-10-26T05:48:23.694579+0000","last_fullsized":"2023-10-26T05:48:23.694579+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.691086+0000","last_change":"2023-10-26T05:48:23.690214+0000","last_active":"2023-10-26T05:48:23.691086+0000","last_peered":"2023-10-26T05:48:23.691086+0000","last_clean":"2023-10-26T05:48:23.691086+0000","last_became_active":"2023-10-26T05:48:23.688325+0000","last_became_peered":"2023-10-26T05:48:23.688325+0000","last_unstale":"2023-10-26T05:48:23.691086+0000","last_undegraded":"2023-10-26T05:48:23.691086+0000","last_fullsized":"2023-10-26T05:48:23.691086+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:55.310630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694075+0000","last_change":"2023-10-26T05:48:23.694075+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694075+0000","last_undegraded":"2023-10-26T05:48:23.694075+0000","last_fullsized":"2023-10-26T05:48:23.694075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692061+0000","last_change":"2023-10-26T05:48:23.691997+0000","last_active":"2023-10-26T05:48:23.692061+0000","last_peered":"2023-10-26T05:48:23.692061+0000","last_clean":"2023-10-26T05:48:23.692061+0000","last_became_active":"2023-10-26T05:48:23.691414+0000","last_became_peered":"2023-10-26T05:48:23.691414+0000","last_unstale":"2023-10-26T05:48:23.692061+0000","last_undegraded":"2023-10-26T05:48:23.692061+0000","last_fullsized":"2023-10-26T05:48:23.692061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.695252+0000","last_change":"2023-10-26T05:48:23.695252+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.695252+0000","last_undegraded":"2023-10-26T05:48:23.695252+0000","last_fullsized":"2023-10-26T05:48:23.695252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705798+0000","last_change":"2023-10-26T05:48:23.705656+0000","last_active":"2023-10-26T05:48:23.705798+0000","last_peered":"2023-10-26T05:48:23.705798+0000","last_clean":"2023-10-26T05:48:23.705798+0000","last_became_active":"2023-10-26T05:48:23.705077+0000","last_became_peered":"2023-10-26T05:48:23.705077+0000","last_unstale":"2023-10-26T05:48:23.705798+0000","last_undegraded":"2023-10-26T05:48:23.705798+0000","last_fullsized":"2023-10-26T05:48:23.705798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694168+0000","last_change":"2023-10-26T05:48:23.694168+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694168+0000","last_undegraded":"2023-10-26T05:48:23.694168+0000","last_fullsized":"2023-10-26T05:48:23.694168+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712295+0000","last_change":"2023-10-26T05:48:23.712097+0000","last_active":"2023-10-26T05:48:23.712295+0000","last_peered":"2023-10-26T05:48:23.712295+0000","last_clean":"2023-10-26T05:48:23.712295+0000","last_became_active":"2023-10-26T05:48:23.704476+0000","last_became_peered":"2023-10-26T05:48:23.704476+0000","last_unstale":"2023-10-26T05:48:23.712295+0000","last_undegraded":"2023-10-26T05:48:23.712295+0000","last_fullsized":"2023-10-26T05:48:23.712295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:04.497346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693873+0000","last_change":"2023-10-26T05:48:23.693873+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693873+0000","last_undegraded":"2023-10-26T05:48:23.693873+0000","last_fullsized":"2023-10-26T05:48:23.693873+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.708766+0000","last_change":"2023-10-26T05:48:23.708587+0000","last_active":"2023-10-26T05:48:23.708766+0000","last_peered":"2023-10-26T05:48:23.708766+0000","last_clean":"2023-10-26T05:48:23.708766+0000","last_became_active":"2023-10-26T05:48:23.707969+0000","last_became_peered":"2023-10-26T05:48:23.707969+0000","last_unstale":"2023-10-26T05:48:23.708766+0000","last_undegraded":"2023-10-26T05:48:23.708766+0000","last_fullsized":"2023-10-26T05:48:23.708766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.694854+0000","last_change":"2023-10-26T05:48:23.694854+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.694854+0000","last_undegraded":"2023-10-26T05:48:23.694854+0000","last_fullsized":"2023-10-26T05:48:23.694854+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705560+0000","last_change":"2023-10-26T05:48:23.705491+0000","last_active":"2023-10-26T05:48:23.705560+0000","last_peered":"2023-10-26T05:48:23.705560+0000","last_clean":"2023-10-26T05:48:23.705560+0000","last_became_active":"2023-10-26T05:48:23.705095+0000","last_became_peered":"2023-10-26T05:48:23.705095+0000","last_unstale":"2023-10-26T05:48:23.705560+0000","last_undegraded":"2023-10-26T05:48:23.705560+0000","last_fullsized":"2023-10-26T05:48:23.705560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.693231+0000","last_change":"2023-10-26T05:48:23.693231+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.693231+0000","last_undegraded":"2023-10-26T05:48:23.693231+0000","last_fullsized":"2023-10-26T05:48:23.693231+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.692208+0000","last_change":"2023-10-26T05:48:23.692126+0000","last_active":"2023-10-26T05:48:23.692208+0000","last_peered":"2023-10-26T05:48:23.692208+0000","last_clean":"2023-10-26T05:48:23.692208+0000","last_became_active":"2023-10-26T05:48:23.691722+0000","last_became_peered":"2023-10-26T05:48:23.691722+0000","last_unstale":"2023-10-26T05:48:23.692208+0000","last_undegraded":"2023-10-26T05:48:23.692208+0000","last_fullsized":"2023-10-26T05:48:23.692208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:20.389621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.688037+0000","last_change":"2023-10-26T05:48:23.688037+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.688037+0000","last_undegraded":"2023-10-26T05:48:23.688037+0000","last_fullsized":"2023-10-26T05:48:23.688037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.699555+0000","last_change":"2023-10-26T05:48:23.699301+0000","last_active":"2023-10-26T05:48:23.699555+0000","last_peered":"2023-10-26T05:48:23.699555+0000","last_clean":"2023-10-26T05:48:23.699555+0000","last_became_active":"2023-10-26T05:48:23.698926+0000","last_became_peered":"2023-10-26T05:48:23.698926+0000","last_unstale":"2023-10-26T05:48:23.699555+0000","last_undegraded":"2023-10-26T05:48:23.699555+0000","last_fullsized":"2023-10-26T05:48:23.699555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:49.414179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.696097+0000","last_change":"2023-10-26T05:48:23.696097+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.696097+0000","last_undegraded":"2023-10-26T05:48:23.696097+0000","last_fullsized":"2023-10-26T05:48:23.696097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.697196+0000","last_change":"2023-10-26T05:48:23.697103+0000","last_active":"2023-10-26T05:48:23.697196+0000","last_peered":"2023-10-26T05:48:23.697196+0000","last_clean":"2023-10-26T05:48:23.697196+0000","last_became_active":"2023-10-26T05:48:23.696770+0000","last_became_peered":"2023-10-26T05:48:23.696770+0000","last_unstale":"2023-10-26T05:48:23.697196+0000","last_undegraded":"2023-10-26T05:48:23.697196+0000","last_fullsized":"2023-10-26T05:48:23.697196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690554+0000","last_change":"2023-10-26T05:48:23.690554+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690554+0000","last_undegraded":"2023-10-26T05:48:23.690554+0000","last_fullsized":"2023-10-26T05:48:23.690554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.705457+0000","last_change":"2023-10-26T05:48:23.705357+0000","last_active":"2023-10-26T05:48:23.705457+0000","last_peered":"2023-10-26T05:48:23.705457+0000","last_clean":"2023-10-26T05:48:23.705457+0000","last_became_active":"2023-10-26T05:48:23.704629+0000","last_became_peered":"2023-10-26T05:48:23.704629+0000","last_unstale":"2023-10-26T05:48:23.705457+0000","last_undegraded":"2023-10-26T05:48:23.705457+0000","last_fullsized":"2023-10-26T05:48:23.705457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.690693+0000","last_change":"2023-10-26T05:48:23.690693+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.690693+0000","last_undegraded":"2023-10-26T05:48:23.690693+0000","last_fullsized":"2023-10-26T05:48:23.690693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.701672+0000","last_change":"2023-10-26T05:48:23.701546+0000","last_active":"2023-10-26T05:48:23.701672+0000","last_peered":"2023-10-26T05:48:23.701672+0000","last_clean":"2023-10-26T05:48:23.701672+0000","last_became_active":"2023-10-26T05:48:23.700654+0000","last_became_peered":"2023-10-26T05:48:23.700654+0000","last_unstale":"2023-10-26T05:48:23.701672+0000","last_undegraded":"2023-10-26T05:48:23.701672+0000","last_fullsized":"2023-10-26T05:48:23.701672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:15.235343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.684207+0000","last_change":"2023-10-26T05:48:23.684207+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.684207+0000","last_undegraded":"2023-10-26T05:48:23.684207+0000","last_fullsized":"2023-10-26T05:48:23.684207+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.703737+0000","last_change":"2023-10-26T05:48:23.703600+0000","last_active":"2023-10-26T05:48:23.703737+0000","last_peered":"2023-10-26T05:48:23.703737+0000","last_clean":"2023-10-26T05:48:23.703737+0000","last_became_active":"2023-10-26T05:48:23.703254+0000","last_became_peered":"2023-10-26T05:48:23.703254+0000","last_unstale":"2023-10-26T05:48:23.703737+0000","last_undegraded":"2023-10-26T05:48:23.703737+0000","last_fullsized":"2023-10-26T05:48:23.703737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:35.361883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-10-26T05:48:23.692904+0000","last_change":"2023-10-26T05:48:23.692904+0000","last_active":"2023-10-26T05:48:23.662302+0000","last_peered":"2023-10-26T05:48:23.662302+0000","last_clean":"2023-10-26T05:48:23.662302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:48:23.692904+0000","last_undegraded":"2023-10-26T05:48:23.692904+0000","last_fullsized":"2023-10-26T05:48:23.692904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-26T05:48:23.712540+0000","last_change":"2023-10-26T05:48:23.712299+0000","last_active":"2023-10-26T05:48:23.712540+0000","last_peered":"2023-10-26T05:48:23.712540+0000","last_clean":"2023-10-26T05:48:23.712540+0000","last_became_active":"2023-10-26T05:48:23.711619+0000","last_became_peered":"2023-10-26T05:48:23.711619+0000","last_unstale":"2023-10-26T05:48:23.712540+0000","last_undegraded":"2023-10-26T05:48:23.712540+0000","last_fullsized":"2023-10-26T05:48:23.712540+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:45:09.567350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-26T05:48:22.672770+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:48:22.672770+0000","last_peered":"2023-10-26T05:48:22.672770+0000","last_clean":"2023-10-26T05:48:22.672770+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:48:22.672770+0000","last_undegraded":"2023-10-26T05:48:22.672770+0000","last_fullsized":"2023-10-26T05:48:22.672770+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194606,"statfs":{"total":200000000,"available":199276907,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":1.022},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.05}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.147},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2150000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.292},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.1890000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.2390000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.879}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.98599999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.0409999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":1.0700000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0349999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.76100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.622},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.196}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.087}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.012},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":1.159}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.751},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.24}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.95599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":135,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199861008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.752},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.036}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.77800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199867275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.44}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.99099999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.083},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.1519999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.871}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194600,"statfs":{"total":200000000,"available":199270640,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.63},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.53700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.86699999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.80900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.90400000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.73799999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:48:32.125 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:48:32.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- 172.21.15.71:0/3355130008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe80047d0 msgr2=0x7f5fe8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 --2- 172.21.15.71:0/3355130008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe80047d0 0x7f5fe8004c50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009b60 tx=0x7f5fec01d250 comp rx=0 tx=0).stop 2023-10-26T05:48:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- 172.21.15.71:0/3355130008 shutdown_connections 2023-10-26T05:48:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 --2- 172.21.15.71:0/3355130008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe80047d0 0x7f5fe8004c50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- 172.21.15.71:0/3355130008 >> 172.21.15.71:0/3355130008 conn(0x7f5ffc069770 msgr2=0x7f5ffc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- 172.21.15.71:0/3355130008 shutdown_connections 2023-10-26T05:48:32.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- 172.21.15.71:0/3355130008 wait complete. 2023-10-26T05:48:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 Processor -- start 2023-10-26T05:48:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- start start 2023-10-26T05:48:32.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5fe80047d0 0x7f5ffc0ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ffc14f030 0x7f5ffc14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 0x7f5ffc14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc1370e0 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc0ab0f0 con 0x7f5ffc14f030 2023-10-26T05:48:32.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600477e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc0aa4e0 con 0x7f5fe80047d0 2023-10-26T05:48:32.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 0x7f5ffc14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 0x7f5ffc14f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39382/0 (socket says 172.21.15.71:39382) 2023-10-26T05:48:32.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 -- 172.21.15.71:0/2458892811 learned_addr learned my addr 172.21.15.71:0/2458892811 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:32.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f600251a640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5fe80047d0 0x7f5ffc0ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6001d19640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ffc14f030 0x7f5ffc14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 -- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5fe80047d0 msgr2=0x7f5ffc0ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5fe80047d0 0x7f5ffc0ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 -- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ffc14f030 msgr2=0x7f5ffc14e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.257+0000 7f6002d1b640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ffc14f030 0x7f5ffc14e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f6002d1b640 1 -- 172.21.15.71:0/2458892811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0aa7c0 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f6002d1b640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 0x7f5ffc14f960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc072f10 tx=0x7f5ff4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff401a030 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ff4013a10 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc0aaaa0 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4013e20 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc0aa020 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ffc066b10 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ff401d070 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f5ff2ffd640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5fe0066470 0x7f5fe0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ff401b070 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f600251a640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5fe0066470 0x7f5fe0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.261+0000 7f600251a640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5fe0066470 0x7f5fe0068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc154f80 tx=0x7f5ff8006b10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:32.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.265+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ff4054930 con 0x7f5ffc0a9b70 2023-10-26T05:48:32.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.413+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ffc13d800 con 0x7f5fe0066470 2023-10-26T05:48:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.437+0000 7f5ff2ffd640 1 -- 172.21.15.71:0/2458892811 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283774 (secure 0 0 0) 0x7f5ff8012500 con 0x7f5fe0066470 2023-10-26T05:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:48:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5fe0066470 msgr2=0x7f5fe0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5fe0066470 0x7f5fe0068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc154f80 tx=0x7f5ff8006b10 comp rx=0 tx=0).stop 2023-10-26T05:48:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 msgr2=0x7f5ffc14f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 0x7f5ffc14f960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc072f10 tx=0x7f5ff4011a30 comp rx=0 tx=0).stop 2023-10-26T05:48:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 shutdown_connections 2023-10-26T05:48:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5fe0066470 0x7f5fe0068930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ffc0a9b70 0x7f5ffc14f960 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ffc14f030 0x7f5ffc14e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 --2- 172.21.15.71:0/2458892811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5fe80047d0 0x7f5ffc0ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 >> 172.21.15.71:0/2458892811 conn(0x7f5ffc069770 msgr2=0x7f5ffc0b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 shutdown_connections 2023-10-26T05:48:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.441+0000 7f600477e640 1 -- 172.21.15.71:0/2458892811 wait complete. 2023-10-26T05:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-10-26T05:48:31.435409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2345,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560154,"statfs":{"total":1600000000,"available":1597597908,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.750901"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677670+0000","last_change":"2023-10-26T05:48:24.713217+0000","last_active":"2023-10-26T05:48:25.677670+0000","last_peered":"2023-10-26T05:48:25.677670+0000","last_clean":"2023-10-26T05:48:25.677670+0000","last_became_active":"2023-10-26T05:48:24.712727+0000","last_became_peered":"2023-10-26T05:48:24.712727+0000","last_unstale":"2023-10-26T05:48:25.677670+0000","last_undegraded":"2023-10-26T05:48:25.677670+0000","last_fullsized":"2023-10-26T05:48:25.677670+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:14:23.558708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.017620+0000","last_change":"2023-10-26T05:48:24.697626+0000","last_active":"2023-10-26T05:48:26.017620+0000","last_peered":"2023-10-26T05:48:26.017620+0000","last_clean":"2023-10-26T05:48:26.017620+0000","last_became_active":"2023-10-26T05:48:24.696857+0000","last_became_peered":"2023-10-26T05:48:24.696857+0000","last_unstale":"2023-10-26T05:48:26.017620+0000","last_undegraded":"2023-10-26T05:48:26.017620+0000","last_fullsized":"2023-10-26T05:48:26.017620+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:05.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018991+0000","last_change":"2023-10-26T05:48:24.726711+0000","last_active":"2023-10-26T05:48:26.018991+0000","last_peered":"2023-10-26T05:48:26.018991+0000","last_clean":"2023-10-26T05:48:26.018991+0000","last_became_active":"2023-10-26T05:48:24.724343+0000","last_became_peered":"2023-10-26T05:48:24.724343+0000","last_unstale":"2023-10-26T05:48:26.018991+0000","last_undegraded":"2023-10-26T05:48:26.018991+0000","last_fullsized":"2023-10-26T05:48:26.018991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:03.302780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.673453+0000","last_change":"2023-10-26T05:48:24.692668+0000","last_active":"2023-10-26T05:48:25.673453+0000","last_peered":"2023-10-26T05:48:25.673453+0000","last_clean":"2023-10-26T05:48:25.673453+0000","last_became_active":"2023-10-26T05:48:24.688638+0000","last_became_peered":"2023-10-26T05:48:24.688638+0000","last_unstale":"2023-10-26T05:48:25.673453+0000","last_undegraded":"2023-10-26T05:48:25.673453+0000","last_fullsized":"2023-10-26T05:48:25.673453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:25.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.019157+0000","last_change":"2023-10-26T05:48:24.696369+0000","last_active":"2023-10-26T05:48:26.019157+0000","last_peered":"2023-10-26T05:48:26.019157+0000","last_clean":"2023-10-26T05:48:26.019157+0000","last_became_active":"2023-10-26T05:48:24.695832+0000","last_became_peered":"2023-10-26T05:48:24.695832+0000","last_unstale":"2023-10-26T05:48:26.019157+0000","last_undegraded":"2023-10-26T05:48:26.019157+0000","last_fullsized":"2023-10-26T05:48:26.019157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:30.455725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018386+0000","last_change":"2023-10-26T05:48:24.703594+0000","last_active":"2023-10-26T05:48:26.018386+0000","last_peered":"2023-10-26T05:48:26.018386+0000","last_clean":"2023-10-26T05:48:26.018386+0000","last_became_active":"2023-10-26T05:48:24.702478+0000","last_became_peered":"2023-10-26T05:48:24.702478+0000","last_unstale":"2023-10-26T05:48:26.018386+0000","last_undegraded":"2023-10-26T05:48:26.018386+0000","last_fullsized":"2023-10-26T05:48:26.018386+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:50.223925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676917+0000","last_change":"2023-10-26T05:48:24.719640+0000","last_active":"2023-10-26T05:48:25.676917+0000","last_peered":"2023-10-26T05:48:25.676917+0000","last_clean":"2023-10-26T05:48:25.676917+0000","last_became_active":"2023-10-26T05:48:24.698961+0000","last_became_peered":"2023-10-26T05:48:24.698961+0000","last_unstale":"2023-10-26T05:48:25.676917+0000","last_undegraded":"2023-10-26T05:48:25.676917+0000","last_fullsized":"2023-10-26T05:48:25.676917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:03.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676022+0000","last_change":"2023-10-26T05:48:24.704435+0000","last_active":"2023-10-26T05:48:25.676022+0000","last_peered":"2023-10-26T05:48:25.676022+0000","last_clean":"2023-10-26T05:48:25.676022+0000","last_became_active":"2023-10-26T05:48:24.702815+0000","last_became_peered":"2023-10-26T05:48:24.702815+0000","last_unstale":"2023-10-26T05:48:25.676022+0000","last_undegraded":"2023-10-26T05:48:25.676022+0000","last_fullsized":"2023-10-26T05:48:25.676022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:44:20.030937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787413+0000","last_change":"2023-10-26T05:48:24.688627+0000","last_active":"2023-10-26T05:48:25.787413+0000","last_peered":"2023-10-26T05:48:25.787413+0000","last_clean":"2023-10-26T05:48:25.787413+0000","last_became_active":"2023-10-26T05:48:24.686766+0000","last_became_peered":"2023-10-26T05:48:24.686766+0000","last_unstale":"2023-10-26T05:48:25.787413+0000","last_undegraded":"2023-10-26T05:48:25.787413+0000","last_fullsized":"2023-10-26T05:48:25.787413+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:44:20.030937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.674533+0000","last_change":"2023-10-26T05:48:24.701197+0000","last_active":"2023-10-26T05:48:25.674533+0000","last_peered":"2023-10-26T05:48:25.674533+0000","last_clean":"2023-10-26T05:48:25.674533+0000","last_became_active":"2023-10-26T05:48:24.700010+0000","last_became_peered":"2023-10-26T05:48:24.700010+0000","last_unstale":"2023-10-26T05:48:25.674533+0000","last_undegraded":"2023-10-26T05:48:25.674533+0000","last_fullsized":"2023-10-26T05:48:25.674533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:13.315366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677478+0000","last_change":"2023-10-26T05:48:24.701793+0000","last_active":"2023-10-26T05:48:25.677478+0000","last_peered":"2023-10-26T05:48:25.677478+0000","last_clean":"2023-10-26T05:48:25.677478+0000","last_became_active":"2023-10-26T05:48:24.700846+0000","last_became_peered":"2023-10-26T05:48:24.700846+0000","last_unstale":"2023-10-26T05:48:25.677478+0000","last_undegraded":"2023-10-26T05:48:25.677478+0000","last_fullsized":"2023-10-26T05:48:25.677478+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:22.140651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018235+0000","last_change":"2023-10-26T05:48:24.723992+0000","last_active":"2023-10-26T05:48:26.018235+0000","last_peered":"2023-10-26T05:48:26.018235+0000","last_clean":"2023-10-26T05:48:26.018235+0000","last_became_active":"2023-10-26T05:48:24.717898+0000","last_became_peered":"2023-10-26T05:48:24.717898+0000","last_unstale":"2023-10-26T05:48:26.018235+0000","last_undegraded":"2023-10-26T05:48:26.018235+0000","last_fullsized":"2023-10-26T05:48:26.018235+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:11.342892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016552+0000","last_change":"2023-10-26T05:48:24.684124+0000","last_active":"2023-10-26T05:48:26.016552+0000","last_peered":"2023-10-26T05:48:26.016552+0000","last_clean":"2023-10-26T05:48:26.016552+0000","last_became_active":"2023-10-26T05:48:24.683431+0000","last_became_peered":"2023-10-26T05:48:24.683431+0000","last_unstale":"2023-10-26T05:48:26.016552+0000","last_undegraded":"2023-10-26T05:48:26.016552+0000","last_fullsized":"2023-10-26T05:48:26.016552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:54.000966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.017377+0000","last_change":"2023-10-26T05:48:24.702994+0000","last_active":"2023-10-26T05:48:26.017377+0000","last_peered":"2023-10-26T05:48:26.017377+0000","last_clean":"2023-10-26T05:48:26.017377+0000","last_became_active":"2023-10-26T05:48:24.701653+0000","last_became_peered":"2023-10-26T05:48:24.701653+0000","last_unstale":"2023-10-26T05:48:26.017377+0000","last_undegraded":"2023-10-26T05:48:26.017377+0000","last_fullsized":"2023-10-26T05:48:26.017377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:45.783172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018575+0000","last_change":"2023-10-26T05:48:24.703575+0000","last_active":"2023-10-26T05:48:26.018575+0000","last_peered":"2023-10-26T05:48:26.018575+0000","last_clean":"2023-10-26T05:48:26.018575+0000","last_became_active":"2023-10-26T05:48:24.702945+0000","last_became_peered":"2023-10-26T05:48:24.702945+0000","last_unstale":"2023-10-26T05:48:26.018575+0000","last_undegraded":"2023-10-26T05:48:26.018575+0000","last_fullsized":"2023-10-26T05:48:26.018575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:03.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677792+0000","last_change":"2023-10-26T05:48:24.719274+0000","last_active":"2023-10-26T05:48:25.677792+0000","last_peered":"2023-10-26T05:48:25.677792+0000","last_clean":"2023-10-26T05:48:25.677792+0000","last_became_active":"2023-10-26T05:48:24.705838+0000","last_became_peered":"2023-10-26T05:48:24.705838+0000","last_unstale":"2023-10-26T05:48:25.677792+0000","last_undegraded":"2023-10-26T05:48:25.677792+0000","last_fullsized":"2023-10-26T05:48:25.677792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:38.930048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676214+0000","last_change":"2023-10-26T05:48:24.719167+0000","last_active":"2023-10-26T05:48:25.676214+0000","last_peered":"2023-10-26T05:48:25.676214+0000","last_clean":"2023-10-26T05:48:25.676214+0000","last_became_active":"2023-10-26T05:48:24.697895+0000","last_became_peered":"2023-10-26T05:48:24.697895+0000","last_unstale":"2023-10-26T05:48:25.676214+0000","last_undegraded":"2023-10-26T05:48:25.676214+0000","last_fullsized":"2023-10-26T05:48:25.676214+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:41:40.650960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016839+0000","last_change":"2023-10-26T05:48:24.696299+0000","last_active":"2023-10-26T05:48:26.016839+0000","last_peered":"2023-10-26T05:48:26.016839+0000","last_clean":"2023-10-26T05:48:26.016839+0000","last_became_active":"2023-10-26T05:48:24.695694+0000","last_became_peered":"2023-10-26T05:48:24.695694+0000","last_unstale":"2023-10-26T05:48:26.016839+0000","last_undegraded":"2023-10-26T05:48:26.016839+0000","last_fullsized":"2023-10-26T05:48:26.016839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:38.930048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016368+0000","last_change":"2023-10-26T05:48:24.703464+0000","last_active":"2023-10-26T05:48:26.016368+0000","last_peered":"2023-10-26T05:48:26.016368+0000","last_clean":"2023-10-26T05:48:26.016368+0000","last_became_active":"2023-10-26T05:48:24.702431+0000","last_became_peered":"2023-10-26T05:48:24.702431+0000","last_unstale":"2023-10-26T05:48:26.016368+0000","last_undegraded":"2023-10-26T05:48:26.016368+0000","last_fullsized":"2023-10-26T05:48:26.016368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:03.932031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.019485+0000","last_change":"2023-10-26T05:48:24.700612+0000","last_active":"2023-10-26T05:48:26.019485+0000","last_peered":"2023-10-26T05:48:26.019485+0000","last_clean":"2023-10-26T05:48:26.019485+0000","last_became_active":"2023-10-26T05:48:24.700022+0000","last_became_peered":"2023-10-26T05:48:24.700022+0000","last_unstale":"2023-10-26T05:48:26.019485+0000","last_undegraded":"2023-10-26T05:48:26.019485+0000","last_fullsized":"2023-10-26T05:48:26.019485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:15.723732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.786998+0000","last_change":"2023-10-26T05:48:24.678752+0000","last_active":"2023-10-26T05:48:25.786998+0000","last_peered":"2023-10-26T05:48:25.786998+0000","last_clean":"2023-10-26T05:48:25.786998+0000","last_became_active":"2023-10-26T05:48:24.678128+0000","last_became_peered":"2023-10-26T05:48:24.678128+0000","last_unstale":"2023-10-26T05:48:25.786998+0000","last_undegraded":"2023-10-26T05:48:25.786998+0000","last_fullsized":"2023-10-26T05:48:25.786998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:25.860325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787849+0000","last_change":"2023-10-26T05:48:24.708069+0000","last_active":"2023-10-26T05:48:25.787849+0000","last_peered":"2023-10-26T05:48:25.787849+0000","last_clean":"2023-10-26T05:48:25.787849+0000","last_became_active":"2023-10-26T05:48:24.698397+0000","last_became_peered":"2023-10-26T05:48:24.698397+0000","last_unstale":"2023-10-26T05:48:25.787849+0000","last_undegraded":"2023-10-26T05:48:25.787849+0000","last_fullsized":"2023-10-26T05:48:25.787849+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:52.315290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.017442+0000","last_change":"2023-10-26T05:48:24.695186+0000","last_active":"2023-10-26T05:48:26.017442+0000","last_peered":"2023-10-26T05:48:26.017442+0000","last_clean":"2023-10-26T05:48:26.017442+0000","last_became_active":"2023-10-26T05:48:24.694555+0000","last_became_peered":"2023-10-26T05:48:24.694555+0000","last_unstale":"2023-10-26T05:48:26.017442+0000","last_undegraded":"2023-10-26T05:48:26.017442+0000","last_fullsized":"2023-10-26T05:48:26.017442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:29.521135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018252+0000","last_change":"2023-10-26T05:48:24.702581+0000","last_active":"2023-10-26T05:48:26.018252+0000","last_peered":"2023-10-26T05:48:26.018252+0000","last_clean":"2023-10-26T05:48:26.018252+0000","last_became_active":"2023-10-26T05:48:24.701534+0000","last_became_peered":"2023-10-26T05:48:24.701534+0000","last_unstale":"2023-10-26T05:48:26.018252+0000","last_undegraded":"2023-10-26T05:48:26.018252+0000","last_fullsized":"2023-10-26T05:48:26.018252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:13.315366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.673858+0000","last_change":"2023-10-26T05:48:24.704000+0000","last_active":"2023-10-26T05:48:25.673858+0000","last_peered":"2023-10-26T05:48:25.673858+0000","last_clean":"2023-10-26T05:48:25.673858+0000","last_became_active":"2023-10-26T05:48:24.703766+0000","last_became_peered":"2023-10-26T05:48:24.703766+0000","last_unstale":"2023-10-26T05:48:25.673858+0000","last_undegraded":"2023-10-26T05:48:25.673858+0000","last_fullsized":"2023-10-26T05:48:25.673858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:44:20.030937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018700+0000","last_change":"2023-10-26T05:48:24.702845+0000","last_active":"2023-10-26T05:48:26.018700+0000","last_peered":"2023-10-26T05:48:26.018700+0000","last_clean":"2023-10-26T05:48:26.018700+0000","last_became_active":"2023-10-26T05:48:24.701637+0000","last_became_peered":"2023-10-26T05:48:24.701637+0000","last_unstale":"2023-10-26T05:48:26.018700+0000","last_undegraded":"2023-10-26T05:48:26.018700+0000","last_fullsized":"2023-10-26T05:48:26.018700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:35:29.528071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.674032+0000","last_change":"2023-10-26T05:48:24.693386+0000","last_active":"2023-10-26T05:48:25.674032+0000","last_peered":"2023-10-26T05:48:25.674032+0000","last_clean":"2023-10-26T05:48:25.674032+0000","last_became_active":"2023-10-26T05:48:24.690328+0000","last_became_peered":"2023-10-26T05:48:24.690328+0000","last_unstale":"2023-10-26T05:48:25.674032+0000","last_undegraded":"2023-10-26T05:48:25.674032+0000","last_fullsized":"2023-10-26T05:48:25.674032+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:41:40.650960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018260+0000","last_change":"2023-10-26T05:48:24.726495+0000","last_active":"2023-10-26T05:48:26.018260+0000","last_peered":"2023-10-26T05:48:26.018260+0000","last_clean":"2023-10-26T05:48:26.018260+0000","last_became_active":"2023-10-26T05:48:24.723633+0000","last_became_peered":"2023-10-26T05:48:24.723633+0000","last_unstale":"2023-10-26T05:48:26.018260+0000","last_undegraded":"2023-10-26T05:48:26.018260+0000","last_fullsized":"2023-10-26T05:48:26.018260+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:52.315290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787750+0000","last_change":"2023-10-26T05:48:24.715207+0000","last_active":"2023-10-26T05:48:25.787750+0000","last_peered":"2023-10-26T05:48:25.787750+0000","last_clean":"2023-10-26T05:48:25.787750+0000","last_became_active":"2023-10-26T05:48:24.710456+0000","last_became_peered":"2023-10-26T05:48:24.710456+0000","last_unstale":"2023-10-26T05:48:25.787750+0000","last_undegraded":"2023-10-26T05:48:25.787750+0000","last_fullsized":"2023-10-26T05:48:25.787750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:11.342892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677859+0000","last_change":"2023-10-26T05:48:24.703640+0000","last_active":"2023-10-26T05:48:25.677859+0000","last_peered":"2023-10-26T05:48:25.677859+0000","last_clean":"2023-10-26T05:48:25.677859+0000","last_became_active":"2023-10-26T05:48:24.703117+0000","last_became_peered":"2023-10-26T05:48:24.703117+0000","last_unstale":"2023-10-26T05:48:25.677859+0000","last_undegraded":"2023-10-26T05:48:25.677859+0000","last_fullsized":"2023-10-26T05:48:25.677859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:45.783172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677940+0000","last_change":"2023-10-26T05:48:24.701094+0000","last_active":"2023-10-26T05:48:25.677940+0000","last_peered":"2023-10-26T05:48:25.677940+0000","last_clean":"2023-10-26T05:48:25.677940+0000","last_became_active":"2023-10-26T05:48:24.700464+0000","last_became_peered":"2023-10-26T05:48:24.700464+0000","last_unstale":"2023-10-26T05:48:25.677940+0000","last_undegraded":"2023-10-26T05:48:25.677940+0000","last_fullsized":"2023-10-26T05:48:25.677940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:07:51.253800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676590+0000","last_change":"2023-10-26T05:48:24.699575+0000","last_active":"2023-10-26T05:48:25.676590+0000","last_peered":"2023-10-26T05:48:25.676590+0000","last_clean":"2023-10-26T05:48:25.676590+0000","last_became_active":"2023-10-26T05:48:24.696170+0000","last_became_peered":"2023-10-26T05:48:24.696170+0000","last_unstale":"2023-10-26T05:48:25.676590+0000","last_undegraded":"2023-10-26T05:48:25.676590+0000","last_fullsized":"2023-10-26T05:48:25.676590+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:52.315290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016903+0000","last_change":"2023-10-26T05:48:24.693230+0000","last_active":"2023-10-26T05:48:26.016903+0000","last_peered":"2023-10-26T05:48:26.016903+0000","last_clean":"2023-10-26T05:48:26.016903+0000","last_became_active":"2023-10-26T05:48:24.692568+0000","last_became_peered":"2023-10-26T05:48:24.692568+0000","last_unstale":"2023-10-26T05:48:26.016903+0000","last_undegraded":"2023-10-26T05:48:26.016903+0000","last_fullsized":"2023-10-26T05:48:26.016903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:07:51.253800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.726731+0000","last_change":"2023-10-26T05:48:23.711070+0000","last_active":"2023-10-26T05:48:24.726731+0000","last_peered":"2023-10-26T05:48:24.726731+0000","last_clean":"2023-10-26T05:48:24.726731+0000","last_became_active":"2023-10-26T05:48:23.710421+0000","last_became_peered":"2023-10-26T05:48:23.710421+0000","last_unstale":"2023-10-26T05:48:24.726731+0000","last_undegraded":"2023-10-26T05:48:24.726731+0000","last_fullsized":"2023-10-26T05:48:24.726731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676921+0000","last_change":"2023-10-26T05:48:24.709865+0000","last_active":"2023-10-26T05:48:25.676921+0000","last_peered":"2023-10-26T05:48:25.676921+0000","last_clean":"2023-10-26T05:48:25.676921+0000","last_became_active":"2023-10-26T05:48:24.709355+0000","last_became_peered":"2023-10-26T05:48:24.709355+0000","last_unstale":"2023-10-26T05:48:25.676921+0000","last_undegraded":"2023-10-26T05:48:25.676921+0000","last_fullsized":"2023-10-26T05:48:25.676921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:30.455725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.718791+0000","last_change":"2023-10-26T05:48:23.703810+0000","last_active":"2023-10-26T05:48:24.718791+0000","last_peered":"2023-10-26T05:48:24.718791+0000","last_clean":"2023-10-26T05:48:24.718791+0000","last_became_active":"2023-10-26T05:48:23.703500+0000","last_became_peered":"2023-10-26T05:48:23.703500+0000","last_unstale":"2023-10-26T05:48:24.718791+0000","last_undegraded":"2023-10-26T05:48:24.718791+0000","last_fullsized":"2023-10-26T05:48:24.718791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:03.464765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787366+0000","last_change":"2023-10-26T05:48:24.699591+0000","last_active":"2023-10-26T05:48:25.787366+0000","last_peered":"2023-10-26T05:48:25.787366+0000","last_clean":"2023-10-26T05:48:25.787366+0000","last_became_active":"2023-10-26T05:48:24.698937+0000","last_became_peered":"2023-10-26T05:48:24.698937+0000","last_unstale":"2023-10-26T05:48:25.787366+0000","last_undegraded":"2023-10-26T05:48:25.787366+0000","last_fullsized":"2023-10-26T05:48:25.787366+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:23.763067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.711865+0000","last_change":"2023-10-26T05:48:23.690022+0000","last_active":"2023-10-26T05:48:24.711865+0000","last_peered":"2023-10-26T05:48:24.711865+0000","last_clean":"2023-10-26T05:48:24.711865+0000","last_became_active":"2023-10-26T05:48:23.689605+0000","last_became_peered":"2023-10-26T05:48:23.689605+0000","last_unstale":"2023-10-26T05:48:24.711865+0000","last_undegraded":"2023-10-26T05:48:24.711865+0000","last_fullsized":"2023-10-26T05:48:24.711865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:28.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.674224+0000","last_change":"2023-10-26T05:48:24.691840+0000","last_active":"2023-10-26T05:48:25.674224+0000","last_peered":"2023-10-26T05:48:25.674224+0000","last_clean":"2023-10-26T05:48:25.674224+0000","last_became_active":"2023-10-26T05:48:24.691228+0000","last_became_peered":"2023-10-26T05:48:24.691228+0000","last_unstale":"2023-10-26T05:48:25.674224+0000","last_undegraded":"2023-10-26T05:48:25.674224+0000","last_fullsized":"2023-10-26T05:48:25.674224+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:52.315290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.711263+0000","last_change":"2023-10-26T05:48:23.701287+0000","last_active":"2023-10-26T05:48:24.711263+0000","last_peered":"2023-10-26T05:48:24.711263+0000","last_clean":"2023-10-26T05:48:24.711263+0000","last_became_active":"2023-10-26T05:48:23.700949+0000","last_became_peered":"2023-10-26T05:48:23.700949+0000","last_unstale":"2023-10-26T05:48:24.711263+0000","last_undegraded":"2023-10-26T05:48:24.711263+0000","last_fullsized":"2023-10-26T05:48:24.711263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:17.343245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676554+0000","last_change":"2023-10-26T05:48:24.703424+0000","last_active":"2023-10-26T05:48:25.676554+0000","last_peered":"2023-10-26T05:48:25.676554+0000","last_clean":"2023-10-26T05:48:25.676554+0000","last_became_active":"2023-10-26T05:48:24.695856+0000","last_became_peered":"2023-10-26T05:48:24.695856+0000","last_unstale":"2023-10-26T05:48:25.676554+0000","last_undegraded":"2023-10-26T05:48:25.676554+0000","last_fullsized":"2023-10-26T05:48:25.676554+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:11.342892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.713664+0000","last_change":"2023-10-26T05:48:23.690572+0000","last_active":"2023-10-26T05:48:24.713664+0000","last_peered":"2023-10-26T05:48:24.713664+0000","last_clean":"2023-10-26T05:48:24.713664+0000","last_became_active":"2023-10-26T05:48:23.689616+0000","last_became_peered":"2023-10-26T05:48:23.689616+0000","last_unstale":"2023-10-26T05:48:24.713664+0000","last_undegraded":"2023-10-26T05:48:24.713664+0000","last_fullsized":"2023-10-26T05:48:24.713664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018591+0000","last_change":"2023-10-26T05:48:24.717210+0000","last_active":"2023-10-26T05:48:26.018591+0000","last_peered":"2023-10-26T05:48:26.018591+0000","last_clean":"2023-10-26T05:48:26.018591+0000","last_became_active":"2023-10-26T05:48:24.713812+0000","last_became_peered":"2023-10-26T05:48:24.713812+0000","last_unstale":"2023-10-26T05:48:26.018591+0000","last_undegraded":"2023-10-26T05:48:26.018591+0000","last_fullsized":"2023-10-26T05:48:26.018591+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:22.140651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.714619+0000","last_change":"2023-10-26T05:48:23.701241+0000","last_active":"2023-10-26T05:48:24.714619+0000","last_peered":"2023-10-26T05:48:24.714619+0000","last_clean":"2023-10-26T05:48:24.714619+0000","last_became_active":"2023-10-26T05:48:23.699629+0000","last_became_peered":"2023-10-26T05:48:23.699629+0000","last_unstale":"2023-10-26T05:48:24.714619+0000","last_undegraded":"2023-10-26T05:48:24.714619+0000","last_fullsized":"2023-10-26T05:48:24.714619+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:52.984869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787078+0000","last_change":"2023-10-26T05:48:24.689303+0000","last_active":"2023-10-26T05:48:25.787078+0000","last_peered":"2023-10-26T05:48:25.787078+0000","last_clean":"2023-10-26T05:48:25.787078+0000","last_became_active":"2023-10-26T05:48:24.688681+0000","last_became_peered":"2023-10-26T05:48:24.688681+0000","last_unstale":"2023-10-26T05:48:25.787078+0000","last_undegraded":"2023-10-26T05:48:25.787078+0000","last_fullsized":"2023-10-26T05:48:25.787078+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:25.613225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.692754+0000","last_change":"2023-10-26T05:48:23.700412+0000","last_active":"2023-10-26T05:48:24.692754+0000","last_peered":"2023-10-26T05:48:24.692754+0000","last_clean":"2023-10-26T05:48:24.692754+0000","last_became_active":"2023-10-26T05:48:23.698966+0000","last_became_peered":"2023-10-26T05:48:23.698966+0000","last_unstale":"2023-10-26T05:48:24.692754+0000","last_undegraded":"2023-10-26T05:48:24.692754+0000","last_fullsized":"2023-10-26T05:48:24.692754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:41:39.634863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.673567+0000","last_change":"2023-10-26T05:48:24.712046+0000","last_active":"2023-10-26T05:48:25.673567+0000","last_peered":"2023-10-26T05:48:25.673567+0000","last_clean":"2023-10-26T05:48:25.673567+0000","last_became_active":"2023-10-26T05:48:24.711834+0000","last_became_peered":"2023-10-26T05:48:24.711834+0000","last_unstale":"2023-10-26T05:48:25.673567+0000","last_undegraded":"2023-10-26T05:48:25.673567+0000","last_fullsized":"2023-10-26T05:48:25.673567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:18.214571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.677192+0000","last_change":"2023-10-26T05:48:23.715497+0000","last_active":"2023-10-26T05:48:24.677192+0000","last_peered":"2023-10-26T05:48:24.677192+0000","last_clean":"2023-10-26T05:48:24.677192+0000","last_became_active":"2023-10-26T05:48:23.714277+0000","last_became_peered":"2023-10-26T05:48:23.714277+0000","last_unstale":"2023-10-26T05:48:24.677192+0000","last_undegraded":"2023-10-26T05:48:24.677192+0000","last_fullsized":"2023-10-26T05:48:24.677192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:29.439627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.019331+0000","last_change":"2023-10-26T05:48:24.723712+0000","last_active":"2023-10-26T05:48:26.019331+0000","last_peered":"2023-10-26T05:48:26.019331+0000","last_clean":"2023-10-26T05:48:26.019331+0000","last_became_active":"2023-10-26T05:48:24.716107+0000","last_became_peered":"2023-10-26T05:48:24.716107+0000","last_unstale":"2023-10-26T05:48:26.019331+0000","last_undegraded":"2023-10-26T05:48:26.019331+0000","last_fullsized":"2023-10-26T05:48:26.019331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:50:43.526589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.695932+0000","last_change":"2023-10-26T05:48:23.705748+0000","last_active":"2023-10-26T05:48:24.695932+0000","last_peered":"2023-10-26T05:48:24.695932+0000","last_clean":"2023-10-26T05:48:24.695932+0000","last_became_active":"2023-10-26T05:48:23.705389+0000","last_became_peered":"2023-10-26T05:48:23.705389+0000","last_unstale":"2023-10-26T05:48:24.695932+0000","last_undegraded":"2023-10-26T05:48:24.695932+0000","last_fullsized":"2023-10-26T05:48:24.695932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:12.299269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.019002+0000","last_change":"2023-10-26T05:48:24.703025+0000","last_active":"2023-10-26T05:48:26.019002+0000","last_peered":"2023-10-26T05:48:26.019002+0000","last_clean":"2023-10-26T05:48:26.019002+0000","last_became_active":"2023-10-26T05:48:24.702011+0000","last_became_peered":"2023-10-26T05:48:24.702011+0000","last_unstale":"2023-10-26T05:48:26.019002+0000","last_undegraded":"2023-10-26T05:48:26.019002+0000","last_fullsized":"2023-10-26T05:48:26.019002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:56:38.223629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.671012+0000","last_change":"2023-10-26T05:48:23.696857+0000","last_active":"2023-10-26T05:48:24.671012+0000","last_peered":"2023-10-26T05:48:24.671012+0000","last_clean":"2023-10-26T05:48:24.671012+0000","last_became_active":"2023-10-26T05:48:23.696471+0000","last_became_peered":"2023-10-26T05:48:23.696471+0000","last_unstale":"2023-10-26T05:48:24.671012+0000","last_undegraded":"2023-10-26T05:48:24.671012+0000","last_fullsized":"2023-10-26T05:48:24.671012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.768450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018111+0000","last_change":"2023-10-26T05:48:24.702679+0000","last_active":"2023-10-26T05:48:26.018111+0000","last_peered":"2023-10-26T05:48:26.018111+0000","last_clean":"2023-10-26T05:48:26.018111+0000","last_became_active":"2023-10-26T05:48:24.698570+0000","last_became_peered":"2023-10-26T05:48:24.698570+0000","last_unstale":"2023-10-26T05:48:26.018111+0000","last_undegraded":"2023-10-26T05:48:26.018111+0000","last_fullsized":"2023-10-26T05:48:26.018111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:50:43.526589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.729353+0000","last_change":"2023-10-26T05:48:23.709670+0000","last_active":"2023-10-26T05:48:24.729353+0000","last_peered":"2023-10-26T05:48:24.729353+0000","last_clean":"2023-10-26T05:48:24.729353+0000","last_became_active":"2023-10-26T05:48:23.709170+0000","last_became_peered":"2023-10-26T05:48:23.709170+0000","last_unstale":"2023-10-26T05:48:24.729353+0000","last_undegraded":"2023-10-26T05:48:24.729353+0000","last_fullsized":"2023-10-26T05:48:24.729353+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:22.746970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018650+0000","last_change":"2023-10-26T05:48:24.726998+0000","last_active":"2023-10-26T05:48:26.018650+0000","last_peered":"2023-10-26T05:48:26.018650+0000","last_clean":"2023-10-26T05:48:26.018650+0000","last_became_active":"2023-10-26T05:48:24.724997+0000","last_became_peered":"2023-10-26T05:48:24.724997+0000","last_unstale":"2023-10-26T05:48:26.018650+0000","last_undegraded":"2023-10-26T05:48:26.018650+0000","last_fullsized":"2023-10-26T05:48:26.018650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:38.930048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.713554+0000","last_change":"2023-10-26T05:48:23.689973+0000","last_active":"2023-10-26T05:48:24.713554+0000","last_peered":"2023-10-26T05:48:24.713554+0000","last_clean":"2023-10-26T05:48:24.713554+0000","last_became_active":"2023-10-26T05:48:23.689550+0000","last_became_peered":"2023-10-26T05:48:23.689550+0000","last_unstale":"2023-10-26T05:48:24.713554+0000","last_undegraded":"2023-10-26T05:48:24.713554+0000","last_fullsized":"2023-10-26T05:48:24.713554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:10.326795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.674131+0000","last_change":"2023-10-26T05:48:24.711454+0000","last_active":"2023-10-26T05:48:25.674131+0000","last_peered":"2023-10-26T05:48:25.674131+0000","last_clean":"2023-10-26T05:48:25.674131+0000","last_became_active":"2023-10-26T05:48:24.711077+0000","last_became_peered":"2023-10-26T05:48:24.711077+0000","last_unstale":"2023-10-26T05:48:25.674131+0000","last_undegraded":"2023-10-26T05:48:25.674131+0000","last_fullsized":"2023-10-26T05:48:25.674131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:49:11.342892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.722429+0000","last_change":"2023-10-26T05:48:23.715244+0000","last_active":"2023-10-26T05:48:24.722429+0000","last_peered":"2023-10-26T05:48:24.722429+0000","last_clean":"2023-10-26T05:48:24.722429+0000","last_became_active":"2023-10-26T05:48:23.713621+0000","last_became_peered":"2023-10-26T05:48:23.713621+0000","last_unstale":"2023-10-26T05:48:24.722429+0000","last_undegraded":"2023-10-26T05:48:24.722429+0000","last_fullsized":"2023-10-26T05:48:24.722429+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:14.707635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.017453+0000","last_change":"2023-10-26T05:48:24.690353+0000","last_active":"2023-10-26T05:48:26.017453+0000","last_peered":"2023-10-26T05:48:26.017453+0000","last_clean":"2023-10-26T05:48:26.017453+0000","last_became_active":"2023-10-26T05:48:24.689761+0000","last_became_peered":"2023-10-26T05:48:24.689761+0000","last_unstale":"2023-10-26T05:48:26.017453+0000","last_undegraded":"2023-10-26T05:48:26.017453+0000","last_fullsized":"2023-10-26T05:48:26.017453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:03.302780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.681562+0000","last_change":"2023-10-26T05:48:23.704259+0000","last_active":"2023-10-26T05:48:24.681562+0000","last_peered":"2023-10-26T05:48:24.681562+0000","last_clean":"2023-10-26T05:48:24.681562+0000","last_became_active":"2023-10-26T05:48:23.703997+0000","last_became_peered":"2023-10-26T05:48:23.703997+0000","last_unstale":"2023-10-26T05:48:24.681562+0000","last_undegraded":"2023-10-26T05:48:24.681562+0000","last_fullsized":"2023-10-26T05:48:24.681562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.676871+0000","last_change":"2023-10-26T05:48:24.703974+0000","last_active":"2023-10-26T05:48:25.676871+0000","last_peered":"2023-10-26T05:48:25.676871+0000","last_clean":"2023-10-26T05:48:25.676871+0000","last_became_active":"2023-10-26T05:48:24.696469+0000","last_became_peered":"2023-10-26T05:48:24.696469+0000","last_unstale":"2023-10-26T05:48:25.676871+0000","last_undegraded":"2023-10-26T05:48:25.676871+0000","last_fullsized":"2023-10-26T05:48:25.676871+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:33:13.315366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.687233+0000","last_change":"2023-10-26T05:48:23.706487+0000","last_active":"2023-10-26T05:48:24.687233+0000","last_peered":"2023-10-26T05:48:24.687233+0000","last_clean":"2023-10-26T05:48:24.687233+0000","last_became_active":"2023-10-26T05:48:23.706232+0000","last_became_peered":"2023-10-26T05:48:23.706232+0000","last_unstale":"2023-10-26T05:48:24.687233+0000","last_undegraded":"2023-10-26T05:48:24.687233+0000","last_fullsized":"2023-10-26T05:48:24.687233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:24.844228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677113+0000","last_change":"2023-10-26T05:48:24.692332+0000","last_active":"2023-10-26T05:48:25.677113+0000","last_peered":"2023-10-26T05:48:25.677113+0000","last_clean":"2023-10-26T05:48:25.677113+0000","last_became_active":"2023-10-26T05:48:24.690816+0000","last_became_peered":"2023-10-26T05:48:24.690816+0000","last_unstale":"2023-10-26T05:48:25.677113+0000","last_undegraded":"2023-10-26T05:48:25.677113+0000","last_fullsized":"2023-10-26T05:48:25.677113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:17.389020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.714413+0000","last_change":"2023-10-26T05:48:23.688956+0000","last_active":"2023-10-26T05:48:24.714413+0000","last_peered":"2023-10-26T05:48:24.714413+0000","last_clean":"2023-10-26T05:48:24.714413+0000","last_became_active":"2023-10-26T05:48:23.688311+0000","last_became_peered":"2023-10-26T05:48:23.688311+0000","last_unstale":"2023-10-26T05:48:24.714413+0000","last_undegraded":"2023-10-26T05:48:24.714413+0000","last_fullsized":"2023-10-26T05:48:24.714413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:17.198474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787595+0000","last_change":"2023-10-26T05:48:24.710641+0000","last_active":"2023-10-26T05:48:25.787595+0000","last_peered":"2023-10-26T05:48:25.787595+0000","last_clean":"2023-10-26T05:48:25.787595+0000","last_became_active":"2023-10-26T05:48:24.708464+0000","last_became_peered":"2023-10-26T05:48:24.708464+0000","last_unstale":"2023-10-26T05:48:25.787595+0000","last_undegraded":"2023-10-26T05:48:25.787595+0000","last_fullsized":"2023-10-26T05:48:25.787595+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:41:40.650960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.710816+0000","last_change":"2023-10-26T05:48:23.688351+0000","last_active":"2023-10-26T05:48:24.710816+0000","last_peered":"2023-10-26T05:48:24.710816+0000","last_clean":"2023-10-26T05:48:24.710816+0000","last_became_active":"2023-10-26T05:48:23.687015+0000","last_became_peered":"2023-10-26T05:48:23.687015+0000","last_unstale":"2023-10-26T05:48:24.710816+0000","last_undegraded":"2023-10-26T05:48:24.710816+0000","last_fullsized":"2023-10-26T05:48:24.710816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:44:28.505038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018639+0000","last_change":"2023-10-26T05:48:24.702179+0000","last_active":"2023-10-26T05:48:26.018639+0000","last_peered":"2023-10-26T05:48:26.018639+0000","last_clean":"2023-10-26T05:48:26.018639+0000","last_became_active":"2023-10-26T05:48:24.700665+0000","last_became_peered":"2023-10-26T05:48:24.700665+0000","last_unstale":"2023-10-26T05:48:26.018639+0000","last_undegraded":"2023-10-26T05:48:26.018639+0000","last_fullsized":"2023-10-26T05:48:26.018639+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:45.783172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.720045+0000","last_change":"2023-10-26T05:48:23.714853+0000","last_active":"2023-10-26T05:48:24.720045+0000","last_peered":"2023-10-26T05:48:24.720045+0000","last_clean":"2023-10-26T05:48:24.720045+0000","last_became_active":"2023-10-26T05:48:23.712919+0000","last_became_peered":"2023-10-26T05:48:23.712919+0000","last_unstale":"2023-10-26T05:48:24.720045+0000","last_undegraded":"2023-10-26T05:48:24.720045+0000","last_fullsized":"2023-10-26T05:48:24.720045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:14:22.542611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.673743+0000","last_change":"2023-10-26T05:48:24.692966+0000","last_active":"2023-10-26T05:48:25.673743+0000","last_peered":"2023-10-26T05:48:25.673743+0000","last_clean":"2023-10-26T05:48:25.673743+0000","last_became_active":"2023-10-26T05:48:24.689310+0000","last_became_peered":"2023-10-26T05:48:24.689310+0000","last_unstale":"2023-10-26T05:48:25.673743+0000","last_undegraded":"2023-10-26T05:48:25.673743+0000","last_fullsized":"2023-10-26T05:48:25.673743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:23.763067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.713071+0000","last_change":"2023-10-26T05:48:23.703179+0000","last_active":"2023-10-26T05:48:24.713071+0000","last_peered":"2023-10-26T05:48:24.713071+0000","last_clean":"2023-10-26T05:48:24.713071+0000","last_became_active":"2023-10-26T05:48:23.702603+0000","last_became_peered":"2023-10-26T05:48:23.702603+0000","last_unstale":"2023-10-26T05:48:24.713071+0000","last_undegraded":"2023-10-26T05:48:24.713071+0000","last_fullsized":"2023-10-26T05:48:24.713071+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.019009+0000","last_change":"2023-10-26T05:48:24.695967+0000","last_active":"2023-10-26T05:48:26.019009+0000","last_peered":"2023-10-26T05:48:26.019009+0000","last_clean":"2023-10-26T05:48:26.019009+0000","last_became_active":"2023-10-26T05:48:24.695312+0000","last_became_peered":"2023-10-26T05:48:24.695312+0000","last_unstale":"2023-10-26T05:48:26.019009+0000","last_undegraded":"2023-10-26T05:48:26.019009+0000","last_fullsized":"2023-10-26T05:48:26.019009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:37.965274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.677213+0000","last_change":"2023-10-26T05:48:23.690214+0000","last_active":"2023-10-26T05:48:24.677213+0000","last_peered":"2023-10-26T05:48:24.677213+0000","last_clean":"2023-10-26T05:48:24.677213+0000","last_became_active":"2023-10-26T05:48:23.688325+0000","last_became_peered":"2023-10-26T05:48:23.688325+0000","last_unstale":"2023-10-26T05:48:24.677213+0000","last_undegraded":"2023-10-26T05:48:24.677213+0000","last_fullsized":"2023-10-26T05:48:24.677213+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:16.372923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016565+0000","last_change":"2023-10-26T05:48:24.678150+0000","last_active":"2023-10-26T05:48:26.016565+0000","last_peered":"2023-10-26T05:48:26.016565+0000","last_clean":"2023-10-26T05:48:26.016565+0000","last_became_active":"2023-10-26T05:48:24.677470+0000","last_became_peered":"2023-10-26T05:48:24.677470+0000","last_unstale":"2023-10-26T05:48:26.016565+0000","last_undegraded":"2023-10-26T05:48:26.016565+0000","last_fullsized":"2023-10-26T05:48:26.016565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:36.377980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.709911+0000","last_change":"2023-10-26T05:48:23.691997+0000","last_active":"2023-10-26T05:48:24.709911+0000","last_peered":"2023-10-26T05:48:24.709911+0000","last_clean":"2023-10-26T05:48:24.709911+0000","last_became_active":"2023-10-26T05:48:23.691414+0000","last_became_peered":"2023-10-26T05:48:23.691414+0000","last_unstale":"2023-10-26T05:48:24.709911+0000","last_undegraded":"2023-10-26T05:48:24.709911+0000","last_fullsized":"2023-10-26T05:48:24.709911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:00.129232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018583+0000","last_change":"2023-10-26T05:48:24.680107+0000","last_active":"2023-10-26T05:48:26.018583+0000","last_peered":"2023-10-26T05:48:26.018583+0000","last_clean":"2023-10-26T05:48:26.018583+0000","last_became_active":"2023-10-26T05:48:24.679473+0000","last_became_peered":"2023-10-26T05:48:24.679473+0000","last_unstale":"2023-10-26T05:48:26.018583+0000","last_undegraded":"2023-10-26T05:48:26.018583+0000","last_fullsized":"2023-10-26T05:48:26.018583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:01.145329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.683441+0000","last_change":"2023-10-26T05:48:23.705656+0000","last_active":"2023-10-26T05:48:24.683441+0000","last_peered":"2023-10-26T05:48:24.683441+0000","last_clean":"2023-10-26T05:48:24.683441+0000","last_became_active":"2023-10-26T05:48:23.705077+0000","last_became_peered":"2023-10-26T05:48:23.705077+0000","last_unstale":"2023-10-26T05:48:24.683441+0000","last_undegraded":"2023-10-26T05:48:24.683441+0000","last_fullsized":"2023-10-26T05:48:24.683441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:21:46.476272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.019326+0000","last_change":"2023-10-26T05:48:24.679959+0000","last_active":"2023-10-26T05:48:26.019326+0000","last_peered":"2023-10-26T05:48:26.019326+0000","last_clean":"2023-10-26T05:48:26.019326+0000","last_became_active":"2023-10-26T05:48:24.679517+0000","last_became_peered":"2023-10-26T05:48:24.679517+0000","last_unstale":"2023-10-26T05:48:26.019326+0000","last_undegraded":"2023-10-26T05:48:26.019326+0000","last_fullsized":"2023-10-26T05:48:26.019326+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:17.389020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.678225+0000","last_change":"2023-10-26T05:48:23.712097+0000","last_active":"2023-10-26T05:48:24.678225+0000","last_peered":"2023-10-26T05:48:24.678225+0000","last_clean":"2023-10-26T05:48:24.678225+0000","last_became_active":"2023-10-26T05:48:23.704476+0000","last_became_peered":"2023-10-26T05:48:23.704476+0000","last_unstale":"2023-10-26T05:48:24.678225+0000","last_undegraded":"2023-10-26T05:48:24.678225+0000","last_fullsized":"2023-10-26T05:48:24.678225+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:07:50.237702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.678161+0000","last_change":"2023-10-26T05:48:24.720124+0000","last_active":"2023-10-26T05:48:25.678161+0000","last_peered":"2023-10-26T05:48:25.678161+0000","last_clean":"2023-10-26T05:48:25.678161+0000","last_became_active":"2023-10-26T05:48:24.705381+0000","last_became_peered":"2023-10-26T05:48:24.705381+0000","last_unstale":"2023-10-26T05:48:25.678161+0000","last_undegraded":"2023-10-26T05:48:25.678161+0000","last_fullsized":"2023-10-26T05:48:25.678161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:03.302780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.685197+0000","last_change":"2023-10-26T05:48:23.708587+0000","last_active":"2023-10-26T05:48:24.685197+0000","last_peered":"2023-10-26T05:48:24.685197+0000","last_clean":"2023-10-26T05:48:24.685197+0000","last_became_active":"2023-10-26T05:48:23.707969+0000","last_became_peered":"2023-10-26T05:48:23.707969+0000","last_unstale":"2023-10-26T05:48:24.685197+0000","last_undegraded":"2023-10-26T05:48:24.685197+0000","last_fullsized":"2023-10-26T05:48:24.685197+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:16.372923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018721+0000","last_change":"2023-10-26T05:48:24.701776+0000","last_active":"2023-10-26T05:48:26.018721+0000","last_peered":"2023-10-26T05:48:26.018721+0000","last_clean":"2023-10-26T05:48:26.018721+0000","last_became_active":"2023-10-26T05:48:24.701412+0000","last_became_peered":"2023-10-26T05:48:24.701412+0000","last_unstale":"2023-10-26T05:48:26.018721+0000","last_undegraded":"2023-10-26T05:48:26.018721+0000","last_fullsized":"2023-10-26T05:48:26.018721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:04.480862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.715586+0000","last_change":"2023-10-26T05:48:23.705491+0000","last_active":"2023-10-26T05:48:24.715586+0000","last_peered":"2023-10-26T05:48:24.715586+0000","last_clean":"2023-10-26T05:48:24.715586+0000","last_became_active":"2023-10-26T05:48:23.705095+0000","last_became_peered":"2023-10-26T05:48:23.705095+0000","last_unstale":"2023-10-26T05:48:24.715586+0000","last_undegraded":"2023-10-26T05:48:24.715586+0000","last_fullsized":"2023-10-26T05:48:24.715586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:10:52.984869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018970+0000","last_change":"2023-10-26T05:48:24.718390+0000","last_active":"2023-10-26T05:48:26.018970+0000","last_peered":"2023-10-26T05:48:26.018970+0000","last_clean":"2023-10-26T05:48:26.018970+0000","last_became_active":"2023-10-26T05:48:24.714693+0000","last_became_peered":"2023-10-26T05:48:24.714693+0000","last_unstale":"2023-10-26T05:48:26.018970+0000","last_undegraded":"2023-10-26T05:48:26.018970+0000","last_fullsized":"2023-10-26T05:48:26.018970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:05.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.017649+0000","last_change":"2023-10-26T05:48:23.692126+0000","last_active":"2023-10-26T05:48:26.017649+0000","last_peered":"2023-10-26T05:48:26.017649+0000","last_clean":"2023-10-26T05:48:26.017649+0000","last_became_active":"2023-10-26T05:48:23.691722+0000","last_became_peered":"2023-10-26T05:48:23.691722+0000","last_unstale":"2023-10-26T05:48:26.017649+0000","last_undegraded":"2023-10-26T05:48:26.017649+0000","last_fullsized":"2023-10-26T05:48:26.017649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:59:36.949177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016265+0000","last_change":"2023-10-26T05:48:24.685925+0000","last_active":"2023-10-26T05:48:26.016265+0000","last_peered":"2023-10-26T05:48:26.016265+0000","last_clean":"2023-10-26T05:48:26.016265+0000","last_became_active":"2023-10-26T05:48:24.685317+0000","last_became_peered":"2023-10-26T05:48:24.685317+0000","last_unstale":"2023-10-26T05:48:26.016265+0000","last_undegraded":"2023-10-26T05:48:26.016265+0000","last_fullsized":"2023-10-26T05:48:26.016265+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:14:23.558708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.684434+0000","last_change":"2023-10-26T05:48:23.699301+0000","last_active":"2023-10-26T05:48:24.684434+0000","last_peered":"2023-10-26T05:48:24.684434+0000","last_clean":"2023-10-26T05:48:24.684434+0000","last_became_active":"2023-10-26T05:48:23.698926+0000","last_became_peered":"2023-10-26T05:48:23.698926+0000","last_unstale":"2023-10-26T05:48:24.684434+0000","last_undegraded":"2023-10-26T05:48:24.684434+0000","last_fullsized":"2023-10-26T05:48:24.684434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:24.844228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.677446+0000","last_change":"2023-10-26T05:48:24.694911+0000","last_active":"2023-10-26T05:48:25.677446+0000","last_peered":"2023-10-26T05:48:25.677446+0000","last_clean":"2023-10-26T05:48:25.677446+0000","last_became_active":"2023-10-26T05:48:24.691725+0000","last_became_peered":"2023-10-26T05:48:24.691725+0000","last_unstale":"2023-10-26T05:48:25.677446+0000","last_undegraded":"2023-10-26T05:48:25.677446+0000","last_fullsized":"2023-10-26T05:48:25.677446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:15.723732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.672551+0000","last_change":"2023-10-26T05:48:23.697103+0000","last_active":"2023-10-26T05:48:24.672551+0000","last_peered":"2023-10-26T05:48:24.672551+0000","last_clean":"2023-10-26T05:48:24.672551+0000","last_became_active":"2023-10-26T05:48:23.696770+0000","last_became_peered":"2023-10-26T05:48:23.696770+0000","last_unstale":"2023-10-26T05:48:24.672551+0000","last_undegraded":"2023-10-26T05:48:24.672551+0000","last_fullsized":"2023-10-26T05:48:24.672551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:14.707635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.018694+0000","last_change":"2023-10-26T05:48:24.688186+0000","last_active":"2023-10-26T05:48:26.018694+0000","last_peered":"2023-10-26T05:48:26.018694+0000","last_clean":"2023-10-26T05:48:26.018694+0000","last_became_active":"2023-10-26T05:48:24.687120+0000","last_became_peered":"2023-10-26T05:48:24.687120+0000","last_unstale":"2023-10-26T05:48:26.018694+0000","last_undegraded":"2023-10-26T05:48:26.018694+0000","last_fullsized":"2023-10-26T05:48:26.018694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:25.860325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.681659+0000","last_change":"2023-10-26T05:48:23.705357+0000","last_active":"2023-10-26T05:48:24.681659+0000","last_peered":"2023-10-26T05:48:24.681659+0000","last_clean":"2023-10-26T05:48:24.681659+0000","last_became_active":"2023-10-26T05:48:23.704629+0000","last_became_peered":"2023-10-26T05:48:23.704629+0000","last_unstale":"2023-10-26T05:48:24.681659+0000","last_undegraded":"2023-10-26T05:48:24.681659+0000","last_fullsized":"2023-10-26T05:48:24.681659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:29.439627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:26.016700+0000","last_change":"2023-10-26T05:48:24.703192+0000","last_active":"2023-10-26T05:48:26.016700+0000","last_peered":"2023-10-26T05:48:26.016700+0000","last_clean":"2023-10-26T05:48:26.016700+0000","last_became_active":"2023-10-26T05:48:24.702117+0000","last_became_peered":"2023-10-26T05:48:24.702117+0000","last_unstale":"2023-10-26T05:48:26.016700+0000","last_undegraded":"2023-10-26T05:48:26.016700+0000","last_fullsized":"2023-10-26T05:48:26.016700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:11:22.140651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.676171+0000","last_change":"2023-10-26T05:48:23.701546+0000","last_active":"2023-10-26T05:48:24.676171+0000","last_peered":"2023-10-26T05:48:24.676171+0000","last_clean":"2023-10-26T05:48:24.676171+0000","last_became_active":"2023-10-26T05:48:23.700654+0000","last_became_peered":"2023-10-26T05:48:23.700654+0000","last_unstale":"2023-10-26T05:48:24.676171+0000","last_undegraded":"2023-10-26T05:48:24.676171+0000","last_fullsized":"2023-10-26T05:48:24.676171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:00.129232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.787187+0000","last_change":"2023-10-26T05:48:24.693637+0000","last_active":"2023-10-26T05:48:25.787187+0000","last_peered":"2023-10-26T05:48:25.787187+0000","last_clean":"2023-10-26T05:48:25.787187+0000","last_became_active":"2023-10-26T05:48:24.692000+0000","last_became_peered":"2023-10-26T05:48:24.692000+0000","last_unstale":"2023-10-26T05:48:25.787187+0000","last_undegraded":"2023-10-26T05:48:25.787187+0000","last_fullsized":"2023-10-26T05:48:25.787187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:18.214571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.693427+0000","last_change":"2023-10-26T05:48:23.703600+0000","last_active":"2023-10-26T05:48:24.693427+0000","last_peered":"2023-10-26T05:48:24.693427+0000","last_clean":"2023-10-26T05:48:24.693427+0000","last_became_active":"2023-10-26T05:48:23.703254+0000","last_became_peered":"2023-10-26T05:48:23.703254+0000","last_unstale":"2023-10-26T05:48:24.693427+0000","last_undegraded":"2023-10-26T05:48:24.693427+0000","last_fullsized":"2023-10-26T05:48:24.693427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:51.299192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-26T05:48:25.678346+0000","last_change":"2023-10-26T05:48:24.720757+0000","last_active":"2023-10-26T05:48:25.678346+0000","last_peered":"2023-10-26T05:48:25.678346+0000","last_clean":"2023-10-26T05:48:25.678346+0000","last_became_active":"2023-10-26T05:48:24.719526+0000","last_became_peered":"2023-10-26T05:48:24.719526+0000","last_unstale":"2023-10-26T05:48:25.678346+0000","last_undegraded":"2023-10-26T05:48:25.678346+0000","last_fullsized":"2023-10-26T05:48:25.678346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:23.662302+0000","last_clean_scrub_stamp":"2023-10-26T05:48:23.662302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:05.698023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.684811+0000","last_change":"2023-10-26T05:48:23.712299+0000","last_active":"2023-10-26T05:48:24.684811+0000","last_peered":"2023-10-26T05:48:24.684811+0000","last_clean":"2023-10-26T05:48:24.684811+0000","last_became_active":"2023-10-26T05:48:23.711619+0000","last_became_peered":"2023-10-26T05:48:23.711619+0000","last_unstale":"2023-10-26T05:48:24.684811+0000","last_undegraded":"2023-10-26T05:48:24.684811+0000","last_fullsized":"2023-10-26T05:48:24.684811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:48:22.646204+0000","last_clean_scrub_stamp":"2023-10-26T05:48:22.646204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:00.129232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":177,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-26T05:48:24.685087+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:48:24.685087+0000","last_peered":"2023-10-26T05:48:24.685087+0000","last_clean":"2023-10-26T05:48:24.685087+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:48:24.685087+0000","last_undegraded":"2023-10-26T05:48:24.685087+0000","last_fullsized":"2023-10-26T05:48:24.685087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":725,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257475,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.66000000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.91900000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.97299999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.097},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":1.044}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.21}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.65700000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.012},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.056}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.89700000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":1.165},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.032}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.117},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.0840000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847261,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97799999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.109},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.873}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.95299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847829,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.0680000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.877},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.94299999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.034},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.2270000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.103},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.1910000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195162,"statfs":{"total":200000000,"available":199846299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.752},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86799999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.92100000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.95899999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.1870000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.036}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847863,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.46899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.88700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195162,"statfs":{"total":200000000,"available":199846299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.5},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.873}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.84399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":725,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257565,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.097}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.95799999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":1.163}]}]}],"pool_statfs":[]}} 2023-10-26T05:48:32.470 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:48:32.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:48:32.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 -- 172.21.15.71:0/23752555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d0141980 msgr2=0x7f82d00b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 --2- 172.21.15.71:0/23752555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d0141980 0x7f82d00b1e50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f82d00ca250 tx=0x7f82cc01c5b0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 -- 172.21.15.71:0/23752555 shutdown_connections 2023-10-26T05:48:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 --2- 172.21.15.71:0/23752555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82d00704b0 0x7f82d00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 --2- 172.21.15.71:0/23752555 >> v2:172.21.15.86:3300/0 conn(0x7f82d00b2390 0x7f82d006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 --2- 172.21.15.71:0/23752555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d0141980 0x7f82d00b1e50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 -- 172.21.15.71:0/23752555 >> 172.21.15.71:0/23752555 conn(0x7f82d00aec60 msgr2=0x7f82d00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:32.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 -- 172.21.15.71:0/23752555 shutdown_connections 2023-10-26T05:48:32.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 -- 172.21.15.71:0/23752555 wait complete. 2023-10-26T05:48:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 Processor -- start 2023-10-26T05:48:32.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.637+0000 7f82d805b640 1 -- start start 2023-10-26T05:48:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 0x7f82d0159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d00b2390 0x7f82d01f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82d0141980 0x7f82d01f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d015cb50 con 0x7f82d00704b0 2023-10-26T05:48:32.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d01581a0 con 0x7f82d00b2390 2023-10-26T05:48:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d01609a0 con 0x7f82d0141980 2023-10-26T05:48:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 0x7f82d0159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 0x7f82d0159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39400/0 (socket says 172.21.15.71:39400) 2023-10-26T05:48:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 -- 172.21.15.71:0/420270372 learned_addr learned my addr 172.21.15.71:0/420270372 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:32.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d55f6640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d00b2390 0x7f82d01f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d65f8640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82d0141980 0x7f82d01f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 -- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82d0141980 msgr2=0x7f82d01f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82d0141980 0x7f82d01f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 -- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d00b2390 msgr2=0x7f82d01f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d00b2390 0x7f82d01f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 -- 172.21.15.71:0/420270372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc009560 con 0x7f82d00704b0 2023-10-26T05:48:32.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d55f6640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d00b2390 0x7f82d01f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:32.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d5df7640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 0x7f82d0159ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009850 tx=0x7f82cc0a6b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:32.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc004030 con 0x7f82d00704b0 2023-10-26T05:48:32.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82cc004280 con 0x7f82d00704b0 2023-10-26T05:48:32.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d0160bf0 con 0x7f82d00704b0 2023-10-26T05:48:32.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005040 con 0x7f82d00704b0 2023-10-26T05:48:32.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d01628b0 con 0x7f82d00704b0 2023-10-26T05:48:32.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d0066b10 con 0x7f82d00704b0 2023-10-26T05:48:32.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82cc0051a0 con 0x7f82d00704b0 2023-10-26T05:48:32.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82c67fc640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82b4066470 0x7f82b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82cc11e080 con 0x7f82d00704b0 2023-10-26T05:48:32.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.641+0000 7f82d55f6640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82b4066470 0x7f82b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.645+0000 7f82d55f6640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82b4066470 0x7f82b4068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82c0005fd0 tx=0x7f82c0005da0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:32.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.645+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82cc0eb590 con 0x7f82d00704b0 2023-10-26T05:48:32.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.861+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f82d00721e0 con 0x7f82d00704b0 2023-10-26T05:48:32.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82c67fc640 1 -- 172.21.15.71:0/420270372 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) v1 ==== 190+0+147 (secure 0 0 0) 0x7f82cc0ac0b0 con 0x7f82d00704b0 2023-10-26T05:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:48:32.869 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:48:32.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82b4066470 msgr2=0x7f82b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82b4066470 0x7f82b4068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f82c0005fd0 tx=0x7f82c0005da0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 msgr2=0x7f82d0159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 0x7f82d0159ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009850 tx=0x7f82cc0a6b60 comp rx=0 tx=0).stop 2023-10-26T05:48:32.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 shutdown_connections 2023-10-26T05:48:32.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82d0141980 0x7f82d01f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82d00b2390 0x7f82d01f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82b4066470 0x7f82b4068930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 --2- 172.21.15.71:0/420270372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82d00704b0 0x7f82d0159ab0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 >> 172.21.15.71:0/420270372 conn(0x7f82d00aec60 msgr2=0x7f82d00cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:32.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 shutdown_connections 2023-10-26T05:48:32.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.865+0000 7f82d805b640 1 -- 172.21.15.71:0/420270372 wait complete. 2023-10-26T05:48:32.886 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 -- 172.21.15.71:0/3236677833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0850 msgr2=0x7f1b9c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/3236677833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0850 0x7f1b9c15b3a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c08a140 tx=0x7f1b8c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 -- 172.21.15.71:0/3236677833 shutdown_connections 2023-10-26T05:48:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/3236677833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0850 0x7f1b9c15b3a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/3236677833 >> v2:172.21.15.86:3300/0 conn(0x7f1b9c0d0470 0x7f1b9c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:32.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 -- 172.21.15.71:0/3236677833 >> 172.21.15.71:0/3236677833 conn(0x7f1b9c0cdd80 msgr2=0x7f1b9c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:32.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 -- 172.21.15.71:0/3236677833 shutdown_connections 2023-10-26T05:48:32.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.989+0000 7f1ba47f4640 1 -- 172.21.15.71:0/3236677833 wait complete. 2023-10-26T05:48:32.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 Processor -- start 2023-10-26T05:48:32.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- start start 2023-10-26T05:48:32.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cd880 0x7f1b9c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 0x7f1b9c15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:32.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b9c0d0850 0x7f1b9c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c15abc0 con 0x7f1b9c0d0470 2023-10-26T05:48:33.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c15ce60 con 0x7f1b9c0cd880 2023-10-26T05:48:33.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c158e30 con 0x7f1b9c0d0850 2023-10-26T05:48:33.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 0x7f1b9c15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 0x7f1b9c15a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39420/0 (socket says 172.21.15.71:39420) 2023-10-26T05:48:33.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 -- 172.21.15.71:0/1098238420 learned_addr learned my addr 172.21.15.71:0/1098238420 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 -- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b9c0d0850 msgr2=0x7f1b9c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b9c0d0850 0x7f1b9c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 -- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cd880 msgr2=0x7f1b9c1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba2590640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cd880 0x7f1b9c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cd880 0x7f1b9c1532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 -- 172.21.15.71:0/1098238420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c009560 con 0x7f1b9c0d0470 2023-10-26T05:48:33.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba1d8f640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 0x7f1b9c15a050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009850 tx=0x7f1b8c0a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c003800 con 0x7f1b9c0d0470 2023-10-26T05:48:33.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b8c003960 con 0x7f1b9c0d0470 2023-10-26T05:48:33.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c159110 con 0x7f1b9c0d0470 2023-10-26T05:48:33.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c003d70 con 0x7f1b9c0d0470 2023-10-26T05:48:33.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c1596d0 con 0x7f1b9c0d0470 2023-10-26T05:48:33.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c066b10 con 0x7f1b9c0d0470 2023-10-26T05:48:33.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b8c0b4040 con 0x7f1b9c0d0470 2023-10-26T05:48:33.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1b92ffd640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b80066470 0x7f1b80068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.993+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b8c11eb40 con 0x7f1b9c0d0470 2023-10-26T05:48:33.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.997+0000 7f1ba2590640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b80066470 0x7f1b80068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.997+0000 7f1ba2590640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b80066470 0x7f1b80068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800ba40 tx=0x7f1b98005cd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:32.997+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b8c0ec110 con 0x7f1b9c0d0470 2023-10-26T05:48:33.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b9c1598c0 con 0x7f1b9c0d0470 2023-10-26T05:48:33.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1b92ffd640 1 -- 172.21.15.71:0/1098238420 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f1b8c0ef730 con 0x7f1b9c0d0470 2023-10-26T05:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:33.179 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:33.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b80066470 msgr2=0x7f1b80068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b80066470 0x7f1b80068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800ba40 tx=0x7f1b98005cd0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 msgr2=0x7f1b9c15a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 0x7f1b9c15a050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009850 tx=0x7f1b8c0a56e0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 shutdown_connections 2023-10-26T05:48:33.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b9c0d0850 0x7f1b9c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b80066470 0x7f1b80068930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b9c0d0470 0x7f1b9c15a050 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 --2- 172.21.15.71:0/1098238420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b9c0cd880 0x7f1b9c1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 >> 172.21.15.71:0/1098238420 conn(0x7f1b9c0cdd80 msgr2=0x7f1b9c154d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 shutdown_connections 2023-10-26T05:48:33.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.173+0000 7f1ba47f4640 1 -- 172.21.15.71:0/1098238420 wait complete. 2023-10-26T05:48:33.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:48:24.033549+0000', 'modified': '2023-10-26T05:48:26.052225+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5280}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5280': {'gid': 5280, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.71:6835/4006610413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 4006610413}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 4006610413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:48:33.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:48:33.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T05:48:33.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 -- 172.21.15.71:0/3532151892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824069d30 msgr2=0x7f782406a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 --2- 172.21.15.71:0/3532151892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824069d30 0x7f782406a1c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f781c009bc0 tx=0x7f781c01cab0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 -- 172.21.15.71:0/3532151892 shutdown_connections 2023-10-26T05:48:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 --2- 172.21.15.71:0/3532151892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824069d30 0x7f782406a1c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 --2- 172.21.15.71:0/3532151892 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7824159700 0x7f78240b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 -- 172.21.15.71:0/3532151892 >> 172.21.15.71:0/3532151892 conn(0x7f782406b8c0 msgr2=0x7f782406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 -- 172.21.15.71:0/3532151892 shutdown_connections 2023-10-26T05:48:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 -- 172.21.15.71:0/3532151892 wait complete. 2023-10-26T05:48:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 Processor -- start 2023-10-26T05:48:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.729+0000 7f782c1d9640 1 -- start start 2023-10-26T05:48:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7824069d30 0x7f782414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78240ca7f0 0x7f782413e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 0x7f782413cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782414ffa0 con 0x7f7824159700 2023-10-26T05:48:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7824152e00 con 0x7f78240ca7f0 2023-10-26T05:48:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782413ed60 con 0x7f7824069d30 2023-10-26T05:48:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 0x7f782413cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 0x7f782413cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39430/0 (socket says 172.21.15.71:39430) 2023-10-26T05:48:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 -- 172.21.15.71:0/2806269470 learned_addr learned my addr 172.21.15.71:0/2806269470 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f7829774640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78240ca7f0 0x7f782413e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 -- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7824069d30 msgr2=0x7f782414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f7829f75640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7824069d30 0x7f782414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7824069d30 0x7f782414fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 -- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78240ca7f0 msgr2=0x7f782413e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78240ca7f0 0x7f782413e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 -- 172.21.15.71:0/2806269470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009820 con 0x7f7824159700 2023-10-26T05:48:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f7829774640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78240ca7f0 0x7f782413e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782a776640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 0x7f782413cf80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f781c01a6e0 tx=0x7f781c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f781a7fc640 1 -- 172.21.15.71:0/2806269470 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0ba070 con 0x7f7824159700 2023-10-26T05:48:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f781a7fc640 1 -- 172.21.15.71:0/2806269470 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f781c0a5e90 con 0x7f7824159700 2023-10-26T05:48:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782413d4c0 con 0x7f7824159700 2023-10-26T05:48:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f781a7fc640 1 -- 172.21.15.71:0/2806269470 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c004c90 con 0x7f7824159700 2023-10-26T05:48:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782413da60 con 0x7f7824159700 2023-10-26T05:48:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f781a7fc640 1 -- 172.21.15.71:0/2806269470 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f781c0b4040 con 0x7f7824159700 2023-10-26T05:48:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f781a7fc640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7808066470 0x7f7808068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f781a7fc640 1 -- 172.21.15.71:0/2806269470 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f781c11e340 con 0x7f7824159700 2023-10-26T05:48:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.733+0000 7f7829f75640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7808066470 0x7f7808068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 Processor -- start 2023-10-26T05:48:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- start start 2023-10-26T05:48:33.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829f75640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7808066470 0x7f7808068930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78200011f0 tx=0x7f7820009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a2601dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 0x564a25fd3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a25fd3f80 0x564a25fd4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25dc8cc0 con 0x564a2601e2e0 2023-10-26T05:48:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25e2b920 con 0x564a25fd3f80 2023-10-26T05:48:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a25e28e50 con 0x564a2601d9c0 2023-10-26T05:48:33.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 0x564a25fd3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 0x564a25fd3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39446/0 (socket says 172.21.15.71:39446) 2023-10-26T05:48:33.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 -- 172.21.15.71:0/3231762597 learned_addr learned my addr 172.21.15.71:0/3231762597 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829f75640 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a2601dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 -- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 msgr2=0x564a2601dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782a776640 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a25fd3f80 0x564a25fd4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a2601dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 -- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a25fd3f80 msgr2=0x564a25fd4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a25fd3f80 0x564a25fd4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 -- 172.21.15.71:0/3231762597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a25b1d890 con 0x564a2601e2e0 2023-10-26T05:48:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f7829774640 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 0x564a25fd3a40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f78140058d0 tx=0x7f781400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f284f0b221a1d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f78077fe640 1 -- 172.21.15.71:0/3231762597 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781400de50 con 0x564a2601e2e0 2023-10-26T05:48:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f78077fe640 1 -- 172.21.15.71:0/3231762597 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7814017420 con 0x564a2601e2e0 2023-10-26T05:48:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f78077fe640 1 -- 172.21.15.71:0/3231762597 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814017830 con 0x564a2601e2e0 2023-10-26T05:48:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- 172.21.15.71:0/3231762597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 msgr2=0x564a25fd3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 0x564a25fd3a40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f78140058d0 tx=0x7f781400bec0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- 172.21.15.71:0/3231762597 shutdown_connections 2023-10-26T05:48:33.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a25fd3f80 0x564a25fd4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a2601e2e0 0x564a25fd3a40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- 172.21.15.71:0/3231762597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a2601dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- 172.21.15.71:0/3231762597 >> 172.21.15.71:0/3231762597 conn(0x564a25fcb2f0 msgr2=0x564a25fd2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- 172.21.15.71:0/3231762597 shutdown_connections 2023-10-26T05:48:33.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- 172.21.15.71:0/3231762597 wait complete. 2023-10-26T05:48:33.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 -- start start 2023-10-26T05:48:33.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 Processor -- start 2023-10-26T05:48:33.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a25fd9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.737+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601e2e0 0x564a25fd98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25e24740 con 0x564a25fd3f80 2023-10-26T05:48:33.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25de3d10 con 0x564a2601e2e0 2023-10-26T05:48:33.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a25de39b0 con 0x564a2601d9c0 2023-10-26T05:48:33.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 10 client.0 _ll_register_callbacks cb 0x564a25fbf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:48:33.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39462/0 (socket says 172.21.15.71:39462) 2023-10-26T05:48:33.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 -- 172.21.15.71:0/2308126320 learned_addr learned my addr 172.21.15.71:0/2308126320 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829774640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a25fd9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782a776640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601e2e0 0x564a25fd98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 -- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 msgr2=0x564a25fd9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a25fd9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 -- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601e2e0 msgr2=0x564a25fd98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601e2e0 0x564a25fd98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 -- 172.21.15.71:0/2308126320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781400b690 con 0x564a25fd3f80 2023-10-26T05:48:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829774640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a25fd9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829f75640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601d410 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f782000ddd0 tx=0x7f7820000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587df03c065786b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:48:33.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 1 -- 172.21.15.71:0/2308126320 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820020420 con 0x564a25fd3f80 2023-10-26T05:48:33.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 1 -- 172.21.15.71:0/2308126320 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7820020580 con 0x564a25fd3f80 2023-10-26T05:48:33.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a25dd8370 con 0x564a25fd3f80 2023-10-26T05:48:33.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 1 -- 172.21.15.71:0/2308126320 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7820020740 con 0x564a25fd3f80 2023-10-26T05:48:33.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 10 client.5487 fetch_fsmap learned FSMap version 12 2023-10-26T05:48:33.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x564a25de6460 con 0x564a25fd3f80 2023-10-26T05:48:33.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 1 -- 172.21.15.71:0/2308126320 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x564a25dd8370 con 0x564a25fd3f80 2023-10-26T05:48:33.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 1 -- 172.21.15.71:0/2308126320 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 12) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f78200208a0 con 0x564a25fd3f80 2023-10-26T05:48:33.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 10 client.5487 fetch_fsmap finished waiting for FSMap version 12 2023-10-26T05:48:33.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 10 client.5487 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:48:33.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 20 client.5487 populate_metadata read hostname 'smithi071' 2023-10-26T05:48:33.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a25fe0440 0x564a2602fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 4 client.5487 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:48:33.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 --> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a25dbe2d0 con 0x564a25fe0440 2023-10-26T05:48:33.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829774640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a25fe0440 0x564a2602fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7829774640 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a25fe0440 0x564a2602fa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576602267697b2e0 server_cookie=9b441f54bdbbf861 in_seq=0 out_seq=0 2023-10-26T05:48:33.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 10 client.5487 ms_handle_connect on v2:172.21.15.71:6834/4006610413 2023-10-26T05:48:33.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 1 -- 172.21.15.71:0/2308126320 <== mds.0 v2:172.21.15.71:6834/4006610413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a25dbe2d0 con 0x564a25fe0440 2023-10-26T05:48:33.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 10 client.5487 handle_command_reply: tid=0 2023-10-26T05:48:33.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 client.5487 shutdown 2023-10-26T05:48:33.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 msgr2=0x564a2601d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601d410 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f782000ddd0 tx=0x7f7820000ee0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 shutdown_connections 2023-10-26T05:48:33.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a25fe0440 0x564a2602fa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601e2e0 0x564a25fd98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601d9c0 0x564a25fd9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 --2- 172.21.15.71:0/2308126320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601d410 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f7804ff9640 0 client.5487 ms_handle_reset on v2:172.21.15.71:6834/4006610413 2023-10-26T05:48:33.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 >> 172.21.15.71:0/2308126320 conn(0x564a25fcb2f0 msgr2=0x564a25fd2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 shutdown_connections 2023-10-26T05:48:33.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 1 -- 172.21.15.71:0/2308126320 wait complete. 2023-10-26T05:48:33.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.741+0000 7f782f323000 20 client.5487 trim_cache size 0 max 16384 2023-10-26T05:48:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 Processor -- start 2023-10-26T05:48:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 -- start start 2023-10-26T05:48:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a2601bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 0x564a2602da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25ade940 con 0x564a25fd3f80 2023-10-26T05:48:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25b1e1b0 con 0x564a2601d9c0 2023-10-26T05:48:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a25fda090 con 0x564a2601e2e0 2023-10-26T05:48:33.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39468/0 (socket says 172.21.15.71:39468) 2023-10-26T05:48:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 -- 172.21.15.71:0/1387221069 learned_addr learned my addr 172.21.15.71:0/1387221069 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829774640 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a2601bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 -- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 msgr2=0x564a2602da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 0x564a2602da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 -- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 msgr2=0x564a2601bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a2601bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 -- 172.21.15.71:0/1387221069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a25dcc760 con 0x564a25fd3f80 2023-10-26T05:48:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829774640 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a2601bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:33.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7829f75640 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601ba00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7820001440 tx=0x7f7820001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dccc620e69695f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7807fff640 1 -- 172.21.15.71:0/1387221069 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782000d5d0 con 0x564a25fd3f80 2023-10-26T05:48:33.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7807fff640 1 -- 172.21.15.71:0/1387221069 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7820020d60 con 0x564a25fd3f80 2023-10-26T05:48:33.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.757+0000 7f7807fff640 1 -- 172.21.15.71:0/1387221069 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820020680 con 0x564a25fd3f80 2023-10-26T05:48:33.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/1387221069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 msgr2=0x564a2601ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601ba00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7820001440 tx=0x7f7820001160 comp rx=0 tx=0).stop 2023-10-26T05:48:33.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/1387221069 shutdown_connections 2023-10-26T05:48:33.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 0x564a2602da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a2601bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/1387221069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a2601ba00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/1387221069 >> 172.21.15.71:0/1387221069 conn(0x564a25fcb2f0 msgr2=0x564a25fdf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/1387221069 shutdown_connections 2023-10-26T05:48:33.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/1387221069 wait complete. 2023-10-26T05:48:33.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- start start 2023-10-26T05:48:33.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 Processor -- start 2023-10-26T05:48:33.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a26030ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a26041640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 0x564a26041b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25e2e200 con 0x564a25fd3f80 2023-10-26T05:48:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a25e200b0 con 0x564a2601d9c0 2023-10-26T05:48:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a25fd7ae0 con 0x564a2601e2e0 2023-10-26T05:48:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 10 client.0 _ll_register_callbacks cb 0x564a25e22ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:48:33.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a26030ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a26030ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39470/0 (socket says 172.21.15.71:39470) 2023-10-26T05:48:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 -- 172.21.15.71:0/470318053 learned_addr learned my addr 172.21.15.71:0/470318053 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 -- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 msgr2=0x564a26041b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 0x564a26041b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829774640 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a26041640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 -- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 msgr2=0x564a26041640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:48:33.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a26041640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 -- 172.21.15.71:0/470318053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a25de6460 con 0x564a25fd3f80 2023-10-26T05:48:33.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829f75640 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a26030ae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f782000d1c0 tx=0x7f7820020650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e410bed57e82264a server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:48:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 1 -- 172.21.15.71:0/470318053 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78200209d0 con 0x564a25fd3f80 2023-10-26T05:48:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 1 -- 172.21.15.71:0/470318053 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f782002a450 con 0x564a25fd3f80 2023-10-26T05:48:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a25fd66e0 con 0x564a25fd3f80 2023-10-26T05:48:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 1 -- 172.21.15.71:0/470318053 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f782002a610 con 0x564a25fd3f80 2023-10-26T05:48:33.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 10 client.5493 fetch_fsmap learned FSMap version 12 2023-10-26T05:48:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x564a25b1dcc0 con 0x564a25fd3f80 2023-10-26T05:48:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 1 -- 172.21.15.71:0/470318053 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x564a25fd66e0 con 0x564a25fd3f80 2023-10-26T05:48:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 1 -- 172.21.15.71:0/470318053 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 12) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f782002a770 con 0x564a25fd3f80 2023-10-26T05:48:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 10 client.5493 fetch_fsmap finished waiting for FSMap version 12 2023-10-26T05:48:33.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 10 client.5493 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:48:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 20 client.5493 populate_metadata read hostname 'smithi071' 2023-10-26T05:48:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a26084a70 0x564a26084e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 4 client.5493 mds_command: new command op to 5280 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:48:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 --> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a25dbc6f0 con 0x564a26084a70 2023-10-26T05:48:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829774640 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a26084a70 0x564a26084e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7829774640 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a26084a70 0x564a26084e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be875cf057b591d server_cookie=ef636c90134ed950 in_seq=0 out_seq=0 2023-10-26T05:48:33.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 10 client.5493 ms_handle_connect on v2:172.21.15.71:6834/4006610413 2023-10-26T05:48:33.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 1 -- 172.21.15.71:0/470318053 <== mds.0 v2:172.21.15.71:6834/4006610413 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564a25dbc6f0 con 0x564a26084a70 2023-10-26T05:48:33.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 10 client.5493 handle_command_reply: tid=0 2023-10-26T05:48:33.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 client.5493 shutdown 2023-10-26T05:48:33.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 msgr2=0x564a26030ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a26030ae0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f782000d1c0 tx=0x7f7820020650 comp rx=0 tx=0).stop 2023-10-26T05:48:33.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 shutdown_connections 2023-10-26T05:48:33.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:6834/4006610413,v1:172.21.15.71:6835/4006610413] conn(0x564a26084a70 0x564a26084e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x564a2601e2e0 0x564a26041b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f7804ff9640 0 client.5493 ms_handle_reset on v2:172.21.15.71:6834/4006610413 2023-10-26T05:48:33.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a2601d9c0 0x564a26041640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 --2- 172.21.15.71:0/470318053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564a25fd3f80 0x564a26030ae0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 >> 172.21.15.71:0/470318053 conn(0x564a25fcb2f0 msgr2=0x564a2601a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 shutdown_connections 2023-10-26T05:48:33.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 1 -- 172.21.15.71:0/470318053 wait complete. 2023-10-26T05:48:33.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.761+0000 7f782f323000 20 client.5493 trim_cache size 0 max 16384 2023-10-26T05:48:33.788 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:48:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:48:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5280, 2023-10-26T05:48:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:48:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:48:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:48:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 9.0664753779999998, 2023-10-26T05:48:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 12, 2023-10-26T05:48:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 34, 2023-10-26T05:48:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 34, 2023-10-26T05:48:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 17.063152809000002 2023-10-26T05:48:33.790 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:33.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7808066470 msgr2=0x7f7808068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7808066470 0x7f7808068930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f78200011f0 tx=0x7f7820009040 comp rx=0 tx=0).stop 2023-10-26T05:48:33.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 msgr2=0x7f782413cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 0x7f782413cf80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f781c01a6e0 tx=0x7f781c009300 comp rx=0 tx=0).stop 2023-10-26T05:48:33.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 shutdown_connections 2023-10-26T05:48:33.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7808066470 0x7f7808068930 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7824159700 0x7f782413cf80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78240ca7f0 0x7f782413e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 --2- 172.21.15.71:0/2806269470 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7824069d30 0x7f782414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 >> 172.21.15.71:0/2806269470 conn(0x7f782406b8c0 msgr2=0x7f782413baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 shutdown_connections 2023-10-26T05:48:33.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.765+0000 7f782c1d9640 1 -- 172.21.15.71:0/2806269470 wait complete. 2023-10-26T05:48:33.796 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:48:33.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- 172.21.15.71:0/2531658528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9000047d0 msgr2=0x7fc900004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2531658528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9000047d0 0x7fc900004c50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c700 tx=0x7fc90401c660 comp rx=0 tx=0).stop 2023-10-26T05:48:33.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- 172.21.15.71:0/2531658528 shutdown_connections 2023-10-26T05:48:33.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2531658528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9000047d0 0x7fc900004c50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- 172.21.15.71:0/2531658528 >> 172.21.15.71:0/2531658528 conn(0x7fc914069770 msgr2=0x7fc914069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:33.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- 172.21.15.71:0/2531658528 shutdown_connections 2023-10-26T05:48:33.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- 172.21.15.71:0/2531658528 wait complete. 2023-10-26T05:48:33.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 Processor -- start 2023-10-26T05:48:33.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- start start 2023-10-26T05:48:33.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9000047d0 0x7fc914159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 0x7fc914140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc914141000 0x7fc9141585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141523d0 con 0x7fc91415b4f0 2023-10-26T05:48:33.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141415f0 con 0x7fc9000047d0 2023-10-26T05:48:33.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc91b46c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141418d0 con 0x7fc914141000 2023-10-26T05:48:33.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc918a07640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 0x7fc914140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc918a07640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 0x7fc914140ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39492/0 (socket says 172.21.15.71:39492) 2023-10-26T05:48:33.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.905+0000 7fc918a07640 1 -- 172.21.15.71:0/2584736273 learned_addr learned my addr 172.21.15.71:0/2584736273 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:33.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc919a09640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc914141000 0x7fc9141585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc919208640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9000047d0 0x7fc914159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc918a07640 1 -- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc914141000 msgr2=0x7fc9141585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc918a07640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc914141000 0x7fc9141585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc918a07640 1 -- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9000047d0 msgr2=0x7fc914159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:33.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc918a07640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9000047d0 0x7fc914159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:33.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc918a07640 1 -- 172.21.15.71:0/2584736273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc914152060 con 0x7fc91415b4f0 2023-10-26T05:48:33.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc918a07640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 0x7fc914140ac0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc90401c490 tx=0x7fc904004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904006330 con 0x7fc91415b4f0 2023-10-26T05:48:33.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc904004e60 con 0x7fc91415b4f0 2023-10-26T05:48:33.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040a76b0 con 0x7fc91415b4f0 2023-10-26T05:48:33.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91415b970 con 0x7fc91415b4f0 2023-10-26T05:48:33.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc914151d10 con 0x7fc91415b4f0 2023-10-26T05:48:33.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc914141a90 con 0x7fc91415b4f0 2023-10-26T05:48:33.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc9040a7810 con 0x7fc91415b4f0 2023-10-26T05:48:33.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc909ffb640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8f80664a0 0x7fc8f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:33.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc90411e0b0 con 0x7fc91415b4f0 2023-10-26T05:48:33.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.909+0000 7fc919208640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8f80664a0 0x7fc8f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:33.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.913+0000 7fc919208640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8f80664a0 0x7fc8f8068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9100046e0 tx=0x7fc9100016f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:33.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:33.913+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc9040eb6a0 con 0x7fc91415b4f0 2023-10-26T05:48:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.097+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc914066b10 con 0x7fc91415b4f0 2023-10-26T05:48:34.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.097+0000 7fc909ffb640 1 -- 172.21.15.71:0/2584736273 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc9040ad060 con 0x7fc91415b4f0 2023-10-26T05:48:34.102 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:48:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8f80664a0 msgr2=0x7fc8f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:34.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8f80664a0 0x7fc8f8068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9100046e0 tx=0x7fc9100016f0 comp rx=0 tx=0).stop 2023-10-26T05:48:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 msgr2=0x7fc914140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 0x7fc914140ac0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc90401c490 tx=0x7fc904004c70 comp rx=0 tx=0).stop 2023-10-26T05:48:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 shutdown_connections 2023-10-26T05:48:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc914141000 0x7fc9141585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:34.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8f80664a0 0x7fc8f8068960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc91415b4f0 0x7fc914140ac0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 --2- 172.21.15.71:0/2584736273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9000047d0 0x7fc914159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 >> 172.21.15.71:0/2584736273 conn(0x7fc914069770 msgr2=0x7fc9141377a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 shutdown_connections 2023-10-26T05:48:34.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:34.101+0000 7fc91b46c640 1 -- 172.21.15.71:0/2584736273 wait complete. 2023-10-26T05:48:34.119 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:48:34.119 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:48:34.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:48:34.120 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:48:34.120 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:34.121 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:48:34.121 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:48:34.121 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:48:34.122 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:48:34.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.123 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:48:34.129 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:48:34.129 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:48:34.129 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:48:34.129 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.130 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:48:34.130 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.130 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:48:34.130 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:34.130 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:48:34.131 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:48:34.131 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:34.131 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:48:34.131 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:48:34.131 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:34.132 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:48:34.132 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:34.132 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:48:34.132 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42463sec preferred_lft 42463sec 2023-10-26T05:48:34.132 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:48:34.132 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.133 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:48:34.133 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:34.133 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:48:34.133 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.133 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:48:34.134 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:34.134 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:48:34.134 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.134 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:48:34.134 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.135 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:48:34.135 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:34.135 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:48:34.135 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.135 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:48:34.136 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:34.136 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:48:34.136 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:34.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.138 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:48:34.180 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:48:34.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.182 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:48:34.231 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:48:34.231 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:48:34.231 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:48:34.231 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:48:34.232 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:48:34.232 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:48:34.232 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:48:34.232 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:48:34.232 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:48:34.233 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:48:34.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.233 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:48:34.234 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:48:34.234 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:48:34.234 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:48:34.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:48:34.235 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:48:34.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.303 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:48:34.348 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:48:34.349 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:48:34.349 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:48:34.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.350 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:48:34.407 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:48:34.408 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:48:34.408 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:34.456 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:48:34.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:48:34.457 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:34.504 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:34.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.562 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:48:34.612 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:48:34.613 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:48:34.657 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:48:34.664 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:48:34.665 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:48:34.665 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:48:34.665 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:48:34.665 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:48:34.665 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:48:34.666 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:48:34.666 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:48:34.666 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:48:34.666 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:48:34.666 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:48:34.667 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:48:34.667 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:48:34.667 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:48:34.667 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:48:34.667 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:48:34.668 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:48:34.668 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:48:34.668 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:48:34.668 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:48:34.668 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:48:34.669 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:48:34.669 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:48:34.669 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:48:34.669 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:48:34.669 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:48:34.669 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:48:34.670 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:48:34.670 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:48:34.670 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:34.670 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:34.670 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:34.671 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:34.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:34.672 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:48:34.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:48:34.733+0000 7f2d4b6e1580 -1 init, newargv = 0x55e081a739f0 newargc=15 2023-10-26T05:48:34.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[38648]: starting ceph client 2023-10-26T05:48:34.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[38648]: starting fuse 2023-10-26T05:48:35.823 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:48:35.832 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:48:35.832 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:48:35.833 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:48:35.833 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:48:35.833 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:48:35.833 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:48:35.834 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:48:35.834 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:48:35.834 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:48:35.834 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:48:35.834 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:48:35.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:48:35.835 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:48:35.835 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:48:35.835 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:48:35.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:48:35.836 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:48:35.836 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:48:35.836 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:48:35.836 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:48:35.836 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:48:35.837 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:48:35.837 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:48:35.837 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:48:35.837 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:48:35.837 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:48:35.838 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:48:35.838 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:48:35.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:48:35.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:35.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:48:35.839 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:35.839 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:48:35.839 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:35.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:35.841 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:48:35.889 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:48:35.889 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:48:35.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:48:35.890 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:48:35.891 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:48:35.891 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:35.891 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:48:35.892 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:35.892 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:48:35.893 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:48:35.893 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:48:35.893 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:48:35.894 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:35.894 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:48:35.895 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:48:35.895 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:48:35.895 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:48:35.896 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:35.896 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:48:35.897 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:48:35.897 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:48:35.897 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:48:35.898 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:48:35.898 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:48:35.898 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:48:35.899 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:48:35.899 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:35.900 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:48:35.900 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:36.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:36.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:36.026 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:36.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.38648.asok status 2023-10-26T05:48:36.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:36.218 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:36.218 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:48:36.219 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:48:36.219 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:48:36.219 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:48:36.219 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:48:36.219 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:48:36.220 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "38648", 2023-10-26T05:48:36.220 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:48:36.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:36.220 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:48:36.220 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:48:36.220 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5505, 2023-10-26T05:48:36.221 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:48:36.221 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:48:36.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:48:36.221 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 5505 2023-10-26T05:48:36.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:36.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:36.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:36.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:36.222 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:36.222 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:36.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:36.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:36.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:36.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:36.223 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.5505 192.168.0.1:0/4143885926", 2023-10-26T05:48:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/4143885926", 2023-10-26T05:48:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:48:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 12, 2023-10-26T05:48:36.224 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 34, 2023-10-26T05:48:36.225 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:48:36.225 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:48:36.225 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:48:36.225 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:36.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:36.237 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:36.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:48:36.238 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:36.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:36.282 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:48:36.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:36.288 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:36.289 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:36.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:36.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:36.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:36.390 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:48:36.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 -- 172.21.15.71:0/1839830994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 msgr2=0x7f8d2c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:36.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 --2- 172.21.15.71:0/1839830994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c15aa20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009900 tx=0x7f8d2401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 -- 172.21.15.71:0/1839830994 shutdown_connections 2023-10-26T05:48:36.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 --2- 172.21.15.71:0/1839830994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c15aa20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 --2- 172.21.15.71:0/1839830994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d2c15c720 0x7f8d2c0b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 -- 172.21.15.71:0/1839830994 >> 172.21.15.71:0/1839830994 conn(0x7f8d2c0b0f20 msgr2=0x7f8d2c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:36.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 -- 172.21.15.71:0/1839830994 shutdown_connections 2023-10-26T05:48:36.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 -- 172.21.15.71:0/1839830994 wait complete. 2023-10-26T05:48:36.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 Processor -- start 2023-10-26T05:48:36.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.513+0000 7f8d32c01640 1 -- start start 2023-10-26T05:48:36.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d2c0cede0 0x7f8d2c14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:36.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d2c15c720 0x7f8d2c150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:36.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c137b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:36.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c06d910 con 0x7f8d2c15d560 2023-10-26T05:48:36.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c150ec0 con 0x7f8d2c0cede0 2023-10-26T05:48:36.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c06dbb0 con 0x7f8d2c15c720 2023-10-26T05:48:36.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c137b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:36.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c137b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39506/0 (socket says 172.21.15.71:39506) 2023-10-26T05:48:36.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 -- 172.21.15.71:0/3734393062 learned_addr learned my addr 172.21.15.71:0/3734393062 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:36.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3099d640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d2c0cede0 0x7f8d2c14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:36.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d2bfff640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d2c15c720 0x7f8d2c150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:36.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 -- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d2c15c720 msgr2=0x7f8d2c150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:36.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d2c15c720 0x7f8d2c150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 -- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d2c0cede0 msgr2=0x7f8d2c14bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:36.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d2c0cede0 0x7f8d2c14bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 -- 172.21.15.71:0/3734393062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009560 con 0x7f8d2c15d560 2023-10-26T05:48:36.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3119e640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c137b60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8d240b0040 tx=0x7f8d240a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:36.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24005230 con 0x7f8d2c15d560 2023-10-26T05:48:36.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d24005480 con 0x7f8d2c15d560 2023-10-26T05:48:36.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c06de90 con 0x7f8d2c15d560 2023-10-26T05:48:36.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24004930 con 0x7f8d2c15d560 2023-10-26T05:48:36.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c14c610 con 0x7f8d2c15d560 2023-10-26T05:48:36.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c06dcf0 con 0x7f8d2c15d560 2023-10-26T05:48:36.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d240b4070 con 0x7f8d2c15d560 2023-10-26T05:48:36.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d297fa640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066470 0x7f8d0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:36.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d2411e2a0 con 0x7f8d2c15d560 2023-10-26T05:48:36.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.517+0000 7f8d3099d640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066470 0x7f8d0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:36.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.521+0000 7f8d3099d640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066470 0x7f8d0c068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005c50 tx=0x7f8d1c00a3b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:36.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.521+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d2c06dcf0 con 0x7f8d2c15d560 2023-10-26T05:48:36.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.709+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8d2c06dcf0 con 0x7f8d2c15d560 2023-10-26T05:48:36.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d297fa640 1 -- 172.21.15.71:0/3734393062 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8d2c06dcf0 con 0x7f8d2c15d560 2023-10-26T05:48:36.716 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:48:36.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066470 msgr2=0x7f8d0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:36.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066470 0x7f8d0c068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c005c50 tx=0x7f8d1c00a3b0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 msgr2=0x7f8d2c137b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c137b60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8d240b0040 tx=0x7f8d240a5730 comp rx=0 tx=0).stop 2023-10-26T05:48:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 shutdown_connections 2023-10-26T05:48:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066470 0x7f8d0c068930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d2c15d560 0x7f8d2c137b60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d2c15c720 0x7f8d2c150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 --2- 172.21.15.71:0/3734393062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d2c0cede0 0x7f8d2c14bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 >> 172.21.15.71:0/3734393062 conn(0x7f8d2c0b0f20 msgr2=0x7f8d2c159760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:36.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 shutdown_connections 2023-10-26T05:48:36.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:36.713+0000 7f8d32c01640 1 -- 172.21.15.71:0/3734393062 wait complete. 2023-10-26T05:48:36.734 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:48:36.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:48:36.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:48:36.735 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:48:36.736 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:36.736 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:48:36.736 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:48:36.737 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:48:36.737 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:48:36.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:36.738 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:48:36.746 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:48:36.746 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:48:36.746 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:48:36.747 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.747 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:48:36.747 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.748 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:48:36.748 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:36.748 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:48:36.748 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:48:36.748 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:36.749 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:48:36.749 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:48:36.749 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:36.749 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:48:36.749 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:36.750 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:48:36.750 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42454sec preferred_lft 42454sec 2023-10-26T05:48:36.750 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:48:36.750 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.750 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:48:36.751 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:36.751 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:48:36.751 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.751 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:48:36.752 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:48:36.752 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:48:36.752 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.752 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:48:36.752 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.753 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:48:36.753 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:36.753 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:48:36.753 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.754 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:48:36.754 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:36.754 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:48:36.755 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:48:36.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:36.756 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:48:36.802 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:48:36.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:36.804 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:48:36.855 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:48:36.855 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:48:36.855 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:48:36.856 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:48:36.856 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:48:36.856 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:48:36.856 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:48:36.856 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:48:36.857 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:48:36.857 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:48:36.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:36.858 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:48:36.858 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:48:36.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:48:36.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:48:36.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:48:36.859 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:48:36.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:36.935 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:48:36.983 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:48:36.983 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:48:36.984 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:48:36.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:36.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:48:37.039 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:48:37.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:48:37.040 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:37.092 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:48:37.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:48:37.093 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:37.138 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:48:37.190 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:48:37.191 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:48:37.191 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:48:37.191 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:48:37.191 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:48:37.191 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:48:37.192 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:48:37.192 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:48:37.192 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:48:37.192 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:48:37.192 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:48:37.192 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:48:37.193 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:48:37.193 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:48:37.193 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:48:37.193 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:48:37.193 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:48:37.194 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:48:37.194 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:48:37.194 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:48:37.194 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:48:37.194 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:48:37.195 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:48:37.195 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:48:37.195 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:48:37.195 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:48:37.195 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:48:37.195 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:48:37.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:48:37.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:37.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:37.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:37.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:37.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:37.198 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:48:37.243 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:48:37.244 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:48:37.289 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:48:37.296 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:48:37.296 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:48:37.296 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:48:37.297 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:48:37.297 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:48:37.297 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:48:37.297 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:48:37.298 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:48:37.298 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:48:37.298 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:48:37.298 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:48:37.298 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:48:37.299 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:48:37.299 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:48:37.299 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:48:37.299 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:48:37.300 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:48:37.300 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:48:37.300 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:48:37.300 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:48:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:48:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:48:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:48:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:48:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:48:37.301 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:48:37.302 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:48:37.302 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:48:37.302 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:48:37.302 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:37.303 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:37.303 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:37.303 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:37.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:37.306 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:48:37.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:48:37.369+0000 7f1053a0d580 -1 init, newargv = 0x555fd068b500 newargc=15 2023-10-26T05:48:37.372 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34055]: starting ceph client 2023-10-26T05:48:37.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34055]: starting fuse 2023-10-26T05:48:38.452 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:48:38.460 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:48:38.460 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:48:38.461 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:48:38.461 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:48:38.461 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:48:38.461 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:48:38.461 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:48:38.462 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:48:38.462 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:48:38.462 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:48:38.462 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:48:38.462 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:48:38.463 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:48:38.463 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:48:38.463 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:48:38.463 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:48:38.463 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:48:38.464 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:48:38.464 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:48:38.464 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:48:38.464 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:48:38.464 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:48:38.465 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:48:38.465 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:48:38.465 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:48:38.465 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:48:38.465 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:48:38.466 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:48:38.466 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:48:38.466 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:38.466 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:48:38.466 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:38.467 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:48:38.467 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:38.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:38.468 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:48:38.515 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:48:38.516 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:48:38.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:48:38.517 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:48:38.517 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:48:38.518 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:48:38.518 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:48:38.518 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:48:38.519 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:48:38.519 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:48:38.520 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:48:38.520 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:48:38.520 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:48:38.521 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:48:38.521 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:48:38.521 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:48:38.522 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:48:38.522 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:48:38.523 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:48:38.523 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:48:38.523 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:48:38.524 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:48:38.524 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:48:38.524 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:48:38.525 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:48:38.525 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:48:38.526 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:48:38.526 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:48:38.526 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:48:38.640 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.34055.asok 2023-10-26T05:48:38.643 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:38.651 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34055.asok 2023-10-26T05:48:38.651 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:38.652 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.34055.asok status 2023-10-26T05:48:38.739 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:38.841 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:48:38.842 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:48:38.842 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:48:38.842 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:48:38.843 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:48:38.843 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:48:38.843 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:48:38.843 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34055", 2023-10-26T05:48:38.843 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:48:38.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:48:38.844 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:48:38.844 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:48:38.844 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4754, 2023-10-26T05:48:38.844 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:48:38.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:48:38.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:48:38.845 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4754 2023-10-26T05:48:38.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:48:38.845 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:48:38.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:48:38.846 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:38.846 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1342550074 2023-10-26T05:48:38.846 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:48:38.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:48:38.846 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:48:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:48:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1342550074 2023-10-26T05:48:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:48:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4754 192.168.0.1:0/1342550074", 2023-10-26T05:48:38.847 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1342550074", 2023-10-26T05:48:38.848 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:48:38.848 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 12, 2023-10-26T05:48:38.848 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 34, 2023-10-26T05:48:38.848 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:48:38.848 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:48:38.849 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:48:38.849 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:48:38.855 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:38.862 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:38.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:48:38.863 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:38.957 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:38.959 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:48:38.959 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:38.965 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:38.966 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:38.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:39.017 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:39.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:39.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.153+0000 7fce18d64640 1 -- 172.21.15.71:0/1673305610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0470 msgr2=0x7fce1406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.153+0000 7fce18d64640 1 --2- 172.21.15.71:0/1673305610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0470 0x7fce1406aa70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fce04009840 tx=0x7fce0401c700 comp rx=0 tx=0).stop 2023-10-26T05:48:39.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.153+0000 7fce18d64640 1 -- 172.21.15.71:0/1673305610 shutdown_connections 2023-10-26T05:48:39.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 --2- 172.21.15.71:0/1673305610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0470 0x7fce1406aa70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- 172.21.15.71:0/1673305610 >> 172.21.15.71:0/1673305610 conn(0x7fce140cdd80 msgr2=0x7fce140ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:39.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- 172.21.15.71:0/1673305610 shutdown_connections 2023-10-26T05:48:39.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- 172.21.15.71:0/1673305610 wait complete. 2023-10-26T05:48:39.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 Processor -- start 2023-10-26T05:48:39.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- start start 2023-10-26T05:48:39.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce140cd880 0x7fce14155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce140d0470 0x7fce140c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 0x7fce140c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1406a650 con 0x7fce140d0850 2023-10-26T05:48:39.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14150330 con 0x7fce140cd880 2023-10-26T05:48:39.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce14156090 con 0x7fce140d0470 2023-10-26T05:48:39.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 0x7fce140c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 0x7fce140c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39532/0 (socket says 172.21.15.71:39532) 2023-10-26T05:48:39.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 -- 172.21.15.71:0/3990695215 learned_addr learned my addr 172.21.15.71:0/3990695215 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:39.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce11d9b640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce140d0470 0x7fce140c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce1259c640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce140cd880 0x7fce14155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 -- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce140d0470 msgr2=0x7fce140c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce140d0470 0x7fce140c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 -- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce140cd880 msgr2=0x7fce14155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce140cd880 0x7fce14155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 -- 172.21.15.71:0/3990695215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce040094a0 con 0x7fce140d0850 2023-10-26T05:48:39.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce11d9b640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce140d0470 0x7fce140c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:39.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce1259c640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce140cd880 0x7fce14155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:39.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce12d9d640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 0x7fce140c4e30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00cae0 tx=0x7fce0c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:39.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c003b80 con 0x7fce140d0850 2023-10-26T05:48:39.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce0c003dd0 con 0x7fce140d0850 2023-10-26T05:48:39.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1414f720 con 0x7fce140d0850 2023-10-26T05:48:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00aab0 con 0x7fce140d0850 2023-10-26T05:48:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1414fce0 con 0x7fce140d0850 2023-10-26T05:48:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.157+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce14066b10 con 0x7fce140d0850 2023-10-26T05:48:39.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.161+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce0c00ac10 con 0x7fce140d0850 2023-10-26T05:48:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.161+0000 7fce02ffd640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdf4066470 0x7fcdf4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.161+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce0c084550 con 0x7fce140d0850 2023-10-26T05:48:39.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.161+0000 7fce1259c640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdf4066470 0x7fcdf4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.161+0000 7fce1259c640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdf4066470 0x7fcdf4068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce040af000 tx=0x7fce0401af40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:39.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.165+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce0c051930 con 0x7fce140d0850 2023-10-26T05:48:39.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce141504e0 con 0x7fce140d0850 2023-10-26T05:48:39.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce02ffd640 1 -- 172.21.15.71:0/3990695215 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fce141504e0 con 0x7fce140d0850 2023-10-26T05:48:39.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:39.357 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:39.357 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdf4066470 msgr2=0x7fcdf4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdf4066470 0x7fcdf4068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fce040af000 tx=0x7fce0401af40 comp rx=0 tx=0).stop 2023-10-26T05:48:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 msgr2=0x7fce140c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 0x7fce140c4e30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00cae0 tx=0x7fce0c00cbd0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 shutdown_connections 2023-10-26T05:48:39.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdf4066470 0x7fcdf4068930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce140d0850 0x7fce140c4e30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce140d0470 0x7fce140c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 --2- 172.21.15.71:0/3990695215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce140cd880 0x7fce14155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 >> 172.21.15.71:0/3990695215 conn(0x7fce140cdd80 msgr2=0x7fce140b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:39.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 shutdown_connections 2023-10-26T05:48:39.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.353+0000 7fce18d64640 1 -- 172.21.15.71:0/3990695215 wait complete. 2023-10-26T05:48:39.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T05:48:39.513 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":5280,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.798842350999999,"mdsmap_epoch":12,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":22.795523380999999} 2023-10-26T05:48:39.513 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 5280, "mdsmap_epoch": 12, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 14.798842351, "state": "up:active", "uptime": 22.795523381, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:48:39.514 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-10-26T05:48:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 -- 172.21.15.71:0/3244331750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 msgr2=0x7f07b815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 --2- 172.21.15.71:0/3244331750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b815bab0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f07b808a140 tx=0x7f07a801c7a0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 -- 172.21.15.71:0/3244331750 shutdown_connections 2023-10-26T05:48:39.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 --2- 172.21.15.71:0/3244331750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b815bab0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 --2- 172.21.15.71:0/3244331750 >> v2:172.21.15.86:3300/0 conn(0x7f07b80d0470 0x7f07b806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 -- 172.21.15.71:0/3244331750 >> 172.21.15.71:0/3244331750 conn(0x7f07b80cdd80 msgr2=0x7f07b80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:39.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 -- 172.21.15.71:0/3244331750 shutdown_connections 2023-10-26T05:48:39.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.625+0000 7f07bff86640 1 -- 172.21.15.71:0/3244331750 wait complete. 2023-10-26T05:48:39.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 Processor -- start 2023-10-26T05:48:39.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- start start 2023-10-26T05:48:39.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b8151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b80d0470 0x7f07b815e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07b80d0850 0x7f07b8159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b81528c0 con 0x7f07b80cd880 2023-10-26T05:48:39.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b8151cb0 con 0x7f07b80d0470 2023-10-26T05:48:39.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b8151f90 con 0x7f07b80d0850 2023-10-26T05:48:39.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b8151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b8151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39554/0 (socket says 172.21.15.71:39554) 2023-10-26T05:48:39.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 -- 172.21.15.71:0/2001983390 learned_addr learned my addr 172.21.15.71:0/2001983390 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:39.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bd521640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b80d0470 0x7f07b815e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07be523640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07b80d0850 0x7f07b8159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 -- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07b80d0850 msgr2=0x7f07b8159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07b80d0850 0x7f07b8159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 -- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b80d0470 msgr2=0x7f07b815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b80d0470 0x7f07b815e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 -- 172.21.15.71:0/2001983390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8009560 con 0x7f07b80cd880 2023-10-26T05:48:39.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bdd22640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b8151620 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f07b000ead0 tx=0x7f07b000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:39.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b000ce40 con 0x7f07b80cd880 2023-10-26T05:48:39.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07b0004c40 con 0x7f07b80cd880 2023-10-26T05:48:39.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b8152270 con 0x7f07b80cd880 2023-10-26T05:48:39.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b0005020 con 0x7f07b80cd880 2023-10-26T05:48:39.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b815db60 con 0x7f07b80cd880 2023-10-26T05:48:39.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b8066b10 con 0x7f07b80cd880 2023-10-26T05:48:39.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.629+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07b0005180 con 0x7f07b80cd880 2023-10-26T05:48:39.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.633+0000 7f07b67fc640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a4066470 0x7f07a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:39.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.633+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07b00849d0 con 0x7f07b80cd880 2023-10-26T05:48:39.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.633+0000 7f07bd521640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a4066470 0x7f07a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:39.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.633+0000 7f07bd521640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a4066470 0x7f07a4068930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07b815a330 tx=0x7f07ac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:39.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.637+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07b0051dc0 con 0x7f07b80cd880 2023-10-26T05:48:39.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.845+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f07b815de40 con 0x7f07b80cd880 2023-10-26T05:48:39.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.845+0000 7f07b67fc640 1 -- 172.21.15.71:0/2001983390 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7f07b001f030 con 0x7f07b80cd880 2023-10-26T05:48:39.851 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:39.851 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:39.851 INFO:teuthology.orchestra.run.smithi071.stdout: "stats": { 2023-10-26T05:48:39.852 INFO:teuthology.orchestra.run.smithi071.stdout: "total_bytes": 1600000000, 2023-10-26T05:48:39.852 INFO:teuthology.orchestra.run.smithi071.stdout: "total_avail_bytes": 1597597908, 2023-10-26T05:48:39.852 INFO:teuthology.orchestra.run.smithi071.stdout: "total_used_bytes": 2402092, 2023-10-26T05:48:39.852 INFO:teuthology.orchestra.run.smithi071.stdout: "total_used_raw_bytes": 2402092, 2023-10-26T05:48:39.852 INFO:teuthology.orchestra.run.smithi071.stdout: "total_used_raw_ratio": 0.0015013074735179543, 2023-10-26T05:48:39.853 INFO:teuthology.orchestra.run.smithi071.stdout: "num_osds": 8, 2023-10-26T05:48:39.853 INFO:teuthology.orchestra.run.smithi071.stdout: "num_per_pool_osds": 0, 2023-10-26T05:48:39.853 INFO:teuthology.orchestra.run.smithi071.stdout: "num_per_pool_omap_osds": 0 2023-10-26T05:48:39.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:39.853 INFO:teuthology.orchestra.run.smithi071.stdout: "stats_by_class": { 2023-10-26T05:48:39.853 INFO:teuthology.orchestra.run.smithi071.stdout: "hdd": { 2023-10-26T05:48:39.854 INFO:teuthology.orchestra.run.smithi071.stdout: "total_bytes": 1600000000, 2023-10-26T05:48:39.854 INFO:teuthology.orchestra.run.smithi071.stdout: "total_avail_bytes": 1597597908, 2023-10-26T05:48:39.854 INFO:teuthology.orchestra.run.smithi071.stdout: "total_used_bytes": 2402092, 2023-10-26T05:48:39.854 INFO:teuthology.orchestra.run.smithi071.stdout: "total_used_raw_bytes": 2402092, 2023-10-26T05:48:39.854 INFO:teuthology.orchestra.run.smithi071.stdout: "total_used_raw_ratio": 0.0015013074735179543 2023-10-26T05:48:39.855 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:39.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:39.855 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:48:39.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:39.855 INFO:teuthology.orchestra.run.smithi071.stdout: "name": ".mgr", 2023-10-26T05:48:39.856 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 1, 2023-10-26T05:48:39.856 INFO:teuthology.orchestra.run.smithi071.stdout: "stats": { 2023-10-26T05:48:39.856 INFO:teuthology.orchestra.run.smithi071.stdout: "stored": 590368, 2023-10-26T05:48:39.856 INFO:teuthology.orchestra.run.smithi071.stdout: "objects": 2, 2023-10-26T05:48:39.856 INFO:teuthology.orchestra.run.smithi071.stdout: "kb_used": 577, 2023-10-26T05:48:39.857 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 590368, 2023-10-26T05:48:39.857 INFO:teuthology.orchestra.run.smithi071.stdout: "percent_used": 0.00052964512724429369, 2023-10-26T05:48:39.857 INFO:teuthology.orchestra.run.smithi071.stdout: "max_avail": 557028928 2023-10-26T05:48:39.857 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:39.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:39.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:39.858 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs_metadata", 2023-10-26T05:48:39.858 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 4, 2023-10-26T05:48:39.858 INFO:teuthology.orchestra.run.smithi071.stdout: "stats": { 2023-10-26T05:48:39.858 INFO:teuthology.orchestra.run.smithi071.stdout: "stored": 2410, 2023-10-26T05:48:39.859 INFO:teuthology.orchestra.run.smithi071.stdout: "objects": 22, 2023-10-26T05:48:39.859 INFO:teuthology.orchestra.run.smithi071.stdout: "kb_used": 3, 2023-10-26T05:48:39.859 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 2410, 2023-10-26T05:48:39.859 INFO:teuthology.orchestra.run.smithi071.stdout: "percent_used": 2.1632581592712086e-06, 2023-10-26T05:48:39.860 INFO:teuthology.orchestra.run.smithi071.stdout: "max_avail": 557028928 2023-10-26T05:48:39.860 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:39.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:39.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:39.860 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephfs_data", 2023-10-26T05:48:39.860 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5, 2023-10-26T05:48:39.861 INFO:teuthology.orchestra.run.smithi071.stdout: "stats": { 2023-10-26T05:48:39.861 INFO:teuthology.orchestra.run.smithi071.stdout: "stored": 0, 2023-10-26T05:48:39.861 INFO:teuthology.orchestra.run.smithi071.stdout: "objects": 0, 2023-10-26T05:48:39.861 INFO:teuthology.orchestra.run.smithi071.stdout: "kb_used": 0, 2023-10-26T05:48:39.861 INFO:teuthology.orchestra.run.smithi071.stdout: "bytes_used": 0, 2023-10-26T05:48:39.862 INFO:teuthology.orchestra.run.smithi071.stdout: "percent_used": 0, 2023-10-26T05:48:39.862 INFO:teuthology.orchestra.run.smithi071.stdout: "max_avail": 557028928 2023-10-26T05:48:39.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:39.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:39.862 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:48:39.863 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:39.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a4066470 msgr2=0x7f07a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a4066470 0x7f07a4068930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07b815a330 tx=0x7f07ac008040 comp rx=0 tx=0).stop 2023-10-26T05:48:39.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 msgr2=0x7f07b8151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:39.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b8151620 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f07b000ead0 tx=0x7f07b000ef90 comp rx=0 tx=0).stop 2023-10-26T05:48:39.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 shutdown_connections 2023-10-26T05:48:39.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07b80d0850 0x7f07b8159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07b80d0470 0x7f07b815e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a4066470 0x7f07a4068930 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 --2- 172.21.15.71:0/2001983390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07b80cd880 0x7f07b8151620 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:39.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 >> 172.21.15.71:0/2001983390 conn(0x7f07b80cdd80 msgr2=0x7f07b80b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 shutdown_connections 2023-10-26T05:48:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39.849+0000 7f07bff86640 1 -- 172.21.15.71:0/2001983390 wait complete. 2023-10-26T05:48:39.868 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:48:39.868 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:39.869 DEBUG:teuthology.orchestra.run.smithi071:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T05:48:39.870 DEBUG:teuthology.orchestra.run.smithi071:> os.fchmod(fd, 0O666) 2023-10-26T05:48:39.870 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(fd) 2023-10-26T05:48:39.871 DEBUG:teuthology.orchestra.run.smithi071:> os.close(fd) 2023-10-26T05:48:39.871 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:39.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:39.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:48:39.947 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:48:39.947 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:48:39.947 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:39.948 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:48:39.948 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:39.949 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:48:39.949 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:48:39.949 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:48:39.950 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:48:39.950 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:39.950 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:48:39.951 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:48:39.951 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:48:39.951 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:48:39.952 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:39.952 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:48:39.952 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:48:39.953 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:48:39.953 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:48:39.954 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:48:39.954 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:48:39.954 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:48:39.955 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:48:39.955 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:39.955 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:48:39.957 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:40.055 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:40.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:40.066 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:40.067 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:40.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.38648.asok status 2023-10-26T05:48:40.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:40.252 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:40.252 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:48:40.252 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:48:40.253 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:48:40.253 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:48:40.253 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:48:40.253 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:48:40.253 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "38648", 2023-10-26T05:48:40.253 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:48:40.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:40.254 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 1, 2023-10-26T05:48:40.254 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:48:40.254 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5505, 2023-10-26T05:48:40.254 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:48:40.255 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:48:40.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:48:40.255 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 5505 2023-10-26T05:48:40.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:40.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:40.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:40.257 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.5505 192.168.0.1:0/4143885926", 2023-10-26T05:48:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/4143885926", 2023-10-26T05:48:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 2, 2023-10-26T05:48:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 12, 2023-10-26T05:48:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 34, 2023-10-26T05:48:40.258 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 34, 2023-10-26T05:48:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:48:40.259 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:48:40.259 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:40.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:40.271 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-26T05:48:40.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/832299119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141af0 msgr2=0x7f52940b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 --2- 172.21.15.71:0/832299119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141af0 0x7f52940b2030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5290009900 tx=0x7f529001c850 comp rx=0 tx=0).stop 2023-10-26T05:48:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/832299119 shutdown_connections 2023-10-26T05:48:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 --2- 172.21.15.71:0/832299119 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5294141ed0 0x7f52941423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 --2- 172.21.15.71:0/832299119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141af0 0x7f52940b2030 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:40.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/832299119 >> 172.21.15.71:0/832299119 conn(0x7f529406b8c0 msgr2=0x7f529406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/832299119 shutdown_connections 2023-10-26T05:48:40.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/832299119 wait complete. 2023-10-26T05:48:40.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 Processor -- start 2023-10-26T05:48:40.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- start start 2023-10-26T05:48:40.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294141af0 0x7f5294158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:40.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 0x7f5294161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:40.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52941596b0 0x7f5294162250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:40.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5294163650 con 0x7f5294141ed0 2023-10-26T05:48:40.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5294162790 con 0x7f5294141af0 2023-10-26T05:48:40.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5294162a40 con 0x7f52941596b0 2023-10-26T05:48:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 0x7f5294161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 0x7f5294161d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39580/0 (socket says 172.21.15.71:39580) 2023-10-26T05:48:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 -- 172.21.15.71:0/947051819 learned_addr learned my addr 172.21.15.71:0/947051819 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529a17c640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52941596b0 0x7f5294162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529997b640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294141af0 0x7f5294158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 -- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52941596b0 msgr2=0x7f5294162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52941596b0 0x7f5294162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 -- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294141af0 msgr2=0x7f5294158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294141af0 0x7f5294158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 -- 172.21.15.71:0/947051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290009560 con 0x7f5294141ed0 2023-10-26T05:48:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529a17c640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52941596b0 0x7f5294162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529917a640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 0x7f5294161d10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f528400c970 tx=0x7f528400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284007bb0 con 0x7f5294141ed0 2023-10-26T05:48:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5284007e00 con 0x7f5294141ed0 2023-10-26T05:48:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5294162d20 con 0x7f5294141ed0 2023-10-26T05:48:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284005280 con 0x7f5294141ed0 2023-10-26T05:48:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.441+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52941632e0 con 0x7f5294141ed0 2023-10-26T05:48:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.445+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5294066b10 con 0x7f5294141ed0 2023-10-26T05:48:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.445+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f52840053e0 con 0x7f5294141ed0 2023-10-26T05:48:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.445+0000 7f528a7fc640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5278066470 0x7f5278068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.445+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5284084550 con 0x7f5294141ed0 2023-10-26T05:48:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.445+0000 7f529997b640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5278066470 0x7f5278068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.445+0000 7f529997b640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5278066470 0x7f5278068930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52900af000 tx=0x7f529001b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.449+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f52840553d0 con 0x7f5294141ed0 2023-10-26T05:48:40.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:40.613+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f529406c7d0 con 0x7f5294141ed0 2023-10-26T05:48:41.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.433+0000 7f528a7fc640 1 -- 172.21.15.71:0/947051819 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7f528401d020 con 0x7f5294141ed0 2023-10-26T05:48:41.437 INFO:teuthology.orchestra.run.smithi071.stderr:pauserd,pausewr is set 2023-10-26T05:48:41.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5278066470 msgr2=0x7f5278068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:41.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5278066470 0x7f5278068930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f52900af000 tx=0x7f529001b090 comp rx=0 tx=0).stop 2023-10-26T05:48:41.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 msgr2=0x7f5294161d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:41.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 0x7f5294161d10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f528400c970 tx=0x7f528400ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:41.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 shutdown_connections 2023-10-26T05:48:41.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f52941596b0 0x7f5294162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5278066470 0x7f5278068930 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5294141ed0 0x7f5294161d10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 --2- 172.21.15.71:0/947051819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5294141af0 0x7f5294158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 >> 172.21.15.71:0/947051819 conn(0x7f529406b8c0 msgr2=0x7f529413e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:41.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 shutdown_connections 2023-10-26T05:48:41.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.437+0000 7f529bbdf640 1 -- 172.21.15.71:0/947051819 wait complete. 2023-10-26T05:48:41.465 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-26T05:48:41.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- 172.21.15.71:0/3607932123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841591a0 msgr2=0x7f87840ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:41.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 --2- 172.21.15.71:0/3607932123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841591a0 0x7f87840ce9e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f87840cade0 tx=0x7f877c01c7f0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- 172.21.15.71:0/3607932123 shutdown_connections 2023-10-26T05:48:41.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 --2- 172.21.15.71:0/3607932123 >> v2:172.21.15.86:3300/0 conn(0x7f87840cd3a0 0x7f87840cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 --2- 172.21.15.71:0/3607932123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87841591a0 0x7f87840ce9e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- 172.21.15.71:0/3607932123 >> 172.21.15.71:0/3607932123 conn(0x7f878406fd90 msgr2=0x7f87840b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:41.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- 172.21.15.71:0/3607932123 shutdown_connections 2023-10-26T05:48:41.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- 172.21.15.71:0/3607932123 wait complete. 2023-10-26T05:48:41.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 Processor -- start 2023-10-26T05:48:41.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- start start 2023-10-26T05:48:41.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 0x7f878414f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:41.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87840cd3a0 0x7f87840b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:41.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87841591a0 0x7f87840b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:41.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878406cf60 con 0x7f878406ed80 2023-10-26T05:48:41.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878406c350 con 0x7f87840cd3a0 2023-10-26T05:48:41.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878891d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878406c630 con 0x7f87841591a0 2023-10-26T05:48:41.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 0x7f878414f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:41.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 0x7f878414f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39598/0 (socket says 172.21.15.71:39598) 2023-10-26T05:48:41.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 -- 172.21.15.71:0/456088320 learned_addr learned my addr 172.21.15.71:0/456088320 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:41.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 -- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87841591a0 msgr2=0x7f87840b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:41.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87841591a0 0x7f87840b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 -- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87840cd3a0 msgr2=0x7f87840b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:41.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87840cd3a0 0x7f87840b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:41.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.593+0000 7f878259c640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c009560 con 0x7f878406ed80 2023-10-26T05:48:41.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f878259c640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 0x7f878414f530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f877c019400 tx=0x7f877c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:41.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c003800 con 0x7f878406ed80 2023-10-26T05:48:41.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f877c0a58a0 con 0x7f878406ed80 2023-10-26T05:48:41.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878406c910 con 0x7f878406ed80 2023-10-26T05:48:41.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c003cf0 con 0x7f878406ed80 2023-10-26T05:48:41.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878414fa70 con 0x7f878406ed80 2023-10-26T05:48:41.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878406a7b0 con 0x7f878406ed80 2023-10-26T05:48:41.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f877c0b4040 con 0x7f878406ed80 2023-10-26T05:48:41.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8764066470 0x7f8764068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:41.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f877c11eae0 con 0x7f878406ed80 2023-10-26T05:48:41.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=36+}) v3 -- 0x7f876406dfa0 con 0x7f878406ed80 2023-10-26T05:48:41.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f8781d9b640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8764066470 0x7f8764068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:41.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.597+0000 7f8781d9b640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8764066470 0x7f8764068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f876c007fd0 tx=0x7f876c007d60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:41.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.601+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f877c0ba060 con 0x7f878406ed80 2023-10-26T05:48:41.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:41.765+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f878404db60 con 0x7f878406ed80 2023-10-26T05:48:42.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.441+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f877c0b2030 con 0x7f878406ed80 2023-10-26T05:48:42.446 INFO:teuthology.orchestra.run.smithi071.stderr:pauserd,pausewr is unset 2023-10-26T05:48:42.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.445+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 <== mon.0 v2:172.21.15.71:3300/0 8 ==== osd_map(36..36 src has 1..36) v4 ==== 288+0+0 (secure 0 0 0) 0x7f877c0f0320 con 0x7f878406ed80 2023-10-26T05:48:42.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.445+0000 7f877affd640 1 -- 172.21.15.71:0/456088320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=37}) v3 -- 0x7f876406e930 con 0x7f878406ed80 2023-10-26T05:48:42.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.445+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8764066470 msgr2=0x7f8764068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.445+0000 7f878891d640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8764066470 0x7f8764068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f876c007fd0 tx=0x7f876c007d60 comp rx=0 tx=0).stop 2023-10-26T05:48:42.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.445+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 msgr2=0x7f878414f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.445+0000 7f878891d640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 0x7f878414f530 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f877c019400 tx=0x7f877c0a56b0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 shutdown_connections 2023-10-26T05:48:42.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87841591a0 0x7f87840b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87840cd3a0 0x7f87840b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8764066470 0x7f8764068930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 --2- 172.21.15.71:0/456088320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f878406ed80 0x7f878414f530 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 >> 172.21.15.71:0/456088320 conn(0x7f878406fd90 msgr2=0x7f87840c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:42.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 shutdown_connections 2023-10-26T05:48:42.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.449+0000 7f878891d640 1 -- 172.21.15.71:0/456088320 wait complete. 2023-10-26T05:48:42.473 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:42.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/1853734425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b815d560 msgr2=0x7f32b815aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 --2- 172.21.15.71:0/1853734425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b815d560 0x7f32b815aa20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f32b8071290 tx=0x7f32b001cc60 comp rx=0 tx=0).stop 2023-10-26T05:48:42.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/1853734425 shutdown_connections 2023-10-26T05:48:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 --2- 172.21.15.71:0/1853734425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b815d560 0x7f32b815aa20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 --2- 172.21.15.71:0/1853734425 >> v2:172.21.15.86:3300/0 conn(0x7f32b815c720 0x7f32b80b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/1853734425 >> 172.21.15.71:0/1853734425 conn(0x7f32b80b0f20 msgr2=0x7f32b80b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:42.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/1853734425 shutdown_connections 2023-10-26T05:48:42.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/1853734425 wait complete. 2023-10-26T05:48:42.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 Processor -- start 2023-10-26T05:48:42.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- start start 2023-10-26T05:48:42.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 0x7f32b81532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:42.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32b815c720 0x7f32b806baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:42.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b815d560 0x7f32b8158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:42.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8153830 con 0x7f32b80cede0 2023-10-26T05:48:42.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80b3330 con 0x7f32b815d560 2023-10-26T05:48:42.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80b35d0 con 0x7f32b815c720 2023-10-26T05:48:42.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 0x7f32b81532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:42.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 0x7f32b81532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50696/0 (socket says 172.21.15.71:50696) 2023-10-26T05:48:42.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 -- 172.21.15.71:0/2527400140 learned_addr learned my addr 172.21.15.71:0/2527400140 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:42.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bdb2e640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b815d560 0x7f32b8158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:42.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 -- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32b815c720 msgr2=0x7f32b806baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bcb2c640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32b815c720 0x7f32b806baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:42.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32b815c720 0x7f32b806baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 -- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b815d560 msgr2=0x7f32b8158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b815d560 0x7f32b8158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 -- 172.21.15.71:0/2527400140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00098e0 con 0x7f32b80cede0 2023-10-26T05:48:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bcb2c640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32b815c720 0x7f32b806baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:42.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bd32d640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 0x7f32b81532f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b550 tx=0x7f32b400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:42.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4010870 con 0x7f32b80cede0 2023-10-26T05:48:42.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80b38b0 con 0x7f32b80cede0 2023-10-26T05:48:42.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f32b4010ae0 con 0x7f32b80cede0 2023-10-26T05:48:42.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b400f3c0 con 0x7f32b80cede0 2023-10-26T05:48:42.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.601+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b80b3e70 con 0x7f32b80cede0 2023-10-26T05:48:42.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.605+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8066d00 con 0x7f32b80cede0 2023-10-26T05:48:42.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.605+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f32b400f520 con 0x7f32b80cede0 2023-10-26T05:48:42.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.605+0000 7f32adffb640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f329c0664a0 0x7f329c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:42.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.605+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32b4084180 con 0x7f32b80cede0 2023-10-26T05:48:42.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.605+0000 7f32bcb2c640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f329c0664a0 0x7f329c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:42.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.605+0000 7f32bcb2c640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f329c0664a0 0x7f329c068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f32a80016e0 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:42.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.609+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32b4051580 con 0x7f32b80cede0 2023-10-26T05:48:42.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.765+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b80b00a0 con 0x7f32b80cede0 2023-10-26T05:48:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.765+0000 7f32adffb640 1 -- 172.21.15.71:0/2527400140 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f32b4054ba0 con 0x7f32b80cede0 2023-10-26T05:48:42.772 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:42.772 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:42.435500+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f329c0664a0 msgr2=0x7f329c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f329c0664a0 0x7f329c068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f32a80016e0 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2023-10-26T05:48:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 msgr2=0x7f32b81532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 0x7f32b81532f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f32b400b550 tx=0x7f32b400ba10 comp rx=0 tx=0).stop 2023-10-26T05:48:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 shutdown_connections 2023-10-26T05:48:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32b815d560 0x7f32b8158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32b815c720 0x7f32b806baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f329c0664a0 0x7f329c068960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 --2- 172.21.15.71:0/2527400140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b80cede0 0x7f32b81532f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 >> 172.21.15.71:0/2527400140 conn(0x7f32b80b0f20 msgr2=0x7f32b814fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 shutdown_connections 2023-10-26T05:48:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42.769+0000 7f32bf591640 1 -- 172.21.15.71:0/2527400140 wait complete. 2023-10-26T05:48:42.790 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:48:42.791 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:42.791 DEBUG:teuthology.orchestra.run.smithi071:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T05:48:42.792 DEBUG:teuthology.orchestra.run.smithi071:> os.fchmod(fd, 0O666) 2023-10-26T05:48:42.792 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(fd) 2023-10-26T05:48:42.792 DEBUG:teuthology.orchestra.run.smithi071:> os.close(fd) 2023-10-26T05:48:42.793 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:42.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:42.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:48:42.867 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:48:42.868 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:48:42.868 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:42.869 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:48:42.869 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:42.869 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:48:42.870 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:48:42.870 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:48:42.870 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:48:42.871 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:42.871 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:48:42.871 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:48:42.872 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:48:42.872 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:48:42.872 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:42.873 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:48:42.873 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:48:42.873 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:48:42.874 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:48:42.874 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:48:42.875 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:48:42.875 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:48:42.875 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:48:42.876 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:42.876 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:48:42.876 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:42.989 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:42.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:43.001 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:43.001 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:43.002 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.38648.asok status 2023-10-26T05:48:43.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:43.199 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:43.200 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:48:43.200 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:48:43.200 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:48:43.200 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:48:43.201 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:48:43.201 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:48:43.201 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "38648", 2023-10-26T05:48:43.202 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:48:43.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:43.202 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 1, 2023-10-26T05:48:43.202 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:48:43.202 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5505, 2023-10-26T05:48:43.203 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:48:43.203 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:48:43.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:48:43.203 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 5505 2023-10-26T05:48:43.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:43.204 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:43.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:43.205 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:43.205 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:43.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:43.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:43.205 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:43.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:43.206 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:43.206 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:43.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:43.207 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.5505 192.168.0.1:0/4143885926", 2023-10-26T05:48:43.207 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/4143885926", 2023-10-26T05:48:43.207 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 2, 2023-10-26T05:48:43.207 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 12, 2023-10-26T05:48:43.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 34, 2023-10-26T05:48:43.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 34, 2023-10-26T05:48:43.208 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:48:43.208 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:48:43.208 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:43.220 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/2231065887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0b2890 msgr2=0x7f9e8c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- 172.21.15.71:0/2231065887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c0caad0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c06f190 tx=0x7f9e8001c5e0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/2231065887 shutdown_connections 2023-10-26T05:48:43.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- 172.21.15.71:0/2231065887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c0cb010 0x7f9e8c06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- 172.21.15.71:0/2231065887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c0caad0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- 172.21.15.71:0/2231065887 >> v2:172.21.15.86:3300/0 conn(0x7f9e8c13c2f0 0x7f9e8c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/2231065887 >> 172.21.15.71:0/2231065887 conn(0x7f9e8c069770 msgr2=0x7f9e8c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:43.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/2231065887 shutdown_connections 2023-10-26T05:48:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/2231065887 wait complete. 2023-10-26T05:48:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 Processor -- start 2023-10-26T05:48:43.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- start start 2023-10-26T05:48:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:43.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 0x7f9e8c155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c13c2f0 0x7f9e8c13cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c13d0c0 con 0x7f9e8c0cb010 2023-10-26T05:48:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c135cd0 con 0x7f9e8c0b2890 2023-10-26T05:48:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c154560 con 0x7f9e8c13c2f0 2023-10-26T05:48:43.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 0x7f9e8c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 0x7f9e8c155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50714/0 (socket says 172.21.15.71:50714) 2023-10-26T05:48:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 -- 172.21.15.71:0/799186574 learned_addr learned my addr 172.21.15.71:0/799186574 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8b7fe640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8bfff640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c13c2f0 0x7f9e8c13cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:43.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 -- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c13c2f0 msgr2=0x7f9e8c13cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c13c2f0 0x7f9e8c13cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 -- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c0b2890 msgr2=0x7f9e8c153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:43.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 -- 172.21.15.71:0/799186574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80009560 con 0x7f9e8c0cb010 2023-10-26T05:48:43.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8b7fe640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8bfff640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c13c2f0 0x7f9e8c13cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e8affd640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 0x7f9e8c155140 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9e800ac040 tx=0x7f9e800aa5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:43.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800aaae0 con 0x7f9e8c0cb010 2023-10-26T05:48:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9e800b8070 con 0x7f9e8c0cb010 2023-10-26T05:48:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c154840 con 0x7f9e8c0cb010 2023-10-26T05:48:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80003600 con 0x7f9e8c0cb010 2023-10-26T05:48:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c154e00 con 0x7f9e8c0cb010 2023-10-26T05:48:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.337+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c066b10 con 0x7f9e8c0cb010 2023-10-26T05:48:43.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.341+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e80003760 con 0x7f9e8c0cb010 2023-10-26T05:48:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.341+0000 7f9e73fff640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e78066470 0x7f9e78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.341+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e800ba070 con 0x7f9e8c0cb010 2023-10-26T05:48:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.341+0000 7f9e8b7fe640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e78066470 0x7f9e78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:43.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.341+0000 7f9e8b7fe640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e78066470 0x7f9e78068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0058d0 tx=0x7f9e7c005cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:43.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.345+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e800ec530 con 0x7f9e8c0cb010 2023-10-26T05:48:43.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e8c0721e0 con 0x7f9e8c0cb010 2023-10-26T05:48:43.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e73fff640 1 -- 172.21.15.71:0/799186574 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f9e800efb50 con 0x7f9e8c0cb010 2023-10-26T05:48:43.533 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:43.533 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:43.534 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:43.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e78066470 msgr2=0x7f9e78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:43.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e78066470 0x7f9e78068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c0058d0 tx=0x7f9e7c005cc0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 msgr2=0x7f9e8c155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:43.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 0x7f9e8c155140 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9e800ac040 tx=0x7f9e800aa5f0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 shutdown_connections 2023-10-26T05:48:43.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e8c13c2f0 0x7f9e8c13cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e78066470 0x7f9e78068930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e8c0cb010 0x7f9e8c155140 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 --2- 172.21.15.71:0/799186574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8c0b2890 0x7f9e8c153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:43.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 >> 172.21.15.71:0/799186574 conn(0x7f9e8c069770 msgr2=0x7f9e8c152940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:43.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 shutdown_connections 2023-10-26T05:48:43.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43.529+0000 7f9e91aee640 1 -- 172.21.15.71:0/799186574 wait complete. 2023-10-26T05:48:43.551 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 36 2023-10-26T05:48:43.672 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-26T05:48:43.673 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:48:43.673 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:43.673 DEBUG:teuthology.orchestra.run.smithi071:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T05:48:43.674 DEBUG:teuthology.orchestra.run.smithi071:> os.fchmod(fd, 0O666) 2023-10-26T05:48:43.674 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(fd) 2023-10-26T05:48:43.674 DEBUG:teuthology.orchestra.run.smithi071:> os.close(fd) 2023-10-26T05:48:43.675 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:43.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:43.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:48:43.747 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:48:43.747 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:48:43.748 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:43.748 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:48:43.748 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:43.749 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:48:43.750 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:48:43.750 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:48:43.751 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:48:43.751 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:43.751 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:48:43.752 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:48:43.752 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:48:43.752 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:48:43.753 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:43.753 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:48:43.754 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:48:43.754 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:48:43.754 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:48:43.755 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:48:43.755 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:48:43.755 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:48:43.756 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:48:43.756 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:43.756 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:48:43.757 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:43.859 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:43.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:43.870 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:43.870 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:43.873 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.38648.asok status 2023-10-26T05:48:43.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:44.054 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:44.054 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:48:44.054 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:48:44.055 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:48:44.055 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:48:44.055 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:48:44.055 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:48:44.055 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "38648", 2023-10-26T05:48:44.056 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:48:44.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.056 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 2, 2023-10-26T05:48:44.056 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:48:44.056 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5505, 2023-10-26T05:48:44.057 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:48:44.057 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:48:44.057 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:48:44.057 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 5505 2023-10-26T05:48:44.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.058 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:44.058 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:44.058 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:44.058 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:44.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:44.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.059 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:44.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:44.059 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:44.059 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:44.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.060 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.5505 192.168.0.1:0/4143885926", 2023-10-26T05:48:44.060 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/4143885926", 2023-10-26T05:48:44.060 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 3, 2023-10-26T05:48:44.060 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 12, 2023-10-26T05:48:44.061 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 36, 2023-10-26T05:48:44.061 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 36, 2023-10-26T05:48:44.061 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:48:44.061 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:48:44.061 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:44.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:44.074 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:48:44.075 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:48:44.075 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:48:44.076 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:48:44.076 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:48:44.076 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:44.077 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:48:44.077 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:48:44.077 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:48:44.078 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:48:44.078 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:44.079 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:48:44.079 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:48:44.079 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:48:44.080 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:48:44.080 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:44.080 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:48:44.081 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:48:44.081 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:48:44.081 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:48:44.082 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:48:44.082 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:48:44.083 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:48:44.083 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:48:44.083 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:48:44.084 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:48:44.084 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:48:44.159 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:44.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:44.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38648.asok 2023-10-26T05:48:44.171 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:44.171 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.38648.asok status 2023-10-26T05:48:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:44.356 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:48:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:48:44.356 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:48:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:48:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:48:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:48:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "38648", 2023-10-26T05:48:44.357 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:48:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 2, 2023-10-26T05:48:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:48:44.358 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5505, 2023-10-26T05:48:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:48:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:48:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:48:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 5505 2023-10-26T05:48:44.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:44.360 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:48:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:48:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:48:44.361 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4143885926 2023-10-26T05:48:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.5505 192.168.0.1:0/4143885926", 2023-10-26T05:48:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/4143885926", 2023-10-26T05:48:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 3, 2023-10-26T05:48:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 12, 2023-10-26T05:48:44.363 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 36, 2023-10-26T05:48:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 36, 2023-10-26T05:48:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:48:44.366 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:48:44.367 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:44.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:44.375 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:48:44.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:48:44.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:48:44.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:48:44.472 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:48:44.473 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:44.473 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:44.480 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:44.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/1195068129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 msgr2=0x7ff8140b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:44.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/1195068129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 0x7ff8140b1900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00ad00 tx=0x7ff80c01de60 comp rx=0 tx=0).stop 2023-10-26T05:48:44.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/1195068129 shutdown_connections 2023-10-26T05:48:44.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/1195068129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 0x7ff8140b1900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/1195068129 >> 172.21.15.71:0/1195068129 conn(0x7ff814069770 msgr2=0x7ff814069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:44.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/1195068129 shutdown_connections 2023-10-26T05:48:44.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/1195068129 wait complete. 2023-10-26T05:48:44.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 Processor -- start 2023-10-26T05:48:44.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- start start 2023-10-26T05:48:44.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 0x7ff814147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:44.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81414e6e0 0x7ff814134160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:44.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 0x7ff8140ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:44.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140aeba0 con 0x7ff81414e300 2023-10-26T05:48:44.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8141346a0 con 0x7ff81414e6e0 2023-10-26T05:48:44.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140ae640 con 0x7ff81415b700 2023-10-26T05:48:44.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 0x7ff814147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:44.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 0x7ff814147b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50720/0 (socket says 172.21.15.71:50720) 2023-10-26T05:48:44.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 -- 172.21.15.71:0/2318393342 learned_addr learned my addr 172.21.15.71:0/2318393342 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:44.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff818894640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 0x7ff8140ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:44.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff8137fe640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81414e6e0 0x7ff814134160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:44.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 -- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 msgr2=0x7ff8140ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:44.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 0x7ff8140ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 -- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81414e6e0 msgr2=0x7ff814134160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:44.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81414e6e0 0x7ff814134160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 -- 172.21.15.71:0/2318393342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c00a930 con 0x7ff81414e300 2023-10-26T05:48:44.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff813fff640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 0x7ff814147b70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff80400c500 tx=0x7ff80400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:44.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804012830 con 0x7ff81414e300 2023-10-26T05:48:44.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff814159c70 con 0x7ff81414e300 2023-10-26T05:48:44.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff804012a80 con 0x7ff81414e300 2023-10-26T05:48:44.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804012e90 con 0x7ff81414e300 2023-10-26T05:48:44.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.649+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff814159f10 con 0x7ff81414e300 2023-10-26T05:48:44.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.653+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff814066b10 con 0x7ff81414e300 2023-10-26T05:48:44.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.653+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff80401a3b0 con 0x7ff81414e300 2023-10-26T05:48:44.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.653+0000 7ff810ff9640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8000664a0 0x7ff800068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:44.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.653+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8040840b0 con 0x7ff81414e300 2023-10-26T05:48:44.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.653+0000 7ff8137fe640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8000664a0 0x7ff800068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:44.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.653+0000 7ff8137fe640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8000664a0 0x7ff800068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff808007fd0 tx=0x7ff808009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:44.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.657+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff804051600 con 0x7ff81414e300 2023-10-26T05:48:44.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:48:44.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:48:44.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8140721e0 con 0x7ff81414e300 2023-10-26T05:48:44.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff810ff9640 1 -- 172.21.15.71:0/2318393342 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16485 (secure 0 0 0) 0x7ff804054c20 con 0x7ff81414e300 2023-10-26T05:48:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:44.818 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:42.435500+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:44.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8000664a0 msgr2=0x7ff800068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:44.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8000664a0 0x7ff800068960 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff808007fd0 tx=0x7ff808009040 comp rx=0 tx=0).stop 2023-10-26T05:48:44.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 msgr2=0x7ff814147b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:44.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 0x7ff814147b70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff80400c500 tx=0x7ff80400c9c0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 shutdown_connections 2023-10-26T05:48:44.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81415b700 0x7ff8140ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81414e6e0 0x7ff814134160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8000664a0 0x7ff800068960 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 --2- 172.21.15.71:0/2318393342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff81414e300 0x7ff814147b70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:44.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.813+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 >> 172.21.15.71:0/2318393342 conn(0x7ff814069770 msgr2=0x7ff814150d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:44.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.817+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 shutdown_connections 2023-10-26T05:48:44.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.817+0000 7ff81a2f7640 1 -- 172.21.15.71:0/2318393342 wait complete. 2023-10-26T05:48:44.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:48:21.848187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:48:23.028324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:44.837 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.993+0000 7f17d0b1f640 1 -- 172.21.15.71:0/1471675248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc0cbf30 msgr2=0x7f17cc069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.993+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/1471675248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc0cbf30 0x7f17cc069dc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f17cc06e920 tx=0x7f17c401cc60 comp rx=0 tx=0).stop 2023-10-26T05:48:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/1471675248 shutdown_connections 2023-10-26T05:48:45.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/1471675248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc0cbf30 0x7f17cc069dc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/1471675248 >> v2:172.21.15.86:3300/0 conn(0x7f17cc13c220 0x7f17cc15cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/1471675248 >> 172.21.15.71:0/1471675248 conn(0x7f17cc0aec60 msgr2=0x7f17cc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/1471675248 shutdown_connections 2023-10-26T05:48:45.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/1471675248 wait complete. 2023-10-26T05:48:45.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 Processor -- start 2023-10-26T05:48:45.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- start start 2023-10-26T05:48:45.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc0cbf30 0x7f17cc139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 0x7f17cc0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d220 0x7f17cc135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0ae8f0 con 0x7f17cc13c220 2023-10-26T05:48:45.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0acb90 con 0x7f17cc0cbf30 2023-10-26T05:48:45.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0ace70 con 0x7f17cc15d220 2023-10-26T05:48:45.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 0x7f17cc0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 0x7f17cc0ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50736/0 (socket says 172.21.15.71:50736) 2023-10-26T05:48:45.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 -- 172.21.15.71:0/2428906080 learned_addr learned my addr 172.21.15.71:0/2428906080 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:45.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17ca59c640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc0cbf30 0x7f17cc139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17cad9d640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d220 0x7f17cc135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 -- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d220 msgr2=0x7f17cc135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d220 0x7f17cc135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 -- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc0cbf30 msgr2=0x7f17cc139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc0cbf30 0x7f17cc139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 -- 172.21.15.71:0/2428906080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c40098e0 con 0x7f17cc13c220 2023-10-26T05:48:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17ca59c640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc0cbf30 0x7f17cc139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17c9d9b640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 0x7f17cc0ae3b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b3e0 tx=0x7f17c000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0011830 con 0x7f17cc13c220 2023-10-26T05:48:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f17c0011aa0 con 0x7f17cc13c220 2023-10-26T05:48:45.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc0ad150 con 0x7f17cc13c220 2023-10-26T05:48:45.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0010380 con 0x7f17cc13c220 2023-10-26T05:48:45.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc0ad710 con 0x7f17cc13c220 2023-10-26T05:48:45.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:44.997+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc066b10 con 0x7f17cc13c220 2023-10-26T05:48:45.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.001+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f17c00104e0 con 0x7f17cc13c220 2023-10-26T05:48:45.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.001+0000 7f17baffd640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17ac066470 0x7f17ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.001+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17c0084200 con 0x7f17cc13c220 2023-10-26T05:48:45.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.001+0000 7f17ca59c640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17ac066470 0x7f17ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.001+0000 7f17ca59c640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17ac066470 0x7f17ac068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0058d0 tx=0x7f17bc005eb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.005+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f17c0051710 con 0x7f17cc13c220 2023-10-26T05:48:45.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.157+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17cc06d8f0 con 0x7f17cc13c220 2023-10-26T05:48:45.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.157+0000 7f17baffd640 1 -- 172.21.15.71:0/2428906080 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17c0054d30 con 0x7f17cc13c220 2023-10-26T05:48:45.162 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:45.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17ac066470 msgr2=0x7f17ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17ac066470 0x7f17ac068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0058d0 tx=0x7f17bc005eb0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 msgr2=0x7f17cc0ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 0x7f17cc0ae3b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b3e0 tx=0x7f17c000b8a0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 shutdown_connections 2023-10-26T05:48:45.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17cc15d220 0x7f17cc135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17ac066470 0x7f17ac068930 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17cc13c220 0x7f17cc0ae3b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 --2- 172.21.15.71:0/2428906080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17cc0cbf30 0x7f17cc139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 >> 172.21.15.71:0/2428906080 conn(0x7f17cc0aec60 msgr2=0x7f17cc15cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 shutdown_connections 2023-10-26T05:48:45.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.161+0000 7f17d0b1f640 1 -- 172.21.15.71:0/2428906080 wait complete. 2023-10-26T05:48:45.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:45.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- 172.21.15.71:0/1846854066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c0cbd60 msgr2=0x7fce4c13e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 --2- 172.21.15.71:0/1846854066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c0cbd60 0x7fce4c13e8c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009900 tx=0x7fce3c01c850 comp rx=0 tx=0).stop 2023-10-26T05:48:45.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- 172.21.15.71:0/1846854066 shutdown_connections 2023-10-26T05:48:45.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 --2- 172.21.15.71:0/1846854066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c0cbd60 0x7fce4c13e8c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 --2- 172.21.15.71:0/1846854066 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce4c13c190 0x7fce4c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- 172.21.15.71:0/1846854066 >> 172.21.15.71:0/1846854066 conn(0x7fce4c0aec60 msgr2=0x7fce4c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- 172.21.15.71:0/1846854066 shutdown_connections 2023-10-26T05:48:45.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- 172.21.15.71:0/1846854066 wait complete. 2023-10-26T05:48:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 Processor -- start 2023-10-26T05:48:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- start start 2023-10-26T05:48:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce4c0cbd60 0x7fce4c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce4c13c190 0x7fce4c13f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 0x7fce4c15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c142090 con 0x7fce4c13ee00 2023-10-26T05:48:45.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0ad170 con 0x7fce4c0cbd60 2023-10-26T05:48:45.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce52b1a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c135bc0 con 0x7fce4c13c190 2023-10-26T05:48:45.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce510b7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 0x7fce4c15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce510b7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 0x7fce4c15bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50756/0 (socket says 172.21.15.71:50756) 2023-10-26T05:48:45.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce510b7640 1 -- 172.21.15.71:0/66752263 learned_addr learned my addr 172.21.15.71:0/66752263 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:45.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce510b7640 1 -- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce4c13c190 msgr2=0x7fce4c13f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.305+0000 7fce4bfff640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce4c13c190 0x7fce4c13f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce508b6640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce4c0cbd60 0x7fce4c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce510b7640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce4c13c190 0x7fce4c13f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce510b7640 1 -- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce4c0cbd60 msgr2=0x7fce4c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce510b7640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce4c0cbd60 0x7fce4c1522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce510b7640 1 -- 172.21.15.71:0/66752263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c009560 con 0x7fce4c13ee00 2023-10-26T05:48:45.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce510b7640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 0x7fce4c15bde0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce4000b410 tx=0x7fce4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011830 con 0x7fce4c13ee00 2023-10-26T05:48:45.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce40011ac0 con 0x7fce4c13ee00 2023-10-26T05:48:45.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40010380 con 0x7fce4c13ee00 2023-10-26T05:48:45.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c1333b0 con 0x7fce4c13ee00 2023-10-26T05:48:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c1422e0 con 0x7fce4c13ee00 2023-10-26T05:48:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c066b10 con 0x7fce4c13ee00 2023-10-26T05:48:45.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce400104e0 con 0x7fce4c13ee00 2023-10-26T05:48:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce497fa640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce38066470 0x7fce38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce40083ef0 con 0x7fce4c13ee00 2023-10-26T05:48:45.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce508b6640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce38066470 0x7fce38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.309+0000 7fce508b6640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce38066470 0x7fce38068930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce44005c50 tx=0x7fce44005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.313+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce40051370 con 0x7fce4c13ee00 2023-10-26T05:48:45.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.457+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce4c0721e0 con 0x7fce4c13ee00 2023-10-26T05:48:45.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.457+0000 7fce497fa640 1 -- 172.21.15.71:0/66752263 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce40054990 con 0x7fce4c13ee00 2023-10-26T05:48:45.462 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce38066470 msgr2=0x7fce38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce38066470 0x7fce38068930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce44005c50 tx=0x7fce44005be0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 msgr2=0x7fce4c15bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 0x7fce4c15bde0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce4000b410 tx=0x7fce4000b8d0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 shutdown_connections 2023-10-26T05:48:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce38066470 0x7fce38068930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce4c13ee00 0x7fce4c15bde0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce4c13c190 0x7fce4c13f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 --2- 172.21.15.71:0/66752263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce4c0cbd60 0x7fce4c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 >> 172.21.15.71:0/66752263 conn(0x7fce4c0aec60 msgr2=0x7fce4c13d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 shutdown_connections 2023-10-26T05:48:45.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.461+0000 7fce52b1a640 1 -- 172.21.15.71:0/66752263 wait complete. 2023-10-26T05:48:45.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:45.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/445021039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf800039b0 msgr2=0x7fcf80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/445021039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf800039b0 0x7fcf80003dd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009b60 tx=0x7fcf8c01d260 comp rx=0 tx=0).stop 2023-10-26T05:48:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 -- 172.21.15.71:0/445021039 reap_dead start 2023-10-26T05:48:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/445021039 shutdown_connections 2023-10-26T05:48:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/445021039 >> 172.21.15.71:0/445021039 conn(0x7fcf940cdd80 msgr2=0x7fcf940ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/445021039 shutdown_connections 2023-10-26T05:48:45.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/445021039 wait complete. 2023-10-26T05:48:45.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 Processor -- start 2023-10-26T05:48:45.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- start start 2023-10-26T05:48:45.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf800039b0 0x7fcf94156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf940c83d0 0x7fcf9414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 0x7fcf940b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9406a650 con 0x7fcf940b58c0 2023-10-26T05:48:45.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940b6280 con 0x7fcf940c83d0 2023-10-26T05:48:45.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940cb610 con 0x7fcf800039b0 2023-10-26T05:48:45.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 0x7fcf940b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 0x7fcf940b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50780/0 (socket says 172.21.15.71:50780) 2023-10-26T05:48:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 -- 172.21.15.71:0/2246166542 learned_addr learned my addr 172.21.15.71:0/2246166542 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf937fe640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf800039b0 0x7fcf94156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf92ffd640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf940c83d0 0x7fcf9414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 -- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf800039b0 msgr2=0x7fcf94156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf800039b0 0x7fcf94156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 -- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf940c83d0 msgr2=0x7fcf9414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf940c83d0 0x7fcf9414fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 -- 172.21.15.71:0/2246166542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf941502e0 con 0x7fcf940b58c0 2023-10-26T05:48:45.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf93fff640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 0x7fcf940b5d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcf940cb900 tx=0x7fcf8400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84005d50 con 0x7fcf940b58c0 2023-10-26T05:48:45.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf8400ced0 con 0x7fcf940b58c0 2023-10-26T05:48:45.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf940caa00 con 0x7fcf940b58c0 2023-10-26T05:48:45.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400a810 con 0x7fcf940b58c0 2023-10-26T05:48:45.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.593+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf940cafc0 con 0x7fcf940b58c0 2023-10-26T05:48:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.597+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf94066b10 con 0x7fcf940b58c0 2023-10-26T05:48:45.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.597+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf8400a970 con 0x7fcf940b58c0 2023-10-26T05:48:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.597+0000 7fcf8bfff640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf740664a0 0x7fcf74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.597+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf84082fc0 con 0x7fcf940b58c0 2023-10-26T05:48:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.597+0000 7fcf937fe640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf740664a0 0x7fcf74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.597+0000 7fcf937fe640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf740664a0 0x7fcf74068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0ae000 tx=0x7fcf8c01a000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.601+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf84050440 con 0x7fcf940b58c0 2023-10-26T05:48:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf940721e0 con 0x7fcf940b58c0 2023-10-26T05:48:45.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf8bfff640 1 -- 172.21.15.71:0/2246166542 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf840170a0 con 0x7fcf940b58c0 2023-10-26T05:48:45.746 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf740664a0 msgr2=0x7fcf74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf740664a0 0x7fcf74068960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0ae000 tx=0x7fcf8c01a000 comp rx=0 tx=0).stop 2023-10-26T05:48:45.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 msgr2=0x7fcf940b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 0x7fcf940b5d40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcf940cb900 tx=0x7fcf8400cb90 comp rx=0 tx=0).stop 2023-10-26T05:48:45.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 shutdown_connections 2023-10-26T05:48:45.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf740664a0 0x7fcf74068960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940b58c0 0x7fcf940b5d40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf940c83d0 0x7fcf9414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 --2- 172.21.15.71:0/2246166542 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf800039b0 0x7fcf94156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.741+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 >> 172.21.15.71:0/2246166542 conn(0x7fcf940cdd80 msgr2=0x7fcf940b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.745+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 shutdown_connections 2023-10-26T05:48:45.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.745+0000 7fcf9a09b640 1 -- 172.21.15.71:0/2246166542 wait complete. 2023-10-26T05:48:45.762 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:45.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d76a4640 1 -- 172.21.15.71:0/1434112242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95bc004d10 msgr2=0x7f95bc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d76a4640 1 --2- 172.21.15.71:0/1434112242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95bc004d10 0x7f95bc005190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009eb0 tx=0x7f95c801d5d0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d5c41640 1 -- 172.21.15.71:0/1434112242 reap_dead start 2023-10-26T05:48:45.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d76a4640 1 -- 172.21.15.71:0/1434112242 shutdown_connections 2023-10-26T05:48:45.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d76a4640 1 -- 172.21.15.71:0/1434112242 >> 172.21.15.71:0/1434112242 conn(0x7f95d006b8c0 msgr2=0x7f95d006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:45.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d76a4640 1 -- 172.21.15.71:0/1434112242 shutdown_connections 2023-10-26T05:48:45.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.869+0000 7f95d76a4640 1 -- 172.21.15.71:0/1434112242 wait complete. 2023-10-26T05:48:45.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 Processor -- start 2023-10-26T05:48:45.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- start start 2023-10-26T05:48:45.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95bc004d10 0x7f95d0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 0x7f95d01581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d01586f0 0x7f95d013c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d01f27a0 con 0x7f95d0069d80 2023-10-26T05:48:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0071360 con 0x7f95d01586f0 2023-10-26T05:48:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d0158b70 con 0x7f95bc004d10 2023-10-26T05:48:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 0x7f95d01581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 0x7f95d01581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50810/0 (socket says 172.21.15.71:50810) 2023-10-26T05:48:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 -- 172.21.15.71:0/791547278 learned_addr learned my addr 172.21.15.71:0/791547278 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d5440640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95bc004d10 0x7f95d0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d5c41640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d01586f0 0x7f95d013c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 -- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95bc004d10 msgr2=0x7f95d0070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95bc004d10 0x7f95d0070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 -- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d01586f0 msgr2=0x7f95d013c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:45.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d01586f0 0x7f95d013c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:45.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 -- 172.21.15.71:0/791547278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0071570 con 0x7f95d0069d80 2023-10-26T05:48:45.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d4c3f640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 0x7f95d01581b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f95c000f370 tx=0x7f95c000f830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c00084e0 con 0x7f95d0069d80 2023-10-26T05:48:45.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f95c0006720 con 0x7f95d0069d80 2023-10-26T05:48:45.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d01f1e00 con 0x7f95d0069d80 2023-10-26T05:48:45.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0006b30 con 0x7f95d0069d80 2023-10-26T05:48:45.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d01f23c0 con 0x7f95d0069d80 2023-10-26T05:48:45.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d01f2580 con 0x7f95d0069d80 2023-10-26T05:48:45.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f95c0006c90 con 0x7f95d0069d80 2023-10-26T05:48:45.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95c5ffb640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95b40664a0 0x7f95b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:45.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95c0086630 con 0x7f95d0069d80 2023-10-26T05:48:45.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.873+0000 7f95d5440640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95b40664a0 0x7f95b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:45.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.877+0000 7f95d5440640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95b40664a0 0x7f95b4068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95d00ca440 tx=0x7f95cc006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:45.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:45.877+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f95c00539a0 con 0x7f95d0069d80 2023-10-26T05:48:46.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.029+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95d0066b10 con 0x7f95d0069d80 2023-10-26T05:48:46.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95c5ffb640 1 -- 172.21.15.71:0/791547278 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95c001a080 con 0x7f95d0069d80 2023-10-26T05:48:46.036 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:46.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95b40664a0 msgr2=0x7f95b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95b40664a0 0x7f95b4068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f95d00ca440 tx=0x7f95cc006af0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 msgr2=0x7f95d01581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 0x7f95d01581b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f95c000f370 tx=0x7f95c000f830 comp rx=0 tx=0).stop 2023-10-26T05:48:46.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 shutdown_connections 2023-10-26T05:48:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d01586f0 0x7f95d013c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f95b40664a0 0x7f95b4068960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d0069d80 0x7f95d01581b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 --2- 172.21.15.71:0/791547278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f95bc004d10 0x7f95d0070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 >> 172.21.15.71:0/791547278 conn(0x7f95d006b8c0 msgr2=0x7f95d00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:46.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 shutdown_connections 2023-10-26T05:48:46.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.033+0000 7f95d76a4640 1 -- 172.21.15.71:0/791547278 wait complete. 2023-10-26T05:48:46.052 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:48:46.053 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:46.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:48:46.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:46.102 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[38648]: fuse finished with error 0 and tester_r 0 2023-10-26T05:48:46.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:46.118 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:48:46.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:46.119 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:48:46.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:48:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:46.142 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:46.142 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:46.155 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:46.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/2410980172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec06b720 msgr2=0x7f65ec157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 --2- 172.21.15.71:0/2410980172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec06b720 0x7f65ec157ee0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0cb1c0 tx=0x7f65dc01dad0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/2410980172 shutdown_connections 2023-10-26T05:48:46.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 --2- 172.21.15.71:0/2410980172 >> v2:172.21.15.86:3300/0 conn(0x7f65ec0ca270 0x7f65ec158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 --2- 172.21.15.71:0/2410980172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec06b720 0x7f65ec157ee0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/2410980172 >> 172.21.15.71:0/2410980172 conn(0x7f65ec070170 msgr2=0x7f65ec070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:46.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/2410980172 shutdown_connections 2023-10-26T05:48:46.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/2410980172 wait complete. 2023-10-26T05:48:46.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 Processor -- start 2023-10-26T05:48:46.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- start start 2023-10-26T05:48:46.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65ec06b720 0x7f65ec148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 0x7f65ec13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65ec0ca270 0x7f65ec0ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0adcc0 con 0x7f65ec0c9e90 2023-10-26T05:48:46.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0adf60 con 0x7f65ec06b720 2023-10-26T05:48:46.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec0ae240 con 0x7f65ec0ca270 2023-10-26T05:48:46.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 0x7f65ec13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 0x7f65ec13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50826/0 (socket says 172.21.15.71:50826) 2023-10-26T05:48:46.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 -- 172.21.15.71:0/721161077 learned_addr learned my addr 172.21.15.71:0/721161077 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:46.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 -- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65ec0ca270 msgr2=0x7f65ec0ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65ec0ca270 0x7f65ec0ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 -- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65ec06b720 msgr2=0x7f65ec148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65ec06b720 0x7f65ec148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 -- 172.21.15.71:0/721161077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc00a670 con 0x7f65ec0c9e90 2023-10-26T05:48:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f0907640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 0x7f65ec13a160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0b1040 tx=0x7f65dc0a6af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc005440 con 0x7f65ec0c9e90 2023-10-26T05:48:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65dc0046d0 con 0x7f65ec0c9e90 2023-10-26T05:48:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec0ae520 con 0x7f65ec0c9e90 2023-10-26T05:48:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc004ae0 con 0x7f65ec0c9e90 2023-10-26T05:48:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec0aeae0 con 0x7f65ec0c9e90 2023-10-26T05:48:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.289+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec0aecd0 con 0x7f65ec0c9e90 2023-10-26T05:48:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.293+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f65dc004c40 con 0x7f65ec0c9e90 2023-10-26T05:48:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.293+0000 7f65e1ffb640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65d0066470 0x7f65d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.293+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65dc11e9a0 con 0x7f65ec0c9e90 2023-10-26T05:48:46.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.293+0000 7f65f1108640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65d0066470 0x7f65d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.293+0000 7f65f1108640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65d0066470 0x7f65d0068930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f65e8007fd0 tx=0x7f65e80079a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:46.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.297+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f65dc0ebe30 con 0x7f65ec0c9e90 2023-10-26T05:48:46.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.453+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65ec066d00 con 0x7f65ec0c9e90 2023-10-26T05:48:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.457+0000 7f65e1ffb640 1 -- 172.21.15.71:0/721161077 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f65dc0ef450 con 0x7f65ec0c9e90 2023-10-26T05:48:46.462 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:46.462 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:42.435500+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:46.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.457+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65d0066470 msgr2=0x7f65d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.457+0000 7f65f336c640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65d0066470 0x7f65d0068930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f65e8007fd0 tx=0x7f65e80079a0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.457+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 msgr2=0x7f65ec13a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.457+0000 7f65f336c640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 0x7f65ec13a160 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0b1040 tx=0x7f65dc0a6af0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 shutdown_connections 2023-10-26T05:48:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65ec0ca270 0x7f65ec0ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65d0066470 0x7f65d0068930 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65ec0c9e90 0x7f65ec13a160 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 --2- 172.21.15.71:0/721161077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65ec06b720 0x7f65ec148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 >> 172.21.15.71:0/721161077 conn(0x7f65ec070170 msgr2=0x7f65ec157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 shutdown_connections 2023-10-26T05:48:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.461+0000 7f65f336c640 1 -- 172.21.15.71:0/721161077 wait complete. 2023-10-26T05:48:46.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:48:21.848187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:48:23.028324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:46.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:46.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/3526455839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c0b1520 msgr2=0x7f7d0c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- 172.21.15.71:0/3526455839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c0b1520 0x7f7d0c0b1900 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c072f30 tx=0x7f7cfc01c8a0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/3526455839 shutdown_connections 2023-10-26T05:48:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- 172.21.15.71:0/3526455839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d0c0b1040 0x7f7d0c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- 172.21.15.71:0/3526455839 >> v2:172.21.15.86:3300/0 conn(0x7f7d0c070e10 0x7f7d0c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- 172.21.15.71:0/3526455839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c0b1520 0x7f7d0c0b1900 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/3526455839 >> 172.21.15.71:0/3526455839 conn(0x7f7d0c0ca230 msgr2=0x7f7d0c0ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/3526455839 shutdown_connections 2023-10-26T05:48:46.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/3526455839 wait complete. 2023-10-26T05:48:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 Processor -- start 2023-10-26T05:48:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- start start 2023-10-26T05:48:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 0x7f7d0c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0b1040 0x7f7d0c14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d0c0b1520 0x7f7d0c14dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c14a620 con 0x7f7d0c070e10 2023-10-26T05:48:46.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c1487a0 con 0x7f7d0c0b1040 2023-10-26T05:48:46.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c14fee0 con 0x7f7d0c0b1520 2023-10-26T05:48:46.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 0x7f7d0c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 0x7f7d0c151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50838/0 (socket says 172.21.15.71:50838) 2023-10-26T05:48:46.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 -- 172.21.15.71:0/2295605152 learned_addr learned my addr 172.21.15.71:0/2295605152 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:46.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 -- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d0c0b1520 msgr2=0x7f7d0c14dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0affd640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0b1040 0x7f7d0c14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d0c0b1520 0x7f7d0c14dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 -- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0b1040 msgr2=0x7f7d0c14d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0b1040 0x7f7d0c14d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 -- 172.21.15.71:0/2295605152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009560 con 0x7f7d0c070e10 2023-10-26T05:48:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d0b7fe640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 0x7f7d0c151840 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b0040 tx=0x7f7cfc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc005230 con 0x7f7d0c070e10 2023-10-26T05:48:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7cfc005480 con 0x7f7d0c070e10 2023-10-26T05:48:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc004930 con 0x7f7d0c070e10 2023-10-26T05:48:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c14e770 con 0x7f7d0c070e10 2023-10-26T05:48:46.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.601+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c14e260 con 0x7f7d0c070e10 2023-10-26T05:48:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.605+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c14a760 con 0x7f7d0c070e10 2023-10-26T05:48:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.605+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7cfc0b4070 con 0x7f7d0c070e10 2023-10-26T05:48:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.605+0000 7f7cf3fff640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7cf80664a0 0x7f7cf8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.605+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cfc11e170 con 0x7f7d0c070e10 2023-10-26T05:48:46.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.605+0000 7f7d0affd640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7cf80664a0 0x7f7cf8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.605+0000 7f7d0affd640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7cf80664a0 0x7f7cf8068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c158900 tx=0x7f7d00005c70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:46.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.609+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7cfc0eb680 con 0x7f7d0c070e10 2023-10-26T05:48:46.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.761+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d0c066d00 con 0x7f7d0c070e10 2023-10-26T05:48:46.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.761+0000 7f7cf3fff640 1 -- 172.21.15.71:0/2295605152 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cfc0eeca0 con 0x7f7d0c070e10 2023-10-26T05:48:46.767 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:46.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7cf80664a0 msgr2=0x7f7cf8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7cf80664a0 0x7f7cf8068960 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c158900 tx=0x7f7d00005c70 comp rx=0 tx=0).stop 2023-10-26T05:48:46.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 msgr2=0x7f7d0c151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 0x7f7d0c151840 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0b0040 tx=0x7f7cfc004600 comp rx=0 tx=0).stop 2023-10-26T05:48:46.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 shutdown_connections 2023-10-26T05:48:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d0c0b1520 0x7f7d0c14dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0b1040 0x7f7d0c14d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7cf80664a0 0x7f7cf8068960 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 --2- 172.21.15.71:0/2295605152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d0c070e10 0x7f7d0c151840 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 >> 172.21.15.71:0/2295605152 conn(0x7f7d0c0ca230 msgr2=0x7f7d0c14c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:46.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 shutdown_connections 2023-10-26T05:48:46.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.765+0000 7f7d11acf640 1 -- 172.21.15.71:0/2295605152 wait complete. 2023-10-26T05:48:46.786 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:46.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/719398648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec0aa8e0 msgr2=0x7fe0ec0ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 --2- 172.21.15.71:0/719398648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec0ac750 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009900 tx=0x7fe0e001c7c0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/719398648 shutdown_connections 2023-10-26T05:48:46.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 --2- 172.21.15.71:0/719398648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec0ac750 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 --2- 172.21.15.71:0/719398648 >> v2:172.21.15.86:3300/0 conn(0x7fe0ec159090 0x7fe0ec15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/719398648 >> 172.21.15.71:0/719398648 conn(0x7fe0ec070d10 msgr2=0x7fe0ec06a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:46.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/719398648 shutdown_connections 2023-10-26T05:48:46.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/719398648 wait complete. 2023-10-26T05:48:46.924 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 Processor -- start 2023-10-26T05:48:46.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- start start 2023-10-26T05:48:46.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0ec0acc90 0x7fe0ec15af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 0x7fe0ec152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15a7c0 con 0x7fe0ec159090 2023-10-26T05:48:46.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec155040 con 0x7fe0ec0aa8e0 2023-10-26T05:48:46.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15b490 con 0x7fe0ec0acc90 2023-10-26T05:48:46.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 0x7fe0ec152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 0x7fe0ec152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50856/0 (socket says 172.21.15.71:50856) 2023-10-26T05:48:46.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 -- 172.21.15.71:0/3024222618 learned_addr learned my addr 172.21.15.71:0/3024222618 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:46.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eaffd640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0ea7fc640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0ec0acc90 0x7fe0ec15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 -- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0ec0acc90 msgr2=0x7fe0ec15af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0ec0acc90 0x7fe0ec15af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 -- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec0aa8e0 msgr2=0x7fe0ec1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:46.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:46.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 -- 172.21.15.71:0/3024222618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0009560 con 0x7fe0ec159090 2023-10-26T05:48:46.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eaffd640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:46.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0eb7fe640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 0x7fe0ec152810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400c7c0 tx=0x7fe0e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:46.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4003db0 con 0x7fe0ec159090 2023-10-26T05:48:46.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0e4005070 con 0x7fe0ec159090 2023-10-26T05:48:46.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec159c40 con 0x7fe0ec159090 2023-10-26T05:48:46.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4005500 con 0x7fe0ec159090 2023-10-26T05:48:46.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.917+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec15a200 con 0x7fe0ec159090 2023-10-26T05:48:46.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.921+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec066b10 con 0x7fe0ec159090 2023-10-26T05:48:46.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.921+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe0e4005660 con 0x7fe0ec159090 2023-10-26T05:48:46.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.921+0000 7fe0d37fe640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0d80664a0 0x7fe0d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:46.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.921+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0e4083830 con 0x7fe0ec159090 2023-10-26T05:48:46.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.921+0000 7fe0eaffd640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0d80664a0 0x7fe0d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:46.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.921+0000 7fe0eaffd640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0d80664a0 0x7fe0d8068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005fd0 tx=0x7fe0dc005dc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:46.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:46.925+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe0e4050cb0 con 0x7fe0ec159090 2023-10-26T05:48:47.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.073+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0ec0ad0e0 con 0x7fe0ec159090 2023-10-26T05:48:47.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.073+0000 7fe0d37fe640 1 -- 172.21.15.71:0/3024222618 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0ec0ad0e0 con 0x7fe0ec159090 2023-10-26T05:48:47.079 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:47.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0d80664a0 msgr2=0x7fe0d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0d80664a0 0x7fe0d8068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005fd0 tx=0x7fe0dc005dc0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 msgr2=0x7fe0ec152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 0x7fe0ec152810 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe0e400c7c0 tx=0x7fe0e400cc80 comp rx=0 tx=0).stop 2023-10-26T05:48:47.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 shutdown_connections 2023-10-26T05:48:47.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0d80664a0 0x7fe0d8068960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0ec159090 0x7fe0ec152810 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0ec0acc90 0x7fe0ec15af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 --2- 172.21.15.71:0/3024222618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0ec0aa8e0 0x7fe0ec1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 >> 172.21.15.71:0/3024222618 conn(0x7fe0ec070d10 msgr2=0x7fe0ec15d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:47.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 shutdown_connections 2023-10-26T05:48:47.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.077+0000 7fe0f146f640 1 -- 172.21.15.71:0/3024222618 wait complete. 2023-10-26T05:48:47.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:47.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 -- 172.21.15.71:0/1707366480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 msgr2=0x7f157c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 --2- 172.21.15.71:0/1707366480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c15b3a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1570009900 tx=0x7f157001c730 comp rx=0 tx=0).stop 2023-10-26T05:48:47.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 -- 172.21.15.71:0/1707366480 shutdown_connections 2023-10-26T05:48:47.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 --2- 172.21.15.71:0/1707366480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c15b3a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 --2- 172.21.15.71:0/1707366480 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0d0470 0x7f157c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 -- 172.21.15.71:0/1707366480 >> 172.21.15.71:0/1707366480 conn(0x7f157c0cdd80 msgr2=0x7f157c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:47.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 -- 172.21.15.71:0/1707366480 shutdown_connections 2023-10-26T05:48:47.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.205+0000 7f1582358640 1 -- 172.21.15.71:0/1707366480 wait complete. 2023-10-26T05:48:47.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 Processor -- start 2023-10-26T05:48:47.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- start start 2023-10-26T05:48:47.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0cd880 0x7f157c14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c0d0470 0x7f157c153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c14fed0 con 0x7f157c0d0850 2023-10-26T05:48:47.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1513e0 con 0x7f157c0d0470 2023-10-26T05:48:47.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c150e20 con 0x7f157c0cd880 2023-10-26T05:48:47.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50874/0 (socket says 172.21.15.71:50874) 2023-10-26T05:48:47.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 -- 172.21.15.71:0/912577161 learned_addr learned my addr 172.21.15.71:0/912577161 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:47.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f157bfff640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0cd880 0x7f157c14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f157b7fe640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c0d0470 0x7f157c153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 -- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0cd880 msgr2=0x7f157c14f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0cd880 0x7f157c14f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 -- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c0d0470 msgr2=0x7f157c153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c0d0470 0x7f157c153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 -- 172.21.15.71:0/912577161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009560 con 0x7f157c0d0850 2023-10-26T05:48:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f157bfff640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0cd880 0x7f157c14f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f15808f5640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c153770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f157400c780 tx=0x7f157400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574003d90 con 0x7f157c0d0850 2023-10-26T05:48:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f157400ce90 con 0x7f157c0d0850 2023-10-26T05:48:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15740052d0 con 0x7f157c0d0850 2023-10-26T05:48:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c151100 con 0x7f157c0d0850 2023-10-26T05:48:47.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c155b70 con 0x7f157c0d0850 2023-10-26T05:48:47.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.209+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c066b10 con 0x7f157c0d0850 2023-10-26T05:48:47.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.213+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1574005430 con 0x7f157c0d0850 2023-10-26T05:48:47.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.213+0000 7f1578ff9640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1568066470 0x7f1568068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.213+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1574083510 con 0x7f157c0d0850 2023-10-26T05:48:47.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.213+0000 7f157bfff640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1568066470 0x7f1568068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.213+0000 7f157bfff640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1568066470 0x7f1568068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f157c0b6b50 tx=0x7f156c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:47.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.217+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1574050a20 con 0x7f157c0d0850 2023-10-26T05:48:47.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.357+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f157c155d90 con 0x7f157c0d0850 2023-10-26T05:48:47.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.357+0000 7f1578ff9640 1 -- 172.21.15.71:0/912577161 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1574081e60 con 0x7f157c0d0850 2023-10-26T05:48:47.362 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:47.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1568066470 msgr2=0x7f1568068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1568066470 0x7f1568068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f157c0b6b50 tx=0x7f156c008040 comp rx=0 tx=0).stop 2023-10-26T05:48:47.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 msgr2=0x7f157c153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c153770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f157400c780 tx=0x7f157400cc40 comp rx=0 tx=0).stop 2023-10-26T05:48:47.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 shutdown_connections 2023-10-26T05:48:47.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1568066470 0x7f1568068930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f157c0d0850 0x7f157c153770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f157c0d0470 0x7f157c153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 --2- 172.21.15.71:0/912577161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f157c0cd880 0x7f157c14f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 >> 172.21.15.71:0/912577161 conn(0x7f157c0cdd80 msgr2=0x7f157c15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 shutdown_connections 2023-10-26T05:48:47.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.361+0000 7f1582358640 1 -- 172.21.15.71:0/912577161 wait complete. 2023-10-26T05:48:47.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:48:47.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.489+0000 7f9205843640 1 -- 172.21.15.71:0/3923812541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f920015b410 msgr2=0x7f920015ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.489+0000 7f9205843640 1 --2- 172.21.15.71:0/3923812541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f920015b410 0x7f920015ef00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009840 tx=0x7f91f001c790 comp rx=0 tx=0).stop 2023-10-26T05:48:47.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.489+0000 7f9205843640 1 -- 172.21.15.71:0/3923812541 shutdown_connections 2023-10-26T05:48:47.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.489+0000 7f9205843640 1 --2- 172.21.15.71:0/3923812541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f920015b410 0x7f920015ef00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.489+0000 7f9205843640 1 -- 172.21.15.71:0/3923812541 >> 172.21.15.71:0/3923812541 conn(0x7f92000b3b90 msgr2=0x7f92000d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:47.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.489+0000 7f9205843640 1 -- 172.21.15.71:0/3923812541 shutdown_connections 2023-10-26T05:48:47.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- 172.21.15.71:0/3923812541 wait complete. 2023-10-26T05:48:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 Processor -- start 2023-10-26T05:48:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- start start 2023-10-26T05:48:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 0x7f9200150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92000cf2e0 0x7f92000cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920015b410 0x7f920014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000cab40 con 0x7f92000b0440 2023-10-26T05:48:47.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000cae20 con 0x7f920015b410 2023-10-26T05:48:47.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92000cb100 con 0x7f92000cf2e0 2023-10-26T05:48:47.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 0x7f9200150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 0x7f9200150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50904/0 (socket says 172.21.15.71:50904) 2023-10-26T05:48:47.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 -- 172.21.15.71:0/2069226090 learned_addr learned my addr 172.21.15.71:0/2069226090 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:47.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91fe7fc640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92000cf2e0 0x7f92000cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91ff7fe640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920015b410 0x7f920014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 -- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92000cf2e0 msgr2=0x7f92000cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92000cf2e0 0x7f92000cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 -- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920015b410 msgr2=0x7f920014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920015b410 0x7f920014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 -- 172.21.15.71:0/2069226090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00094a0 con 0x7f92000b0440 2023-10-26T05:48:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91ff7fe640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920015b410 0x7f920014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91fe7fc640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92000cf2e0 0x7f92000cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91feffd640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 0x7f9200150a70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f91f001cc70 tx=0x7f91f00a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f00bc070 con 0x7f92000b0440 2023-10-26T05:48:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f91f0005230 con 0x7f92000b0440 2023-10-26T05:48:47.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0005600 con 0x7f92000b0440 2023-10-26T05:48:47.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92000cb3e0 con 0x7f92000b0440 2023-10-26T05:48:47.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92000ccf80 con 0x7f92000b0440 2023-10-26T05:48:47.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.493+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9200066b10 con 0x7f92000b0440 2023-10-26T05:48:47.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.497+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f91f00046d0 con 0x7f92000b0440 2023-10-26T05:48:47.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.497+0000 7f91e7fff640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f91ec066470 0x7f91ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.497+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91f011e4c0 con 0x7f92000b0440 2023-10-26T05:48:47.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.497+0000 7f91fe7fc640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f91ec066470 0x7f91ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.497+0000 7f91fe7fc640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f91ec066470 0x7f91ec068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f92000b4020 tx=0x7f91f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:47.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.501+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f91f00eb8a0 con 0x7f92000b0440 2023-10-26T05:48:47.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.653+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f920006c2c0 con 0x7f92000b0440 2023-10-26T05:48:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.653+0000 7f91e7fff640 1 -- 172.21.15.71:0/2069226090 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f91f00b2080 con 0x7f92000b0440 2023-10-26T05:48:47.659 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:48:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f91ec066470 msgr2=0x7f91ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f91ec066470 0x7f91ec068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f92000b4020 tx=0x7f91f4009040 comp rx=0 tx=0).stop 2023-10-26T05:48:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 msgr2=0x7f9200150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 0x7f9200150a70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f91f001cc70 tx=0x7f91f00a5860 comp rx=0 tx=0).stop 2023-10-26T05:48:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 shutdown_connections 2023-10-26T05:48:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f920015b410 0x7f920014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92000cf2e0 0x7f92000cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f91ec066470 0x7f91ec068930 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 --2- 172.21.15.71:0/2069226090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92000b0440 0x7f9200150a70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 >> 172.21.15.71:0/2069226090 conn(0x7f92000b3b90 msgr2=0x7f9200157280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:47.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 shutdown_connections 2023-10-26T05:48:47.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.657+0000 7f9205843640 1 -- 172.21.15.71:0/2069226090 wait complete. 2023-10-26T05:48:47.676 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:48:47.676 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:48:47.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:48:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:48:47.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34055]: fuse finished with error 0 and tester_r 0 2023-10-26T05:48:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:48:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:47.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:48:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:48:47.741 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:48:47.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:47.742 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:48:47.750 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:47.750 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:47.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:48:47.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:48:47.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:47.758 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:48:47.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:48:47.805 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:48:47.806 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:47.806 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:47.811 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:48:47.812 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:48:47.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:48:47.813 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:48:47.858 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:48:47.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:48:47.860 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:47.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- 172.21.15.71:0/1488586483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480b2a60 msgr2=0x7f10480caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 --2- 172.21.15.71:0/1488586483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480b2a60 0x7f10480caad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1044009840 tx=0x7f104401c800 comp rx=0 tx=0).stop 2023-10-26T05:48:47.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- 172.21.15.71:0/1488586483 shutdown_connections 2023-10-26T05:48:47.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 --2- 172.21.15.71:0/1488586483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480cb010 0x7f1048159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 --2- 172.21.15.71:0/1488586483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480b2a60 0x7f10480caad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- 172.21.15.71:0/1488586483 >> 172.21.15.71:0/1488586483 conn(0x7f1048069770 msgr2=0x7f1048069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:47.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- 172.21.15.71:0/1488586483 shutdown_connections 2023-10-26T05:48:47.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- 172.21.15.71:0/1488586483 wait complete. 2023-10-26T05:48:47.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 Processor -- start 2023-10-26T05:48:47.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- start start 2023-10-26T05:48:47.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480b2a60 0x7f10481378a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480cb010 0x7f10481548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 0x7f1048154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048156c20 con 0x7f104813c2f0 2023-10-26T05:48:47.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048153070 con 0x7f10480cb010 2023-10-26T05:48:47.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f1050105640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1048137de0 con 0x7f10480b2a60 2023-10-26T05:48:47.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f104e6a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 0x7f1048154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f104e6a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 0x7f1048154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50912/0 (socket says 172.21.15.71:50912) 2023-10-26T05:48:47.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f104e6a2640 1 -- 172.21.15.71:0/1649925692 learned_addr learned my addr 172.21.15.71:0/1649925692 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:47.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.981+0000 7f104dea1640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480b2a60 0x7f10481378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104d6a0640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480cb010 0x7f10481548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104e6a2640 1 -- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480b2a60 msgr2=0x7f10481378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104e6a2640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480b2a60 0x7f10481378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104e6a2640 1 -- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480cb010 msgr2=0x7f10481548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:47.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104e6a2640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480cb010 0x7f10481548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:47.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104e6a2640 1 -- 172.21.15.71:0/1649925692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040009820 con 0x7f104813c2f0 2023-10-26T05:48:47.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104d6a0640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480cb010 0x7f10481548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:47.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104dea1640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480b2a60 0x7f10481378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:47.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104e6a2640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 0x7f1048154e20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1040009d30 tx=0x7f104000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:47.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040009b80 con 0x7f104813c2f0 2023-10-26T05:48:47.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1040003b80 con 0x7f104813c2f0 2023-10-26T05:48:47.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10440094a0 con 0x7f104813c2f0 2023-10-26T05:48:47.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040004020 con 0x7f104813c2f0 2023-10-26T05:48:47.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1048152740 con 0x7f104813c2f0 2023-10-26T05:48:47.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048066b10 con 0x7f104813c2f0 2023-10-26T05:48:47.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1040004180 con 0x7f104813c2f0 2023-10-26T05:48:47.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f103e7fc640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f102c066470 0x7f102c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:47.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1040014070 con 0x7f104813c2f0 2023-10-26T05:48:47.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.985+0000 7f104dea1640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f102c066470 0x7f102c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:47.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.989+0000 7f104dea1640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f102c066470 0x7f102c068930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1038001670 tx=0x7f1038009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:47.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:47.989+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10400521f0 con 0x7f104813c2f0 2023-10-26T05:48:48.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.161+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1048159b40 con 0x7f104813c2f0 2023-10-26T05:48:48.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f103e7fc640 1 -- 172.21.15.71:0/1649925692 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f1040055810 con 0x7f104813c2f0 2023-10-26T05:48:48.168 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:48.168 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:48.169 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:48.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f102c066470 msgr2=0x7f102c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f102c066470 0x7f102c068930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1038001670 tx=0x7f1038009040 comp rx=0 tx=0).stop 2023-10-26T05:48:48.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 msgr2=0x7f1048154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 0x7f1048154e20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f1040009d30 tx=0x7f104000c6a0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 shutdown_connections 2023-10-26T05:48:48.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f102c066470 0x7f102c068930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104813c2f0 0x7f1048154e20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10480cb010 0x7f10481548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 --2- 172.21.15.71:0/1649925692 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10480b2a60 0x7f10481378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 >> 172.21.15.71:0/1649925692 conn(0x7f1048069770 msgr2=0x7f10480abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:48.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 shutdown_connections 2023-10-26T05:48:48.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.165+0000 7f1050105640 1 -- 172.21.15.71:0/1649925692 wait complete. 2023-10-26T05:48:48.185 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 -- 172.21.15.71:0/532246691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0415bc50 msgr2=0x7ffa0415c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/532246691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0415bc50 0x7ffa0415c030 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ffa040719c0 tx=0x7ff9f401c8a0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 -- 172.21.15.71:0/532246691 shutdown_connections 2023-10-26T05:48:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/532246691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa040b1f10 0x7ffa041416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/532246691 >> v2:172.21.15.86:3300/0 conn(0x7ffa040b1a60 0x7ffa04141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/532246691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa0415bc50 0x7ffa0415c030 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 -- 172.21.15.71:0/532246691 >> 172.21.15.71:0/532246691 conn(0x7ffa040caa90 msgr2=0x7ffa040caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 -- 172.21.15.71:0/532246691 shutdown_connections 2023-10-26T05:48:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.297+0000 7ffa0acbe640 1 -- 172.21.15.71:0/532246691 wait complete. 2023-10-26T05:48:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 Processor -- start 2023-10-26T05:48:48.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- start start 2023-10-26T05:48:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa040b1a60 0x7ffa04154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 0x7ffa04158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0415bc50 0x7ffa04159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa04155060 con 0x7ffa040b1f10 2023-10-26T05:48:48.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0413cdc0 con 0x7ffa0415bc50 2023-10-26T05:48:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0413d0a0 con 0x7ffa040b1a60 2023-10-26T05:48:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 0x7ffa04158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 0x7ffa04158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50936/0 (socket says 172.21.15.71:50936) 2023-10-26T05:48:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 -- 172.21.15.71:0/1110421772 learned_addr learned my addr 172.21.15.71:0/1110421772 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:48.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0925b640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0415bc50 0x7ffa04159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa08a5a640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa040b1a60 0x7ffa04154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 -- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa040b1a60 msgr2=0x7ffa04154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa040b1a60 0x7ffa04154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 -- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0415bc50 msgr2=0x7ffa04159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0415bc50 0x7ffa04159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 -- 172.21.15.71:0/1110421772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f4009560 con 0x7ffa040b1f10 2023-10-26T05:48:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa03fff640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 0x7ffa04158ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c940 tx=0x7ff9f800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8004220 con 0x7ffa040b1f10 2023-10-26T05:48:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff9f8004470 con 0x7ffa040b1f10 2023-10-26T05:48:48.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0413d380 con 0x7ffa040b1f10 2023-10-26T05:48:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8007bf0 con 0x7ffa040b1f10 2023-10-26T05:48:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0413c090 con 0x7ffa040b1f10 2023-10-26T05:48:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.301+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa04066d00 con 0x7ffa040b1f10 2023-10-26T05:48:48.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.305+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff9f8007d50 con 0x7ffa040b1f10 2023-10-26T05:48:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.305+0000 7ffa017fa640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9f00664a0 0x7ff9f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.305+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9f8083f90 con 0x7ffa040b1f10 2023-10-26T05:48:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.305+0000 7ffa08a5a640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9f00664a0 0x7ff9f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.305+0000 7ffa08a5a640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9f00664a0 0x7ff9f0068960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40af000 tx=0x7ff9f401b0e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.309+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff9f80b5090 con 0x7ffa040b1f10 2023-10-26T05:48:48.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.465+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa040ca1d0 con 0x7ffa040b1f10 2023-10-26T05:48:48.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.465+0000 7ffa017fa640 1 -- 172.21.15.71:0/1110421772 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16485 (secure 0 0 0) 0x7ff9f801d090 con 0x7ffa040b1f10 2023-10-26T05:48:48.470 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:48.471 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:42.435500+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:48.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9f00664a0 msgr2=0x7ff9f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9f00664a0 0x7ff9f0068960 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff9f40af000 tx=0x7ff9f401b0e0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 msgr2=0x7ffa04158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 0x7ffa04158ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c940 tx=0x7ff9f800ce00 comp rx=0 tx=0).stop 2023-10-26T05:48:48.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 shutdown_connections 2023-10-26T05:48:48.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa0415bc50 0x7ffa04159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9f00664a0 0x7ff9f0068960 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa040b1f10 0x7ffa04158ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 --2- 172.21.15.71:0/1110421772 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa040b1a60 0x7ffa04154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 >> 172.21.15.71:0/1110421772 conn(0x7ffa040caa90 msgr2=0x7ffa04157030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:48.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 shutdown_connections 2023-10-26T05:48:48.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.469+0000 7ffa0acbe640 1 -- 172.21.15.71:0/1110421772 wait complete. 2023-10-26T05:48:48.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:48:21.848187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:48:23.028324+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:48.490 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:48.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bdcd15640 1 -- 172.21.15.71:0/13453951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bc0004d10 msgr2=0x7f3bc0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/13453951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bc0004d10 0x7f3bc0005190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0001da0 tx=0x7f3bd001caf0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bd6d9d640 1 -- 172.21.15.71:0/13453951 reap_dead start 2023-10-26T05:48:48.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bdcd15640 1 -- 172.21.15.71:0/13453951 shutdown_connections 2023-10-26T05:48:48.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bdcd15640 1 -- 172.21.15.71:0/13453951 >> 172.21.15.71:0/13453951 conn(0x7f3bd80b0c10 msgr2=0x7f3bd80b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:48.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bdcd15640 1 -- 172.21.15.71:0/13453951 shutdown_connections 2023-10-26T05:48:48.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.613+0000 7f3bdcd15640 1 -- 172.21.15.71:0/13453951 wait complete. 2023-10-26T05:48:48.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 Processor -- start 2023-10-26T05:48:48.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- start start 2023-10-26T05:48:48.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bd8152720 0x7f3bd80c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd80b26c0 0x7f3bd81f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 0x7f3bd8164220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8163180 con 0x7f3bd81f2d80 2023-10-26T05:48:48.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8163460 con 0x7f3bd80b26c0 2023-10-26T05:48:48.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8163740 con 0x7f3bd8152720 2023-10-26T05:48:48.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 0x7f3bd8164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 0x7f3bd8164220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50944/0 (socket says 172.21.15.71:50944) 2023-10-26T05:48:48.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 -- 172.21.15.71:0/385909889 learned_addr learned my addr 172.21.15.71:0/385909889 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:48.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd659c640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bd8152720 0x7f3bd80c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd5d9b640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd80b26c0 0x7f3bd81f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 -- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bd8152720 msgr2=0x7f3bd80c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bd8152720 0x7f3bd80c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 -- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd80b26c0 msgr2=0x7f3bd81f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd80b26c0 0x7f3bd81f2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 -- 172.21.15.71:0/385909889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd8163a20 con 0x7f3bd81f2d80 2023-10-26T05:48:48.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bd6d9d640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 0x7f3bd8164220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00aa040 tx=0x7f3bd00a7d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:48.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00a7e80 con 0x7f3bd81f2d80 2023-10-26T05:48:48.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3bd00012e0 con 0x7f3bd81f2d80 2023-10-26T05:48:48.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd8163d00 con 0x7f3bd81f2d80 2023-10-26T05:48:48.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00a7070 con 0x7f3bd81f2d80 2023-10-26T05:48:48.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd8152b00 con 0x7f3bd81f2d80 2023-10-26T05:48:48.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd8066b10 con 0x7f3bd81f2d80 2023-10-26T05:48:48.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3bd00b5070 con 0x7f3bd81f2d80 2023-10-26T05:48:48.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bc6ffd640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bb8066470 0x7f3bb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.617+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bd00aa070 con 0x7f3bd81f2d80 2023-10-26T05:48:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.621+0000 7f3bd659c640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bb8066470 0x7f3bb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.621+0000 7f3bd659c640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bb8066470 0x7f3bb8068930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80010f0 tx=0x7f3bc8000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.621+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3bd00eb070 con 0x7f3bd81f2d80 2023-10-26T05:48:48.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3bd80721e0 con 0x7f3bd81f2d80 2023-10-26T05:48:48.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bc6ffd640 1 -- 172.21.15.71:0/385909889 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bd00ee690 con 0x7f3bd81f2d80 2023-10-26T05:48:48.785 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:48.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bb8066470 msgr2=0x7f3bb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bb8066470 0x7f3bb8068930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80010f0 tx=0x7f3bc8000ee0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 msgr2=0x7f3bd8164220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 0x7f3bd8164220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00aa040 tx=0x7f3bd00a7d90 comp rx=0 tx=0).stop 2023-10-26T05:48:48.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 shutdown_connections 2023-10-26T05:48:48.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bb8066470 0x7f3bb8068930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bd81f2d80 0x7f3bd8164220 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bd80b26c0 0x7f3bd81f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 --2- 172.21.15.71:0/385909889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bd8152720 0x7f3bd80c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 >> 172.21.15.71:0/385909889 conn(0x7f3bd80b0c10 msgr2=0x7f3bd806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:48.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 shutdown_connections 2023-10-26T05:48:48.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.781+0000 7f3bdcd15640 1 -- 172.21.15.71:0/385909889 wait complete. 2023-10-26T05:48:48.804 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:48:48.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3768961462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b815b5c0 msgr2=0x7fa3b80b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3768961462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b815b5c0 0x7fa3b80b02a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8009bc0 tx=0x7fa3a801cac0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3768961462 shutdown_connections 2023-10-26T05:48:48.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3768961462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b815b5c0 0x7fa3b80b02a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3768961462 >> 172.21.15.71:0/3768961462 conn(0x7fa3b80d12d0 msgr2=0x7fa3b80d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:48.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3768961462 shutdown_connections 2023-10-26T05:48:48.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.929+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3768961462 wait complete. 2023-10-26T05:48:48.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 Processor -- start 2023-10-26T05:48:48.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- start start 2023-10-26T05:48:48.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b80b6910 0x7fa3b80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 0x7fa3b80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3b815b5c0 0x7fa3b80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b815d9a0 con 0x7fa3b80b6d80 2023-10-26T05:48:48.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80c83a0 con 0x7fa3b80b6910 2023-10-26T05:48:48.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80c8680 con 0x7fa3b815b5c0 2023-10-26T05:48:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 0x7fa3b80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 0x7fa3b80c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50970/0 (socket says 172.21.15.71:50970) 2023-10-26T05:48:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 -- 172.21.15.71:0/3319872775 learned_addr learned my addr 172.21.15.71:0/3319872775 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b659c640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b80b6910 0x7fa3b80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b6d9d640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3b815b5c0 0x7fa3b80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 -- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3b815b5c0 msgr2=0x7fa3b80b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3b815b5c0 0x7fa3b80b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 -- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b80b6910 msgr2=0x7fa3b80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b80b6910 0x7fa3b80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 -- 172.21.15.71:0/3319872775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8009820 con 0x7fa3b80b6d80 2023-10-26T05:48:48.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b6d9d640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3b815b5c0 0x7fa3b80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:48.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b659c640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b80b6910 0x7fa3b80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:48.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3b5d9b640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 0x7fa3b80c6df0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c750 tx=0x7fa3a000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:48.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0005270 con 0x7fa3b80b6d80 2023-10-26T05:48:48.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa3a00053f0 con 0x7fa3b80b6d80 2023-10-26T05:48:48.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a000a880 con 0x7fa3b80b6d80 2023-10-26T05:48:48.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80cdb70 con 0x7fa3b80b6d80 2023-10-26T05:48:48.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b80c8c40 con 0x7fa3b80b6d80 2023-10-26T05:48:48.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.933+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b8066b10 con 0x7fa3b80b6d80 2023-10-26T05:48:48.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.937+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa3a000a9e0 con 0x7fa3b80b6d80 2023-10-26T05:48:48.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.937+0000 7fa3aeffd640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa398066470 0x7fa398068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:48.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.937+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3a0083560 con 0x7fa3b80b6d80 2023-10-26T05:48:48.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.937+0000 7fa3b659c640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa398066470 0x7fa398068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:48.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.937+0000 7fa3b659c640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa398066470 0x7fa398068930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80af000 tx=0x7fa3a801b300 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:48.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:48.941+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa3a00509e0 con 0x7fa3b80b6d80 2023-10-26T05:48:49.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3b815b9a0 con 0x7fa3b80b6d80 2023-10-26T05:48:49.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3aeffd640 1 -- 172.21.15.71:0/3319872775 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3a0017020 con 0x7fa3b80b6d80 2023-10-26T05:48:49.089 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:48:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa398066470 msgr2=0x7fa398068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa398066470 0x7fa398068930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80af000 tx=0x7fa3a801b300 comp rx=0 tx=0).stop 2023-10-26T05:48:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 msgr2=0x7fa3b80c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 0x7fa3b80c6df0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000c750 tx=0x7fa3a000cc10 comp rx=0 tx=0).stop 2023-10-26T05:48:49.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 shutdown_connections 2023-10-26T05:48:49.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa3b815b5c0 0x7fa3b80b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa398066470 0x7fa398068930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3b80b6d80 0x7fa3b80c6df0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 --2- 172.21.15.71:0/3319872775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3b80b6910 0x7fa3b80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 >> 172.21.15.71:0/3319872775 conn(0x7fa3b80d12d0 msgr2=0x7fa3b80b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 shutdown_connections 2023-10-26T05:48:49.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.085+0000 7fa3bcf11640 1 -- 172.21.15.71:0/3319872775 wait complete. 2023-10-26T05:48:49.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:48:49.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 -- 172.21.15.71:0/3148507553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 msgr2=0x7f77740b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 --2- 172.21.15.71:0/3148507553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f77740b1e50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7764009c80 tx=0x7f776401cd10 comp rx=0 tx=0).stop 2023-10-26T05:48:49.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 -- 172.21.15.71:0/3148507553 shutdown_connections 2023-10-26T05:48:49.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 --2- 172.21.15.71:0/3148507553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f77740b1e50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 -- 172.21.15.71:0/3148507553 >> 172.21.15.71:0/3148507553 conn(0x7f77740aec60 msgr2=0x7f77740af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:49.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 -- 172.21.15.71:0/3148507553 shutdown_connections 2023-10-26T05:48:49.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.229+0000 7f7779c95640 1 -- 172.21.15.71:0/3148507553 wait complete. 2023-10-26T05:48:49.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 Processor -- start 2023-10-26T05:48:49.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- start start 2023-10-26T05:48:49.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774070680 0x7f7774137a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77740b2390 0x7f777413dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f777413e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774142950 con 0x7f7774141980 2023-10-26T05:48:49.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774137f40 con 0x7f7774070680 2023-10-26T05:48:49.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77740c43e0 con 0x7f77740b2390 2023-10-26T05:48:49.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f777413e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f777413e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50990/0 (socket says 172.21.15.71:50990) 2023-10-26T05:48:49.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 -- 172.21.15.71:0/2138119259 learned_addr learned my addr 172.21.15.71:0/2138119259 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:49.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f77737fe640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774070680 0x7f7774137a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7772ffd640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77740b2390 0x7f777413dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 -- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77740b2390 msgr2=0x7f777413dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77740b2390 0x7f777413dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 -- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774070680 msgr2=0x7f7774137a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774070680 0x7f7774137a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 -- 172.21.15.71:0/2138119259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77640098e0 con 0x7f7774141980 2023-10-26T05:48:49.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7773fff640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f777413e320 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c3c0 tx=0x7f776c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:49.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c011790 con 0x7f7774141980 2023-10-26T05:48:49.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f776c0119e0 con 0x7f7774141980 2023-10-26T05:48:49.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c011df0 con 0x7f7774141980 2023-10-26T05:48:49.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77740c46c0 con 0x7f7774141980 2023-10-26T05:48:49.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7774142570 con 0x7f7774141980 2023-10-26T05:48:49.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774066b10 con 0x7f7774141980 2023-10-26T05:48:49.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.233+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f776c01c460 con 0x7f7774141980 2023-10-26T05:48:49.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.237+0000 7f775bfff640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7760066470 0x7f7760068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.237+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f776c083f60 con 0x7f7774141980 2023-10-26T05:48:49.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.237+0000 7f77737fe640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7760066470 0x7f7760068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.237+0000 7f77737fe640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7760066470 0x7f7760068930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f77640af000 tx=0x7f776401b550 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:49.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.241+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f776c0513e0 con 0x7f7774141980 2023-10-26T05:48:49.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7774142a90 con 0x7f7774141980 2023-10-26T05:48:49.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f775bfff640 1 -- 172.21.15.71:0/2138119259 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7774142a90 con 0x7f7774141980 2023-10-26T05:48:49.397 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:48:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7760066470 msgr2=0x7f7760068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7760066470 0x7f7760068930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f77640af000 tx=0x7f776401b550 comp rx=0 tx=0).stop 2023-10-26T05:48:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 msgr2=0x7f777413e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f777413e320 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c3c0 tx=0x7f776c00c880 comp rx=0 tx=0).stop 2023-10-26T05:48:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 shutdown_connections 2023-10-26T05:48:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7760066470 0x7f7760068930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7774141980 0x7f777413e320 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77740b2390 0x7f777413dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 --2- 172.21.15.71:0/2138119259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774070680 0x7f7774137a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 >> 172.21.15.71:0/2138119259 conn(0x7f77740aec60 msgr2=0x7f777413a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 shutdown_connections 2023-10-26T05:48:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.393+0000 7f7779c95640 1 -- 172.21.15.71:0/2138119259 wait complete. 2023-10-26T05:48:49.415 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 -- 172.21.15.71:0/2561874250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 msgr2=0x7fd65806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 --2- 172.21.15.71:0/2561874250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd65806aa70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd6580c4dd0 tx=0x7fd64801cc60 comp rx=0 tx=0).stop 2023-10-26T05:48:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 -- 172.21.15.71:0/2561874250 shutdown_connections 2023-10-26T05:48:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 --2- 172.21.15.71:0/2561874250 >> v2:172.21.15.86:3300/0 conn(0x7fd6580d0850 0x7fd65815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 --2- 172.21.15.71:0/2561874250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd65806aa70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 -- 172.21.15.71:0/2561874250 >> 172.21.15.71:0/2561874250 conn(0x7fd6580cdd80 msgr2=0x7fd6580ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 -- 172.21.15.71:0/2561874250 shutdown_connections 2023-10-26T05:48:49.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.533+0000 7fd65d435640 1 -- 172.21.15.71:0/2561874250 wait complete. 2023-10-26T05:48:49.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 Processor -- start 2023-10-26T05:48:49.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- start start 2023-10-26T05:48:49.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6580cd880 0x7fd65815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd658157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6580d0850 0x7fd6581f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65815d6c0 con 0x7fd6580d0470 2023-10-26T05:48:49.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65815a0e0 con 0x7fd6580cd880 2023-10-26T05:48:49.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581594d0 con 0x7fd6580d0850 2023-10-26T05:48:49.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd658157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd658157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51020/0 (socket says 172.21.15.71:51020) 2023-10-26T05:48:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 -- 172.21.15.71:0/1729093390 learned_addr learned my addr 172.21.15.71:0/1729093390 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd656ffd640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6580cd880 0x7fd65815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6577fe640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6580d0850 0x7fd6581f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 -- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6580d0850 msgr2=0x7fd6581f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6580d0850 0x7fd6581f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 -- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6580cd880 msgr2=0x7fd65815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6580cd880 0x7fd65815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 -- 172.21.15.71:0/1729093390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480098e0 con 0x7fd6580d0470 2023-10-26T05:48:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd656ffd640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6580cd880 0x7fd65815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:48:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd6567fc640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd658157f50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c940 tx=0x7fd64000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640004220 con 0x7fd6580d0470 2023-10-26T05:48:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd640004470 con 0x7fd6580d0470 2023-10-26T05:48:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6581597b0 con 0x7fd6580d0470 2023-10-26T05:48:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640007c10 con 0x7fd6580d0470 2023-10-26T05:48:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd658159d70 con 0x7fd6580d0470 2023-10-26T05:48:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.537+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd658066b10 con 0x7fd6580d0470 2023-10-26T05:48:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.541+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd640003600 con 0x7fd6580d0470 2023-10-26T05:48:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.541+0000 7fd64f7fe640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd638066470 0x7fd638068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.541+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd640083ee0 con 0x7fd6580d0470 2023-10-26T05:48:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.541+0000 7fd656ffd640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd638066470 0x7fd638068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.541+0000 7fd656ffd640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd638066470 0x7fd638068930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6480af000 tx=0x7fd64801b4a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:49.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.545+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd6400514a0 con 0x7fd6580d0470 2023-10-26T05:48:49.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.721+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6580721e0 con 0x7fd6580d0470 2023-10-26T05:48:49.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.721+0000 7fd64f7fe640 1 -- 172.21.15.71:0/1729093390 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fd64001d090 con 0x7fd6580d0470 2023-10-26T05:48:49.727 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:49.727 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:49.727 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:49.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd638066470 msgr2=0x7fd638068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd638066470 0x7fd638068930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6480af000 tx=0x7fd64801b4a0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 msgr2=0x7fd658157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd658157f50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c940 tx=0x7fd64000ce00 comp rx=0 tx=0).stop 2023-10-26T05:48:49.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 shutdown_connections 2023-10-26T05:48:49.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6580d0850 0x7fd6581f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd638066470 0x7fd638068930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6580d0470 0x7fd658157f50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 --2- 172.21.15.71:0/1729093390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6580cd880 0x7fd65815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 >> 172.21.15.71:0/1729093390 conn(0x7fd6580cdd80 msgr2=0x7fd6580cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:49.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 shutdown_connections 2023-10-26T05:48:49.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.725+0000 7fd65d435640 1 -- 172.21.15.71:0/1729093390 wait complete. 2023-10-26T05:48:49.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:49.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 -- 172.21.15.71:0/3268531221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0cd880 msgr2=0x7f713c15b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 --2- 172.21.15.71:0/3268531221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0cd880 0x7f713c15b8e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f713c0c4dd0 tx=0x7f713401cc70 comp rx=0 tx=0).stop 2023-10-26T05:48:49.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 -- 172.21.15.71:0/3268531221 shutdown_connections 2023-10-26T05:48:49.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 --2- 172.21.15.71:0/3268531221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0cd880 0x7f713c15b8e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 --2- 172.21.15.71:0/3268531221 >> v2:172.21.15.86:3300/0 conn(0x7f713c0d0850 0x7f713c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 -- 172.21.15.71:0/3268531221 >> 172.21.15.71:0/3268531221 conn(0x7f713c0cdd80 msgr2=0x7f713c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:49.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 -- 172.21.15.71:0/3268531221 shutdown_connections 2023-10-26T05:48:49.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.849+0000 7f714239e640 1 -- 172.21.15.71:0/3268531221 wait complete. 2023-10-26T05:48:49.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 Processor -- start 2023-10-26T05:48:49.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- start start 2023-10-26T05:48:49.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f713c0cd880 0x7f713c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 0x7f713c150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f713c0d0850 0x7f713c150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1558e0 con 0x7f713c0d0470 2023-10-26T05:48:49.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c152030 con 0x7f713c0d0850 2023-10-26T05:48:49.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713c155b10 con 0x7f713c0cd880 2023-10-26T05:48:49.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 0x7f713c150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 0x7f713c150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51034/0 (socket says 172.21.15.71:51034) 2023-10-26T05:48:49.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 -- 172.21.15.71:0/3800521741 learned_addr learned my addr 172.21.15.71:0/3800521741 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:49.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714093b640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f713c0d0850 0x7f713c150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713bfff640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f713c0cd880 0x7f713c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 -- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f713c0cd880 msgr2=0x7f713c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f713c0cd880 0x7f713c0c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 -- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f713c0d0850 msgr2=0x7f713c150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:49.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f713c0d0850 0x7f713c150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:49.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 -- 172.21.15.71:0/3800521741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340098e0 con 0x7f713c0d0470 2023-10-26T05:48:49.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713bfff640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f713c0cd880 0x7f713c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:49.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714093b640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f713c0d0850 0x7f713c150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:49.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f713b7fe640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 0x7f713c150730 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f713000c940 tx=0x7f713000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:49.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130004220 con 0x7f713c0d0470 2023-10-26T05:48:49.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7130004470 con 0x7f713c0d0470 2023-10-26T05:48:49.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c155df0 con 0x7f713c0d0470 2023-10-26T05:48:49.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130007c10 con 0x7f713c0d0470 2023-10-26T05:48:49.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c1563b0 con 0x7f713c0d0470 2023-10-26T05:48:49.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713c066b10 con 0x7f713c0d0470 2023-10-26T05:48:49.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7130003600 con 0x7f713c0d0470 2023-10-26T05:48:49.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f7138ff9640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7128066470 0x7f7128068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:49.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.853+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7130084400 con 0x7f713c0d0470 2023-10-26T05:48:49.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.857+0000 7f713bfff640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7128066470 0x7f7128068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:49.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.857+0000 7f713bfff640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7128066470 0x7f7128068930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f712c003e40 tx=0x7f712c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:49.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:49.857+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7130051880 con 0x7f713c0d0470 2023-10-26T05:48:50.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.005+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f713c0721e0 con 0x7f713c0d0470 2023-10-26T05:48:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.005+0000 7f7138ff9640 1 -- 172.21.15.71:0/3800521741 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f713001d090 con 0x7f713c0d0470 2023-10-26T05:48:50.011 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:50.012 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:42.435500+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:50.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7128066470 msgr2=0x7f7128068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7128066470 0x7f7128068930 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f712c003e40 tx=0x7f712c009040 comp rx=0 tx=0).stop 2023-10-26T05:48:50.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 msgr2=0x7f713c150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 0x7f713c150730 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f713000c940 tx=0x7f713000ce00 comp rx=0 tx=0).stop 2023-10-26T05:48:50.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 shutdown_connections 2023-10-26T05:48:50.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f713c0d0850 0x7f713c150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7128066470 0x7f7128068930 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c0d0470 0x7f713c150730 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 --2- 172.21.15.71:0/3800521741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f713c0cd880 0x7f713c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 >> 172.21.15.71:0/3800521741 conn(0x7f713c0cdd80 msgr2=0x7f713c15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 shutdown_connections 2023-10-26T05:48:50.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.009+0000 7f714239e640 1 -- 172.21.15.71:0/3800521741 wait complete. 2023-10-26T05:48:50.030 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:48:50.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- 172.21.15.71:0/2372434015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300cd880 msgr2=0x7fd53015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 --2- 172.21.15.71:0/2372434015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300cd880 0x7fd53015b8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd528009cb0 tx=0x7fd52801cc60 comp rx=0 tx=0).stop 2023-10-26T05:48:50.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- 172.21.15.71:0/2372434015 shutdown_connections 2023-10-26T05:48:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 --2- 172.21.15.71:0/2372434015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300cd880 0x7fd53015b8e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 --2- 172.21.15.71:0/2372434015 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5300d0850 0x7fd53015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- 172.21.15.71:0/2372434015 >> 172.21.15.71:0/2372434015 conn(0x7fd5300cdd80 msgr2=0x7fd5300ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- 172.21.15.71:0/2372434015 shutdown_connections 2023-10-26T05:48:50.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- 172.21.15.71:0/2372434015 wait complete. 2023-10-26T05:48:50.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 Processor -- start 2023-10-26T05:48:50.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- start start 2023-10-26T05:48:50.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5300cd880 0x7fd53015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300d0470 0x7fd53014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 0x7fd53015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53015c950 con 0x7fd5300d0850 2023-10-26T05:48:50.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53015c040 con 0x7fd5300d0470 2023-10-26T05:48:50.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd536834640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53014f610 con 0x7fd5300cd880 2023-10-26T05:48:50.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd534dd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 0x7fd53015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd534dd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 0x7fd53015dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51046/0 (socket says 172.21.15.71:51046) 2023-10-26T05:48:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd534dd1640 1 -- 172.21.15.71:0/544969561 learned_addr learned my addr 172.21.15.71:0/544969561 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd52f7fe640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300d0470 0x7fd53014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.149+0000 7fd52ffff640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5300cd880 0x7fd53015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd534dd1640 1 -- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5300cd880 msgr2=0x7fd53015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd534dd1640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5300cd880 0x7fd53015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd534dd1640 1 -- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300d0470 msgr2=0x7fd53014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd534dd1640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300d0470 0x7fd53014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd534dd1640 1 -- 172.21.15.71:0/544969561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280098e0 con 0x7fd5300d0850 2023-10-26T05:48:50.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52f7fe640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300d0470 0x7fd53014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-26T05:48:50.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd534dd1640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 0x7fd53015dae0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd52801b310 tx=0x7fd5280a5c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528003fd0 con 0x7fd5300d0850 2023-10-26T05:48:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd528004150 con 0x7fd5300d0850 2023-10-26T05:48:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53015e450 con 0x7fd5300d0850 2023-10-26T05:48:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528004560 con 0x7fd5300d0850 2023-10-26T05:48:50.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53015c570 con 0x7fd5300d0850 2023-10-26T05:48:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd530066b10 con 0x7fd5300d0850 2023-10-26T05:48:50.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd5280b4040 con 0x7fd5300d0850 2023-10-26T05:48:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52cff9640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd51c0664a0 0x7fd51c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd52811e870 con 0x7fd5300d0850 2023-10-26T05:48:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52ffff640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd51c0664a0 0x7fd51c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.153+0000 7fd52ffff640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd51c0664a0 0x7fd51c068960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd524007fd0 tx=0x7fd524009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:50.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.157+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd5280ebda0 con 0x7fd5300d0850 2023-10-26T05:48:50.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd53015ca90 con 0x7fd5300d0850 2023-10-26T05:48:50.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd52cff9640 1 -- 172.21.15.71:0/544969561 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fd53015ca90 con 0x7fd5300d0850 2023-10-26T05:48:50.341 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:50.341 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:50.342 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd51c0664a0 msgr2=0x7fd51c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd51c0664a0 0x7fd51c068960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd524007fd0 tx=0x7fd524009040 comp rx=0 tx=0).stop 2023-10-26T05:48:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 msgr2=0x7fd53015dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 0x7fd53015dae0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd52801b310 tx=0x7fd5280a5c50 comp rx=0 tx=0).stop 2023-10-26T05:48:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 shutdown_connections 2023-10-26T05:48:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd51c0664a0 0x7fd51c068960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5300d0850 0x7fd53015dae0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5300d0470 0x7fd53014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 --2- 172.21.15.71:0/544969561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5300cd880 0x7fd53015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 >> 172.21.15.71:0/544969561 conn(0x7fd5300cdd80 msgr2=0x7fd53015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 shutdown_connections 2023-10-26T05:48:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.337+0000 7fd536834640 1 -- 172.21.15.71:0/544969561 wait complete. 2023-10-26T05:48:50.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:50.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/2077628375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e040ce300 msgr2=0x7f0e0415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/2077628375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e040ce300 0x7f0e0415d880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009840 tx=0x7f0df401c800 comp rx=0 tx=0).stop 2023-10-26T05:48:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/2077628375 shutdown_connections 2023-10-26T05:48:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/2077628375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e040ce300 0x7f0e0415d880 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/2077628375 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e040cbad0 0x7f0e0406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/2077628375 >> 172.21.15.71:0/2077628375 conn(0x7f0e0406fd90 msgr2=0x7f0e040b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/2077628375 shutdown_connections 2023-10-26T05:48:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/2077628375 wait complete. 2023-10-26T05:48:50.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 Processor -- start 2023-10-26T05:48:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- start start 2023-10-26T05:48:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e040cbad0 0x7f0e041585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e040ce300 0x7f0e04153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 0x7f0e04154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0414f8c0 con 0x7f0e041594c0 2023-10-26T05:48:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04162380 con 0x7f0e040cbad0 2023-10-26T05:48:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0c6f0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e04162660 con 0x7f0e040ce300 2023-10-26T05:48:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 0x7f0e04154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 0x7f0e04154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51076/0 (socket says 172.21.15.71:51076) 2023-10-26T05:48:50.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 -- 172.21.15.71:0/916121586 learned_addr learned my addr 172.21.15.71:0/916121586 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:50.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0a48c640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e040cbad0 0x7f0e041585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e09c8b640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e040ce300 0x7f0e04153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 -- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e040ce300 msgr2=0x7f0e04153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e040ce300 0x7f0e04153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 -- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e040cbad0 msgr2=0x7f0e041585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e040cbad0 0x7f0e041585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.477+0000 7f0e0ac8d640 1 -- 172.21.15.71:0/916121586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40094a0 con 0x7f0e041594c0 2023-10-26T05:48:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0e0ac8d640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 0x7f0e04154cf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0df401a430 tx=0x7f0df40a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4005020 con 0x7f0e041594c0 2023-10-26T05:48:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0df4005270 con 0x7f0e041594c0 2023-10-26T05:48:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e04162940 con 0x7f0e041594c0 2023-10-26T05:48:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df40046d0 con 0x7f0e041594c0 2023-10-26T05:48:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e04158cb0 con 0x7f0e041594c0 2023-10-26T05:48:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e04066d00 con 0x7f0e041594c0 2023-10-26T05:48:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0df40b4040 con 0x7f0e041594c0 2023-10-26T05:48:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0dfaffd640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0de8066470 0x7f0de8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0df411de40 con 0x7f0e041594c0 2023-10-26T05:48:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0e0a48c640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0de8066470 0x7f0de8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.481+0000 7f0e0a48c640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0de8066470 0x7f0de8068930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005c50 tx=0x7f0dfc005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.485+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0df40eb2c0 con 0x7f0e041594c0 2023-10-26T05:48:50.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.653+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e04162df0 con 0x7f0e041594c0 2023-10-26T05:48:50.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0dfaffd640 1 -- 172.21.15.71:0/916121586 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f0df40b2090 con 0x7f0e041594c0 2023-10-26T05:48:50.660 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:50.661 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:48:24.033549+0000","modified":"2023-10-26T05:48:26.052225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5280},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5280":{"gid":5280,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":4,"addr":"172.21.15.71:6835/4006610413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":4006610413},{"type":"v1","addr":"172.21.15.71:6835","nonce":4006610413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-26T05:48:50.661 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2023-10-26T05:48:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0de8066470 msgr2=0x7f0de8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0de8066470 0x7f0de8068930 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005c50 tx=0x7f0dfc005be0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 msgr2=0x7f0e04154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 0x7f0e04154cf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0df401a430 tx=0x7f0df40a5890 comp rx=0 tx=0).stop 2023-10-26T05:48:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 shutdown_connections 2023-10-26T05:48:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0de8066470 0x7f0de8068930 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e041594c0 0x7f0e04154cf0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e040ce300 0x7f0e04153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 --2- 172.21.15.71:0/916121586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e040cbad0 0x7f0e041585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 >> 172.21.15.71:0/916121586 conn(0x7f0e0406fd90 msgr2=0x7f0e040cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 shutdown_connections 2023-10-26T05:48:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.657+0000 7f0e0c6f0640 1 -- 172.21.15.71:0/916121586 wait complete. 2023-10-26T05:48:50.677 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:50.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 -- 172.21.15.71:0/3584095089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40b1360 msgr2=0x7f54d40b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 --2- 172.21.15.71:0/3584095089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40b1360 0x7f54d40b17d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009c80 tx=0x7f54cc01cd10 comp rx=0 tx=0).stop 2023-10-26T05:48:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 -- 172.21.15.71:0/3584095089 shutdown_connections 2023-10-26T05:48:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 --2- 172.21.15.71:0/3584095089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d40b1360 0x7f54d40b17d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 --2- 172.21.15.71:0/3584095089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d4070fe0 0x7f54d4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 -- 172.21.15.71:0/3584095089 >> 172.21.15.71:0/3584095089 conn(0x7f54d4069770 msgr2=0x7f54d4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 -- 172.21.15.71:0/3584095089 shutdown_connections 2023-10-26T05:48:50.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.793+0000 7f54da57f640 1 -- 172.21.15.71:0/3584095089 wait complete. 2023-10-26T05:48:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 Processor -- start 2023-10-26T05:48:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- start start 2023-10-26T05:48:50.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54d4070fe0 0x7f54d415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d40b1360 0x7f54d40ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 0x7f54d4157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d415d720 con 0x7f54d4140cc0 2023-10-26T05:48:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40ae120 con 0x7f54d4070fe0 2023-10-26T05:48:50.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d415cb10 con 0x7f54d40b1360 2023-10-26T05:48:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 0x7f54d4157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 0x7f54d4157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51096/0 (socket says 172.21.15.71:51096) 2023-10-26T05:48:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 -- 172.21.15.71:0/2126590024 learned_addr learned my addr 172.21.15.71:0/2126590024 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d37fe640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d40b1360 0x7f54d40ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d3fff640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54d4070fe0 0x7f54d415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 -- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d40b1360 msgr2=0x7f54d40ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d40b1360 0x7f54d40ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 -- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54d4070fe0 msgr2=0x7f54d415e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54d4070fe0 0x7f54d415e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 -- 172.21.15.71:0/2126590024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0098e0 con 0x7f54d4140cc0 2023-10-26T05:48:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d37fe640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d40b1360 0x7f54d40ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d3fff640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54d4070fe0 0x7f54d415e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d8b1c640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 0x7f54d4157790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f54cc01a940 tx=0x7f54cc004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0ad7b0 con 0x7f54d4140cc0 2023-10-26T05:48:50.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f54cc0ada00 con 0x7f54d4140cc0 2023-10-26T05:48:50.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d415cdf0 con 0x7f54d4140cc0 2023-10-26T05:48:50.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0addd0 con 0x7f54d4140cc0 2023-10-26T05:48:50.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d415d3b0 con 0x7f54d4140cc0 2023-10-26T05:48:50.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.797+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d4066b10 con 0x7f54d4140cc0 2023-10-26T05:48:50.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.801+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f54cc0b4040 con 0x7f54d4140cc0 2023-10-26T05:48:50.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.801+0000 7f54d0ff9640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54c0066470 0x7f54c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:50.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.801+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54cc11e2b0 con 0x7f54d4140cc0 2023-10-26T05:48:50.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.801+0000 7f54d3fff640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54c0066470 0x7f54c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:50.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.801+0000 7f54d3fff640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54c0066470 0x7f54c0068930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54c400a830 tx=0x7f54c4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:50.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.805+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f54cc0eb730 con 0x7f54d4140cc0 2023-10-26T05:48:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.949+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54d413cf50 con 0x7f54d4140cc0 2023-10-26T05:48:50.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.949+0000 7f54d0ff9640 1 -- 172.21.15.71:0/2126590024 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f54cc0eed50 con 0x7f54d4140cc0 2023-10-26T05:48:50.954 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:50.955 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:42.435500+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:48:21.848187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-26T05:48:23.028324+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54c0066470 msgr2=0x7f54c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54c0066470 0x7f54c0068930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f54c400a830 tx=0x7f54c4008040 comp rx=0 tx=0).stop 2023-10-26T05:48:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 msgr2=0x7f54d4157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 0x7f54d4157790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f54cc01a940 tx=0x7f54cc004de0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 shutdown_connections 2023-10-26T05:48:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54c0066470 0x7f54c0068930 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54d4140cc0 0x7f54d4157790 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f54d40b1360 0x7f54d40ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 --2- 172.21.15.71:0/2126590024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54d4070fe0 0x7f54d415e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 >> 172.21.15.71:0/2126590024 conn(0x7f54d4069770 msgr2=0x7f54d406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 shutdown_connections 2023-10-26T05:48:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:50.953+0000 7f54da57f640 1 -- 172.21.15.71:0/2126590024 wait complete. 2023-10-26T05:48:50.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:48:51.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/2805152061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a1c0039b0 msgr2=0x7f1a1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/2805152061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a1c0039b0 0x7f1a1c003dd0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1a28006fd0 tx=0x7f1a2801cfa0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 -- 172.21.15.71:0/2805152061 reap_dead start 2023-10-26T05:48:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/2805152061 shutdown_connections 2023-10-26T05:48:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/2805152061 >> 172.21.15.71:0/2805152061 conn(0x7f1a340addc0 msgr2=0x7f1a340b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/2805152061 shutdown_connections 2023-10-26T05:48:51.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/2805152061 wait complete. 2023-10-26T05:48:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 Processor -- start 2023-10-26T05:48:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- start start 2023-10-26T05:48:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a340ac9d0 0x7f1a340b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a340b3260 0x7f1a341416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 0x7f1a340a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340a8d10 con 0x7f1a340a8350 2023-10-26T05:48:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340b21c0 con 0x7f1a340b3260 2023-10-26T05:48:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a340b24a0 con 0x7f1a340ac9d0 2023-10-26T05:48:51.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 0x7f1a340a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 0x7f1a340a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51114/0 (socket says 172.21.15.71:51114) 2023-10-26T05:48:51.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 -- 172.21.15.71:0/1269486120 learned_addr learned my addr 172.21.15.71:0/1269486120 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:51.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a3259c640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a340ac9d0 0x7f1a340b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a31d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a340b3260 0x7f1a341416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 -- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a340ac9d0 msgr2=0x7f1a340b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a340ac9d0 0x7f1a340b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 -- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a340b3260 msgr2=0x7f1a341416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a340b3260 0x7f1a341416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 -- 172.21.15.71:0/1269486120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340b2780 con 0x7f1a340a8350 2023-10-26T05:48:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a31d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a340b3260 0x7f1a341416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a32d9d640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 0x7f1a340a87d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1a340704c0 tx=0x7f1a2c00e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00c070 con 0x7f1a340a8350 2023-10-26T05:48:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a2c0135f0 con 0x7f1a340a8350 2023-10-26T05:48:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a340b2a60 con 0x7f1a340a8350 2023-10-26T05:48:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c013a00 con 0x7f1a340a8350 2023-10-26T05:48:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.085+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a34141c10 con 0x7f1a340a8350 2023-10-26T05:48:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.089+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a34066b10 con 0x7f1a340a8350 2023-10-26T05:48:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.089+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1a2c01c3b0 con 0x7f1a340a8350 2023-10-26T05:48:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.089+0000 7f1a22ffd640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a14066470 0x7f1a14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.089+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a2c083220 con 0x7f1a340a8350 2023-10-26T05:48:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.089+0000 7f1a3259c640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a14066470 0x7f1a14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.089+0000 7f1a3259c640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a14066470 0x7f1a14068930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1a280ac000 tx=0x7f1a280b0040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.093+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1a2c0506a0 con 0x7f1a340a8350 2023-10-26T05:48:51.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.265+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1a34069920 con 0x7f1a340a8350 2023-10-26T05:48:51.271 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:48:51.269+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:48:51.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.269+0000 7f1a22ffd640 1 -- 172.21.15.71:0/1269486120 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v13) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1a2c0014b0 con 0x7f1a340a8350 2023-10-26T05:48:51.275 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:48:51.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.273+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a14066470 msgr2=0x7f1a14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.273+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a14066470 0x7f1a14068930 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1a280ac000 tx=0x7f1a280b0040 comp rx=0 tx=0).stop 2023-10-26T05:48:51.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.273+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 msgr2=0x7f1a340a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.273+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 0x7f1a340a87d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1a340704c0 tx=0x7f1a2c00e920 comp rx=0 tx=0).stop 2023-10-26T05:48:51.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 shutdown_connections 2023-10-26T05:48:51.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a14066470 0x7f1a14068930 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a340a8350 0x7f1a340a87d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a340b3260 0x7f1a341416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 --2- 172.21.15.71:0/1269486120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a340ac9d0 0x7f1a340b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 >> 172.21.15.71:0/1269486120 conn(0x7f1a340addc0 msgr2=0x7f1a34161140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:51.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 shutdown_connections 2023-10-26T05:48:51.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.277+0000 7f1a38d9b640 1 -- 172.21.15.71:0/1269486120 wait complete. 2023-10-26T05:48:51.282 INFO:tasks.ceph.mds.a.smithi071.stderr: -3361> 2023-10-26T05:48:27.565+0000 7fbcc234f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:48:51.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:48:51.310 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:48:51.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3540163953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40b2560 msgr2=0x7fd9d406ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3540163953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40b2560 0x7fd9d406ff70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40c9f70 tx=0x7fd9c401c830 comp rx=0 tx=0).stop 2023-10-26T05:48:51.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3540163953 shutdown_connections 2023-10-26T05:48:51.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3540163953 >> v2:172.21.15.86:3300/0 conn(0x7fd9d40704b0 0x7fd9d40cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3540163953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40b2560 0x7fd9d406ff70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3540163953 >> 172.21.15.71:0/3540163953 conn(0x7fd9d40aec60 msgr2=0x7fd9d40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3540163953 shutdown_connections 2023-10-26T05:48:51.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3540163953 wait complete. 2023-10-26T05:48:51.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 Processor -- start 2023-10-26T05:48:51.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- start start 2023-10-26T05:48:51.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 0x7fd9d4154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9d40b2560 0x7fd9d415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9d4141980 0x7fd9d413e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415d830 con 0x7fd9d40704b0 2023-10-26T05:48:51.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415cf90 con 0x7fd9d4141980 2023-10-26T05:48:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9dac7a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d415d270 con 0x7fd9d40b2560 2023-10-26T05:48:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 0x7fd9d4154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 0x7fd9d4154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51164/0 (socket says 172.21.15.71:51164) 2023-10-26T05:48:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 -- 172.21.15.71:0/3857257596 learned_addr learned my addr 172.21.15.71:0/3857257596 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:51.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d9217640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9d4141980 0x7fd9d413e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d3fff640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9d40b2560 0x7fd9d415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 -- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9d40b2560 msgr2=0x7fd9d415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9d40b2560 0x7fd9d415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 -- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9d4141980 msgr2=0x7fd9d413e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:51.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9d4141980 0x7fd9d413e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 -- 172.21.15.71:0/3857257596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4009560 con 0x7fd9d40704b0 2023-10-26T05:48:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d9217640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9d4141980 0x7fd9d413e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d8a16640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 0x7fd9d4154960 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00b410 tx=0x7fd9bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc004020 con 0x7fd9d40704b0 2023-10-26T05:48:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.413+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9bc004270 con 0x7fd9d40704b0 2023-10-26T05:48:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d415d550 con 0x7fd9d40704b0 2023-10-26T05:48:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc011890 con 0x7fd9d40704b0 2023-10-26T05:48:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d4154ea0 con 0x7fd9d40704b0 2023-10-26T05:48:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d4066b10 con 0x7fd9d40704b0 2023-10-26T05:48:51.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9bc003600 con 0x7fd9d40704b0 2023-10-26T05:48:51.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9d17fa640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9b4066470 0x7fd9b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd9bc083ea0 con 0x7fd9d40704b0 2023-10-26T05:48:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9d3fff640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9b4066470 0x7fd9b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.417+0000 7fd9d3fff640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9b4066470 0x7fd9b4068930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40af000 tx=0x7fd9c401b070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:51.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.421+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9bc051350 con 0x7fd9d40704b0 2023-10-26T05:48:51.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:51.605+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd9d40721e0 con 0x7fd9d40704b0 2023-10-26T05:48:52.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.281+0000 7fd9d17fa640 1 -- 172.21.15.71:0/3857257596 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9bc054970 con 0x7fd9d40704b0 2023-10-26T05:48:52.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9b4066470 msgr2=0x7fd9b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:52.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9b4066470 0x7fd9b4068930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40af000 tx=0x7fd9c401b070 comp rx=0 tx=0).stop 2023-10-26T05:48:52.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 msgr2=0x7fd9d4154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:52.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 0x7fd9d4154960 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00b410 tx=0x7fd9bc00b8d0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 shutdown_connections 2023-10-26T05:48:52.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9d4141980 0x7fd9d413e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9d40b2560 0x7fd9d415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9b4066470 0x7fd9b4068930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 --2- 172.21.15.71:0/3857257596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9d40704b0 0x7fd9d4154960 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 >> 172.21.15.71:0/3857257596 conn(0x7fd9d40aec60 msgr2=0x7fd9d40b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:52.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 shutdown_connections 2023-10-26T05:48:52.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.285+0000 7fd9dac7a640 1 -- 172.21.15.71:0/3857257596 wait complete. 2023-10-26T05:48:52.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:48:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 -- 172.21.15.71:0/3706385037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 msgr2=0x7f2ae80cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/3706385037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae80cb8c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009900 tx=0x7f2ad801c890 comp rx=0 tx=0).stop 2023-10-26T05:48:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 -- 172.21.15.71:0/3706385037 shutdown_connections 2023-10-26T05:48:52.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/3706385037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae813ef60 0x7f2ae8069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/3706385037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae80cb8c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 -- 172.21.15.71:0/3706385037 >> 172.21.15.71:0/3706385037 conn(0x7f2ae80aec60 msgr2=0x7f2ae80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 -- 172.21.15.71:0/3706385037 shutdown_connections 2023-10-26T05:48:52.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.473+0000 7f2aee5a2640 1 -- 172.21.15.71:0/3706385037 wait complete. 2023-10-26T05:48:52.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 Processor -- start 2023-10-26T05:48:52.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- start start 2023-10-26T05:48:52.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae80cbe00 0x7f2ae80abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:52.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae814d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:52.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae813ef60 0x7f2ae81362a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:52.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae80aba50 con 0x7f2ae813c120 2023-10-26T05:48:52.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae814df00 con 0x7f2ae80cbe00 2023-10-26T05:48:52.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae814d420 con 0x7f2ae813ef60 2023-10-26T05:48:52.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae814d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:52.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae814d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36654/0 (socket says 172.21.15.71:36654) 2023-10-26T05:48:52.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 -- 172.21.15.71:0/4282020725 learned_addr learned my addr 172.21.15.71:0/4282020725 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:52.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae7fff640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae80cbe00 0x7f2ae80abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:52.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aecb3f640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae813ef60 0x7f2ae81362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:52.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 -- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae813ef60 msgr2=0x7f2ae81362a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:52.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae813ef60 0x7f2ae81362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 -- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae80cbe00 msgr2=0x7f2ae80abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:52.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae80cbe00 0x7f2ae80abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:52.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 -- 172.21.15.71:0/4282020725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad8009560 con 0x7f2ae813c120 2023-10-26T05:48:52.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae7fff640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae80cbe00 0x7f2ae80abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:52.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae77fe640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae814d9c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00c4d0 tx=0x7f2adc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:52.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc005480 con 0x7f2ae813c120 2023-10-26T05:48:52.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2adc0036f0 con 0x7f2ae813c120 2023-10-26T05:48:52.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae80ab390 con 0x7f2ae813c120 2023-10-26T05:48:52.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc003b00 con 0x7f2ae813c120 2023-10-26T05:48:52.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae815ba90 con 0x7f2ae813c120 2023-10-26T05:48:52.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae814ac70 con 0x7f2ae813c120 2023-10-26T05:48:52.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.477+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2adc003c60 con 0x7f2ae813c120 2023-10-26T05:48:52.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.481+0000 7f2ae4ff9640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ad4066470 0x7f2ad4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:52.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.481+0000 7f2ae7fff640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ad4066470 0x7f2ad4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:52.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.481+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2adc054fc0 con 0x7f2ae813c120 2023-10-26T05:48:52.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.481+0000 7f2ae7fff640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ad4066470 0x7f2ad4068930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80af000 tx=0x7f2ad801b0d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:52.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.485+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2adc050c00 con 0x7f2ae813c120 2023-10-26T05:48:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:52.629+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ae8066b10 con 0x7f2ae813c120 2023-10-26T05:48:53.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.489+0000 7f2ae4ff9640 1 -- 172.21.15.71:0/4282020725 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2adc017070 con 0x7f2ae813c120 2023-10-26T05:48:53.496 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:48:53.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ad4066470 msgr2=0x7f2ad4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:53.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ad4066470 0x7f2ad4068930 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80af000 tx=0x7f2ad801b0d0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 msgr2=0x7f2ae814d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:53.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae814d9c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00c4d0 tx=0x7f2adc00c990 comp rx=0 tx=0).stop 2023-10-26T05:48:53.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 shutdown_connections 2023-10-26T05:48:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae813ef60 0x7f2ae81362a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ad4066470 0x7f2ad4068930 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae813c120 0x7f2ae814d9c0 secure :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00c4d0 tx=0x7f2adc00c990 comp rx=0 tx=0).stop 2023-10-26T05:48:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 --2- 172.21.15.71:0/4282020725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae80cbe00 0x7f2ae80abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 >> 172.21.15.71:0/4282020725 conn(0x7f2ae80aec60 msgr2=0x7f2ae815b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:53.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 shutdown_connections 2023-10-26T05:48:53.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.497+0000 7f2aee5a2640 1 -- 172.21.15.71:0/4282020725 wait complete. 2023-10-26T05:48:53.520 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:48:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/573807298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf000039b0 msgr2=0x7fcf00003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/573807298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf000039b0 0x7fcf00003dd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009e60 tx=0x7fcf0801d0e0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf16d9d640 1 -- 172.21.15.71:0/573807298 reap_dead start 2023-10-26T05:48:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/573807298 shutdown_connections 2023-10-26T05:48:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/573807298 >> 172.21.15.71:0/573807298 conn(0x7fcf180d13f0 msgr2=0x7fcf180d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/573807298 shutdown_connections 2023-10-26T05:48:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/573807298 wait complete. 2023-10-26T05:48:53.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 Processor -- start 2023-10-26T05:48:53.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- start start 2023-10-26T05:48:53.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf000039b0 0x7fcf181e8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:53.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 0x7fcf18154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:53.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf181534a0 0x7fcf18153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:53.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18155780 con 0x7fcf181540f0 2023-10-26T05:48:53.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18153e60 con 0x7fcf000039b0 2023-10-26T05:48:53.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18154b70 con 0x7fcf181534a0 2023-10-26T05:48:53.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 0x7fcf18154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:53.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 0x7fcf18154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36668/0 (socket says 172.21.15.71:36668) 2023-10-26T05:48:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 -- 172.21.15.71:0/2225710570 learned_addr learned my addr 172.21.15.71:0/2225710570 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1659c640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf000039b0 0x7fcf181e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf16d9d640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf181534a0 0x7fcf18153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:53.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 -- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf181534a0 msgr2=0x7fcf18153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf181534a0 0x7fcf18153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 -- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf000039b0 msgr2=0x7fcf181e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf000039b0 0x7fcf181e8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 -- 172.21.15.71:0/2225710570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18154e50 con 0x7fcf181540f0 2023-10-26T05:48:53.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf15d9b640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 0x7fcf18154550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00e590 tx=0x7fcf0c00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00c8f0 con 0x7fcf181540f0 2023-10-26T05:48:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf0c00cb40 con 0x7fcf181540f0 2023-10-26T05:48:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf18155130 con 0x7fcf181540f0 2023-10-26T05:48:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c011460 con 0x7fcf181540f0 2023-10-26T05:48:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.653+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf181e8e00 con 0x7fcf181540f0 2023-10-26T05:48:53.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.657+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18066b10 con 0x7fcf181540f0 2023-10-26T05:48:53.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.657+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf0c0115c0 con 0x7fcf181540f0 2023-10-26T05:48:53.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.657+0000 7fcf06ffd640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcef80664a0 0x7fcef8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:53.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.657+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fcf0c0844c0 con 0x7fcf181540f0 2023-10-26T05:48:53.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.657+0000 7fcf1659c640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcef80664a0 0x7fcef8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:53.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.657+0000 7fcf1659c640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcef80664a0 0x7fcef8068960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcf080ae000 tx=0x7fcf08019e80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:53.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.661+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf0c051aa0 con 0x7fcf181540f0 2023-10-26T05:48:53.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:53.817+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf180721e0 con 0x7fcf181540f0 2023-10-26T05:48:54.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.505+0000 7fcf06ffd640 1 -- 172.21.15.71:0/2225710570 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcf0c0150d0 con 0x7fcf181540f0 2023-10-26T05:48:54.509 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:48:54.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.509+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcef80664a0 msgr2=0x7fcef8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.509+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcef80664a0 0x7fcef8068960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcf080ae000 tx=0x7fcf08019e80 comp rx=0 tx=0).stop 2023-10-26T05:48:54.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.509+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 msgr2=0x7fcf18154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.509+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 0x7fcf18154550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00e590 tx=0x7fcf0c00ea50 comp rx=0 tx=0).stop 2023-10-26T05:48:54.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.513+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 shutdown_connections 2023-10-26T05:48:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.513+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf181534a0 0x7fcf18153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.517+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcef80664a0 0x7fcef8068960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.517+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf181540f0 0x7fcf18154550 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.517+0000 7fcf1ca5f640 1 --2- 172.21.15.71:0/2225710570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf000039b0 0x7fcf181e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.517+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 >> 172.21.15.71:0/2225710570 conn(0x7fcf180d13f0 msgr2=0x7fcf180cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.517+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 shutdown_connections 2023-10-26T05:48:54.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.517+0000 7fcf1ca5f640 1 -- 172.21.15.71:0/2225710570 wait complete. 2023-10-26T05:48:54.547 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:48:54.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- 172.21.15.71:0/2377764788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71080b15f0 msgr2=0x7f71081586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 --2- 172.21.15.71:0/2377764788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71080b15f0 0x7f71081586c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f710806d750 tx=0x7f70fc01c870 comp rx=0 tx=0).stop 2023-10-26T05:48:54.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- 172.21.15.71:0/2377764788 shutdown_connections 2023-10-26T05:48:54.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 --2- 172.21.15.71:0/2377764788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71080b15f0 0x7f71081586c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 --2- 172.21.15.71:0/2377764788 >> v2:172.21.15.86:3300/0 conn(0x7f71080cc730 0x7f710815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- 172.21.15.71:0/2377764788 >> 172.21.15.71:0/2377764788 conn(0x7f710806b8c0 msgr2=0x7f710806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:54.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- 172.21.15.71:0/2377764788 shutdown_connections 2023-10-26T05:48:54.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- 172.21.15.71:0/2377764788 wait complete. 2023-10-26T05:48:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 Processor -- start 2023-10-26T05:48:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- start start 2023-10-26T05:48:54.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 0x7f7108145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71080b15f0 0x7f7108156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71080cc730 0x7f7108156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108153790 con 0x7f710806a000 2023-10-26T05:48:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108147620 con 0x7f71080b15f0 2023-10-26T05:48:54.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710d14d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71081531d0 con 0x7f71080cc730 2023-10-26T05:48:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 0x7f7108145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 0x7f7108145b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36684/0 (socket says 172.21.15.71:36684) 2023-10-26T05:48:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 -- 172.21.15.71:0/3473579010 learned_addr learned my addr 172.21.15.71:0/3473579010 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710659c640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71080b15f0 0x7f7108156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 -- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71080cc730 msgr2=0x7f7108156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f710759e640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71080cc730 0x7f7108156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71080cc730 0x7f7108156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 -- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71080b15f0 msgr2=0x7f7108156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71080b15f0 0x7f7108156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.669+0000 7f7106d9d640 1 -- 172.21.15.71:0/3473579010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc009560 con 0x7f710806a000 2023-10-26T05:48:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f7106d9d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 0x7f7108145b30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c970 tx=0x7f70f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8004220 con 0x7f710806a000 2023-10-26T05:48:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f70f8004470 con 0x7f710806a000 2023-10-26T05:48:54.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71080ccd20 con 0x7f710806a000 2023-10-26T05:48:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f8007bf0 con 0x7f710806a000 2023-10-26T05:48:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081472b0 con 0x7f710806a000 2023-10-26T05:48:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7108066b10 con 0x7f710806a000 2023-10-26T05:48:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f70f8007d50 con 0x7f710806a000 2023-10-26T05:48:54.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f70ef7fe640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f70f40664a0 0x7f70f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f70f8083c40 con 0x7f710806a000 2023-10-26T05:48:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f710659c640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f70f40664a0 0x7f70f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.673+0000 7f710659c640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f70f40664a0 0x7f70f4068960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0af000 tx=0x7f70fc01b0b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:54.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.677+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f70f8051450 con 0x7f710806a000 2023-10-26T05:48:54.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.833+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f710806c7d0 con 0x7f710806a000 2023-10-26T05:48:54.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.837+0000 7f70ef7fe640 1 -- 172.21.15.71:0/3473579010 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v20) v1 ==== 71+0+0 (secure 0 0 0) 0x7f70f801d080 con 0x7f710806a000 2023-10-26T05:48:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f70f40664a0 msgr2=0x7f70f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f70f40664a0 0x7f70f4068960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0af000 tx=0x7f70fc01b0b0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 msgr2=0x7f7108145b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 0x7f7108145b30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c970 tx=0x7f70f800ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 shutdown_connections 2023-10-26T05:48:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f71080cc730 0x7f7108156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71080b15f0 0x7f7108156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f70f40664a0 0x7f70f4068960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 --2- 172.21.15.71:0/3473579010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f710806a000 0x7f7108145b30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 >> 172.21.15.71:0/3473579010 conn(0x7f710806b8c0 msgr2=0x7f710806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 shutdown_connections 2023-10-26T05:48:54.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.841+0000 7f710d14d640 1 -- 172.21.15.71:0/3473579010 wait complete. 2023-10-26T05:48:54.862 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-10-26T05:48:54.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1660069013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb394003980 msgr2=0x7fb394003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1660069013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb394003980 0x7fb394003da0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00c610 tx=0x7fb39c01d2d0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b276e640 1 -- 172.21.15.71:0/1660069013 reap_dead start 2023-10-26T05:48:54.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1660069013 shutdown_connections 2023-10-26T05:48:54.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1660069013 >> 172.21.15.71:0/1660069013 conn(0x7fb3ac069770 msgr2=0x7fb3ac069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:54.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1660069013 shutdown_connections 2023-10-26T05:48:54.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1660069013 wait complete. 2023-10-26T05:48:54.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 Processor -- start 2023-10-26T05:48:54.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.985+0000 7fb3b41d1640 1 -- start start 2023-10-26T05:48:54.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394003980 0x7fb3ac159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ac15b6e0 0x7fb3ac06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 0x7fb3ac133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:54.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac070710 con 0x7fb3ac133200 2023-10-26T05:48:54.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac1352a0 con 0x7fb394003980 2023-10-26T05:48:54.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac135510 con 0x7fb3ac15b6e0 2023-10-26T05:48:54.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 0x7fb3ac133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 0x7fb3ac133690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36698/0 (socket says 172.21.15.71:36698) 2023-10-26T05:48:54.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 -- 172.21.15.71:0/1933216054 learned_addr learned my addr 172.21.15.71:0/1933216054 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:54.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b1f6d640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394003980 0x7fb3ac159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b176c640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ac15b6e0 0x7fb3ac06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:54.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 -- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ac15b6e0 msgr2=0x7fb3ac06a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ac15b6e0 0x7fb3ac06a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 -- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394003980 msgr2=0x7fb3ac159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:54.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394003980 0x7fb3ac159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:54.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 -- 172.21.15.71:0/1933216054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac1357f0 con 0x7fb3ac133200 2023-10-26T05:48:54.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b276e640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 0x7fb3ac133690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac06f780 tx=0x7fb3a400e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:54.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4005020 con 0x7fb3ac133200 2023-10-26T05:48:54.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3a4005270 con 0x7fb3ac133200 2023-10-26T05:48:54.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac135ad0 con 0x7fb3ac133200 2023-10-26T05:48:54.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4005660 con 0x7fb3ac133200 2023-10-26T05:48:54.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac133bd0 con 0x7fb3ac133200 2023-10-26T05:48:54.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac066b10 con 0x7fb3ac133200 2023-10-26T05:48:54.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb3a40047b0 con 0x7fb3ac133200 2023-10-26T05:48:54.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3aa7fc640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb398066470 0x7fb398068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.989+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3a40839d0 con 0x7fb3ac133200 2023-10-26T05:48:55.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.993+0000 7fb3b1f6d640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb398066470 0x7fb398068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.993+0000 7fb3b1f6d640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb398066470 0x7fb398068930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0ac000 tx=0x7fb39c01a070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:55.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:54.993+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb3a40511e0 con 0x7fb3ac133200 2023-10-26T05:48:55.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.205+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fb3ac15bac0 con 0x7fb3ac133200 2023-10-26T05:48:55.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3aa7fc640 1 -- 172.21.15.71:0/1933216054 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v109) v1 ==== 130+0+0 (secure 0 0 0) 0x7fb3a4054800 con 0x7fb3ac133200 2023-10-26T05:48:55.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb398066470 msgr2=0x7fb398068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb398066470 0x7fb398068930 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0ac000 tx=0x7fb39c01a070 comp rx=0 tx=0).stop 2023-10-26T05:48:55.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 msgr2=0x7fb3ac133690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 0x7fb3ac133690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac06f780 tx=0x7fb3a400e9b0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 shutdown_connections 2023-10-26T05:48:55.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb398066470 0x7fb398068930 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ac133200 0x7fb3ac133690 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ac15b6e0 0x7fb3ac06a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 --2- 172.21.15.71:0/1933216054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb394003980 0x7fb3ac159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 >> 172.21.15.71:0/1933216054 conn(0x7fb3ac069770 msgr2=0x7fb3ac13f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:55.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.509+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 shutdown_connections 2023-10-26T05:48:55.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.513+0000 7fb3b41d1640 1 -- 172.21.15.71:0/1933216054 wait complete. 2023-10-26T05:48:55.536 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-26T05:48:55.537 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T05:48:55.537 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:55.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- 172.21.15.71:0/901183111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8200cb010 msgr2=0x7fe820159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 --2- 172.21.15.71:0/901183111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8200cb010 0x7fe820159810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe818009bc0 tx=0x7fe81801cb80 comp rx=0 tx=0).stop 2023-10-26T05:48:55.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- 172.21.15.71:0/901183111 shutdown_connections 2023-10-26T05:48:55.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 --2- 172.21.15.71:0/901183111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8200cb010 0x7fe820159810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 --2- 172.21.15.71:0/901183111 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8200b2a60 0x7fe8200caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- 172.21.15.71:0/901183111 >> 172.21.15.71:0/901183111 conn(0x7fe820069770 msgr2=0x7fe820069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:55.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- 172.21.15.71:0/901183111 shutdown_connections 2023-10-26T05:48:55.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- 172.21.15.71:0/901183111 wait complete. 2023-10-26T05:48:55.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 Processor -- start 2023-10-26T05:48:55.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- start start 2023-10-26T05:48:55.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8200b2a60 0x7fe82014d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 0x7fe8201369c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82013c2f0 0x7fe820069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b1110 con 0x7fe8200cb010 2023-10-26T05:48:55.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820132200 con 0x7fe82013c2f0 2023-10-26T05:48:55.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.661+0000 7fe8280a9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200b1fc0 con 0x7fe8200b2a60 2023-10-26T05:48:55.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 0x7fe8201369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 0x7fe8201369c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36714/0 (socket says 172.21.15.71:36714) 2023-10-26T05:48:55.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 -- 172.21.15.71:0/1453978351 learned_addr learned my addr 172.21.15.71:0/1453978351 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:55.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe826646640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82013c2f0 0x7fe820069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825e45640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8200b2a60 0x7fe82014d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 -- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8200b2a60 msgr2=0x7fe82014d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8200b2a60 0x7fe82014d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 -- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82013c2f0 msgr2=0x7fe820069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82013c2f0 0x7fe820069b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 -- 172.21.15.71:0/1453978351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe818009820 con 0x7fe8200cb010 2023-10-26T05:48:55.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe826646640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82013c2f0 0x7fe820069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:55.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825644640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 0x7fe8201369c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00e9f0 tx=0x7fe81c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:55.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c009710 con 0x7fe8200cb010 2023-10-26T05:48:55.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe81c00ce70 con 0x7fe8200cb010 2023-10-26T05:48:55.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c010690 con 0x7fe8200cb010 2023-10-26T05:48:55.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8200b1360 con 0x7fe8200cb010 2023-10-26T05:48:55.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8200b1920 con 0x7fe8200cb010 2023-10-26T05:48:55.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe820066b10 con 0x7fe8200cb010 2023-10-26T05:48:55.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe81c0107f0 con 0x7fe8200cb010 2023-10-26T05:48:55.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8127fc640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe804066470 0x7fe804068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe81c014070 con 0x7fe8200cb010 2023-10-26T05:48:55.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825e45640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe804066470 0x7fe804068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.665+0000 7fe825e45640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe804066470 0x7fe804068930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe814000f70 tx=0x7fe814009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:55.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.669+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe81c052030 con 0x7fe8200cb010 2023-10-26T05:48:55.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.825+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8200721e0 con 0x7fe8200cb010 2023-10-26T05:48:55.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.825+0000 7fe8127fc640 1 -- 172.21.15.71:0/1453978351 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+12902 (secure 0 0 0) 0x7fe81c055650 con 0x7fe8200cb010 2023-10-26T05:48:55.831 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:55.831 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":39,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:54.494899+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/4006610413":"2023-10-27T05:48:51.270056+0000","172.21.15.71:6834/4006610413":"2023-10-27T05:48:51.270056+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:55.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe804066470 msgr2=0x7fe804068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe804066470 0x7fe804068930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe814000f70 tx=0x7fe814009040 comp rx=0 tx=0).stop 2023-10-26T05:48:55.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 msgr2=0x7fe8201369c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 0x7fe8201369c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00e9f0 tx=0x7fe81c00eeb0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 shutdown_connections 2023-10-26T05:48:55.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82013c2f0 0x7fe820069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe804066470 0x7fe804068930 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8200cb010 0x7fe8201369c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 --2- 172.21.15.71:0/1453978351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8200b2a60 0x7fe82014d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 >> 172.21.15.71:0/1453978351 conn(0x7fe820069770 msgr2=0x7fe8200b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:55.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 shutdown_connections 2023-10-26T05:48:55.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.829+0000 7fe8280a9640 1 -- 172.21.15.71:0/1453978351 wait complete. 2023-10-26T05:48:55.850 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:55.851 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:48:55.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/17079652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 msgr2=0x7fb36c158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 --2- 172.21.15.71:0/17079652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c158920 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb360009870 tx=0x7fb36001c730 comp rx=0 tx=0).stop 2023-10-26T05:48:55.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/17079652 shutdown_connections 2023-10-26T05:48:55.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 --2- 172.21.15.71:0/17079652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c158920 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/17079652 >> 172.21.15.71:0/17079652 conn(0x7fb36c06b980 msgr2=0x7fb36c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:55.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/17079652 shutdown_connections 2023-10-26T05:48:55.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/17079652 wait complete. 2023-10-26T05:48:55.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 Processor -- start 2023-10-26T05:48:55.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- start start 2023-10-26T05:48:55.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb36c0cc950 0x7fb36c151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb36c1527e0 0x7fb36c06a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c151d60 con 0x7fb36c152bc0 2023-10-26T05:48:55.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c156510 con 0x7fb36c0cc950 2023-10-26T05:48:55.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c149b40 con 0x7fb36c1527e0 2023-10-26T05:48:55.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36726/0 (socket says 172.21.15.71:36726) 2023-10-26T05:48:55.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 -- 172.21.15.71:0/374547850 learned_addr learned my addr 172.21.15.71:0/374547850 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:55.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb36b7fe640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb36c1527e0 0x7fb36c06a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb36bfff640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb36c0cc950 0x7fb36c151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 -- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb36c1527e0 msgr2=0x7fb36c06a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb36c1527e0 0x7fb36c06a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 -- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb36c0cc950 msgr2=0x7fb36c151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:55.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb36c0cc950 0x7fb36c151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:55.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 -- 172.21.15.71:0/374547850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600094a0 con 0x7fb36c152bc0 2023-10-26T05:48:55.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb370b89640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c145d50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb36400cb70 tx=0x7fb36400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:55.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400aa50 con 0x7fb36c152bc0 2023-10-26T05:48:55.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb36400aca0 con 0x7fb36c152bc0 2023-10-26T05:48:55.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364012680 con 0x7fb36c152bc0 2023-10-26T05:48:55.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c149160 con 0x7fb36c152bc0 2023-10-26T05:48:55.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.977+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c1494f0 con 0x7fb36c152bc0 2023-10-26T05:48:55.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.981+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c066b10 con 0x7fb36c152bc0 2023-10-26T05:48:55.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.981+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb36401a3b0 con 0x7fb36c152bc0 2023-10-26T05:48:55.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.981+0000 7fb368ff9640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb358066470 0x7fb358068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:55.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.981+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb364083d30 con 0x7fb36c152bc0 2023-10-26T05:48:55.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.981+0000 7fb36bfff640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb358066470 0x7fb358068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:55.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.981+0000 7fb36bfff640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb358066470 0x7fb358068930 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb35c007fd0 tx=0x7fb35c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:55.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:55.985+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb3640515f0 con 0x7fb36c152bc0 2023-10-26T05:48:56.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb36c149350 con 0x7fb36c152bc0 2023-10-26T05:48:56.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb368ff9640 1 -- 172.21.15.71:0/374547850 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb36c149350 con 0x7fb36c152bc0 2023-10-26T05:48:56.133 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:48:56.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb358066470 msgr2=0x7fb358068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb358066470 0x7fb358068930 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb35c007fd0 tx=0x7fb35c009040 comp rx=0 tx=0).stop 2023-10-26T05:48:56.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 msgr2=0x7fb36c145d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c145d50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb36400cb70 tx=0x7fb36400cc60 comp rx=0 tx=0).stop 2023-10-26T05:48:56.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 shutdown_connections 2023-10-26T05:48:56.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb358066470 0x7fb358068930 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb36c152bc0 0x7fb36c145d50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb36c1527e0 0x7fb36c06a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 --2- 172.21.15.71:0/374547850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb36c0cc950 0x7fb36c151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 >> 172.21.15.71:0/374547850 conn(0x7fb36c06b980 msgr2=0x7fb36c14f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:56.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 shutdown_connections 2023-10-26T05:48:56.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.129+0000 7fb3725ec640 1 -- 172.21.15.71:0/374547850 wait complete. 2023-10-26T05:48:56.150 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-26T05:48:56.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- 172.21.15.71:0/3379389963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a50003980 msgr2=0x7f2a50003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/3379389963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a50003980 0x7f2a50003da0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2a60005fd0 tx=0x7f2a6001cd40 comp rx=0 tx=0).stop 2023-10-26T05:48:56.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 -- 172.21.15.71:0/3379389963 reap_dead start 2023-10-26T05:48:56.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- 172.21.15.71:0/3379389963 shutdown_connections 2023-10-26T05:48:56.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- 172.21.15.71:0/3379389963 >> 172.21.15.71:0/3379389963 conn(0x7f2a64069770 msgr2=0x7f2a64069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:56.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- 172.21.15.71:0/3379389963 shutdown_connections 2023-10-26T05:48:56.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- 172.21.15.71:0/3379389963 wait complete. 2023-10-26T05:48:56.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 Processor -- start 2023-10-26T05:48:56.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- start start 2023-10-26T05:48:56.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a50003980 0x7f2a641f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a64142b20 0x7f2a64156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 0x7f2a640adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640ae120 con 0x7f2a640ad750 2023-10-26T05:48:56.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641400e0 con 0x7f2a64142b20 2023-10-26T05:48:56.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6c03d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641f70c0 con 0x7f2a50003980 2023-10-26T05:48:56.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 0x7f2a640adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 0x7f2a640adbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36730/0 (socket says 172.21.15.71:36730) 2023-10-26T05:48:56.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 -- 172.21.15.71:0/1129660505 learned_addr learned my addr 172.21.15.71:0/1129660505 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:56.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a69dd9640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a50003980 0x7f2a641f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a695d8640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a64142b20 0x7f2a64156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 -- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a50003980 msgr2=0x7f2a641f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a50003980 0x7f2a641f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 -- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a64142b20 msgr2=0x7f2a64156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a64142b20 0x7f2a64156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.269+0000 7f2a6a5da640 1 -- 172.21.15.71:0/1129660505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a641f6b00 con 0x7f2a640ad750 2023-10-26T05:48:56.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a695d8640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a64142b20 0x7f2a64156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:56.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a6a5da640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 0x7f2a640adbe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c001340 tx=0x7f2a5c00dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:56.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c015030 con 0x7f2a640ad750 2023-10-26T05:48:56.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a5c00b800 con 0x7f2a640ad750 2023-10-26T05:48:56.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a641f6de0 con 0x7f2a640ad750 2023-10-26T05:48:56.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c003ea0 con 0x7f2a640ad750 2023-10-26T05:48:56.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a641405f0 con 0x7f2a640ad750 2023-10-26T05:48:56.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a64066b10 con 0x7f2a640ad750 2023-10-26T05:48:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2a5c018070 con 0x7f2a640ad750 2023-10-26T05:48:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a5a7fc640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a480664a0 0x7f2a48068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a5c016070 con 0x7f2a640ad750 2023-10-26T05:48:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.273+0000 7f2a69dd9640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a480664a0 0x7f2a48068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.277+0000 7f2a69dd9640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a480664a0 0x7f2a48068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b0000 tx=0x7f2a600a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:56.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.277+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2a5c0521b0 con 0x7f2a640ad750 2023-10-26T05:48:56.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.481+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f2a6415a620 con 0x7f2a640ad750 2023-10-26T05:48:56.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.509+0000 7f2a5a7fc640 1 -- 172.21.15.71:0/1129660505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v110) v1 ==== 145+0+0 (secure 0 0 0) 0x7f2a5c023380 con 0x7f2a640ad750 2023-10-26T05:48:56.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a480664a0 msgr2=0x7f2a48068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a480664a0 0x7f2a48068960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2a600b0000 tx=0x7f2a600a7040 comp rx=0 tx=0).stop 2023-10-26T05:48:56.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 msgr2=0x7f2a640adbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 0x7f2a640adbe0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c001340 tx=0x7f2a5c00dd60 comp rx=0 tx=0).stop 2023-10-26T05:48:56.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 shutdown_connections 2023-10-26T05:48:56.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a480664a0 0x7f2a48068960 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a640ad750 0x7f2a640adbe0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a64142b20 0x7f2a64156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 --2- 172.21.15.71:0/1129660505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a50003980 0x7f2a641f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 >> 172.21.15.71:0/1129660505 conn(0x7f2a64069770 msgr2=0x7f2a64159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 shutdown_connections 2023-10-26T05:48:56.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.513+0000 7f2a6c03d640 1 -- 172.21.15.71:0/1129660505 wait complete. 2023-10-26T05:48:56.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T05:48:56.646 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_objectstore":"memstore"} 2023-10-26T05:48:56.647 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T05:48:56.647 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:48:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 -- 172.21.15.71:0/1351120566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc815bbc0 msgr2=0x7f9dc8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1351120566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8070690 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0009c80 tx=0x7f9dc001cd40 comp rx=0 tx=0).stop 2023-10-26T05:48:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 -- 172.21.15.71:0/1351120566 shutdown_connections 2023-10-26T05:48:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1351120566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8070690 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1351120566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 0x7f9dc80ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 -- 172.21.15.71:0/1351120566 >> 172.21.15.71:0/1351120566 conn(0x7f9dc8069770 msgr2=0x7f9dc8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 -- 172.21.15.71:0/1351120566 shutdown_connections 2023-10-26T05:48:56.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 -- 172.21.15.71:0/1351120566 wait complete. 2023-10-26T05:48:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 Processor -- start 2023-10-26T05:48:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.757+0000 7f9dccab2640 1 -- start start 2023-10-26T05:48:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 0x7f9dc81328a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 0x7f9dc813e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8132de0 con 0x7f9dc813d800 2023-10-26T05:48:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc81379d0 con 0x7f9dc815bbc0 2023-10-26T05:48:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8136dc0 con 0x7f9dc813d420 2023-10-26T05:48:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 0x7f9dc813e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 0x7f9dc813e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36746/0 (socket says 172.21.15.71:36746) 2023-10-26T05:48:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 -- 172.21.15.71:0/1806378075 learned_addr learned my addr 172.21.15.71:0/1806378075 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc659c640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 0x7f9dc81328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc6d9d640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 -- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 msgr2=0x7f9dc81328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 0x7f9dc81328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 -- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc815bbc0 msgr2=0x7f9dc8137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 -- 172.21.15.71:0/1806378075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00098e0 con 0x7f9dc813d800 2023-10-26T05:48:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc6d9d640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc659c640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 0x7f9dc81328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dc5d9b640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 0x7f9dc813e0f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c490 tx=0x7f9db8001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8003600 con 0x7f9dc813d800 2023-10-26T05:48:56.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9db8003780 con 0x7f9dc813d800 2023-10-26T05:48:56.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc81370a0 con 0x7f9dc813d800 2023-10-26T05:48:56.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8003b90 con 0x7f9dc813d800 2023-10-26T05:48:56.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc8137660 con 0x7f9dc813d800 2023-10-26T05:48:56.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc8066b10 con 0x7f9dc813d800 2023-10-26T05:48:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9db8003cf0 con 0x7f9dc813d800 2023-10-26T05:48:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dbeffd640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9da80664a0 0x7f9da8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.761+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9db8083390 con 0x7f9dc813d800 2023-10-26T05:48:56.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.765+0000 7f9dc659c640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9da80664a0 0x7f9da8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.765+0000 7f9dc659c640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9da80664a0 0x7f9da8068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9db000a8b0 tx=0x7f9db0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.765+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9db8050a80 con 0x7f9dc813d800 2023-10-26T05:48:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.917+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f9dc80721e0 con 0x7f9dc813d800 2023-10-26T05:48:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.917+0000 7f9dbeffd640 1 -- 172.21.15.71:0/1806378075 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v20) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f9db8017080 con 0x7f9dc813d800 2023-10-26T05:48:56.923 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:48:56.923 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:48:56.923 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:48:56.923 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:48:56.924 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:48:56.924 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:48:56.924 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:48:56.924 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:48:56.924 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:48:56.925 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:48:56.925 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:48:56.925 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:48:56.925 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:48:56.925 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:48:56.926 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:48:56.926 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:48:56.926 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:48:56.926 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:48:56.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9da80664a0 msgr2=0x7f9da8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9da80664a0 0x7f9da8068960 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9db000a8b0 tx=0x7f9db0008040 comp rx=0 tx=0).stop 2023-10-26T05:48:56.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 msgr2=0x7f9dc813e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:56.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 0x7f9dc813e0f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c490 tx=0x7f9db8001e80 comp rx=0 tx=0).stop 2023-10-26T05:48:56.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 shutdown_connections 2023-10-26T05:48:56.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9dc815bbc0 0x7f9dc8137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9da80664a0 0x7f9da8068960 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dc813d800 0x7f9dc813e0f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 --2- 172.21.15.71:0/1806378075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9dc813d420 0x7f9dc81328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:56.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 >> 172.21.15.71:0/1806378075 conn(0x7f9dc8069770 msgr2=0x7f9dc815a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:56.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 shutdown_connections 2023-10-26T05:48:56.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:56.921+0000 7f9dccab2640 1 -- 172.21.15.71:0/1806378075 wait complete. 2023-10-26T05:48:56.940 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:48:57.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/562042004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 msgr2=0x7fb1b815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 --2- 172.21.15.71:0/562042004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b815bab0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009bc0 tx=0x7fb1b001ca80 comp rx=0 tx=0).stop 2023-10-26T05:48:57.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/562042004 shutdown_connections 2023-10-26T05:48:57.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 --2- 172.21.15.71:0/562042004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b815bab0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/562042004 >> 172.21.15.71:0/562042004 conn(0x7fb1b80cdd80 msgr2=0x7fb1b80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:57.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/562042004 shutdown_connections 2023-10-26T05:48:57.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/562042004 wait complete. 2023-10-26T05:48:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 Processor -- start 2023-10-26T05:48:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- start start 2023-10-26T05:48:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b8158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b80d0470 0x7fb1b80c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1b80d0850 0x7fb1b8148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b806a650 con 0x7fb1b80cd880 2023-10-26T05:48:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81593c0 con 0x7fb1b80d0470 2023-10-26T05:48:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80c9bc0 con 0x7fb1b80d0850 2023-10-26T05:48:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b8158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b8158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36768/0 (socket says 172.21.15.71:36768) 2023-10-26T05:48:57.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 -- 172.21.15.71:0/1976199832 learned_addr learned my addr 172.21.15.71:0/1976199832 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:57.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 -- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1b80d0850 msgr2=0x7fb1b8148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1b80d0850 0x7fb1b8148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 -- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b80d0470 msgr2=0x7fb1b80c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1affff640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b80d0470 0x7fb1b80c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b80d0470 0x7fb1b80c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 -- 172.21.15.71:0/1976199832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0009820 con 0x7fb1b80cd880 2023-10-26T05:48:57.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b7fff640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b8158520 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c480 tx=0x7fb1a800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:57.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8012830 con 0x7fb1b80cd880 2023-10-26T05:48:57.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1a8012a80 con 0x7fb1b80cd880 2023-10-26T05:48:57.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b8158b20 con 0x7fb1b80cd880 2023-10-26T05:48:57.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8011380 con 0x7fb1b80cd880 2023-10-26T05:48:57.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.061+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b81590e0 con 0x7fb1b80cd880 2023-10-26T05:48:57.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.065+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b8066b10 con 0x7fb1b80cd880 2023-10-26T05:48:57.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.065+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb1a80114e0 con 0x7fb1b80cd880 2023-10-26T05:48:57.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.065+0000 7fb1b57fa640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb198066470 0x7fb198068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.065+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb1a8083c70 con 0x7fb1b80cd880 2023-10-26T05:48:57.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.065+0000 7fb1affff640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb198066470 0x7fb198068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.065+0000 7fb1affff640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb198066470 0x7fb198068930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb1b81568d0 tx=0x7fb1a0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:57.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.069+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb1a8051480 con 0x7fb1b80cd880 2023-10-26T05:48:57.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.213+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb1b80c9920 con 0x7fb1b80cd880 2023-10-26T05:48:57.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.221+0000 7fb1b57fa640 1 -- 172.21.15.71:0/1976199832 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v21)=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1b80c9920 con 0x7fb1b80cd880 2023-10-26T05:48:57.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb198066470 msgr2=0x7fb198068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb198066470 0x7fb198068930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb1b81568d0 tx=0x7fb1a0008040 comp rx=0 tx=0).stop 2023-10-26T05:48:57.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 msgr2=0x7fb1b8158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b8158520 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c480 tx=0x7fb1a800c940 comp rx=0 tx=0).stop 2023-10-26T05:48:57.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 shutdown_connections 2023-10-26T05:48:57.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1b80d0850 0x7fb1b8148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1b80d0470 0x7fb1b80c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb198066470 0x7fb198068930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 --2- 172.21.15.71:0/1976199832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1b80cd880 0x7fb1b8158520 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 >> 172.21.15.71:0/1976199832 conn(0x7fb1b80cdd80 msgr2=0x7fb1b806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:57.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 shutdown_connections 2023-10-26T05:48:57.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.225+0000 7fb1be62c640 1 -- 172.21.15.71:0/1976199832 wait complete. 2023-10-26T05:48:57.243 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:48:57.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:48:57.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:48:57.348 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:48:57.349 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:57.349 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:57.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:48:57.355 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:57.355 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:57.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:48:57.361 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:48:57.362 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:57.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:48:57.404 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:48:57.404 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:48:57.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:48:57.410 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:48:57.410 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:48:57.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 -- 172.21.15.71:0/2434293660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 msgr2=0x7f92840b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 --2- 172.21.15.71:0/2434293660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f92840b2350 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f92840b32d0 tx=0x7f927401c5e0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 -- 172.21.15.71:0/2434293660 shutdown_connections 2023-10-26T05:48:57.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 --2- 172.21.15.71:0/2434293660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 0x7f92840caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 --2- 172.21.15.71:0/2434293660 >> v2:172.21.15.86:3300/0 conn(0x7f92840cb010 0x7f928406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 --2- 172.21.15.71:0/2434293660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f92840b2350 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 -- 172.21.15.71:0/2434293660 >> 172.21.15.71:0/2434293660 conn(0x7f9284069770 msgr2=0x7f9284069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:57.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 -- 172.21.15.71:0/2434293660 shutdown_connections 2023-10-26T05:48:57.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.561+0000 7f92892c3640 1 -- 172.21.15.71:0/2434293660 wait complete. 2023-10-26T05:48:57.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 Processor -- start 2023-10-26T05:48:57.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- start start 2023-10-26T05:48:57.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 0x7f9284158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92840cb010 0x7f928413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f9284133f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928413ffb0 con 0x7f928413c2f0 2023-10-26T05:48:57.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9284132ae0 con 0x7f92840cb010 2023-10-26T05:48:57.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92841335d0 con 0x7f92840b2890 2023-10-26T05:48:57.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f9284133f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f9284133f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36792/0 (socket says 172.21.15.71:36792) 2023-10-26T05:48:57.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 -- 172.21.15.71:0/1606306476 learned_addr learned my addr 172.21.15.71:0/1606306476 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:57.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f9282ffd640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 0x7f9284158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 -- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 msgr2=0x7f9284158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92827fc640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92840cb010 0x7f928413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 0x7f9284158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 -- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92840cb010 msgr2=0x7f928413fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92840cb010 0x7f928413fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 -- 172.21.15.71:0/1606306476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009560 con 0x7f928413c2f0 2023-10-26T05:48:57.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f9282ffd640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 0x7f9284158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:57.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92827fc640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92840cb010 0x7f928413fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:57.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92837fe640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f9284133f70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b970 tx=0x7f927c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:57.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00cb40 con 0x7f928413c2f0 2023-10-26T05:48:57.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f927c00cdb0 con 0x7f928413c2f0 2023-10-26T05:48:57.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9284132d30 con 0x7f928413c2f0 2023-10-26T05:48:57.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0146a0 con 0x7f928413c2f0 2023-10-26T05:48:57.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92841332f0 con 0x7f928413c2f0 2023-10-26T05:48:57.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.565+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9284066b10 con 0x7f928413c2f0 2023-10-26T05:48:57.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.569+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f927c01d3b0 con 0x7f928413c2f0 2023-10-26T05:48:57.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.569+0000 7f926b7fe640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9270066470 0x7f9270068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.569+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f927c083d90 con 0x7f928413c2f0 2023-10-26T05:48:57.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.569+0000 7f9282ffd640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9270066470 0x7f9270068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.569+0000 7f9282ffd640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9270066470 0x7f9270068930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92740ae000 tx=0x7f927401ae20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:57.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.573+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f927c051510 con 0x7f928413c2f0 2023-10-26T05:48:57.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.741+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92840721e0 con 0x7f928413c2f0 2023-10-26T05:48:57.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f926b7fe640 1 -- 172.21.15.71:0/1606306476 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f927c019020 con 0x7f928413c2f0 2023-10-26T05:48:57.748 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:57.749 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2273351228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2273351228},{"type":"v1","addr":"172.21.15.86:6836","nonce":2273351228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4864,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1841130724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1841130724},{"type":"v1","addr":"172.21.15.71:6833","nonce":1841130724}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":4879,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/445173848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":445173848},{"type":"v1","addr":"172.21.15.86:6837","nonce":445173848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5697,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2592963661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2592963661},{"type":"v1","addr":"172.21.15.71:6835","nonce":2592963661}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":14}],"filesystems":[]} 2023-10-26T05:48:57.749 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 14 2023-10-26T05:48:57.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9270066470 msgr2=0x7f9270068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9270066470 0x7f9270068930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f92740ae000 tx=0x7f927401ae20 comp rx=0 tx=0).stop 2023-10-26T05:48:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 msgr2=0x7f9284133f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f9284133f70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f927c00b970 tx=0x7f927c00be30 comp rx=0 tx=0).stop 2023-10-26T05:48:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 shutdown_connections 2023-10-26T05:48:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9270066470 0x7f9270068930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928413c2f0 0x7f9284133f70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92840cb010 0x7f928413fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 --2- 172.21.15.71:0/1606306476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92840b2890 0x7f9284158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 >> 172.21.15.71:0/1606306476 conn(0x7f9284069770 msgr2=0x7f92840ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 shutdown_connections 2023-10-26T05:48:57.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.745+0000 7f92892c3640 1 -- 172.21.15.71:0/1606306476 wait complete. 2023-10-26T05:48:57.766 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:48:57.766 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:48:57.767 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:48:57.767 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:57.767 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:48:57.768 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:48:57.768 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:48:57.768 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:57.769 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:48:57.769 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:48:57.769 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:48:57.769 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:57.770 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:48:57.770 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:48:57.770 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:48:57.771 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:48:57.776 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:48:57.777 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:48:57.779 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:48:57.780 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:48:57.789 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:48:57.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:48:57.790 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:48:57.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:48:57.828 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:48:57.829 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:57.830 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:48:57.830 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:57.836 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:48:57.836 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:57.837 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:48:57.837 DEBUG:teuthology.parallel:result is None 2023-10-26T05:48:57.838 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:48:57.884 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:48:57.886 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:48:57.908 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:48:57.909 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:48:57.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- 172.21.15.71:0/4274531735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 msgr2=0x7f25d80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 --2- 172.21.15.71:0/4274531735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d80ccbf0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f25d8072e20 tx=0x7f25d001f510 comp rx=0 tx=0).stop 2023-10-26T05:48:57.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- 172.21.15.71:0/4274531735 shutdown_connections 2023-10-26T05:48:57.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 --2- 172.21.15.71:0/4274531735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d80ccbf0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 --2- 172.21.15.71:0/4274531735 >> v2:172.21.15.86:3300/0 conn(0x7f25d806a390 0x7f25d806a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- 172.21.15.71:0/4274531735 >> 172.21.15.71:0/4274531735 conn(0x7f25d80d12d0 msgr2=0x7f25d80d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:57.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- 172.21.15.71:0/4274531735 shutdown_connections 2023-10-26T05:48:57.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- 172.21.15.71:0/4274531735 wait complete. 2023-10-26T05:48:57.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 Processor -- start 2023-10-26T05:48:57.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- start start 2023-10-26T05:48:57.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d806a390 0x7f25d81579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d8154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d815b3b0 0x7f25d81f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d815c310 con 0x7f25d80cc7c0 2023-10-26T05:48:57.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d815bd50 con 0x7f25d806a390 2023-10-26T05:48:57.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25df221640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d815c030 con 0x7f25d815b3b0 2023-10-26T05:48:57.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d8154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d8154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36824/0 (socket says 172.21.15.71:36824) 2023-10-26T05:48:57.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 -- 172.21.15.71:0/3940950546 learned_addr learned my addr 172.21.15.71:0/3940950546 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:57.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25dcfbd640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d806a390 0x7f25d81579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25dd7be640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d815b3b0 0x7f25d81f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 -- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d815b3b0 msgr2=0x7f25d81f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d815b3b0 0x7f25d81f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 -- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d806a390 msgr2=0x7f25d81579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:57.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d806a390 0x7f25d81579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:57.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 -- 172.21.15.71:0/3940950546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00098e0 con 0x7f25d80cc7c0 2023-10-26T05:48:57.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.949+0000 7f25d7fff640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d8154a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00c970 tx=0x7f25cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:57.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc007c70 con 0x7f25d80cc7c0 2023-10-26T05:48:57.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f25cc00f450 con 0x7f25d80cc7c0 2023-10-26T05:48:57.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d81571a0 con 0x7f25d80cc7c0 2023-10-26T05:48:57.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00f860 con 0x7f25d80cc7c0 2023-10-26T05:48:57.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d8154fb0 con 0x7f25d80cc7c0 2023-10-26T05:48:57.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d8066b10 con 0x7f25d80cc7c0 2023-10-26T05:48:57.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f25cc00f9c0 con 0x7f25d80cc7c0 2023-10-26T05:48:57.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25d57fa640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25c4066470 0x7f25c4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:57.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f25cc0842e0 con 0x7f25d80cc7c0 2023-10-26T05:48:57.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25dcfbd640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25c4066470 0x7f25c4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:57.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.953+0000 7f25dcfbd640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25c4066470 0x7f25c4068930 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f25c80098e0 tx=0x7f25c8009060 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:57.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:57.957+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f25cc051b10 con 0x7f25d80cc7c0 2023-10-26T05:48:58.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:58.121+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f25d806c350 con 0x7f25d80cc7c0 2023-10-26T05:48:59.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.249+0000 7f25d57fa640 1 -- 172.21.15.71:0/3940950546 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7f25cc055130 con 0x7f25d80cc7c0 2023-10-26T05:48:59.254 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:48:59.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25c4066470 msgr2=0x7f25c4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25c4066470 0x7f25c4068930 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f25c80098e0 tx=0x7f25c8009060 comp rx=0 tx=0).stop 2023-10-26T05:48:59.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 msgr2=0x7f25d8154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d8154a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00c970 tx=0x7f25cc00ce30 comp rx=0 tx=0).stop 2023-10-26T05:48:59.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 shutdown_connections 2023-10-26T05:48:59.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d815b3b0 0x7f25d81f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25c4066470 0x7f25c4068930 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25d80cc7c0 0x7f25d8154a70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 --2- 172.21.15.71:0/3940950546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d806a390 0x7f25d81579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 >> 172.21.15.71:0/3940950546 conn(0x7f25d80d12d0 msgr2=0x7f25d80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:59.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 shutdown_connections 2023-10-26T05:48:59.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.253+0000 7f25df221640 1 -- 172.21.15.71:0/3940950546 wait complete. 2023-10-26T05:48:59.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:48:59.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/2460833053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93880b0040 msgr2=0x7f93880b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- 172.21.15.71:0/2460833053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93880b0040 0x7f93880b04b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f93880afd40 tx=0x7f938001cd30 comp rx=0 tx=0).stop 2023-10-26T05:48:59.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/2460833053 shutdown_connections 2023-10-26T05:48:59.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- 172.21.15.71:0/2460833053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93880b0040 0x7f93880b04b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- 172.21.15.71:0/2460833053 >> v2:172.21.15.86:3300/0 conn(0x7f9388070fe0 0x7f9388071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- 172.21.15.71:0/2460833053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9388140cc0 0x7f938815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/2460833053 >> 172.21.15.71:0/2460833053 conn(0x7f9388069770 msgr2=0x7f9388069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:59.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/2460833053 shutdown_connections 2023-10-26T05:48:59.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/2460833053 wait complete. 2023-10-26T05:48:59.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 Processor -- start 2023-10-26T05:48:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- start start 2023-10-26T05:48:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9388070fe0 0x7f93881550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93880b0040 0x7f938813b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 0x7f93881535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388155610 con 0x7f9388140cc0 2023-10-26T05:48:59.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388154c40 con 0x7f93880b0040 2023-10-26T05:48:59.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9388154030 con 0x7f9388070fe0 2023-10-26T05:48:59.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 0x7f93881535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 0x7f93881535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36842/0 (socket says 172.21.15.71:36842) 2023-10-26T05:48:59.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 -- 172.21.15.71:0/1269195102 learned_addr learned my addr 172.21.15.71:0/1269195102 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:59.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93867fc640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93880b0040 0x7f938813b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f9386ffd640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9388070fe0 0x7f93881550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 -- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9388070fe0 msgr2=0x7f93881550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9388070fe0 0x7f93881550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 -- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93880b0040 msgr2=0x7f938813b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93880b0040 0x7f938813b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 -- 172.21.15.71:0/1269195102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93800098e0 con 0x7f9388140cc0 2023-10-26T05:48:59.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f93877fe640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 0x7f93881535e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9380009bd0 tx=0x7f93800a5c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:59.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380004e60 con 0x7f9388140cc0 2023-10-26T05:48:59.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93800ad5b0 con 0x7f9388140cc0 2023-10-26T05:48:59.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93800ad980 con 0x7f9388140cc0 2023-10-26T05:48:59.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9388154310 con 0x7f9388140cc0 2023-10-26T05:48:59.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.405+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881548d0 con 0x7f9388140cc0 2023-10-26T05:48:59.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.409+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9388066b10 con 0x7f9388140cc0 2023-10-26T05:48:59.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.409+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f93800b4040 con 0x7f9388140cc0 2023-10-26T05:48:59.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.409+0000 7f936ffff640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f93740664a0 0x7f9374068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.409+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f938011dd10 con 0x7f9388140cc0 2023-10-26T05:48:59.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.409+0000 7f9386ffd640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f93740664a0 0x7f9374068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.409+0000 7f9386ffd640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f93740664a0 0x7f9374068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9378005fd0 tx=0x7f937800c460 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:59.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.413+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f93800eb620 con 0x7f9388140cc0 2023-10-26T05:48:59.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.621+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9388154dd0 con 0x7f9388140cc0 2023-10-26T05:48:59.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.621+0000 7f936ffff640 1 -- 172.21.15.71:0/1269195102 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f93800b2020 con 0x7f9388140cc0 2023-10-26T05:48:59.626 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:59.626 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:48:59.626 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:48:59.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.627 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:48:59.627 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:48:59.627 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.627 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:59.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:59.628 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.629 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:48:59.629 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:48:59.629 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.629 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:59.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:59.630 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.631 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:48:59.631 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:48:59.631 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.631 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.632 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:59.632 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:59.632 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.633 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:48:59.633 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:48:59.633 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.633 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.634 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:59.634 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:59.634 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.635 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:48:59.635 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:48:59.635 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.636 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.636 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.636 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.637 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:48:59.637 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:48:59.637 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.637 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.638 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.638 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.639 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:48:59.639 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:48:59.639 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.639 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.639 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.640 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.641 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:48:59.641 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:48:59.641 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.641 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.641 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.642 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.642 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:48:59.643 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:48:59.643 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.643 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.643 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.643 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.644 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.644 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:48:59.644 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:48:59.645 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.645 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.645 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.645 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.645 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.646 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:48:59.646 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:48:59.646 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.647 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.647 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.648 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:48:59.648 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:48:59.648 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.648 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:48:59.649 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:48:59.649 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.650 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:48:59.650 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:48:59.650 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.650 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.650 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:48:59.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:48:59.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.651 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:48:59.652 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:48:59.652 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:59.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:59.653 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:59.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.653 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:48:59.654 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:48:59.654 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.654 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.654 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:59.654 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:59.655 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:59.655 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.655 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:48:59.656 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:48:59.656 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:48:59.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:48:59.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:48:59.656 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:48:59.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.657 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:48:59.657 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:48:59.658 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:48:59.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:48:59.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:48:59.659 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.659 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:48:59.660 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:48:59.660 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:48:59.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:48:59.660 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:48:59.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:48:59.661 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:48:59.661 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:48:59.662 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:48:59.662 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:48:59.662 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:48:59.662 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:48:59.662 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:48:59.663 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:48:59.663 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:48:59.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f93740664a0 msgr2=0x7f9374068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f93740664a0 0x7f9374068960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9378005fd0 tx=0x7f937800c460 comp rx=0 tx=0).stop 2023-10-26T05:48:59.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 msgr2=0x7f93881535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 0x7f93881535e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9380009bd0 tx=0x7f93800a5c20 comp rx=0 tx=0).stop 2023-10-26T05:48:59.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 shutdown_connections 2023-10-26T05:48:59.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f93740664a0 0x7f9374068960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9388140cc0 0x7f93881535e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93880b0040 0x7f938813b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 --2- 172.21.15.71:0/1269195102 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9388070fe0 0x7f93881550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 >> 172.21.15.71:0/1269195102 conn(0x7f9388069770 msgr2=0x7f9388069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:59.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 shutdown_connections 2023-10-26T05:48:59.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.625+0000 7f938d8f0640 1 -- 172.21.15.71:0/1269195102 wait complete. 2023-10-26T05:48:59.673 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:48:59.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 -- 172.21.15.71:0/20191528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28a80047d0 msgr2=0x7f28a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 --2- 172.21.15.71:0/20191528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28a80047d0 0x7f28a8004c50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f28b4009b30 tx=0x7f28b401d200 comp rx=0 tx=0).stop 2023-10-26T05:48:59.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 -- 172.21.15.71:0/20191528 shutdown_connections 2023-10-26T05:48:59.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 --2- 172.21.15.71:0/20191528 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28a80047d0 0x7f28a8004c50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 -- 172.21.15.71:0/20191528 >> 172.21.15.71:0/20191528 conn(0x7f28c00cdd80 msgr2=0x7f28c00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:59.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 -- 172.21.15.71:0/20191528 shutdown_connections 2023-10-26T05:48:59.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 -- 172.21.15.71:0/20191528 wait complete. 2023-10-26T05:48:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 Processor -- start 2023-10-26T05:48:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.797+0000 7f28c689a640 1 -- start start 2023-10-26T05:48:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28a80047d0 0x7f28c01f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 0x7f28c00cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28c0152520 0x7f28c01529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c006a650 con 0x7f28c00cb930 2023-10-26T05:48:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c0153ed0 con 0x7f28a80047d0 2023-10-26T05:48:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c01f8670 con 0x7f28c0152520 2023-10-26T05:48:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 0x7f28c00cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 0x7f28c00cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36854/0 (socket says 172.21.15.71:36854) 2023-10-26T05:48:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 -- 172.21.15.71:0/996425581 learned_addr learned my addr 172.21.15.71:0/996425581 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:48:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bffff640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28a80047d0 0x7f28c01f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 -- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28c0152520 msgr2=0x7f28c01529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c4e37640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28c0152520 0x7f28c01529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28c0152520 0x7f28c01529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 -- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28a80047d0 msgr2=0x7f28c01f8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28a80047d0 0x7f28c01f8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 -- 172.21.15.71:0/996425581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c0153250 con 0x7f28c00cb930 2023-10-26T05:48:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bffff640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28a80047d0 0x7f28c01f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:48:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bf7fe640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 0x7f28c00cbdb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28b401d030 tx=0x7f28b40056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4004a10 con 0x7f28c00cb930 2023-10-26T05:48:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f28b4004c60 con 0x7f28c00cb930 2023-10-26T05:48:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c0153530 con 0x7f28c00cb930 2023-10-26T05:48:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b40b6410 con 0x7f28c00cb930 2023-10-26T05:48:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c0153af0 con 0x7f28c00cb930 2023-10-26T05:48:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c0066b10 con 0x7f28c00cb930 2023-10-26T05:48:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f28b40b3070 con 0x7f28c00cb930 2023-10-26T05:48:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bcff9640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f28ac066470 0x7f28ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:48:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f28b411ecc0 con 0x7f28c00cb930 2023-10-26T05:48:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.801+0000 7f28bffff640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f28ac066470 0x7f28ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:48:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.805+0000 7f28bffff640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f28ac066470 0x7f28ac068930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0006a90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:48:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.805+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f28b4125080 con 0x7f28c00cb930 2023-10-26T05:48:59.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.957+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28c00cc2f0 con 0x7f28c00cb930 2023-10-26T05:48:59.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28bcff9640 1 -- 172.21.15.71:0/996425581 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f28b40ec4b0 con 0x7f28c00cb930 2023-10-26T05:48:59.964 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:48:59.964 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:48:59.249258+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:48:59.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f28ac066470 msgr2=0x7f28ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f28ac066470 0x7f28ac068930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0006a90 comp rx=0 tx=0).stop 2023-10-26T05:48:59.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 msgr2=0x7f28c00cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:48:59.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 0x7f28c00cbdb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28b401d030 tx=0x7f28b40056c0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 shutdown_connections 2023-10-26T05:48:59.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f28c0152520 0x7f28c01529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f28ac066470 0x7f28ac068930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28c00cb930 0x7f28c00cbdb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 --2- 172.21.15.71:0/996425581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28a80047d0 0x7f28c01f8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:48:59.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 >> 172.21.15.71:0/996425581 conn(0x7f28c00cdd80 msgr2=0x7f28c015a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:48:59.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 shutdown_connections 2023-10-26T05:48:59.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:48:59.961+0000 7f28c689a640 1 -- 172.21.15.71:0/996425581 wait complete. 2023-10-26T05:48:59.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:48:59.983 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:49:00.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/912554248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 msgr2=0x7f22480b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 --2- 172.21.15.71:0/912554248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f22480b0fa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2238009900 tx=0x7f223801c890 comp rx=0 tx=0).stop 2023-10-26T05:49:00.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/912554248 shutdown_connections 2023-10-26T05:49:00.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 --2- 172.21.15.71:0/912554248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f224815bf70 0x7f22480ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 --2- 172.21.15.71:0/912554248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f22480b0fa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/912554248 >> 172.21.15.71:0/912554248 conn(0x7f22480aec60 msgr2=0x7f22480af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/912554248 shutdown_connections 2023-10-26T05:49:00.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/912554248 wait complete. 2023-10-26T05:49:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 Processor -- start 2023-10-26T05:49:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- start start 2023-10-26T05:49:00.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2248070dc0 0x7f2248151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f2248137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224815bf70 0x7f224815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248137b60 con 0x7f224815bb90 2023-10-26T05:49:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248151a40 con 0x7f224815bf70 2023-10-26T05:49:00.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22481384a0 con 0x7f2248070dc0 2023-10-26T05:49:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f2248137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f2248137620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36870/0 (socket says 172.21.15.71:36870) 2023-10-26T05:49:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 -- 172.21.15.71:0/577303126 learned_addr learned my addr 172.21.15.71:0/577303126 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f2246ffd640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2248070dc0 0x7f2248151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 -- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2248070dc0 msgr2=0x7f2248151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f22477fe640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224815bf70 0x7f224815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2248070dc0 0x7f2248151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 -- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224815bf70 msgr2=0x7f224815a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224815bf70 0x7f224815a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 -- 172.21.15.71:0/577303126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009560 con 0x7f224815bb90 2023-10-26T05:49:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f22477fe640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224815bf70 0x7f224815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223e7fc640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f2248137620 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f223000c780 tx=0x7f223000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:00.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230005270 con 0x7f224815bb90 2023-10-26T05:49:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f22300054c0 con 0x7f224815bb90 2023-10-26T05:49:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22481516b0 con 0x7f224815bb90 2023-10-26T05:49:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000a880 con 0x7f224815bb90 2023-10-26T05:49:00.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.097+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248151f70 con 0x7f224815bb90 2023-10-26T05:49:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.101+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248066b10 con 0x7f224815bb90 2023-10-26T05:49:00.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.101+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f223000a9e0 con 0x7f224815bb90 2023-10-26T05:49:00.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.101+0000 7f223ffff640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f22280664a0 0x7f2228068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.101+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f22300830c0 con 0x7f224815bb90 2023-10-26T05:49:00.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.101+0000 7f2246ffd640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f22280664a0 0x7f2228068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.101+0000 7f2246ffd640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f22280664a0 0x7f2228068960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22380af000 tx=0x7f223801b0d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.105+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2230050920 con 0x7f224815bb90 2023-10-26T05:49:00.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.261+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2248071220 con 0x7f224815bb90 2023-10-26T05:49:00.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.261+0000 7f223ffff640 1 -- 172.21.15.71:0/577303126 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2248071220 con 0x7f224815bb90 2023-10-26T05:49:00.265 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:49:00.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f22280664a0 msgr2=0x7f2228068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f22280664a0 0x7f2228068960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f22380af000 tx=0x7f223801b0d0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 msgr2=0x7f2248137620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f2248137620 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f223000c780 tx=0x7f223000cc40 comp rx=0 tx=0).stop 2023-10-26T05:49:00.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 shutdown_connections 2023-10-26T05:49:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224815bf70 0x7f224815a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f22280664a0 0x7f2228068960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f224815bb90 0x7f2248137620 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 --2- 172.21.15.71:0/577303126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2248070dc0 0x7f2248151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 >> 172.21.15.71:0/577303126 conn(0x7f22480aec60 msgr2=0x7f22480cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:00.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 shutdown_connections 2023-10-26T05:49:00.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.265+0000 7f224d748640 1 -- 172.21.15.71:0/577303126 wait complete. 2023-10-26T05:49:00.284 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:49:00.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/603873636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 msgr2=0x7ff0f8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 --2- 172.21.15.71:0/603873636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff0f8004850 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c290 tx=0x7ff0f401f760 comp rx=0 tx=0).stop 2023-10-26T05:49:00.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10bfff640 1 -- 172.21.15.71:0/603873636 reap_dead start 2023-10-26T05:49:00.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/603873636 shutdown_connections 2023-10-26T05:49:00.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/603873636 >> 172.21.15.71:0/603873636 conn(0x7ff10c069770 msgr2=0x7ff10c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:00.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/603873636 shutdown_connections 2023-10-26T05:49:00.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/603873636 wait complete. 2023-10-26T05:49:00.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 Processor -- start 2023-10-26T05:49:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- start start 2023-10-26T05:49:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff10c136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff10c13f3b0 0x7ff10c14fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c13dce0 0x7ff10c13e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c13b1f0 con 0x7ff0f8004370 2023-10-26T05:49:00.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c13b4d0 con 0x7ff10c13dce0 2023-10-26T05:49:00.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c13ebb0 con 0x7ff10c13f3b0 2023-10-26T05:49:00.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff10c136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff10c136ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36896/0 (socket says 172.21.15.71:36896) 2023-10-26T05:49:00.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 -- 172.21.15.71:0/860328512 learned_addr learned my addr 172.21.15.71:0/860328512 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:00.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10bfff640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c13dce0 0x7ff10c13e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10affd640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff10c13f3b0 0x7ff10c14fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 -- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff10c13f3b0 msgr2=0x7ff10c14fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff10c13f3b0 0x7ff10c14fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 -- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c13dce0 msgr2=0x7ff10c13e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c13dce0 0x7ff10c13e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 -- 172.21.15.71:0/860328512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c13ee90 con 0x7ff0f8004370 2023-10-26T05:49:00.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff10b7fe640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff10c136ab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00d6d0 tx=0x7ff0fc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:00.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004230 con 0x7ff0f8004370 2023-10-26T05:49:00.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff0fc0044a0 con 0x7ff0f8004370 2023-10-26T05:49:00.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10c13adb0 con 0x7ff0f8004370 2023-10-26T05:49:00.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc009a80 con 0x7ff0f8004370 2023-10-26T05:49:00.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.409+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff10c12cda0 con 0x7ff0f8004370 2023-10-26T05:49:00.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.413+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10c066b10 con 0x7ff0f8004370 2023-10-26T05:49:00.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.413+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff0fc009be0 con 0x7ff0f8004370 2023-10-26T05:49:00.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.413+0000 7ff103fff640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0ec0664a0 0x7ff0ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.413+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff0fc083c50 con 0x7ff0f8004370 2023-10-26T05:49:00.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.413+0000 7ff10affd640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0ec0664a0 0x7ff0ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.413+0000 7ff10affd640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0ec0664a0 0x7ff0ec068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40b0000 tx=0x7ff0f401c500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:00.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.417+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0fc089020 con 0x7ff0f8004370 2023-10-26T05:49:00.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.561+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff10c13afa0 con 0x7ff0f8004370 2023-10-26T05:49:00.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff103fff640 1 -- 172.21.15.71:0/860328512 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff0fc051420 con 0x7ff0f8004370 2023-10-26T05:49:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:00.568 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:49:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:49:00.568 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:49:00.569 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:00.569 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:49:00.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0ec0664a0 msgr2=0x7ff0ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0ec0664a0 0x7ff0ec068960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40b0000 tx=0x7ff0f401c500 comp rx=0 tx=0).stop 2023-10-26T05:49:00.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 msgr2=0x7ff10c136ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff10c136ab0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00d6d0 tx=0x7ff0fc00db90 comp rx=0 tx=0).stop 2023-10-26T05:49:00.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 shutdown_connections 2023-10-26T05:49:00.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c13dce0 0x7ff10c13e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff10c13f3b0 0x7ff10c14fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0ec0664a0 0x7ff0ec068960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 --2- 172.21.15.71:0/860328512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0f8004370 0x7ff10c136ab0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 >> 172.21.15.71:0/860328512 conn(0x7ff10c069770 msgr2=0x7ff10c1376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:00.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 shutdown_connections 2023-10-26T05:49:00.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.565+0000 7ff1121d6640 1 -- 172.21.15.71:0/860328512 wait complete. 2023-10-26T05:49:00.585 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:49:00.585 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:49:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- 172.21.15.71:0/801826041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425006a7e0 msgr2=0x7f425006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 --2- 172.21.15.71:0/801826041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425006a7e0 0x7f425006ac90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f42500c4ec0 tx=0x7f424401c7b0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- 172.21.15.71:0/801826041 shutdown_connections 2023-10-26T05:49:00.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 --2- 172.21.15.71:0/801826041 >> v2:172.21.15.86:3300/0 conn(0x7f42500cf2f0 0x7f42500cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 --2- 172.21.15.71:0/801826041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425006a7e0 0x7f425006ac90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- 172.21.15.71:0/801826041 >> 172.21.15.71:0/801826041 conn(0x7f42500d13f0 msgr2=0x7f42500d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- 172.21.15.71:0/801826041 shutdown_connections 2023-10-26T05:49:00.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- 172.21.15.71:0/801826041 wait complete. 2023-10-26T05:49:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 Processor -- start 2023-10-26T05:49:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- start start 2023-10-26T05:49:00.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425006a7e0 0x7f42500ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42500cf2f0 0x7f4250158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 0x7f425014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501528c0 con 0x7f425015b4d0 2023-10-26T05:49:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42500cab80 con 0x7f425006a7e0 2023-10-26T05:49:00.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4256990640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42500cae20 con 0x7f42500cf2f0 2023-10-26T05:49:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.709+0000 7f4254f2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 0x7f425014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 0x7f425014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36920/0 (socket says 172.21.15.71:36920) 2023-10-26T05:49:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 -- 172.21.15.71:0/3693871796 learned_addr learned my addr 172.21.15.71:0/3693871796 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424ffff640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425006a7e0 0x7f42500ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424f7fe640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42500cf2f0 0x7f4250158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 -- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42500cf2f0 msgr2=0x7f4250158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42500cf2f0 0x7f4250158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 -- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425006a7e0 msgr2=0x7f42500ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425006a7e0 0x7f42500ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 -- 172.21.15.71:0/3693871796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440094a0 con 0x7f425015b4d0 2023-10-26T05:49:00.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424ffff640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425006a7e0 0x7f42500ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4254f2d640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 0x7f425014e760 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f424800ccb0 tx=0x7f424800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248004b30 con 0x7f425015b4d0 2023-10-26T05:49:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4248004da0 con 0x7f425015b4d0 2023-10-26T05:49:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42500cb100 con 0x7f425015b4d0 2023-10-26T05:49:00.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248005280 con 0x7f425015b4d0 2023-10-26T05:49:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4250152db0 con 0x7f425015b4d0 2023-10-26T05:49:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4250066b10 con 0x7f425015b4d0 2023-10-26T05:49:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f42480053e0 con 0x7f425015b4d0 2023-10-26T05:49:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424cff9640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c066470 0x7f423c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:00.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42480843c0 con 0x7f425015b4d0 2023-10-26T05:49:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424ffff640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c066470 0x7f423c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.713+0000 7f424ffff640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c066470 0x7f423c068930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4240001650 tx=0x7f4240009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:00.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.717+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4248051c20 con 0x7f425015b4d0 2023-10-26T05:49:00.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:00.873+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42500721e0 con 0x7f425015b4d0 2023-10-26T05:49:01.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.549+0000 7f424cff9640 1 -- 172.21.15.71:0/3693871796 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7f424801d020 con 0x7f425015b4d0 2023-10-26T05:49:01.554 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:49:01.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c066470 msgr2=0x7f423c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:01.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c066470 0x7f423c068930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4240001650 tx=0x7f4240009040 comp rx=0 tx=0).stop 2023-10-26T05:49:01.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 msgr2=0x7f425014e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:01.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 0x7f425014e760 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f424800ccb0 tx=0x7f424800a620 comp rx=0 tx=0).stop 2023-10-26T05:49:01.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 shutdown_connections 2023-10-26T05:49:01.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c066470 0x7f423c068930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f425015b4d0 0x7f425014e760 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42500cf2f0 0x7f4250158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 --2- 172.21.15.71:0/3693871796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425006a7e0 0x7f42500ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 >> 172.21.15.71:0/3693871796 conn(0x7f42500d13f0 msgr2=0x7f425015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:01.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 shutdown_connections 2023-10-26T05:49:01.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.553+0000 7f4256990640 1 -- 172.21.15.71:0/3693871796 wait complete. 2023-10-26T05:49:01.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:49:01.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- 172.21.15.71:0/412743328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 msgr2=0x7f398c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:01.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 --2- 172.21.15.71:0/412743328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c0b1ff0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f398c0cfc30 tx=0x7f398401c7e0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- 172.21.15.71:0/412743328 shutdown_connections 2023-10-26T05:49:01.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 --2- 172.21.15.71:0/412743328 >> v2:172.21.15.86:3300/0 conn(0x7f398c0cc220 0x7f398c0cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 --2- 172.21.15.71:0/412743328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c0b1ff0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- 172.21.15.71:0/412743328 >> 172.21.15.71:0/412743328 conn(0x7f398c06f440 msgr2=0x7f398c06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:01.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- 172.21.15.71:0/412743328 shutdown_connections 2023-10-26T05:49:01.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- 172.21.15.71:0/412743328 wait complete. 2023-10-26T05:49:01.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 Processor -- start 2023-10-26T05:49:01.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- start start 2023-10-26T05:49:01.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c0af990 0x7f398c138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:01.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0cc220 0x7f398c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:01.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c13da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:01.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c15c1d0 con 0x7f398c142b30 2023-10-26T05:49:01.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c06bb20 con 0x7f398c0af990 2023-10-26T05:49:01.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f39926d8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c06c980 con 0x7f398c0cc220 2023-10-26T05:49:01.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c13da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:01.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c13da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52638/0 (socket says 172.21.15.71:52638) 2023-10-26T05:49:01.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 -- 172.21.15.71:0/1773992513 learned_addr learned my addr 172.21.15.71:0/1773992513 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:01.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f398bfff640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c0af990 0x7f398c138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:01.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f398b7fe640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0cc220 0x7f398c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:01.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 -- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0cc220 msgr2=0x7f398c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:01.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0cc220 0x7f398c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 -- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c0af990 msgr2=0x7f398c138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:01.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c0af990 0x7f398c138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:01.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.713+0000 7f3990c75640 1 -- 172.21.15.71:0/1773992513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984009560 con 0x7f398c142b30 2023-10-26T05:49:01.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3990c75640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c13da40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f397c00c970 tx=0x7f397c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:01.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c007bb0 con 0x7f398c142b30 2023-10-26T05:49:01.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f397c007e20 con 0x7f398c142b30 2023-10-26T05:49:01.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c06bd70 con 0x7f398c142b30 2023-10-26T05:49:01.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00f6b0 con 0x7f398c142b30 2023-10-26T05:49:01.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c06c330 con 0x7f398c142b30 2023-10-26T05:49:01.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c066b10 con 0x7f398c142b30 2023-10-26T05:49:01.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f397c00f810 con 0x7f398c142b30 2023-10-26T05:49:01.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3988ff9640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3978066470 0x7f3978068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:01.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f397c0848b0 con 0x7f398c142b30 2023-10-26T05:49:01.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f398bfff640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3978066470 0x7f3978068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:01.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.717+0000 7f398bfff640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3978066470 0x7f3978068930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f39840af000 tx=0x7f398401b020 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:01.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.721+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f397c051f00 con 0x7f398c142b30 2023-10-26T05:49:01.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:01.865+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f398c06caf0 con 0x7f398c142b30 2023-10-26T05:49:02.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.553+0000 7f3988ff9640 1 -- 172.21.15.71:0/1773992513 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7f397c055520 con 0x7f398c142b30 2023-10-26T05:49:02.559 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:49:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3978066470 msgr2=0x7f3978068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3978066470 0x7f3978068930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f39840af000 tx=0x7f398401b020 comp rx=0 tx=0).stop 2023-10-26T05:49:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 msgr2=0x7f398c13da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:02.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c13da40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f397c00c970 tx=0x7f397c00ce30 comp rx=0 tx=0).stop 2023-10-26T05:49:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 shutdown_connections 2023-10-26T05:49:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3978066470 0x7f3978068930 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f398c142b30 0x7f398c13da40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f398c0cc220 0x7f398c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:02.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 --2- 172.21.15.71:0/1773992513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f398c0af990 0x7f398c138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:02.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 >> 172.21.15.71:0/1773992513 conn(0x7f398c06f440 msgr2=0x7f398c15c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:02.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 shutdown_connections 2023-10-26T05:49:02.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.557+0000 7f39926d8640 1 -- 172.21.15.71:0/1773992513 wait complete. 2023-10-26T05:49:02.590 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:49:02.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- 172.21.15.71:0/1029399767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 msgr2=0x7f2dbc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/1029399767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dbc003da0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009b60 tx=0x7f2dc001d260 comp rx=0 tx=0).stop 2023-10-26T05:49:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd62e2640 1 -- 172.21.15.71:0/1029399767 reap_dead start 2023-10-26T05:49:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- 172.21.15.71:0/1029399767 shutdown_connections 2023-10-26T05:49:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- 172.21.15.71:0/1029399767 >> 172.21.15.71:0/1029399767 conn(0x7f2dd00aec60 msgr2=0x7f2dd00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- 172.21.15.71:0/1029399767 shutdown_connections 2023-10-26T05:49:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- 172.21.15.71:0/1029399767 wait complete. 2023-10-26T05:49:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 Processor -- start 2023-10-26T05:49:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- start start 2023-10-26T05:49:02.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dd01596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:02.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd01365f0 0x7f2dd0135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:02.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd013d5c0 0x7f2dd013da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:02.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd013df80 con 0x7f2dbc003980 2023-10-26T05:49:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0140a50 con 0x7f2dd013d5c0 2023-10-26T05:49:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd7d45640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd015a070 con 0x7f2dd01365f0 2023-10-26T05:49:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.713+0000 7f2dd5ae1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dd01596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dd01596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52670/0 (socket says 172.21.15.71:52670) 2023-10-26T05:49:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 -- 172.21.15.71:0/2314762329 learned_addr learned my addr 172.21.15.71:0/2314762329 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd52e0640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd01365f0 0x7f2dd0135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd62e2640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd013d5c0 0x7f2dd013da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 -- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd01365f0 msgr2=0x7f2dd0135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd01365f0 0x7f2dd0135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 -- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd013d5c0 msgr2=0x7f2dd013da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:02.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd013d5c0 0x7f2dd013da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:02.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 -- 172.21.15.71:0/2314762329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0159c30 con 0x7f2dbc003980 2023-10-26T05:49:02.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd5ae1640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dd01596f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009a80 tx=0x7f2dc0005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:02.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004a10 con 0x7f2dbc003980 2023-10-26T05:49:02.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2dc0004c60 con 0x7f2dbc003980 2023-10-26T05:49:02.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd015a330 con 0x7f2dbc003980 2023-10-26T05:49:02.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00b6410 con 0x7f2dbc003980 2023-10-26T05:49:02.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd0140ca0 con 0x7f2dbc003980 2023-10-26T05:49:02.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd0066b10 con 0x7f2dbc003980 2023-10-26T05:49:02.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2dc00b3070 con 0x7f2dbc003980 2023-10-26T05:49:02.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dc67fc640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2db4066470 0x7f2db4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:02.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2dc011f3a0 con 0x7f2dbc003980 2023-10-26T05:49:02.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd52e0640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2db4066470 0x7f2db4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:02.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.717+0000 7f2dd52e0640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2db4066470 0x7f2db4068930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0142e00 tx=0x7f2dcc009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:02.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.721+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2dc00ec9c0 con 0x7f2dbc003980 2023-10-26T05:49:02.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:02.909+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2dd00721e0 con 0x7f2dbc003980 2023-10-26T05:49:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.589+0000 7f2dc67fc640 1 -- 172.21.15.71:0/2314762329 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v16) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2dc00ba0c0 con 0x7f2dbc003980 2023-10-26T05:49:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 6 and data pool 7 2023-10-26T05:49:03.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2db4066470 msgr2=0x7f2db4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2db4066470 0x7f2db4068930 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0142e00 tx=0x7f2dcc009500 comp rx=0 tx=0).stop 2023-10-26T05:49:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 msgr2=0x7f2dd01596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dd01596f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009a80 tx=0x7f2dc0005640 comp rx=0 tx=0).stop 2023-10-26T05:49:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 shutdown_connections 2023-10-26T05:49:03.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2dd013d5c0 0x7f2dd013da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2dd01365f0 0x7f2dd0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2db4066470 0x7f2db4068930 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 --2- 172.21.15.71:0/2314762329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2dbc003980 0x7f2dd01596f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 >> 172.21.15.71:0/2314762329 conn(0x7f2dd00aec60 msgr2=0x7f2dd015e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:03.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 shutdown_connections 2023-10-26T05:49:03.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.593+0000 7f2dd7d45640 1 -- 172.21.15.71:0/2314762329 wait complete. 2023-10-26T05:49:03.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:03.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/1048988672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 msgr2=0x7f6ab006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/1048988672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab006aa70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6ab00ccd20 tx=0x7f6aa401c780 comp rx=0 tx=0).stop 2023-10-26T05:49:03.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/1048988672 shutdown_connections 2023-10-26T05:49:03.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/1048988672 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ab00d0850 0x7f6ab015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/1048988672 >> v2:172.21.15.86:3300/0 conn(0x7f6ab00cd880 0x7f6ab015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/1048988672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab006aa70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/1048988672 >> 172.21.15.71:0/1048988672 conn(0x7f6ab00cdd80 msgr2=0x7f6ab00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:03.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/1048988672 shutdown_connections 2023-10-26T05:49:03.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/1048988672 wait complete. 2023-10-26T05:49:03.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 Processor -- start 2023-10-26T05:49:03.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- start start 2023-10-26T05:49:03.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ab00cd880 0x7f6ab00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:03.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab014d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:03.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab00d0850 0x7f6ab015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00c5570 con 0x7f6ab00d0470 2023-10-26T05:49:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00c8500 con 0x7f6ab00d0850 2023-10-26T05:49:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab00c4fb0 con 0x7f6ab00cd880 2023-10-26T05:49:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab014d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:03.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab014d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52686/0 (socket says 172.21.15.71:52686) 2023-10-26T05:49:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 -- 172.21.15.71:0/857616462 learned_addr learned my addr 172.21.15.71:0/857616462 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aae59c640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ab00cd880 0x7f6ab00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aaed9d640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab00d0850 0x7f6ab015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 -- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ab00cd880 msgr2=0x7f6ab00c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ab00cd880 0x7f6ab00c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 -- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab00d0850 msgr2=0x7f6ab015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab00d0850 0x7f6ab015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 -- 172.21.15.71:0/857616462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4009560 con 0x7f6ab00d0470 2023-10-26T05:49:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6aadd9b640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab014d750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000b410 tx=0x7f6aa000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0011830 con 0x7f6ab00d0470 2023-10-26T05:49:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6aa0011a80 con 0x7f6ab00d0470 2023-10-26T05:49:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab00c5290 con 0x7f6ab00d0470 2023-10-26T05:49:03.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0011e90 con 0x7f6ab00d0470 2023-10-26T05:49:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.745+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab00c5d10 con 0x7f6ab00d0470 2023-10-26T05:49:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.749+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab0066b10 con 0x7f6ab00d0470 2023-10-26T05:49:03.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.749+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6aa001a400 con 0x7f6ab00d0470 2023-10-26T05:49:03.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.749+0000 7f6a9effd640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a90066470 0x7f6a90068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:03.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.749+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6aa00841d0 con 0x7f6ab00d0470 2023-10-26T05:49:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.749+0000 7f6aae59c640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a90066470 0x7f6a90068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.749+0000 7f6aae59c640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a90066470 0x7f6a90068930 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40af000 tx=0x7f6aa401afc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:03.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.753+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6aa00516e0 con 0x7f6ab00d0470 2023-10-26T05:49:03.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.909+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ab00721e0 con 0x7f6ab00d0470 2023-10-26T05:49:03.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.909+0000 7f6a9effd640 1 -- 172.21.15.71:0/857616462 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16442 (secure 0 0 0) 0x7f6aa0054d00 con 0x7f6ab00d0470 2023-10-26T05:49:03.915 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:03.916 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:02.915085+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:03.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a90066470 msgr2=0x7f6a90068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a90066470 0x7f6a90068930 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40af000 tx=0x7f6aa401afc0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 msgr2=0x7f6ab014d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:03.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab014d750 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000b410 tx=0x7f6aa000b8d0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 shutdown_connections 2023-10-26T05:49:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ab00d0850 0x7f6ab015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a90066470 0x7f6a90068930 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab00d0470 0x7f6ab014d750 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 --2- 172.21.15.71:0/857616462 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6ab00cd880 0x7f6ab00c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 >> 172.21.15.71:0/857616462 conn(0x7f6ab00cdd80 msgr2=0x7f6ab00b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 shutdown_connections 2023-10-26T05:49:03.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:03.913+0000 7f6ab4a2e640 1 -- 172.21.15.71:0/857616462 wait complete. 2023-10-26T05:49:03.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:04.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/469916410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 msgr2=0x7f1b5415aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/469916410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5415aa20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009c80 tx=0x7f1b4c01ccc0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/469916410 shutdown_connections 2023-10-26T05:49:04.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/469916410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5415aa20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/469916410 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b5415c720 0x7f1b540b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/469916410 >> 172.21.15.71:0/469916410 conn(0x7f1b540b0f20 msgr2=0x7f1b540b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:04.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/469916410 shutdown_connections 2023-10-26T05:49:04.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/469916410 wait complete. 2023-10-26T05:49:04.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 Processor -- start 2023-10-26T05:49:04.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- start start 2023-10-26T05:49:04.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b540cede0 0x7f1b54143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 0x7f1b541441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54145500 con 0x7f1b5415c720 2023-10-26T05:49:04.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54143f60 con 0x7f1b5415d560 2023-10-26T05:49:04.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5ba8a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54144c60 con 0x7f1b540cede0 2023-10-26T05:49:04.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 0x7f1b541441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 0x7f1b541441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52688/0 (socket says 172.21.15.71:52688) 2023-10-26T05:49:04.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 -- 172.21.15.71:0/1180598678 learned_addr learned my addr 172.21.15.71:0/1180598678 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:04.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59826640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b540cede0 0x7f1b54143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5a027640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 -- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b540cede0 msgr2=0x7f1b54143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b540cede0 0x7f1b54143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 -- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 msgr2=0x7f1b5406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59025640 1 -- 172.21.15.71:0/1180598678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0098e0 con 0x7f1b5415c720 2023-10-26T05:49:04.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b5a027640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:04.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.053+0000 7f1b59826640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b540cede0 0x7f1b54143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:04.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b59025640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 0x7f1b541441a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c420 tx=0x7f1b4400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:04.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440052d0 con 0x7f1b5415c720 2023-10-26T05:49:04.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b44007020 con 0x7f1b5415c720 2023-10-26T05:49:04.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b54144f40 con 0x7f1b5415c720 2023-10-26T05:49:04.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44007430 con 0x7f1b5415c720 2023-10-26T05:49:04.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b54148db0 con 0x7f1b5415c720 2023-10-26T05:49:04.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b54148c10 con 0x7f1b5415c720 2023-10-26T05:49:04.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b44007590 con 0x7f1b5415c720 2023-10-26T05:49:04.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b4a7fc640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b38066470 0x7f1b38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b44083c00 con 0x7f1b5415c720 2023-10-26T05:49:04.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b59826640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b38066470 0x7f1b38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.057+0000 7f1b59826640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b38066470 0x7f1b38068930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009c30 tx=0x7f1b500091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:04.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.061+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b44051080 con 0x7f1b5415c720 2023-10-26T05:49:04.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.213+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b54148c10 con 0x7f1b5415c720 2023-10-26T05:49:04.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.217+0000 7f1b4a7fc640 1 -- 172.21.15.71:0/1180598678 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16442 (secure 0 0 0) 0x7f1b54148c10 con 0x7f1b5415c720 2023-10-26T05:49:04.221 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:04.221 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":47,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:02.915085+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.217+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b38066470 msgr2=0x7f1b38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.217+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b38066470 0x7f1b38068930 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009c30 tx=0x7f1b500091e0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.217+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 msgr2=0x7f1b541441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.217+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 0x7f1b541441a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c420 tx=0x7f1b4400c8e0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 shutdown_connections 2023-10-26T05:49:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b5415d560 0x7f1b5406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1b38066470 0x7f1b38068930 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5415c720 0x7f1b541441a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 --2- 172.21.15.71:0/1180598678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b540cede0 0x7f1b54143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 >> 172.21.15.71:0/1180598678 conn(0x7f1b540b0f20 msgr2=0x7f1b540b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 shutdown_connections 2023-10-26T05:49:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.221+0000 7f1b5ba8a640 1 -- 172.21.15.71:0/1180598678 wait complete. 2023-10-26T05:49:04.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:49:04.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 -- 172.21.15.71:0/1598588329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80d0850 msgr2=0x7f6de815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 --2- 172.21.15.71:0/1598588329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80d0850 0x7f6de815b3a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6de808a140 tx=0x7f6dd001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 -- 172.21.15.71:0/1598588329 shutdown_connections 2023-10-26T05:49:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 --2- 172.21.15.71:0/1598588329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80d0850 0x7f6de815b3a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 --2- 172.21.15.71:0/1598588329 >> v2:172.21.15.86:3300/0 conn(0x7f6de80d0470 0x7f6de806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 -- 172.21.15.71:0/1598588329 >> 172.21.15.71:0/1598588329 conn(0x7f6de80cdd80 msgr2=0x7f6de80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 -- 172.21.15.71:0/1598588329 shutdown_connections 2023-10-26T05:49:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 -- 172.21.15.71:0/1598588329 wait complete. 2023-10-26T05:49:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 Processor -- start 2023-10-26T05:49:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.361+0000 7f6deca2e640 1 -- start start 2023-10-26T05:49:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 0x7f6de815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de80d0470 0x7f6de815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de80d0850 0x7f6de8150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de81503d0 con 0x7f6de80cd880 2023-10-26T05:49:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8150da0 con 0x7f6de80d0850 2023-10-26T05:49:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de814fac0 con 0x7f6de80d0470 2023-10-26T05:49:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 0x7f6de815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 0x7f6de815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52706/0 (socket says 172.21.15.71:52706) 2023-10-26T05:49:04.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 -- 172.21.15.71:0/2582057400 learned_addr learned my addr 172.21.15.71:0/2582057400 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de6d9d640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de80d0850 0x7f6de8150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de5d9b640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de80d0470 0x7f6de815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 -- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de80d0470 msgr2=0x7f6de815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de80d0470 0x7f6de815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 -- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de80d0850 msgr2=0x7f6de8150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de80d0850 0x7f6de8150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 -- 172.21.15.71:0/2582057400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0009560 con 0x7f6de80cd880 2023-10-26T05:49:04.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de6d9d640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de80d0850 0x7f6de8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:04.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de659c640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 0x7f6de815d790 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6de000cc30 tx=0x7f6de000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:04.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000f650 con 0x7f6de80cd880 2023-10-26T05:49:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6de000f8a0 con 0x7f6de80cd880 2023-10-26T05:49:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de80cae00 con 0x7f6de80cd880 2023-10-26T05:49:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000fcb0 con 0x7f6de80cd880 2023-10-26T05:49:04.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de814fd80 con 0x7f6de80cd880 2023-10-26T05:49:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de8066b10 con 0x7f6de80cd880 2023-10-26T05:49:04.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6de0018400 con 0x7f6de80cd880 2023-10-26T05:49:04.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6ddeffd640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dc8066470 0x7f6dc8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6de0084ba0 con 0x7f6de80cd880 2023-10-26T05:49:04.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.365+0000 7f6de5d9b640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dc8066470 0x7f6dc8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.369+0000 7f6de5d9b640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dc8066470 0x7f6dc8068930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00af000 tx=0x7f6dd001afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:04.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.369+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6de0052020 con 0x7f6de80cd880 2023-10-26T05:49:04.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.561+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6de80721e0 con 0x7f6de80cd880 2023-10-26T05:49:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.621+0000 7f6ddeffd640 1 -- 172.21.15.71:0/2582057400 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6de0055640 con 0x7f6de80cd880 2023-10-26T05:49:04.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dc8066470 msgr2=0x7f6dc8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dc8066470 0x7f6dc8068930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00af000 tx=0x7f6dd001afe0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 msgr2=0x7f6de815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 0x7f6de815d790 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6de000cc30 tx=0x7f6de000a620 comp rx=0 tx=0).stop 2023-10-26T05:49:04.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 shutdown_connections 2023-10-26T05:49:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6de80d0850 0x7f6de8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6de80d0470 0x7f6de815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dc8066470 0x7f6dc8068930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 --2- 172.21.15.71:0/2582057400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6de80cd880 0x7f6de815d790 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 >> 172.21.15.71:0/2582057400 conn(0x7f6de80cdd80 msgr2=0x7f6de8154c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 shutdown_connections 2023-10-26T05:49:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.625+0000 7f6deca2e640 1 -- 172.21.15.71:0/2582057400 wait complete. 2023-10-26T05:49:04.649 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:49:04.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:49:04.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/2513092787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 msgr2=0x7fed04003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/2513092787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed04003da0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fed1400bbe0 tx=0x7fed1401cdb0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/2513092787 shutdown_connections 2023-10-26T05:49:04.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/2513092787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed04003da0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/2513092787 >> 172.21.15.71:0/2513092787 conn(0x7fed180aec60 msgr2=0x7fed180af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:04.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/2513092787 shutdown_connections 2023-10-26T05:49:04.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/2513092787 wait complete. 2023-10-26T05:49:04.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 Processor -- start 2023-10-26T05:49:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- start start 2023-10-26T05:49:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed181e4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed181559e0 0x7fed180adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed180ae520 0x7fed1815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ae9a0 con 0x7fed04003980 2023-10-26T05:49:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed181f4d90 con 0x7fed181559e0 2023-10-26T05:49:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed181f5070 con 0x7fed180ae520 2023-10-26T05:49:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed181e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed181e4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52722/0 (socket says 172.21.15.71:52722) 2023-10-26T05:49:04.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 -- 172.21.15.71:0/3780286440 learned_addr learned my addr 172.21.15.71:0/3780286440 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:04.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d180640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed181559e0 0x7fed180adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1e182640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed180ae520 0x7fed1815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 -- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed180ae520 msgr2=0x7fed1815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed180ae520 0x7fed1815a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 -- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed181559e0 msgr2=0x7fed180adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:04.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed181559e0 0x7fed180adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:04.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 -- 172.21.15.71:0/3780286440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed181f5350 con 0x7fed04003980 2023-10-26T05:49:04.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d180640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed181559e0 0x7fed180adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:04.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1d981640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed181e4890 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fed140b1040 tx=0x7fed14005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:04.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140be070 con 0x7fed04003980 2023-10-26T05:49:04.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed14004980 con 0x7fed04003980 2023-10-26T05:49:04.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14004d10 con 0x7fed04003980 2023-10-26T05:49:04.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed181f5630 con 0x7fed04003980 2023-10-26T05:49:04.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.773+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed18155e60 con 0x7fed04003980 2023-10-26T05:49:04.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.777+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed180acb90 con 0x7fed04003980 2023-10-26T05:49:04.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.777+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed140c3410 con 0x7fed04003980 2023-10-26T05:49:04.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.777+0000 7fed0e7fc640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fecfc066470 0x7fecfc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:04.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.777+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed1411e7d0 con 0x7fed04003980 2023-10-26T05:49:04.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.777+0000 7fed1d180640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fecfc066470 0x7fecfc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:04.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.777+0000 7fed1d180640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fecfc066470 0x7fecfc068930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed180c9ea0 tx=0x7fed08006a80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:04.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.781+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed140ebc60 con 0x7fed04003980 2023-10-26T05:49:04.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:04.969+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fed18066b10 con 0x7fed04003980 2023-10-26T05:49:05.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.637+0000 7fed0e7fc640 1 -- 172.21.15.71:0/3780286440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed140ab020 con 0x7fed04003980 2023-10-26T05:49:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fecfc066470 msgr2=0x7fecfc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fecfc066470 0x7fecfc068930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed180c9ea0 tx=0x7fed08006a80 comp rx=0 tx=0).stop 2023-10-26T05:49:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 msgr2=0x7fed181e4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:05.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed181e4890 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fed140b1040 tx=0x7fed14005700 comp rx=0 tx=0).stop 2023-10-26T05:49:05.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 shutdown_connections 2023-10-26T05:49:05.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed180ae520 0x7fed1815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed181559e0 0x7fed180adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fecfc066470 0x7fecfc068930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 --2- 172.21.15.71:0/3780286440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04003980 0x7fed181e4890 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 >> 172.21.15.71:0/3780286440 conn(0x7fed180aec60 msgr2=0x7fed180aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:05.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 shutdown_connections 2023-10-26T05:49:05.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.641+0000 7fed1fbe5640 1 -- 172.21.15.71:0/3780286440 wait complete. 2023-10-26T05:49:05.668 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:49:05.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:05.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.797+0000 7f1384416640 1 -- 172.21.15.71:0/688144817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c069770 msgr2=0x7f137c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:05.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.797+0000 7f1384416640 1 --2- 172.21.15.71:0/688144817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c069770 0x7f137c06a110 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f137c0d1ff0 tx=0x7f137801c8a0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/688144817 shutdown_connections 2023-10-26T05:49:05.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 --2- 172.21.15.71:0/688144817 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f137c069bb0 0x7f137c06a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 --2- 172.21.15.71:0/688144817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c069770 0x7f137c06a110 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 --2- 172.21.15.71:0/688144817 >> v2:172.21.15.86:3300/0 conn(0x7f137c15b5c0 0x7f137c0ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/688144817 >> 172.21.15.71:0/688144817 conn(0x7f137c0d12d0 msgr2=0x7f137c0d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:05.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/688144817 shutdown_connections 2023-10-26T05:49:05.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/688144817 wait complete. 2023-10-26T05:49:05.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 Processor -- start 2023-10-26T05:49:05.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- start start 2023-10-26T05:49:05.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137c069770 0x7f137c15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:05.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f137c069bb0 0x7f137c0c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:05.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 0x7f137c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:05.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b0590 con 0x7f137c15b5c0 2023-10-26T05:49:05.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137c160250 con 0x7f137c069770 2023-10-26T05:49:05.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137c0b02b0 con 0x7f137c069bb0 2023-10-26T05:49:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 0x7f137c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 0x7f137c15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52742/0 (socket says 172.21.15.71:52742) 2023-10-26T05:49:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 -- 172.21.15.71:0/1261974496 learned_addr learned my addr 172.21.15.71:0/1261974496 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13821b2640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137c069770 0x7f137c15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13819b1640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f137c069bb0 0x7f137c0c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 -- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f137c069bb0 msgr2=0x7f137c0c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f137c069bb0 0x7f137c0c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 -- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137c069770 msgr2=0x7f137c15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137c069770 0x7f137c15de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 -- 172.21.15.71:0/1261974496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1378009560 con 0x7f137c15b5c0 2023-10-26T05:49:05.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13821b2640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137c069770 0x7f137c15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:05.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f13829b3640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 0x7f137c15a190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f136c00b430 tx=0x7f136c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:05.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c011830 con 0x7f137c15b5c0 2023-10-26T05:49:05.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f136c011a80 con 0x7f137c15b5c0 2023-10-26T05:49:05.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137c160b40 con 0x7f137c15b5c0 2023-10-26T05:49:05.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c011e90 con 0x7f137c15b5c0 2023-10-26T05:49:05.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137c15e3d0 con 0x7f137c15b5c0 2023-10-26T05:49:05.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.801+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137c066b10 con 0x7f137c15b5c0 2023-10-26T05:49:05.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.805+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f136c01a400 con 0x7f137c15b5c0 2023-10-26T05:49:05.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.805+0000 7f1372ffd640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1368066470 0x7f1368068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:05.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.805+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f136c084210 con 0x7f137c15b5c0 2023-10-26T05:49:05.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.805+0000 7f13821b2640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1368066470 0x7f1368068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:05.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.805+0000 7f13821b2640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1368066470 0x7f1368068930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1374005c50 tx=0x7f13740082b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:05.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.809+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f136c051720 con 0x7f137c15b5c0 2023-10-26T05:49:06.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.997+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f137c15e620 con 0x7f137c15b5c0 2023-10-26T05:49:06.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.997+0000 7f1372ffd640 1 -- 172.21.15.71:0/1261974496 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f137c15e620 con 0x7f137c15b5c0 2023-10-26T05:49:06.001 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:06.001 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:06.002 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:05.997+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1368066470 msgr2=0x7f1368068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1368066470 0x7f1368068930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1374005c50 tx=0x7f13740082b0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 msgr2=0x7f137c15a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 0x7f137c15a190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f136c00b430 tx=0x7f136c00b8f0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 shutdown_connections 2023-10-26T05:49:06.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1368066470 0x7f1368068930 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f137c15b5c0 0x7f137c15a190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f137c069bb0 0x7f137c0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 --2- 172.21.15.71:0/1261974496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f137c069770 0x7f137c15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 >> 172.21.15.71:0/1261974496 conn(0x7f137c0d12d0 msgr2=0x7f137c0b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 shutdown_connections 2023-10-26T05:49:06.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.001+0000 7f1384416640 1 -- 172.21.15.71:0/1261974496 wait complete. 2023-10-26T05:49:06.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:06.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/1788586804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780039b0 msgr2=0x7efd78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 --2- 172.21.15.71:0/1788586804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd780039b0 0x7efd78003dd0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00bc00 tx=0x7efd7c01ce50 comp rx=0 tx=0).stop 2023-10-26T05:49:06.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd9173e640 1 -- 172.21.15.71:0/1788586804 reap_dead start 2023-10-26T05:49:06.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/1788586804 shutdown_connections 2023-10-26T05:49:06.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/1788586804 >> 172.21.15.71:0/1788586804 conn(0x7efd8c0cdd80 msgr2=0x7efd8c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/1788586804 shutdown_connections 2023-10-26T05:49:06.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/1788586804 wait complete. 2023-10-26T05:49:06.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 Processor -- start 2023-10-26T05:49:06.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- start start 2023-10-26T05:49:06.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd780039b0 0x7efd8c151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 0x7efd8c0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c15fa60 0x7efd8c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c160030 con 0x7efd8c15f620 2023-10-26T05:49:06.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c160310 con 0x7efd8c15fa60 2023-10-26T05:49:06.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c1605f0 con 0x7efd780039b0 2023-10-26T05:49:06.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 0x7efd8c0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 0x7efd8c0b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52758/0 (socket says 172.21.15.71:52758) 2023-10-26T05:49:06.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 -- 172.21.15.71:0/2144361642 learned_addr learned my addr 172.21.15.71:0/2144361642 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:06.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd90f3d640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd780039b0 0x7efd8c151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd9173e640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c15fa60 0x7efd8c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 -- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd780039b0 msgr2=0x7efd8c151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd780039b0 0x7efd8c151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 -- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c15fa60 msgr2=0x7efd8c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c15fa60 0x7efd8c0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 -- 172.21.15.71:0/2144361642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c1608d0 con 0x7efd8c15f620 2023-10-26T05:49:06.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd9173e640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c15fa60 0x7efd8c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:06.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd8bfff640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 0x7efd8c0b5340 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd8400fbd0 tx=0x7efd8400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:06.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8400dee0 con 0x7efd8c15f620 2023-10-26T05:49:06.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd8400fed0 con 0x7efd8c15f620 2023-10-26T05:49:06.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c151dd0 con 0x7efd8c15f620 2023-10-26T05:49:06.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd840118e0 con 0x7efd8c15f620 2023-10-26T05:49:06.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.137+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c1f82e0 con 0x7efd8c15f620 2023-10-26T05:49:06.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.141+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c066b10 con 0x7efd8c15f620 2023-10-26T05:49:06.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.141+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd84011a40 con 0x7efd8c15f620 2023-10-26T05:49:06.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.141+0000 7efd897fa640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd6c0664a0 0x7efd6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.141+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd84084de0 con 0x7efd8c15f620 2023-10-26T05:49:06.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.141+0000 7efd90f3d640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd6c0664a0 0x7efd6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.141+0000 7efd90f3d640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd6c0664a0 0x7efd6c068960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0af000 tx=0x7efd7c019bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:06.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.145+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd840522f0 con 0x7efd8c15f620 2023-10-26T05:49:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.297+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd8c0721e0 con 0x7efd8c15f620 2023-10-26T05:49:06.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.297+0000 7efd897fa640 1 -- 172.21.15.71:0/2144361642 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16429 (secure 0 0 0) 0x7efd84055910 con 0x7efd8c15f620 2023-10-26T05:49:06.302 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:06.303 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":48,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:04.590515+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd6c0664a0 msgr2=0x7efd6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd6c0664a0 0x7efd6c068960 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0af000 tx=0x7efd7c019bf0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 msgr2=0x7efd8c0b5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 0x7efd8c0b5340 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd8400fbd0 tx=0x7efd8400d6b0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 shutdown_connections 2023-10-26T05:49:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd8c15fa60 0x7efd8c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd6c0664a0 0x7efd6c068960 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd8c15f620 0x7efd8c0b5340 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 --2- 172.21.15.71:0/2144361642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd780039b0 0x7efd8c151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 >> 172.21.15.71:0/2144361642 conn(0x7efd8c0cdd80 msgr2=0x7efd8c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 shutdown_connections 2023-10-26T05:49:06.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.301+0000 7efd931a1640 1 -- 172.21.15.71:0/2144361642 wait complete. 2023-10-26T05:49:06.320 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:49:06.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:49:06.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.437+0000 7f84c50bc640 1 -- 172.21.15.71:0/150851873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 msgr2=0x7f84a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.437+0000 7f84c50bc640 1 --2- 172.21.15.71:0/150851873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84a8004c50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f84b4006fd0 tx=0x7f84b401f720 comp rx=0 tx=0).stop 2023-10-26T05:49:06.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.437+0000 7f84c50bc640 1 -- 172.21.15.71:0/150851873 shutdown_connections 2023-10-26T05:49:06.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.437+0000 7f84c50bc640 1 --2- 172.21.15.71:0/150851873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84a8004c50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.437+0000 7f84c50bc640 1 -- 172.21.15.71:0/150851873 >> 172.21.15.71:0/150851873 conn(0x7f84c0069770 msgr2=0x7f84c0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- 172.21.15.71:0/150851873 shutdown_connections 2023-10-26T05:49:06.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- 172.21.15.71:0/150851873 wait complete. 2023-10-26T05:49:06.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 Processor -- start 2023-10-26T05:49:06.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- start start 2023-10-26T05:49:06.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84c00a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c01f3250 0x7f84c01f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84c01f3c10 0x7f84c01eec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c0134250 con 0x7f84a80047d0 2023-10-26T05:49:06.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c014c6e0 con 0x7f84c01f3250 2023-10-26T05:49:06.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c014cca0 con 0x7f84c01f3c10 2023-10-26T05:49:06.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84c00a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84c00a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52770/0 (socket says 172.21.15.71:52770) 2023-10-26T05:49:06.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 -- 172.21.15.71:0/1226485734 learned_addr learned my addr 172.21.15.71:0/1226485734 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:06.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84be59c640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c01f3250 0x7f84c01f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bf59e640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84c01f3c10 0x7f84c01eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 -- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84c01f3c10 msgr2=0x7f84c01eec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84c01f3c10 0x7f84c01eec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 -- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c01f3250 msgr2=0x7f84c01f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c01f3250 0x7f84c01f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 -- 172.21.15.71:0/1226485734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c014f5e0 con 0x7f84a80047d0 2023-10-26T05:49:06.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84be59c640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c01f3250 0x7f84c01f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:06.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84bed9d640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84c00a64d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d680 tx=0x7f84b000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:06.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0005220 con 0x7f84a80047d0 2023-10-26T05:49:06.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84b0005470 con 0x7f84a80047d0 2023-10-26T05:49:06.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c01374e0 con 0x7f84a80047d0 2023-10-26T05:49:06.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0009a20 con 0x7f84a80047d0 2023-10-26T05:49:06.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c014c930 con 0x7f84a80047d0 2023-10-26T05:49:06.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.441+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c0066b10 con 0x7f84a80047d0 2023-10-26T05:49:06.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.445+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f84b0009b80 con 0x7f84a80047d0 2023-10-26T05:49:06.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.445+0000 7f84a77fe640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84ac066470 0x7f84ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.445+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84b0083ea0 con 0x7f84a80047d0 2023-10-26T05:49:06.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.445+0000 7f84be59c640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84ac066470 0x7f84ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.445+0000 7f84be59c640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84ac066470 0x7f84ac068930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f84b40b0000 tx=0x7f84b401c4c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:06.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.449+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f84b00513b0 con 0x7f84a80047d0 2023-10-26T05:49:06.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.597+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84c00cba10 con 0x7f84ac066470 2023-10-26T05:49:06.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.621+0000 7f84a77fe640 1 -- 172.21.15.71:0/1226485734 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283664 (secure 0 0 0) 0x7f84b4000ee0 con 0x7f84ac066470 2023-10-26T05:49:06.629 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:06.631 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:49:06.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84ac066470 msgr2=0x7f84ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84ac066470 0x7f84ac068930 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f84b40b0000 tx=0x7f84b401c4c0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 msgr2=0x7f84c00a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84c00a64d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f84b000d680 tx=0x7f84b000db40 comp rx=0 tx=0).stop 2023-10-26T05:49:06.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 shutdown_connections 2023-10-26T05:49:06.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84c01f3c10 0x7f84c01eec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84c01f3250 0x7f84c01f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84ac066470 0x7f84ac068930 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 --2- 172.21.15.71:0/1226485734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a80047d0 0x7f84c00a64d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 >> 172.21.15.71:0/1226485734 conn(0x7f84c0069770 msgr2=0x7f84c0158840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 shutdown_connections 2023-10-26T05:49:06.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.625+0000 7f84c50bc640 1 -- 172.21.15.71:0/1226485734 wait complete. 2023-10-26T05:49:06.647 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-10-26T05:49:05.446608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559597,"statfs":{"total":1600000000,"available":1597027841,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001103"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609426+0000","last_change":"2023-10-26T05:49:03.607461+0000","last_active":"2023-10-26T05:49:03.609426+0000","last_peered":"2023-10-26T05:49:03.609426+0000","last_clean":"2023-10-26T05:49:03.609426+0000","last_became_active":"2023-10-26T05:49:03.605415+0000","last_became_peered":"2023-10-26T05:49:03.605415+0000","last_unstale":"2023-10-26T05:49:03.609426+0000","last_undegraded":"2023-10-26T05:49:03.609426+0000","last_fullsized":"2023-10-26T05:49:03.609426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:44.388032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626757+0000","last_change":"2023-10-26T05:49:03.625701+0000","last_active":"2023-10-26T05:49:03.626757+0000","last_peered":"2023-10-26T05:49:03.626757+0000","last_clean":"2023-10-26T05:49:03.626757+0000","last_became_active":"2023-10-26T05:49:03.623694+0000","last_became_peered":"2023-10-26T05:49:03.623694+0000","last_unstale":"2023-10-26T05:49:03.626757+0000","last_undegraded":"2023-10-26T05:49:03.626757+0000","last_fullsized":"2023-10-26T05:49:03.626757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:07.865684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609329+0000","last_change":"2023-10-26T05:49:03.608806+0000","last_active":"2023-10-26T05:49:03.609329+0000","last_peered":"2023-10-26T05:49:03.609329+0000","last_clean":"2023-10-26T05:49:03.609329+0000","last_became_active":"2023-10-26T05:49:03.606696+0000","last_became_peered":"2023-10-26T05:49:03.606696+0000","last_unstale":"2023-10-26T05:49:03.609329+0000","last_undegraded":"2023-10-26T05:49:03.609329+0000","last_fullsized":"2023-10-26T05:49:03.609329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:36.604418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.633642+0000","last_change":"2023-10-26T05:49:03.633583+0000","last_active":"2023-10-26T05:49:03.633642+0000","last_peered":"2023-10-26T05:49:03.633642+0000","last_clean":"2023-10-26T05:49:03.633642+0000","last_became_active":"2023-10-26T05:49:03.633329+0000","last_became_peered":"2023-10-26T05:49:03.633329+0000","last_unstale":"2023-10-26T05:49:03.633642+0000","last_undegraded":"2023-10-26T05:49:03.633642+0000","last_fullsized":"2023-10-26T05:49:03.633642+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:17.113253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.627989+0000","last_change":"2023-10-26T05:49:03.627526+0000","last_active":"2023-10-26T05:49:03.627989+0000","last_peered":"2023-10-26T05:49:03.627989+0000","last_clean":"2023-10-26T05:49:03.627989+0000","last_became_active":"2023-10-26T05:49:03.620638+0000","last_became_peered":"2023-10-26T05:49:03.620638+0000","last_unstale":"2023-10-26T05:49:03.627989+0000","last_undegraded":"2023-10-26T05:49:03.627989+0000","last_fullsized":"2023-10-26T05:49:03.627989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:44.388032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.618923+0000","last_change":"2023-10-26T05:49:03.618749+0000","last_active":"2023-10-26T05:49:03.618923+0000","last_peered":"2023-10-26T05:49:03.618923+0000","last_clean":"2023-10-26T05:49:03.618923+0000","last_became_active":"2023-10-26T05:49:03.618399+0000","last_became_peered":"2023-10-26T05:49:03.618399+0000","last_unstale":"2023-10-26T05:49:03.618923+0000","last_undegraded":"2023-10-26T05:49:03.618923+0000","last_fullsized":"2023-10-26T05:49:03.618923+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:40.549589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606555+0000","last_change":"2023-10-26T05:49:03.604123+0000","last_active":"2023-10-26T05:49:03.606555+0000","last_peered":"2023-10-26T05:49:03.606555+0000","last_clean":"2023-10-26T05:49:03.606555+0000","last_became_active":"2023-10-26T05:49:03.603512+0000","last_became_peered":"2023-10-26T05:49:03.603512+0000","last_unstale":"2023-10-26T05:49:03.606555+0000","last_undegraded":"2023-10-26T05:49:03.606555+0000","last_fullsized":"2023-10-26T05:49:03.606555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:43.246142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.622309+0000","last_change":"2023-10-26T05:49:03.622243+0000","last_active":"2023-10-26T05:49:03.622309+0000","last_peered":"2023-10-26T05:49:03.622309+0000","last_clean":"2023-10-26T05:49:03.622309+0000","last_became_active":"2023-10-26T05:49:03.622019+0000","last_became_peered":"2023-10-26T05:49:03.622019+0000","last_unstale":"2023-10-26T05:49:03.622309+0000","last_undegraded":"2023-10-26T05:49:03.622309+0000","last_fullsized":"2023-10-26T05:49:03.622309+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:10.235795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.600548+0000","last_change":"2023-10-26T05:49:03.599891+0000","last_active":"2023-10-26T05:49:03.600548+0000","last_peered":"2023-10-26T05:49:03.600548+0000","last_clean":"2023-10-26T05:49:03.600548+0000","last_became_active":"2023-10-26T05:49:03.599663+0000","last_became_peered":"2023-10-26T05:49:03.599663+0000","last_unstale":"2023-10-26T05:49:03.600548+0000","last_undegraded":"2023-10-26T05:49:03.600548+0000","last_fullsized":"2023-10-26T05:49:03.600548+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:16:19.574921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.618420+0000","last_change":"2023-10-26T05:49:03.618345+0000","last_active":"2023-10-26T05:49:03.618420+0000","last_peered":"2023-10-26T05:49:03.618420+0000","last_clean":"2023-10-26T05:49:03.618420+0000","last_became_active":"2023-10-26T05:49:03.617876+0000","last_became_peered":"2023-10-26T05:49:03.617876+0000","last_unstale":"2023-10-26T05:49:03.618420+0000","last_undegraded":"2023-10-26T05:49:03.618420+0000","last_fullsized":"2023-10-26T05:49:03.618420+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:36.604418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606468+0000","last_change":"2023-10-26T05:49:03.602519+0000","last_active":"2023-10-26T05:49:03.606468+0000","last_peered":"2023-10-26T05:49:03.606468+0000","last_clean":"2023-10-26T05:49:03.606468+0000","last_became_active":"2023-10-26T05:49:03.602228+0000","last_became_peered":"2023-10-26T05:49:03.602228+0000","last_unstale":"2023-10-26T05:49:03.606468+0000","last_undegraded":"2023-10-26T05:49:03.606468+0000","last_fullsized":"2023-10-26T05:49:03.606468+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:42.052128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626709+0000","last_change":"2023-10-26T05:49:03.625242+0000","last_active":"2023-10-26T05:49:03.626709+0000","last_peered":"2023-10-26T05:49:03.626709+0000","last_clean":"2023-10-26T05:49:03.626709+0000","last_became_active":"2023-10-26T05:49:03.623003+0000","last_became_peered":"2023-10-26T05:49:03.623003+0000","last_unstale":"2023-10-26T05:49:03.626709+0000","last_undegraded":"2023-10-26T05:49:03.626709+0000","last_fullsized":"2023-10-26T05:49:03.626709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:53.593057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.611977+0000","last_change":"2023-10-26T05:49:03.611889+0000","last_active":"2023-10-26T05:49:03.611977+0000","last_peered":"2023-10-26T05:49:03.611977+0000","last_clean":"2023-10-26T05:49:03.611977+0000","last_became_active":"2023-10-26T05:49:03.611578+0000","last_became_peered":"2023-10-26T05:49:03.611578+0000","last_unstale":"2023-10-26T05:49:03.611977+0000","last_undegraded":"2023-10-26T05:49:03.611977+0000","last_fullsized":"2023-10-26T05:49:03.611977+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:17.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.607669+0000","last_change":"2023-10-26T05:49:03.607577+0000","last_active":"2023-10-26T05:49:03.607669+0000","last_peered":"2023-10-26T05:49:03.607669+0000","last_clean":"2023-10-26T05:49:03.607669+0000","last_became_active":"2023-10-26T05:49:03.607270+0000","last_became_peered":"2023-10-26T05:49:03.607270+0000","last_unstale":"2023-10-26T05:49:03.607669+0000","last_undegraded":"2023-10-26T05:49:03.607669+0000","last_fullsized":"2023-10-26T05:49:03.607669+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:56.606656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621398+0000","last_change":"2023-10-26T05:49:03.621325+0000","last_active":"2023-10-26T05:49:03.621398+0000","last_peered":"2023-10-26T05:49:03.621398+0000","last_clean":"2023-10-26T05:49:03.621398+0000","last_became_active":"2023-10-26T05:49:03.620987+0000","last_became_peered":"2023-10-26T05:49:03.620987+0000","last_unstale":"2023-10-26T05:49:03.621398+0000","last_undegraded":"2023-10-26T05:49:03.621398+0000","last_fullsized":"2023-10-26T05:49:03.621398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:24:45.984512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621759+0000","last_change":"2023-10-26T05:49:03.621639+0000","last_active":"2023-10-26T05:49:03.621759+0000","last_peered":"2023-10-26T05:49:03.621759+0000","last_clean":"2023-10-26T05:49:03.621759+0000","last_became_active":"2023-10-26T05:49:03.620138+0000","last_became_peered":"2023-10-26T05:49:03.620138+0000","last_unstale":"2023-10-26T05:49:03.621759+0000","last_undegraded":"2023-10-26T05:49:03.621759+0000","last_fullsized":"2023-10-26T05:49:03.621759+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:42.192405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.625860+0000","last_change":"2023-10-26T05:49:03.625692+0000","last_active":"2023-10-26T05:49:03.625860+0000","last_peered":"2023-10-26T05:49:03.625860+0000","last_clean":"2023-10-26T05:49:03.625860+0000","last_became_active":"2023-10-26T05:49:03.624996+0000","last_became_peered":"2023-10-26T05:49:03.624996+0000","last_unstale":"2023-10-26T05:49:03.625860+0000","last_undegraded":"2023-10-26T05:49:03.625860+0000","last_fullsized":"2023-10-26T05:49:03.625860+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:42.052128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604189+0000","last_change":"2023-10-26T05:49:03.603205+0000","last_active":"2023-10-26T05:49:03.604189+0000","last_peered":"2023-10-26T05:49:03.604189+0000","last_clean":"2023-10-26T05:49:03.604189+0000","last_became_active":"2023-10-26T05:49:03.602436+0000","last_became_peered":"2023-10-26T05:49:03.602436+0000","last_unstale":"2023-10-26T05:49:03.604189+0000","last_undegraded":"2023-10-26T05:49:03.604189+0000","last_fullsized":"2023-10-26T05:49:03.604189+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:24.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.633477+0000","last_change":"2023-10-26T05:49:03.633178+0000","last_active":"2023-10-26T05:49:03.633477+0000","last_peered":"2023-10-26T05:49:03.633477+0000","last_clean":"2023-10-26T05:49:03.633477+0000","last_became_active":"2023-10-26T05:49:03.625796+0000","last_became_peered":"2023-10-26T05:49:03.625796+0000","last_unstale":"2023-10-26T05:49:03.633477+0000","last_undegraded":"2023-10-26T05:49:03.633477+0000","last_fullsized":"2023-10-26T05:49:03.633477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:29.113550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.615064+0000","last_change":"2023-10-26T05:49:03.614999+0000","last_active":"2023-10-26T05:49:03.615064+0000","last_peered":"2023-10-26T05:49:03.615064+0000","last_clean":"2023-10-26T05:49:03.615064+0000","last_became_active":"2023-10-26T05:49:03.614783+0000","last_became_peered":"2023-10-26T05:49:03.614783+0000","last_unstale":"2023-10-26T05:49:03.615064+0000","last_undegraded":"2023-10-26T05:49:03.615064+0000","last_fullsized":"2023-10-26T05:49:03.615064+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:29.113550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620868+0000","last_change":"2023-10-26T05:49:03.620795+0000","last_active":"2023-10-26T05:49:03.620868+0000","last_peered":"2023-10-26T05:49:03.620868+0000","last_clean":"2023-10-26T05:49:03.620868+0000","last_became_active":"2023-10-26T05:49:03.620571+0000","last_became_peered":"2023-10-26T05:49:03.620571+0000","last_unstale":"2023-10-26T05:49:03.620868+0000","last_undegraded":"2023-10-26T05:49:03.620868+0000","last_fullsized":"2023-10-26T05:49:03.620868+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:54:46.778631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.633734+0000","last_change":"2023-10-26T05:49:03.633679+0000","last_active":"2023-10-26T05:49:03.633734+0000","last_peered":"2023-10-26T05:49:03.633734+0000","last_clean":"2023-10-26T05:49:03.633734+0000","last_became_active":"2023-10-26T05:49:03.633436+0000","last_became_peered":"2023-10-26T05:49:03.633436+0000","last_unstale":"2023-10-26T05:49:03.633734+0000","last_undegraded":"2023-10-26T05:49:03.633734+0000","last_fullsized":"2023-10-26T05:49:03.633734+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:01:29.828911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.641400+0000","last_change":"2023-10-26T05:49:03.641248+0000","last_active":"2023-10-26T05:49:03.641400+0000","last_peered":"2023-10-26T05:49:03.641400+0000","last_clean":"2023-10-26T05:49:03.641400+0000","last_became_active":"2023-10-26T05:49:03.640126+0000","last_became_peered":"2023-10-26T05:49:03.640126+0000","last_unstale":"2023-10-26T05:49:03.641400+0000","last_undegraded":"2023-10-26T05:49:03.641400+0000","last_fullsized":"2023-10-26T05:49:03.641400+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:44.587647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.640234+0000","last_change":"2023-10-26T05:49:03.640056+0000","last_active":"2023-10-26T05:49:03.640234+0000","last_peered":"2023-10-26T05:49:03.640234+0000","last_clean":"2023-10-26T05:49:03.640234+0000","last_became_active":"2023-10-26T05:49:03.637500+0000","last_became_peered":"2023-10-26T05:49:03.637500+0000","last_unstale":"2023-10-26T05:49:03.640234+0000","last_undegraded":"2023-10-26T05:49:03.640234+0000","last_fullsized":"2023-10-26T05:49:03.640234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:01:29.828911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.617491+0000","last_change":"2023-10-26T05:49:03.617006+0000","last_active":"2023-10-26T05:49:03.617491+0000","last_peered":"2023-10-26T05:49:03.617491+0000","last_clean":"2023-10-26T05:49:03.617491+0000","last_became_active":"2023-10-26T05:49:03.615738+0000","last_became_peered":"2023-10-26T05:49:03.615738+0000","last_unstale":"2023-10-26T05:49:03.617491+0000","last_undegraded":"2023-10-26T05:49:03.617491+0000","last_fullsized":"2023-10-26T05:49:03.617491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:01.030276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.607474+0000","last_change":"2023-10-26T05:49:03.607209+0000","last_active":"2023-10-26T05:49:03.607474+0000","last_peered":"2023-10-26T05:49:03.607474+0000","last_clean":"2023-10-26T05:49:03.607474+0000","last_became_active":"2023-10-26T05:49:03.606978+0000","last_became_peered":"2023-10-26T05:49:03.606978+0000","last_unstale":"2023-10-26T05:49:03.607474+0000","last_undegraded":"2023-10-26T05:49:03.607474+0000","last_fullsized":"2023-10-26T05:49:03.607474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:01.030276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.629211+0000","last_change":"2023-10-26T05:49:03.628978+0000","last_active":"2023-10-26T05:49:03.629211+0000","last_peered":"2023-10-26T05:49:03.629211+0000","last_clean":"2023-10-26T05:49:03.629211+0000","last_became_active":"2023-10-26T05:49:03.628221+0000","last_became_peered":"2023-10-26T05:49:03.628221+0000","last_unstale":"2023-10-26T05:49:03.629211+0000","last_undegraded":"2023-10-26T05:49:03.629211+0000","last_fullsized":"2023-10-26T05:49:03.629211+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:42.844061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626545+0000","last_change":"2023-10-26T05:49:03.623123+0000","last_active":"2023-10-26T05:49:03.626545+0000","last_peered":"2023-10-26T05:49:03.626545+0000","last_clean":"2023-10-26T05:49:03.626545+0000","last_became_active":"2023-10-26T05:49:03.619317+0000","last_became_peered":"2023-10-26T05:49:03.619317+0000","last_unstale":"2023-10-26T05:49:03.626545+0000","last_undegraded":"2023-10-26T05:49:03.626545+0000","last_fullsized":"2023-10-26T05:49:03.626545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:39.212676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.638656+0000","last_change":"2023-10-26T05:49:03.638475+0000","last_active":"2023-10-26T05:49:03.638656+0000","last_peered":"2023-10-26T05:49:03.638656+0000","last_clean":"2023-10-26T05:49:03.638656+0000","last_became_active":"2023-10-26T05:49:03.632432+0000","last_became_peered":"2023-10-26T05:49:03.632432+0000","last_unstale":"2023-10-26T05:49:03.638656+0000","last_undegraded":"2023-10-26T05:49:03.638656+0000","last_fullsized":"2023-10-26T05:49:03.638656+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:17.113253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604233+0000","last_change":"2023-10-26T05:49:03.604049+0000","last_active":"2023-10-26T05:49:03.604233+0000","last_peered":"2023-10-26T05:49:03.604233+0000","last_clean":"2023-10-26T05:49:03.604233+0000","last_became_active":"2023-10-26T05:49:03.603458+0000","last_became_peered":"2023-10-26T05:49:03.603458+0000","last_unstale":"2023-10-26T05:49:03.604233+0000","last_undegraded":"2023-10-26T05:49:03.604233+0000","last_fullsized":"2023-10-26T05:49:03.604233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:56.707421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.623351+0000","last_change":"2023-10-26T05:49:03.623303+0000","last_active":"2023-10-26T05:49:03.623351+0000","last_peered":"2023-10-26T05:49:03.623351+0000","last_clean":"2023-10-26T05:49:03.623351+0000","last_became_active":"2023-10-26T05:49:03.623157+0000","last_became_peered":"2023-10-26T05:49:03.623157+0000","last_unstale":"2023-10-26T05:49:03.623351+0000","last_undegraded":"2023-10-26T05:49:03.623351+0000","last_fullsized":"2023-10-26T05:49:03.623351+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.608691+0000","last_change":"2023-10-26T05:49:03.608566+0000","last_active":"2023-10-26T05:49:03.608691+0000","last_peered":"2023-10-26T05:49:03.608691+0000","last_clean":"2023-10-26T05:49:03.608691+0000","last_became_active":"2023-10-26T05:49:03.608339+0000","last_became_peered":"2023-10-26T05:49:03.608339+0000","last_unstale":"2023-10-26T05:49:03.608691+0000","last_undegraded":"2023-10-26T05:49:03.608691+0000","last_fullsized":"2023-10-26T05:49:03.608691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:42.821655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.610611+0000","last_change":"2023-10-26T05:49:03.610496+0000","last_active":"2023-10-26T05:49:03.610611+0000","last_peered":"2023-10-26T05:49:03.610611+0000","last_clean":"2023-10-26T05:49:03.610611+0000","last_became_active":"2023-10-26T05:49:03.610182+0000","last_became_peered":"2023-10-26T05:49:03.610182+0000","last_unstale":"2023-10-26T05:49:03.610611+0000","last_undegraded":"2023-10-26T05:49:03.610611+0000","last_fullsized":"2023-10-26T05:49:03.610611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:56.606656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621288+0000","last_change":"2023-10-26T05:49:02.588420+0000","last_active":"2023-10-26T05:49:03.621288+0000","last_peered":"2023-10-26T05:49:03.621288+0000","last_clean":"2023-10-26T05:49:03.621288+0000","last_became_active":"2023-10-26T05:49:02.588270+0000","last_became_peered":"2023-10-26T05:49:02.588270+0000","last_unstale":"2023-10-26T05:49:03.621288+0000","last_undegraded":"2023-10-26T05:49:03.621288+0000","last_fullsized":"2023-10-26T05:49:03.621288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:11.532031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.640992+0000","last_change":"2023-10-26T05:49:03.640850+0000","last_active":"2023-10-26T05:49:03.640992+0000","last_peered":"2023-10-26T05:49:03.640992+0000","last_clean":"2023-10-26T05:49:03.640992+0000","last_became_active":"2023-10-26T05:49:03.624309+0000","last_became_peered":"2023-10-26T05:49:03.624309+0000","last_unstale":"2023-10-26T05:49:03.640992+0000","last_undegraded":"2023-10-26T05:49:03.640992+0000","last_fullsized":"2023-10-26T05:49:03.640992+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:22.416213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.628451+0000","last_change":"2023-10-26T05:49:02.590322+0000","last_active":"2023-10-26T05:49:03.628451+0000","last_peered":"2023-10-26T05:49:03.628451+0000","last_clean":"2023-10-26T05:49:03.628451+0000","last_became_active":"2023-10-26T05:49:02.589733+0000","last_became_peered":"2023-10-26T05:49:02.589733+0000","last_unstale":"2023-10-26T05:49:03.628451+0000","last_undegraded":"2023-10-26T05:49:03.628451+0000","last_fullsized":"2023-10-26T05:49:03.628451+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:42.232057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626662+0000","last_change":"2023-10-26T05:49:03.625150+0000","last_active":"2023-10-26T05:49:03.626662+0000","last_peered":"2023-10-26T05:49:03.626662+0000","last_clean":"2023-10-26T05:49:03.626662+0000","last_became_active":"2023-10-26T05:49:03.624344+0000","last_became_peered":"2023-10-26T05:49:03.624344+0000","last_unstale":"2023-10-26T05:49:03.626662+0000","last_undegraded":"2023-10-26T05:49:03.626662+0000","last_fullsized":"2023-10-26T05:49:03.626662+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:43.246142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620319+0000","last_change":"2023-10-26T05:49:02.581477+0000","last_active":"2023-10-26T05:49:03.620319+0000","last_peered":"2023-10-26T05:49:03.620319+0000","last_clean":"2023-10-26T05:49:03.620319+0000","last_became_active":"2023-10-26T05:49:02.580448+0000","last_became_peered":"2023-10-26T05:49:02.580448+0000","last_unstale":"2023-10-26T05:49:03.620319+0000","last_undegraded":"2023-10-26T05:49:03.620319+0000","last_fullsized":"2023-10-26T05:49:03.620319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:55.693336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.617618+0000","last_change":"2023-10-26T05:49:03.617551+0000","last_active":"2023-10-26T05:49:03.617618+0000","last_peered":"2023-10-26T05:49:03.617618+0000","last_clean":"2023-10-26T05:49:03.617618+0000","last_became_active":"2023-10-26T05:49:03.617252+0000","last_became_peered":"2023-10-26T05:49:03.617252+0000","last_unstale":"2023-10-26T05:49:03.617618+0000","last_undegraded":"2023-10-26T05:49:03.617618+0000","last_fullsized":"2023-10-26T05:49:03.617618+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:36.604418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.636925+0000","last_change":"2023-10-26T05:49:02.595139+0000","last_active":"2023-10-26T05:49:03.636925+0000","last_peered":"2023-10-26T05:49:03.636925+0000","last_clean":"2023-10-26T05:49:03.636925+0000","last_became_active":"2023-10-26T05:49:02.589877+0000","last_became_peered":"2023-10-26T05:49:02.589877+0000","last_unstale":"2023-10-26T05:49:03.636925+0000","last_undegraded":"2023-10-26T05:49:03.636925+0000","last_fullsized":"2023-10-26T05:49:03.636925+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:57.499803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.624524+0000","last_change":"2023-10-26T05:49:03.624076+0000","last_active":"2023-10-26T05:49:03.624524+0000","last_peered":"2023-10-26T05:49:03.624524+0000","last_clean":"2023-10-26T05:49:03.624524+0000","last_became_active":"2023-10-26T05:49:03.623066+0000","last_became_peered":"2023-10-26T05:49:03.623066+0000","last_unstale":"2023-10-26T05:49:03.624524+0000","last_undegraded":"2023-10-26T05:49:03.624524+0000","last_fullsized":"2023-10-26T05:49:03.624524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:56.606656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616138+0000","last_change":"2023-10-26T05:49:02.584779+0000","last_active":"2023-10-26T05:49:03.616138+0000","last_peered":"2023-10-26T05:49:03.616138+0000","last_clean":"2023-10-26T05:49:03.616138+0000","last_became_active":"2023-10-26T05:49:02.584565+0000","last_became_peered":"2023-10-26T05:49:02.584565+0000","last_unstale":"2023-10-26T05:49:03.616138+0000","last_undegraded":"2023-10-26T05:49:03.616138+0000","last_fullsized":"2023-10-26T05:49:03.616138+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:47.467126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.608619+0000","last_change":"2023-10-26T05:49:03.607733+0000","last_active":"2023-10-26T05:49:03.608619+0000","last_peered":"2023-10-26T05:49:03.608619+0000","last_clean":"2023-10-26T05:49:03.608619+0000","last_became_active":"2023-10-26T05:49:03.607528+0000","last_became_peered":"2023-10-26T05:49:03.607528+0000","last_unstale":"2023-10-26T05:49:03.608619+0000","last_undegraded":"2023-10-26T05:49:03.608619+0000","last_fullsized":"2023-10-26T05:49:03.608619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:36.193315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.630198+0000","last_change":"2023-10-26T05:49:02.582583+0000","last_active":"2023-10-26T05:49:03.630198+0000","last_peered":"2023-10-26T05:49:03.630198+0000","last_clean":"2023-10-26T05:49:03.630198+0000","last_became_active":"2023-10-26T05:49:02.581814+0000","last_became_peered":"2023-10-26T05:49:02.581814+0000","last_unstale":"2023-10-26T05:49:03.630198+0000","last_undegraded":"2023-10-26T05:49:03.630198+0000","last_fullsized":"2023-10-26T05:49:03.630198+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:05.329949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.618242+0000","last_change":"2023-10-26T05:49:03.618164+0000","last_active":"2023-10-26T05:49:03.618242+0000","last_peered":"2023-10-26T05:49:03.618242+0000","last_clean":"2023-10-26T05:49:03.618242+0000","last_became_active":"2023-10-26T05:49:03.617470+0000","last_became_peered":"2023-10-26T05:49:03.617470+0000","last_unstale":"2023-10-26T05:49:03.618242+0000","last_undegraded":"2023-10-26T05:49:03.618242+0000","last_fullsized":"2023-10-26T05:49:03.618242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:42.844061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.613180+0000","last_change":"2023-10-26T05:49:02.587615+0000","last_active":"2023-10-26T05:49:03.613180+0000","last_peered":"2023-10-26T05:49:03.613180+0000","last_clean":"2023-10-26T05:49:03.613180+0000","last_became_active":"2023-10-26T05:49:02.587060+0000","last_became_peered":"2023-10-26T05:49:02.587060+0000","last_unstale":"2023-10-26T05:49:03.613180+0000","last_undegraded":"2023-10-26T05:49:03.613180+0000","last_fullsized":"2023-10-26T05:49:03.613180+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:23.643656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.639834+0000","last_change":"2023-10-26T05:49:03.639695+0000","last_active":"2023-10-26T05:49:03.639834+0000","last_peered":"2023-10-26T05:49:03.639834+0000","last_clean":"2023-10-26T05:49:03.639834+0000","last_became_active":"2023-10-26T05:49:03.633815+0000","last_became_peered":"2023-10-26T05:49:03.633815+0000","last_unstale":"2023-10-26T05:49:03.639834+0000","last_undegraded":"2023-10-26T05:49:03.639834+0000","last_fullsized":"2023-10-26T05:49:03.639834+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:36:08.417695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.599708+0000","last_change":"2023-10-26T05:49:02.579062+0000","last_active":"2023-10-26T05:49:03.599708+0000","last_peered":"2023-10-26T05:49:03.599708+0000","last_clean":"2023-10-26T05:49:03.599708+0000","last_became_active":"2023-10-26T05:49:02.578404+0000","last_became_peered":"2023-10-26T05:49:02.578404+0000","last_unstale":"2023-10-26T05:49:03.599708+0000","last_undegraded":"2023-10-26T05:49:03.599708+0000","last_fullsized":"2023-10-26T05:49:03.599708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:38.198591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.602746+0000","last_change":"2023-10-26T05:49:03.602287+0000","last_active":"2023-10-26T05:49:03.602746+0000","last_peered":"2023-10-26T05:49:03.602746+0000","last_clean":"2023-10-26T05:49:03.602746+0000","last_became_active":"2023-10-26T05:49:03.601690+0000","last_became_peered":"2023-10-26T05:49:03.601690+0000","last_unstale":"2023-10-26T05:49:03.602746+0000","last_undegraded":"2023-10-26T05:49:03.602746+0000","last_fullsized":"2023-10-26T05:49:03.602746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:39.212676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.600746+0000","last_change":"2023-10-26T05:49:02.578207+0000","last_active":"2023-10-26T05:49:03.600746+0000","last_peered":"2023-10-26T05:49:03.600746+0000","last_clean":"2023-10-26T05:49:03.600746+0000","last_became_active":"2023-10-26T05:49:02.577456+0000","last_became_peered":"2023-10-26T05:49:02.577456+0000","last_unstale":"2023-10-26T05:49:03.600746+0000","last_undegraded":"2023-10-26T05:49:03.600746+0000","last_fullsized":"2023-10-26T05:49:03.600746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:57.499803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.631136+0000","last_change":"2023-10-26T05:49:03.630941+0000","last_active":"2023-10-26T05:49:03.631136+0000","last_peered":"2023-10-26T05:49:03.631136+0000","last_clean":"2023-10-26T05:49:03.631136+0000","last_became_active":"2023-10-26T05:49:03.630088+0000","last_became_peered":"2023-10-26T05:49:03.630088+0000","last_unstale":"2023-10-26T05:49:03.631136+0000","last_undegraded":"2023-10-26T05:49:03.631136+0000","last_fullsized":"2023-10-26T05:49:03.631136+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:24.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.599370+0000","last_change":"2023-10-26T05:49:02.579468+0000","last_active":"2023-10-26T05:49:03.599370+0000","last_peered":"2023-10-26T05:49:03.599370+0000","last_clean":"2023-10-26T05:49:03.599370+0000","last_became_active":"2023-10-26T05:49:02.578778+0000","last_became_peered":"2023-10-26T05:49:02.578778+0000","last_unstale":"2023-10-26T05:49:03.599370+0000","last_undegraded":"2023-10-26T05:49:03.599370+0000","last_fullsized":"2023-10-26T05:49:03.599370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:55.693336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616417+0000","last_change":"2023-10-26T05:49:03.616285+0000","last_active":"2023-10-26T05:49:03.616417+0000","last_peered":"2023-10-26T05:49:03.616417+0000","last_clean":"2023-10-26T05:49:03.616417+0000","last_became_active":"2023-10-26T05:49:03.615695+0000","last_became_peered":"2023-10-26T05:49:03.615695+0000","last_unstale":"2023-10-26T05:49:03.616417+0000","last_undegraded":"2023-10-26T05:49:03.616417+0000","last_fullsized":"2023-10-26T05:49:03.616417+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:21.204226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.641355+0000","last_change":"2023-10-26T05:49:02.590859+0000","last_active":"2023-10-26T05:49:03.641355+0000","last_peered":"2023-10-26T05:49:03.641355+0000","last_clean":"2023-10-26T05:49:03.641355+0000","last_became_active":"2023-10-26T05:49:02.590216+0000","last_became_peered":"2023-10-26T05:49:02.590216+0000","last_unstale":"2023-10-26T05:49:03.641355+0000","last_undegraded":"2023-10-26T05:49:03.641355+0000","last_fullsized":"2023-10-26T05:49:03.641355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:24:44.970427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.612377+0000","last_change":"2023-10-26T05:49:03.612220+0000","last_active":"2023-10-26T05:49:03.612377+0000","last_peered":"2023-10-26T05:49:03.612377+0000","last_clean":"2023-10-26T05:49:03.612377+0000","last_became_active":"2023-10-26T05:49:03.611605+0000","last_became_peered":"2023-10-26T05:49:03.611605+0000","last_unstale":"2023-10-26T05:49:03.612377+0000","last_undegraded":"2023-10-26T05:49:03.612377+0000","last_fullsized":"2023-10-26T05:49:03.612377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:54:46.778631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.619967+0000","last_change":"2023-10-26T05:49:02.578690+0000","last_active":"2023-10-26T05:49:03.619967+0000","last_peered":"2023-10-26T05:49:03.619967+0000","last_clean":"2023-10-26T05:49:03.619967+0000","last_became_active":"2023-10-26T05:49:02.578385+0000","last_became_peered":"2023-10-26T05:49:02.578385+0000","last_unstale":"2023-10-26T05:49:03.619967+0000","last_undegraded":"2023-10-26T05:49:03.619967+0000","last_fullsized":"2023-10-26T05:49:03.619967+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:43.573562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621711+0000","last_change":"2023-10-26T05:49:03.621261+0000","last_active":"2023-10-26T05:49:03.621711+0000","last_peered":"2023-10-26T05:49:03.621711+0000","last_clean":"2023-10-26T05:49:03.621711+0000","last_became_active":"2023-10-26T05:49:03.619502+0000","last_became_peered":"2023-10-26T05:49:03.619502+0000","last_unstale":"2023-10-26T05:49:03.621711+0000","last_undegraded":"2023-10-26T05:49:03.621711+0000","last_fullsized":"2023-10-26T05:49:03.621711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:24.672797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.615429+0000","last_change":"2023-10-26T05:49:02.582599+0000","last_active":"2023-10-26T05:49:03.615429+0000","last_peered":"2023-10-26T05:49:03.615429+0000","last_clean":"2023-10-26T05:49:03.615429+0000","last_became_active":"2023-10-26T05:49:02.581943+0000","last_became_peered":"2023-10-26T05:49:02.581943+0000","last_unstale":"2023-10-26T05:49:03.615429+0000","last_undegraded":"2023-10-26T05:49:03.615429+0000","last_fullsized":"2023-10-26T05:49:03.615429+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:57:05.628834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606521+0000","last_change":"2023-10-26T05:49:03.604966+0000","last_active":"2023-10-26T05:49:03.606521+0000","last_peered":"2023-10-26T05:49:03.606521+0000","last_clean":"2023-10-26T05:49:03.606521+0000","last_became_active":"2023-10-26T05:49:03.604335+0000","last_became_peered":"2023-10-26T05:49:03.604335+0000","last_unstale":"2023-10-26T05:49:03.606521+0000","last_undegraded":"2023-10-26T05:49:03.606521+0000","last_fullsized":"2023-10-26T05:49:03.606521+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:36:40.893138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.603841+0000","last_change":"2023-10-26T05:49:02.595617+0000","last_active":"2023-10-26T05:49:03.603841+0000","last_peered":"2023-10-26T05:49:03.603841+0000","last_clean":"2023-10-26T05:49:03.603841+0000","last_became_active":"2023-10-26T05:49:02.592539+0000","last_became_peered":"2023-10-26T05:49:02.592539+0000","last_unstale":"2023-10-26T05:49:03.603841+0000","last_undegraded":"2023-10-26T05:49:03.603841+0000","last_fullsized":"2023-10-26T05:49:03.603841+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:39.535504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609073+0000","last_change":"2023-10-26T05:49:03.608454+0000","last_active":"2023-10-26T05:49:03.609073+0000","last_peered":"2023-10-26T05:49:03.609073+0000","last_clean":"2023-10-26T05:49:03.609073+0000","last_became_active":"2023-10-26T05:49:03.608045+0000","last_became_peered":"2023-10-26T05:49:03.608045+0000","last_unstale":"2023-10-26T05:49:03.609073+0000","last_undegraded":"2023-10-26T05:49:03.609073+0000","last_fullsized":"2023-10-26T05:49:03.609073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:58.513888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606167+0000","last_change":"2023-10-26T05:49:02.575131+0000","last_active":"2023-10-26T05:49:03.606167+0000","last_peered":"2023-10-26T05:49:03.606167+0000","last_clean":"2023-10-26T05:49:03.606167+0000","last_became_active":"2023-10-26T05:49:02.574318+0000","last_became_peered":"2023-10-26T05:49:02.574318+0000","last_unstale":"2023-10-26T05:49:03.606167+0000","last_undegraded":"2023-10-26T05:49:03.606167+0000","last_fullsized":"2023-10-26T05:49:03.606167+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:04.382797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.637702+0000","last_change":"2023-10-26T05:49:03.637444+0000","last_active":"2023-10-26T05:49:03.637702+0000","last_peered":"2023-10-26T05:49:03.637702+0000","last_clean":"2023-10-26T05:49:03.637702+0000","last_became_active":"2023-10-26T05:49:03.636919+0000","last_became_peered":"2023-10-26T05:49:03.636919+0000","last_unstale":"2023-10-26T05:49:03.637702+0000","last_undegraded":"2023-10-26T05:49:03.637702+0000","last_fullsized":"2023-10-26T05:49:03.637702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621823+0000","last_change":"2023-10-26T05:49:02.574731+0000","last_active":"2023-10-26T05:49:03.621823+0000","last_peered":"2023-10-26T05:49:03.621823+0000","last_clean":"2023-10-26T05:49:03.621823+0000","last_became_active":"2023-10-26T05:49:02.574503+0000","last_became_peered":"2023-10-26T05:49:02.574503+0000","last_unstale":"2023-10-26T05:49:03.621823+0000","last_undegraded":"2023-10-26T05:49:03.621823+0000","last_fullsized":"2023-10-26T05:49:03.621823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:43.117799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.627668+0000","last_change":"2023-10-26T05:49:03.627608+0000","last_active":"2023-10-26T05:49:03.627668+0000","last_peered":"2023-10-26T05:49:03.627668+0000","last_clean":"2023-10-26T05:49:03.627668+0000","last_became_active":"2023-10-26T05:49:03.627450+0000","last_became_peered":"2023-10-26T05:49:03.627450+0000","last_unstale":"2023-10-26T05:49:03.627668+0000","last_undegraded":"2023-10-26T05:49:03.627668+0000","last_fullsized":"2023-10-26T05:49:03.627668+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:24.672797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.617730+0000","last_change":"2023-10-26T05:49:02.584218+0000","last_active":"2023-10-26T05:49:03.617730+0000","last_peered":"2023-10-26T05:49:03.617730+0000","last_clean":"2023-10-26T05:49:03.617730+0000","last_became_active":"2023-10-26T05:49:02.583933+0000","last_became_peered":"2023-10-26T05:49:02.583933+0000","last_unstale":"2023-10-26T05:49:03.617730+0000","last_undegraded":"2023-10-26T05:49:03.617730+0000","last_fullsized":"2023-10-26T05:49:03.617730+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:21.402128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609959+0000","last_change":"2023-10-26T05:49:03.609814+0000","last_active":"2023-10-26T05:49:03.609959+0000","last_peered":"2023-10-26T05:49:03.609959+0000","last_clean":"2023-10-26T05:49:03.609959+0000","last_became_active":"2023-10-26T05:49:03.609622+0000","last_became_peered":"2023-10-26T05:49:03.609622+0000","last_unstale":"2023-10-26T05:49:03.609959+0000","last_undegraded":"2023-10-26T05:49:03.609959+0000","last_fullsized":"2023-10-26T05:49:03.609959+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:13.301659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616861+0000","last_change":"2023-10-26T05:49:02.571804+0000","last_active":"2023-10-26T05:49:03.616861+0000","last_peered":"2023-10-26T05:49:03.616861+0000","last_clean":"2023-10-26T05:49:03.616861+0000","last_became_active":"2023-10-26T05:49:02.571014+0000","last_became_peered":"2023-10-26T05:49:02.571014+0000","last_unstale":"2023-10-26T05:49:03.616861+0000","last_undegraded":"2023-10-26T05:49:03.616861+0000","last_fullsized":"2023-10-26T05:49:03.616861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:52.578972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.635515+0000","last_change":"2023-10-26T05:49:03.635293+0000","last_active":"2023-10-26T05:49:03.635515+0000","last_peered":"2023-10-26T05:49:03.635515+0000","last_clean":"2023-10-26T05:49:03.635515+0000","last_became_active":"2023-10-26T05:49:03.634169+0000","last_became_peered":"2023-10-26T05:49:03.634169+0000","last_unstale":"2023-10-26T05:49:03.635515+0000","last_undegraded":"2023-10-26T05:49:03.635515+0000","last_fullsized":"2023-10-26T05:49:03.635515+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:58:38.756380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.635908+0000","last_change":"2023-10-26T05:49:02.587219+0000","last_active":"2023-10-26T05:49:03.635908+0000","last_peered":"2023-10-26T05:49:03.635908+0000","last_clean":"2023-10-26T05:49:03.635908+0000","last_became_active":"2023-10-26T05:49:02.586237+0000","last_became_peered":"2023-10-26T05:49:02.586237+0000","last_unstale":"2023-10-26T05:49:03.635908+0000","last_undegraded":"2023-10-26T05:49:03.635908+0000","last_fullsized":"2023-10-26T05:49:03.635908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:54:45.764547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621657+0000","last_change":"2023-10-26T05:49:03.620903+0000","last_active":"2023-10-26T05:49:03.621657+0000","last_peered":"2023-10-26T05:49:03.621657+0000","last_clean":"2023-10-26T05:49:03.621657+0000","last_became_active":"2023-10-26T05:49:03.618838+0000","last_became_peered":"2023-10-26T05:49:03.618838+0000","last_unstale":"2023-10-26T05:49:03.621657+0000","last_undegraded":"2023-10-26T05:49:03.621657+0000","last_fullsized":"2023-10-26T05:49:03.621657+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:17.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606292+0000","last_change":"2023-10-26T05:49:02.588033+0000","last_active":"2023-10-26T05:49:03.606292+0000","last_peered":"2023-10-26T05:49:03.606292+0000","last_clean":"2023-10-26T05:49:03.606292+0000","last_became_active":"2023-10-26T05:49:02.587683+0000","last_became_peered":"2023-10-26T05:49:02.587683+0000","last_unstale":"2023-10-26T05:49:03.606292+0000","last_undegraded":"2023-10-26T05:49:03.606292+0000","last_fullsized":"2023-10-26T05:49:03.606292+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:38.628722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616426+0000","last_change":"2023-10-26T05:49:03.616378+0000","last_active":"2023-10-26T05:49:03.616426+0000","last_peered":"2023-10-26T05:49:03.616426+0000","last_clean":"2023-10-26T05:49:03.616426+0000","last_became_active":"2023-10-26T05:49:03.615942+0000","last_became_peered":"2023-10-26T05:49:03.615942+0000","last_unstale":"2023-10-26T05:49:03.616426+0000","last_undegraded":"2023-10-26T05:49:03.616426+0000","last_fullsized":"2023-10-26T05:49:03.616426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:22.619812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620937+0000","last_change":"2023-10-26T05:49:02.585731+0000","last_active":"2023-10-26T05:49:03.620937+0000","last_peered":"2023-10-26T05:49:03.620937+0000","last_clean":"2023-10-26T05:49:03.620937+0000","last_became_active":"2023-10-26T05:49:02.585422+0000","last_became_peered":"2023-10-26T05:49:02.585422+0000","last_unstale":"2023-10-26T05:49:03.620937+0000","last_undegraded":"2023-10-26T05:49:03.620937+0000","last_fullsized":"2023-10-26T05:49:03.620937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:45.401293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620968+0000","last_change":"2023-10-26T05:49:03.620885+0000","last_active":"2023-10-26T05:49:03.620968+0000","last_peered":"2023-10-26T05:49:03.620968+0000","last_clean":"2023-10-26T05:49:03.620968+0000","last_became_active":"2023-10-26T05:49:03.620586+0000","last_became_peered":"2023-10-26T05:49:03.620586+0000","last_unstale":"2023-10-26T05:49:03.620968+0000","last_undegraded":"2023-10-26T05:49:03.620968+0000","last_fullsized":"2023-10-26T05:49:03.620968+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:42.052128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.613496+0000","last_change":"2023-10-26T05:49:02.591418+0000","last_active":"2023-10-26T05:49:03.613496+0000","last_peered":"2023-10-26T05:49:03.613496+0000","last_clean":"2023-10-26T05:49:03.613496+0000","last_became_active":"2023-10-26T05:49:02.589938+0000","last_became_peered":"2023-10-26T05:49:02.589938+0000","last_unstale":"2023-10-26T05:49:03.613496+0000","last_undegraded":"2023-10-26T05:49:03.613496+0000","last_fullsized":"2023-10-26T05:49:03.613496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:05.329949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.614683+0000","last_change":"2023-10-26T05:49:03.614564+0000","last_active":"2023-10-26T05:49:03.614683+0000","last_peered":"2023-10-26T05:49:03.614683+0000","last_clean":"2023-10-26T05:49:03.614683+0000","last_became_active":"2023-10-26T05:49:03.614245+0000","last_became_peered":"2023-10-26T05:49:03.614245+0000","last_unstale":"2023-10-26T05:49:03.614683+0000","last_undegraded":"2023-10-26T05:49:03.614683+0000","last_fullsized":"2023-10-26T05:49:03.614683+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:42.192405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.599822+0000","last_change":"2023-10-26T05:49:02.577364+0000","last_active":"2023-10-26T05:49:03.599822+0000","last_peered":"2023-10-26T05:49:03.599822+0000","last_clean":"2023-10-26T05:49:03.599822+0000","last_became_active":"2023-10-26T05:49:02.576874+0000","last_became_peered":"2023-10-26T05:49:02.576874+0000","last_unstale":"2023-10-26T05:49:03.599822+0000","last_undegraded":"2023-10-26T05:49:03.599822+0000","last_fullsized":"2023-10-26T05:49:03.599822+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:52.578972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606488+0000","last_change":"2023-10-26T05:49:03.604623+0000","last_active":"2023-10-26T05:49:03.606488+0000","last_peered":"2023-10-26T05:49:03.606488+0000","last_clean":"2023-10-26T05:49:03.606488+0000","last_became_active":"2023-10-26T05:49:03.603551+0000","last_became_peered":"2023-10-26T05:49:03.603551+0000","last_unstale":"2023-10-26T05:49:03.606488+0000","last_undegraded":"2023-10-26T05:49:03.606488+0000","last_fullsized":"2023-10-26T05:49:03.606488+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:54.512246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604188+0000","last_change":"2023-10-26T05:49:02.596028+0000","last_active":"2023-10-26T05:49:03.604188+0000","last_peered":"2023-10-26T05:49:03.604188+0000","last_clean":"2023-10-26T05:49:03.604188+0000","last_became_active":"2023-10-26T05:49:02.593417+0000","last_became_peered":"2023-10-26T05:49:02.593417+0000","last_unstale":"2023-10-26T05:49:03.604188+0000","last_undegraded":"2023-10-26T05:49:03.604188+0000","last_fullsized":"2023-10-26T05:49:03.604188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:57:05.628834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.634970+0000","last_change":"2023-10-26T05:49:03.634816+0000","last_active":"2023-10-26T05:49:03.634970+0000","last_peered":"2023-10-26T05:49:03.634970+0000","last_clean":"2023-10-26T05:49:03.634970+0000","last_became_active":"2023-10-26T05:49:03.632307+0000","last_became_peered":"2023-10-26T05:49:03.632307+0000","last_unstale":"2023-10-26T05:49:03.634970+0000","last_undegraded":"2023-10-26T05:49:03.634970+0000","last_fullsized":"2023-10-26T05:49:03.634970+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:10.235795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.603923+0000","last_change":"2023-10-26T05:49:02.582939+0000","last_active":"2023-10-26T05:49:03.603923+0000","last_peered":"2023-10-26T05:49:03.603923+0000","last_clean":"2023-10-26T05:49:03.603923+0000","last_became_active":"2023-10-26T05:49:02.582477+0000","last_became_peered":"2023-10-26T05:49:02.582477+0000","last_unstale":"2023-10-26T05:49:03.603923+0000","last_undegraded":"2023-10-26T05:49:03.603923+0000","last_fullsized":"2023-10-26T05:49:03.603923+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:19.215900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606504+0000","last_change":"2023-10-26T05:49:03.604796+0000","last_active":"2023-10-26T05:49:03.606504+0000","last_peered":"2023-10-26T05:49:03.606504+0000","last_clean":"2023-10-26T05:49:03.606504+0000","last_became_active":"2023-10-26T05:49:03.603948+0000","last_became_peered":"2023-10-26T05:49:03.603948+0000","last_unstale":"2023-10-26T05:49:03.606504+0000","last_undegraded":"2023-10-26T05:49:03.606504+0000","last_fullsized":"2023-10-26T05:49:03.606504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:58:38.756380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.597783+0000","last_change":"2023-10-26T05:49:02.591231+0000","last_active":"2023-10-26T05:49:03.597783+0000","last_peered":"2023-10-26T05:49:03.597783+0000","last_clean":"2023-10-26T05:49:03.597783+0000","last_became_active":"2023-10-26T05:49:02.590628+0000","last_became_peered":"2023-10-26T05:49:02.590628+0000","last_unstale":"2023-10-26T05:49:03.597783+0000","last_undegraded":"2023-10-26T05:49:03.597783+0000","last_fullsized":"2023-10-26T05:49:03.597783+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:06.851599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609477+0000","last_change":"2023-10-26T05:49:03.608673+0000","last_active":"2023-10-26T05:49:03.609477+0000","last_peered":"2023-10-26T05:49:03.609477+0000","last_clean":"2023-10-26T05:49:03.609477+0000","last_became_active":"2023-10-26T05:49:03.608012+0000","last_became_peered":"2023-10-26T05:49:03.608012+0000","last_unstale":"2023-10-26T05:49:03.609477+0000","last_undegraded":"2023-10-26T05:49:03.609477+0000","last_fullsized":"2023-10-26T05:49:03.609477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:53.593057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.594591+0000","last_change":"2023-10-26T05:49:02.581625+0000","last_active":"2023-10-26T05:49:03.594591+0000","last_peered":"2023-10-26T05:49:03.594591+0000","last_clean":"2023-10-26T05:49:03.594591+0000","last_became_active":"2023-10-26T05:49:02.581076+0000","last_became_peered":"2023-10-26T05:49:02.581076+0000","last_unstale":"2023-10-26T05:49:03.594591+0000","last_undegraded":"2023-10-26T05:49:03.594591+0000","last_fullsized":"2023-10-26T05:49:03.594591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:38.198591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609099+0000","last_change":"2023-10-26T05:49:03.608970+0000","last_active":"2023-10-26T05:49:03.609099+0000","last_peered":"2023-10-26T05:49:03.609099+0000","last_clean":"2023-10-26T05:49:03.609099+0000","last_became_active":"2023-10-26T05:49:03.608612+0000","last_became_peered":"2023-10-26T05:49:03.608612+0000","last_unstale":"2023-10-26T05:49:03.609099+0000","last_undegraded":"2023-10-26T05:49:03.609099+0000","last_fullsized":"2023-10-26T05:49:03.609099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:43.246142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604491+0000","last_change":"2023-10-26T05:49:02.596545+0000","last_active":"2023-10-26T05:49:03.604491+0000","last_peered":"2023-10-26T05:49:03.604491+0000","last_clean":"2023-10-26T05:49:03.604491+0000","last_became_active":"2023-10-26T05:49:02.594398+0000","last_became_peered":"2023-10-26T05:49:02.594398+0000","last_unstale":"2023-10-26T05:49:03.604491+0000","last_undegraded":"2023-10-26T05:49:03.604491+0000","last_fullsized":"2023-10-26T05:49:03.604491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:11.532031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.607072+0000","last_change":"2023-10-26T05:49:03.606989+0000","last_active":"2023-10-26T05:49:03.607072+0000","last_peered":"2023-10-26T05:49:03.607072+0000","last_clean":"2023-10-26T05:49:03.607072+0000","last_became_active":"2023-10-26T05:49:03.606366+0000","last_became_peered":"2023-10-26T05:49:03.606366+0000","last_unstale":"2023-10-26T05:49:03.607072+0000","last_undegraded":"2023-10-26T05:49:03.607072+0000","last_fullsized":"2023-10-26T05:49:03.607072+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:24.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.598884+0000","last_change":"2023-10-26T05:49:02.581599+0000","last_active":"2023-10-26T05:49:03.598884+0000","last_peered":"2023-10-26T05:49:03.598884+0000","last_clean":"2023-10-26T05:49:03.598884+0000","last_became_active":"2023-10-26T05:49:02.580235+0000","last_became_peered":"2023-10-26T05:49:02.580235+0000","last_unstale":"2023-10-26T05:49:03.598884+0000","last_undegraded":"2023-10-26T05:49:03.598884+0000","last_fullsized":"2023-10-26T05:49:03.598884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:06.851599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.622708+0000","last_change":"2023-10-26T05:49:03.620491+0000","last_active":"2023-10-26T05:49:03.622708+0000","last_peered":"2023-10-26T05:49:03.622708+0000","last_clean":"2023-10-26T05:49:03.622708+0000","last_became_active":"2023-10-26T05:49:03.619862+0000","last_became_peered":"2023-10-26T05:49:03.619862+0000","last_unstale":"2023-10-26T05:49:03.622708+0000","last_undegraded":"2023-10-26T05:49:03.622708+0000","last_fullsized":"2023-10-26T05:49:03.622708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:56.707421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.593824+0000","last_change":"2023-10-26T05:49:02.588587+0000","last_active":"2023-10-26T05:49:03.593824+0000","last_peered":"2023-10-26T05:49:03.593824+0000","last_clean":"2023-10-26T05:49:03.593824+0000","last_became_active":"2023-10-26T05:49:02.587943+0000","last_became_peered":"2023-10-26T05:49:02.587943+0000","last_unstale":"2023-10-26T05:49:03.593824+0000","last_undegraded":"2023-10-26T05:49:03.593824+0000","last_fullsized":"2023-10-26T05:49:03.593824+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:41.829976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609451+0000","last_change":"2023-10-26T05:49:03.609183+0000","last_active":"2023-10-26T05:49:03.609451+0000","last_peered":"2023-10-26T05:49:03.609451+0000","last_clean":"2023-10-26T05:49:03.609451+0000","last_became_active":"2023-10-26T05:49:03.608050+0000","last_became_peered":"2023-10-26T05:49:03.608050+0000","last_unstale":"2023-10-26T05:49:03.609451+0000","last_undegraded":"2023-10-26T05:49:03.609451+0000","last_fullsized":"2023-10-26T05:49:03.609451+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:44.388032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.598665+0000","last_change":"2023-10-26T05:49:02.573176+0000","last_active":"2023-10-26T05:49:03.598665+0000","last_peered":"2023-10-26T05:49:03.598665+0000","last_clean":"2023-10-26T05:49:03.598665+0000","last_became_active":"2023-10-26T05:49:02.572710+0000","last_became_peered":"2023-10-26T05:49:02.572710+0000","last_unstale":"2023-10-26T05:49:03.598665+0000","last_undegraded":"2023-10-26T05:49:03.598665+0000","last_fullsized":"2023-10-26T05:49:03.598665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:57:53.394547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.611674+0000","last_change":"2023-10-26T05:49:03.611200+0000","last_active":"2023-10-26T05:49:03.611674+0000","last_peered":"2023-10-26T05:49:03.611674+0000","last_clean":"2023-10-26T05:49:03.611674+0000","last_became_active":"2023-10-26T05:49:03.610952+0000","last_became_peered":"2023-10-26T05:49:03.610952+0000","last_unstale":"2023-10-26T05:49:03.611674+0000","last_undegraded":"2023-10-26T05:49:03.611674+0000","last_fullsized":"2023-10-26T05:49:03.611674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:47.718823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.636324+0000","last_change":"2023-10-26T05:49:02.580467+0000","last_active":"2023-10-26T05:49:03.636324+0000","last_peered":"2023-10-26T05:49:03.636324+0000","last_clean":"2023-10-26T05:49:03.636324+0000","last_became_active":"2023-10-26T05:49:02.579843+0000","last_became_peered":"2023-10-26T05:49:02.579843+0000","last_unstale":"2023-10-26T05:49:03.636324+0000","last_undegraded":"2023-10-26T05:49:03.636324+0000","last_fullsized":"2023-10-26T05:49:03.636324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:57.499803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":188,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-26T05:49:02.574701+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:49:02.574701+0000","last_peered":"2023-10-26T05:49:02.574701+0000","last_clean":"2023-10-26T05:49:02.574701+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:49:02.574701+0000","last_undegraded":"2023-10-26T05:49:02.574701+0000","last_fullsized":"2023-10-26T05:49:02.574701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":797,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194515,"statfs":{"total":200000000,"available":199183450,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.262}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.135},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0629999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.81999999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2250000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.75}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.026},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.016},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.23}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.085}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.254}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.056},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":1.1180000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.204},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.2869999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":220,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773908,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.875}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.91500000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0529999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.95099999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1020000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199772296,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.075}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.88400000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.0249999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.121}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773270,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.438}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.65200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195098,"statfs":{"total":200000000,"available":199780683,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":1.1950000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.26700000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":2.8809999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":2.6139999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":5.9550000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":2.8319999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.046}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":2.5760000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":6.7729999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":15.352}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195098,"statfs":{"total":200000000,"available":199780683,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.497}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.107}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.141},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.026},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.2270000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194521,"statfs":{"total":200000000,"available":199190315,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79000000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.753}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98499999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:49:06.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:49:06.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3091649285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 msgr2=0x7ff54006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 --2- 172.21.15.71:0/3091649285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff54006aa70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff54008a140 tx=0x7ff53401c780 comp rx=0 tx=0).stop 2023-10-26T05:49:06.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3091649285 shutdown_connections 2023-10-26T05:49:06.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 --2- 172.21.15.71:0/3091649285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5400cd880 0x7ff54015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 --2- 172.21.15.71:0/3091649285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff54006aa70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3091649285 >> 172.21.15.71:0/3091649285 conn(0x7ff5400cdd80 msgr2=0x7ff5400ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3091649285 shutdown_connections 2023-10-26T05:49:06.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3091649285 wait complete. 2023-10-26T05:49:06.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 Processor -- start 2023-10-26T05:49:06.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- start start 2023-10-26T05:49:06.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5400cd880 0x7ff540159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff540150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5400d0850 0x7ff54014d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54014e460 con 0x7ff5400d0470 2023-10-26T05:49:06.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54014e740 con 0x7ff5400d0850 2023-10-26T05:49:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54014ea20 con 0x7ff5400cd880 2023-10-26T05:49:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff540150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff540150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52784/0 (socket says 172.21.15.71:52784) 2023-10-26T05:49:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 -- 172.21.15.71:0/3361079615 learned_addr learned my addr 172.21.15.71:0/3361079615 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 -- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5400cd880 msgr2=0x7ff540159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5400cd880 0x7ff540159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff544deb640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5400d0850 0x7ff54014d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 -- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5400d0850 msgr2=0x7ff54014d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5400d0850 0x7ff54014d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 -- 172.21.15.71:0/3361079615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534009560 con 0x7ff5400d0470 2023-10-26T05:49:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53f7fe640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff540150200 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff53000b410 tx=0x7ff53000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530004020 con 0x7ff5400d0470 2023-10-26T05:49:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff530004270 con 0x7ff5400d0470 2023-10-26T05:49:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54014ed00 con 0x7ff5400d0470 2023-10-26T05:49:06.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530011890 con 0x7ff5400d0470 2023-10-26T05:49:06.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54014f850 con 0x7ff5400d0470 2023-10-26T05:49:06.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.813+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540066b10 con 0x7ff5400d0470 2023-10-26T05:49:06.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.817+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff530003600 con 0x7ff5400d0470 2023-10-26T05:49:06.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.817+0000 7ff53cff9640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff52c0664a0 0x7ff52c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:06.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.817+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff530083de0 con 0x7ff5400d0470 2023-10-26T05:49:06.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.817+0000 7ff53ffff640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff52c0664a0 0x7ff52c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:06.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.817+0000 7ff53ffff640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff52c0664a0 0x7ff52c068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff5340af000 tx=0x7ff534019520 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:06.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.821+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5300512f0 con 0x7ff5400d0470 2023-10-26T05:49:06.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.957+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff54014f220 con 0x7ff52c0664a0 2023-10-26T05:49:06.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff53cff9640 1 -- 172.21.15.71:0/3361079615 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283664 (secure 0 0 0) 0x7ff54014f220 con 0x7ff52c0664a0 2023-10-26T05:49:06.988 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:49:06.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff52c0664a0 msgr2=0x7ff52c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff52c0664a0 0x7ff52c068960 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff5340af000 tx=0x7ff534019520 comp rx=0 tx=0).stop 2023-10-26T05:49:06.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 msgr2=0x7ff540150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:06.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff540150200 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff53000b410 tx=0x7ff53000b8d0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 shutdown_connections 2023-10-26T05:49:06.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5400d0850 0x7ff54014d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff52c0664a0 0x7ff52c068960 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5400d0470 0x7ff540150200 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 --2- 172.21.15.71:0/3361079615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5400cd880 0x7ff540159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:06.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 >> 172.21.15.71:0/3361079615 conn(0x7ff5400cdd80 msgr2=0x7ff5400ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:06.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 shutdown_connections 2023-10-26T05:49:06.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:06.981+0000 7ff54684e640 1 -- 172.21.15.71:0/3361079615 wait complete. 2023-10-26T05:49:07.002 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-10-26T05:49:05.446608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2902,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559597,"statfs":{"total":1600000000,"available":1597027841,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001103"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609426+0000","last_change":"2023-10-26T05:49:03.607461+0000","last_active":"2023-10-26T05:49:03.609426+0000","last_peered":"2023-10-26T05:49:03.609426+0000","last_clean":"2023-10-26T05:49:03.609426+0000","last_became_active":"2023-10-26T05:49:03.605415+0000","last_became_peered":"2023-10-26T05:49:03.605415+0000","last_unstale":"2023-10-26T05:49:03.609426+0000","last_undegraded":"2023-10-26T05:49:03.609426+0000","last_fullsized":"2023-10-26T05:49:03.609426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:44.388032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626757+0000","last_change":"2023-10-26T05:49:03.625701+0000","last_active":"2023-10-26T05:49:03.626757+0000","last_peered":"2023-10-26T05:49:03.626757+0000","last_clean":"2023-10-26T05:49:03.626757+0000","last_became_active":"2023-10-26T05:49:03.623694+0000","last_became_peered":"2023-10-26T05:49:03.623694+0000","last_unstale":"2023-10-26T05:49:03.626757+0000","last_undegraded":"2023-10-26T05:49:03.626757+0000","last_fullsized":"2023-10-26T05:49:03.626757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:07.865684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609329+0000","last_change":"2023-10-26T05:49:03.608806+0000","last_active":"2023-10-26T05:49:03.609329+0000","last_peered":"2023-10-26T05:49:03.609329+0000","last_clean":"2023-10-26T05:49:03.609329+0000","last_became_active":"2023-10-26T05:49:03.606696+0000","last_became_peered":"2023-10-26T05:49:03.606696+0000","last_unstale":"2023-10-26T05:49:03.609329+0000","last_undegraded":"2023-10-26T05:49:03.609329+0000","last_fullsized":"2023-10-26T05:49:03.609329+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:36.604418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.633642+0000","last_change":"2023-10-26T05:49:03.633583+0000","last_active":"2023-10-26T05:49:03.633642+0000","last_peered":"2023-10-26T05:49:03.633642+0000","last_clean":"2023-10-26T05:49:03.633642+0000","last_became_active":"2023-10-26T05:49:03.633329+0000","last_became_peered":"2023-10-26T05:49:03.633329+0000","last_unstale":"2023-10-26T05:49:03.633642+0000","last_undegraded":"2023-10-26T05:49:03.633642+0000","last_fullsized":"2023-10-26T05:49:03.633642+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:17.113253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.627989+0000","last_change":"2023-10-26T05:49:03.627526+0000","last_active":"2023-10-26T05:49:03.627989+0000","last_peered":"2023-10-26T05:49:03.627989+0000","last_clean":"2023-10-26T05:49:03.627989+0000","last_became_active":"2023-10-26T05:49:03.620638+0000","last_became_peered":"2023-10-26T05:49:03.620638+0000","last_unstale":"2023-10-26T05:49:03.627989+0000","last_undegraded":"2023-10-26T05:49:03.627989+0000","last_fullsized":"2023-10-26T05:49:03.627989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:44.388032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.618923+0000","last_change":"2023-10-26T05:49:03.618749+0000","last_active":"2023-10-26T05:49:03.618923+0000","last_peered":"2023-10-26T05:49:03.618923+0000","last_clean":"2023-10-26T05:49:03.618923+0000","last_became_active":"2023-10-26T05:49:03.618399+0000","last_became_peered":"2023-10-26T05:49:03.618399+0000","last_unstale":"2023-10-26T05:49:03.618923+0000","last_undegraded":"2023-10-26T05:49:03.618923+0000","last_fullsized":"2023-10-26T05:49:03.618923+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:40.549589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606555+0000","last_change":"2023-10-26T05:49:03.604123+0000","last_active":"2023-10-26T05:49:03.606555+0000","last_peered":"2023-10-26T05:49:03.606555+0000","last_clean":"2023-10-26T05:49:03.606555+0000","last_became_active":"2023-10-26T05:49:03.603512+0000","last_became_peered":"2023-10-26T05:49:03.603512+0000","last_unstale":"2023-10-26T05:49:03.606555+0000","last_undegraded":"2023-10-26T05:49:03.606555+0000","last_fullsized":"2023-10-26T05:49:03.606555+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:43.246142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.622309+0000","last_change":"2023-10-26T05:49:03.622243+0000","last_active":"2023-10-26T05:49:03.622309+0000","last_peered":"2023-10-26T05:49:03.622309+0000","last_clean":"2023-10-26T05:49:03.622309+0000","last_became_active":"2023-10-26T05:49:03.622019+0000","last_became_peered":"2023-10-26T05:49:03.622019+0000","last_unstale":"2023-10-26T05:49:03.622309+0000","last_undegraded":"2023-10-26T05:49:03.622309+0000","last_fullsized":"2023-10-26T05:49:03.622309+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:10.235795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.600548+0000","last_change":"2023-10-26T05:49:03.599891+0000","last_active":"2023-10-26T05:49:03.600548+0000","last_peered":"2023-10-26T05:49:03.600548+0000","last_clean":"2023-10-26T05:49:03.600548+0000","last_became_active":"2023-10-26T05:49:03.599663+0000","last_became_peered":"2023-10-26T05:49:03.599663+0000","last_unstale":"2023-10-26T05:49:03.600548+0000","last_undegraded":"2023-10-26T05:49:03.600548+0000","last_fullsized":"2023-10-26T05:49:03.600548+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:16:19.574921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.618420+0000","last_change":"2023-10-26T05:49:03.618345+0000","last_active":"2023-10-26T05:49:03.618420+0000","last_peered":"2023-10-26T05:49:03.618420+0000","last_clean":"2023-10-26T05:49:03.618420+0000","last_became_active":"2023-10-26T05:49:03.617876+0000","last_became_peered":"2023-10-26T05:49:03.617876+0000","last_unstale":"2023-10-26T05:49:03.618420+0000","last_undegraded":"2023-10-26T05:49:03.618420+0000","last_fullsized":"2023-10-26T05:49:03.618420+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:36.604418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606468+0000","last_change":"2023-10-26T05:49:03.602519+0000","last_active":"2023-10-26T05:49:03.606468+0000","last_peered":"2023-10-26T05:49:03.606468+0000","last_clean":"2023-10-26T05:49:03.606468+0000","last_became_active":"2023-10-26T05:49:03.602228+0000","last_became_peered":"2023-10-26T05:49:03.602228+0000","last_unstale":"2023-10-26T05:49:03.606468+0000","last_undegraded":"2023-10-26T05:49:03.606468+0000","last_fullsized":"2023-10-26T05:49:03.606468+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:42.052128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626709+0000","last_change":"2023-10-26T05:49:03.625242+0000","last_active":"2023-10-26T05:49:03.626709+0000","last_peered":"2023-10-26T05:49:03.626709+0000","last_clean":"2023-10-26T05:49:03.626709+0000","last_became_active":"2023-10-26T05:49:03.623003+0000","last_became_peered":"2023-10-26T05:49:03.623003+0000","last_unstale":"2023-10-26T05:49:03.626709+0000","last_undegraded":"2023-10-26T05:49:03.626709+0000","last_fullsized":"2023-10-26T05:49:03.626709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:53.593057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.611977+0000","last_change":"2023-10-26T05:49:03.611889+0000","last_active":"2023-10-26T05:49:03.611977+0000","last_peered":"2023-10-26T05:49:03.611977+0000","last_clean":"2023-10-26T05:49:03.611977+0000","last_became_active":"2023-10-26T05:49:03.611578+0000","last_became_peered":"2023-10-26T05:49:03.611578+0000","last_unstale":"2023-10-26T05:49:03.611977+0000","last_undegraded":"2023-10-26T05:49:03.611977+0000","last_fullsized":"2023-10-26T05:49:03.611977+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:17.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.607669+0000","last_change":"2023-10-26T05:49:03.607577+0000","last_active":"2023-10-26T05:49:03.607669+0000","last_peered":"2023-10-26T05:49:03.607669+0000","last_clean":"2023-10-26T05:49:03.607669+0000","last_became_active":"2023-10-26T05:49:03.607270+0000","last_became_peered":"2023-10-26T05:49:03.607270+0000","last_unstale":"2023-10-26T05:49:03.607669+0000","last_undegraded":"2023-10-26T05:49:03.607669+0000","last_fullsized":"2023-10-26T05:49:03.607669+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:56.606656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621398+0000","last_change":"2023-10-26T05:49:03.621325+0000","last_active":"2023-10-26T05:49:03.621398+0000","last_peered":"2023-10-26T05:49:03.621398+0000","last_clean":"2023-10-26T05:49:03.621398+0000","last_became_active":"2023-10-26T05:49:03.620987+0000","last_became_peered":"2023-10-26T05:49:03.620987+0000","last_unstale":"2023-10-26T05:49:03.621398+0000","last_undegraded":"2023-10-26T05:49:03.621398+0000","last_fullsized":"2023-10-26T05:49:03.621398+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:24:45.984512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621759+0000","last_change":"2023-10-26T05:49:03.621639+0000","last_active":"2023-10-26T05:49:03.621759+0000","last_peered":"2023-10-26T05:49:03.621759+0000","last_clean":"2023-10-26T05:49:03.621759+0000","last_became_active":"2023-10-26T05:49:03.620138+0000","last_became_peered":"2023-10-26T05:49:03.620138+0000","last_unstale":"2023-10-26T05:49:03.621759+0000","last_undegraded":"2023-10-26T05:49:03.621759+0000","last_fullsized":"2023-10-26T05:49:03.621759+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:42.192405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.625860+0000","last_change":"2023-10-26T05:49:03.625692+0000","last_active":"2023-10-26T05:49:03.625860+0000","last_peered":"2023-10-26T05:49:03.625860+0000","last_clean":"2023-10-26T05:49:03.625860+0000","last_became_active":"2023-10-26T05:49:03.624996+0000","last_became_peered":"2023-10-26T05:49:03.624996+0000","last_unstale":"2023-10-26T05:49:03.625860+0000","last_undegraded":"2023-10-26T05:49:03.625860+0000","last_fullsized":"2023-10-26T05:49:03.625860+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:42.052128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604189+0000","last_change":"2023-10-26T05:49:03.603205+0000","last_active":"2023-10-26T05:49:03.604189+0000","last_peered":"2023-10-26T05:49:03.604189+0000","last_clean":"2023-10-26T05:49:03.604189+0000","last_became_active":"2023-10-26T05:49:03.602436+0000","last_became_peered":"2023-10-26T05:49:03.602436+0000","last_unstale":"2023-10-26T05:49:03.604189+0000","last_undegraded":"2023-10-26T05:49:03.604189+0000","last_fullsized":"2023-10-26T05:49:03.604189+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:24.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.633477+0000","last_change":"2023-10-26T05:49:03.633178+0000","last_active":"2023-10-26T05:49:03.633477+0000","last_peered":"2023-10-26T05:49:03.633477+0000","last_clean":"2023-10-26T05:49:03.633477+0000","last_became_active":"2023-10-26T05:49:03.625796+0000","last_became_peered":"2023-10-26T05:49:03.625796+0000","last_unstale":"2023-10-26T05:49:03.633477+0000","last_undegraded":"2023-10-26T05:49:03.633477+0000","last_fullsized":"2023-10-26T05:49:03.633477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:29.113550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.615064+0000","last_change":"2023-10-26T05:49:03.614999+0000","last_active":"2023-10-26T05:49:03.615064+0000","last_peered":"2023-10-26T05:49:03.615064+0000","last_clean":"2023-10-26T05:49:03.615064+0000","last_became_active":"2023-10-26T05:49:03.614783+0000","last_became_peered":"2023-10-26T05:49:03.614783+0000","last_unstale":"2023-10-26T05:49:03.615064+0000","last_undegraded":"2023-10-26T05:49:03.615064+0000","last_fullsized":"2023-10-26T05:49:03.615064+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:42:29.113550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620868+0000","last_change":"2023-10-26T05:49:03.620795+0000","last_active":"2023-10-26T05:49:03.620868+0000","last_peered":"2023-10-26T05:49:03.620868+0000","last_clean":"2023-10-26T05:49:03.620868+0000","last_became_active":"2023-10-26T05:49:03.620571+0000","last_became_peered":"2023-10-26T05:49:03.620571+0000","last_unstale":"2023-10-26T05:49:03.620868+0000","last_undegraded":"2023-10-26T05:49:03.620868+0000","last_fullsized":"2023-10-26T05:49:03.620868+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:54:46.778631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.633734+0000","last_change":"2023-10-26T05:49:03.633679+0000","last_active":"2023-10-26T05:49:03.633734+0000","last_peered":"2023-10-26T05:49:03.633734+0000","last_clean":"2023-10-26T05:49:03.633734+0000","last_became_active":"2023-10-26T05:49:03.633436+0000","last_became_peered":"2023-10-26T05:49:03.633436+0000","last_unstale":"2023-10-26T05:49:03.633734+0000","last_undegraded":"2023-10-26T05:49:03.633734+0000","last_fullsized":"2023-10-26T05:49:03.633734+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:01:29.828911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.641400+0000","last_change":"2023-10-26T05:49:03.641248+0000","last_active":"2023-10-26T05:49:03.641400+0000","last_peered":"2023-10-26T05:49:03.641400+0000","last_clean":"2023-10-26T05:49:03.641400+0000","last_became_active":"2023-10-26T05:49:03.640126+0000","last_became_peered":"2023-10-26T05:49:03.640126+0000","last_unstale":"2023-10-26T05:49:03.641400+0000","last_undegraded":"2023-10-26T05:49:03.641400+0000","last_fullsized":"2023-10-26T05:49:03.641400+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:44.587647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.640234+0000","last_change":"2023-10-26T05:49:03.640056+0000","last_active":"2023-10-26T05:49:03.640234+0000","last_peered":"2023-10-26T05:49:03.640234+0000","last_clean":"2023-10-26T05:49:03.640234+0000","last_became_active":"2023-10-26T05:49:03.637500+0000","last_became_peered":"2023-10-26T05:49:03.637500+0000","last_unstale":"2023-10-26T05:49:03.640234+0000","last_undegraded":"2023-10-26T05:49:03.640234+0000","last_fullsized":"2023-10-26T05:49:03.640234+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:01:29.828911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.617491+0000","last_change":"2023-10-26T05:49:03.617006+0000","last_active":"2023-10-26T05:49:03.617491+0000","last_peered":"2023-10-26T05:49:03.617491+0000","last_clean":"2023-10-26T05:49:03.617491+0000","last_became_active":"2023-10-26T05:49:03.615738+0000","last_became_peered":"2023-10-26T05:49:03.615738+0000","last_unstale":"2023-10-26T05:49:03.617491+0000","last_undegraded":"2023-10-26T05:49:03.617491+0000","last_fullsized":"2023-10-26T05:49:03.617491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:01.030276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.607474+0000","last_change":"2023-10-26T05:49:03.607209+0000","last_active":"2023-10-26T05:49:03.607474+0000","last_peered":"2023-10-26T05:49:03.607474+0000","last_clean":"2023-10-26T05:49:03.607474+0000","last_became_active":"2023-10-26T05:49:03.606978+0000","last_became_peered":"2023-10-26T05:49:03.606978+0000","last_unstale":"2023-10-26T05:49:03.607474+0000","last_undegraded":"2023-10-26T05:49:03.607474+0000","last_fullsized":"2023-10-26T05:49:03.607474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:12:01.030276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.629211+0000","last_change":"2023-10-26T05:49:03.628978+0000","last_active":"2023-10-26T05:49:03.629211+0000","last_peered":"2023-10-26T05:49:03.629211+0000","last_clean":"2023-10-26T05:49:03.629211+0000","last_became_active":"2023-10-26T05:49:03.628221+0000","last_became_peered":"2023-10-26T05:49:03.628221+0000","last_unstale":"2023-10-26T05:49:03.629211+0000","last_undegraded":"2023-10-26T05:49:03.629211+0000","last_fullsized":"2023-10-26T05:49:03.629211+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:42.844061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626545+0000","last_change":"2023-10-26T05:49:03.623123+0000","last_active":"2023-10-26T05:49:03.626545+0000","last_peered":"2023-10-26T05:49:03.626545+0000","last_clean":"2023-10-26T05:49:03.626545+0000","last_became_active":"2023-10-26T05:49:03.619317+0000","last_became_peered":"2023-10-26T05:49:03.619317+0000","last_unstale":"2023-10-26T05:49:03.626545+0000","last_undegraded":"2023-10-26T05:49:03.626545+0000","last_fullsized":"2023-10-26T05:49:03.626545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:39.212676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.638656+0000","last_change":"2023-10-26T05:49:03.638475+0000","last_active":"2023-10-26T05:49:03.638656+0000","last_peered":"2023-10-26T05:49:03.638656+0000","last_clean":"2023-10-26T05:49:03.638656+0000","last_became_active":"2023-10-26T05:49:03.632432+0000","last_became_peered":"2023-10-26T05:49:03.632432+0000","last_unstale":"2023-10-26T05:49:03.638656+0000","last_undegraded":"2023-10-26T05:49:03.638656+0000","last_fullsized":"2023-10-26T05:49:03.638656+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:17.113253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604233+0000","last_change":"2023-10-26T05:49:03.604049+0000","last_active":"2023-10-26T05:49:03.604233+0000","last_peered":"2023-10-26T05:49:03.604233+0000","last_clean":"2023-10-26T05:49:03.604233+0000","last_became_active":"2023-10-26T05:49:03.603458+0000","last_became_peered":"2023-10-26T05:49:03.603458+0000","last_unstale":"2023-10-26T05:49:03.604233+0000","last_undegraded":"2023-10-26T05:49:03.604233+0000","last_fullsized":"2023-10-26T05:49:03.604233+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:56.707421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.623351+0000","last_change":"2023-10-26T05:49:03.623303+0000","last_active":"2023-10-26T05:49:03.623351+0000","last_peered":"2023-10-26T05:49:03.623351+0000","last_clean":"2023-10-26T05:49:03.623351+0000","last_became_active":"2023-10-26T05:49:03.623157+0000","last_became_peered":"2023-10-26T05:49:03.623157+0000","last_unstale":"2023-10-26T05:49:03.623351+0000","last_undegraded":"2023-10-26T05:49:03.623351+0000","last_fullsized":"2023-10-26T05:49:03.623351+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.608691+0000","last_change":"2023-10-26T05:49:03.608566+0000","last_active":"2023-10-26T05:49:03.608691+0000","last_peered":"2023-10-26T05:49:03.608691+0000","last_clean":"2023-10-26T05:49:03.608691+0000","last_became_active":"2023-10-26T05:49:03.608339+0000","last_became_peered":"2023-10-26T05:49:03.608339+0000","last_unstale":"2023-10-26T05:49:03.608691+0000","last_undegraded":"2023-10-26T05:49:03.608691+0000","last_fullsized":"2023-10-26T05:49:03.608691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:42.821655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.610611+0000","last_change":"2023-10-26T05:49:03.610496+0000","last_active":"2023-10-26T05:49:03.610611+0000","last_peered":"2023-10-26T05:49:03.610611+0000","last_clean":"2023-10-26T05:49:03.610611+0000","last_became_active":"2023-10-26T05:49:03.610182+0000","last_became_peered":"2023-10-26T05:49:03.610182+0000","last_unstale":"2023-10-26T05:49:03.610611+0000","last_undegraded":"2023-10-26T05:49:03.610611+0000","last_fullsized":"2023-10-26T05:49:03.610611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:56.606656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621288+0000","last_change":"2023-10-26T05:49:02.588420+0000","last_active":"2023-10-26T05:49:03.621288+0000","last_peered":"2023-10-26T05:49:03.621288+0000","last_clean":"2023-10-26T05:49:03.621288+0000","last_became_active":"2023-10-26T05:49:02.588270+0000","last_became_peered":"2023-10-26T05:49:02.588270+0000","last_unstale":"2023-10-26T05:49:03.621288+0000","last_undegraded":"2023-10-26T05:49:03.621288+0000","last_fullsized":"2023-10-26T05:49:03.621288+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:11.532031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.640992+0000","last_change":"2023-10-26T05:49:03.640850+0000","last_active":"2023-10-26T05:49:03.640992+0000","last_peered":"2023-10-26T05:49:03.640992+0000","last_clean":"2023-10-26T05:49:03.640992+0000","last_became_active":"2023-10-26T05:49:03.624309+0000","last_became_peered":"2023-10-26T05:49:03.624309+0000","last_unstale":"2023-10-26T05:49:03.640992+0000","last_undegraded":"2023-10-26T05:49:03.640992+0000","last_fullsized":"2023-10-26T05:49:03.640992+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:22.416213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.628451+0000","last_change":"2023-10-26T05:49:02.590322+0000","last_active":"2023-10-26T05:49:03.628451+0000","last_peered":"2023-10-26T05:49:03.628451+0000","last_clean":"2023-10-26T05:49:03.628451+0000","last_became_active":"2023-10-26T05:49:02.589733+0000","last_became_peered":"2023-10-26T05:49:02.589733+0000","last_unstale":"2023-10-26T05:49:03.628451+0000","last_undegraded":"2023-10-26T05:49:03.628451+0000","last_fullsized":"2023-10-26T05:49:03.628451+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:42.232057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.626662+0000","last_change":"2023-10-26T05:49:03.625150+0000","last_active":"2023-10-26T05:49:03.626662+0000","last_peered":"2023-10-26T05:49:03.626662+0000","last_clean":"2023-10-26T05:49:03.626662+0000","last_became_active":"2023-10-26T05:49:03.624344+0000","last_became_peered":"2023-10-26T05:49:03.624344+0000","last_unstale":"2023-10-26T05:49:03.626662+0000","last_undegraded":"2023-10-26T05:49:03.626662+0000","last_fullsized":"2023-10-26T05:49:03.626662+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:43.246142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620319+0000","last_change":"2023-10-26T05:49:02.581477+0000","last_active":"2023-10-26T05:49:03.620319+0000","last_peered":"2023-10-26T05:49:03.620319+0000","last_clean":"2023-10-26T05:49:03.620319+0000","last_became_active":"2023-10-26T05:49:02.580448+0000","last_became_peered":"2023-10-26T05:49:02.580448+0000","last_unstale":"2023-10-26T05:49:03.620319+0000","last_undegraded":"2023-10-26T05:49:03.620319+0000","last_fullsized":"2023-10-26T05:49:03.620319+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:55.693336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.617618+0000","last_change":"2023-10-26T05:49:03.617551+0000","last_active":"2023-10-26T05:49:03.617618+0000","last_peered":"2023-10-26T05:49:03.617618+0000","last_clean":"2023-10-26T05:49:03.617618+0000","last_became_active":"2023-10-26T05:49:03.617252+0000","last_became_peered":"2023-10-26T05:49:03.617252+0000","last_unstale":"2023-10-26T05:49:03.617618+0000","last_undegraded":"2023-10-26T05:49:03.617618+0000","last_fullsized":"2023-10-26T05:49:03.617618+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:10:36.604418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.636925+0000","last_change":"2023-10-26T05:49:02.595139+0000","last_active":"2023-10-26T05:49:03.636925+0000","last_peered":"2023-10-26T05:49:03.636925+0000","last_clean":"2023-10-26T05:49:03.636925+0000","last_became_active":"2023-10-26T05:49:02.589877+0000","last_became_peered":"2023-10-26T05:49:02.589877+0000","last_unstale":"2023-10-26T05:49:03.636925+0000","last_undegraded":"2023-10-26T05:49:03.636925+0000","last_fullsized":"2023-10-26T05:49:03.636925+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:57.499803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.624524+0000","last_change":"2023-10-26T05:49:03.624076+0000","last_active":"2023-10-26T05:49:03.624524+0000","last_peered":"2023-10-26T05:49:03.624524+0000","last_clean":"2023-10-26T05:49:03.624524+0000","last_became_active":"2023-10-26T05:49:03.623066+0000","last_became_peered":"2023-10-26T05:49:03.623066+0000","last_unstale":"2023-10-26T05:49:03.624524+0000","last_undegraded":"2023-10-26T05:49:03.624524+0000","last_fullsized":"2023-10-26T05:49:03.624524+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:20:56.606656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616138+0000","last_change":"2023-10-26T05:49:02.584779+0000","last_active":"2023-10-26T05:49:03.616138+0000","last_peered":"2023-10-26T05:49:03.616138+0000","last_clean":"2023-10-26T05:49:03.616138+0000","last_became_active":"2023-10-26T05:49:02.584565+0000","last_became_peered":"2023-10-26T05:49:02.584565+0000","last_unstale":"2023-10-26T05:49:03.616138+0000","last_undegraded":"2023-10-26T05:49:03.616138+0000","last_fullsized":"2023-10-26T05:49:03.616138+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:47.467126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.608619+0000","last_change":"2023-10-26T05:49:03.607733+0000","last_active":"2023-10-26T05:49:03.608619+0000","last_peered":"2023-10-26T05:49:03.608619+0000","last_clean":"2023-10-26T05:49:03.608619+0000","last_became_active":"2023-10-26T05:49:03.607528+0000","last_became_peered":"2023-10-26T05:49:03.607528+0000","last_unstale":"2023-10-26T05:49:03.608619+0000","last_undegraded":"2023-10-26T05:49:03.608619+0000","last_fullsized":"2023-10-26T05:49:03.608619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:57:36.193315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.630198+0000","last_change":"2023-10-26T05:49:02.582583+0000","last_active":"2023-10-26T05:49:03.630198+0000","last_peered":"2023-10-26T05:49:03.630198+0000","last_clean":"2023-10-26T05:49:03.630198+0000","last_became_active":"2023-10-26T05:49:02.581814+0000","last_became_peered":"2023-10-26T05:49:02.581814+0000","last_unstale":"2023-10-26T05:49:03.630198+0000","last_undegraded":"2023-10-26T05:49:03.630198+0000","last_fullsized":"2023-10-26T05:49:03.630198+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:05.329949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.618242+0000","last_change":"2023-10-26T05:49:03.618164+0000","last_active":"2023-10-26T05:49:03.618242+0000","last_peered":"2023-10-26T05:49:03.618242+0000","last_clean":"2023-10-26T05:49:03.618242+0000","last_became_active":"2023-10-26T05:49:03.617470+0000","last_became_peered":"2023-10-26T05:49:03.617470+0000","last_unstale":"2023-10-26T05:49:03.618242+0000","last_undegraded":"2023-10-26T05:49:03.618242+0000","last_fullsized":"2023-10-26T05:49:03.618242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:42.844061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.613180+0000","last_change":"2023-10-26T05:49:02.587615+0000","last_active":"2023-10-26T05:49:03.613180+0000","last_peered":"2023-10-26T05:49:03.613180+0000","last_clean":"2023-10-26T05:49:03.613180+0000","last_became_active":"2023-10-26T05:49:02.587060+0000","last_became_peered":"2023-10-26T05:49:02.587060+0000","last_unstale":"2023-10-26T05:49:03.613180+0000","last_undegraded":"2023-10-26T05:49:03.613180+0000","last_fullsized":"2023-10-26T05:49:03.613180+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:23.643656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.639834+0000","last_change":"2023-10-26T05:49:03.639695+0000","last_active":"2023-10-26T05:49:03.639834+0000","last_peered":"2023-10-26T05:49:03.639834+0000","last_clean":"2023-10-26T05:49:03.639834+0000","last_became_active":"2023-10-26T05:49:03.633815+0000","last_became_peered":"2023-10-26T05:49:03.633815+0000","last_unstale":"2023-10-26T05:49:03.639834+0000","last_undegraded":"2023-10-26T05:49:03.639834+0000","last_fullsized":"2023-10-26T05:49:03.639834+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:36:08.417695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.599708+0000","last_change":"2023-10-26T05:49:02.579062+0000","last_active":"2023-10-26T05:49:03.599708+0000","last_peered":"2023-10-26T05:49:03.599708+0000","last_clean":"2023-10-26T05:49:03.599708+0000","last_became_active":"2023-10-26T05:49:02.578404+0000","last_became_peered":"2023-10-26T05:49:02.578404+0000","last_unstale":"2023-10-26T05:49:03.599708+0000","last_undegraded":"2023-10-26T05:49:03.599708+0000","last_fullsized":"2023-10-26T05:49:03.599708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:38.198591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.602746+0000","last_change":"2023-10-26T05:49:03.602287+0000","last_active":"2023-10-26T05:49:03.602746+0000","last_peered":"2023-10-26T05:49:03.602746+0000","last_clean":"2023-10-26T05:49:03.602746+0000","last_became_active":"2023-10-26T05:49:03.601690+0000","last_became_peered":"2023-10-26T05:49:03.601690+0000","last_unstale":"2023-10-26T05:49:03.602746+0000","last_undegraded":"2023-10-26T05:49:03.602746+0000","last_fullsized":"2023-10-26T05:49:03.602746+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:39.212676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.600746+0000","last_change":"2023-10-26T05:49:02.578207+0000","last_active":"2023-10-26T05:49:03.600746+0000","last_peered":"2023-10-26T05:49:03.600746+0000","last_clean":"2023-10-26T05:49:03.600746+0000","last_became_active":"2023-10-26T05:49:02.577456+0000","last_became_peered":"2023-10-26T05:49:02.577456+0000","last_unstale":"2023-10-26T05:49:03.600746+0000","last_undegraded":"2023-10-26T05:49:03.600746+0000","last_fullsized":"2023-10-26T05:49:03.600746+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:57.499803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.631136+0000","last_change":"2023-10-26T05:49:03.630941+0000","last_active":"2023-10-26T05:49:03.631136+0000","last_peered":"2023-10-26T05:49:03.631136+0000","last_clean":"2023-10-26T05:49:03.631136+0000","last_became_active":"2023-10-26T05:49:03.630088+0000","last_became_peered":"2023-10-26T05:49:03.630088+0000","last_unstale":"2023-10-26T05:49:03.631136+0000","last_undegraded":"2023-10-26T05:49:03.631136+0000","last_fullsized":"2023-10-26T05:49:03.631136+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:24.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.599370+0000","last_change":"2023-10-26T05:49:02.579468+0000","last_active":"2023-10-26T05:49:03.599370+0000","last_peered":"2023-10-26T05:49:03.599370+0000","last_clean":"2023-10-26T05:49:03.599370+0000","last_became_active":"2023-10-26T05:49:02.578778+0000","last_became_peered":"2023-10-26T05:49:02.578778+0000","last_unstale":"2023-10-26T05:49:03.599370+0000","last_undegraded":"2023-10-26T05:49:03.599370+0000","last_fullsized":"2023-10-26T05:49:03.599370+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:55.693336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616417+0000","last_change":"2023-10-26T05:49:03.616285+0000","last_active":"2023-10-26T05:49:03.616417+0000","last_peered":"2023-10-26T05:49:03.616417+0000","last_clean":"2023-10-26T05:49:03.616417+0000","last_became_active":"2023-10-26T05:49:03.615695+0000","last_became_peered":"2023-10-26T05:49:03.615695+0000","last_unstale":"2023-10-26T05:49:03.616417+0000","last_undegraded":"2023-10-26T05:49:03.616417+0000","last_fullsized":"2023-10-26T05:49:03.616417+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:21.204226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.641355+0000","last_change":"2023-10-26T05:49:02.590859+0000","last_active":"2023-10-26T05:49:03.641355+0000","last_peered":"2023-10-26T05:49:03.641355+0000","last_clean":"2023-10-26T05:49:03.641355+0000","last_became_active":"2023-10-26T05:49:02.590216+0000","last_became_peered":"2023-10-26T05:49:02.590216+0000","last_unstale":"2023-10-26T05:49:03.641355+0000","last_undegraded":"2023-10-26T05:49:03.641355+0000","last_fullsized":"2023-10-26T05:49:03.641355+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:24:44.970427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.612377+0000","last_change":"2023-10-26T05:49:03.612220+0000","last_active":"2023-10-26T05:49:03.612377+0000","last_peered":"2023-10-26T05:49:03.612377+0000","last_clean":"2023-10-26T05:49:03.612377+0000","last_became_active":"2023-10-26T05:49:03.611605+0000","last_became_peered":"2023-10-26T05:49:03.611605+0000","last_unstale":"2023-10-26T05:49:03.612377+0000","last_undegraded":"2023-10-26T05:49:03.612377+0000","last_fullsized":"2023-10-26T05:49:03.612377+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:54:46.778631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.619967+0000","last_change":"2023-10-26T05:49:02.578690+0000","last_active":"2023-10-26T05:49:03.619967+0000","last_peered":"2023-10-26T05:49:03.619967+0000","last_clean":"2023-10-26T05:49:03.619967+0000","last_became_active":"2023-10-26T05:49:02.578385+0000","last_became_peered":"2023-10-26T05:49:02.578385+0000","last_unstale":"2023-10-26T05:49:03.619967+0000","last_undegraded":"2023-10-26T05:49:03.619967+0000","last_fullsized":"2023-10-26T05:49:03.619967+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:02:43.573562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621711+0000","last_change":"2023-10-26T05:49:03.621261+0000","last_active":"2023-10-26T05:49:03.621711+0000","last_peered":"2023-10-26T05:49:03.621711+0000","last_clean":"2023-10-26T05:49:03.621711+0000","last_became_active":"2023-10-26T05:49:03.619502+0000","last_became_peered":"2023-10-26T05:49:03.619502+0000","last_unstale":"2023-10-26T05:49:03.621711+0000","last_undegraded":"2023-10-26T05:49:03.621711+0000","last_fullsized":"2023-10-26T05:49:03.621711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:24.672797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.615429+0000","last_change":"2023-10-26T05:49:02.582599+0000","last_active":"2023-10-26T05:49:03.615429+0000","last_peered":"2023-10-26T05:49:03.615429+0000","last_clean":"2023-10-26T05:49:03.615429+0000","last_became_active":"2023-10-26T05:49:02.581943+0000","last_became_peered":"2023-10-26T05:49:02.581943+0000","last_unstale":"2023-10-26T05:49:03.615429+0000","last_undegraded":"2023-10-26T05:49:03.615429+0000","last_fullsized":"2023-10-26T05:49:03.615429+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:57:05.628834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606521+0000","last_change":"2023-10-26T05:49:03.604966+0000","last_active":"2023-10-26T05:49:03.606521+0000","last_peered":"2023-10-26T05:49:03.606521+0000","last_clean":"2023-10-26T05:49:03.606521+0000","last_became_active":"2023-10-26T05:49:03.604335+0000","last_became_peered":"2023-10-26T05:49:03.604335+0000","last_unstale":"2023-10-26T05:49:03.606521+0000","last_undegraded":"2023-10-26T05:49:03.606521+0000","last_fullsized":"2023-10-26T05:49:03.606521+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:36:40.893138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.603841+0000","last_change":"2023-10-26T05:49:02.595617+0000","last_active":"2023-10-26T05:49:03.603841+0000","last_peered":"2023-10-26T05:49:03.603841+0000","last_clean":"2023-10-26T05:49:03.603841+0000","last_became_active":"2023-10-26T05:49:02.592539+0000","last_became_peered":"2023-10-26T05:49:02.592539+0000","last_unstale":"2023-10-26T05:49:03.603841+0000","last_undegraded":"2023-10-26T05:49:03.603841+0000","last_fullsized":"2023-10-26T05:49:03.603841+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:39.535504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609073+0000","last_change":"2023-10-26T05:49:03.608454+0000","last_active":"2023-10-26T05:49:03.609073+0000","last_peered":"2023-10-26T05:49:03.609073+0000","last_clean":"2023-10-26T05:49:03.609073+0000","last_became_active":"2023-10-26T05:49:03.608045+0000","last_became_peered":"2023-10-26T05:49:03.608045+0000","last_unstale":"2023-10-26T05:49:03.609073+0000","last_undegraded":"2023-10-26T05:49:03.609073+0000","last_fullsized":"2023-10-26T05:49:03.609073+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:58.513888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606167+0000","last_change":"2023-10-26T05:49:02.575131+0000","last_active":"2023-10-26T05:49:03.606167+0000","last_peered":"2023-10-26T05:49:03.606167+0000","last_clean":"2023-10-26T05:49:03.606167+0000","last_became_active":"2023-10-26T05:49:02.574318+0000","last_became_peered":"2023-10-26T05:49:02.574318+0000","last_unstale":"2023-10-26T05:49:03.606167+0000","last_undegraded":"2023-10-26T05:49:03.606167+0000","last_fullsized":"2023-10-26T05:49:03.606167+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:04.382797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.637702+0000","last_change":"2023-10-26T05:49:03.637444+0000","last_active":"2023-10-26T05:49:03.637702+0000","last_peered":"2023-10-26T05:49:03.637702+0000","last_clean":"2023-10-26T05:49:03.637702+0000","last_became_active":"2023-10-26T05:49:03.636919+0000","last_became_peered":"2023-10-26T05:49:03.636919+0000","last_unstale":"2023-10-26T05:49:03.637702+0000","last_undegraded":"2023-10-26T05:49:03.637702+0000","last_fullsized":"2023-10-26T05:49:03.637702+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:06.344034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621823+0000","last_change":"2023-10-26T05:49:02.574731+0000","last_active":"2023-10-26T05:49:03.621823+0000","last_peered":"2023-10-26T05:49:03.621823+0000","last_clean":"2023-10-26T05:49:03.621823+0000","last_became_active":"2023-10-26T05:49:02.574503+0000","last_became_peered":"2023-10-26T05:49:02.574503+0000","last_unstale":"2023-10-26T05:49:03.621823+0000","last_undegraded":"2023-10-26T05:49:03.621823+0000","last_fullsized":"2023-10-26T05:49:03.621823+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:12:43.117799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.627668+0000","last_change":"2023-10-26T05:49:03.627608+0000","last_active":"2023-10-26T05:49:03.627668+0000","last_peered":"2023-10-26T05:49:03.627668+0000","last_clean":"2023-10-26T05:49:03.627668+0000","last_became_active":"2023-10-26T05:49:03.627450+0000","last_became_peered":"2023-10-26T05:49:03.627450+0000","last_unstale":"2023-10-26T05:49:03.627668+0000","last_undegraded":"2023-10-26T05:49:03.627668+0000","last_fullsized":"2023-10-26T05:49:03.627668+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:24.672797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.617730+0000","last_change":"2023-10-26T05:49:02.584218+0000","last_active":"2023-10-26T05:49:03.617730+0000","last_peered":"2023-10-26T05:49:03.617730+0000","last_clean":"2023-10-26T05:49:03.617730+0000","last_became_active":"2023-10-26T05:49:02.583933+0000","last_became_peered":"2023-10-26T05:49:02.583933+0000","last_unstale":"2023-10-26T05:49:03.617730+0000","last_undegraded":"2023-10-26T05:49:03.617730+0000","last_fullsized":"2023-10-26T05:49:03.617730+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:51:21.402128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609959+0000","last_change":"2023-10-26T05:49:03.609814+0000","last_active":"2023-10-26T05:49:03.609959+0000","last_peered":"2023-10-26T05:49:03.609959+0000","last_clean":"2023-10-26T05:49:03.609959+0000","last_became_active":"2023-10-26T05:49:03.609622+0000","last_became_peered":"2023-10-26T05:49:03.609622+0000","last_unstale":"2023-10-26T05:49:03.609959+0000","last_undegraded":"2023-10-26T05:49:03.609959+0000","last_fullsized":"2023-10-26T05:49:03.609959+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:13.301659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616861+0000","last_change":"2023-10-26T05:49:02.571804+0000","last_active":"2023-10-26T05:49:03.616861+0000","last_peered":"2023-10-26T05:49:03.616861+0000","last_clean":"2023-10-26T05:49:03.616861+0000","last_became_active":"2023-10-26T05:49:02.571014+0000","last_became_peered":"2023-10-26T05:49:02.571014+0000","last_unstale":"2023-10-26T05:49:03.616861+0000","last_undegraded":"2023-10-26T05:49:03.616861+0000","last_fullsized":"2023-10-26T05:49:03.616861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:52.578972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.635515+0000","last_change":"2023-10-26T05:49:03.635293+0000","last_active":"2023-10-26T05:49:03.635515+0000","last_peered":"2023-10-26T05:49:03.635515+0000","last_clean":"2023-10-26T05:49:03.635515+0000","last_became_active":"2023-10-26T05:49:03.634169+0000","last_became_peered":"2023-10-26T05:49:03.634169+0000","last_unstale":"2023-10-26T05:49:03.635515+0000","last_undegraded":"2023-10-26T05:49:03.635515+0000","last_fullsized":"2023-10-26T05:49:03.635515+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:58:38.756380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.635908+0000","last_change":"2023-10-26T05:49:02.587219+0000","last_active":"2023-10-26T05:49:03.635908+0000","last_peered":"2023-10-26T05:49:03.635908+0000","last_clean":"2023-10-26T05:49:03.635908+0000","last_became_active":"2023-10-26T05:49:02.586237+0000","last_became_peered":"2023-10-26T05:49:02.586237+0000","last_unstale":"2023-10-26T05:49:03.635908+0000","last_undegraded":"2023-10-26T05:49:03.635908+0000","last_fullsized":"2023-10-26T05:49:03.635908+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:54:45.764547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.621657+0000","last_change":"2023-10-26T05:49:03.620903+0000","last_active":"2023-10-26T05:49:03.621657+0000","last_peered":"2023-10-26T05:49:03.621657+0000","last_clean":"2023-10-26T05:49:03.621657+0000","last_became_active":"2023-10-26T05:49:03.618838+0000","last_became_peered":"2023-10-26T05:49:03.618838+0000","last_unstale":"2023-10-26T05:49:03.621657+0000","last_undegraded":"2023-10-26T05:49:03.621657+0000","last_fullsized":"2023-10-26T05:49:03.621657+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:17.819673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606292+0000","last_change":"2023-10-26T05:49:02.588033+0000","last_active":"2023-10-26T05:49:03.606292+0000","last_peered":"2023-10-26T05:49:03.606292+0000","last_clean":"2023-10-26T05:49:03.606292+0000","last_became_active":"2023-10-26T05:49:02.587683+0000","last_became_peered":"2023-10-26T05:49:02.587683+0000","last_unstale":"2023-10-26T05:49:03.606292+0000","last_undegraded":"2023-10-26T05:49:03.606292+0000","last_fullsized":"2023-10-26T05:49:03.606292+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:38.628722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.616426+0000","last_change":"2023-10-26T05:49:03.616378+0000","last_active":"2023-10-26T05:49:03.616426+0000","last_peered":"2023-10-26T05:49:03.616426+0000","last_clean":"2023-10-26T05:49:03.616426+0000","last_became_active":"2023-10-26T05:49:03.615942+0000","last_became_peered":"2023-10-26T05:49:03.615942+0000","last_unstale":"2023-10-26T05:49:03.616426+0000","last_undegraded":"2023-10-26T05:49:03.616426+0000","last_fullsized":"2023-10-26T05:49:03.616426+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:03:22.619812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620937+0000","last_change":"2023-10-26T05:49:02.585731+0000","last_active":"2023-10-26T05:49:03.620937+0000","last_peered":"2023-10-26T05:49:03.620937+0000","last_clean":"2023-10-26T05:49:03.620937+0000","last_became_active":"2023-10-26T05:49:02.585422+0000","last_became_peered":"2023-10-26T05:49:02.585422+0000","last_unstale":"2023-10-26T05:49:03.620937+0000","last_undegraded":"2023-10-26T05:49:03.620937+0000","last_fullsized":"2023-10-26T05:49:03.620937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:45.401293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.620968+0000","last_change":"2023-10-26T05:49:03.620885+0000","last_active":"2023-10-26T05:49:03.620968+0000","last_peered":"2023-10-26T05:49:03.620968+0000","last_clean":"2023-10-26T05:49:03.620968+0000","last_became_active":"2023-10-26T05:49:03.620586+0000","last_became_peered":"2023-10-26T05:49:03.620586+0000","last_unstale":"2023-10-26T05:49:03.620968+0000","last_undegraded":"2023-10-26T05:49:03.620968+0000","last_fullsized":"2023-10-26T05:49:03.620968+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:53:42.052128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.613496+0000","last_change":"2023-10-26T05:49:02.591418+0000","last_active":"2023-10-26T05:49:03.613496+0000","last_peered":"2023-10-26T05:49:03.613496+0000","last_clean":"2023-10-26T05:49:03.613496+0000","last_became_active":"2023-10-26T05:49:02.589938+0000","last_became_peered":"2023-10-26T05:49:02.589938+0000","last_unstale":"2023-10-26T05:49:03.613496+0000","last_undegraded":"2023-10-26T05:49:03.613496+0000","last_fullsized":"2023-10-26T05:49:03.613496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:05.329949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.614683+0000","last_change":"2023-10-26T05:49:03.614564+0000","last_active":"2023-10-26T05:49:03.614683+0000","last_peered":"2023-10-26T05:49:03.614683+0000","last_clean":"2023-10-26T05:49:03.614683+0000","last_became_active":"2023-10-26T05:49:03.614245+0000","last_became_peered":"2023-10-26T05:49:03.614245+0000","last_unstale":"2023-10-26T05:49:03.614683+0000","last_undegraded":"2023-10-26T05:49:03.614683+0000","last_fullsized":"2023-10-26T05:49:03.614683+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:42.192405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.599822+0000","last_change":"2023-10-26T05:49:02.577364+0000","last_active":"2023-10-26T05:49:03.599822+0000","last_peered":"2023-10-26T05:49:03.599822+0000","last_clean":"2023-10-26T05:49:03.599822+0000","last_became_active":"2023-10-26T05:49:02.576874+0000","last_became_peered":"2023-10-26T05:49:02.576874+0000","last_unstale":"2023-10-26T05:49:03.599822+0000","last_undegraded":"2023-10-26T05:49:03.599822+0000","last_fullsized":"2023-10-26T05:49:03.599822+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:52.578972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606488+0000","last_change":"2023-10-26T05:49:03.604623+0000","last_active":"2023-10-26T05:49:03.606488+0000","last_peered":"2023-10-26T05:49:03.606488+0000","last_clean":"2023-10-26T05:49:03.606488+0000","last_became_active":"2023-10-26T05:49:03.603551+0000","last_became_peered":"2023-10-26T05:49:03.603551+0000","last_unstale":"2023-10-26T05:49:03.606488+0000","last_undegraded":"2023-10-26T05:49:03.606488+0000","last_fullsized":"2023-10-26T05:49:03.606488+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:54.512246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604188+0000","last_change":"2023-10-26T05:49:02.596028+0000","last_active":"2023-10-26T05:49:03.604188+0000","last_peered":"2023-10-26T05:49:03.604188+0000","last_clean":"2023-10-26T05:49:03.604188+0000","last_became_active":"2023-10-26T05:49:02.593417+0000","last_became_peered":"2023-10-26T05:49:02.593417+0000","last_unstale":"2023-10-26T05:49:03.604188+0000","last_undegraded":"2023-10-26T05:49:03.604188+0000","last_fullsized":"2023-10-26T05:49:03.604188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:57:05.628834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.634970+0000","last_change":"2023-10-26T05:49:03.634816+0000","last_active":"2023-10-26T05:49:03.634970+0000","last_peered":"2023-10-26T05:49:03.634970+0000","last_clean":"2023-10-26T05:49:03.634970+0000","last_became_active":"2023-10-26T05:49:03.632307+0000","last_became_peered":"2023-10-26T05:49:03.632307+0000","last_unstale":"2023-10-26T05:49:03.634970+0000","last_undegraded":"2023-10-26T05:49:03.634970+0000","last_fullsized":"2023-10-26T05:49:03.634970+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:10.235795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.603923+0000","last_change":"2023-10-26T05:49:02.582939+0000","last_active":"2023-10-26T05:49:03.603923+0000","last_peered":"2023-10-26T05:49:03.603923+0000","last_clean":"2023-10-26T05:49:03.603923+0000","last_became_active":"2023-10-26T05:49:02.582477+0000","last_became_peered":"2023-10-26T05:49:02.582477+0000","last_unstale":"2023-10-26T05:49:03.603923+0000","last_undegraded":"2023-10-26T05:49:03.603923+0000","last_fullsized":"2023-10-26T05:49:03.603923+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:31:19.215900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.606504+0000","last_change":"2023-10-26T05:49:03.604796+0000","last_active":"2023-10-26T05:49:03.606504+0000","last_peered":"2023-10-26T05:49:03.606504+0000","last_clean":"2023-10-26T05:49:03.606504+0000","last_became_active":"2023-10-26T05:49:03.603948+0000","last_became_peered":"2023-10-26T05:49:03.603948+0000","last_unstale":"2023-10-26T05:49:03.606504+0000","last_undegraded":"2023-10-26T05:49:03.606504+0000","last_fullsized":"2023-10-26T05:49:03.606504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:58:38.756380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.597783+0000","last_change":"2023-10-26T05:49:02.591231+0000","last_active":"2023-10-26T05:49:03.597783+0000","last_peered":"2023-10-26T05:49:03.597783+0000","last_clean":"2023-10-26T05:49:03.597783+0000","last_became_active":"2023-10-26T05:49:02.590628+0000","last_became_peered":"2023-10-26T05:49:02.590628+0000","last_unstale":"2023-10-26T05:49:03.597783+0000","last_undegraded":"2023-10-26T05:49:03.597783+0000","last_fullsized":"2023-10-26T05:49:03.597783+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:06.851599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609477+0000","last_change":"2023-10-26T05:49:03.608673+0000","last_active":"2023-10-26T05:49:03.609477+0000","last_peered":"2023-10-26T05:49:03.609477+0000","last_clean":"2023-10-26T05:49:03.609477+0000","last_became_active":"2023-10-26T05:49:03.608012+0000","last_became_peered":"2023-10-26T05:49:03.608012+0000","last_unstale":"2023-10-26T05:49:03.609477+0000","last_undegraded":"2023-10-26T05:49:03.609477+0000","last_fullsized":"2023-10-26T05:49:03.609477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:53.593057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.594591+0000","last_change":"2023-10-26T05:49:02.581625+0000","last_active":"2023-10-26T05:49:03.594591+0000","last_peered":"2023-10-26T05:49:03.594591+0000","last_clean":"2023-10-26T05:49:03.594591+0000","last_became_active":"2023-10-26T05:49:02.581076+0000","last_became_peered":"2023-10-26T05:49:02.581076+0000","last_unstale":"2023-10-26T05:49:03.594591+0000","last_undegraded":"2023-10-26T05:49:03.594591+0000","last_fullsized":"2023-10-26T05:49:03.594591+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:38.198591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609099+0000","last_change":"2023-10-26T05:49:03.608970+0000","last_active":"2023-10-26T05:49:03.609099+0000","last_peered":"2023-10-26T05:49:03.609099+0000","last_clean":"2023-10-26T05:49:03.609099+0000","last_became_active":"2023-10-26T05:49:03.608612+0000","last_became_peered":"2023-10-26T05:49:03.608612+0000","last_unstale":"2023-10-26T05:49:03.609099+0000","last_undegraded":"2023-10-26T05:49:03.609099+0000","last_fullsized":"2023-10-26T05:49:03.609099+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:43.246142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.604491+0000","last_change":"2023-10-26T05:49:02.596545+0000","last_active":"2023-10-26T05:49:03.604491+0000","last_peered":"2023-10-26T05:49:03.604491+0000","last_clean":"2023-10-26T05:49:03.604491+0000","last_became_active":"2023-10-26T05:49:02.594398+0000","last_became_peered":"2023-10-26T05:49:02.594398+0000","last_unstale":"2023-10-26T05:49:03.604491+0000","last_undegraded":"2023-10-26T05:49:03.604491+0000","last_fullsized":"2023-10-26T05:49:03.604491+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:11.532031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.607072+0000","last_change":"2023-10-26T05:49:03.606989+0000","last_active":"2023-10-26T05:49:03.607072+0000","last_peered":"2023-10-26T05:49:03.607072+0000","last_clean":"2023-10-26T05:49:03.607072+0000","last_became_active":"2023-10-26T05:49:03.606366+0000","last_became_peered":"2023-10-26T05:49:03.606366+0000","last_unstale":"2023-10-26T05:49:03.607072+0000","last_undegraded":"2023-10-26T05:49:03.607072+0000","last_fullsized":"2023-10-26T05:49:03.607072+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:24.657741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.598884+0000","last_change":"2023-10-26T05:49:02.581599+0000","last_active":"2023-10-26T05:49:03.598884+0000","last_peered":"2023-10-26T05:49:03.598884+0000","last_clean":"2023-10-26T05:49:03.598884+0000","last_became_active":"2023-10-26T05:49:02.580235+0000","last_became_peered":"2023-10-26T05:49:02.580235+0000","last_unstale":"2023-10-26T05:49:03.598884+0000","last_undegraded":"2023-10-26T05:49:03.598884+0000","last_fullsized":"2023-10-26T05:49:03.598884+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:06.851599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.622708+0000","last_change":"2023-10-26T05:49:03.620491+0000","last_active":"2023-10-26T05:49:03.622708+0000","last_peered":"2023-10-26T05:49:03.622708+0000","last_clean":"2023-10-26T05:49:03.622708+0000","last_became_active":"2023-10-26T05:49:03.619862+0000","last_became_peered":"2023-10-26T05:49:03.619862+0000","last_unstale":"2023-10-26T05:49:03.622708+0000","last_undegraded":"2023-10-26T05:49:03.622708+0000","last_fullsized":"2023-10-26T05:49:03.622708+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:56.707421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.593824+0000","last_change":"2023-10-26T05:49:02.588587+0000","last_active":"2023-10-26T05:49:03.593824+0000","last_peered":"2023-10-26T05:49:03.593824+0000","last_clean":"2023-10-26T05:49:03.593824+0000","last_became_active":"2023-10-26T05:49:02.587943+0000","last_became_peered":"2023-10-26T05:49:02.587943+0000","last_unstale":"2023-10-26T05:49:03.593824+0000","last_undegraded":"2023-10-26T05:49:03.593824+0000","last_fullsized":"2023-10-26T05:49:03.593824+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:41.829976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.609451+0000","last_change":"2023-10-26T05:49:03.609183+0000","last_active":"2023-10-26T05:49:03.609451+0000","last_peered":"2023-10-26T05:49:03.609451+0000","last_clean":"2023-10-26T05:49:03.609451+0000","last_became_active":"2023-10-26T05:49:03.608050+0000","last_became_peered":"2023-10-26T05:49:03.608050+0000","last_unstale":"2023-10-26T05:49:03.609451+0000","last_undegraded":"2023-10-26T05:49:03.609451+0000","last_fullsized":"2023-10-26T05:49:03.609451+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:44.388032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.598665+0000","last_change":"2023-10-26T05:49:02.573176+0000","last_active":"2023-10-26T05:49:03.598665+0000","last_peered":"2023-10-26T05:49:03.598665+0000","last_clean":"2023-10-26T05:49:03.598665+0000","last_became_active":"2023-10-26T05:49:02.572710+0000","last_became_peered":"2023-10-26T05:49:02.572710+0000","last_unstale":"2023-10-26T05:49:03.598665+0000","last_undegraded":"2023-10-26T05:49:03.598665+0000","last_fullsized":"2023-10-26T05:49:03.598665+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:57:53.394547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.611674+0000","last_change":"2023-10-26T05:49:03.611200+0000","last_active":"2023-10-26T05:49:03.611674+0000","last_peered":"2023-10-26T05:49:03.611674+0000","last_clean":"2023-10-26T05:49:03.611674+0000","last_became_active":"2023-10-26T05:49:03.610952+0000","last_became_peered":"2023-10-26T05:49:03.610952+0000","last_unstale":"2023-10-26T05:49:03.611674+0000","last_undegraded":"2023-10-26T05:49:03.611674+0000","last_fullsized":"2023-10-26T05:49:03.611674+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:02.551926+0000","last_clean_scrub_stamp":"2023-10-26T05:49:02.551926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:47.718823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-10-26T05:49:03.636324+0000","last_change":"2023-10-26T05:49:02.580467+0000","last_active":"2023-10-26T05:49:03.636324+0000","last_peered":"2023-10-26T05:49:03.636324+0000","last_clean":"2023-10-26T05:49:03.636324+0000","last_became_active":"2023-10-26T05:49:02.579843+0000","last_became_peered":"2023-10-26T05:49:02.579843+0000","last_unstale":"2023-10-26T05:49:03.636324+0000","last_undegraded":"2023-10-26T05:49:03.636324+0000","last_fullsized":"2023-10-26T05:49:03.636324+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:01.537841+0000","last_clean_scrub_stamp":"2023-10-26T05:49:01.537841+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:57.499803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":188,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-26T05:49:02.574701+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:49:02.574701+0000","last_peered":"2023-10-26T05:49:02.574701+0000","last_clean":"2023-10-26T05:49:02.574701+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:49:02.574701+0000","last_undegraded":"2023-10-26T05:49:02.574701+0000","last_fullsized":"2023-10-26T05:49:02.574701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:19.502572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":797,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194515,"statfs":{"total":200000000,"available":199183450,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.262}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.135},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.0629999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.81999999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":1.2250000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.75}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.026},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.99299999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.016},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.1579999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":1.23}]},{"osd":3,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":1.085}]},{"osd":4,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.254}]},{"osd":5,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":1.056},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":1.1180000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":1.204},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.2869999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":220,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773908,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.875}]},{"osd":1,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.91500000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.78700000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.587,"5min":1.587,"15min":1.587},"last":0.73699999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0529999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.95099999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:48:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1020000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199772296,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":1.075}]},{"osd":1,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.88400000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.97199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":1.0249999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Thu Oct 26 05:48:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.121}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195091,"statfs":{"total":200000000,"available":199773270,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.438}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.65200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195098,"statfs":{"total":200000000,"available":199780683,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":1.1950000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.26700000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":2.8809999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.82699999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":2.6139999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":5.9550000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":2.8319999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4379999999999999,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":1.046}]},{"osd":6,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":2.5760000000000001},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":6.7729999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:48:11 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":15.352}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195098,"statfs":{"total":200000000,"available":199780683,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.497}]},{"osd":3,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.85999999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.107}]},{"osd":5,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.141},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.891,"5min":1.891,"15min":1.891},"last":1.026},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.0700000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:48:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.2270000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194521,"statfs":{"total":200000000,"available":199190315,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79000000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.39400000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.753}]},{"osd":6,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:48:09 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98499999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:49:07.009 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:49:07.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:49:07.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/3857107347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 msgr2=0x7fe11c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/3857107347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe11c0047d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe12800c260 tx=0x7fe12801f760 comp rx=0 tx=0).stop 2023-10-26T05:49:07.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe138fab640 1 -- 172.21.15.71:0/3857107347 reap_dead start 2023-10-26T05:49:07.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/3857107347 shutdown_connections 2023-10-26T05:49:07.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/3857107347 >> 172.21.15.71:0/3857107347 conn(0x7fe1340ce070 msgr2=0x7fe1340ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:07.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/3857107347 shutdown_connections 2023-10-26T05:49:07.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/3857107347 wait complete. 2023-10-26T05:49:07.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 Processor -- start 2023-10-26T05:49:07.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- start start 2023-10-26T05:49:07.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe1341f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1340ca3c0 0x7fe1340ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1340cad80 0x7fe13414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13413f450 con 0x7fe11c003f00 2023-10-26T05:49:07.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340c8aa0 con 0x7fe1340cad80 2023-10-26T05:49:07.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340c8d80 con 0x7fe1340ca3c0 2023-10-26T05:49:07.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe1341f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe1341f1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52798/0 (socket says 172.21.15.71:52798) 2023-10-26T05:49:07.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 -- 172.21.15.71:0/2287219241 learned_addr learned my addr 172.21.15.71:0/2287219241 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:07.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe1337fe640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1340ca3c0 0x7fe1340ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe138fab640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1340cad80 0x7fe13414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 -- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1340ca3c0 msgr2=0x7fe1340ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1340ca3c0 0x7fe1340ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 -- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1340cad80 msgr2=0x7fe13414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1340cad80 0x7fe13414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 -- 172.21.15.71:0/2287219241 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13413f0e0 con 0x7fe11c003f00 2023-10-26T05:49:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe138fab640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1340cad80 0x7fe13414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:07.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe133fff640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe1341f1110 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe12400fa00 tx=0x7fe12400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400bcb0 con 0x7fe11c003f00 2023-10-26T05:49:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe124004030 con 0x7fe11c003f00 2023-10-26T05:49:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13413edd0 con 0x7fe11c003f00 2023-10-26T05:49:07.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124004420 con 0x7fe11c003f00 2023-10-26T05:49:07.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.177+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1340cb200 con 0x7fe11c003f00 2023-10-26T05:49:07.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.181+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134066d00 con 0x7fe11c003f00 2023-10-26T05:49:07.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.181+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe124003600 con 0x7fe11c003f00 2023-10-26T05:49:07.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.181+0000 7fe130ff9640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe120066470 0x7fe120068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.181+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1240849e0 con 0x7fe11c003f00 2023-10-26T05:49:07.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.181+0000 7fe1337fe640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe120066470 0x7fe120068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.181+0000 7fe1337fe640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe120066470 0x7fe120068930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1280b0000 tx=0x7fe12801c500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:07.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.185+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe124051ef0 con 0x7fe11c003f00 2023-10-26T05:49:07.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.377+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe13413f270 con 0x7fe11c003f00 2023-10-26T05:49:07.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe130ff9640 1 -- 172.21.15.71:0/2287219241 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe124020020 con 0x7fe11c003f00 2023-10-26T05:49:07.385 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:49:07.386 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:49:07.386 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:49:07.386 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:49:07.386 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:49:07.388 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:49:07.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe120066470 msgr2=0x7fe120068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe120066470 0x7fe120068930 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe1280b0000 tx=0x7fe12801c500 comp rx=0 tx=0).stop 2023-10-26T05:49:07.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 msgr2=0x7fe1341f1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe1341f1110 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe12400fa00 tx=0x7fe12400fec0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 shutdown_connections 2023-10-26T05:49:07.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe1340cad80 0x7fe13414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1340ca3c0 0x7fe1340ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe120066470 0x7fe120068930 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.381+0000 7fe13aa0e640 1 --2- 172.21.15.71:0/2287219241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c003f00 0x7fe1341f1110 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.385+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 >> 172.21.15.71:0/2287219241 conn(0x7fe1340ce070 msgr2=0x7fe13414df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:07.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.385+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 shutdown_connections 2023-10-26T05:49:07.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.385+0000 7fe13aa0e640 1 -- 172.21.15.71:0/2287219241 wait complete. 2023-10-26T05:49:07.402 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 -- 172.21.15.71:0/1905610517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7006fb80 msgr2=0x7f6b7015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 --2- 172.21.15.71:0/1905610517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7006fb80 0x7f6b7015b4b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b70155a00 tx=0x7f6b6001c8a0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 -- 172.21.15.71:0/1905610517 shutdown_connections 2023-10-26T05:49:07.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 --2- 172.21.15.71:0/1905610517 >> v2:172.21.15.86:3300/0 conn(0x7f6b7015b9f0 0x7f6b7013d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 --2- 172.21.15.71:0/1905610517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7006fb80 0x7f6b7015b4b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 --2- 172.21.15.71:0/1905610517 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b700ac740 0x7f6b7006f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 -- 172.21.15.71:0/1905610517 >> 172.21.15.71:0/1905610517 conn(0x7f6b70069770 msgr2=0x7f6b70069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:07.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 -- 172.21.15.71:0/1905610517 shutdown_connections 2023-10-26T05:49:07.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 -- 172.21.15.71:0/1905610517 wait complete. 2023-10-26T05:49:07.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 Processor -- start 2023-10-26T05:49:07.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.513+0000 7f6b784fa640 1 -- start start 2023-10-26T05:49:07.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b7006fb80 0x7f6b70138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b700ac740 0x7f6b700aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 0x7f6b700a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7014eb70 con 0x7f6b7015b9f0 2023-10-26T05:49:07.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700aa020 con 0x7f6b700ac740 2023-10-26T05:49:07.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7014e890 con 0x7f6b7006fb80 2023-10-26T05:49:07.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 0x7f6b700a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 0x7f6b700a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52812/0 (socket says 172.21.15.71:52812) 2023-10-26T05:49:07.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 -- 172.21.15.71:0/2929962912 learned_addr learned my addr 172.21.15.71:0/2929962912 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:07.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b75a95640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b700ac740 0x7f6b700aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76296640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b7006fb80 0x7f6b70138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 -- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b7006fb80 msgr2=0x7f6b70138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b7006fb80 0x7f6b70138820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 -- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b700ac740 msgr2=0x7f6b700aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b700ac740 0x7f6b700aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 -- 172.21.15.71:0/2929962912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009560 con 0x7f6b7015b9f0 2023-10-26T05:49:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b75a95640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b700ac740 0x7f6b700aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76a97640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 0x7f6b700a64d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e970 tx=0x7f6b6800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:07.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68004d30 con 0x7f6b7015b9f0 2023-10-26T05:49:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b6800cb20 con 0x7f6b7015b9f0 2023-10-26T05:49:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b700aa2c0 con 0x7f6b7015b9f0 2023-10-26T05:49:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68005020 con 0x7f6b7015b9f0 2023-10-26T05:49:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b700aa880 con 0x7f6b7015b9f0 2023-10-26T05:49:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b70066b10 con 0x7f6b7015b9f0 2023-10-26T05:49:07.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6b68005180 con 0x7f6b7015b9f0 2023-10-26T05:49:07.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b66ffd640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b540664a0 0x7f6b54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:07.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b68084cd0 con 0x7f6b7015b9f0 2023-10-26T05:49:07.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.517+0000 7f6b76296640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b540664a0 0x7f6b54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:07.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.521+0000 7f6b76296640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b540664a0 0x7f6b54068960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005d90 tx=0x7f6b6c005ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:07.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.521+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6b68052130 con 0x7f6b7015b9f0 2023-10-26T05:49:07.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b700721e0 con 0x7f6b7015b9f0 2023-10-26T05:49:07.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b66ffd640 1 -- 172.21.15.71:0/2929962912 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f6b6801f090 con 0x7f6b7015b9f0 2023-10-26T05:49:07.701 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:07.701 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:07.701 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:07.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b540664a0 msgr2=0x7f6b54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b540664a0 0x7f6b54068960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005d90 tx=0x7f6b6c005ca0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 msgr2=0x7f6b700a64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:07.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 0x7f6b700a64d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800e970 tx=0x7f6b6800ee30 comp rx=0 tx=0).stop 2023-10-26T05:49:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 shutdown_connections 2023-10-26T05:49:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b540664a0 0x7f6b54068960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b7015b9f0 0x7f6b700a64d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b700ac740 0x7f6b700aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 --2- 172.21.15.71:0/2929962912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b7006fb80 0x7f6b70138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 >> 172.21.15.71:0/2929962912 conn(0x7f6b70069770 msgr2=0x7f6b70158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 shutdown_connections 2023-10-26T05:49:07.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:07.697+0000 7f6b784fa640 1 -- 172.21.15.71:0/2929962912 wait complete. 2023-10-26T05:49:08.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:49:02.915412+0000', 'modified': '2023-10-26T05:49:04.973374+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5778': {'gid': 5778, 'name': 'c', 'rank': 0, 'incarnation': 16, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/1106692810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 1106692810}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1106692810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:49:08.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:49:08.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T05:49:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 -- 172.21.15.71:0/206625966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 msgr2=0x7f1c44004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 --2- 172.21.15.71:0/206625966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c44004c50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c700 tx=0x7f1c5001c650 comp rx=0 tx=0).stop 2023-10-26T05:49:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 -- 172.21.15.71:0/206625966 shutdown_connections 2023-10-26T05:49:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 --2- 172.21.15.71:0/206625966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c44004c50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 -- 172.21.15.71:0/206625966 >> 172.21.15.71:0/206625966 conn(0x7f1c5c06b8c0 msgr2=0x7f1c5c06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 -- 172.21.15.71:0/206625966 shutdown_connections 2023-10-26T05:49:08.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 -- 172.21.15.71:0/206625966 wait complete. 2023-10-26T05:49:08.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 Processor -- start 2023-10-26T05:49:08.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.253+0000 7f1c60d09640 1 -- start start 2023-10-26T05:49:08.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c5c0a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c5c1f85f0 0x7f1c5c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c5c151b60 0x7f1c5c13a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1f5100 con 0x7f1c440047d0 2023-10-26T05:49:08.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1f8b60 con 0x7f1c5c151b60 2023-10-26T05:49:08.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1f8e40 con 0x7f1c5c1f85f0 2023-10-26T05:49:08.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c5c0a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c5c0a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52826/0 (socket says 172.21.15.71:52826) 2023-10-26T05:49:08.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 -- 172.21.15.71:0/171064365 learned_addr learned my addr 172.21.15.71:0/171064365 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:08.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c5c151b60 0x7f1c5c13a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c5c1f85f0 0x7f1c5c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 -- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c5c1f85f0 msgr2=0x7f1c5c0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c5c1f85f0 0x7f1c5c0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 -- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c5c151b60 msgr2=0x7f1c5c13a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c5c151b60 0x7f1c5c13a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 -- 172.21.15.71:0/171064365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c1f9120 con 0x7f1c440047d0 2023-10-26T05:49:08.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c5c1f85f0 0x7f1c5c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:08.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c5c0a6820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00e6a0 tx=0x7f1c4c00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c4affd640 1 -- 172.21.15.71:0/171064365 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00ca50 con 0x7f1c440047d0 2023-10-26T05:49:08.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c4affd640 1 -- 172.21.15.71:0/171064365 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c4c00cca0 con 0x7f1c440047d0 2023-10-26T05:49:08.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c1f9400 con 0x7f1c440047d0 2023-10-26T05:49:08.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c4affd640 1 -- 172.21.15.71:0/171064365 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0116d0 con 0x7f1c440047d0 2023-10-26T05:49:08.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0b0ee0 con 0x7f1c440047d0 2023-10-26T05:49:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c4affd640 1 -- 172.21.15.71:0/171064365 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c4c011830 con 0x7f1c440047d0 2023-10-26T05:49:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c4affd640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c3c0664a0 0x7f1c3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.257+0000 7f1c4affd640 1 -- 172.21.15.71:0/171064365 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c4c084960 con 0x7f1c440047d0 2023-10-26T05:49:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c3c0664a0 0x7f1c3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 Processor -- start 2023-10-26T05:49:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- start start 2023-10-26T05:49:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c3c0664a0 0x7f1c3c068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1c440057b0 tx=0x7f1c500193f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c103da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c0b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0ba360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0beaecc0 con 0x560d0c0b9f80 2023-10-26T05:49:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bf11920 con 0x560d0c1042e0 2023-10-26T05:49:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0bf0ee50 con 0x560d0c1039c0 2023-10-26T05:49:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0ba360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0ba360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52842/0 (socket says 172.21.15.71:52842) 2023-10-26T05:49:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 -- 172.21.15.71:0/2690133389 learned_addr learned my addr 172.21.15.71:0/2690133389 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c0b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 -- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 msgr2=0x560d0c103da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c103da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 -- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 msgr2=0x560d0c0b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c0b9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 -- 172.21.15.71:0/2690133389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0bc03890 con 0x560d0c0b9f80 2023-10-26T05:49:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0ba360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0cb690 tx=0x7f1c54004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f427a3695c180593 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c3bfff640 1 -- 172.21.15.71:0/2690133389 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54010bd0 con 0x560d0c0b9f80 2023-10-26T05:49:08.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c3bfff640 1 -- 172.21.15.71:0/2690133389 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c54010e20 con 0x560d0c0b9f80 2023-10-26T05:49:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c3bfff640 1 -- 172.21.15.71:0/2690133389 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540126a0 con 0x560d0c0b9f80 2023-10-26T05:49:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- 172.21.15.71:0/2690133389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 msgr2=0x560d0c0ba360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0ba360 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0cb690 tx=0x7f1c54004ed0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- 172.21.15.71:0/2690133389 shutdown_connections 2023-10-26T05:49:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0ba360 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c0b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 --2- 172.21.15.71:0/2690133389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- 172.21.15.71:0/2690133389 >> 172.21.15.71:0/2690133389 conn(0x560d0c0b12f0 msgr2=0x560d0c0b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- 172.21.15.71:0/2690133389 shutdown_connections 2023-10-26T05:49:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- 172.21.15.71:0/2690133389 wait complete. 2023-10-26T05:49:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 -- start start 2023-10-26T05:49:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.261+0000 7f1c63e53000 1 Processor -- start 2023-10-26T05:49:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1039c0 0x560d0c0bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1042e0 0x560d0c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bf0a740 con 0x560d0c0b9f80 2023-10-26T05:49:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bec9d10 con 0x560d0c1039c0 2023-10-26T05:49:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0bec99b0 con 0x560d0c1042e0 2023-10-26T05:49:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 10 client.0 _ll_register_callbacks cb 0x560d0c0a56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:49:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0bf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52848/0 (socket says 172.21.15.71:52848) 2023-10-26T05:49:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 -- 172.21.15.71:0/3999112560 learned_addr learned my addr 172.21.15.71:0/3999112560 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1039c0 0x560d0c0bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1042e0 0x560d0c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 -- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1042e0 msgr2=0x560d0c0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1042e0 0x560d0c0c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 -- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1039c0 msgr2=0x560d0c0bf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1039c0 0x560d0c0bf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 -- 172.21.15.71:0/3999112560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c0cc4f0 con 0x560d0c0b9f80 2023-10-26T05:49:08.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0bf3b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c05d7d0 tx=0x7f1c4c056100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aec703945ea62e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:49:08.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 1 -- 172.21.15.71:0/3999112560 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0550f0 con 0x560d0c0b9f80 2023-10-26T05:49:08.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 1 -- 172.21.15.71:0/3999112560 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c4c059b50 con 0x560d0c0b9f80 2023-10-26T05:49:08.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d0bebe370 con 0x560d0c0b9f80 2023-10-26T05:49:08.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 1 -- 172.21.15.71:0/3999112560 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c4c05dd00 con 0x560d0c0b9f80 2023-10-26T05:49:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 10 client.5922 fetch_fsmap learned FSMap version 18 2023-10-26T05:49:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x560d0becc460 con 0x560d0c0b9f80 2023-10-26T05:49:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 1 -- 172.21.15.71:0/3999112560 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x560d0bebe370 con 0x560d0c0b9f80 2023-10-26T05:49:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 1 -- 172.21.15.71:0/3999112560 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c4c018020 con 0x560d0c0b9f80 2023-10-26T05:49:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 10 client.5922 fetch_fsmap finished waiting for FSMap version 18 2023-10-26T05:49:08.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 10 client.5922 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:49:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 20 client.5922 populate_metadata read hostname 'smithi071' 2023-10-26T05:49:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c0c65f0 0x560d0c115a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 4 client.5922 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:49:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 --> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d0bea42d0 con 0x560d0c0c65f0 2023-10-26T05:49:08.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c0c65f0 0x560d0c115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 10 client.5922 ms_handle_connect on v2:172.21.15.71:6834/1106692810 2023-10-26T05:49:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c0c65f0 0x560d0c115a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb16756d9297c73f server_cookie=5c48d98bf463f26 in_seq=0 out_seq=0 2023-10-26T05:49:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 1 -- 172.21.15.71:0/3999112560 <== mds.0 v2:172.21.15.71:6834/1106692810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560d0bea42d0 con 0x560d0c0c65f0 2023-10-26T05:49:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 10 client.5922 handle_command_reply: tid=0 2023-10-26T05:49:08.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 client.5922 shutdown 2023-10-26T05:49:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 msgr2=0x560d0c0bf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0bf3b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c05d7d0 tx=0x7f1c4c056100 comp rx=0 tx=0).stop 2023-10-26T05:49:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 shutdown_connections 2023-10-26T05:49:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c0c65f0 0x560d0c115a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1042e0 0x560d0c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c397fa640 0 client.5922 ms_handle_reset on v2:172.21.15.71:6834/1106692810 2023-10-26T05:49:08.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1039c0 0x560d0c0bf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 --2- 172.21.15.71:0/3999112560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c0bf3b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 >> 172.21.15.71:0/3999112560 conn(0x560d0c0b12f0 msgr2=0x560d0c0b8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 shutdown_connections 2023-10-26T05:49:08.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 1 -- 172.21.15.71:0/3999112560 wait complete. 2023-10-26T05:49:08.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.265+0000 7f1c63e53000 20 client.5922 trim_cache size 0 max 16384 2023-10-26T05:49:08.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 Processor -- start 2023-10-26T05:49:08.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 -- start start 2023-10-26T05:49:08.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c102800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c113a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bbc4940 con 0x560d0c0b9f80 2023-10-26T05:49:08.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bf04a40 con 0x560d0c1042e0 2023-10-26T05:49:08.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0bec94d0 con 0x560d0c1039c0 2023-10-26T05:49:08.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c1022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52858/0 (socket says 172.21.15.71:52858) 2023-10-26T05:49:08.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 -- 172.21.15.71:0/4213657483 learned_addr learned my addr 172.21.15.71:0/4213657483 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:08.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 -- 172.21.15.71:0/4213657483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 msgr2=0x560d0c102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 -- 172.21.15.71:0/4213657483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 msgr2=0x560d0c113a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 -- 172.21.15.71:0/4213657483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0beb2760 con 0x560d0c0b9f80 2023-10-26T05:49:08.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c1022c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c055990 tx=0x7f1c4c082cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5176d7aedcba3fbf server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c48ff9640 1 -- 172.21.15.71:0/4213657483 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c05d8c0 con 0x560d0c0b9f80 2023-10-26T05:49:08.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c48ff9640 1 -- 172.21.15.71:0/4213657483 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c4c059b50 con 0x560d0c0b9f80 2023-10-26T05:49:08.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.281+0000 7f1c48ff9640 1 -- 172.21.15.71:0/4213657483 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c05dd00 con 0x560d0c0b9f80 2023-10-26T05:49:08.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/4213657483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 msgr2=0x560d0c1022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c1022c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c055990 tx=0x7f1c4c082cc0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/4213657483 shutdown_connections 2023-10-26T05:49:08.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c113a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c1039c0 0x560d0c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/4213657483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c0b9f80 0x560d0c1022c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/4213657483 >> 172.21.15.71:0/4213657483 conn(0x560d0c0b12f0 msgr2=0x560d0c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/4213657483 shutdown_connections 2023-10-26T05:49:08.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/4213657483 wait complete. 2023-10-26T05:49:08.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- start start 2023-10-26T05:49:08.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 Processor -- start 2023-10-26T05:49:08.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c0b9f80 0x560d0c1168d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 0x560d0c11ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c11f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bed3720 con 0x560d0c1039c0 2023-10-26T05:49:08.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0bf060b0 con 0x560d0c1042e0 2023-10-26T05:49:08.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d0bf0d2a0 con 0x560d0c0b9f80 2023-10-26T05:49:08.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 10 client.0 _ll_register_callbacks cb 0x560d0bf08ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:49:08.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 0x560d0c11ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 0x560d0c11ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52870/0 (socket says 172.21.15.71:52870) 2023-10-26T05:49:08.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 -- 172.21.15.71:0/36260631 learned_addr learned my addr 172.21.15.71:0/36260631 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:08.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c0b9f80 0x560d0c1168d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c11f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 -- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c0b9f80 msgr2=0x560d0c1168d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c0b9f80 0x560d0c1168d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 -- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 msgr2=0x560d0c11f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:49:08.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c11f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 -- 172.21.15.71:0/36260631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d0becc460 con 0x560d0c1039c0 2023-10-26T05:49:08.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c5ad9d640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c11f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:08.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:49:08.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c59d9b640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 0x560d0c11ee30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1c5001cb30 tx=0x7f1c44005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2e562e25801b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 1 -- 172.21.15.71:0/36260631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500c8070 con 0x560d0c1039c0 2023-10-26T05:49:08.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 1 -- 172.21.15.71:0/36260631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c50004c60 con 0x560d0c1039c0 2023-10-26T05:49:08.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d0c110e90 con 0x560d0c1039c0 2023-10-26T05:49:08.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 1 -- 172.21.15.71:0/36260631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c50004e20 con 0x560d0c1039c0 2023-10-26T05:49:08.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 10 client.5928 fetch_fsmap learned FSMap version 18 2023-10-26T05:49:08.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x560d0bc03cc0 con 0x560d0c1039c0 2023-10-26T05:49:08.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 1 -- 172.21.15.71:0/36260631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x560d0c110e90 con 0x560d0c1039c0 2023-10-26T05:49:08.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 1 -- 172.21.15.71:0/36260631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1c500b6890 con 0x560d0c1039c0 2023-10-26T05:49:08.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 10 client.5928 fetch_fsmap finished waiting for FSMap version 18 2023-10-26T05:49:08.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 10 client.5928 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:49:08.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 20 client.5928 populate_metadata read hostname 'smithi071' 2023-10-26T05:49:08.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c11f8b0 0x560d0c16a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 4 client.5928 mds_command: new command op to 5778 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:49:08.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 --> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560d0bea26f0 con 0x560d0c11f8b0 2023-10-26T05:49:08.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c11f8b0 0x560d0c16a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c5a59c640 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c11f8b0 0x560d0c16a2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33bcc8b70f486244 server_cookie=8f7983aba78330c9 in_seq=0 out_seq=0 2023-10-26T05:49:08.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 10 client.5928 ms_handle_connect on v2:172.21.15.71:6834/1106692810 2023-10-26T05:49:08.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 1 -- 172.21.15.71:0/36260631 <== mds.0 v2:172.21.15.71:6834/1106692810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560d0bea26f0 con 0x560d0c11f8b0 2023-10-26T05:49:08.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 10 client.5928 handle_command_reply: tid=0 2023-10-26T05:49:08.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 client.5928 shutdown 2023-10-26T05:49:08.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 msgr2=0x560d0c11ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 0x560d0c11ee30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1c5001cb30 tx=0x7f1c44005800 comp rx=0 tx=0).stop 2023-10-26T05:49:08.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 shutdown_connections 2023-10-26T05:49:08.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:6834/1106692810,v1:172.21.15.71:6835/1106692810] conn(0x560d0c11f8b0 0x560d0c16a2a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560d0c1042e0 0x560d0c11f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d0c1039c0 0x560d0c11ee30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c397fa640 0 client.5928 ms_handle_reset on v2:172.21.15.71:6834/1106692810 2023-10-26T05:49:08.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 --2- 172.21.15.71:0/36260631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x560d0c0b9f80 0x560d0c1168d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 >> 172.21.15.71:0/36260631 conn(0x560d0c0b12f0 msgr2=0x560d0c0c5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 shutdown_connections 2023-10-26T05:49:08.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.285+0000 7f1c63e53000 1 -- 172.21.15.71:0/36260631 wait complete. 2023-10-26T05:49:08.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c63e53000 20 client.5928 trim_cache size 0 max 16384 2023-10-26T05:49:08.314 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:08.315 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:08.315 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:49:08.315 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5778, 2023-10-26T05:49:08.315 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:49:08.315 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:49:08.316 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:49:08.316 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 4.6982234969999999, 2023-10-26T05:49:08.316 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 18, 2023-10-26T05:49:08.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 48, 2023-10-26T05:49:08.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 48, 2023-10-26T05:49:08.317 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 10.403020054000001 2023-10-26T05:49:08.317 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:08.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c3c0664a0 msgr2=0x7f1c3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c3c0664a0 0x7f1c3c068960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1c440057b0 tx=0x7f1c500193f0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 msgr2=0x7f1c5c0a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c5c0a6820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00e6a0 tx=0x7f1c4c00eb60 comp rx=0 tx=0).stop 2023-10-26T05:49:08.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 shutdown_connections 2023-10-26T05:49:08.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c5c151b60 0x7f1c5c13a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c5c1f85f0 0x7f1c5c0b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c3c0664a0 0x7f1c3c068960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 --2- 172.21.15.71:0/171064365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c440047d0 0x7f1c5c0a6820 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 >> 172.21.15.71:0/171064365 conn(0x7f1c5c06b8c0 msgr2=0x7f1c5c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 shutdown_connections 2023-10-26T05:49:08.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.289+0000 7f1c60d09640 1 -- 172.21.15.71:0/171064365 wait complete. 2023-10-26T05:49:08.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:49:08.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 -- 172.21.15.71:0/2825167296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b00cb010 msgr2=0x7f10b006f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 --2- 172.21.15.71:0/2825167296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b00cb010 0x7f10b006f640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f10b00afd40 tx=0x7f10a801caa0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 -- 172.21.15.71:0/2825167296 shutdown_connections 2023-10-26T05:49:08.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 --2- 172.21.15.71:0/2825167296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b00cb010 0x7f10b006f640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 --2- 172.21.15.71:0/2825167296 >> v2:172.21.15.86:3300/0 conn(0x7f10b00b2890 0x7f10b00caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 --2- 172.21.15.71:0/2825167296 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10b013c2f0 0x7f10b00b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 -- 172.21.15.71:0/2825167296 >> 172.21.15.71:0/2825167296 conn(0x7f10b0069770 msgr2=0x7f10b0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 -- 172.21.15.71:0/2825167296 shutdown_connections 2023-10-26T05:49:08.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 -- 172.21.15.71:0/2825167296 wait complete. 2023-10-26T05:49:08.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 Processor -- start 2023-10-26T05:49:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.437+0000 7f10b73b9640 1 -- start start 2023-10-26T05:49:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b00b2890 0x7f10b0136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10b00cb010 0x7f10b013afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 0x7f10b013b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0138eb0 con 0x7f10b013c2f0 2023-10-26T05:49:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01381b0 con 0x7f10b00b2890 2023-10-26T05:49:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0138490 con 0x7f10b00cb010 2023-10-26T05:49:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 0x7f10b013b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 0x7f10b013b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52884/0 (socket says 172.21.15.71:52884) 2023-10-26T05:49:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 -- 172.21.15.71:0/3516058371 learned_addr learned my addr 172.21.15.71:0/3516058371 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b4954640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10b00cb010 0x7f10b013afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 -- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10b00cb010 msgr2=0x7f10b013afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5155640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b00b2890 0x7f10b0136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10b00cb010 0x7f10b013afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 -- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b00b2890 msgr2=0x7f10b0136ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b00b2890 0x7f10b0136ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 -- 172.21.15.71:0/3516058371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a80098e0 con 0x7f10b013c2f0 2023-10-26T05:49:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5956640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 0x7f10b013b4f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f10a801b150 tx=0x7f10a80a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8003b90 con 0x7f10b013c2f0 2023-10-26T05:49:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f10a8003de0 con 0x7f10b013c2f0 2023-10-26T05:49:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b0138860 con 0x7f10b013c2f0 2023-10-26T05:49:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004290 con 0x7f10b013c2f0 2023-10-26T05:49:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b00a9eb0 con 0x7f10b013c2f0 2023-10-26T05:49:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0066b10 con 0x7f10b013c2f0 2023-10-26T05:49:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10a80b4040 con 0x7f10b013c2f0 2023-10-26T05:49:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10a5ffb640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1094066470 0x7f1094068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:08.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10a811e750 con 0x7f10b013c2f0 2023-10-26T05:49:08.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.441+0000 7f10b5155640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1094066470 0x7f1094068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:08.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.445+0000 7f10b5155640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1094066470 0x7f1094068930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f10b015c1f0 tx=0x7f10ac005c50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:08.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.445+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f10a80ebc60 con 0x7f10b013c2f0 2023-10-26T05:49:08.532 INFO:tasks.ceph.mds.c.smithi071.stderr:2023-10-26T05:49:08.529+0000 7f86ea962640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:49:08.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f10b00721e0 con 0x7f10b013c2f0 2023-10-26T05:49:08.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10a5ffb640 1 -- 172.21.15.71:0/3516058371 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f10a80b2060 con 0x7f10b013c2f0 2023-10-26T05:49:08.629 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:49:08.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1094066470 msgr2=0x7f1094068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1094066470 0x7f1094068930 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f10b015c1f0 tx=0x7f10ac005c50 comp rx=0 tx=0).stop 2023-10-26T05:49:08.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 msgr2=0x7f10b013b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:08.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 0x7f10b013b4f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f10a801b150 tx=0x7f10a80a5890 comp rx=0 tx=0).stop 2023-10-26T05:49:08.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 shutdown_connections 2023-10-26T05:49:08.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1094066470 0x7f1094068930 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b013c2f0 0x7f10b013b4f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10b00cb010 0x7f10b013afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 --2- 172.21.15.71:0/3516058371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b00b2890 0x7f10b0136ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:08.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 >> 172.21.15.71:0/3516058371 conn(0x7f10b0069770 msgr2=0x7f10b0157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:08.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 shutdown_connections 2023-10-26T05:49:08.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:08.625+0000 7f10b73b9640 1 -- 172.21.15.71:0/3516058371 wait complete. 2023-10-26T05:49:08.646 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:49:08.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:49:08.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:49:08.647 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:49:08.647 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:08.647 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:49:08.648 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:49:08.648 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:49:08.648 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:49:08.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:08.649 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:49:08.655 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:49:08.655 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:49:08.655 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:49:08.655 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.655 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:49:08.656 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.656 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:49:08.656 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:08.656 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:49:08.656 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:49:08.657 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:08.657 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:49:08.657 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:49:08.657 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:08.657 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:49:08.657 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:08.658 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:49:08.658 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42428sec preferred_lft 42428sec 2023-10-26T05:49:08.658 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:49:08.658 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.658 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:49:08.659 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:08.659 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:49:08.659 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.659 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:49:08.659 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:08.659 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:49:08.660 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.660 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:49:08.660 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.660 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:49:08.660 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:08.661 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:49:08.661 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.661 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:49:08.661 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:08.661 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:49:08.662 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:08.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:08.662 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:49:08.709 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:49:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:08.711 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:49:08.762 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:49:08.762 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:49:08.763 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:49:08.763 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:49:08.763 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:49:08.763 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:49:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:49:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:49:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:49:08.764 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:49:08.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:08.765 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:49:08.765 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:49:08.766 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:49:08.766 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:49:08.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:49:08.767 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:49:08.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:08.836 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:49:08.888 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:49:08.889 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:49:08.889 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:49:08.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:08.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:49:08.943 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:49:08.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:49:08.944 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:08.992 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:49:08.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:49:08.993 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:09.040 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:49:09.089 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:49:09.089 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:49:09.090 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:49:09.090 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:49:09.090 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:49:09.091 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:49:09.091 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:49:09.091 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:49:09.091 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:49:09.091 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:49:09.092 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:49:09.092 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:49:09.092 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:49:09.092 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:49:09.093 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:49:09.093 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:49:09.093 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:49:09.093 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:49:09.093 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:49:09.094 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:49:09.094 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:49:09.094 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:49:09.094 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:49:09.095 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:49:09.095 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:49:09.095 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:49:09.095 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:49:09.096 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:49:09.096 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:49:09.096 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:09.096 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:09.096 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:09.097 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:09.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:09.098 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:49:09.144 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:49:09.144 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:49:09.189 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:49:09.195 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:49:09.196 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:49:09.196 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:49:09.196 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:49:09.196 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:49:09.197 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:49:09.197 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:49:09.197 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:49:09.197 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:49:09.197 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:49:09.198 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:49:09.198 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:49:09.198 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:49:09.198 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:49:09.198 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:49:09.199 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:49:09.199 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:49:09.199 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:49:09.199 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:49:09.200 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:49:09.200 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:49:09.200 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:49:09.200 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:49:09.200 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:49:09.201 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:49:09.201 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:49:09.201 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:49:09.201 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:49:09.201 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:49:09.202 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:09.202 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:09.202 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:09.202 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:09.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:09.204 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:49:09.269 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:49:09.265+0000 7f4a0d000580 -1 init, newargv = 0x5630a38ed210 newargc=15 2023-10-26T05:49:09.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[40433]: starting ceph client 2023-10-26T05:49:09.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[40433]: starting fuse 2023-10-26T05:49:10.357 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:49:10.367 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:49:10.367 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:49:10.367 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:49:10.367 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:49:10.368 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:49:10.368 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:49:10.368 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:49:10.368 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:49:10.368 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:49:10.369 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:49:10.369 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:49:10.369 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:49:10.369 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:49:10.369 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:49:10.370 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:49:10.370 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:49:10.370 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:49:10.370 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:49:10.370 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:49:10.371 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:49:10.371 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:49:10.371 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:49:10.371 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:49:10.371 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:49:10.372 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:49:10.372 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:49:10.372 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:49:10.372 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:49:10.372 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:49:10.373 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:10.373 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:49:10.373 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:10.373 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:49:10.373 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:10.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:10.375 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:49:10.420 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:49:10.420 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:49:10.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:49:10.421 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:49:10.422 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:49:10.422 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:49:10.422 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:49:10.423 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:49:10.423 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:49:10.424 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:49:10.424 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:49:10.424 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:49:10.425 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:49:10.425 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:49:10.426 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:49:10.426 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:49:10.426 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:49:10.427 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:49:10.427 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:49:10.427 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:49:10.428 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:49:10.428 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:49:10.429 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:49:10.429 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:49:10.429 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:49:10.430 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:49:10.430 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:49:10.431 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:49:10.431 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:49:10.544 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.40433.asok 2023-10-26T05:49:10.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:10.556 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40433.asok 2023-10-26T05:49:10.556 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:10.557 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.40433.asok status 2023-10-26T05:49:10.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:10.750 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:10.751 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:49:10.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:49:10.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:49:10.752 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:49:10.752 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:49:10.752 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:49:10.752 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "40433", 2023-10-26T05:49:10.752 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:49:10.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:10.753 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:49:10.753 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:49:10.753 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 5943, 2023-10-26T05:49:10.753 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:49:10.754 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:49:10.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:49:10.754 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 5943 2023-10-26T05:49:10.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:10.754 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:49:10.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:10.755 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:49:10.755 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 47289067 2023-10-26T05:49:10.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:10.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:10.756 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:49:10.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:10.756 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:49:10.756 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 47289067 2023-10-26T05:49:10.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:10.757 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.5943 192.168.0.1:0/47289067", 2023-10-26T05:49:10.757 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/47289067", 2023-10-26T05:49:10.757 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:49:10.757 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 18, 2023-10-26T05:49:10.758 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 48, 2023-10-26T05:49:10.758 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:49:10.758 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:49:10.758 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:49:10.758 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:10.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:10.770 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:10.770 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:49:10.771 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:10.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:10.815 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:49:10.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:10.822 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:10.822 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:10.823 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:10.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:10.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:10.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:49:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/3455943675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 msgr2=0x7f16480b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 --2- 172.21.15.71:0/3455943675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f16480b1900 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1640009bc0 tx=0x7f164001cab0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/3455943675 shutdown_connections 2023-10-26T05:49:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 --2- 172.21.15.71:0/3455943675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f16480b1900 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/3455943675 >> 172.21.15.71:0/3455943675 conn(0x7f1648069770 msgr2=0x7f1648069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/3455943675 shutdown_connections 2023-10-26T05:49:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/3455943675 wait complete. 2023-10-26T05:49:11.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 Processor -- start 2023-10-26T05:49:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- start start 2023-10-26T05:49:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164814e300 0x7f16481493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164814e6e0 0x7f16480abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f164814eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:11.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648151a10 con 0x7f164815b700 2023-10-26T05:49:11.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1648148630 con 0x7f164814e6e0 2023-10-26T05:49:11.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1648148910 con 0x7f164814e300 2023-10-26T05:49:11.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f164814eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:11.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f164814eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52900/0 (socket says 172.21.15.71:52900) 2023-10-26T05:49:11.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 -- 172.21.15.71:0/4228391722 learned_addr learned my addr 172.21.15.71:0/4228391722 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:11.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f1647fff640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164814e6e0 0x7f16480abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:11.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164cafb640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164814e300 0x7f16481493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:11.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 -- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164814e300 msgr2=0x7f16481493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:11.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164814e300 0x7f16481493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 -- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164814e6e0 msgr2=0x7f16480abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164814e6e0 0x7f16480abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 -- 172.21.15.71:0/4228391722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640009820 con 0x7f164815b700 2023-10-26T05:49:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164d2fc640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f164814eb20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f16400196c0 tx=0x7f16400a59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640004c50 con 0x7f164815b700 2023-10-26T05:49:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1640003f60 con 0x7f164815b700 2023-10-26T05:49:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1648148bf0 con 0x7f164815b700 2023-10-26T05:49:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640004ea0 con 0x7f164815b700 2023-10-26T05:49:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.041+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16480abff0 con 0x7f164815b700 2023-10-26T05:49:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.045+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648066b10 con 0x7f164815b700 2023-10-26T05:49:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.045+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f16400b4040 con 0x7f164815b700 2023-10-26T05:49:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.045+0000 7f16457fa640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1628066470 0x7f1628068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.045+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f164011e0d0 con 0x7f164815b700 2023-10-26T05:49:11.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.045+0000 7f164cafb640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1628066470 0x7f1628068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.045+0000 7f164cafb640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1628066470 0x7f1628068930 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1638007fd0 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:11.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.049+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f16400eb680 con 0x7f164815b700 2023-10-26T05:49:11.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1648070b40 con 0x7f164815b700 2023-10-26T05:49:11.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f16457fa640 1 -- 172.21.15.71:0/4228391722 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f16400b2070 con 0x7f164815b700 2023-10-26T05:49:11.241 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:49:11.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1628066470 msgr2=0x7f1628068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:11.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1628066470 0x7f1628068930 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1638007fd0 tx=0x7f1638009040 comp rx=0 tx=0).stop 2023-10-26T05:49:11.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 msgr2=0x7f164814eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:11.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f164814eb20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f16400196c0 tx=0x7f16400a59f0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 shutdown_connections 2023-10-26T05:49:11.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1628066470 0x7f1628068930 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f164815b700 0x7f164814eb20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f164814e6e0 0x7f16480abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 --2- 172.21.15.71:0/4228391722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f164814e300 0x7f16481493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:11.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 >> 172.21.15.71:0/4228391722 conn(0x7f1648069770 msgr2=0x7f1648159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:11.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 shutdown_connections 2023-10-26T05:49:11.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:11.237+0000 7f164ed5f640 1 -- 172.21.15.71:0/4228391722 wait complete. 2023-10-26T05:49:11.260 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:49:11.260 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:49:11.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:49:11.261 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:49:11.261 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:11.261 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:49:11.262 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:49:11.262 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:49:11.262 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:49:11.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.263 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:49:11.270 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:49:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:49:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:49:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:49:11.271 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.272 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:49:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:49:11.272 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:49:11.272 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:49:11.273 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:49:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:11.273 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:49:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:11.273 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:49:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42420sec preferred_lft 42420sec 2023-10-26T05:49:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:49:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.274 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:49:11.274 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:49:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.275 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:49:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:49:11.275 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:49:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:49:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.276 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:49:11.276 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:49:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.277 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:49:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:49:11.277 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:49:11.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.278 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:49:11.327 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:49:11.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.328 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:49:11.382 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:49:11.382 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:49:11.383 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:49:11.383 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:49:11.383 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:49:11.383 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:49:11.383 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:49:11.384 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:49:11.384 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:49:11.384 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:49:11.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.385 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:49:11.385 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:49:11.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:49:11.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:49:11.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:49:11.387 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:49:11.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.461 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:49:11.507 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:49:11.508 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:49:11.508 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:49:11.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:49:11.562 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:49:11.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:49:11.563 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:11.611 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:49:11.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:49:11.612 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:11.659 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:49:11.708 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:49:11.708 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:49:11.709 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:49:11.709 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:49:11.709 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:49:11.709 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:49:11.709 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:49:11.710 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:49:11.710 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:49:11.710 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:49:11.710 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:49:11.710 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:49:11.711 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:49:11.711 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:49:11.711 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:49:11.711 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:49:11.711 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:49:11.712 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:49:11.712 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:49:11.712 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:49:11.712 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:49:11.712 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:49:11.713 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:49:11.713 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:49:11.713 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:49:11.713 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:49:11.713 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:49:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:49:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:49:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:11.715 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:11.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.716 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:49:11.763 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:49:11.764 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:49:11.808 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:49:11.815 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:49:11.815 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:49:11.815 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:49:11.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:49:11.816 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:49:11.816 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:49:11.816 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:49:11.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:49:11.817 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:49:11.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:49:11.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:49:11.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:49:11.817 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:49:11.818 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:49:11.818 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:49:11.818 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:49:11.818 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:49:11.818 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:49:11.819 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:49:11.819 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:49:11.819 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:49:11.819 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:49:11.819 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:49:11.820 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:49:11.820 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:49:11.820 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:49:11.820 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:49:11.820 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:11.821 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:11.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:11.823 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:49:11.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:49:11.888+0000 7f90668d2580 -1 init, newargv = 0x55b18a1c5240 newargc=15 2023-10-26T05:49:11.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34206]: starting ceph client 2023-10-26T05:49:11.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34206]: starting fuse 2023-10-26T05:49:12.972 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:49:12.980 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:49:12.980 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:49:12.980 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:49:12.981 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:49:12.981 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:49:12.981 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:49:12.981 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:49:12.981 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:49:12.982 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:49:12.982 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:49:12.982 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:49:12.982 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:49:12.982 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:49:12.983 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:49:12.983 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:49:12.983 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:49:12.983 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:49:12.983 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:49:12.984 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:49:12.984 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:49:12.984 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:49:12.984 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:49:12.984 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:49:12.985 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:49:12.985 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:49:12.985 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:49:12.985 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:49:12.985 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:49:12.986 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:49:12.986 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:12.986 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:49:12.986 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:12.986 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:49:12.986 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:12.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:12.988 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:49:13.035 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:49:13.036 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:49:13.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:49:13.037 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:49:13.037 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:49:13.038 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:49:13.038 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:49:13.038 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:49:13.039 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:49:13.039 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:49:13.040 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:49:13.040 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:49:13.040 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:49:13.041 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:49:13.041 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:49:13.041 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:49:13.042 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:49:13.042 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:49:13.043 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:49:13.043 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:49:13.044 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:49:13.044 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:49:13.045 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:49:13.045 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:49:13.046 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:49:13.046 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:49:13.046 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:49:13.047 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:49:13.047 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:49:13.159 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.34206.asok 2023-10-26T05:49:13.163 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:13.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34206.asok 2023-10-26T05:49:13.172 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:13.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.34206.asok status 2023-10-26T05:49:13.263 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:13.364 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:49:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:49:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:49:13.365 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:49:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:49:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:49:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:49:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34206", 2023-10-26T05:49:13.366 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:49:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:49:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:49:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:49:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4922, 2023-10-26T05:49:13.367 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:49:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:49:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:49:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4922 2023-10-26T05:49:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:49:13.368 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:49:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:49:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:49:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4070151629 2023-10-26T05:49:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:49:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:49:13.369 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:49:13.370 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:49:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:49:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4070151629 2023-10-26T05:49:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:49:13.371 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4922 192.168.0.1:0/4070151629", 2023-10-26T05:49:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/4070151629", 2023-10-26T05:49:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:49:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 18, 2023-10-26T05:49:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 50, 2023-10-26T05:49:13.372 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:49:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:49:13.373 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:49:13.373 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:49:13.377 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:13.384 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:13.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:49:13.385 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:13.431 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:13.432 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:49:13.432 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:13.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:13.439 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:13.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:13.537 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:13.570 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:13.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2622178330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 msgr2=0x7fd0b4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2622178330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0b4005190 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c5c0 tx=0x7fd0c401d340 comp rx=0 tx=0).stop 2023-10-26T05:49:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0cad9d640 1 -- 172.21.15.71:0/2622178330 reap_dead start 2023-10-26T05:49:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2622178330 shutdown_connections 2023-10-26T05:49:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2622178330 >> 172.21.15.71:0/2622178330 conn(0x7fd0cc069770 msgr2=0x7fd0cc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2622178330 shutdown_connections 2023-10-26T05:49:13.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2622178330 wait complete. 2023-10-26T05:49:13.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 Processor -- start 2023-10-26T05:49:13.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- start start 2023-10-26T05:49:13.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0cc152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0cc153490 0x7fd0cc1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0cc1f49e0 0x7fd0cc158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b21c0 con 0x7fd0b4004d10 2023-10-26T05:49:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc13d180 con 0x7fd0cc153490 2023-10-26T05:49:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc151980 con 0x7fd0cc1f49e0 2023-10-26T05:49:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0cc152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0cc152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50532/0 (socket says 172.21.15.71:50532) 2023-10-26T05:49:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 -- 172.21.15.71:0/2918112951 learned_addr learned my addr 172.21.15.71:0/2918112951 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0cad9d640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0cc1f49e0 0x7fd0cc158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0c9d9b640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0cc153490 0x7fd0cc1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 -- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0cc1f49e0 msgr2=0x7fd0cc158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0cc1f49e0 0x7fd0cc158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 -- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0cc153490 msgr2=0x7fd0cc1582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0cc153490 0x7fd0cc1582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 -- 172.21.15.71:0/2918112951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc152c70 con 0x7fd0b4004d10 2023-10-26T05:49:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0cad9d640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0cc1f49e0 0x7fd0cc158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0c9d9b640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0cc153490 0x7fd0cc1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:13.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0ca59c640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0cc152f50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00f9d0 tx=0x7fd0bc00fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:13.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00bcb0 con 0x7fd0b4004d10 2023-10-26T05:49:13.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0bc011440 con 0x7fd0b4004d10 2023-10-26T05:49:13.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc151bd0 con 0x7fd0b4004d10 2023-10-26T05:49:13.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc011810 con 0x7fd0b4004d10 2023-10-26T05:49:13.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.705+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc152190 con 0x7fd0b4004d10 2023-10-26T05:49:13.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.709+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc066b10 con 0x7fd0b4004d10 2023-10-26T05:49:13.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.709+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0bc011970 con 0x7fd0b4004d10 2023-10-26T05:49:13.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.709+0000 7fd0baffd640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0ac0664a0 0x7fd0ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:13.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.709+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd0bc0850a0 con 0x7fd0b4004d10 2023-10-26T05:49:13.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.709+0000 7fd0c9d9b640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0ac0664a0 0x7fd0ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:13.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.709+0000 7fd0c9d9b640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0ac0664a0 0x7fd0ac068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc06a0d0 tx=0x7fd0c0006a50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:13.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.713+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0bc0524d0 con 0x7fd0b4004d10 2023-10-26T05:49:13.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.893+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0cc0721e0 con 0x7fd0b4004d10 2023-10-26T05:49:13.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.893+0000 7fd0baffd640 1 -- 172.21.15.71:0/2918112951 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd0bc055af0 con 0x7fd0b4004d10 2023-10-26T05:49:13.899 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:13.900 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:13.900 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:13.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0ac0664a0 msgr2=0x7fd0ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:13.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0ac0664a0 0x7fd0ac068960 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc06a0d0 tx=0x7fd0c0006a50 comp rx=0 tx=0).stop 2023-10-26T05:49:13.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 msgr2=0x7fd0cc152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:13.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0cc152f50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00f9d0 tx=0x7fd0bc00fe90 comp rx=0 tx=0).stop 2023-10-26T05:49:13.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 shutdown_connections 2023-10-26T05:49:13.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0cc1f49e0 0x7fd0cc158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0cc153490 0x7fd0cc1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0ac0664a0 0x7fd0ac068960 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 --2- 172.21.15.71:0/2918112951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0b4004d10 0x7fd0cc152f50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 >> 172.21.15.71:0/2918112951 conn(0x7fd0cc069770 msgr2=0x7fd0cc15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:13.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 shutdown_connections 2023-10-26T05:49:13.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:13.897+0000 7fd0d0dd2640 1 -- 172.21.15.71:0/2918112951 wait complete. 2023-10-26T05:49:13.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:49:14.038 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":5778,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.431818955000001,"mdsmap_epoch":18,"osdmap_epoch":50,"osdmap_epoch_barrier":48,"uptime":16.136626437} 2023-10-26T05:49:14.038 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 5778, "mdsmap_epoch": 18, "osdmap_epoch": 50, "osdmap_epoch_barrier": 48, "rank_uptime": 10.431818955, "state": "up:active", "uptime": 16.136626437, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:49:14.039 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:49:14.157 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:49:14.157 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:49:14.158 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-26T05:49:14.158 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:14.159 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:14.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:14.165 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-26T05:49:14.842 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:49:14.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:49:16.453 INFO:teuthology.orchestra.run.smithi071.stderr:265+0 records in 2023-10-26T05:49:16.453 INFO:teuthology.orchestra.run.smithi071.stderr:265+0 records out 2023-10-26T05:49:16.454 INFO:teuthology.orchestra.run.smithi071.stderr:277872640 bytes (278 MB, 265 MiB) copied, 2.23553 s, 124 MB/s 2023-10-26T05:49:16.455 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:16.455 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:16.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:16.462 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-26T05:49:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:292+0 records in 2023-10-26T05:49:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:292+0 records out 2023-10-26T05:49:18.798 INFO:teuthology.orchestra.run.smithi071.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.28444 s, 134 MB/s 2023-10-26T05:49:18.799 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T05:49:18.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:18.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- 172.21.15.71:0/2363675616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890140cc0 msgr2=0x7fd89015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:18.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 --2- 172.21.15.71:0/2363675616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890140cc0 0x7fd89015c760 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88801c810 comp rx=0 tx=0).stop 2023-10-26T05:49:18.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- 172.21.15.71:0/2363675616 shutdown_connections 2023-10-26T05:49:18.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 --2- 172.21.15.71:0/2363675616 >> v2:172.21.15.86:3300/0 conn(0x7fd8900b2370 0x7fd890071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:18.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 --2- 172.21.15.71:0/2363675616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890140cc0 0x7fd89015c760 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:18.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- 172.21.15.71:0/2363675616 >> 172.21.15.71:0/2363675616 conn(0x7fd890069770 msgr2=0x7fd890069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:18.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- 172.21.15.71:0/2363675616 shutdown_connections 2023-10-26T05:49:18.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- 172.21.15.71:0/2363675616 wait complete. 2023-10-26T05:49:18.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 Processor -- start 2023-10-26T05:49:18.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- start start 2023-10-26T05:49:18.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890070fe0 0x7fd8901625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:18.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 0x7fd8900c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:18.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd890140cc0 0x7fd8900c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:18.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89015bb40 con 0x7fd8900b2370 2023-10-26T05:49:18.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900b3220 con 0x7fd890070fe0 2023-10-26T05:49:18.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd896807640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900b2f40 con 0x7fd890140cc0 2023-10-26T05:49:18.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 0x7fd8900c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:18.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 0x7fd8900c37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50548/0 (socket says 172.21.15.71:50548) 2023-10-26T05:49:18.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 -- 172.21.15.71:0/3991295299 learned_addr learned my addr 172.21.15.71:0/3991295299 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:18.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd894da4640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd890140cc0 0x7fd8900c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:18.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88ffff640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890070fe0 0x7fd8901625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:18.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 -- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd890140cc0 msgr2=0x7fd8900c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:18.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd890140cc0 0x7fd8900c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:18.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 -- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890070fe0 msgr2=0x7fd8901625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:18.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890070fe0 0x7fd8901625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:18.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.933+0000 7fd88f7fe640 1 -- 172.21.15.71:0/3991295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888009560 con 0x7fd8900b2370 2023-10-26T05:49:18.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88f7fe640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 0x7fd8900c37c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c450 tx=0x7fd87800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:18.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878012830 con 0x7fd8900b2370 2023-10-26T05:49:18.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd878012aa0 con 0x7fd8900b2370 2023-10-26T05:49:18.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd890161dc0 con 0x7fd8900b2370 2023-10-26T05:49:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878011380 con 0x7fd8900b2370 2023-10-26T05:49:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd890162b00 con 0x7fd8900b2370 2023-10-26T05:49:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd890066b10 con 0x7fd8900b2370 2023-10-26T05:49:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd8780114e0 con 0x7fd8900b2370 2023-10-26T05:49:18.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88cff9640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8700664a0 0x7fd870068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:18.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd878083f40 con 0x7fd8900b2370 2023-10-26T05:49:18.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88ffff640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8700664a0 0x7fd870068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:18.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.937+0000 7fd88ffff640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8700664a0 0x7fd870068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd8880af000 tx=0x7fd88801b050 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:18.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:18.941+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd878051410 con 0x7fd8900b2370 2023-10-26T05:49:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.101+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd8900721e0 con 0x7fd8900b2370 2023-10-26T05:49:19.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd88cff9640 1 -- 172.21.15.71:0/3991295299 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) v1 ==== 81+0+34235 (secure 0 0 0) 0x7fd878054a30 con 0x7fd8900b2370 2023-10-26T05:49:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:19.110 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 50, 2023-10-26T05:49:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:10.401712+0000", 2023-10-26T05:49:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:19.115 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:19.115 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:19.115 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:19.115 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:19.116 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:19.116 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:19.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.116 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:19.117 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:19.117 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:19.117 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:19.117 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:19.117 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:19.117 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:19.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:19.118 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:19.118 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:19.118 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:19.118 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:19.119 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:19.119 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:19.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:19.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:19.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:19.120 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:19.120 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:19.120 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:19.120 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:19.120 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:19.121 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:19.121 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:19.121 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:19.121 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:19.121 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:19.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.122 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "16", 2023-10-26T05:49:19.122 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:19.122 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:19.123 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:19.123 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:19.123 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:19.123 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:19.123 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:19.124 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:19.124 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:19.124 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:19.124 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:19.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:19.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:19.125 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:19.126 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:19.126 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:19.126 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:19.127 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:19.127 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:19.127 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:19.128 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:19.129 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:19.129 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:19.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.129 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:19.129 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:19.130 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:19.130 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:19.130 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:19.130 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:19.130 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:19.131 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:19.131 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:19.131 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:19.131 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:19.131 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:19.132 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:19.132 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:19.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.132 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:19.132 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:19.133 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:19.134 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:19.134 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:19.134 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:19.134 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:19.135 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:19.135 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.135 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:19.136 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:19.136 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:19.136 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:19.136 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:19.137 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:19.137 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:19.137 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:19.138 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:19.138 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:19.138 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:19.139 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:19.139 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:19.140 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:19.140 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:19.140 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:19.141 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:19.141 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:19.141 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:19.141 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:19.141 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:19.142 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:19.142 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:19.142 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:19.142 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:19.142 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:19.143 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:19.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "47", 2023-10-26T05:49:19.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:19.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:19.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:19.144 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:19.144 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:19.144 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:19.144 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:19.144 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:19.145 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:19.145 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:19.145 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:19.145 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:19.145 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:19.146 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:19.146 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:19.146 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:19.146 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:19.146 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:19.147 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:19.147 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:19.147 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:19.147 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:19.147 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:19.147 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:19.148 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:19.148 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:19.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.148 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:19.148 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:19.149 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:19.149 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:19.149 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:19.149 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:19.150 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:19.150 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:19.150 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:19.150 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:19.150 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:19.151 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:19.151 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:19.151 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:19.151 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:19.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.151 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:19.152 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:19.152 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:19.152 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.152 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:19.153 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:19.153 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:19.153 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:19.153 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:19.153 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:19.154 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:19.154 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:19.154 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:19.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.155 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:19.155 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:19.155 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:19.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:19.155 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:19.156 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:19.156 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:19.156 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:19.156 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:19.156 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:19.157 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:19.157 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:19.157 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:19.157 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:19.157 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:19.157 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:19.158 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:19.158 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:19.158 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:19.158 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:19.158 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:19.159 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:19.159 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:19.159 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:19.159 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:19.159 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:19.160 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:19.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.160 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "48", 2023-10-26T05:49:19.160 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:19.160 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:19.161 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:19.161 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:19.161 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:19.161 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:19.161 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:19.161 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:19.162 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:19.162 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:19.162 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:19.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:19.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:19.163 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:19.163 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:19.163 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:19.163 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:19.163 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:19.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:19.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:19.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:19.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:19.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:19.164 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:19.165 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:19.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:19.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.165 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:19.165 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:19.166 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:19.166 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:19.166 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:19.166 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:19.166 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:19.167 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:19.167 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:19.167 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:19.167 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:19.167 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:19.167 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:19.168 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:19.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.168 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:19.168 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:19.168 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:19.169 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.169 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:19.169 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:19.169 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:19.170 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:19.170 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:19.170 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:19.170 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:19.170 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:19.170 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:19.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.171 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:19.171 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:19.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.172 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:19.172 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.172 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.172 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.172 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.173 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.173 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.173 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.173 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.173 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:19.173 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.174 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.174 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.174 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.174 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.175 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:19.175 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.176 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:19.176 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.176 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.176 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:19.177 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.178 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.178 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:19.178 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.179 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.179 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.180 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:19.180 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.181 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:19.181 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.181 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.182 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.182 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:19.183 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.183 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:19.184 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:19.184 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.184 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.184 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:19.184 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:19.185 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:19.185 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:19.185 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.185 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.185 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.186 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:19.186 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.187 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.187 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.187 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.187 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.187 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.188 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.188 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.188 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:19.188 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.188 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.189 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.189 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.189 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.189 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:19.189 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.190 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:19.190 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.191 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.191 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.191 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.191 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.192 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:19.192 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.193 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:19.193 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.193 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.194 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.194 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:19.195 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:19.196 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.196 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.197 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.197 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.197 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:19.198 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.198 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.198 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:19.199 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:19.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.199 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.199 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:19.199 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:19.200 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:19.200 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:19.200 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.200 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.200 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.201 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.201 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:19.201 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.202 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.202 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.202 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.202 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.202 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.203 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.203 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.203 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:19.203 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.203 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.203 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.204 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.204 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:19.204 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.205 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:19.205 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.206 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.206 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.206 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.207 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.207 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:19.207 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.208 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:19.208 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.208 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.209 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.209 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.209 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.209 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:19.210 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.210 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:19.211 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.211 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.211 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.211 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.211 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.212 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:19.212 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.213 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:19.213 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:19.213 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.214 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:19.214 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:19.214 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:19.214 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:19.215 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.215 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.215 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.215 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.216 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:19.216 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.216 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.216 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.217 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.217 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.217 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.217 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.217 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.217 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:19.218 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.218 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.218 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.218 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.219 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:19.219 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.220 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:19.220 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.220 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.220 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.221 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.221 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.221 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:19.222 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:19.222 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.223 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.223 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:19.224 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:19.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.225 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.225 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.226 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.226 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.227 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:19.227 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.228 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:19.228 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:19.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.229 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:19.229 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:19.229 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:19.229 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:19.229 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.229 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.230 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.230 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.230 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:19.231 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.231 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.231 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.231 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.231 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.232 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.232 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.232 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.232 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:19.232 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.232 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.233 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.233 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.233 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:19.234 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.234 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:19.235 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.235 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.235 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.235 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:19.236 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:19.237 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.238 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.238 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.239 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:19.239 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:19.240 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.240 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:19.242 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.242 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:19.243 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:19.243 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.243 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.243 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:19.244 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:19.244 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:19.244 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:19.244 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.244 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.244 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.245 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:19.245 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.246 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.246 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.246 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.246 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.247 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.247 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.247 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:19.247 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.247 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.247 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:19.248 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:19.249 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.250 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.250 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.251 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:19.251 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:19.252 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.252 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.253 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:19.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:19.255 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.255 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.255 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:19.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.257 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:19.257 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:19.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:19.258 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.259 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.260 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.261 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.261 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.261 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.261 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.261 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.262 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:19.262 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.262 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.262 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.262 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.263 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.263 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:19.263 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.264 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.264 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:19.264 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.264 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.265 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.265 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:19.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:19.267 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.267 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.268 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:19.268 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.269 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:19.269 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.270 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.270 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:19.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:19.272 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:19.272 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.272 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.273 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:19.273 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:19.273 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:19.273 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:19.274 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.274 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.274 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.275 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:19.275 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:19.275 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:19.275 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:19.275 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:19.276 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:19.276 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:19.276 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:19.276 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:19.276 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:19.277 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:19.277 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:19.277 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:19.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.278 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:19.278 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.279 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:19.279 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.280 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:19.280 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.280 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.280 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:19.281 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.281 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:19.282 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.282 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.282 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.282 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:19.283 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.283 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:19.283 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.285 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.285 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:19.285 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:19.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:19.286 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:19.286 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:19.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:19.287 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:19.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.288 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:19.288 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:19.288 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:19.288 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:19.288 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:19.289 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:19.289 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:19.289 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.289 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:19.290 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:19.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.290 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:19.290 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.290 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.291 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.291 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.291 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.291 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:19.291 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.292 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:19.292 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.292 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.292 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.293 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.293 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.293 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:19.293 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.294 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.295 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.295 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:19.295 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.296 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:19.296 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.296 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.296 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.296 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.297 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.297 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:19.297 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.298 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:19.298 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.298 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.298 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.298 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.298 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.299 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:19.299 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.299 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:19.300 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.300 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.300 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.300 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.300 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.301 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:19.301 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.301 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:19.302 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.302 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.302 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.302 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.302 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.302 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:19.303 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.303 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:19.303 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:19.304 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:19.304 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:19.304 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:19.304 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:19.304 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:19.305 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:19.305 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.305 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:19.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:19.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.306 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:19.308 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:19.309 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:19.310 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:19.311 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:19.311 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:19.311 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:19.311 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:19.311 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:19.312 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:19.312 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:19.312 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:19.312 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:19.313 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:19.313 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:19.313 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:19.313 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:19.313 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:19.314 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:19.314 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:19.314 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:19.314 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:19.314 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:19.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8700664a0 msgr2=0x7fd870068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:19.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8700664a0 0x7fd870068960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd8880af000 tx=0x7fd88801b050 comp rx=0 tx=0).stop 2023-10-26T05:49:19.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 msgr2=0x7fd8900c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:19.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 0x7fd8900c37c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c450 tx=0x7fd87800c910 comp rx=0 tx=0).stop 2023-10-26T05:49:19.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 shutdown_connections 2023-10-26T05:49:19.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd890140cc0 0x7fd8900c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:19.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd8700664a0 0x7fd870068960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:19.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd8900b2370 0x7fd8900c37c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:19.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 --2- 172.21.15.71:0/3991295299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd890070fe0 0x7fd8901625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:19.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 >> 172.21.15.71:0/3991295299 conn(0x7fd890069770 msgr2=0x7fd89015a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:19.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 shutdown_connections 2023-10-26T05:49:19.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:19.105+0000 7fd896807640 1 -- 172.21.15.71:0/3991295299 wait complete. 2023-10-26T05:49:19.319 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:49:20.163 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:49:20.157+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2023-10-26T05:49:24.320 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:24.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- 172.21.15.71:0/1169119683 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 msgr2=0x7fbaf80caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:24.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 --2- 172.21.15.71:0/1169119683 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 0x7fbaf80caad0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009bf0 tx=0x7fbaf401cbc0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- 172.21.15.71:0/1169119683 shutdown_connections 2023-10-26T05:49:24.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 --2- 172.21.15.71:0/1169119683 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 0x7fbaf80caad0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- 172.21.15.71:0/1169119683 >> 172.21.15.71:0/1169119683 conn(0x7fbaf8069770 msgr2=0x7fbaf8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:24.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- 172.21.15.71:0/1169119683 shutdown_connections 2023-10-26T05:49:24.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- 172.21.15.71:0/1169119683 wait complete. 2023-10-26T05:49:24.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 Processor -- start 2023-10-26T05:49:24.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- start start 2023-10-26T05:49:24.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 0x7fbaf80ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:24.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 0x7fbaf80ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:24.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaf813c2f0 0x7fbaf8134420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:24.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80b03e0 con 0x7fbaf80cb010 2023-10-26T05:49:24.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf81350a0 con 0x7fbaf813c2f0 2023-10-26T05:49:24.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbb003ec640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf80ac290 con 0x7fbaf80b2a60 2023-10-26T05:49:24.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 0x7fbaf80ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 0x7fbaf80ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53814/0 (socket says 172.21.15.71:53814) 2023-10-26T05:49:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 -- 172.21.15.71:0/4155168105 learned_addr learned my addr 172.21.15.71:0/4155168105 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafe188640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 0x7fbaf80ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafe989640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaf813c2f0 0x7fbaf8134420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 -- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 msgr2=0x7fbaf80ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:24.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 0x7fbaf80ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 -- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaf813c2f0 msgr2=0x7fbaf8134420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaf813c2f0 0x7fbaf8134420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 -- 172.21.15.71:0/4155168105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4009820 con 0x7fbaf80cb010 2023-10-26T05:49:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafe989640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaf813c2f0 0x7fbaf8134420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:24.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.453+0000 7fbafd987640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 0x7fbaf80ab2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40197d0 tx=0x7fbaf40a5910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4005240 con 0x7fbaf80cb010 2023-10-26T05:49:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbaf40053a0 con 0x7fbaf80cb010 2023-10-26T05:49:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf80ab9f0 con 0x7fbaf80cb010 2023-10-26T05:49:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4005790 con 0x7fbaf80cb010 2023-10-26T05:49:24.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf80abfb0 con 0x7fbaf80cb010 2023-10-26T05:49:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8066b10 con 0x7fbaf80cb010 2023-10-26T05:49:24.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbaf40b7440 con 0x7fbaf80cb010 2023-10-26T05:49:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbadc0664a0 0x7fbadc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbaf411e750 con 0x7fbaf80cb010 2023-10-26T05:49:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=54}) v3 -- 0x7fbadc06e090 con 0x7fbaf80cb010 2023-10-26T05:49:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbafe188640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbadc0664a0 0x7fbadc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.457+0000 7fbafe188640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbadc0664a0 0x7fbadc068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00015e0 tx=0x7fbaf0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.461+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbaf40b2070 con 0x7fbaf80cb010 2023-10-26T05:49:24.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.617+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbaf80b0520 con 0x7fbaf80cb010 2023-10-26T05:49:24.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbaeeffd640 1 -- 172.21.15.71:0/4155168105 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+34378 (secure 0 0 0) 0x7fbaf80b0520 con 0x7fbaf80cb010 2023-10-26T05:49:24.625 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:24.625 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:24.625 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 53, 2023-10-26T05:49:24.625 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:24.625 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:24.626 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:23.068585+0000", 2023-10-26T05:49:24.626 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:24.626 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:24.626 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:24.626 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:24.627 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:24.627 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:24.627 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:24.627 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:24.627 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:24.628 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:24.628 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:24.628 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:24.628 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:24.629 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:24.630 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:24.630 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:24.630 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:24.631 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:24.631 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:24.631 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:24.631 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:24.632 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:24.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.632 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:24.632 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:24.632 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:24.633 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:24.633 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:24.633 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:24.633 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:24.633 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:24.634 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:24.634 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:24.634 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:24.634 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:24.634 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:24.635 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:24.635 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:24.635 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:24.635 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:24.635 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:24.636 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:24.636 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:24.636 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:24.636 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:24.636 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:24.637 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:24.637 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:24.637 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:24.637 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:24.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:24.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:24.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:24.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:24.638 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:24.639 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:24.639 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:24.639 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:24.639 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:24.639 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:24.639 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:24.640 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:24.640 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:24.640 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:24.640 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:24.640 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:24.641 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:24.641 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:24.641 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:24.641 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:24.642 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:24.642 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:24.642 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:24.642 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:24.642 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:24.643 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:24.643 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:24.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.643 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:24.643 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:24.644 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:24.644 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:24.644 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:24.644 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:24.644 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:24.645 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:24.645 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:24.645 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:24.645 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:24.645 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:24.646 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:24.646 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:24.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.646 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:24.646 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:24.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.647 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:24.647 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:24.647 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:24.647 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:24.648 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:24.648 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:24.648 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:24.648 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:24.648 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:24.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.649 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:24.649 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:24.650 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:24.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:24.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:24.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:24.651 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:24.651 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:24.651 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:24.651 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:24.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:24.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:24.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:24.652 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:24.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:24.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:24.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:24.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:24.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:24.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:24.654 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:24.654 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:24.654 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:24.654 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:24.655 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:24.655 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:24.655 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:24.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.655 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:24.656 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:24.656 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:24.656 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:24.656 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:24.656 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:24.657 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:24.657 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:24.657 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:24.657 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:24.657 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:24.657 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:24.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:24.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:24.658 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:24.658 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:24.658 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:24.659 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:24.659 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:24.659 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:24.659 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:24.659 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:24.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:24.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:24.660 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:24.660 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:24.660 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:24.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.661 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:24.661 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:24.661 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:24.661 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:24.661 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:24.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:24.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:24.662 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:24.662 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:24.662 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:24.663 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:24.663 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:24.663 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:24.663 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:24.663 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:24.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.664 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:24.664 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:24.664 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:24.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.665 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:24.665 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:24.665 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:24.665 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:24.666 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:24.666 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:24.666 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:24.666 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:24.667 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:24.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.667 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:24.668 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:24.668 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:24.668 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:24.668 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:24.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:24.669 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:24.669 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:24.669 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:24.669 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:24.669 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:24.670 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:24.670 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:24.670 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:24.670 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:24.670 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:24.671 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:24.671 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:24.671 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:24.671 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:24.671 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:24.672 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:24.672 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:24.672 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:24.672 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:24.672 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:24.672 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:24.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.673 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:24.673 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:24.673 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:24.673 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:24.674 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:24.674 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:24.674 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:24.674 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:24.674 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:24.675 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:24.675 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:24.675 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:24.675 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:24.675 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:24.676 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:24.676 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:24.676 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:24.676 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:24.676 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:24.677 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:24.677 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:24.677 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:24.677 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:24.677 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:24.677 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:24.678 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:24.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:24.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.678 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:24.679 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:24.679 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:24.679 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:24.679 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:24.679 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:24.680 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:24.680 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:24.680 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:24.680 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:24.680 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:24.681 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:24.681 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:24.681 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:24.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.681 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:24.682 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:24.682 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:24.682 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.682 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:24.683 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:24.683 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:24.683 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:24.683 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:24.683 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:24.684 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:24.684 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:24.684 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:24.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.684 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.685 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:24.685 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:24.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.685 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:24.685 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.686 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.686 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.686 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.686 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.686 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.686 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.687 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.687 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:24.687 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.687 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.687 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:24.688 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.689 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.689 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:24.689 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.690 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.690 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.690 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:24.691 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:24.692 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.692 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.692 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.693 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:24.694 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:24.695 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.695 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.695 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.695 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.696 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.696 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.696 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:24.696 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:24.697 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:24.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.698 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.698 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:24.698 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:24.699 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:24.699 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:24.699 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.699 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.699 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:24.700 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.700 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:24.701 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.701 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.701 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.701 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.701 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.702 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.702 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.702 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.702 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:24.702 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.703 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.703 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:24.704 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.704 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:24.705 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.705 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.705 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.705 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.706 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.706 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:24.707 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.707 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.707 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:24.708 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.708 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.708 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.709 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:24.709 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.710 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:24.710 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.711 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.713 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.713 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:24.714 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:24.715 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:24.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.716 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:24.716 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:24.716 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:24.716 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.718 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:24.718 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.718 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.718 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.718 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.719 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.719 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.719 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.719 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.719 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:24.720 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.720 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.720 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.720 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.721 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:24.721 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.722 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.722 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:24.722 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.722 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.723 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.723 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:24.724 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.724 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.724 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:24.725 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.725 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.725 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.725 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.726 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.726 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.726 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:24.726 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.727 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:24.727 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.728 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.728 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.728 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:24.729 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.730 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:24.730 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:24.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.730 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.731 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:24.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:24.731 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:24.731 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:24.732 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.732 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.732 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:24.732 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.732 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.733 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:24.733 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.733 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.734 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.734 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.734 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.734 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.734 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:24.736 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:24.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.738 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.738 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.739 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.739 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:24.739 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.739 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.740 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:24.740 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.741 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.741 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.741 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:24.742 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.742 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.742 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:24.743 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.743 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.743 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:24.744 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:24.745 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:24.746 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.746 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.746 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:24.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:24.747 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:24.747 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:24.747 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.747 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:49:24.747 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.748 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.748 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.748 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:24.748 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.749 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.749 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.749 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.749 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.749 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.750 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.750 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.750 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:24.750 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.750 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.751 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.751 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.751 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.751 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:24.752 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.752 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:24.752 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.753 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.753 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.753 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.754 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.754 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:24.754 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.755 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:24.755 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.755 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.756 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.756 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.757 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:24.757 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.757 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.757 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.758 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:24.758 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.758 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.758 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.759 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.759 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.759 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:24.759 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.760 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.760 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:24.760 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:24.761 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.761 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.761 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:24.761 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:24.762 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:24.762 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:24.762 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.762 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.762 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.763 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.763 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:24.763 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.763 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.764 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.764 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.764 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.764 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.764 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.765 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.765 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:24.765 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.765 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.765 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.766 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.766 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.766 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:24.766 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.767 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:24.767 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.768 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.768 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.768 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.769 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:24.769 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.770 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:24.770 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.770 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.770 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.771 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.771 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.771 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:24.772 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.772 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.772 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:24.773 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.773 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.773 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.773 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.773 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.774 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:24.774 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:24.775 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:24.775 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.776 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.776 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:24.776 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:24.776 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.778 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:24.778 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.778 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.778 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.779 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.779 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.779 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.779 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.779 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.780 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:24.780 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.780 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.780 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.780 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.781 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:24.781 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.782 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:24.782 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.783 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.783 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.783 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.784 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:24.784 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.785 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:24.785 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.785 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.786 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.786 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.786 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.786 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:24.787 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.787 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:24.787 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.788 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.788 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.788 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.789 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:24.789 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.790 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:24.790 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:24.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.790 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.791 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:24.791 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:24.791 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:24.791 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:24.792 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.792 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.792 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:24.792 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.792 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.793 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:24.793 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:24.793 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:24.793 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:24.794 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:24.794 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:24.794 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:24.794 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:24.794 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:24.795 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:24.795 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:24.795 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:24.795 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:24.795 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.796 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:24.796 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.797 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:24.797 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.797 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.797 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.798 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:24.798 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.798 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.799 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:24.799 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.800 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:24.800 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.800 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.800 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.800 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:24.801 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.801 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:24.801 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.802 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.802 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:24.802 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.803 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:24.803 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:24.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:24.804 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:24.804 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:24.805 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:24.805 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:24.805 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.805 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.806 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:24.806 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:24.806 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:24.806 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:24.807 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:24.807 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:24.807 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:24.807 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:24.807 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.808 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:24.808 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:24.808 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.808 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:24.809 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.809 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.809 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.809 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.809 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.809 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:24.810 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.810 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:24.810 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.811 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.811 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.811 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.811 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.811 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:24.812 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.812 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.812 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:24.812 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.812 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.813 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.813 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.813 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.813 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:24.813 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.814 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.814 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.814 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:24.814 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.814 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.815 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.815 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.815 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.815 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:24.815 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.816 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:24.816 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.816 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.816 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.817 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.817 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.817 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:24.817 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.818 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:24.818 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.818 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.818 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.819 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.819 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.819 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:24.819 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.820 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:24.820 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.820 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.820 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.820 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.821 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.821 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:24.821 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.822 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:24.822 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:24.822 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:24.822 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:24.822 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:24.823 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:24.823 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:24.823 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:24.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.823 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:24.823 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:24.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:24.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.824 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:24.824 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:24.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.825 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:24.825 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:24.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.825 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.826 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:24.826 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:24.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:24.826 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:24.826 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:24.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.827 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:24.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.827 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:24.827 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:24.828 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:24.828 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:24.828 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:24.828 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:24.828 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:24.829 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:24.829 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:24.829 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:24.829 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:24.829 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:24.830 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:24.831 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:24.831 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:24.831 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:24.831 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:24.831 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:24.832 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:24.832 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:24.832 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:24.832 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:24.832 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:24.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbadc0664a0 msgr2=0x7fbadc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:24.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbadc0664a0 0x7fbadc068960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00015e0 tx=0x7fbaf0009040 comp rx=0 tx=0).stop 2023-10-26T05:49:24.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 msgr2=0x7fbaf80ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:24.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 0x7fbaf80ab2f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40197d0 tx=0x7fbaf40a5910 comp rx=0 tx=0).stop 2023-10-26T05:49:24.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 shutdown_connections 2023-10-26T05:49:24.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaf813c2f0 0x7fbaf8134420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbadc0664a0 0x7fbadc068960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf80cb010 0x7fbaf80ab2f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 --2- 172.21.15.71:0/4155168105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbaf80b2a60 0x7fbaf80ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:24.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 >> 172.21.15.71:0/4155168105 conn(0x7fbaf8069770 msgr2=0x7fbaf806a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:24.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 shutdown_connections 2023-10-26T05:49:24.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:24.621+0000 7fbb003ec640 1 -- 172.21.15.71:0/4155168105 wait complete. 2023-10-26T05:49:24.837 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T05:49:24.838 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:24.838 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:24.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:24.844 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-26T05:49:24.992 INFO:teuthology.orchestra.run.smithi071.stderr:dd: error writing 'large_file_b': No space left on device 2023-10-26T05:49:24.993 INFO:teuthology.orchestra.run.smithi071.stderr:dd: closing output file 'large_file_b': No space left on device 2023-10-26T05:49:24.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:49:24.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:25.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- 172.21.15.71:0/1617670613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 msgr2=0x7f9f0415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 --2- 172.21.15.71:0/1617670613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0415bab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009c80 tx=0x7f9efc01cc30 comp rx=0 tx=0).stop 2023-10-26T05:49:25.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- 172.21.15.71:0/1617670613 shutdown_connections 2023-10-26T05:49:25.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 --2- 172.21.15.71:0/1617670613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0415bab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 --2- 172.21.15.71:0/1617670613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f040d0470 0x7f9f0406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- 172.21.15.71:0/1617670613 >> 172.21.15.71:0/1617670613 conn(0x7f9f040cdd80 msgr2=0x7f9f040ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:25.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- 172.21.15.71:0/1617670613 shutdown_connections 2023-10-26T05:49:25.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- 172.21.15.71:0/1617670613 wait complete. 2023-10-26T05:49:25.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 Processor -- start 2023-10-26T05:49:25.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- start start 2023-10-26T05:49:25.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f040d0470 0x7f9f04157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 0x7f9f041576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f04157ce0 con 0x7f9f040d0850 2023-10-26T05:49:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040c4840 con 0x7f9f040cd880 2023-10-26T05:49:25.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f0a611640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f040c4fd0 con 0x7f9f040d0470 2023-10-26T05:49:25.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f08bae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 0x7f9f041576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f08bae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 0x7f9f041576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53820/0 (socket says 172.21.15.71:53820) 2023-10-26T05:49:25.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f08bae640 1 -- 172.21.15.71:0/717811224 learned_addr learned my addr 172.21.15.71:0/717811224 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:25.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f03fff640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.121+0000 7f9f037fe640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f040d0470 0x7f9f04157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f08bae640 1 -- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f040d0470 msgr2=0x7f9f04157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f08bae640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f040d0470 0x7f9f04157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f08bae640 1 -- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 msgr2=0x7f9f0414a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f08bae640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0414a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f08bae640 1 -- 172.21.15.71:0/717811224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0098e0 con 0x7f9f040d0850 2023-10-26T05:49:25.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f03fff640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0414a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:25.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f08bae640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 0x7f9f041576b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009bd0 tx=0x7f9efc003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:25.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc004c50 con 0x7f9f040d0850 2023-10-26T05:49:25.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9efc0ad5b0 con 0x7f9f040d0850 2023-10-26T05:49:25.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f040c52b0 con 0x7f9f040d0850 2023-10-26T05:49:25.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc004ea0 con 0x7f9f040d0850 2023-10-26T05:49:25.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f040c5870 con 0x7f9f040d0850 2023-10-26T05:49:25.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f04066b10 con 0x7f9f040d0850 2023-10-26T05:49:25.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9efc0b4040 con 0x7f9f040d0850 2023-10-26T05:49:25.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9ef0066470 0x7f9ef0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9efc11e310 con 0x7f9f040d0850 2023-10-26T05:49:25.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=54}) v3 -- 0x7f9ef006e060 con 0x7f9f040d0850 2023-10-26T05:49:25.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f03fff640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9ef0066470 0x7f9ef0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.125+0000 7f9f03fff640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9ef0066470 0x7f9ef0068930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009c30 tx=0x7f9ef80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:25.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.129+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9efc0bc060 con 0x7f9f040d0850 2023-10-26T05:49:25.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.273+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9f040cd2d0 con 0x7f9f040d0850 2023-10-26T05:49:25.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.273+0000 7f9f00ff9640 1 -- 172.21.15.71:0/717811224 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+34378 (secure 0 0 0) 0x7f9efc0b2030 con 0x7f9f040d0850 2023-10-26T05:49:25.279 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:25.279 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:25.280 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 53, 2023-10-26T05:49:25.280 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:25.280 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:25.280 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:23.068585+0000", 2023-10-26T05:49:25.280 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:25.281 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:25.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:25.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:25.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:25.281 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:25.282 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:25.282 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:25.282 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:25.282 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:25.283 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:25.283 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:25.283 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:25.283 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:25.283 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:25.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:25.284 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:25.284 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:25.284 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:25.284 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:25.285 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:25.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:25.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:25.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:25.286 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:25.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:25.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:25.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:25.286 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:25.287 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:25.287 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:25.287 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:25.287 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:25.287 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:25.288 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:25.288 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:25.288 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:25.288 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:25.288 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:25.288 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:25.289 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:25.289 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:25.289 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:25.289 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:25.289 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:25.290 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:25.290 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:25.290 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:25.291 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:25.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.291 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:25.291 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:25.291 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:25.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:25.292 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:25.292 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:25.292 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:25.292 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:25.293 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:25.293 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:25.293 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:25.293 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:25.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:25.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:25.294 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:25.294 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:25.294 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:25.294 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:25.296 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:25.296 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:25.296 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:25.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.296 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:25.297 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:25.297 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:25.297 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:25.297 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:25.297 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:25.298 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:25.298 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:25.298 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:25.298 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:25.298 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:25.299 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:25.299 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:25.299 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:25.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.299 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:25.300 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:25.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.300 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:25.300 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:25.300 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:25.301 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:25.301 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:25.301 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:25.301 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:25.301 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:25.302 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:25.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.302 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:25.303 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:25.303 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:25.303 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:25.303 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:25.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:25.303 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:25.304 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:25.304 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:25.304 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:25.304 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:25.304 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:25.305 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:25.305 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:25.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:25.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:25.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:25.306 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:25.306 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:25.306 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:25.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:25.306 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:25.307 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:25.307 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:25.307 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:25.307 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:25.307 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:25.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.308 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:25.308 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:25.308 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:25.308 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:25.308 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:25.309 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:25.309 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:25.309 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:25.309 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:25.309 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:25.310 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:25.310 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:25.310 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:25.310 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:25.310 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:25.311 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:25.311 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:25.311 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:25.311 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:25.311 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:25.312 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:25.312 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:25.312 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:25.312 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:25.312 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:25.313 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:25.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:25.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.313 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:25.313 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:25.314 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:25.314 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:25.314 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:25.314 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:25.314 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:25.315 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:25.315 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:25.315 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:25.315 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:25.315 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:25.316 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:25.316 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:25.316 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:25.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.316 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:25.317 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:25.317 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:25.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.317 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:25.318 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:25.318 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:25.318 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:25.318 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:25.318 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:25.319 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:25.319 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:25.319 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:25.319 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.320 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:25.320 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:25.321 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:25.321 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:25.321 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:25.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:25.321 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:25.322 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:25.322 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:25.322 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:25.322 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:25.322 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:25.323 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:25.323 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:25.323 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:25.323 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:25.325 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:25.325 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:25.325 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:25.326 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:25.326 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:25.326 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:25.326 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:25.326 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:25.327 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:25.327 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:25.327 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:25.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.327 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:25.328 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:25.328 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:25.328 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:25.328 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:25.328 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:25.329 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:25.329 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:25.329 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:25.329 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:25.329 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:25.330 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:25.330 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:25.330 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:25.330 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:25.330 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:25.331 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:25.331 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:25.331 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:25.331 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:25.331 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:25.332 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:25.332 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:25.332 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:25.332 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:25.332 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:25.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:25.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.333 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:25.333 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:25.333 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:25.334 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:25.334 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:25.334 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:25.334 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:25.334 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:25.335 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:25.335 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:25.335 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:25.335 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:25.335 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:25.336 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:25.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.336 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:25.336 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:25.336 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:25.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.337 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:25.337 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:25.337 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:25.337 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:25.338 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:25.338 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:25.338 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:25.338 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:25.338 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:25.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.339 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:25.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:25.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.340 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:25.340 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.340 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.340 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.340 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.341 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.341 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.341 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.341 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.341 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:25.341 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.342 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.342 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.342 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.343 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:25.343 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.344 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:25.344 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.344 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.345 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.345 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.345 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:25.345 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.346 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:25.346 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.347 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.347 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.347 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.347 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.348 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.348 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:25.348 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:25.349 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.349 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.350 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.350 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:25.353 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.355 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.356 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.359 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.360 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.362 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:25.363 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:25.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.366 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:25.366 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:25.367 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:25.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.368 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:25.368 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:25.368 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:25.368 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:25.369 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.369 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.369 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.369 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.370 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:25.370 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.370 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.370 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.371 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.371 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.371 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.371 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:25.372 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.372 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.372 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.372 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.373 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:25.373 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.374 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:25.374 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.374 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.375 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.375 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.376 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:25.376 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.377 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:25.377 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.377 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.377 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.378 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.378 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:25.379 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:25.380 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.380 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.380 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:25.381 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:25.382 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:25.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.383 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.383 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:25.383 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:25.384 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:25.384 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:25.384 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.384 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.384 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:25.385 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.385 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:25.386 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.386 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.386 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.386 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.386 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.387 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.387 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.387 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.387 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:25.387 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.388 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.388 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:25.389 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.389 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:25.390 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.390 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.390 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.391 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:25.392 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.392 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:25.393 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.393 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.393 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.393 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:25.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.395 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:25.395 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.396 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.396 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.397 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:25.397 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.398 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:25.398 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:25.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.398 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.399 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:25.399 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:25.399 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:25.399 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:25.399 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.400 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:49:25.400 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.400 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.400 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.401 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:25.401 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.401 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.401 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.402 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.402 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.402 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.402 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.402 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.403 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:25.403 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.403 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.403 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.403 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.404 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:25.404 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:25.405 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.405 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.405 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.406 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.406 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:25.407 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:25.408 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.408 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.408 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.408 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:25.409 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.410 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.410 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:25.410 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.411 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.411 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.411 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:25.412 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.413 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:25.413 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:25.413 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.413 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.414 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:25.414 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:25.414 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:25.414 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:25.415 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.415 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.415 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.415 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.416 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:25.416 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.416 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.416 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.416 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.417 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.417 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.417 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.417 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.417 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:25.418 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.418 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.418 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.418 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.419 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:25.419 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.420 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.420 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:25.420 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.420 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.421 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.421 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.421 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:25.422 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.422 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:25.423 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.423 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.423 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:25.424 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:25.425 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.426 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.427 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:25.427 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:25.428 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:25.428 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.428 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.429 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:25.429 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:25.429 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:25.429 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:25.430 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.430 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.430 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.430 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.431 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:25.431 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.431 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.431 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.432 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.432 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.432 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.433 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:25.433 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.433 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.433 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.433 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.434 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:25.434 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.435 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.435 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:25.435 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.436 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.437 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:25.437 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.437 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:25.438 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.438 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.438 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.439 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.439 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:25.439 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.440 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:25.440 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.441 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.441 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.441 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.442 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.442 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:25.442 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.443 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:25.443 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:25.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.443 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.444 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:25.444 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:25.444 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:25.444 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:25.445 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.445 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.445 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:25.445 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.445 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.446 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:25.446 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:25.446 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:25.447 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:25.447 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:25.447 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:25.447 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:25.447 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:25.448 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:25.448 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:25.448 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:25.448 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:25.448 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:25.449 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.449 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.449 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:25.449 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.451 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:25.451 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.452 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:25.452 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.453 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:25.453 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.453 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.454 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:25.454 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.454 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.455 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:25.455 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.455 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.455 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:25.456 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.456 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.456 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:25.457 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:25.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:25.457 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:25.457 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:25.458 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:25.458 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:25.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.459 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.459 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:25.459 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:25.460 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:25.460 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:25.460 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:25.460 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:25.460 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:25.460 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:25.461 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.461 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:25.461 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:25.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.462 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:25.462 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.462 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.462 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.462 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.463 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.463 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:25.463 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.464 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:25.464 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.464 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.464 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.464 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.464 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.465 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:25.465 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.465 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.466 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.467 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:25.467 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.467 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:25.467 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.468 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.468 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.468 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.468 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.468 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:25.469 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.469 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:25.469 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.470 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.470 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.470 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.470 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.470 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:25.471 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.471 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:25.471 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.471 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.472 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.472 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.472 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.472 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:25.472 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.473 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:25.473 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.473 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.474 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.474 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.474 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.474 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:25.474 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.475 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:25.475 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:25.475 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:25.476 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:25.476 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:25.476 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:25.476 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:25.476 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:25.476 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.477 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:25.477 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:25.477 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:25.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.477 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:25.478 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:25.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.478 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:25.478 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:25.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.479 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.479 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:25.479 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:25.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:25.480 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:25.480 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:25.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.480 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:25.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.481 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:25.481 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:25.481 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:25.481 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:25.481 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:25.482 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:25.482 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:25.482 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:25.482 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:25.482 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:25.483 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:25.483 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:25.483 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:25.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:25.484 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:25.484 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:25.484 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:25.484 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:25.484 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:25.484 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:25.485 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:25.485 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:25.485 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:25.485 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:25.485 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:25.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:25.486 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:25.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9ef0066470 msgr2=0x7f9ef0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9ef0066470 0x7f9ef0068930 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009c30 tx=0x7f9ef80091e0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 msgr2=0x7f9f041576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 0x7f9f041576b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009bd0 tx=0x7f9efc003f40 comp rx=0 tx=0).stop 2023-10-26T05:49:25.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 shutdown_connections 2023-10-26T05:49:25.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9ef0066470 0x7f9ef0068930 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f040d0850 0x7f9f041576b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f040d0470 0x7f9f04157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 --2- 172.21.15.71:0/717811224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f040cd880 0x7f9f0414a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 >> 172.21.15.71:0/717811224 conn(0x7f9f040cdd80 msgr2=0x7f9f040b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:25.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 shutdown_connections 2023-10-26T05:49:25.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.277+0000 7f9f0a611640 1 -- 172.21.15.71:0/717811224 wait complete. 2023-10-26T05:49:25.491 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:25.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 -- 172.21.15.71:0/3518810358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 msgr2=0x7feb5400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 --2- 172.21.15.71:0/3518810358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb5400b990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7feb64006fd0 tx=0x7feb6401f9a0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 -- 172.21.15.71:0/3518810358 shutdown_connections 2023-10-26T05:49:25.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 --2- 172.21.15.71:0/3518810358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb5400b990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 -- 172.21.15.71:0/3518810358 >> 172.21.15.71:0/3518810358 conn(0x7feb6c0cdd80 msgr2=0x7feb6c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:25.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 -- 172.21.15.71:0/3518810358 shutdown_connections 2023-10-26T05:49:25.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 -- 172.21.15.71:0/3518810358 wait complete. 2023-10-26T05:49:25.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 Processor -- start 2023-10-26T05:49:25.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.617+0000 7feb72c20640 1 -- start start 2023-10-26T05:49:25.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb6c1f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6c0b47f0 0x7feb6c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb6c15d130 0x7feb6c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c06a650 con 0x7feb540051e0 2023-10-26T05:49:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1f2cd0 con 0x7feb6c0b47f0 2023-10-26T05:49:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c0b51b0 con 0x7feb6c15d130 2023-10-26T05:49:25.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb6c1f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb6c1f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53838/0 (socket says 172.21.15.71:53838) 2023-10-26T05:49:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 -- 172.21.15.71:0/4269523207 learned_addr learned my addr 172.21.15.71:0/4269523207 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb6bfff640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6c0b47f0 0x7feb6c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb711bd640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb6c15d130 0x7feb6c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 -- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb6c15d130 msgr2=0x7feb6c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb6c15d130 0x7feb6c15d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 -- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6c0b47f0 msgr2=0x7feb6c0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6c0b47f0 0x7feb6c0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 -- 172.21.15.71:0/4269523207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c1f2050 con 0x7feb540051e0 2023-10-26T05:49:25.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb709bc640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb6c1f17b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7feb60004d20 tx=0x7feb60001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:25.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60016030 con 0x7feb540051e0 2023-10-26T05:49:25.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb60011750 con 0x7feb540051e0 2023-10-26T05:49:25.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c1f2330 con 0x7feb540051e0 2023-10-26T05:49:25.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb60011b20 con 0x7feb540051e0 2023-10-26T05:49:25.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c1f28f0 con 0x7feb540051e0 2023-10-26T05:49:25.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c066b10 con 0x7feb540051e0 2023-10-26T05:49:25.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb60019070 con 0x7feb540051e0 2023-10-26T05:49:25.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb697fa640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb58066470 0x7feb58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:25.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.621+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feb60017070 con 0x7feb540051e0 2023-10-26T05:49:25.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.625+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=54}) v3 -- 0x7feb5806e060 con 0x7feb540051e0 2023-10-26T05:49:25.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.625+0000 7feb6bfff640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb58066470 0x7feb58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:25.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.625+0000 7feb6bfff640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb58066470 0x7feb58068930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb6c066dc0 tx=0x7feb5c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:25.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.629+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb60051860 con 0x7feb540051e0 2023-10-26T05:49:25.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.809+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb6c0721e0 con 0x7feb540051e0 2023-10-26T05:49:25.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.809+0000 7feb697fa640 1 -- 172.21.15.71:0/4269523207 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7feb60054e80 con 0x7feb540051e0 2023-10-26T05:49:25.814 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:25.814 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:25.815 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:25.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb58066470 msgr2=0x7feb58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb58066470 0x7feb58068930 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feb6c066dc0 tx=0x7feb5c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 msgr2=0x7feb6c1f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:25.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb6c1f17b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7feb60004d20 tx=0x7feb60001640 comp rx=0 tx=0).stop 2023-10-26T05:49:25.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 shutdown_connections 2023-10-26T05:49:25.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb6c15d130 0x7feb6c15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6c0b47f0 0x7feb6c0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb58066470 0x7feb58068930 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 --2- 172.21.15.71:0/4269523207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb540051e0 0x7feb6c1f17b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:25.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 >> 172.21.15.71:0/4269523207 conn(0x7feb6c0cdd80 msgr2=0x7feb6c0b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:25.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 shutdown_connections 2023-10-26T05:49:25.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:25.813+0000 7feb72c20640 1 -- 172.21.15.71:0/4269523207 wait complete. 2023-10-26T05:49:25.832 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:49:25.962 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":5778,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.355516430000002,"mdsmap_epoch":18,"osdmap_epoch":53,"osdmap_epoch_barrier":48,"uptime":28.060358621999999} 2023-10-26T05:49:25.964 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 5778, "mdsmap_epoch": 18, "osdmap_epoch": 53, "osdmap_epoch_barrier": 48, "rank_uptime": 22.35551643, "state": "up:active", "uptime": 28.060358622, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:49:25.964 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:49:25.964 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:25.965 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:25.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:25.971 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-26T05:49:26.022 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:49:26.017+0000 7f49f27fc640 -1 client.5943 handle_client_reply no pending request on tid 14 safe is:1 2023-10-26T05:49:26.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:49:26.024 INFO:teuthology.orchestra.run.smithi071.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-26T05:49:26.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:26.024 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T05:49:26.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:26.098 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T05:49:26.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:26.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/3790274880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64980039b0 msgr2=0x7f6498003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:26.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 --2- 172.21.15.71:0/3790274880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64980039b0 0x7f6498003dd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f64a800c260 tx=0x7f64a801cfc0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b23bf640 1 -- 172.21.15.71:0/3790274880 reap_dead start 2023-10-26T05:49:26.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/3790274880 shutdown_connections 2023-10-26T05:49:26.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/3790274880 >> 172.21.15.71:0/3790274880 conn(0x7f64ac069770 msgr2=0x7f64ac069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:26.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/3790274880 shutdown_connections 2023-10-26T05:49:26.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/3790274880 wait complete. 2023-10-26T05:49:26.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 Processor -- start 2023-10-26T05:49:26.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- start start 2023-10-26T05:49:26.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64980039b0 0x7f64ac13c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:26.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 0x7f64ac13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:26.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64ac13d350 0x7f64ac13d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:26.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac13dd10 con 0x7f64ac13f960 2023-10-26T05:49:26.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac140d50 con 0x7f64980039b0 2023-10-26T05:49:26.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac15f110 con 0x7f64ac13d350 2023-10-26T05:49:26.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 0x7f64ac13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:26.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 0x7f64ac13fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53846/0 (socket says 172.21.15.71:53846) 2023-10-26T05:49:26.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 -- 172.21.15.71:0/541250687 learned_addr learned my addr 172.21.15.71:0/541250687 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:26.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b1bbe640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64980039b0 0x7f64ac13c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:26.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b23bf640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64ac13d350 0x7f64ac13d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:26.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 -- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64ac13d350 msgr2=0x7f64ac13d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:26.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64ac13d350 0x7f64ac13d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 -- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64980039b0 msgr2=0x7f64ac13c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:26.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64980039b0 0x7f64ac13c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 -- 172.21.15.71:0/541250687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0b31a0 con 0x7f64ac13f960 2023-10-26T05:49:26.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b1bbe640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64980039b0 0x7f64ac13c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:26.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b13bd640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 0x7f64ac13fdc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b680 tx=0x7f649c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:26.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c0041f0 con 0x7f64ac13f960 2023-10-26T05:49:26.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f649c010b50 con 0x7f64ac13f960 2023-10-26T05:49:26.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac1404b0 con 0x7f64ac13f960 2023-10-26T05:49:26.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c01a470 con 0x7f64ac13f960 2023-10-26T05:49:26.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.329+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac140a70 con 0x7f64ac13f960 2023-10-26T05:49:26.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac066b10 con 0x7f64ac13f960 2023-10-26T05:49:26.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f649c017070 con 0x7f64ac13f960 2023-10-26T05:49:26.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64a27fc640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6490066470 0x7f6490068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:26.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f649c084ae0 con 0x7f64ac13f960 2023-10-26T05:49:26.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=54}) v3 -- 0x7f649006e060 con 0x7f64ac13f960 2023-10-26T05:49:26.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64b1bbe640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6490066470 0x7f6490068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:26.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.333+0000 7f64b1bbe640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6490066470 0x7f6490068930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f64a80ac000 tx=0x7f64a80a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:26.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.337+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f649c051f20 con 0x7f64ac13f960 2023-10-26T05:49:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.489+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f64ac0721e0 con 0x7f64ac13f960 2023-10-26T05:49:26.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.493+0000 7f64a27fc640 1 -- 172.21.15.71:0/541250687 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v53) v1 ==== 81+0+34378 (secure 0 0 0) 0x7f649c055540 con 0x7f64ac13f960 2023-10-26T05:49:26.498 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:26.498 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 53, 2023-10-26T05:49:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:23.068585+0000", 2023-10-26T05:49:26.499 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:26.500 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:26.501 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:26.502 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:26.503 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:26.504 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:26.505 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:26.506 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:26.507 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:26.508 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.509 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:26.510 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:26.511 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:26.512 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:26.513 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:26.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:26.515 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:26.516 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:26.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:26.518 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:26.519 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.520 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:26.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:26.522 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:26.523 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:26.524 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:26.525 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:26.526 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:26.527 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:26.528 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:26.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:26.530 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:26.531 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:26.532 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:26.533 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:26.534 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:26.535 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:26.536 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.537 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:26.538 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:26.539 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:26.540 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:26.541 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:26.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:26.543 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:26.544 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:26.545 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:26.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:26.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:26.548 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:26.549 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:26.550 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:26.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:26.552 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:26.553 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:26.554 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.555 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.556 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:26.557 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:26.558 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.562 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.563 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.564 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:26.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.567 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:26.569 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.570 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.571 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.572 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.573 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.575 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.576 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.577 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.578 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.580 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.581 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.582 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.583 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:26.584 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.586 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:26.587 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.588 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.589 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.590 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.593 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.596 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:26.597 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:26.598 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:26.599 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:26.600 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.601 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.602 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:26.603 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.608 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:26.609 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.610 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.612 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:26.613 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:49:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.615 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.616 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.620 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:26.621 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.622 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.624 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.626 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.627 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.628 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:26.629 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.630 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.631 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.632 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.633 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.634 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.637 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.638 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:26.640 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:26.641 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:26.642 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.643 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.644 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.645 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.646 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.648 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.648 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:26.648 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.648 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.649 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.649 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.650 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:26.650 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.650 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.650 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:26.651 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.651 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.651 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.652 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.652 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.652 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:26.652 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.653 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.653 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:26.653 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.654 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.654 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.654 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.655 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:26.655 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.656 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:26.656 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:26.656 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.656 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.657 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:26.657 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:26.657 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:26.657 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:26.657 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.657 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.658 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:26.658 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.658 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.659 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:26.659 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:26.659 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:26.659 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:26.659 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:26.660 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:26.661 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:26.661 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:26.661 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.662 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:26.662 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.663 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:26.663 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.663 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.663 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:26.664 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.664 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:26.664 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.665 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.665 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:26.665 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.666 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.666 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:26.666 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.667 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:26.667 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.668 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:26.668 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.669 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:26.669 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:26.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:26.669 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:26.670 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:26.670 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:26.671 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:26.671 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.671 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.671 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:26.671 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:26.672 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:26.672 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:26.672 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:26.672 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:26.672 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:26.673 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:26.673 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.673 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.673 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:26.673 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:26.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.674 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:26.674 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.674 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.674 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.674 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.675 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.675 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:26.675 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.676 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:26.676 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.676 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.676 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.676 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.676 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.677 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:26.677 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.677 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:26.678 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.678 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.678 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.678 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.678 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.678 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:26.679 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.679 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:26.679 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.680 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.680 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.680 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.680 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.680 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:26.680 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.681 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:26.681 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.681 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.682 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.682 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.682 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.682 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:26.682 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.683 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:26.683 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.683 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.683 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.684 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.684 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.684 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:26.684 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.685 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:26.685 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.685 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.685 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.685 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.686 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.686 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:26.686 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.686 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.687 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:26.687 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:26.687 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:26.687 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:26.687 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:26.688 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:26.688 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:26.688 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:26.688 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.688 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:26.688 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:26.689 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:26.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.689 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:26.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:26.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.690 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:26.690 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:26.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.690 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.691 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:26.691 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:26.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:26.691 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:26.691 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:26.692 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.692 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:26.692 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.692 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:26.692 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:26.693 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:26.693 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:26.693 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:26.693 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:26.693 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:26.693 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:26.694 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:26.694 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:26.694 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:26.694 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:26.694 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:26.695 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:26.695 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:26.695 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:26.695 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:26.695 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:26.696 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:26.697 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:26.697 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6490066470 msgr2=0x7f6490068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6490066470 0x7f6490068930 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f64a80ac000 tx=0x7f64a80a7040 comp rx=0 tx=0).stop 2023-10-26T05:49:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 msgr2=0x7f64ac13fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 0x7f64ac13fdc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b680 tx=0x7f649c00bb40 comp rx=0 tx=0).stop 2023-10-26T05:49:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 shutdown_connections 2023-10-26T05:49:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64ac13d350 0x7f64ac13d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6490066470 0x7f6490068930 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64ac13f960 0x7f64ac13fdc0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 --2- 172.21.15.71:0/541250687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64980039b0 0x7f64ac13c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 >> 172.21.15.71:0/541250687 conn(0x7f64ac069770 msgr2=0x7f64ac166350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 shutdown_connections 2023-10-26T05:49:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:26.497+0000 7f64b3e22640 1 -- 172.21.15.71:0/541250687 wait complete. 2023-10-26T05:49:26.702 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:49:27.067 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:49:27.065+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T05:49:31.705 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:31.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/1224913666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a980047d0 msgr2=0x7f2a98004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:31.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 --2- 172.21.15.71:0/1224913666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a980047d0 0x7f2a98004c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009b60 tx=0x7f2aa001d230 comp rx=0 tx=0).stop 2023-10-26T05:49:31.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 -- 172.21.15.71:0/1224913666 reap_dead start 2023-10-26T05:49:31.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/1224913666 shutdown_connections 2023-10-26T05:49:31.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/1224913666 >> 172.21.15.71:0/1224913666 conn(0x7f2ab0069770 msgr2=0x7f2ab0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:31.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/1224913666 shutdown_connections 2023-10-26T05:49:31.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/1224913666 wait complete. 2023-10-26T05:49:31.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 Processor -- start 2023-10-26T05:49:31.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- start start 2023-10-26T05:49:31.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a980047d0 0x7f2ab013bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:31.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab01596a0 0x7f2ab014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:31.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 0x7f2ab01f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:31.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00abdb0 con 0x7f2ab01f1a00 2023-10-26T05:49:31.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab014e170 con 0x7f2ab01596a0 2023-10-26T05:49:31.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab013b740 con 0x7f2a980047d0 2023-10-26T05:49:31.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 0x7f2ab01f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:31.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 0x7f2ab01f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46682/0 (socket says 172.21.15.71:46682) 2023-10-26T05:49:31.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 -- 172.21.15.71:0/3608963976 learned_addr learned my addr 172.21.15.71:0/3608963976 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:31.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2aaffff640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab01596a0 0x7f2ab014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:31.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab4e05640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a980047d0 0x7f2ab013bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 -- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a980047d0 msgr2=0x7f2ab013bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a980047d0 0x7f2ab013bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 -- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab01596a0 msgr2=0x7f2ab014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab01596a0 0x7f2ab014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:31.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 -- 172.21.15.71:0/3608963976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab013b9e0 con 0x7f2ab01f1a00 2023-10-26T05:49:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab5606640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 0x7f2ab01f1e80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2ab013e070 tx=0x7f2aa8010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8019030 con 0x7f2ab01f1a00 2023-10-26T05:49:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2aa8013480 con 0x7f2ab01f1a00 2023-10-26T05:49:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8013890 con 0x7f2ab01f1a00 2023-10-26T05:49:31.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab01588e0 con 0x7f2ab01f1a00 2023-10-26T05:49:31.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.837+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0158ea0 con 0x7f2ab01f1a00 2023-10-26T05:49:31.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab0066b10 con 0x7f2ab01f1a00 2023-10-26T05:49:31.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2aa801c070 con 0x7f2ab01f1a00 2023-10-26T05:49:31.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2aad7fa640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a9c066470 0x7f2a9c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:31.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2aa801a070 con 0x7f2ab01f1a00 2023-10-26T05:49:31.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=57}) v3 -- 0x7f2a9c06e060 con 0x7f2ab01f1a00 2023-10-26T05:49:31.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2ab4e05640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a9c066470 0x7f2a9c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:31.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.841+0000 7f2ab4e05640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a9c066470 0x7f2a9c068930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005d10 tx=0x7f2aa4005c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:31.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:31.845+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2aa8055010 con 0x7f2ab01f1a00 2023-10-26T05:49:32.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.001+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2ab00721e0 con 0x7f2ab01f1a00 2023-10-26T05:49:32.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.005+0000 7f2aad7fa640 1 -- 172.21.15.71:0/3608963976 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+34274 (secure 0 0 0) 0x7f2aa8058630 con 0x7f2ab01f1a00 2023-10-26T05:49:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:32.011 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 56, 2023-10-26T05:49:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:31.131143+0000", 2023-10-26T05:49:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:49:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:49:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "51", 2023-10-26T05:49:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:49:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:32.214 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:32.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a9c066470 msgr2=0x7f2a9c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:32.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a9c066470 0x7f2a9c068930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4005d10 tx=0x7f2aa4005c60 comp rx=0 tx=0).stop 2023-10-26T05:49:32.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 msgr2=0x7f2ab01f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:32.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 0x7f2ab01f1e80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2ab013e070 tx=0x7f2aa8010ef0 comp rx=0 tx=0).stop 2023-10-26T05:49:32.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 shutdown_connections 2023-10-26T05:49:32.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a9c066470 0x7f2a9c068930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ab01f1a00 0x7f2ab01f1e80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab01596a0 0x7f2ab014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 --2- 172.21.15.71:0/3608963976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a980047d0 0x7f2ab013bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 >> 172.21.15.71:0/3608963976 conn(0x7f2ab0069770 msgr2=0x7f2ab00b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 shutdown_connections 2023-10-26T05:49:32.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:32.009+0000 7f2ab7069640 1 -- 172.21.15.71:0/3608963976 wait complete. 2023-10-26T05:49:32.218 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:49:32.219 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:49:32.065+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-10-26T05:49:37.220 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/2075709646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c15b530 msgr2=0x7f529c157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- 172.21.15.71:0/2075709646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c15b530 0x7f529c157a00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f529c0afd40 tx=0x7f529401d2c0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/2075709646 shutdown_connections 2023-10-26T05:49:37.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- 172.21.15.71:0/2075709646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c15b530 0x7f529c157a00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- 172.21.15.71:0/2075709646 >> v2:172.21.15.86:3300/0 conn(0x7f529c14e6e0 0x7f529c15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- 172.21.15.71:0/2075709646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529c14e300 0x7f529c0cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/2075709646 >> 172.21.15.71:0/2075709646 conn(0x7f529c069770 msgr2=0x7f529c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:37.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/2075709646 shutdown_connections 2023-10-26T05:49:37.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/2075709646 wait complete. 2023-10-26T05:49:37.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 Processor -- start 2023-10-26T05:49:37.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- start start 2023-10-26T05:49:37.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 0x7f529c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529c14e6e0 0x7f529c1433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c15b530 0x7f529c159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14b410 con 0x7f529c14e300 2023-10-26T05:49:37.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14b6f0 con 0x7f529c15b530 2023-10-26T05:49:37.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c144a30 con 0x7f529c14e6e0 2023-10-26T05:49:37.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 0x7f529c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 0x7f529c14abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46698/0 (socket says 172.21.15.71:46698) 2023-10-26T05:49:37.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 -- 172.21.15.71:0/725758681 learned_addr learned my addr 172.21.15.71:0/725758681 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:37.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529b7fe640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529c14e6e0 0x7f529c1433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a0acb640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c15b530 0x7f529c159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 -- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529c14e6e0 msgr2=0x7f529c1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529c14e6e0 0x7f529c1433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 -- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c15b530 msgr2=0x7f529c159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c15b530 0x7f529c159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 -- 172.21.15.71:0/725758681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529400b510 con 0x7f529c14e300 2023-10-26T05:49:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f529bfff640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 0x7f529c14abc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c940 tx=0x7f528c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c007bb0 con 0x7f529c14e300 2023-10-26T05:49:37.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f528c007e20 con 0x7f529c14e300 2023-10-26T05:49:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00f6b0 con 0x7f529c14e300 2023-10-26T05:49:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c144d10 con 0x7f529c14e300 2023-10-26T05:49:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.357+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c14b9d0 con 0x7f529c14e300 2023-10-26T05:49:37.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.361+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c066b10 con 0x7f529c14e300 2023-10-26T05:49:37.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.361+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f528c00f810 con 0x7f529c14e300 2023-10-26T05:49:37.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.361+0000 7f5298ff9640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5288066470 0x7f5288068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.361+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f528c0847e0 con 0x7f529c14e300 2023-10-26T05:49:37.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.361+0000 7f529b7fe640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5288066470 0x7f5288068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.361+0000 7f529b7fe640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5288066470 0x7f5288068930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5290005fd0 tx=0x7f529000a4e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:37.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.365+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f528c051cb0 con 0x7f529c14e300 2023-10-26T05:49:37.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.521+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f529c0721e0 con 0x7f529c14e300 2023-10-26T05:49:37.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.521+0000 7f5298ff9640 1 -- 172.21.15.71:0/725758681 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v57) v1 ==== 81+0+34235 (secure 0 0 0) 0x7f528c01d020 con 0x7f529c14e300 2023-10-26T05:49:37.528 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:37.528 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:37.529 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2023-10-26T05:49:37.529 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:37.529 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:37.530 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:32.147514+0000", 2023-10-26T05:49:37.530 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:37.530 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:37.530 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:37.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:37.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:37.531 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:37.531 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:37.531 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:37.532 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:37.532 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:37.532 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:37.532 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:37.533 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:37.533 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:37.533 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:37.533 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:37.534 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:37.534 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:37.534 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:37.534 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:37.534 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:37.535 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:37.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.535 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:37.535 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:37.536 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:37.536 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:37.536 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:37.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:37.537 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:37.537 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:37.537 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:37.537 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:37.537 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:37.538 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:37.538 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:37.538 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:37.538 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:37.539 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:37.539 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:37.539 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:37.539 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:37.539 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:37.540 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:37.540 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:37.540 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:37.540 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:37.541 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:37.541 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:37.541 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:37.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.541 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "57", 2023-10-26T05:49:37.542 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:37.542 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:37.542 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:37.542 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:37.542 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:37.543 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:37.543 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:37.543 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:37.543 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:37.544 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:37.544 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:37.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:37.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:37.544 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:37.545 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:37.545 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:37.545 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:37.545 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:37.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:37.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:37.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:37.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:37.546 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:37.547 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:37.547 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:37.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:37.547 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.547 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:37.548 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:37.548 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:37.548 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:37.548 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:37.549 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:37.549 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:37.549 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:37.549 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:37.550 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:37.550 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:37.550 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:37.550 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:37.551 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:37.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.551 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:37.551 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:37.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.552 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:37.552 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:37.552 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:37.553 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:37.553 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:37.553 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:37.553 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:37.554 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:37.554 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:37.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.555 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:37.555 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:37.555 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:37.556 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:37.556 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:37.556 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:37.556 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:37.557 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:37.557 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:37.557 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:37.557 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:37.558 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:37.558 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:37.558 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:37.558 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:37.559 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:37.559 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:37.559 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:37.559 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:37.560 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:37.560 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:37.560 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:37.560 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:37.561 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:37.561 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:37.561 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:37.561 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:37.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.562 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "57", 2023-10-26T05:49:37.562 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:37.562 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:37.563 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:37.563 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:37.563 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:37.563 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:37.564 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:37.564 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:37.564 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:37.564 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:37.565 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:37.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:37.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:37.565 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:37.566 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:37.566 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:37.566 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:37.566 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:37.566 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:37.567 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:37.567 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:37.567 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:37.567 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:37.568 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:37.568 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:37.568 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:37.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.569 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:37.569 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:37.569 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:37.569 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:37.569 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:37.570 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:37.570 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:37.570 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:37.570 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:37.570 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:37.571 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:37.571 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:37.571 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:37.571 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:37.572 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:37.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.572 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:37.572 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:37.573 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:37.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.573 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:37.574 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:37.574 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:37.574 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:37.574 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:37.575 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:37.575 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:37.575 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:37.576 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:37.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.577 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:37.577 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:37.577 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:37.577 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:37.578 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:37.578 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:37.578 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:37.578 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:37.578 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:37.579 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:37.579 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:37.579 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:37.579 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:37.580 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:37.580 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:37.580 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:37.580 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:37.580 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:37.581 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:37.581 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:37.581 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:37.581 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:37.582 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:37.582 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:37.582 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:37.582 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:37.582 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "57", 2023-10-26T05:49:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:37.583 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:37.584 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:37.584 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:37.584 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:37.584 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:37.585 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:37.585 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:37.585 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:37.585 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:37.586 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:37.586 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:37.586 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:37.586 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:37.587 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:37.587 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:37.587 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:37.587 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:37.588 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:37.588 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:37.588 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:37.588 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:37.588 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:37.589 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:37.589 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:37.589 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:37.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.590 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:37.590 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:37.590 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:37.590 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:37.591 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:37.591 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:37.591 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:37.591 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:37.592 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:37.592 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:37.592 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:37.592 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:37.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:37.593 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:37.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.593 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:37.594 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:37.594 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:37.594 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.595 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:37.595 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:37.595 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:37.595 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:37.596 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:37.596 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:37.596 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:37.596 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:37.596 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:37.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.597 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:37.597 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:37.598 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.598 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:37.598 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.598 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.599 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.599 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.599 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.599 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.599 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.600 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.600 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:37.600 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.600 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.600 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.601 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.601 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.601 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:37.602 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.602 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.603 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:37.603 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.603 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.603 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.604 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.604 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.604 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.605 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:37.605 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.606 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.606 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:37.606 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.606 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.607 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.607 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.607 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.608 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.608 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:37.608 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.609 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.609 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:37.609 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.609 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.610 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.610 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.611 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.611 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.611 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.611 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:37.611 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.612 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.612 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:37.613 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:37.613 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.613 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.613 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:37.614 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:37.614 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:37.614 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:37.614 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.615 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.615 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.615 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.616 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:37.616 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.616 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.616 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.617 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.617 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.617 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.617 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.618 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.618 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:37.618 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.618 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.619 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.619 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.620 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:37.620 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.621 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:37.621 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.622 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.622 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.623 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:37.623 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.624 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.624 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:37.624 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.624 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.625 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.625 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.625 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.626 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.626 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:37.626 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.627 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.627 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:37.627 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.628 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.628 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.628 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.629 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.629 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:37.629 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.630 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.630 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.630 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:37.630 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:37.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.631 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.631 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:37.631 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.633 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.633 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:37.634 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.634 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.634 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.634 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.634 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.635 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.635 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.635 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.635 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:37.636 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.636 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.636 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.636 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.637 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.637 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:37.637 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.638 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.638 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:37.638 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.639 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.639 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.639 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.640 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.640 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:37.640 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.641 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:37.641 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.641 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.642 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.642 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.643 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.643 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.643 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:37.644 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.644 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.644 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:37.645 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.645 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.645 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.645 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.646 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.646 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.646 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.646 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:37.647 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.648 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:37.648 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:37.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.648 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.649 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:37.649 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:37.649 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:37.649 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:37.649 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.650 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.650 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.650 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.651 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:37.651 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.651 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.652 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.652 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.652 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.652 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.653 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.653 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.653 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:37.653 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.654 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.654 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.654 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.657 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.657 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:37.658 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.658 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.659 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:37.659 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.659 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.660 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.661 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:37.661 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.662 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:37.662 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.662 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.663 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.663 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.663 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.664 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:37.664 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.665 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.665 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:37.665 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.666 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.666 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.666 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.667 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:37.667 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.668 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:37.668 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:37.669 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.669 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.669 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:37.670 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:37.670 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:37.670 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:37.670 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.670 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.671 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.671 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.672 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:37.672 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.672 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.672 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.673 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.673 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.673 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.673 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.674 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.674 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:37.674 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.674 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.675 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.675 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.675 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.675 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:37.676 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.676 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:37.677 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.677 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.677 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:37.678 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:37.680 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.680 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.680 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.680 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.681 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.681 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:37.682 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.682 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.682 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:37.683 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.683 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.683 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.683 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:37.685 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:37.686 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:37.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.686 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.686 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:37.687 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:37.687 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:37.687 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:37.687 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.688 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.688 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.688 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:37.689 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.689 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.689 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.690 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.690 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.690 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.690 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.691 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.691 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:37.691 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.691 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.691 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:37.693 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:37.694 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.694 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.695 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.695 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.695 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.696 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:37.696 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:37.697 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.697 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.697 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.698 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.698 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:37.700 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.701 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.702 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:37.704 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:37.704 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.704 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.705 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:37.705 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:37.705 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:37.705 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:37.706 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.706 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.706 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.706 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.707 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:37.707 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.708 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.708 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.708 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.708 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.708 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.709 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.709 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.709 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:37.710 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.710 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.710 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.710 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.711 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:37.712 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.712 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.713 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:37.713 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.713 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.714 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:37.715 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:37.716 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.717 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.717 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.718 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.718 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:37.718 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.719 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:37.719 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.719 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.720 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.720 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.720 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.721 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:37.721 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.722 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.722 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:37.723 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:37.723 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.723 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.723 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:37.724 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:37.724 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:37.724 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:37.724 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.725 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.725 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.725 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.726 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:37.726 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:37.726 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:37.727 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:37.727 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:37.727 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:37.727 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:37.727 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:37.728 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:37.728 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:37.728 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:37.728 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:37.729 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:37.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:37.730 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.731 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:37.731 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.731 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.732 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:37.732 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.733 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.733 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:37.733 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:37.734 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.734 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.735 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.735 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:37.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.736 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:37.736 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.736 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:37.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.738 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:37.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:37.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.739 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:37.739 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:37.739 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:37.740 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:37.740 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:37.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.741 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.741 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:37.741 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:37.742 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:37.742 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:37.742 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:37.743 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:37.743 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:37.743 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.743 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:37.744 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:37.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.744 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:37.744 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.745 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.745 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.745 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.745 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.746 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:37.746 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.747 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:37.747 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.747 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.748 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.748 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.748 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.748 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:37.748 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.749 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:37.749 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.750 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.750 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.750 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.750 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.750 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:37.751 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.751 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:37.751 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.752 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.752 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.752 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.752 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.753 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:37.753 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.754 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:37.754 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.754 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.754 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.755 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.755 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.755 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:37.755 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.756 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:37.756 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.756 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.757 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.757 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.757 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:37.758 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.758 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:37.759 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.759 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.759 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.759 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.760 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.760 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:37.760 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.761 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:37.761 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:37.761 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:37.761 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:37.762 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:37.762 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:37.762 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:37.762 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:37.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.763 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:37.763 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:37.763 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:37.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.764 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:37.764 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:37.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.764 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:37.765 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:37.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.765 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.766 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:37.766 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:37.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:37.766 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:37.767 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:37.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.767 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:37.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.767 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:37.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:37.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:37.768 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:37.768 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:37.769 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:37.769 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:37.769 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:37.769 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:37.769 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:37.770 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:37.770 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:37.770 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:37.770 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:37.771 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:37.771 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:37.771 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:37.771 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:37.772 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:37.772 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:37.772 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:37.772 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:37.772 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:37.773 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:37.773 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:37.773 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:37.773 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:37.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5288066470 msgr2=0x7f5288068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5288066470 0x7f5288068930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5290005fd0 tx=0x7f529000a4e0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 msgr2=0x7f529c14abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 0x7f529c14abc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c940 tx=0x7f528c00ce00 comp rx=0 tx=0).stop 2023-10-26T05:49:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 shutdown_connections 2023-10-26T05:49:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c15b530 0x7f529c159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f529c14e6e0 0x7f529c1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5288066470 0x7f5288068930 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 --2- 172.21.15.71:0/725758681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f529c14e300 0x7f529c14abc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 >> 172.21.15.71:0/725758681 conn(0x7f529c069770 msgr2=0x7f529c0b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 shutdown_connections 2023-10-26T05:49:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.525+0000 7f52a252e640 1 -- 172.21.15.71:0/725758681 wait complete. 2023-10-26T05:49:37.778 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T05:49:37.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:49:37.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 -- 172.21.15.71:0/857202253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08140cc0 msgr2=0x7f6b0815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/857202253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08140cc0 0x7f6b0815c760 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009840 tx=0x7f6af801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 -- 172.21.15.71:0/857202253 shutdown_connections 2023-10-26T05:49:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/857202253 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b080b2370 0x7f6b08071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/857202253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08140cc0 0x7f6b0815c760 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 -- 172.21.15.71:0/857202253 >> 172.21.15.71:0/857202253 conn(0x7f6b08069770 msgr2=0x7f6b08069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:37.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 -- 172.21.15.71:0/857202253 shutdown_connections 2023-10-26T05:49:37.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.905+0000 7f6b0ef28640 1 -- 172.21.15.71:0/857202253 wait complete. 2023-10-26T05:49:37.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 Processor -- start 2023-10-26T05:49:37.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- start start 2023-10-26T05:49:37.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 0x7f6b081576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b080b2370 0x7f6b0815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b08140cc0 0x7f6b0813dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08157c00 con 0x7f6b08070fe0 2023-10-26T05:49:37.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08158040 con 0x7f6b080b2370 2023-10-26T05:49:37.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08159bd0 con 0x7f6b08140cc0 2023-10-26T05:49:37.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 0x7f6b081576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 0x7f6b081576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46716/0 (socket says 172.21.15.71:46716) 2023-10-26T05:49:37.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 -- 172.21.15.71:0/3654008097 learned_addr learned my addr 172.21.15.71:0/3654008097 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:37.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b07fff640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b080b2370 0x7f6b0815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0d4c5640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b08140cc0 0x7f6b0813dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 -- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b08140cc0 msgr2=0x7f6b0813dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b08140cc0 0x7f6b0813dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 -- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b080b2370 msgr2=0x7f6b0815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:37.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b080b2370 0x7f6b0815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 -- 172.21.15.71:0/3654008097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80094a0 con 0x7f6b08070fe0 2023-10-26T05:49:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b07fff640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b080b2370 0x7f6b0815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0d4c5640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b08140cc0 0x7f6b0813dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ccc4640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 0x7f6b081576c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000c10 tx=0x7f6af8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80bc070 con 0x7f6b08070fe0 2023-10-26T05:49:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6af8005320 con 0x7f6b08070fe0 2023-10-26T05:49:37.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b08159eb0 con 0x7f6b08070fe0 2023-10-26T05:49:37.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af80046d0 con 0x7f6b08070fe0 2023-10-26T05:49:37.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0815a470 con 0x7f6b08070fe0 2023-10-26T05:49:37.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.909+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b08066b10 con 0x7f6b08070fe0 2023-10-26T05:49:37.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.913+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6af80b4040 con 0x7f6b08070fe0 2023-10-26T05:49:37.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.913+0000 7f6b057fa640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ae80664a0 0x7f6ae8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:37.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.913+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6af811e270 con 0x7f6b08070fe0 2023-10-26T05:49:37.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.913+0000 7f6b07fff640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ae80664a0 0x7f6ae8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:37.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.913+0000 7f6b07fff640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ae80664a0 0x7f6ae8068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b0806da10 tx=0x7f6af0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:37.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:37.917+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6af80eb740 con 0x7f6b08070fe0 2023-10-26T05:49:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.069+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6b080721e0 con 0x7f6b08070fe0 2023-10-26T05:49:38.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.073+0000 7f6b057fa640 1 -- 172.21.15.71:0/3654008097 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v57) v1 ==== 81+0+34235 (secure 0 0 0) 0x7f6af80eed60 con 0x7f6b08070fe0 2023-10-26T05:49:38.079 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:38.080 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:38.080 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 57, 2023-10-26T05:49:38.080 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:49:38.080 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:49:38.081 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:49:32.147514+0000", 2023-10-26T05:49:38.081 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:49:38.081 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:49:38.081 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:49:38.081 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:49:38.082 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:49:38.082 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:49:38.082 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:49:38.082 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:49:38.082 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:49:38.083 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:38.083 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:49:38.083 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:49:38.083 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:49:38.083 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:49:38.084 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:49:38.084 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 7, 2023-10-26T05:49:38.084 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:49:38.084 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:49:38.084 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:49:38.085 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:49:38.085 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:49:38.085 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:49:38.085 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.085 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:49:38.086 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:49:38.086 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:49:38.086 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:38.086 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:38.086 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:38.087 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:38.087 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:38.087 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:38.087 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:38.088 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:38.088 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:38.088 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:38.088 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:38.088 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:38.089 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:49:38.089 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:49:38.089 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:49:38.089 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:49:38.089 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:49:38.090 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:38.090 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:38.090 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:38.090 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:38.090 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:38.091 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:38.091 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:38.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.091 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "57", 2023-10-26T05:49:38.091 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:38.092 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:38.092 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:38.093 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:38.093 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:38.093 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:38.094 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:38.094 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:38.094 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:38.094 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:38.095 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:38.095 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:38.095 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:38.095 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:38.095 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:38.096 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:38.096 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:38.096 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:38.096 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:38.097 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:38.097 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:38.097 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:38.098 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:38.098 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:38.098 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:38.098 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:38.099 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.099 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:38.099 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:38.099 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:38.100 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:38.100 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:38.100 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:38.100 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:38.101 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:38.101 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:38.102 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:38.102 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:38.102 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:38.103 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:49:38.103 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:49:38.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.103 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:38.103 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:49:38.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.104 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:38.104 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:49:38.104 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:49:38.104 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:49:38.105 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:49:38.105 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:49:38.105 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:38.105 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:38.105 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:38.106 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.106 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 6, 2023-10-26T05:49:38.106 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:49:38.107 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:00.912702+0000", 2023-10-26T05:49:38.107 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:38.107 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:38.107 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:38.107 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:38.108 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:38.108 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:38.108 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:38.108 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:38.108 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:38.108 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:38.109 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:38.109 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:38.109 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:49:38.109 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:49:38.109 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:49:38.110 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:49:38.110 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:49:38.110 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:38.110 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:38.110 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:38.111 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:38.111 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:38.111 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:38.111 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:38.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "57", 2023-10-26T05:49:38.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:38.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:38.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:38.112 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:38.112 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:38.113 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:38.113 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:38.113 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:38.113 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:38.113 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:38.114 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:38.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:38.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:38.114 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:38.114 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:38.115 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:38.115 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:38.115 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:38.115 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:38.115 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:38.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:38.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:38.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:38.116 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:38.116 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:38.116 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:38.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.117 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:38.117 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:38.117 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:38.117 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:38.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:38.118 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:38.118 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:38.118 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:38.118 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:38.119 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:38.119 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:38.119 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:38.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:49:38.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:49:38.120 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:49:38.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.120 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:38.120 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:38.120 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:49:38.120 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.121 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:38.121 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:49:38.121 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:49:38.121 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:38.122 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:49:38.122 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:49:38.122 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:38.122 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:38.122 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:38.123 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.123 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 7, 2023-10-26T05:49:38.123 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:49:38.123 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:49:01.904497+0000", 2023-10-26T05:49:38.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:49:38.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:49:38.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:49:38.124 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:49:38.124 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:49:38.125 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:49:38.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:49:38.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:49:38.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:49:38.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:49:38.126 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:49:38.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:49:38.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:49:38.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:49:38.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:49:38.127 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:49:38.127 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:49:38.127 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:49:38.127 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:49:38.127 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:49:38.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:49:38.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:49:38.128 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:49:38.128 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:49:38.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "57", 2023-10-26T05:49:38.129 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:49:38.129 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:49:38.129 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:49:38.129 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:49:38.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:49:38.130 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:49:38.130 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:49:38.130 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:49:38.130 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:49:38.131 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:49:38.131 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:49:38.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:49:38.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:49:38.131 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:49:38.132 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:49:38.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:49:38.132 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:49:38.132 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:49:38.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:49:38.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:49:38.133 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:49:38.133 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:49:38.133 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:49:38.133 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:49:38.133 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:49:38.134 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:49:38.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.134 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:49:38.134 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:49:38.134 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:49:38.135 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:49:38.135 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:49:38.135 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:49:38.135 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:49:38.135 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:49:38.136 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:49:38.136 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:49:38.136 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:49:38.136 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:49:38.136 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:49:38.136 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:49:38.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.137 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:49:38.137 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:49:38.137 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:49:38.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.138 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:49:38.138 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:49:38.138 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:49:38.138 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:49:38.139 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:49:38.139 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:49:38.139 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:49:38.139 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:49:38.139 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:49:38.140 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.140 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.140 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:38.140 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:49:38.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.140 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:38.141 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.141 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.141 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.141 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.141 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.142 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.142 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.142 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.142 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:38.142 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.143 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.143 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.143 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.143 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.143 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.144 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:49:38.144 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.144 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.144 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.144 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:49:38.145 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.145 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.145 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.145 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.146 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.146 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.146 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.146 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:49:38.146 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.147 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.147 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:49:38.147 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.147 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.148 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.148 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.148 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.149 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.149 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:49:38.149 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.149 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.150 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:49:38.150 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.150 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.150 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.151 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.151 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.151 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:49:38.152 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.152 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.152 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.152 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:49:38.153 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:49:38.153 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.153 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.153 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:49:38.153 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:49:38.154 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:49:38.154 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:49:38.154 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.154 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.154 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.155 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.155 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.155 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:38.155 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.156 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.156 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.156 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.156 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.156 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.156 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.157 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.157 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:38.157 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.157 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.157 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.158 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.158 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.158 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.158 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:49:38.158 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.159 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:49:38.159 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.159 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:49:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:49:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:49:38.163 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.164 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.164 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:49:38.164 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.165 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:49:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:49:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:49:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.167 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:49:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:49:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:49:38.168 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:49:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.169 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.170 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.171 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.173 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:49:38.173 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:49:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.174 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:49:38.175 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:49:38.176 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:49:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:49:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.179 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.180 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:49:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.181 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:49:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:49:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.182 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:49:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:49:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:49:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:49:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.183 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:38.184 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.185 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.186 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:49:38.187 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.188 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:49:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.189 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:49:38.190 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:49:38.191 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:49:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:49:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:49:38.196 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.197 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:49:38.197 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:49:38.197 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:49:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:49:38.198 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:49:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:49:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.199 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.200 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.201 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.202 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:49:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:49:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.204 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:49:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.206 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:49:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.207 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:49:38.208 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:49:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:49:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.211 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:49:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:49:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:49:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:49:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:49:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:49:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.213 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.214 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.215 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 49, 2023-10-26T05:49:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.216 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.217 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:49:38.218 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.218 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.218 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:49:38.219 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.219 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.219 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.219 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.220 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:49:38.220 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.221 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:49:38.221 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.222 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.222 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:49:38.223 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:49:38.224 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.224 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.225 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:49:38.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.226 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:49:38.226 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:49:38.227 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.227 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.227 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:49:38.227 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:49:38.228 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:49:38.228 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:49:38.228 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.228 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.228 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.229 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:38.229 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.229 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.230 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.230 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.230 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.230 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.230 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.231 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.231 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:38.231 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.231 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.231 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.231 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.232 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:49:38.232 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.233 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:49:38.233 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.233 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.234 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.234 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.235 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.235 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:49:38.235 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.235 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:49:38.236 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.236 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.237 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:49:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:49:38.238 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.239 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.239 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.239 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.239 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:49:38.240 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:49:38.241 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:49:38.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.241 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.242 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:49:38.242 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:49:38.242 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:49:38.242 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:49:38.243 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.243 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.243 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.243 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.244 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:38.244 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:49:38.244 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:49:38.244 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:49:38.244 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:49:38.245 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:49:38.245 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:49:38.245 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:49:38.245 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:49:38.245 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 46, 2023-10-26T05:49:38.245 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:49:38.246 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:49:38.246 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:49:38.246 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.246 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:49:38.247 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:49:38.248 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.248 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.248 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:49:38.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.249 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:49:38.249 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:49:38.250 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.250 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.251 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.251 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:49:38.251 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:49:38.252 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:49:38.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.253 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.253 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.254 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:49:38.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:49:38.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:49:38.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:49:38.255 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:49:38.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:49:38.255 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:49:38.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.256 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.256 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:49:38.256 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:49:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:49:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:49:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:49:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:49:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:49:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:49:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:49:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:49:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:49:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:49:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:49:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:49:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:49:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:49:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:49:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:49:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:49:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:49:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:49:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:49:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:49:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:49:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:49:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:49:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:49:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:49:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:49:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:49:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:49:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:49:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.36", 2023-10-26T05:49:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:49:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "7.3c", 2023-10-26T05:49:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:49:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:49:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 4 2023-10-26T05:49:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:49:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:49:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:49:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:49:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:49:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:49:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:49:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:49:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:49:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:49:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:49:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:49:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:49:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:49:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:49:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:49:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:49:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:49:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:49:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:49:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:49:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:49:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:49:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:49:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:38.284 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:38.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ae80664a0 msgr2=0x7f6ae8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ae80664a0 0x7f6ae8068960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b0806da10 tx=0x7f6af0009040 comp rx=0 tx=0).stop 2023-10-26T05:49:38.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 msgr2=0x7f6b081576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 0x7f6b081576c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000c10 tx=0x7f6af8005040 comp rx=0 tx=0).stop 2023-10-26T05:49:38.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 shutdown_connections 2023-10-26T05:49:38.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b08140cc0 0x7f6b0813dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b080b2370 0x7f6b0815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ae80664a0 0x7f6ae8068960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 --2- 172.21.15.71:0/3654008097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b08070fe0 0x7f6b081576c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 >> 172.21.15.71:0/3654008097 conn(0x7f6b08069770 msgr2=0x7f6b081624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:38.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 shutdown_connections 2023-10-26T05:49:38.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.077+0000 7f6b0ef28640 1 -- 172.21.15.71:0/3654008097 wait complete. 2023-10-26T05:49:38.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:38.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2016614514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e6e0 msgr2=0x7ff45415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2016614514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e6e0 0x7ff45415aff0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff44400a950 tx=0x7ff44401da80 comp rx=0 tx=0).stop 2023-10-26T05:49:38.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2016614514 shutdown_connections 2023-10-26T05:49:38.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2016614514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e6e0 0x7ff45415aff0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2016614514 >> 172.21.15.71:0/2016614514 conn(0x7ff454069770 msgr2=0x7ff454069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:38.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2016614514 shutdown_connections 2023-10-26T05:49:38.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2016614514 wait complete. 2023-10-26T05:49:38.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 Processor -- start 2023-10-26T05:49:38.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- start start 2023-10-26T05:49:38.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 0x7ff454159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:38.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff45414e6e0 0x7ff45415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:38.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff45415b530 0x7ff454154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:38.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454152f20 con 0x7ff45414e300 2023-10-26T05:49:38.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff454152030 con 0x7ff45415b530 2023-10-26T05:49:38.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff454152310 con 0x7ff45414e6e0 2023-10-26T05:49:38.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 0x7ff454159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:38.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 0x7ff454159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46742/0 (socket says 172.21.15.71:46742) 2023-10-26T05:49:38.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 -- 172.21.15.71:0/2229296605 learned_addr learned my addr 172.21.15.71:0/2229296605 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:38.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459164640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff45414e6e0 0x7ff45415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:38.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45a166640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff45415b530 0x7ff454154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:38.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 -- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff45414e6e0 msgr2=0x7ff45415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff45414e6e0 0x7ff45415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 -- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff45415b530 msgr2=0x7ff454154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff45415b530 0x7ff454154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 -- 172.21.15.71:0/2229296605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44400a5b0 con 0x7ff45414e300 2023-10-26T05:49:38.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45a166640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff45415b530 0x7ff454154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:38.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff459965640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 0x7ff454159fc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff45000c450 tx=0x7ff45000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:38.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450012830 con 0x7ff45414e300 2023-10-26T05:49:38.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff450012aa0 con 0x7ff45414e300 2023-10-26T05:49:38.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4541525f0 con 0x7ff45414e300 2023-10-26T05:49:38.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450011380 con 0x7ff45414e300 2023-10-26T05:49:38.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.417+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff454152bb0 con 0x7ff45414e300 2023-10-26T05:49:38.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.421+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454066b10 con 0x7ff45414e300 2023-10-26T05:49:38.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.421+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4500114e0 con 0x7ff45414e300 2023-10-26T05:49:38.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.421+0000 7ff44a7fc640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff438066470 0x7ff438068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:38.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.421+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff450084050 con 0x7ff45414e300 2023-10-26T05:49:38.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.421+0000 7ff459164640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff438066470 0x7ff438068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:38.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.421+0000 7ff459164640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff438066470 0x7ff438068930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b0000 tx=0x7ff44401c2c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:38.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.425+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff450051410 con 0x7ff45414e300 2023-10-26T05:49:38.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.609+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4540721e0 con 0x7ff45414e300 2023-10-26T05:49:38.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.609+0000 7ff44a7fc640 1 -- 172.21.15.71:0/2229296605 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7ff450054a30 con 0x7ff45414e300 2023-10-26T05:49:38.613 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:38.614 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:38.614 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:38.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.609+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff438066470 msgr2=0x7ff438068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.609+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff438066470 0x7ff438068930 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff4440b0000 tx=0x7ff44401c2c0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.609+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 msgr2=0x7ff454159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:38.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.609+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 0x7ff454159fc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff45000c450 tx=0x7ff45000c910 comp rx=0 tx=0).stop 2023-10-26T05:49:38.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 shutdown_connections 2023-10-26T05:49:38.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff45415b530 0x7ff454154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff45414e6e0 0x7ff45415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff438066470 0x7ff438068930 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 --2- 172.21.15.71:0/2229296605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff45414e300 0x7ff454159fc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:38.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 >> 172.21.15.71:0/2229296605 conn(0x7ff454069770 msgr2=0x7ff454158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:38.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 shutdown_connections 2023-10-26T05:49:38.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:38.613+0000 7ff45bbc9640 1 -- 172.21.15.71:0/2229296605 wait complete. 2023-10-26T05:49:38.631 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:49:38.751 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":5778,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.145755459,"mdsmap_epoch":18,"osdmap_epoch":57,"osdmap_epoch_barrier":53,"uptime":40.850565140999997} 2023-10-26T05:49:38.752 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 5778, "mdsmap_epoch": 18, "osdmap_epoch": 57, "osdmap_epoch_barrier": 53, "rank_uptime": 35.145755459, "state": "up:active", "uptime": 40.850565141, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:49:38.753 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:49:38.753 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:38.753 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:38.760 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T05:49:39.254 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records in 2023-10-26T05:49:39.255 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records out 2023-10-26T05:49:39.255 INFO:teuthology.orchestra.run.smithi071.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.440483 s, 119 MB/s 2023-10-26T05:49:39.256 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:49:39.257 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:49:39.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:49:39.372 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:49:39.373 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:39.373 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:39.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:39.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3237253990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00d0470 msgr2=0x7f98c006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3237253990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00d0470 0x7f98c006aa70 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f98c008a3b0 tx=0x7f98b001c720 comp rx=0 tx=0).stop 2023-10-26T05:49:39.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3237253990 shutdown_connections 2023-10-26T05:49:39.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3237253990 >> v2:172.21.15.86:3300/0 conn(0x7f98c00cd880 0x7f98c015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3237253990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00d0470 0x7f98c006aa70 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3237253990 >> 172.21.15.71:0/3237253990 conn(0x7f98c00cdd80 msgr2=0x7f98c00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:39.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3237253990 shutdown_connections 2023-10-26T05:49:39.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3237253990 wait complete. 2023-10-26T05:49:39.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 Processor -- start 2023-10-26T05:49:39.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- start start 2023-10-26T05:49:39.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 0x7f98c0152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c00d0470 0x7f98c0153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98c00d0850 0x7f98c01537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00c9560 con 0x7f98c00cd880 2023-10-26T05:49:39.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0154110 con 0x7f98c00d0470 2023-10-26T05:49:39.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c0153080 con 0x7f98c00d0850 2023-10-26T05:49:39.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 0x7f98c0152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 0x7f98c0152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46766/0 (socket says 172.21.15.71:46766) 2023-10-26T05:49:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 -- 172.21.15.71:0/3452026150 learned_addr learned my addr 172.21.15.71:0/3452026150 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98bf7fe640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98c00d0850 0x7f98c01537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 -- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98c00d0850 msgr2=0x7f98c01537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98be7fc640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c00d0470 0x7f98c0153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98c00d0850 0x7f98c01537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 -- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c00d0470 msgr2=0x7f98c0153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c00d0470 0x7f98c0153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 -- 172.21.15.71:0/3452026150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00094a0 con 0x7f98c00cd880 2023-10-26T05:49:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98beffd640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 0x7f98c0152560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c10 tx=0x7f98b00a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b00042c0 con 0x7f98c00cd880 2023-10-26T05:49:39.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98b0004510 con 0x7f98c00cd880 2023-10-26T05:49:39.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c0153e30 con 0x7f98c00cd880 2023-10-26T05:49:39.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0003840 con 0x7f98c00cd880 2023-10-26T05:49:39.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.553+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00c9180 con 0x7f98c00cd880 2023-10-26T05:49:39.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.557+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0066b10 con 0x7f98c00cd880 2023-10-26T05:49:39.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.557+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f98b00b4040 con 0x7f98c00cd880 2023-10-26T05:49:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.557+0000 7f98a77fe640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98ac066470 0x7f98ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.557+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f98b011f230 con 0x7f98c00cd880 2023-10-26T05:49:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.557+0000 7f98be7fc640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98ac066470 0x7f98ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.557+0000 7f98be7fc640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98ac066470 0x7f98ac068930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f98c015e830 tx=0x7f98b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.561+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98b00ec700 con 0x7f98c00cd880 2023-10-26T05:49:39.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.713+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98c00721e0 con 0x7f98c00cd880 2023-10-26T05:49:39.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.717+0000 7f98a77fe640 1 -- 172.21.15.71:0/3452026150 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f98b00b2020 con 0x7f98c00cd880 2023-10-26T05:49:39.722 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:39.722 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:32.147514+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:39.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98ac066470 msgr2=0x7f98ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98ac066470 0x7f98ac068930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f98c015e830 tx=0x7f98b4009040 comp rx=0 tx=0).stop 2023-10-26T05:49:39.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 msgr2=0x7f98c0152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 0x7f98c0152560 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f98b0000c10 tx=0x7f98b00a57c0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 shutdown_connections 2023-10-26T05:49:39.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98c00d0850 0x7f98c01537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c00d0470 0x7f98c0153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98ac066470 0x7f98ac068930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 --2- 172.21.15.71:0/3452026150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98c00cd880 0x7f98c0152560 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 >> 172.21.15.71:0/3452026150 conn(0x7f98c00cdd80 msgr2=0x7f98c00ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:39.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 shutdown_connections 2023-10-26T05:49:39.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.721+0000 7f98c52a4640 1 -- 172.21.15.71:0/3452026150 wait complete. 2023-10-26T05:49:39.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:49:00.912702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:49:01.904497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:49:39.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:49:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/1142847176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66040047d0 msgr2=0x7f6604004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 --2- 172.21.15.71:0/1142847176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66040047d0 0x7f6604004c50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f660800bbe0 tx=0x7f660801ce30 comp rx=0 tx=0).stop 2023-10-26T05:49:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 -- 172.21.15.71:0/1142847176 reap_dead start 2023-10-26T05:49:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/1142847176 shutdown_connections 2023-10-26T05:49:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/1142847176 >> 172.21.15.71:0/1142847176 conn(0x7f66180b0f20 msgr2=0x7f66180b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/1142847176 shutdown_connections 2023-10-26T05:49:39.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/1142847176 wait complete. 2023-10-26T05:49:39.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 Processor -- start 2023-10-26T05:49:39.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- start start 2023-10-26T05:49:39.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66040047d0 0x7f6618152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f661813ee00 0x7f661806e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 0x7f6618150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6618152450 con 0x7f661806eac0 2023-10-26T05:49:39.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806ef40 con 0x7f66040047d0 2023-10-26T05:49:39.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661813f260 con 0x7f661813ee00 2023-10-26T05:49:39.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 0x7f6618150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 0x7f6618150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46776/0 (socket says 172.21.15.71:46776) 2023-10-26T05:49:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 -- 172.21.15.71:0/182773680 learned_addr learned my addr 172.21.15.71:0/182773680 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661df36640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66040047d0 0x7f6618152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661d735640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f661813ee00 0x7f661806e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 -- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f661813ee00 msgr2=0x7f661806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f661813ee00 0x7f661806e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 -- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66040047d0 msgr2=0x7f6618152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66040047d0 0x7f6618152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 -- 172.21.15.71:0/182773680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66181517d0 con 0x7f661806eac0 2023-10-26T05:49:39.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661df36640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66040047d0 0x7f6618152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:39.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661d735640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f661813ee00 0x7f661806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:39.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f661e737640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 0x7f6618150a60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f661815a990 tx=0x7f6610009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:39.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011710 con 0x7f661806eac0 2023-10-26T05:49:39.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6610011960 con 0x7f661806eac0 2023-10-26T05:49:39.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618151ab0 con 0x7f661806eac0 2023-10-26T05:49:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610011d70 con 0x7f661806eac0 2023-10-26T05:49:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.869+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6618152070 con 0x7f661806eac0 2023-10-26T05:49:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.873+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6618152200 con 0x7f661806eac0 2023-10-26T05:49:39.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.873+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f661000f440 con 0x7f661806eac0 2023-10-26T05:49:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.873+0000 7f660e7fc640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65fc066470 0x7f65fc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:39.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.873+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6610084510 con 0x7f661806eac0 2023-10-26T05:49:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.873+0000 7f661df36640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65fc066470 0x7f65fc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.873+0000 7f661df36640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65fc066470 0x7f65fc068930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6614001000 tx=0x7f6614000bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:39.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:39.877+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6610051950 con 0x7f661806eac0 2023-10-26T05:49:40.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.029+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6618066d00 con 0x7f661806eac0 2023-10-26T05:49:40.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.029+0000 7f660e7fc640 1 -- 172.21.15.71:0/182773680 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6610054f70 con 0x7f661806eac0 2023-10-26T05:49:40.034 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:49:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65fc066470 msgr2=0x7f65fc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65fc066470 0x7f65fc068930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6614001000 tx=0x7f6614000bd0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 msgr2=0x7f6618150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 0x7f6618150a60 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f661815a990 tx=0x7f6610009b00 comp rx=0 tx=0).stop 2023-10-26T05:49:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 shutdown_connections 2023-10-26T05:49:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65fc066470 0x7f65fc068930 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f661806eac0 0x7f6618150a60 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f661813ee00 0x7f661806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 --2- 172.21.15.71:0/182773680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66040047d0 0x7f6618152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 >> 172.21.15.71:0/182773680 conn(0x7f66180b0f20 msgr2=0x7f66180c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 shutdown_connections 2023-10-26T05:49:40.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.033+0000 7f662019a640 1 -- 172.21.15.71:0/182773680 wait complete. 2023-10-26T05:49:40.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:49:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- 172.21.15.71:0/1700088418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 msgr2=0x7fca7c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 --2- 172.21.15.71:0/1700088418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca7c004c50 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fca800016b0 tx=0x7fca8001c750 comp rx=0 tx=0).stop 2023-10-26T05:49:40.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- 172.21.15.71:0/1700088418 shutdown_connections 2023-10-26T05:49:40.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 --2- 172.21.15.71:0/1700088418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca7c004c50 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- 172.21.15.71:0/1700088418 >> 172.21.15.71:0/1700088418 conn(0x7fca900cdd80 msgr2=0x7fca900ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- 172.21.15.71:0/1700088418 shutdown_connections 2023-10-26T05:49:40.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- 172.21.15.71:0/1700088418 wait complete. 2023-10-26T05:49:40.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 Processor -- start 2023-10-26T05:49:40.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- start start 2023-10-26T05:49:40.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca900c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9014c620 0x7fca900b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca900b1910 0x7fca901f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9006a650 con 0x7fca7c0047d0 2023-10-26T05:49:40.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90157f80 con 0x7fca9014c620 2023-10-26T05:49:40.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca98444640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9014d100 con 0x7fca900b1910 2023-10-26T05:49:40.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca900c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca900c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46786/0 (socket says 172.21.15.71:46786) 2023-10-26T05:49:40.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 -- 172.21.15.71:0/2859854845 learned_addr learned my addr 172.21.15.71:0/2859854845 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:40.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca959df640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9014c620 0x7fca900b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca969e1640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca900b1910 0x7fca901f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 -- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca900b1910 msgr2=0x7fca901f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca900b1910 0x7fca901f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 -- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9014c620 msgr2=0x7fca900b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9014c620 0x7fca900b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca961e0640 1 -- 172.21.15.71:0/2859854845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca90157690 con 0x7fca7c0047d0 2023-10-26T05:49:40.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca969e1640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca900b1910 0x7fca901f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:40.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.173+0000 7fca959df640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9014c620 0x7fca900b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:40.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca961e0640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca900c4e30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e600 tx=0x7fca8c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:40.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00c9a0 con 0x7fca7c0047d0 2023-10-26T05:49:40.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca8c00cc10 con 0x7fca7c0047d0 2023-10-26T05:49:40.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca901f9260 con 0x7fca7c0047d0 2023-10-26T05:49:40.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c011460 con 0x7fca7c0047d0 2023-10-26T05:49:40.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca901578c0 con 0x7fca7c0047d0 2023-10-26T05:49:40.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca90066b10 con 0x7fca7c0047d0 2023-10-26T05:49:40.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca8c0115c0 con 0x7fca7c0047d0 2023-10-26T05:49:40.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca86ffd640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca740664a0 0x7fca74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fca8c0846f0 con 0x7fca7c0047d0 2023-10-26T05:49:40.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca959df640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca740664a0 0x7fca74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.177+0000 7fca959df640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca740664a0 0x7fca74068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca80001680 tx=0x7fca800194f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:40.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.181+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fca8c051b30 con 0x7fca7c0047d0 2023-10-26T05:49:40.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.333+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca900721e0 con 0x7fca7c0047d0 2023-10-26T05:49:40.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.333+0000 7fca86ffd640 1 -- 172.21.15.71:0/2859854845 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca8c015020 con 0x7fca7c0047d0 2023-10-26T05:49:40.338 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:49:40.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca740664a0 msgr2=0x7fca74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca740664a0 0x7fca74068960 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca80001680 tx=0x7fca800194f0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 msgr2=0x7fca900c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca900c4e30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e600 tx=0x7fca8c00eac0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 shutdown_connections 2023-10-26T05:49:40.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca900b1910 0x7fca901f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca9014c620 0x7fca900b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca740664a0 0x7fca74068960 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 --2- 172.21.15.71:0/2859854845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca7c0047d0 0x7fca900c4e30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 >> 172.21.15.71:0/2859854845 conn(0x7fca900cdd80 msgr2=0x7fca900b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 shutdown_connections 2023-10-26T05:49:40.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.337+0000 7fca98444640 1 -- 172.21.15.71:0/2859854845 wait complete. 2023-10-26T05:49:40.356 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:49:40.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- 172.21.15.71:0/270309412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950003980 msgr2=0x7fe950003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/270309412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950003980 0x7fe950003da0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fe960009b90 tx=0x7fe96001d260 comp rx=0 tx=0).stop 2023-10-26T05:49:40.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 -- 172.21.15.71:0/270309412 reap_dead start 2023-10-26T05:49:40.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- 172.21.15.71:0/270309412 shutdown_connections 2023-10-26T05:49:40.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- 172.21.15.71:0/270309412 >> 172.21.15.71:0/270309412 conn(0x7fe9640d12d0 msgr2=0x7fe9640d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- 172.21.15.71:0/270309412 shutdown_connections 2023-10-26T05:49:40.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- 172.21.15.71:0/270309412 wait complete. 2023-10-26T05:49:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 Processor -- start 2023-10-26T05:49:40.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- start start 2023-10-26T05:49:40.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950003980 0x7fe9640b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe964156570 0x7fe96415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 0x7fe9641f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96415e3c0 con 0x7fe96415c080 2023-10-26T05:49:40.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9641f83f0 con 0x7fe950003980 2023-10-26T05:49:40.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96b7ce640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640b5270 con 0x7fe964156570 2023-10-26T05:49:40.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 0x7fe9641f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 0x7fe9641f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46806/0 (socket says 172.21.15.71:46806) 2023-10-26T05:49:40.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 -- 172.21.15.71:0/3272208440 learned_addr learned my addr 172.21.15.71:0/3272208440 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:40.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe96956a640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950003980 0x7fe9640b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe968d69640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe964156570 0x7fe96415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 -- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe964156570 msgr2=0x7fe96415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe964156570 0x7fe96415bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 -- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950003980 msgr2=0x7fe9640b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950003980 0x7fe9640b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.469+0000 7fe969d6b640 1 -- 172.21.15.71:0/3272208440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640cdb70 con 0x7fe96415c080 2023-10-26T05:49:40.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe969d6b640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 0x7fe9641f8810 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0012f0 tx=0x7fe95c001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:40.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c00aa20 con 0x7fe96415c080 2023-10-26T05:49:40.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe95c00ac90 con 0x7fe96415c080 2023-10-26T05:49:40.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9640b5480 con 0x7fe96415c080 2023-10-26T05:49:40.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007070 con 0x7fe96415c080 2023-10-26T05:49:40.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9640b5a40 con 0x7fe96415c080 2023-10-26T05:49:40.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964066b10 con 0x7fe96415c080 2023-10-26T05:49:40.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe95c0071d0 con 0x7fe96415c080 2023-10-26T05:49:40.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe959ffb640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9480664a0 0x7fe948068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe95c0839f0 con 0x7fe96415c080 2023-10-26T05:49:40.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe96956a640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9480664a0 0x7fe948068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.473+0000 7fe96956a640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9480664a0 0x7fe948068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9600ae000 tx=0x7fe950005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:40.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.477+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe95c054950 con 0x7fe96415c080 2023-10-26T05:49:40.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.617+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9641f8530 con 0x7fe96415c080 2023-10-26T05:49:40.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.617+0000 7fe959ffb640 1 -- 172.21.15.71:0/3272208440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9641f8530 con 0x7fe96415c080 2023-10-26T05:49:40.622 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:49:40.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.617+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9480664a0 msgr2=0x7fe948068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.617+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9480664a0 0x7fe948068960 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9600ae000 tx=0x7fe950005800 comp rx=0 tx=0).stop 2023-10-26T05:49:40.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.617+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 msgr2=0x7fe9641f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.617+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 0x7fe9641f8810 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0012f0 tx=0x7fe95c001a10 comp rx=0 tx=0).stop 2023-10-26T05:49:40.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 shutdown_connections 2023-10-26T05:49:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9480664a0 0x7fe948068960 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96415c080 0x7fe9641f8810 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe964156570 0x7fe96415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 --2- 172.21.15.71:0/3272208440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe950003980 0x7fe9640b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 >> 172.21.15.71:0/3272208440 conn(0x7fe9640d12d0 msgr2=0x7fe9640b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 shutdown_connections 2023-10-26T05:49:40.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.621+0000 7fe96b7ce640 1 -- 172.21.15.71:0/3272208440 wait complete. 2023-10-26T05:49:40.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:49:40.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 -- 172.21.15.71:0/3039401126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40ca7f0 msgr2=0x7f45d4159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/3039401126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40ca7f0 0x7f45d4159700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f45d406d820 tx=0x7f45cc01c7d0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 -- 172.21.15.71:0/3039401126 shutdown_connections 2023-10-26T05:49:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/3039401126 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d415ccf0 0x7f45d415d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/3039401126 >> v2:172.21.15.86:3300/0 conn(0x7f45d40cabd0 0x7f45d40b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/3039401126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40ca7f0 0x7f45d4159700 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 -- 172.21.15.71:0/3039401126 >> 172.21.15.71:0/3039401126 conn(0x7f45d406b8c0 msgr2=0x7f45d406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 -- 172.21.15.71:0/3039401126 shutdown_connections 2023-10-26T05:49:40.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 -- 172.21.15.71:0/3039401126 wait complete. 2023-10-26T05:49:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 Processor -- start 2023-10-26T05:49:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.761+0000 7f45d8ceb640 1 -- start start 2023-10-26T05:49:40.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d40ca7f0 0x7f45d4158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 0x7f45d4140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d415ccf0 0x7f45d415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4154b60 con 0x7f45d40cabd0 2023-10-26T05:49:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d415ba40 con 0x7f45d40ca7f0 2023-10-26T05:49:40.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d413a1a0 con 0x7f45d415ccf0 2023-10-26T05:49:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 0x7f45d4140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 0x7f45d4140600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46832/0 (socket says 172.21.15.71:46832) 2023-10-26T05:49:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 -- 172.21.15.71:0/1722152023 learned_addr learned my addr 172.21.15.71:0/1722152023 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:40.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d259c640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d40ca7f0 0x7f45d4158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d2d9d640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d415ccf0 0x7f45d415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 -- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d415ccf0 msgr2=0x7f45d415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d415ccf0 0x7f45d415a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 -- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d40ca7f0 msgr2=0x7f45d4158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d40ca7f0 0x7f45d4158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 -- 172.21.15.71:0/1722152023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc009560 con 0x7f45d40cabd0 2023-10-26T05:49:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d1d9b640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 0x7f45d4140600 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c9a0 tx=0x7f45bc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:40.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc007c70 con 0x7f45d40cabd0 2023-10-26T05:49:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45bc00f450 con 0x7f45d40cabd0 2023-10-26T05:49:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d415bc90 con 0x7f45d40cabd0 2023-10-26T05:49:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00f860 con 0x7f45d40cabd0 2023-10-26T05:49:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d415c020 con 0x7f45d40cabd0 2023-10-26T05:49:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d4066b10 con 0x7f45d40cabd0 2023-10-26T05:49:40.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45bc00f9c0 con 0x7f45d40cabd0 2023-10-26T05:49:40.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45caffd640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b4066470 0x7f45b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:40.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f45bc084700 con 0x7f45d40cabd0 2023-10-26T05:49:40.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.765+0000 7f45d259c640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b4066470 0x7f45b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:40.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.769+0000 7f45d259c640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b4066470 0x7f45b4068930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0af000 tx=0x7f45cc01b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:40.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.769+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f45bc051bd0 con 0x7f45d40cabd0 2023-10-26T05:49:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.933+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45d406c7d0 con 0x7f45d40cabd0 2023-10-26T05:49:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.933+0000 7f45caffd640 1 -- 172.21.15.71:0/1722152023 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f45bc0551f0 con 0x7f45d40cabd0 2023-10-26T05:49:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:49:40.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b4066470 msgr2=0x7f45b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b4066470 0x7f45b4068930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0af000 tx=0x7f45cc01b010 comp rx=0 tx=0).stop 2023-10-26T05:49:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 msgr2=0x7f45d4140600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 0x7f45d4140600 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c9a0 tx=0x7f45bc00ce60 comp rx=0 tx=0).stop 2023-10-26T05:49:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 shutdown_connections 2023-10-26T05:49:40.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45d415ccf0 0x7f45d415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45b4066470 0x7f45b4068930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d40cabd0 0x7f45d4140600 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 --2- 172.21.15.71:0/1722152023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d40ca7f0 0x7f45d4158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:40.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 >> 172.21.15.71:0/1722152023 conn(0x7f45d406b8c0 msgr2=0x7f45d4141d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:40.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 shutdown_connections 2023-10-26T05:49:40.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40.937+0000 7f45d8ceb640 1 -- 172.21.15.71:0/1722152023 wait complete. 2023-10-26T05:49:40.956 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:49:40.956 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:40.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:49:40.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:40.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[40433]: fuse finished with error 0 and tester_r 0 2023-10-26T05:49:41.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:41.014 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:49:41.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:41.015 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:49:41.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:49:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:41.051 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:49:41.051 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:41.064 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:41.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 -- 172.21.15.71:0/2859543034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c071470 msgr2=0x7f002c0b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 --2- 172.21.15.71:0/2859543034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c071470 0x7f002c0b1a70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0024009cb0 tx=0x7f002401cbd0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 -- 172.21.15.71:0/2859543034 shutdown_connections 2023-10-26T05:49:41.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 --2- 172.21.15.71:0/2859543034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c071470 0x7f002c0b1a70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 -- 172.21.15.71:0/2859543034 >> 172.21.15.71:0/2859543034 conn(0x7f002c069770 msgr2=0x7f002c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:41.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 -- 172.21.15.71:0/2859543034 shutdown_connections 2023-10-26T05:49:41.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.201+0000 7f0030855640 1 -- 172.21.15.71:0/2859543034 wait complete. 2023-10-26T05:49:41.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 Processor -- start 2023-10-26T05:49:41.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- start start 2023-10-26T05:49:41.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f002c070fe0 0x7f002c0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f002c071470 0x7f002c14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 0x7f002c14ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1599e0 con 0x7f002c140cc0 2023-10-26T05:49:41.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c159cc0 con 0x7f002c070fe0 2023-10-26T05:49:41.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c159fa0 con 0x7f002c071470 2023-10-26T05:49:41.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 0x7f002c14ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 0x7f002c14ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46854/0 (socket says 172.21.15.71:46854) 2023-10-26T05:49:41.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 -- 172.21.15.71:0/2725837267 learned_addr learned my addr 172.21.15.71:0/2725837267 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:41.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 -- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f002c071470 msgr2=0x7f002c14c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f002c071470 0x7f002c14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 -- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f002c070fe0 msgr2=0x7f002c0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f002c070fe0 0x7f002c0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 -- 172.21.15.71:0/2725837267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00240098e0 con 0x7f002c140cc0 2023-10-26T05:49:41.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f002ad9d640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 0x7f002c14ca10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f002401b280 tx=0x7f0024003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:41.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00240ba070 con 0x7f002c140cc0 2023-10-26T05:49:41.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f00240046f0 con 0x7f002c140cc0 2023-10-26T05:49:41.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c15a280 con 0x7f002c140cc0 2023-10-26T05:49:41.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024004cb0 con 0x7f002c140cc0 2023-10-26T05:49:41.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c15aa80 con 0x7f002c140cc0 2023-10-26T05:49:41.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c15aa80 con 0x7f002c140cc0 2023-10-26T05:49:41.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.205+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f00240b4040 con 0x7f002c140cc0 2023-10-26T05:49:41.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.209+0000 7f001affd640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f000c066470 0x7f000c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.209+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f002411ed00 con 0x7f002c140cc0 2023-10-26T05:49:41.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.209+0000 7f002a59c640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f000c066470 0x7f000c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.209+0000 7f002a59c640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f000c066470 0x7f000c068930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f001c009650 tx=0x7f001c006bb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:41.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.213+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f00240ec1d0 con 0x7f002c140cc0 2023-10-26T05:49:41.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.365+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f002c15aa80 con 0x7f002c140cc0 2023-10-26T05:49:41.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.369+0000 7f001affd640 1 -- 172.21.15.71:0/2725837267 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f00240b2020 con 0x7f002c140cc0 2023-10-26T05:49:41.374 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:41.375 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:32.147514+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:41.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f000c066470 msgr2=0x7f000c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f000c066470 0x7f000c068930 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f001c009650 tx=0x7f001c006bb0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 msgr2=0x7f002c14ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 0x7f002c14ca10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f002401b280 tx=0x7f0024003f90 comp rx=0 tx=0).stop 2023-10-26T05:49:41.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 shutdown_connections 2023-10-26T05:49:41.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f000c066470 0x7f000c068930 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f002c140cc0 0x7f002c14ca10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f002c071470 0x7f002c14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 --2- 172.21.15.71:0/2725837267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f002c070fe0 0x7f002c0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 >> 172.21.15.71:0/2725837267 conn(0x7f002c069770 msgr2=0x7f002c06fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:41.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 shutdown_connections 2023-10-26T05:49:41.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.373+0000 7f0030855640 1 -- 172.21.15.71:0/2725837267 wait complete. 2023-10-26T05:49:41.393 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:49:00.912702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:49:01.904497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:49:41.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:49:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 -- 172.21.15.71:0/3109101053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 msgr2=0x7fac1c070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3109101053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c070970 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fac1c06f680 tx=0x7fac1401ca80 comp rx=0 tx=0).stop 2023-10-26T05:49:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 -- 172.21.15.71:0/3109101053 shutdown_connections 2023-10-26T05:49:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3109101053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c070970 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3109101053 >> v2:172.21.15.86:3300/0 conn(0x7fac1c15b280 0x7fac1c070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 -- 172.21.15.71:0/3109101053 >> 172.21.15.71:0/3109101053 conn(0x7fac1c0aec60 msgr2=0x7fac1c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.513+0000 7fac23ff8640 1 -- 172.21.15.71:0/3109101053 shutdown_connections 2023-10-26T05:49:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- 172.21.15.71:0/3109101053 wait complete. 2023-10-26T05:49:41.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 Processor -- start 2023-10-26T05:49:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- start start 2023-10-26T05:49:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac1c0ab080 0x7fac1c14e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c15b280 0x7fac1c1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c15c700 con 0x7fac1c0b2d00 2023-10-26T05:49:41.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c15c9e0 con 0x7fac1c15b280 2023-10-26T05:49:41.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c15ccc0 con 0x7fac1c0ab080 2023-10-26T05:49:41.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c14f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46872/0 (socket says 172.21.15.71:46872) 2023-10-26T05:49:41.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 -- 172.21.15.71:0/3435425860 learned_addr learned my addr 172.21.15.71:0/3435425860 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21d94640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac1c0ab080 0x7fac1c14e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac22595640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c15b280 0x7fac1c1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 -- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac1c0ab080 msgr2=0x7fac1c14e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac1c0ab080 0x7fac1c14e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 -- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c15b280 msgr2=0x7fac1c1551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c15b280 0x7fac1c1551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 -- 172.21.15.71:0/3435425860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009820 con 0x7fac1c0b2d00 2023-10-26T05:49:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac21593640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c14f990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c450 tx=0x7fac0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c012830 con 0x7fac1c0b2d00 2023-10-26T05:49:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac0c012aa0 con 0x7fac1c0b2d00 2023-10-26T05:49:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c15cfa0 con 0x7fac1c0b2d00 2023-10-26T05:49:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0c011380 con 0x7fac1c0b2d00 2023-10-26T05:49:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c15c340 con 0x7fac1c0b2d00 2023-10-26T05:49:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.517+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c066b10 con 0x7fac1c0b2d00 2023-10-26T05:49:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.521+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac0c0114e0 con 0x7fac1c0b2d00 2023-10-26T05:49:41.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.521+0000 7fac127fc640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac00066470 0x7fac00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.521+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fac0c084000 con 0x7fac1c0b2d00 2023-10-26T05:49:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.521+0000 7fac21d94640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac00066470 0x7fac00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.521+0000 7fac21d94640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac00066470 0x7fac00068930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac180058d0 tx=0x7fac18005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.525+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac0c0514d0 con 0x7fac1c0b2d00 2023-10-26T05:49:41.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.669+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac1c0721e0 con 0x7fac1c0b2d00 2023-10-26T05:49:41.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac127fc640 1 -- 172.21.15.71:0/3435425860 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac0c054af0 con 0x7fac1c0b2d00 2023-10-26T05:49:41.676 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:49:41.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac00066470 msgr2=0x7fac00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac00066470 0x7fac00068930 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac180058d0 tx=0x7fac18005860 comp rx=0 tx=0).stop 2023-10-26T05:49:41.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 msgr2=0x7fac1c14f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c14f990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00c450 tx=0x7fac0c00c910 comp rx=0 tx=0).stop 2023-10-26T05:49:41.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 shutdown_connections 2023-10-26T05:49:41.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac1c15b280 0x7fac1c1551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac00066470 0x7fac00068930 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac1c0b2d00 0x7fac1c14f990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 --2- 172.21.15.71:0/3435425860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac1c0ab080 0x7fac1c14e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 >> 172.21.15.71:0/3435425860 conn(0x7fac1c0aec60 msgr2=0x7fac1c0ace70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:41.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 shutdown_connections 2023-10-26T05:49:41.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.673+0000 7fac23ff8640 1 -- 172.21.15.71:0/3435425860 wait complete. 2023-10-26T05:49:41.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:49:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- 172.21.15.71:0/4098923846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf300cbe00 msgr2=0x7fdf3013e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 --2- 172.21.15.71:0/4098923846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf300cbe00 0x7fdf3013e850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009900 tx=0x7fdf2001c890 comp rx=0 tx=0).stop 2023-10-26T05:49:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- 172.21.15.71:0/4098923846 shutdown_connections 2023-10-26T05:49:41.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 --2- 172.21.15.71:0/4098923846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf300cbe00 0x7fdf3013e850 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 --2- 172.21.15.71:0/4098923846 >> v2:172.21.15.86:3300/0 conn(0x7fdf3013c120 0x7fdf300cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- 172.21.15.71:0/4098923846 >> 172.21.15.71:0/4098923846 conn(0x7fdf300aec60 msgr2=0x7fdf300af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:41.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- 172.21.15.71:0/4098923846 shutdown_connections 2023-10-26T05:49:41.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- 172.21.15.71:0/4098923846 wait complete. 2023-10-26T05:49:41.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 Processor -- start 2023-10-26T05:49:41.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- start start 2023-10-26T05:49:41.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf300cbe00 0x7fdf30135830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 0x7fdf3014c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf3015a3f0 0x7fdf3013c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300ae380 con 0x7fdf3013c120 2023-10-26T05:49:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3014b9c0 con 0x7fdf300cbe00 2023-10-26T05:49:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf38250640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf300addc0 con 0x7fdf3015a3f0 2023-10-26T05:49:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf357eb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 0x7fdf3014c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf357eb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 0x7fdf3014c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47788/0 (socket says 172.21.15.71:47788) 2023-10-26T05:49:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf357eb640 1 -- 172.21.15.71:0/4290836009 learned_addr learned my addr 172.21.15.71:0/4290836009 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf367ed640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf3015a3f0 0x7fdf3013c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.805+0000 7fdf35fec640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf300cbe00 0x7fdf30135830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf357eb640 1 -- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf3015a3f0 msgr2=0x7fdf3013c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf357eb640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf3015a3f0 0x7fdf3013c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf357eb640 1 -- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf300cbe00 msgr2=0x7fdf30135830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf357eb640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf300cbe00 0x7fdf30135830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf357eb640 1 -- 172.21.15.71:0/4290836009 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20009560 con 0x7fdf3013c120 2023-10-26T05:49:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf367ed640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf3015a3f0 0x7fdf3013c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:41.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf357eb640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 0x7fdf3014c440 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009850 tx=0x7fdf20005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf20005230 con 0x7fdf3013c120 2023-10-26T05:49:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdf20005480 con 0x7fdf3013c120 2023-10-26T05:49:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf300ae0a0 con 0x7fdf3013c120 2023-10-26T05:49:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf200046d0 con 0x7fdf3013c120 2023-10-26T05:49:41.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3014bf20 con 0x7fdf3013c120 2023-10-26T05:49:41.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30066b10 con 0x7fdf3013c120 2023-10-26T05:49:41.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdf200b4070 con 0x7fdf3013c120 2023-10-26T05:49:41.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf267fc640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf14066470 0x7fdf14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:41.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdf2011df70 con 0x7fdf3013c120 2023-10-26T05:49:41.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf35fec640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf14066470 0x7fdf14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:41.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.809+0000 7fdf35fec640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf14066470 0x7fdf14068930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009b70 tx=0x7fdf2c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:41.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.813+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdf200eb3b0 con 0x7fdf3013c120 2023-10-26T05:49:41.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.953+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf300721e0 con 0x7fdf3013c120 2023-10-26T05:49:41.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.953+0000 7fdf267fc640 1 -- 172.21.15.71:0/4290836009 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf200ee9d0 con 0x7fdf3013c120 2023-10-26T05:49:41.959 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:49:41.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf14066470 msgr2=0x7fdf14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf14066470 0x7fdf14068930 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009b70 tx=0x7fdf2c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 msgr2=0x7fdf3014c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:41.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 0x7fdf3014c440 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009850 tx=0x7fdf20005e70 comp rx=0 tx=0).stop 2023-10-26T05:49:41.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 shutdown_connections 2023-10-26T05:49:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdf3015a3f0 0x7fdf3013c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdf14066470 0x7fdf14068930 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf3013c120 0x7fdf3014c440 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 --2- 172.21.15.71:0/4290836009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf300cbe00 0x7fdf30135830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:41.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 >> 172.21.15.71:0/4290836009 conn(0x7fdf300aec60 msgr2=0x7fdf300cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:41.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 shutdown_connections 2023-10-26T05:49:41.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:41.957+0000 7fdf38250640 1 -- 172.21.15.71:0/4290836009 wait complete. 2023-10-26T05:49:41.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:49:42.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 -- 172.21.15.71:0/1942562457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e806d770 msgr2=0x7fa8e80b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 --2- 172.21.15.71:0/1942562457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e806d770 0x7fa8e80b1c70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009840 tx=0x7fa8e401c770 comp rx=0 tx=0).stop 2023-10-26T05:49:42.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 -- 172.21.15.71:0/1942562457 shutdown_connections 2023-10-26T05:49:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 --2- 172.21.15.71:0/1942562457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8e815fc90 0x7fa8e815f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 --2- 172.21.15.71:0/1942562457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e806d770 0x7fa8e80b1c70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 -- 172.21.15.71:0/1942562457 >> 172.21.15.71:0/1942562457 conn(0x7fa8e80ce270 msgr2=0x7fa8e80ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 -- 172.21.15.71:0/1942562457 shutdown_connections 2023-10-26T05:49:42.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 -- 172.21.15.71:0/1942562457 wait complete. 2023-10-26T05:49:42.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 Processor -- start 2023-10-26T05:49:42.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.105+0000 7fa8f015b640 1 -- start start 2023-10-26T05:49:42.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8e806d770 0x7fa8e806ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 0x7fa8e8150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e815fc90 0x7fa8e8150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e8156020 con 0x7fa8e815f610 2023-10-26T05:49:42.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e806a000 con 0x7fa8e815fc90 2023-10-26T05:49:42.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815a530 con 0x7fa8e806d770 2023-10-26T05:49:42.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 0x7fa8e8150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 0x7fa8e8150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47796/0 (socket says 172.21.15.71:47796) 2023-10-26T05:49:42.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 -- 172.21.15.71:0/4066908487 learned_addr learned my addr 172.21.15.71:0/4066908487 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:42.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8edef7640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8e806d770 0x7fa8e806ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ee6f8640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e815fc90 0x7fa8e8150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 -- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8e806d770 msgr2=0x7fa8e806ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8e806d770 0x7fa8e806ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 -- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e815fc90 msgr2=0x7fa8e8150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e815fc90 0x7fa8e8150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 -- 172.21.15.71:0/4066908487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009560 con 0x7fa8e815f610 2023-10-26T05:49:42.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ee6f8640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e815fc90 0x7fa8e8150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:42.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8ed6f6640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 0x7fa8e8150510 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800e9e0 tx=0x7fa8d800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:42.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d800cdb0 con 0x7fa8e815f610 2023-10-26T05:49:42.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8d8004020 con 0x7fa8e815f610 2023-10-26T05:49:42.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8004430 con 0x7fa8e815f610 2023-10-26T05:49:42.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e40094a0 con 0x7fa8e815f610 2023-10-26T05:49:42.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e806a210 con 0x7fa8e815f610 2023-10-26T05:49:42.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e8066d00 con 0x7fa8e815f610 2023-10-26T05:49:42.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa8d8003600 con 0x7fa8e815f610 2023-10-26T05:49:42.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8de7fc640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa8cc066470 0x7fa8cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa8d8014070 con 0x7fa8e815f610 2023-10-26T05:49:42.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.109+0000 7fa8edef7640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa8cc066470 0x7fa8cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.113+0000 7fa8edef7640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa8cc066470 0x7fa8cc068930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40af000 tx=0x7fa8e401afb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:42.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.113+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa8d8051f50 con 0x7fa8e815f610 2023-10-26T05:49:42.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.265+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8e80cf0a0 con 0x7fa8e815f610 2023-10-26T05:49:42.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8de7fc640 1 -- 172.21.15.71:0/4066908487 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8d8055570 con 0x7fa8e815f610 2023-10-26T05:49:42.272 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:49:42.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa8cc066470 msgr2=0x7fa8cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa8cc066470 0x7fa8cc068930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40af000 tx=0x7fa8e401afb0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 msgr2=0x7fa8e8150510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 0x7fa8e8150510 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa8d800e9e0 tx=0x7fa8d800eea0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 shutdown_connections 2023-10-26T05:49:42.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e815fc90 0x7fa8e8150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa8cc066470 0x7fa8cc068930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8e815f610 0x7fa8e8150510 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 --2- 172.21.15.71:0/4066908487 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8e806d770 0x7fa8e806ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 >> 172.21.15.71:0/4066908487 conn(0x7fa8e80ce270 msgr2=0x7fa8e80adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:42.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 shutdown_connections 2023-10-26T05:49:42.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.269+0000 7fa8f015b640 1 -- 172.21.15.71:0/4066908487 wait complete. 2023-10-26T05:49:42.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:49:42.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- 172.21.15.71:0/1490494441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 msgr2=0x7fc8940b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1490494441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc8940b43e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc884009900 tx=0x7fc88401f070 comp rx=0 tx=0).stop 2023-10-26T05:49:42.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- 172.21.15.71:0/1490494441 shutdown_connections 2023-10-26T05:49:42.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1490494441 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89406a390 0x7fc89406a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1490494441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc8940b43e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- 172.21.15.71:0/1490494441 >> 172.21.15.71:0/1490494441 conn(0x7fc8940d12d0 msgr2=0x7fc8940d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:42.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- 172.21.15.71:0/1490494441 shutdown_connections 2023-10-26T05:49:42.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- 172.21.15.71:0/1490494441 wait complete. 2023-10-26T05:49:42.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 Processor -- start 2023-10-26T05:49:42.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- start start 2023-10-26T05:49:42.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89406a390 0x7fc894159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940cc7c0 0x7fc89415d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc89415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940aa710 con 0x7fc89415b3b0 2023-10-26T05:49:42.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940c9590 con 0x7fc8940cc7c0 2023-10-26T05:49:42.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.409+0000 7fc899ed8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940c9870 con 0x7fc89406a390 2023-10-26T05:49:42.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc89415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc89415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47810/0 (socket says 172.21.15.71:47810) 2023-10-26T05:49:42.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 -- 172.21.15.71:0/1644237445 learned_addr learned my addr 172.21.15.71:0/1644237445 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:42.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc8937fe640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89406a390 0x7fc894159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc892ffd640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940cc7c0 0x7fc89415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 -- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89406a390 msgr2=0x7fc894159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89406a390 0x7fc894159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 -- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940cc7c0 msgr2=0x7fc89415d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940cc7c0 0x7fc89415d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 -- 172.21.15.71:0/1644237445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884009560 con 0x7fc89415b3b0 2023-10-26T05:49:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc892ffd640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940cc7c0 0x7fc89415d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc893fff640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc89415d690 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c780 tx=0x7fc88c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c004b70 con 0x7fc89415b3b0 2023-10-26T05:49:42.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc88c00ce90 con 0x7fc89415b3b0 2023-10-26T05:49:42.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940c9b50 con 0x7fc89415b3b0 2023-10-26T05:49:42.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00aab0 con 0x7fc89415b3b0 2023-10-26T05:49:42.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc894159d90 con 0x7fc89415b3b0 2023-10-26T05:49:42.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc894066b10 con 0x7fc89415b3b0 2023-10-26T05:49:42.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc88c00ac10 con 0x7fc89415b3b0 2023-10-26T05:49:42.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc87bfff640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc880066470 0x7fc880068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc88c0843c0 con 0x7fc89415b3b0 2023-10-26T05:49:42.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc8937fe640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc880066470 0x7fc880068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.413+0000 7fc8937fe640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc880066470 0x7fc880068930 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc8840b2000 tx=0x7fc88401d8b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:42.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.417+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc88c051750 con 0x7fc89415b3b0 2023-10-26T05:49:42.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8940ca070 con 0x7fc89415b3b0 2023-10-26T05:49:42.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc87bfff640 1 -- 172.21.15.71:0/1644237445 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc8940ca070 con 0x7fc89415b3b0 2023-10-26T05:49:42.577 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:49:42.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc880066470 msgr2=0x7fc880068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc880066470 0x7fc880068930 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc8840b2000 tx=0x7fc88401d8b0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 msgr2=0x7fc89415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc89415d690 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00c780 tx=0x7fc88c00cc40 comp rx=0 tx=0).stop 2023-10-26T05:49:42.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 shutdown_connections 2023-10-26T05:49:42.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc880066470 0x7fc880068930 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89415b3b0 0x7fc89415d690 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8940cc7c0 0x7fc89415d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.573+0000 7fc899ed8640 1 --2- 172.21.15.71:0/1644237445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89406a390 0x7fc894159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.577+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 >> 172.21.15.71:0/1644237445 conn(0x7fc8940d12d0 msgr2=0x7fc8940b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:42.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.577+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 shutdown_connections 2023-10-26T05:49:42.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.577+0000 7fc899ed8640 1 -- 172.21.15.71:0/1644237445 wait complete. 2023-10-26T05:49:42.596 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:49:42.596 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:49:42.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:49:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:49:42.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34206]: fuse finished with error 0 and tester_r 0 2023-10-26T05:49:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:49:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:42.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:49:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:49:42.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:49:42.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:42.660 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:49:42.667 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:42.668 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:42.674 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:49:42.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:49:42.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:42.675 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:49:42.719 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:49:42.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:49:42.721 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:49:42.722 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:42.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:49:42.728 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:49:42.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:49:42.729 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:49:42.774 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:49:42.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:49:42.776 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:42.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- 172.21.15.71:0/3999387629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941588f0 msgr2=0x7fe894158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- 172.21.15.71:0/3999387629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941588f0 0x7fe894158cd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe894066310 tx=0x7fe88401c520 comp rx=0 tx=0).stop 2023-10-26T05:49:42.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- 172.21.15.71:0/3999387629 shutdown_connections 2023-10-26T05:49:42.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- 172.21.15.71:0/3999387629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8940b21b0 0x7fe89415be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- 172.21.15.71:0/3999387629 >> v2:172.21.15.86:3300/0 conn(0x7fe894158500 0x7fe8940b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- 172.21.15.71:0/3999387629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8941588f0 0x7fe894158cd0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- 172.21.15.71:0/3999387629 >> 172.21.15.71:0/3999387629 conn(0x7fe8940ce270 msgr2=0x7fe8940ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- 172.21.15.71:0/3999387629 shutdown_connections 2023-10-26T05:49:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- 172.21.15.71:0/3999387629 wait complete. 2023-10-26T05:49:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 Processor -- start 2023-10-26T05:49:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- start start 2023-10-26T05:49:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 0x7fe894149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8941588f0 0x7fe894149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894151640 0x7fe89414adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894149b50 con 0x7fe8940b21b0 2023-10-26T05:49:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe894149d80 con 0x7fe8941588f0 2023-10-26T05:49:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe8989ff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89414a060 con 0x7fe894151640 2023-10-26T05:49:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 0x7fe894149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 0x7fe894149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47832/0 (socket says 172.21.15.71:47832) 2023-10-26T05:49:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 -- 172.21.15.71:0/4232590897 learned_addr learned my addr 172.21.15.71:0/4232590897 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe892d9d640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894151640 0x7fe89414adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe891d9b640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8941588f0 0x7fe894149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 -- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894151640 msgr2=0x7fe89414adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894151640 0x7fe89414adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 -- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8941588f0 msgr2=0x7fe894149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8941588f0 0x7fe894149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 -- 172.21.15.71:0/4232590897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009560 con 0x7fe8940b21b0 2023-10-26T05:49:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe89259c640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 0x7fe894149050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe88401abd0 tx=0x7fe8840a85f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8840042c0 con 0x7fe8940b21b0 2023-10-26T05:49:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe884004510 con 0x7fe8940b21b0 2023-10-26T05:49:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.901+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884005290 con 0x7fe8940b21b0 2023-10-26T05:49:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89414a340 con 0x7fe8940b21b0 2023-10-26T05:49:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89414b2f0 con 0x7fe8940b21b0 2023-10-26T05:49:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe894066d00 con 0x7fe8940b21b0 2023-10-26T05:49:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe8840a6040 con 0x7fe8940b21b0 2023-10-26T05:49:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe882ffd640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe874066470 0x7fe874068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe88411e4e0 con 0x7fe8940b21b0 2023-10-26T05:49:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe891d9b640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe874066470 0x7fe874068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.905+0000 7fe891d9b640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe874066470 0x7fe874068930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe888005fd0 tx=0x7fe88800a4e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:42.909+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe8840eb9f0 con 0x7fe8940b21b0 2023-10-26T05:49:43.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.097+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8940ad5d0 con 0x7fe8940b21b0 2023-10-26T05:49:43.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.097+0000 7fe882ffd640 1 -- 172.21.15.71:0/4232590897 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fe8840053f0 con 0x7fe8940b21b0 2023-10-26T05:49:43.102 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:43.102 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:43.103 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:43.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe874066470 msgr2=0x7fe874068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe874066470 0x7fe874068930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe888005fd0 tx=0x7fe88800a4e0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 msgr2=0x7fe894149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 0x7fe894149050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe88401abd0 tx=0x7fe8840a85f0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 shutdown_connections 2023-10-26T05:49:43.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe894151640 0x7fe89414adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8941588f0 0x7fe894149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe874066470 0x7fe874068930 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 --2- 172.21.15.71:0/4232590897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8940b21b0 0x7fe894149050 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 >> 172.21.15.71:0/4232590897 conn(0x7fe8940ce270 msgr2=0x7fe89406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 shutdown_connections 2023-10-26T05:49:43.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.101+0000 7fe8989ff640 1 -- 172.21.15.71:0/4232590897 wait complete. 2023-10-26T05:49:43.122 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:43.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/824283201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 msgr2=0x7fd9b000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/824283201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9b000b990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00c5c0 tx=0x7fd9bc01d310 comp rx=0 tx=0).stop 2023-10-26T05:49:43.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9ca4f8640 1 -- 172.21.15.71:0/824283201 reap_dead start 2023-10-26T05:49:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/824283201 shutdown_connections 2023-10-26T05:49:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/824283201 >> 172.21.15.71:0/824283201 conn(0x7fd9c40cdd80 msgr2=0x7fd9c40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/824283201 shutdown_connections 2023-10-26T05:49:43.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/824283201 wait complete. 2023-10-26T05:49:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 Processor -- start 2023-10-26T05:49:43.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- start start 2023-10-26T05:49:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9c41f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9c4165fc0 0x7fd9c40b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4164b00 0x7fd9c4164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c406a650 con 0x7fd9b0005260 2023-10-26T05:49:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c4163ad0 con 0x7fd9c4164b00 2023-10-26T05:49:43.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c41f9320 con 0x7fd9c4165fc0 2023-10-26T05:49:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9c41f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9c41f8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47852/0 (socket says 172.21.15.71:47852) 2023-10-26T05:49:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 -- 172.21.15.71:0/3970029713 learned_addr learned my addr 172.21.15.71:0/3970029713 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9ca4f8640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4164b00 0x7fd9c4164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c94f6640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9c4165fc0 0x7fd9c40b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 -- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9c4165fc0 msgr2=0x7fd9c40b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9c4165fc0 0x7fd9c40b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 -- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4164b00 msgr2=0x7fd9c4164f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4164b00 0x7fd9c4164f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 -- 172.21.15.71:0/3970029713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4166480 con 0x7fd9b0005260 2023-10-26T05:49:43.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9c9cf7640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9c41f8de0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e640 tx=0x7fd9c000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:43.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000c9b0 con 0x7fd9b0005260 2023-10-26T05:49:43.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9c000cc00 con 0x7fd9b0005260 2023-10-26T05:49:43.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c4163d40 con 0x7fd9b0005260 2023-10-26T05:49:43.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0011470 con 0x7fd9b0005260 2023-10-26T05:49:43.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.237+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c4164300 con 0x7fd9b0005260 2023-10-26T05:49:43.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.241+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c4066b10 con 0x7fd9b0005260 2023-10-26T05:49:43.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.241+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9c00115d0 con 0x7fd9b0005260 2023-10-26T05:49:43.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.241+0000 7fd9ba7fc640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9a8066470 0x7fd9a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.241+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd9c00848e0 con 0x7fd9b0005260 2023-10-26T05:49:43.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.241+0000 7fd9c94f6640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9a8066470 0x7fd9a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.241+0000 7fd9c94f6640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9a8066470 0x7fd9a8068930 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4166770 tx=0x7fd9b4006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:43.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.245+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9c0051db0 con 0x7fd9b0005260 2023-10-26T05:49:43.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.393+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9c40721e0 con 0x7fd9b0005260 2023-10-26T05:49:43.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9ba7fc640 1 -- 172.21.15.71:0/3970029713 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fd9c00553d0 con 0x7fd9b0005260 2023-10-26T05:49:43.402 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:43.402 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:32.147514+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9a8066470 msgr2=0x7fd9a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9a8066470 0x7fd9a8068930 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4166770 tx=0x7fd9b4006af0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 msgr2=0x7fd9c41f8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9c41f8de0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e640 tx=0x7fd9c000eb00 comp rx=0 tx=0).stop 2023-10-26T05:49:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 shutdown_connections 2023-10-26T05:49:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4164b00 0x7fd9c4164f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9c4165fc0 0x7fd9c40b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd9a8066470 0x7fd9a8068930 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 --2- 172.21.15.71:0/3970029713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b0005260 0x7fd9c41f8de0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 >> 172.21.15.71:0/3970029713 conn(0x7fd9c40cdd80 msgr2=0x7fd9c40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 shutdown_connections 2023-10-26T05:49:43.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.397+0000 7fd9cbf5b640 1 -- 172.21.15.71:0/3970029713 wait complete. 2023-10-26T05:49:43.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:49:00.912702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:49:01.904497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:49:43.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:49:43.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/1002548033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53e4003980 msgr2=0x7f53e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/1002548033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53e4003980 0x7f53e4003da0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f53e8006f90 tx=0x7f53e801cf30 comp rx=0 tx=0).stop 2023-10-26T05:49:43.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/1002548033 shutdown_connections 2023-10-26T05:49:43.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/1002548033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53e4003980 0x7f53e4003da0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/1002548033 >> 172.21.15.71:0/1002548033 conn(0x7f53f80d12d0 msgr2=0x7f53f80d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/1002548033 shutdown_connections 2023-10-26T05:49:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/1002548033 wait complete. 2023-10-26T05:49:43.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 Processor -- start 2023-10-26T05:49:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- start start 2023-10-26T05:49:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e4003980 0x7f53f81e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 0x7f53f81f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53f81f87a0 0x7f53f81f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f81e9870 con 0x7f53f81437e0 2023-10-26T05:49:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f81ea070 con 0x7f53e4003980 2023-10-26T05:49:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f8143c60 con 0x7f53f81f87a0 2023-10-26T05:49:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 0x7f53f81f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 0x7f53f81f8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47862/0 (socket says 172.21.15.71:47862) 2023-10-26T05:49:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 -- 172.21.15.71:0/3726649674 learned_addr learned my addr 172.21.15.71:0/3726649674 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f77fe640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e4003980 0x7f53f81e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f7fff640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53f81f87a0 0x7f53f81f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 -- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53f81f87a0 msgr2=0x7f53f81f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53f81f87a0 0x7f53f81f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 -- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e4003980 msgr2=0x7f53f81e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e4003980 0x7f53f81e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 -- 172.21.15.71:0/3726649674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f81e9ab0 con 0x7f53f81437e0 2023-10-26T05:49:43.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f7fff640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53f81f87a0 0x7f53f81f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:43.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f77fe640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e4003980 0x7f53f81e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:43.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53eeffd640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 0x7f53f81f8260 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c4d0 tx=0x7f53e000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:43.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00052d0 con 0x7f53f81437e0 2023-10-26T05:49:43.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f53e0005450 con 0x7f53f81437e0 2023-10-26T05:49:43.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f81e9d90 con 0x7f53f81437e0 2023-10-26T05:49:43.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0012890 con 0x7f53f81437e0 2023-10-26T05:49:43.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.537+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f8144020 con 0x7f53f81437e0 2023-10-26T05:49:43.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.541+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f8066b10 con 0x7f53f81437e0 2023-10-26T05:49:43.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.541+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f53e0003600 con 0x7f53f81437e0 2023-10-26T05:49:43.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.541+0000 7f53f4ff9640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f53d8066470 0x7f53d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.541+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f53e0083ba0 con 0x7f53f81437e0 2023-10-26T05:49:43.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.541+0000 7f53f77fe640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f53d8066470 0x7f53d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.541+0000 7f53f77fe640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f53d8066470 0x7f53d8068930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f53e80ac000 tx=0x7f53e8019cd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:43.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.545+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f53e0051070 con 0x7f53f81437e0 2023-10-26T05:49:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.689+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53f80721e0 con 0x7f53f81437e0 2023-10-26T05:49:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53f4ff9640 1 -- 172.21.15.71:0/3726649674 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53e0054690 con 0x7f53f81437e0 2023-10-26T05:49:43.696 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:49:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f53d8066470 msgr2=0x7f53d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f53d8066470 0x7f53d8068930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f53e80ac000 tx=0x7f53e8019cd0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 msgr2=0x7f53f81f8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 0x7f53f81f8260 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c4d0 tx=0x7f53e000c990 comp rx=0 tx=0).stop 2023-10-26T05:49:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 shutdown_connections 2023-10-26T05:49:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53f81f87a0 0x7f53f81f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f53d8066470 0x7f53d8068930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53f81437e0 0x7f53f81f8260 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 --2- 172.21.15.71:0/3726649674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53e4003980 0x7f53f81e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 >> 172.21.15.71:0/3726649674 conn(0x7f53f80d12d0 msgr2=0x7f53f8151900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 shutdown_connections 2023-10-26T05:49:43.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.693+0000 7f53fdb4d640 1 -- 172.21.15.71:0/3726649674 wait complete. 2023-10-26T05:49:43.713 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:49:43.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.833+0000 7f7465e76640 1 -- 172.21.15.71:0/4152198655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 msgr2=0x7f74600b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.833+0000 7f7465e76640 1 --2- 172.21.15.71:0/4152198655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f74600b03b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7460070fc0 tx=0x7f745801dc50 comp rx=0 tx=0).stop 2023-10-26T05:49:43.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/4152198655 shutdown_connections 2023-10-26T05:49:43.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 --2- 172.21.15.71:0/4152198655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f74600b03b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 --2- 172.21.15.71:0/4152198655 >> v2:172.21.15.86:3300/0 conn(0x7f7460158770 0x7f746006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/4152198655 >> 172.21.15.71:0/4152198655 conn(0x7f74600aec60 msgr2=0x7f74600af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/4152198655 shutdown_connections 2023-10-26T05:49:43.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/4152198655 wait complete. 2023-10-26T05:49:43.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 Processor -- start 2023-10-26T05:49:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- start start 2023-10-26T05:49:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f7460149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74600cc7c0 0x7f7460146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7460158770 0x7f746014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460147f60 con 0x7f746006a540 2023-10-26T05:49:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460147960 con 0x7f74600cc7c0 2023-10-26T05:49:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74600cb6b0 con 0x7f7460158770 2023-10-26T05:49:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f7460149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f7460149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47874/0 (socket says 172.21.15.71:47874) 2023-10-26T05:49:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 -- 172.21.15.71:0/991696670 learned_addr learned my addr 172.21.15.71:0/991696670 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 -- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7460158770 msgr2=0x7f746014a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7460158770 0x7f746014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 -- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74600cc7c0 msgr2=0x7f7460146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74600cc7c0 0x7f7460146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 -- 172.21.15.71:0/991696670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745800a9f0 con 0x7f746006a540 2023-10-26T05:49:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f745f7fe640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f7460149db0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f745000c4d0 tx=0x7f745000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450005480 con 0x7f746006a540 2023-10-26T05:49:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f74500036f0 con 0x7f746006a540 2023-10-26T05:49:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74600cb990 con 0x7f746006a540 2023-10-26T05:49:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450003b00 con 0x7f746006a540 2023-10-26T05:49:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7460147b60 con 0x7f746006a540 2023-10-26T05:49:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.837+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7460066b10 con 0x7f746006a540 2023-10-26T05:49:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.841+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7450003c60 con 0x7f746006a540 2023-10-26T05:49:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.841+0000 7f7447fff640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f744c0664a0 0x7f744c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.841+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7450083670 con 0x7f746006a540 2023-10-26T05:49:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.841+0000 7f745effd640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f744c0664a0 0x7f744c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.841+0000 7f745effd640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f744c0664a0 0x7f744c068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f74600cbf70 tx=0x7f7454005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.845+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7450050c00 con 0x7f746006a540 2023-10-26T05:49:43.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.985+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74600b0e20 con 0x7f746006a540 2023-10-26T05:49:43.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.985+0000 7f7447fff640 1 -- 172.21.15.71:0/991696670 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7450017020 con 0x7f746006a540 2023-10-26T05:49:43.992 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:49:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f744c0664a0 msgr2=0x7f744c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f744c0664a0 0x7f744c068960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f74600cbf70 tx=0x7f7454005b60 comp rx=0 tx=0).stop 2023-10-26T05:49:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 msgr2=0x7f7460149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f7460149db0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f745000c4d0 tx=0x7f745000c990 comp rx=0 tx=0).stop 2023-10-26T05:49:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 shutdown_connections 2023-10-26T05:49:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7460158770 0x7f746014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74600cc7c0 0x7f7460146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f744c0664a0 0x7f744c068960 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 --2- 172.21.15.71:0/991696670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f746006a540 0x7f7460149db0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:43.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 >> 172.21.15.71:0/991696670 conn(0x7f74600aec60 msgr2=0x7f746014f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:43.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 shutdown_connections 2023-10-26T05:49:43.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:43.989+0000 7f7465e76640 1 -- 172.21.15.71:0/991696670 wait complete. 2023-10-26T05:49:44.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:49:44.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/307203152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194070c60 msgr2=0x7f41940ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 --2- 172.21.15.71:0/307203152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194070c60 0x7f41940ca350 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f418c009cb0 tx=0x7f418c01ccc0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/307203152 shutdown_connections 2023-10-26T05:49:44.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 --2- 172.21.15.71:0/307203152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194070c60 0x7f41940ca350 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 --2- 172.21.15.71:0/307203152 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4194141370 0x7f4194158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/307203152 >> 172.21.15.71:0/307203152 conn(0x7f41940afe40 msgr2=0x7f41940b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:44.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/307203152 shutdown_connections 2023-10-26T05:49:44.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/307203152 wait complete. 2023-10-26T05:49:44.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 Processor -- start 2023-10-26T05:49:44.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- start start 2023-10-26T05:49:44.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4194070c60 0x7f4194140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 0x7f4194140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4194141370 0x7f419406ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419406c3e0 con 0x7f41940b0f70 2023-10-26T05:49:44.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4194156df0 con 0x7f4194141370 2023-10-26T05:49:44.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419406b760 con 0x7f4194070c60 2023-10-26T05:49:44.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 0x7f4194140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 0x7f4194140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47900/0 (socket says 172.21.15.71:47900) 2023-10-26T05:49:44.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 -- 172.21.15.71:0/2293115809 learned_addr learned my addr 172.21.15.71:0/2293115809 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:44.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f41937fe640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4194070c60 0x7f4194140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4193fff640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4194141370 0x7f419406ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 -- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4194070c60 msgr2=0x7f4194140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4194070c60 0x7f4194140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 -- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4194141370 msgr2=0x7f419406ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4194141370 0x7f419406ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 -- 172.21.15.71:0/2293115809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0098e0 con 0x7f41940b0f70 2023-10-26T05:49:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f418affd640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 0x7f4194140590 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c500 tx=0x7f417c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c005220 con 0x7f41940b0f70 2023-10-26T05:49:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f417c005470 con 0x7f41940b0f70 2023-10-26T05:49:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419406ba40 con 0x7f41940b0f70 2023-10-26T05:49:44.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c007290 con 0x7f41940b0f70 2023-10-26T05:49:44.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.133+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419406c000 con 0x7f41940b0f70 2023-10-26T05:49:44.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.137+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419406c520 con 0x7f41940b0f70 2023-10-26T05:49:44.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.137+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f417c0073f0 con 0x7f41940b0f70 2023-10-26T05:49:44.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.137+0000 7f4190ff9640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4174066470 0x7f4174068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.137+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f417c083af0 con 0x7f41940b0f70 2023-10-26T05:49:44.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.137+0000 7f41937fe640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4174066470 0x7f4174068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.137+0000 7f41937fe640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4174066470 0x7f4174068930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4184009ad0 tx=0x7f41840091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:44.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.141+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f419406c520 con 0x7f41940b0f70 2023-10-26T05:49:44.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.281+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f419406c520 con 0x7f41940b0f70 2023-10-26T05:49:44.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.281+0000 7f4190ff9640 1 -- 172.21.15.71:0/2293115809 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f419406c520 con 0x7f41940b0f70 2023-10-26T05:49:44.294 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:49:44.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.281+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4174066470 msgr2=0x7f4174068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.281+0000 7f4199bd6640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4174066470 0x7f4174068930 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4184009ad0 tx=0x7f41840091e0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.281+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 msgr2=0x7f4194140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.281+0000 7f4199bd6640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 0x7f4194140590 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c500 tx=0x7f417c00c9c0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 shutdown_connections 2023-10-26T05:49:44.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4194141370 0x7f419406ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4174066470 0x7f4174068930 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41940b0f70 0x7f4194140590 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 --2- 172.21.15.71:0/2293115809 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4194070c60 0x7f4194140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 >> 172.21.15.71:0/2293115809 conn(0x7f41940afe40 msgr2=0x7f4194070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:44.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 shutdown_connections 2023-10-26T05:49:44.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.285+0000 7f4199bd6640 1 -- 172.21.15.71:0/2293115809 wait complete. 2023-10-26T05:49:44.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/1868333668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 msgr2=0x7fec30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 --2- 172.21.15.71:0/1868333668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec30003dd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fec38001ec0 tx=0x7fec3801ceb0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/1868333668 shutdown_connections 2023-10-26T05:49:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 --2- 172.21.15.71:0/1868333668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec30003dd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/1868333668 >> 172.21.15.71:0/1868333668 conn(0x7fec48069770 msgr2=0x7fec48069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/1868333668 shutdown_connections 2023-10-26T05:49:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/1868333668 wait complete. 2023-10-26T05:49:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 Processor -- start 2023-10-26T05:49:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- start start 2023-10-26T05:49:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec481576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec481587d0 0x7fec481f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec481f4420 0x7fec4815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec481380b0 con 0x7fec300039b0 2023-10-26T05:49:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec481f4930 con 0x7fec481f4420 2023-10-26T05:49:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec481f4c10 con 0x7fec481587d0 2023-10-26T05:49:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec481576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec481576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47912/0 (socket says 172.21.15.71:47912) 2023-10-26T05:49:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 -- 172.21.15.71:0/4053871429 learned_addr learned my addr 172.21.15.71:0/4053871429 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec477fe640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec481f4420 0x7fec4815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec467fc640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec481587d0 0x7fec481f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 -- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec481587d0 msgr2=0x7fec481f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec481587d0 0x7fec481f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 -- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec481f4420 msgr2=0x7fec4815ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec481f4420 0x7fec4815ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 -- 172.21.15.71:0/4053871429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec48137cd0 con 0x7fec300039b0 2023-10-26T05:49:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec46ffd640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec481576c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fec3801c920 tx=0x7fec380060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380b3070 con 0x7fec300039b0 2023-10-26T05:49:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fec38001540 con 0x7fec300039b0 2023-10-26T05:49:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec481f0b80 con 0x7fec300039b0 2023-10-26T05:49:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38003600 con 0x7fec300039b0 2023-10-26T05:49:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.421+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec48158c50 con 0x7fec300039b0 2023-10-26T05:49:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.425+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec48066b10 con 0x7fec300039b0 2023-10-26T05:49:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.425+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fec380ab020 con 0x7fec300039b0 2023-10-26T05:49:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.425+0000 7fec2f7fe640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec340664a0 0x7fec34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.425+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fec3811e0d0 con 0x7fec300039b0 2023-10-26T05:49:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.425+0000 7fec467fc640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec340664a0 0x7fec34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.425+0000 7fec467fc640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec340664a0 0x7fec34068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000f70 tx=0x7fec3c000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.429+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fec380eb480 con 0x7fec300039b0 2023-10-26T05:49:44.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec480721e0 con 0x7fec300039b0 2023-10-26T05:49:44.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec2f7fe640 1 -- 172.21.15.71:0/4053871429 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fec380eeaa0 con 0x7fec300039b0 2023-10-26T05:49:44.621 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:44.622 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:44.622 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:44.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec340664a0 msgr2=0x7fec34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec340664a0 0x7fec34068960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fec3c000f70 tx=0x7fec3c000ee0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 msgr2=0x7fec481576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec481576c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fec3801c920 tx=0x7fec380060d0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 shutdown_connections 2023-10-26T05:49:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec481f4420 0x7fec4815ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fec481587d0 0x7fec481f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fec340664a0 0x7fec34068960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 --2- 172.21.15.71:0/4053871429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec300039b0 0x7fec481576c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 >> 172.21.15.71:0/4053871429 conn(0x7fec48069770 msgr2=0x7fec480ad000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:44.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 shutdown_connections 2023-10-26T05:49:44.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.617+0000 7fec4d465640 1 -- 172.21.15.71:0/4053871429 wait complete. 2023-10-26T05:49:44.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:44.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/223284157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 msgr2=0x7f680415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 --2- 172.21.15.71:0/223284157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f680415bab0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009bc0 tx=0x7f67fc01c910 comp rx=0 tx=0).stop 2023-10-26T05:49:44.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/223284157 shutdown_connections 2023-10-26T05:49:44.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 --2- 172.21.15.71:0/223284157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f680415bab0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/223284157 >> 172.21.15.71:0/223284157 conn(0x7f68040cdd80 msgr2=0x7f68040ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:44.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/223284157 shutdown_connections 2023-10-26T05:49:44.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/223284157 wait complete. 2023-10-26T05:49:44.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 Processor -- start 2023-10-26T05:49:44.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- start start 2023-10-26T05:49:44.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f6804156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68040d0470 0x7f6804155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68040d0850 0x7f68040c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680406a650 con 0x7f68040cd880 2023-10-26T05:49:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68041562c0 con 0x7f68040d0470 2023-10-26T05:49:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68040c5170 con 0x7f68040d0850 2023-10-26T05:49:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f6804156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f6804156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47936/0 (socket says 172.21.15.71:47936) 2023-10-26T05:49:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 -- 172.21.15.71:0/1889400661 learned_addr learned my addr 172.21.15.71:0/1889400661 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:44.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f6803fff640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68040d0850 0x7f68040c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f67fbfff640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68040d0470 0x7f6804155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 -- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68040d0850 msgr2=0x7f68040c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68040d0850 0x7f68040c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 -- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68040d0470 msgr2=0x7f6804155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68040d0470 0x7f6804155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 -- 172.21.15.71:0/1889400661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc009820 con 0x7f68040cd880 2023-10-26T05:49:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f67fbfff640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68040d0470 0x7f6804155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f68037fe640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f6804156450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c450 tx=0x7f67f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4012830 con 0x7f68040cd880 2023-10-26T05:49:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f67f4012aa0 con 0x7f68040cd880 2023-10-26T05:49:44.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68040c5450 con 0x7f68040cd880 2023-10-26T05:49:44.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4011380 con 0x7f68040cd880 2023-10-26T05:49:44.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.757+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68040c5a10 con 0x7f68040cd880 2023-10-26T05:49:44.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.761+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6804066b10 con 0x7f68040cd880 2023-10-26T05:49:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.761+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67f4007020 con 0x7f68040cd880 2023-10-26T05:49:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.761+0000 7f6800ff9640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67e4066470 0x7f67e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.761+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f67f4084650 con 0x7f68040cd880 2023-10-26T05:49:44.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.761+0000 7f67fbfff640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67e4066470 0x7f67e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.761+0000 7f67fbfff640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67e4066470 0x7f67e4068930 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68041555c0 tx=0x7f67ec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:44.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.765+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f67f4051c00 con 0x7f68040cd880 2023-10-26T05:49:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.905+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68040721e0 con 0x7f68040cd880 2023-10-26T05:49:44.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.909+0000 7f6800ff9640 1 -- 172.21.15.71:0/1889400661 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f67f4019310 con 0x7f68040cd880 2023-10-26T05:49:44.914 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:44.914 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:32.147514+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:44.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67e4066470 msgr2=0x7f67e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67e4066470 0x7f67e4068930 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68041555c0 tx=0x7f67ec009040 comp rx=0 tx=0).stop 2023-10-26T05:49:44.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 msgr2=0x7f6804156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f6804156450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c450 tx=0x7f67f400c910 comp rx=0 tx=0).stop 2023-10-26T05:49:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 shutdown_connections 2023-10-26T05:49:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68040d0850 0x7f68040c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68040d0470 0x7f6804155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67e4066470 0x7f67e4068930 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 --2- 172.21.15.71:0/1889400661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68040cd880 0x7f6804156450 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 >> 172.21.15.71:0/1889400661 conn(0x7f68040cdd80 msgr2=0x7f68040b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 shutdown_connections 2023-10-26T05:49:44.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:44.913+0000 7f680a03b640 1 -- 172.21.15.71:0/1889400661 wait complete. 2023-10-26T05:49:44.931 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:49:44.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:45.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/2495790487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300cb3d0 msgr2=0x7fc23006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 --2- 172.21.15.71:0/2495790487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300cb3d0 0x7fc23006a710 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc218009900 tx=0x7fc21801c850 comp rx=0 tx=0).stop 2023-10-26T05:49:45.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/2495790487 shutdown_connections 2023-10-26T05:49:45.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 --2- 172.21.15.71:0/2495790487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300cb3d0 0x7fc23006a710 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/2495790487 >> 172.21.15.71:0/2495790487 conn(0x7fc2300ce270 msgr2=0x7fc2300ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/2495790487 shutdown_connections 2023-10-26T05:49:45.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/2495790487 wait complete. 2023-10-26T05:49:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 Processor -- start 2023-10-26T05:49:45.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- start start 2023-10-26T05:49:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2300cb3d0 0x7fc23013e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 0x7fc230140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc230142d20 0x7fc230140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2300b1500 con 0x7fc2300ccba0 2023-10-26T05:49:45.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23013f310 con 0x7fc230142d20 2023-10-26T05:49:45.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2301415c0 con 0x7fc2300cb3d0 2023-10-26T05:49:45.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 0x7fc230140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 0x7fc230140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47946/0 (socket says 172.21.15.71:47946) 2023-10-26T05:49:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 -- 172.21.15.71:0/1609056560 learned_addr learned my addr 172.21.15.71:0/1609056560 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:45.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc22f7fe640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc230142d20 0x7fc230140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc22effd640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2300cb3d0 0x7fc23013e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 -- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2300cb3d0 msgr2=0x7fc23013e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2300cb3d0 0x7fc23013e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 -- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc230142d20 msgr2=0x7fc230140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc230142d20 0x7fc230140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 -- 172.21.15.71:0/1609056560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc218009560 con 0x7fc2300ccba0 2023-10-26T05:49:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc22f7fe640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc230142d20 0x7fc230140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc2267fc640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 0x7fc230140990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc218009850 tx=0x7fc2180a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218005220 con 0x7fc2300ccba0 2023-10-26T05:49:45.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc218005380 con 0x7fc2300ccba0 2023-10-26T05:49:45.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc230159d00 con 0x7fc2300ccba0 2023-10-26T05:49:45.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180046d0 con 0x7fc2300ccba0 2023-10-26T05:49:45.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.053+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc230141b80 con 0x7fc2300ccba0 2023-10-26T05:49:45.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.057+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc230066d00 con 0x7fc2300ccba0 2023-10-26T05:49:45.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.057+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc2180b4070 con 0x7fc2300ccba0 2023-10-26T05:49:45.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:49:45.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.057+0000 7fc227fff640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc210066460 0x7fc210068920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.057+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc21811e860 con 0x7fc2300ccba0 2023-10-26T05:49:45.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.057+0000 7fc22effd640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc210066460 0x7fc210068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.057+0000 7fc22effd640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc210066460 0x7fc210068920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc220007fd0 tx=0x7fc220009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.061+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc2180ebd30 con 0x7fc2300ccba0 2023-10-26T05:49:45.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:49:45.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.237+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc230066d00 con 0x7fc2300ccba0 2023-10-26T05:49:45.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.237+0000 7fc227fff640 1 -- 172.21.15.71:0/1609056560 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fc21811e020 con 0x7fc2300ccba0 2023-10-26T05:49:45.243 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:45.243 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:45.244 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:45.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc210066460 msgr2=0x7fc210068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc210066460 0x7fc210068920 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc220007fd0 tx=0x7fc220009040 comp rx=0 tx=0).stop 2023-10-26T05:49:45.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 msgr2=0x7fc230140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 0x7fc230140990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc218009850 tx=0x7fc2180a5840 comp rx=0 tx=0).stop 2023-10-26T05:49:45.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 shutdown_connections 2023-10-26T05:49:45.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc230142d20 0x7fc230140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc210066460 0x7fc210068920 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2300ccba0 0x7fc230140990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 --2- 172.21.15.71:0/1609056560 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2300cb3d0 0x7fc23013e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 >> 172.21.15.71:0/1609056560 conn(0x7fc2300ce270 msgr2=0x7fc2300ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 shutdown_connections 2023-10-26T05:49:45.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.241+0000 7fc23531f640 1 -- 172.21.15.71:0/1609056560 wait complete. 2023-10-26T05:49:45.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:45.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/3528857234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b28140cc0 msgr2=0x7f0b2815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/3528857234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b28140cc0 0x7f0b2815c760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009840 tx=0x7f0b1801c790 comp rx=0 tx=0).stop 2023-10-26T05:49:45.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/3528857234 shutdown_connections 2023-10-26T05:49:45.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/3528857234 >> v2:172.21.15.86:3300/0 conn(0x7f0b280b2370 0x7f0b28071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/3528857234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b28140cc0 0x7f0b2815c760 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/3528857234 >> 172.21.15.71:0/3528857234 conn(0x7f0b28069770 msgr2=0x7f0b28069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/3528857234 shutdown_connections 2023-10-26T05:49:45.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/3528857234 wait complete. 2023-10-26T05:49:45.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 Processor -- start 2023-10-26T05:49:45.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- start start 2023-10-26T05:49:45.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b28070fe0 0x7f0b28070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 0x7f0b2812ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b2812f490 0x7f0b28156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2812f980 con 0x7f0b280b2370 2023-10-26T05:49:45.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2812fc60 con 0x7f0b28070fe0 2023-10-26T05:49:45.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b2e3e2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2812ff40 con 0x7f0b2812f490 2023-10-26T05:49:45.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 0x7f0b2812ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 0x7f0b2812ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47956/0 (socket says 172.21.15.71:47956) 2023-10-26T05:49:45.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 -- 172.21.15.71:0/2120868834 learned_addr learned my addr 172.21.15.71:0/2120868834 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:45.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 -- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b2812f490 msgr2=0x7f0b28156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b2812f490 0x7f0b28156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 -- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b28070fe0 msgr2=0x7f0b28070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b28070fe0 0x7f0b28070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.369+0000 7f0b1ffff640 1 -- 172.21.15.71:0/2120868834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180094a0 con 0x7f0b280b2370 2023-10-26T05:49:45.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b1ffff640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 0x7f0b2812ef50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c500 tx=0x7f0b1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10012830 con 0x7f0b280b2370 2023-10-26T05:49:45.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0b10012a80 con 0x7f0b280b2370 2023-10-26T05:49:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28130a90 con 0x7f0b280b2370 2023-10-26T05:49:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10012e90 con 0x7f0b280b2370 2023-10-26T05:49:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b28157c60 con 0x7f0b280b2370 2023-10-26T05:49:45.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b28066b10 con 0x7f0b280b2370 2023-10-26T05:49:45.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0b1001a3b0 con 0x7f0b280b2370 2023-10-26T05:49:45.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b257fa640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0b0c0664a0 0x7f0b0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0b10083fc0 con 0x7f0b280b2370 2023-10-26T05:49:45.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b27fff640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0b0c0664a0 0x7f0b0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.373+0000 7f0b27fff640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0b0c0664a0 0x7f0b0c068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b180af000 tx=0x7f0b18019530 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.377+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0b100b5090 con 0x7f0b280b2370 2023-10-26T05:49:45.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.549+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b280721e0 con 0x7f0b280b2370 2023-10-26T05:49:45.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b257fa640 1 -- 172.21.15.71:0/2120868834 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f0b100514d0 con 0x7f0b280b2370 2023-10-26T05:49:45.556 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:45.556 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":18,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:02.915412+0000","modified":"2023-10-26T05:49:04.973374+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5778":{"gid":5778,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1106692810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1106692810},{"type":"v1","addr":"172.21.15.71:6835","nonce":1106692810}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-26T05:49:45.557 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 18 2023-10-26T05:49:45.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0b0c0664a0 msgr2=0x7f0b0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0b0c0664a0 0x7f0b0c068960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b180af000 tx=0x7f0b18019530 comp rx=0 tx=0).stop 2023-10-26T05:49:45.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 msgr2=0x7f0b2812ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 0x7f0b2812ef50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c500 tx=0x7f0b1000c9c0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 shutdown_connections 2023-10-26T05:49:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0b2812f490 0x7f0b28156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0b0c0664a0 0x7f0b0c068960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b280b2370 0x7f0b2812ef50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 --2- 172.21.15.71:0/2120868834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b28070fe0 0x7f0b28070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 >> 172.21.15.71:0/2120868834 conn(0x7f0b28069770 msgr2=0x7f0b28069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 shutdown_connections 2023-10-26T05:49:45.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.553+0000 7f0b2e3e2640 1 -- 172.21.15.71:0/2120868834 wait complete. 2023-10-26T05:49:45.574 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:45.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/1623223062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8140cc0 msgr2=0x7f72a815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 --2- 172.21.15.71:0/1623223062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8140cc0 0x7f72a815c760 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009900 tx=0x7f72a401c850 comp rx=0 tx=0).stop 2023-10-26T05:49:45.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/1623223062 shutdown_connections 2023-10-26T05:49:45.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 --2- 172.21.15.71:0/1623223062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72a80b2370 0x7f72a8071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 --2- 172.21.15.71:0/1623223062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a8140cc0 0x7f72a815c760 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/1623223062 >> 172.21.15.71:0/1623223062 conn(0x7f72a8069770 msgr2=0x7f72a8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/1623223062 shutdown_connections 2023-10-26T05:49:45.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/1623223062 wait complete. 2023-10-26T05:49:45.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 Processor -- start 2023-10-26T05:49:45.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- start start 2023-10-26T05:49:45.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8070fe0 0x7f72a8157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 0x7f72a80c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72a8140cc0 0x7f72a813f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8161e50 con 0x7f72a80b2370 2023-10-26T05:49:45.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a81581c0 con 0x7f72a8070fe0 2023-10-26T05:49:45.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a8161240 con 0x7f72a8140cc0 2023-10-26T05:49:45.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ae17e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8070fe0 0x7f72a8157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 0x7f72a80c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 0x7f72a80c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47978/0 (socket says 172.21.15.71:47978) 2023-10-26T05:49:45.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ae17e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8070fe0 0x7f72a8157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:44274/0 (socket says 172.21.15.71:44274) 2023-10-26T05:49:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 -- 172.21.15.71:0/3103671902 learned_addr learned my addr 172.21.15.71:0/3103671902 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ae97f640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72a8140cc0 0x7f72a813f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 -- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72a8140cc0 msgr2=0x7f72a813f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72a8140cc0 0x7f72a813f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 -- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8070fe0 msgr2=0x7f72a8157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8070fe0 0x7f72a8157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 -- 172.21.15.71:0/3103671902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009560 con 0x7f72a80b2370 2023-10-26T05:49:45.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72ad97d640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 0x7f72a80c5080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c7f0 tx=0x7f729c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00aa50 con 0x7f72a80b2370 2023-10-26T05:49:45.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f729c00aca0 con 0x7f72a80b2370 2023-10-26T05:49:45.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a8161520 con 0x7f72a80b2370 2023-10-26T05:49:45.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c012680 con 0x7f72a80b2370 2023-10-26T05:49:45.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.681+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a8161ae0 con 0x7f72a80b2370 2023-10-26T05:49:45.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.685+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a8066b10 con 0x7f72a80b2370 2023-10-26T05:49:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.685+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f729c006b60 con 0x7f72a80b2370 2023-10-26T05:49:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.685+0000 7f729affd640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f728c066470 0x7f728c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.685+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f729c0848e0 con 0x7f72a80b2370 2023-10-26T05:49:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.685+0000 7f72ae17e640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f728c066470 0x7f728c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.685+0000 7f72ae17e640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f728c066470 0x7f728c068930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72a40af000 tx=0x7f72a401b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.689+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f729c051eb0 con 0x7f72a80b2370 2023-10-26T05:49:45.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.841+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72a80721e0 con 0x7f72a80b2370 2023-10-26T05:49:45.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f729affd640 1 -- 172.21.15.71:0/3103671902 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f729c016090 con 0x7f72a80b2370 2023-10-26T05:49:45.850 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:45.850 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":57,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:32.147514+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:00.912702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:01.904497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.36","mappings":[{"from":0,"to":5}]},{"pgid":"7.3c","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:45.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f728c066470 msgr2=0x7f728c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f728c066470 0x7f728c068930 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f72a40af000 tx=0x7f72a401b090 comp rx=0 tx=0).stop 2023-10-26T05:49:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 msgr2=0x7f72a80c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 0x7f72a80c5080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f729c00c7f0 tx=0x7f729c00ccb0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 shutdown_connections 2023-10-26T05:49:45.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72a8140cc0 0x7f72a813f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f728c066470 0x7f728c068930 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72a80b2370 0x7f72a80c5080 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 --2- 172.21.15.71:0/3103671902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72a8070fe0 0x7f72a8157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 >> 172.21.15.71:0/3103671902 conn(0x7f72a8069770 msgr2=0x7f72a80b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 shutdown_connections 2023-10-26T05:49:45.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.845+0000 7f72b03e2640 1 -- 172.21.15.71:0/3103671902 wait complete. 2023-10-26T05:49:45.868 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:49:45.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1181175618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 msgr2=0x7fb79c159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1181175618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c159700 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009840 tx=0x7fb78c01c770 comp rx=0 tx=0).stop 2023-10-26T05:49:45.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1181175618 shutdown_connections 2023-10-26T05:49:45.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1181175618 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c069d30 0x7fb79c06a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1181175618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c159700 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1181175618 >> 172.21.15.71:0/1181175618 conn(0x7fb79c06b8c0 msgr2=0x7fb79c06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:45.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1181175618 shutdown_connections 2023-10-26T05:49:45.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1181175618 wait complete. 2023-10-26T05:49:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 Processor -- start 2023-10-26T05:49:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- start start 2023-10-26T05:49:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c069d30 0x7fb79c13c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c0cabd0 0x7fb79c140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c157810 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0a7410 con 0x7fb79c069d30 2023-10-26T05:49:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb7a21e5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c0a76f0 con 0x7fb79c0cabd0 2023-10-26T05:49:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.977+0000 7fb79affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47986/0 (socket says 172.21.15.71:47986) 2023-10-26T05:49:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 -- 172.21.15.71:0/1230077187 learned_addr learned my addr 172.21.15.71:0/1230077187 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79b7fe640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c069d30 0x7fb79c13c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79bfff640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c0cabd0 0x7fb79c140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 -- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c0cabd0 msgr2=0x7fb79c140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c0cabd0 0x7fb79c140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 -- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c069d30 msgr2=0x7fb79c13c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c069d30 0x7fb79c13c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 -- 172.21.15.71:0/1230077187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c0094a0 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79b7fe640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c069d30 0x7fb79c13c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79affd640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c140a00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb790001ec0 tx=0x7fb79000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900054e0 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb790003600 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c157430 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790003a10 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c152de0 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c066b10 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb790003b70 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb783fff640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb788066470 0x7fb788068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:45.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb7900836e0 con 0x7fb79c0ca7f0 2023-10-26T05:49:45.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79b7fe640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb788066470 0x7fb788068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:45.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.981+0000 7fb79b7fe640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb788066470 0x7fb788068930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0af000 tx=0x7fb78c01afb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:45.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:45.985+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb790050a80 con 0x7fb79c0ca7f0 2023-10-26T05:49:46.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.157+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb79c0a7930 con 0x7fb79c0ca7f0 2023-10-26T05:49:46.162 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:49:46.157+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:49:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.161+0000 7fb783fff640 1 -- 172.21.15.71:0/1230077187 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb79c0a7930 con 0x7fb79c0ca7f0 2023-10-26T05:49:46.165 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:49:46.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb788066470 msgr2=0x7fb788068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:46.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb788066470 0x7fb788068930 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0af000 tx=0x7fb78c01afb0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 msgr2=0x7fb79c140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:46.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c140a00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb790001ec0 tx=0x7fb79000c8e0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 shutdown_connections 2023-10-26T05:49:46.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c0cabd0 0x7fb79c140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb788066470 0x7fb788068930 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0ca7f0 0x7fb79c140a00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 --2- 172.21.15.71:0/1230077187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c069d30 0x7fb79c13c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 >> 172.21.15.71:0/1230077187 conn(0x7fb79c06b8c0 msgr2=0x7fb79c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:46.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 shutdown_connections 2023-10-26T05:49:46.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.165+0000 7fb7a21e5640 1 -- 172.21.15.71:0/1230077187 wait complete. 2023-10-26T05:49:46.179 INFO:tasks.ceph.mds.c.smithi071.stderr: -9608> 2023-10-26T05:49:08.529+0000 7f86ea962640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:49:46.187 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:49:46.229 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:49:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 -- 172.21.15.71:0/197569777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68100cc050 msgr2=0x7f68100cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 --2- 172.21.15.71:0/197569777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68100cc050 0x7f68100cd9f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6808009bc0 tx=0x7f680801cac0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 -- 172.21.15.71:0/197569777 shutdown_connections 2023-10-26T05:49:46.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 --2- 172.21.15.71:0/197569777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68100cc050 0x7f68100cd9f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 --2- 172.21.15.71:0/197569777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 0x7f68100cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 -- 172.21.15.71:0/197569777 >> 172.21.15.71:0/197569777 conn(0x7f681006f440 msgr2=0x7f681006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 -- 172.21.15.71:0/197569777 shutdown_connections 2023-10-26T05:49:46.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.293+0000 7f6817ebc640 1 -- 172.21.15.71:0/197569777 wait complete. 2023-10-26T05:49:46.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 Processor -- start 2023-10-26T05:49:46.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- start start 2023-10-26T05:49:46.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 0x7f6810138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:46.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68100cc050 0x7f68100cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:46.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 0x7f68100cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:46.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810150f30 con 0x7f6810142b30 2023-10-26T05:49:46.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68101518a0 con 0x7f68100cc050 2023-10-26T05:49:46.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68101515c0 con 0x7f68100af990 2023-10-26T05:49:46.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 0x7f68100cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:46.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 0x7f68100cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48030/0 (socket says 172.21.15.71:48030) 2023-10-26T05:49:46.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 -- 172.21.15.71:0/1059832989 learned_addr learned my addr 172.21.15.71:0/1059832989 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:46.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6815457640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68100cc050 0x7f68100cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:46.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6815c58640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 0x7f6810138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:46.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 -- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 msgr2=0x7f6810138a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:46.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 0x7f6810138a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 -- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68100cc050 msgr2=0x7f68100cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:46.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68100cc050 0x7f68100cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:46.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 -- 172.21.15.71:0/1059832989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808009820 con 0x7f6810142b30 2023-10-26T05:49:46.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6815c58640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 0x7f6810138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:46.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6815457640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68100cc050 0x7f68100cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:46.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6816459640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 0x7f68100cf340 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f680801a6f0 tx=0x7f68080a59d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:46.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808004c50 con 0x7f6810142b30 2023-10-26T05:49:46.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6808004db0 con 0x7f6810142b30 2023-10-26T05:49:46.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6810150970 con 0x7f6810142b30 2023-10-26T05:49:46.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6808004220 con 0x7f6810142b30 2023-10-26T05:49:46.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6810138f80 con 0x7f6810142b30 2023-10-26T05:49:46.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810066b10 con 0x7f6810142b30 2023-10-26T05:49:46.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68080b4040 con 0x7f6810142b30 2023-10-26T05:49:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.297+0000 7f68067fc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67f4066470 0x7f67f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.301+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f680811e870 con 0x7f6810142b30 2023-10-26T05:49:46.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.301+0000 7f6815c58640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67f4066470 0x7f67f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.301+0000 7f6815c58640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67f4066470 0x7f67f4068930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f680c000f70 tx=0x7f680c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:46.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.305+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68080ebcf0 con 0x7f6810142b30 2023-10-26T05:49:46.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:46.481+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f68100721e0 con 0x7f6810142b30 2023-10-26T05:49:47.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f68067fc640 1 -- 172.21.15.71:0/1059832989 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f68080b20a0 con 0x7f6810142b30 2023-10-26T05:49:47.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67f4066470 msgr2=0x7f67f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:47.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67f4066470 0x7f67f4068930 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f680c000f70 tx=0x7f680c009040 comp rx=0 tx=0).stop 2023-10-26T05:49:47.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 msgr2=0x7f68100cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:47.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 0x7f68100cf340 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f680801a6f0 tx=0x7f68080a59d0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 shutdown_connections 2023-10-26T05:49:47.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67f4066470 0x7f67f4068930 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6810142b30 0x7f68100cf340 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68100cc050 0x7f68100cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 --2- 172.21.15.71:0/1059832989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68100af990 0x7f6810138a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.165+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 >> 172.21.15.71:0/1059832989 conn(0x7f681006f440 msgr2=0x7f6810155f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:47.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.169+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 shutdown_connections 2023-10-26T05:49:47.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.169+0000 7f6817ebc640 1 -- 172.21.15.71:0/1059832989 wait complete. 2023-10-26T05:49:47.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:49:47.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/2654878650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 msgr2=0x7fcd2c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:47.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 --2- 172.21.15.71:0/2654878650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c0cb820 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0aebf0 tx=0x7fcd1c01c850 comp rx=0 tx=0).stop 2023-10-26T05:49:47.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/2654878650 shutdown_connections 2023-10-26T05:49:47.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 --2- 172.21.15.71:0/2654878650 >> v2:172.21.15.86:3300/0 conn(0x7fcd2c13efd0 0x7fcd2c069d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 --2- 172.21.15.71:0/2654878650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c0cb820 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/2654878650 >> 172.21.15.71:0/2654878650 conn(0x7fcd2c0aec60 msgr2=0x7fcd2c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:47.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/2654878650 shutdown_connections 2023-10-26T05:49:47.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/2654878650 wait complete. 2023-10-26T05:49:47.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 Processor -- start 2023-10-26T05:49:47.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- start start 2023-10-26T05:49:47.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2c0cbd60 0x7fcd2c1403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:47.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c13b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:47.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd2c13efd0 0x7fcd2c1309a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:47.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c141690 con 0x7fcd2c13c190 2023-10-26T05:49:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c13b7e0 con 0x7fcd2c13efd0 2023-10-26T05:49:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c140a80 con 0x7fcd2c0cbd60 2023-10-26T05:49:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c13b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:47.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c13b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48056/0 (socket says 172.21.15.71:48056) 2023-10-26T05:49:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 -- 172.21.15.71:0/356292519 learned_addr learned my addr 172.21.15.71:0/356292519 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2bfff640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2c0cbd60 0x7fcd2c1403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 -- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2c0cbd60 msgr2=0x7fcd2c1403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:47.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2c0cbd60 0x7fcd2c1403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 -- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd2c13efd0 msgr2=0x7fcd2c1309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd30922640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd2c13efd0 0x7fcd2c1309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd2c13efd0 0x7fcd2c1309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:47.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 -- 172.21.15.71:0/356292519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c009560 con 0x7fcd2c13c190 2023-10-26T05:49:47.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd30922640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd2c13efd0 0x7fcd2c1309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd2b7fe640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c13b2a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c4c0 tx=0x7fcd20001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20005480 con 0x7fcd2c13c190 2023-10-26T05:49:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd20003600 con 0x7fcd2c13c190 2023-10-26T05:49:47.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c140d60 con 0x7fcd2c13c190 2023-10-26T05:49:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20003a30 con 0x7fcd2c13c190 2023-10-26T05:49:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.369+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c141320 con 0x7fcd2c13c190 2023-10-26T05:49:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.373+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c066b10 con 0x7fcd2c13c190 2023-10-26T05:49:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.373+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd20003b90 con 0x7fcd2c13c190 2023-10-26T05:49:47.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.373+0000 7fcd28ff9640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd18066470 0x7fcd18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.373+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcd200837e0 con 0x7fcd2c13c190 2023-10-26T05:49:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.373+0000 7fcd2bfff640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd18066470 0x7fcd18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:47.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.373+0000 7fcd2bfff640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd18066470 0x7fcd18068930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0af000 tx=0x7fcd1c01b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:47.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.377+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd20050b40 con 0x7fcd2c13c190 2023-10-26T05:49:47.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:47.537+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd2c0721e0 con 0x7fcd2c13c190 2023-10-26T05:49:48.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.169+0000 7fcd28ff9640 1 -- 172.21.15.71:0/356292519 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcd20017030 con 0x7fcd2c13c190 2023-10-26T05:49:48.173 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:49:48.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd18066470 msgr2=0x7fcd18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:48.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd18066470 0x7fcd18068930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0af000 tx=0x7fcd1c01b090 comp rx=0 tx=0).stop 2023-10-26T05:49:48.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 msgr2=0x7fcd2c13b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:48.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c13b2a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c4c0 tx=0x7fcd20001e80 comp rx=0 tx=0).stop 2023-10-26T05:49:48.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 shutdown_connections 2023-10-26T05:49:48.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd2c13efd0 0x7fcd2c1309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd18066470 0x7fcd18068930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd2c13c190 0x7fcd2c13b2a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 --2- 172.21.15.71:0/356292519 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2c0cbd60 0x7fcd2c1403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 >> 172.21.15.71:0/356292519 conn(0x7fcd2c0aec60 msgr2=0x7fcd2c13cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:48.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 shutdown_connections 2023-10-26T05:49:48.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.173+0000 7fcd32385640 1 -- 172.21.15.71:0/356292519 wait complete. 2023-10-26T05:49:48.200 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:49:48.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 -- 172.21.15.71:0/1865583197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980003980 msgr2=0x7f5980003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:48.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 --2- 172.21.15.71:0/1865583197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980003980 0x7f5980003da0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f598800bb10 tx=0x7f598801cd30 comp rx=0 tx=0).stop 2023-10-26T05:49:48.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 -- 172.21.15.71:0/1865583197 shutdown_connections 2023-10-26T05:49:48.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 --2- 172.21.15.71:0/1865583197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5980003980 0x7f5980003da0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 -- 172.21.15.71:0/1865583197 >> 172.21.15.71:0/1865583197 conn(0x7f59980aec60 msgr2=0x7f59980af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:48.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 -- 172.21.15.71:0/1865583197 shutdown_connections 2023-10-26T05:49:48.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.333+0000 7f599d2f0640 1 -- 172.21.15.71:0/1865583197 wait complete. 2023-10-26T05:49:48.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 Processor -- start 2023-10-26T05:49:48.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- start start 2023-10-26T05:49:48.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5980003980 0x7f5998135f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:48.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 0x7f5998136470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:48.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59981369b0 0x7f5998136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:48.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59981f39c0 con 0x7f599815a5a0 2023-10-26T05:49:48.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5998135200 con 0x7f59981369b0 2023-10-26T05:49:48.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5998159ee0 con 0x7f5980003980 2023-10-26T05:49:48.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 0x7f5998136470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:48.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 0x7f5998136470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48072/0 (socket says 172.21.15.71:48072) 2023-10-26T05:49:48.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 -- 172.21.15.71:0/2827689029 learned_addr learned my addr 172.21.15.71:0/2827689029 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:48.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59977fe640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59981369b0 0x7f5998136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:48.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f5996ffd640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5980003980 0x7f5998135f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:48.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 -- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5980003980 msgr2=0x7f5998135f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:48.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5980003980 0x7f5998135f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 -- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59981369b0 msgr2=0x7f5998136e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:48.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59981369b0 0x7f5998136e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:48.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 -- 172.21.15.71:0/2827689029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998135450 con 0x7f599815a5a0 2023-10-26T05:49:48.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f59967fc640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 0x7f5998136470 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d480 tx=0x7f598c00d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:48.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c012630 con 0x7f599815a5a0 2023-10-26T05:49:48.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f598c012790 con 0x7f599815a5a0 2023-10-26T05:49:48.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5998135730 con 0x7f599815a5a0 2023-10-26T05:49:48.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c012b80 con 0x7f599815a5a0 2023-10-26T05:49:48.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59981585f0 con 0x7f599815a5a0 2023-10-26T05:49:48.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5998066b10 con 0x7f599815a5a0 2023-10-26T05:49:48.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f598c01b410 con 0x7f599815a5a0 2023-10-26T05:49:48.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f597f7fe640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59840664a0 0x7f5984068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:48.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.337+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f598c083d10 con 0x7f599815a5a0 2023-10-26T05:49:48.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.341+0000 7f5996ffd640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59840664a0 0x7f5984068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:48.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.341+0000 7f5996ffd640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59840664a0 0x7f5984068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f59880b0000 tx=0x7f59880a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:48.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.341+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f598c051340 con 0x7f599815a5a0 2023-10-26T05:49:48.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:48.497+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59980721e0 con 0x7f599815a5a0 2023-10-26T05:49:49.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.181+0000 7f597f7fe640 1 -- 172.21.15.71:0/2827689029 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7f598c054960 con 0x7f599815a5a0 2023-10-26T05:49:49.188 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:49:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59840664a0 msgr2=0x7f5984068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59840664a0 0x7f5984068960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f59880b0000 tx=0x7f59880a7040 comp rx=0 tx=0).stop 2023-10-26T05:49:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 msgr2=0x7f5998136470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 0x7f5998136470 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d480 tx=0x7f598c00d940 comp rx=0 tx=0).stop 2023-10-26T05:49:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 shutdown_connections 2023-10-26T05:49:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59981369b0 0x7f5998136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59840664a0 0x7f5984068960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f599815a5a0 0x7f5998136470 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 --2- 172.21.15.71:0/2827689029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5980003980 0x7f5998135f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 >> 172.21.15.71:0/2827689029 conn(0x7f59980aec60 msgr2=0x7f59981577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:49.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 shutdown_connections 2023-10-26T05:49:49.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.189+0000 7f599d2f0640 1 -- 172.21.15.71:0/2827689029 wait complete. 2023-10-26T05:49:49.215 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:49:49.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- 172.21.15.71:0/659311566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64140cb8c0 msgr2=0x7f6414159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 --2- 172.21.15.71:0/659311566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64140cb8c0 0x7f6414159060 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f640c009bc0 tx=0x7f640c01cb50 comp rx=0 tx=0).stop 2023-10-26T05:49:49.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- 172.21.15.71:0/659311566 shutdown_connections 2023-10-26T05:49:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 --2- 172.21.15.71:0/659311566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64140cb8c0 0x7f6414159060 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- 172.21.15.71:0/659311566 >> 172.21.15.71:0/659311566 conn(0x7f641406b8c0 msgr2=0x7f641406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:49.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- 172.21.15.71:0/659311566 shutdown_connections 2023-10-26T05:49:49.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- 172.21.15.71:0/659311566 wait complete. 2023-10-26T05:49:49.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 Processor -- start 2023-10-26T05:49:49.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- start start 2023-10-26T05:49:49.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64140cb8c0 0x7f6414152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 0x7f6414153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f641413de80 0x7f641414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6414155660 con 0x7f641413ae90 2023-10-26T05:49:49.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6414133680 con 0x7f641413de80 2023-10-26T05:49:49.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f641ac6e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6414155010 con 0x7f64140cb8c0 2023-10-26T05:49:49.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f6413fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 0x7f6414153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f6413fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 0x7f6414153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48088/0 (socket says 172.21.15.71:48088) 2023-10-26T05:49:49.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.341+0000 7f6413fff640 1 -- 172.21.15.71:0/1451414204 learned_addr learned my addr 172.21.15.71:0/1451414204 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:49.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f641920b640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f641413de80 0x7f641414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6418a0a640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64140cb8c0 0x7f6414152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6413fff640 1 -- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64140cb8c0 msgr2=0x7f6414152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6413fff640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64140cb8c0 0x7f6414152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6413fff640 1 -- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f641413de80 msgr2=0x7f641414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6413fff640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f641413de80 0x7f641414f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6413fff640 1 -- 172.21.15.71:0/1451414204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c009820 con 0x7f641413ae90 2023-10-26T05:49:49.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f641920b640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f641413de80 0x7f641414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:49.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6413fff640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 0x7f6414153210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f640800c4d0 tx=0x7f640800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:49.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408005480 con 0x7f641413ae90 2023-10-26T05:49:49.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64080036f0 con 0x7f641413ae90 2023-10-26T05:49:49.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64141552f0 con 0x7f641413ae90 2023-10-26T05:49:49.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6408003b00 con 0x7f641413ae90 2023-10-26T05:49:49.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641414f680 con 0x7f641413ae90 2023-10-26T05:49:49.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6414066b10 con 0x7f641413ae90 2023-10-26T05:49:49.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6408003c60 con 0x7f641413ae90 2023-10-26T05:49:49.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f64117fa640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6400066470 0x7f6400068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6408083420 con 0x7f641413ae90 2023-10-26T05:49:49.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6418a0a640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6400066470 0x7f6400068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.345+0000 7f6418a0a640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6400066470 0x7f6400068930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6404000f70 tx=0x7f6404009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:49.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.349+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6408050c60 con 0x7f641413ae90 2023-10-26T05:49:49.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.505+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f641406c7d0 con 0x7f641413ae90 2023-10-26T05:49:49.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f64117fa640 1 -- 172.21.15.71:0/1451414204 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v22) v1 ==== 71+0+0 (secure 0 0 0) 0x7f6408017020 con 0x7f641413ae90 2023-10-26T05:49:49.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6400066470 msgr2=0x7f6400068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6400066470 0x7f6400068930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6404000f70 tx=0x7f6404009040 comp rx=0 tx=0).stop 2023-10-26T05:49:49.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 msgr2=0x7f6414153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 0x7f6414153210 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f640800c4d0 tx=0x7f640800c990 comp rx=0 tx=0).stop 2023-10-26T05:49:49.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 shutdown_connections 2023-10-26T05:49:49.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f641413de80 0x7f641414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6400066470 0x7f6400068930 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f641413ae90 0x7f6414153210 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 --2- 172.21.15.71:0/1451414204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64140cb8c0 0x7f6414152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 >> 172.21.15.71:0/1451414204 conn(0x7f641406b8c0 msgr2=0x7f64140ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.513+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 shutdown_connections 2023-10-26T05:49:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.517+0000 7f641ac6e640 1 -- 172.21.15.71:0/1451414204 wait complete. 2023-10-26T05:49:49.537 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-10-26T05:49:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- 172.21.15.71:0/1762622639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 msgr2=0x7f68c4159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 --2- 172.21.15.71:0/1762622639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c4159810 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f68c40ac6e0 tx=0x7f68bc01cd00 comp rx=0 tx=0).stop 2023-10-26T05:49:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- 172.21.15.71:0/1762622639 shutdown_connections 2023-10-26T05:49:49.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 --2- 172.21.15.71:0/1762622639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c4159810 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 --2- 172.21.15.71:0/1762622639 >> v2:172.21.15.86:3300/0 conn(0x7f68c413c2f0 0x7f68c40b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- 172.21.15.71:0/1762622639 >> 172.21.15.71:0/1762622639 conn(0x7f68c4069770 msgr2=0x7f68c4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:49.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- 172.21.15.71:0/1762622639 shutdown_connections 2023-10-26T05:49:49.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- 172.21.15.71:0/1762622639 wait complete. 2023-10-26T05:49:49.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 Processor -- start 2023-10-26T05:49:49.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- start start 2023-10-26T05:49:49.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c40b2890 0x7f68c413b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c41343f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c413c2f0 0x7f68c414f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c413b6f0 con 0x7f68c40cb1e0 2023-10-26T05:49:49.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4134cb0 con 0x7f68c40b2890 2023-10-26T05:49:49.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68cbb68640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c4134f90 con 0x7f68c413c2f0 2023-10-26T05:49:49.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c41343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c41343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48100/0 (socket says 172.21.15.71:48100) 2023-10-26T05:49:49.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 -- 172.21.15.71:0/3207914335 learned_addr learned my addr 172.21.15.71:0/3207914335 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:49.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68ca105640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c413c2f0 0x7f68c414f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9904640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c40b2890 0x7f68c413b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 -- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c413c2f0 msgr2=0x7f68c414f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c413c2f0 0x7f68c414f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 -- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c40b2890 msgr2=0x7f68c413b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:49.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c40b2890 0x7f68c413b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 -- 172.21.15.71:0/3207914335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc0098e0 con 0x7f68c40cb1e0 2023-10-26T05:49:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.661+0000 7f68c9103640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c41343f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c490 tx=0x7f68b4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4003600 con 0x7f68c40cb1e0 2023-10-26T05:49:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f68b4003780 con 0x7f68c40cb1e0 2023-10-26T05:49:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c4135270 con 0x7f68c40cb1e0 2023-10-26T05:49:49.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4003bd0 con 0x7f68c40cb1e0 2023-10-26T05:49:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c4134930 con 0x7f68c40cb1e0 2023-10-26T05:49:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c4066b10 con 0x7f68c40cb1e0 2023-10-26T05:49:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68b4003d30 con 0x7f68c40cb1e0 2023-10-26T05:49:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68ba7fc640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68a8066470 0x7f68a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f68b40832f0 con 0x7f68c40cb1e0 2023-10-26T05:49:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68c9904640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68a8066470 0x7f68a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.665+0000 7f68c9904640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68a8066470 0x7f68a8068930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f68c00058d0 tx=0x7f68c0005bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.669+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68b4050b00 con 0x7f68c40cb1e0 2023-10-26T05:49:49.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:49.885+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f68c40721e0 con 0x7f68c40cb1e0 2023-10-26T05:49:50.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.185+0000 7f68ba7fc640 1 -- 172.21.15.71:0/3207914335 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v157) v1 ==== 142+0+0 (secure 0 0 0) 0x7f68b4054120 con 0x7f68c40cb1e0 2023-10-26T05:49:50.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68a8066470 msgr2=0x7f68a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68a8066470 0x7f68a8068930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f68c00058d0 tx=0x7f68c0005bd0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 msgr2=0x7f68c41343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c41343f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f68b400c490 tx=0x7f68b4001e80 comp rx=0 tx=0).stop 2023-10-26T05:49:50.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 shutdown_connections 2023-10-26T05:49:50.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c413c2f0 0x7f68c414f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68a8066470 0x7f68a8068930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c40cb1e0 0x7f68c41343f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 --2- 172.21.15.71:0/3207914335 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68c40b2890 0x7f68c413b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 >> 172.21.15.71:0/3207914335 conn(0x7f68c4069770 msgr2=0x7f68c40ad620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:50.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 shutdown_connections 2023-10-26T05:49:50.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.189+0000 7f68cbb68640 1 -- 172.21.15.71:0/3207914335 wait complete. 2023-10-26T05:49:50.213 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-26T05:49:50.213 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T05:49:50.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:50.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 -- 172.21.15.71:0/1324065544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c140cc0 msgr2=0x7f5e8c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1324065544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c15c760 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009870 tx=0x7f5e8801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 -- 172.21.15.71:0/1324065544 shutdown_connections 2023-10-26T05:49:50.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1324065544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c15c760 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 -- 172.21.15.71:0/1324065544 >> 172.21.15.71:0/1324065544 conn(0x7f5e8c069770 msgr2=0x7f5e8c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:50.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 -- 172.21.15.71:0/1324065544 shutdown_connections 2023-10-26T05:49:50.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.333+0000 7f5e938c8640 1 -- 172.21.15.71:0/1324065544 wait complete. 2023-10-26T05:49:50.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 Processor -- start 2023-10-26T05:49:50.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- start start 2023-10-26T05:49:50.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 0x7f5e8c13e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e8c0b2370 0x7f5e8c1f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c13f330 con 0x7f5e8c070fe0 2023-10-26T05:49:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c13cc40 con 0x7f5e8c140cc0 2023-10-26T05:49:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c13c070 con 0x7f5e8c0b2370 2023-10-26T05:49:50.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 0x7f5e8c13e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 0x7f5e8c13e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48114/0 (socket says 172.21.15.71:48114) 2023-10-26T05:49:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 -- 172.21.15.71:0/1390242499 learned_addr learned my addr 172.21.15.71:0/1390242499 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 -- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e8c0b2370 msgr2=0x7f5e8c1f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e8c0b2370 0x7f5e8c1f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91e65640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 -- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e8c140cc0 msgr2=0x7f5e8c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 -- 172.21.15.71:0/1390242499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880094a0 con 0x7f5e8c070fe0 2023-10-26T05:49:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91e65640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:50.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e91664640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 0x7f5e8c13e050 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5e8801a400 tx=0x7f5e880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:50.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e880044b0 con 0x7f5e8c070fe0 2023-10-26T05:49:50.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e88005120 con 0x7f5e8c070fe0 2023-10-26T05:49:50.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c13c310 con 0x7f5e8c070fe0 2023-10-26T05:49:50.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e88005530 con 0x7f5e8c070fe0 2023-10-26T05:49:50.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c13c8d0 con 0x7f5e8c070fe0 2023-10-26T05:49:50.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c066b10 con 0x7f5e8c070fe0 2023-10-26T05:49:50.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5e880b4040 con 0x7f5e8c070fe0 2023-10-26T05:49:50.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e81ffb640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e70066470 0x7f5e70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.337+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5e8811d9d0 con 0x7f5e8c070fe0 2023-10-26T05:49:50.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.341+0000 7f5e90e63640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e70066470 0x7f5e70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.341+0000 7f5e90e63640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e70066470 0x7f5e70068930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000f70 tx=0x7f5e7c0079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:50.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.341+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5e880eb150 con 0x7f5e8c070fe0 2023-10-26T05:49:50.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.501+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e8c0721e0 con 0x7f5e8c070fe0 2023-10-26T05:49:50.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.501+0000 7f5e81ffb640 1 -- 172.21.15.71:0/1390242499 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12902 (secure 0 0 0) 0x7f5e880b2020 con 0x7f5e8c070fe0 2023-10-26T05:49:50.507 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:50.507 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":60,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:49.182907+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/1106692810":"2023-10-27T05:49:46.161331+0000","172.21.15.71:6834/1106692810":"2023-10-27T05:49:46.161331+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:50.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e70066470 msgr2=0x7f5e70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e70066470 0x7f5e70068930 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c000f70 tx=0x7f5e7c0079c0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 msgr2=0x7f5e8c13e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 0x7f5e8c13e050 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5e8801a400 tx=0x7f5e880040c0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 shutdown_connections 2023-10-26T05:49:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e8c140cc0 0x7f5e8c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5e8c0b2370 0x7f5e8c1f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e70066470 0x7f5e70068930 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 --2- 172.21.15.71:0/1390242499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e8c070fe0 0x7f5e8c13e050 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 >> 172.21.15.71:0/1390242499 conn(0x7f5e8c069770 msgr2=0x7f5e8c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 shutdown_connections 2023-10-26T05:49:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.505+0000 7f5e938c8640 1 -- 172.21.15.71:0/1390242499 wait complete. 2023-10-26T05:49:50.527 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:49:50.528 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:49:50.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- 172.21.15.71:0/4270687471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f40047d0 msgr2=0x7f01f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 --2- 172.21.15.71:0/4270687471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f40047d0 0x7f01f4004c50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f01f0001680 tx=0x7f01f001c730 comp rx=0 tx=0).stop 2023-10-26T05:49:50.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- 172.21.15.71:0/4270687471 shutdown_connections 2023-10-26T05:49:50.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 --2- 172.21.15.71:0/4270687471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01f40047d0 0x7f01f4004c50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- 172.21.15.71:0/4270687471 >> 172.21.15.71:0/4270687471 conn(0x7f02080cdd80 msgr2=0x7f02080ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:50.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- 172.21.15.71:0/4270687471 shutdown_connections 2023-10-26T05:49:50.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- 172.21.15.71:0/4270687471 wait complete. 2023-10-26T05:49:50.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 Processor -- start 2023-10-26T05:49:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- start start 2023-10-26T05:49:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01f40047d0 0x7f02081577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 0x7f02080c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0208159dc0 0x7f02080b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020806a650 con 0x7f020815dd40 2023-10-26T05:49:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02080c7330 con 0x7f0208159dc0 2023-10-26T05:49:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f020e038640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02080c5320 con 0x7f01f40047d0 2023-10-26T05:49:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f0206ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 0x7f02080c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f0206ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 0x7f02080c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48130/0 (socket says 172.21.15.71:48130) 2023-10-26T05:49:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f0206ffd640 1 -- 172.21.15.71:0/970625590 learned_addr learned my addr 172.21.15.71:0/970625590 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f0207fff640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0208159dc0 0x7f02080b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.653+0000 7f02077fe640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01f40047d0 0x7f02081577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f0206ffd640 1 -- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01f40047d0 msgr2=0x7f02081577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f0206ffd640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01f40047d0 0x7f02081577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f0206ffd640 1 -- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0208159dc0 msgr2=0x7f02080b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f0206ffd640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0208159dc0 0x7f02080b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f0206ffd640 1 -- 172.21.15.71:0/970625590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02080c7560 con 0x7f020815dd40 2023-10-26T05:49:50.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f02077fe640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01f40047d0 0x7f02081577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:50.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f0206ffd640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 0x7f02080c57b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005040 tx=0x7f01f0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:50.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00a9040 con 0x7f020815dd40 2023-10-26T05:49:50.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f01f00068f0 con 0x7f020815dd40 2023-10-26T05:49:50.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02080c7800 con 0x7f020815dd40 2023-10-26T05:49:50.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00a5800 con 0x7f020815dd40 2023-10-26T05:49:50.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02080c7dc0 con 0x7f020815dd40 2023-10-26T05:49:50.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0208066b10 con 0x7f020815dd40 2023-10-26T05:49:50.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f01f00b5020 con 0x7f020815dd40 2023-10-26T05:49:50.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f01fffff640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f01e8066470 0x7f01e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01f00aa070 con 0x7f020815dd40 2023-10-26T05:49:50.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f02077fe640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f01e8066470 0x7f01e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.657+0000 7f02077fe640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f01e8066470 0x7f01e8068930 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0200009ed0 tx=0x7f0200009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:50.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.661+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f01f00eb260 con 0x7f020815dd40 2023-10-26T05:49:50.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.805+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02080721e0 con 0x7f020815dd40 2023-10-26T05:49:50.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.805+0000 7f01fffff640 1 -- 172.21.15.71:0/970625590 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01f00bd020 con 0x7f020815dd40 2023-10-26T05:49:50.811 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:49:50.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f01e8066470 msgr2=0x7f01e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f01e8066470 0x7f01e8068930 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0200009ed0 tx=0x7f0200009500 comp rx=0 tx=0).stop 2023-10-26T05:49:50.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 msgr2=0x7f02080c57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 0x7f02080c57b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f01f0005040 tx=0x7f01f0003c80 comp rx=0 tx=0).stop 2023-10-26T05:49:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 shutdown_connections 2023-10-26T05:49:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0208159dc0 0x7f02080b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f01e8066470 0x7f01e8068930 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f020815dd40 0x7f02080c57b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 --2- 172.21.15.71:0/970625590 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01f40047d0 0x7f02081577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 >> 172.21.15.71:0/970625590 conn(0x7f02080cdd80 msgr2=0x7f020815fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 shutdown_connections 2023-10-26T05:49:50.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.809+0000 7f020e038640 1 -- 172.21.15.71:0/970625590 wait complete. 2023-10-26T05:49:50.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-26T05:49:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/3642708926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cc0003980 msgr2=0x7f9cc0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/3642708926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cc0003980 0x7f9cc0003da0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4005fd0 tx=0x7f9cc401cd30 comp rx=0 tx=0).stop 2023-10-26T05:49:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/3642708926 shutdown_connections 2023-10-26T05:49:50.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/3642708926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cc0003980 0x7f9cc0003da0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/3642708926 >> 172.21.15.71:0/3642708926 conn(0x7f9cd40cdd80 msgr2=0x7f9cd40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:50.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/3642708926 shutdown_connections 2023-10-26T05:49:50.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/3642708926 wait complete. 2023-10-26T05:49:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 Processor -- start 2023-10-26T05:49:50.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- start start 2023-10-26T05:49:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc0003980 0x7f9cd41f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cd4151720 0x7f9cd40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 0x7f9cd41f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd406a650 con 0x7f9cd40cbe70 2023-10-26T05:49:50.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40cc2f0 con 0x7f9cc0003980 2023-10-26T05:49:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cdb2cc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40c92f0 con 0x7f9cd4151720 2023-10-26T05:49:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 0x7f9cd41f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 0x7f9cd41f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48150/0 (socket says 172.21.15.71:48150) 2023-10-26T05:49:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 -- 172.21.15.71:0/2441474663 learned_addr learned my addr 172.21.15.71:0/2441474663 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:50.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 -- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cd4151720 msgr2=0x7f9cd40cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd8867640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cd4151720 0x7f9cd40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9068640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc0003980 0x7f9cd41f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cd4151720 0x7f9cd40cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 -- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc0003980 msgr2=0x7f9cd41f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:50.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc0003980 0x7f9cd41f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:50.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 -- 172.21.15.71:0/2441474663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd41f9170 con 0x7f9cd40cbe70 2023-10-26T05:49:50.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.949+0000 7f9cd9869640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 0x7f9cd41f4ca0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40d0c30 tx=0x7f9ccc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:50.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00ca20 con 0x7f9cd40cbe70 2023-10-26T05:49:50.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ccc00cc90 con 0x7f9cd40cbe70 2023-10-26T05:49:50.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd40c86e0 con 0x7f9cd40cbe70 2023-10-26T05:49:50.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc011460 con 0x7f9cd40cbe70 2023-10-26T05:49:50.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd40c8ca0 con 0x7f9cd40cbe70 2023-10-26T05:49:50.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd4066b10 con 0x7f9cd40cbe70 2023-10-26T05:49:50.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9ccc0115c0 con 0x7f9cd40cbe70 2023-10-26T05:49:50.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cc9ffb640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9cb8066470 0x7f9cb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:50.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ccc084260 con 0x7f9cd40cbe70 2023-10-26T05:49:50.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cd9068640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9cb8066470 0x7f9cb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:50.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.953+0000 7f9cd9068640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9cb8066470 0x7f9cb8068930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40b0000 tx=0x7f9cc4019ad0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:50.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:50.957+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9ccc0519e0 con 0x7f9cd40cbe70 2023-10-26T05:49:51.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.173+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f9cd40cc690 con 0x7f9cd40cbe70 2023-10-26T05:49:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cc9ffb640 1 -- 172.21.15.71:0/2441474663 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v158) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9ccc055000 con 0x7f9cd40cbe70 2023-10-26T05:49:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9cb8066470 msgr2=0x7f9cb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9cb8066470 0x7f9cb8068930 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40b0000 tx=0x7f9cc4019ad0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 msgr2=0x7f9cd41f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 0x7f9cd41f4ca0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40d0c30 tx=0x7f9ccc00ed10 comp rx=0 tx=0).stop 2023-10-26T05:49:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 shutdown_connections 2023-10-26T05:49:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9cb8066470 0x7f9cb8068930 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd40cbe70 0x7f9cd41f4ca0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9cd4151720 0x7f9cd40cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 --2- 172.21.15.71:0/2441474663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cc0003980 0x7f9cd41f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 >> 172.21.15.71:0/2441474663 conn(0x7f9cd40cdd80 msgr2=0x7f9cd40b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 shutdown_connections 2023-10-26T05:49:51.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.189+0000 7f9cdb2cc640 1 -- 172.21.15.71:0/2441474663 wait complete. 2023-10-26T05:49:51.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T05:49:51.344 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_objectstore":"memstore"} 2023-10-26T05:49:51.345 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T05:49:51.345 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:49:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/580319605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0050005190 msgr2=0x7f005000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 --2- 172.21.15.71:0/580319605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0050005190 0x7f005000b990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f006000c5c0 tx=0x7f006001d340 comp rx=0 tx=0).stop 2023-10-26T05:49:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/580319605 shutdown_connections 2023-10-26T05:49:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 --2- 172.21.15.71:0/580319605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0050005190 0x7f005000b990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/580319605 >> 172.21.15.71:0/580319605 conn(0x7f00680caa30 msgr2=0x7f00680cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/580319605 shutdown_connections 2023-10-26T05:49:51.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/580319605 wait complete. 2023-10-26T05:49:51.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 Processor -- start 2023-10-26T05:49:51.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- start start 2023-10-26T05:49:51.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0050005190 0x7f00681ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 0x7f006813c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f006813ce80 0x7f006806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068151a60 con 0x7f0068129d00 2023-10-26T05:49:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00681ef670 con 0x7f0050005190 2023-10-26T05:49:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006813d300 con 0x7f006813ce80 2023-10-26T05:49:51.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 0x7f006813c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 0x7f006813c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48178/0 (socket says 172.21.15.71:48178) 2023-10-26T05:49:51.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 -- 172.21.15.71:0/1194143071 learned_addr learned my addr 172.21.15.71:0/1194143071 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:51.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f00677fe640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0050005190 0x7f00681ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0067fff640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f006813ce80 0x7f006806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 -- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f006813ce80 msgr2=0x7f006806dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f006813ce80 0x7f006806dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 -- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0050005190 msgr2=0x7f00681ef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0050005190 0x7f00681ef130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 -- 172.21.15.71:0/1194143071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068152e20 con 0x7f0068129d00 2023-10-26T05:49:51.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0067fff640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f006813ce80 0x7f006806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f0066ffd640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 0x7f006813c940 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d610 tx=0x7f005c00dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c0062e0 con 0x7f0068129d00 2023-10-26T05:49:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f005c006530 con 0x7f0068129d00 2023-10-26T05:49:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0068150c20 con 0x7f0068129d00 2023-10-26T05:49:51.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c003880 con 0x7f0068129d00 2023-10-26T05:49:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.465+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00681530c0 con 0x7f0068129d00 2023-10-26T05:49:51.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.469+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0068151140 con 0x7f0068129d00 2023-10-26T05:49:51.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.469+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f005c0039e0 con 0x7f0068129d00 2023-10-26T05:49:51.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.469+0000 7f004ffff640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f00540664a0 0x7f0054068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.469+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f005c0843d0 con 0x7f0068129d00 2023-10-26T05:49:51.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.469+0000 7f00677fe640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f00540664a0 0x7f0054068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.469+0000 7f00677fe640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f00540664a0 0x7f0054068960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f005800fa40 tx=0x7f0058005c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:51.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.473+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0068151140 con 0x7f0068129d00 2023-10-26T05:49:51.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.621+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f0068151140 con 0x7f0068129d00 2023-10-26T05:49:51.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f004ffff640 1 -- 172.21.15.71:0/1194143071 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v22) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f0068151140 con 0x7f0068129d00 2023-10-26T05:49:51.628 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:49:51.628 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:49:51.628 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:49:51.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:49:51.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:49:51.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:49:51.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:49:51.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:49:51.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:49:51.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:49:51.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:49:51.630 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:49:51.631 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:49:51.631 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:49:51.631 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:49:51.631 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:49:51.631 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:49:51.632 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:49:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f00540664a0 msgr2=0x7f0054068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f00540664a0 0x7f0054068960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f005800fa40 tx=0x7f0058005c60 comp rx=0 tx=0).stop 2023-10-26T05:49:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 msgr2=0x7f006813c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 0x7f006813c940 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f005c00d610 tx=0x7f005c00dad0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 shutdown_connections 2023-10-26T05:49:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f006813ce80 0x7f006806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f00540664a0 0x7f0054068960 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0068129d00 0x7f006813c940 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 --2- 172.21.15.71:0/1194143071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0050005190 0x7f00681ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 >> 172.21.15.71:0/1194143071 conn(0x7f00680caa30 msgr2=0x7f006813ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:51.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 shutdown_connections 2023-10-26T05:49:51.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.625+0000 7f006df03640 1 -- 172.21.15.71:0/1194143071 wait complete. 2023-10-26T05:49:51.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:49:51.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/4277733709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6558005260 msgr2=0x7f655800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 --2- 172.21.15.71:0/4277733709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6558005260 0x7f655800b990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f656c0afbc0 tx=0x7f656401d110 comp rx=0 tx=0).stop 2023-10-26T05:49:51.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/4277733709 shutdown_connections 2023-10-26T05:49:51.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 --2- 172.21.15.71:0/4277733709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6558005260 0x7f655800b990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/4277733709 >> 172.21.15.71:0/4277733709 conn(0x7f656c0aec60 msgr2=0x7f656c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:51.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/4277733709 shutdown_connections 2023-10-26T05:49:51.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/4277733709 wait complete. 2023-10-26T05:49:51.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 Processor -- start 2023-10-26T05:49:51.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- start start 2023-10-26T05:49:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6558005260 0x7f656c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 0x7f656c14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f656c0a66d0 0x7f656c0a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c14ee40 con 0x7f656c14e480 2023-10-26T05:49:51.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0a7470 con 0x7f6558005260 2023-10-26T05:49:51.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1f0b10 con 0x7f656c0a66d0 2023-10-26T05:49:51.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 0x7f656c14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 0x7f656c14e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52942/0 (socket says 172.21.15.71:52942) 2023-10-26T05:49:51.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 -- 172.21.15.71:0/877584078 learned_addr learned my addr 172.21.15.71:0/877584078 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:51.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f656a59c640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6558005260 0x7f656c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f656ad9d640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f656c0a66d0 0x7f656c0a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 -- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f656c0a66d0 msgr2=0x7f656c0a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f656c0a66d0 0x7f656c0a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 -- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6558005260 msgr2=0x7f656c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6558005260 0x7f656c0b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 -- 172.21.15.71:0/877584078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c1f0df0 con 0x7f656c14e480 2023-10-26T05:49:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f656a59c640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6558005260 0x7f656c0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:49:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6561d9b640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 0x7f656c14e900 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f655400d4d0 tx=0x7f655400d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554012630 con 0x7f656c14e480 2023-10-26T05:49:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6554012790 con 0x7f656c14e480 2023-10-26T05:49:51.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c1f10d0 con 0x7f656c14e480 2023-10-26T05:49:51.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554012b40 con 0x7f656c14e480 2023-10-26T05:49:51.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.757+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c0a7090 con 0x7f656c14e480 2023-10-26T05:49:51.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.761+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c066b10 con 0x7f656c14e480 2023-10-26T05:49:51.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.761+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f655401b400 con 0x7f656c14e480 2023-10-26T05:49:51.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.761+0000 7f65637fe640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f654c066470 0x7f654c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:51.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.761+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6554083f00 con 0x7f656c14e480 2023-10-26T05:49:51.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.761+0000 7f656a59c640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f654c066470 0x7f654c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:51.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.761+0000 7f656a59c640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f654c066470 0x7f654c068930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f655c0046e0 tx=0x7f655c004630 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:51.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.765+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6554051560 con 0x7f656c14e480 2023-10-26T05:49:51.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.905+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f656c0721e0 con 0x7f656c14e480 2023-10-26T05:49:51.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.913+0000 7f65637fe640 1 -- 172.21.15.71:0/877584078 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v23)=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6554054b80 con 0x7f656c14e480 2023-10-26T05:49:51.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f654c066470 msgr2=0x7f654c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f654c066470 0x7f654c068930 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f655c0046e0 tx=0x7f655c004630 comp rx=0 tx=0).stop 2023-10-26T05:49:51.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 msgr2=0x7f656c14e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:51.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 0x7f656c14e900 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f655400d4d0 tx=0x7f655400d990 comp rx=0 tx=0).stop 2023-10-26T05:49:51.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 shutdown_connections 2023-10-26T05:49:51.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f656c0a66d0 0x7f656c0a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f654c066470 0x7f654c068930 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c14e480 0x7f656c14e900 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 --2- 172.21.15.71:0/877584078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6558005260 0x7f656c0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:51.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 >> 172.21.15.71:0/877584078 conn(0x7f656c0aec60 msgr2=0x7f656c14f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:51.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 shutdown_connections 2023-10-26T05:49:51.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:51.917+0000 7f6570f42640 1 -- 172.21.15.71:0/877584078 wait complete. 2023-10-26T05:49:51.936 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:49:51.937 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:49:51.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:49:52.044 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:49:52.045 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:52.045 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:52.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:49:52.051 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:49:52.051 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:52.058 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:49:52.058 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:49:52.059 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:52.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:49:52.096 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:49:52.097 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:49:52.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:49:52.112 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:49:52.113 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:52.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/2118594966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff00b0750 msgr2=0x7f8ff015bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/2118594966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff00b0750 0x7f8ff015bd50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0152a90 tx=0x7f8fd801c750 comp rx=0 tx=0).stop 2023-10-26T05:49:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/2118594966 shutdown_connections 2023-10-26T05:49:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/2118594966 >> v2:172.21.15.86:3300/0 conn(0x7f8ff015c290 0x7f8ff00aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/2118594966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff00b0750 0x7f8ff015bd50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/2118594966 >> 172.21.15.71:0/2118594966 conn(0x7f8ff0069770 msgr2=0x7f8ff0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:52.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/2118594966 shutdown_connections 2023-10-26T05:49:52.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/2118594966 wait complete. 2023-10-26T05:49:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 Processor -- start 2023-10-26T05:49:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- start start 2023-10-26T05:49:52.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff00b0750 0x7f8ff0137740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 0x7f8ff0152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ff015c290 0x7f8ff0137eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0153490 con 0x7f8ff013e430 2023-10-26T05:49:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0138a20 con 0x7f8ff00b0750 2023-10-26T05:49:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0136980 con 0x7f8ff015c290 2023-10-26T05:49:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 0x7f8ff0152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 0x7f8ff0152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52966/0 (socket says 172.21.15.71:52966) 2023-10-26T05:49:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 -- 172.21.15.71:0/658699719 learned_addr learned my addr 172.21.15.71:0/658699719 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 -- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ff015c290 msgr2=0x7f8ff0137eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8feeffd640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff00b0750 0x7f8ff0137740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ff015c290 0x7f8ff0137eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 -- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff00b0750 msgr2=0x7f8ff0137740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff00b0750 0x7f8ff0137740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 -- 172.21.15.71:0/658699719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd80094a0 con 0x7f8ff013e430 2023-10-26T05:49:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fee7fc640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 0x7f8ff0152f50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8019360 tx=0x7f8fd80a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80043c0 con 0x7f8ff013e430 2023-10-26T05:49:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8fd8003600 con 0x7f8ff013e430 2023-10-26T05:49:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff0136c60 con 0x7f8ff013e430 2023-10-26T05:49:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80039b0 con 0x7f8ff013e430 2023-10-26T05:49:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff0137220 con 0x7f8ff013e430 2023-10-26T05:49:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.249+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff0066b10 con 0x7f8ff013e430 2023-10-26T05:49:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.253+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8fd8003b10 con 0x7f8ff013e430 2023-10-26T05:49:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.253+0000 7f8fe77fe640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd4066470 0x7f8fd4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.253+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8fd80aa070 con 0x7f8ff013e430 2023-10-26T05:49:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.253+0000 7f8feeffd640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd4066470 0x7f8fd4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.253+0000 7f8feeffd640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd4066470 0x7f8fd4068930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0007fd0 tx=0x7f8fe0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:52.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.257+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8fd80ed1c0 con 0x7f8ff013e430 2023-10-26T05:49:52.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff0136ac0 con 0x7f8ff013e430 2023-10-26T05:49:52.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8fe77fe640 1 -- 172.21.15.71:0/658699719 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f8fd80bee90 con 0x7f8ff013e430 2023-10-26T05:49:52.441 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:52.442 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4862,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3265654009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3265654009},{"type":"v1","addr":"172.21.15.86:6835","nonce":3265654009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5086,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3275083751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3275083751},{"type":"v1","addr":"172.21.15.86:6837","nonce":3275083751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":5775,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1654745470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1654745470},{"type":"v1","addr":"172.21.15.71:6833","nonce":1654745470}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15},{"gid":6171,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/2352544424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2352544424},{"type":"v1","addr":"172.21.15.71:6835","nonce":2352544424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-10-26T05:49:52.442 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 20 2023-10-26T05:49:52.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd4066470 msgr2=0x7f8fd4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd4066470 0x7f8fd4068930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0007fd0 tx=0x7f8fe0009040 comp rx=0 tx=0).stop 2023-10-26T05:49:52.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 msgr2=0x7f8ff0152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 0x7f8ff0152f50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8019360 tx=0x7f8fd80a67b0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 shutdown_connections 2023-10-26T05:49:52.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8ff015c290 0x7f8ff0137eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8fd4066470 0x7f8fd4068930 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff013e430 0x7f8ff0152f50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 --2- 172.21.15.71:0/658699719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ff00b0750 0x7f8ff0137740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 >> 172.21.15.71:0/658699719 conn(0x7f8ff0069770 msgr2=0x7f8ff015b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:52.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 shutdown_connections 2023-10-26T05:49:52.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.437+0000 7f8ff52c6640 1 -- 172.21.15.71:0/658699719 wait complete. 2023-10-26T05:49:52.459 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:49:52.459 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:49:52.460 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:49:52.460 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:49:52.460 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:49:52.461 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:49:52.461 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:49:52.461 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:49:52.462 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:49:52.462 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:49:52.462 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:49:52.462 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:49:52.463 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:49:52.463 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:49:52.463 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:49:52.463 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:49:52.471 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:49:52.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:49:52.472 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:49:52.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:49:52.474 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:49:52.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:49:52.478 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:49:52.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:49:52.513 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:49:52.514 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:49:52.514 DEBUG:teuthology.parallel:result is None 2023-10-26T05:49:52.515 DEBUG:teuthology.parallel:result is None 2023-10-26T05:49:52.519 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:49:52.520 DEBUG:teuthology.parallel:result is None 2023-10-26T05:49:52.521 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:49:52.521 DEBUG:teuthology.parallel:result is None 2023-10-26T05:49:52.522 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:49:52.574 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:49:52.575 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:49:52.588 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:49:52.594 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:49:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 -- 172.21.15.71:0/1812393901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 msgr2=0x7fc2b40b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/1812393901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40b43e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4072e20 tx=0x7fc2a401f090 comp rx=0 tx=0).stop 2023-10-26T05:49:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 -- 172.21.15.71:0/1812393901 shutdown_connections 2023-10-26T05:49:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/1812393901 >> v2:172.21.15.86:3300/0 conn(0x7fc2b406a390 0x7fc2b406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/1812393901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40b43e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 -- 172.21.15.71:0/1812393901 >> 172.21.15.71:0/1812393901 conn(0x7fc2b40d12d0 msgr2=0x7fc2b40d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:52.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 -- 172.21.15.71:0/1812393901 shutdown_connections 2023-10-26T05:49:52.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.625+0000 7fc2bbe08640 1 -- 172.21.15.71:0/1812393901 wait complete. 2023-10-26T05:49:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 Processor -- start 2023-10-26T05:49:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- start start 2023-10-26T05:49:52.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2b406a390 0x7fc2b41f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2b40cc7c0 0x7fc2b40ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4154af0 con 0x7fc2b415b3b0 2023-10-26T05:49:52.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b4155350 con 0x7fc2b40cc7c0 2023-10-26T05:49:52.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40cb550 con 0x7fc2b406a390 2023-10-26T05:49:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53000/0 (socket says 172.21.15.71:53000) 2023-10-26T05:49:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 -- 172.21.15.71:0/4194819254 learned_addr learned my addr 172.21.15.71:0/4194819254 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2b9ba4640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2b406a390 0x7fc2b41f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2b93a3640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2b40cc7c0 0x7fc2b40ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 -- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2b406a390 msgr2=0x7fc2b41f1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2b406a390 0x7fc2b41f1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 -- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2b40cc7c0 msgr2=0x7fc2b40ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2b40cc7c0 0x7fc2b40ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:52.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 -- 172.21.15.71:0/4194819254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4009560 con 0x7fc2b415b3b0 2023-10-26T05:49:52.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2ba3a5640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40cacf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c770 tx=0x7fc2ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:52.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac003b80 con 0x7fc2b415b3b0 2023-10-26T05:49:52.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc2ac003ce0 con 0x7fc2b415b3b0 2023-10-26T05:49:52.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b4154d00 con 0x7fc2b415b3b0 2023-10-26T05:49:52.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00aab0 con 0x7fc2b415b3b0 2023-10-26T05:49:52.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b41f1b80 con 0x7fc2b415b3b0 2023-10-26T05:49:52.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b4066b10 con 0x7fc2b415b3b0 2023-10-26T05:49:52.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.629+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc2ac00ac10 con 0x7fc2b415b3b0 2023-10-26T05:49:52.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.633+0000 7fc2aa7fc640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc298066470 0x7fc298068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:52.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.633+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc2ac083a20 con 0x7fc2b415b3b0 2023-10-26T05:49:52.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.633+0000 7fc2b9ba4640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc298066470 0x7fc298068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:52.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.633+0000 7fc2b9ba4640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc298066470 0x7fc298068930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40b2000 tx=0x7fc2a401d8d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:52.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.637+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc2ac051250 con 0x7fc2b415b3b0 2023-10-26T05:49:52.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:52.785+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc2b41554c0 con 0x7fc2b415b3b0 2023-10-26T05:49:53.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.221+0000 7fc2aa7fc640 1 -- 172.21.15.71:0/4194819254 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc2ac054870 con 0x7fc2b415b3b0 2023-10-26T05:49:53.227 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:49:53.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc298066470 msgr2=0x7fc298068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc298066470 0x7fc298068930 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40b2000 tx=0x7fc2a401d8d0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 msgr2=0x7fc2b40cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40cacf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00c770 tx=0x7fc2ac00cc30 comp rx=0 tx=0).stop 2023-10-26T05:49:53.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 shutdown_connections 2023-10-26T05:49:53.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc298066470 0x7fc298068930 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b415b3b0 0x7fc2b40cacf0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2b40cc7c0 0x7fc2b40ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 --2- 172.21.15.71:0/4194819254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2b406a390 0x7fc2b41f1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 >> 172.21.15.71:0/4194819254 conn(0x7fc2b40d12d0 msgr2=0x7fc2b415a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:53.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 shutdown_connections 2023-10-26T05:49:53.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.225+0000 7fc2bbe08640 1 -- 172.21.15.71:0/4194819254 wait complete. 2023-10-26T05:49:53.253 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:49:53.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/841822151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 msgr2=0x7f93680047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 --2- 172.21.15.71:0/841822151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93680047d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f9374009b60 tx=0x7f937401d250 comp rx=0 tx=0).stop 2023-10-26T05:49:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937ed9d640 1 -- 172.21.15.71:0/841822151 reap_dead start 2023-10-26T05:49:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/841822151 shutdown_connections 2023-10-26T05:49:53.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/841822151 >> 172.21.15.71:0/841822151 conn(0x7f9380069770 msgr2=0x7f9380069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/841822151 shutdown_connections 2023-10-26T05:49:53.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/841822151 wait complete. 2023-10-26T05:49:53.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 Processor -- start 2023-10-26T05:49:53.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- start start 2023-10-26T05:49:53.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93801576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93801f5790 0x7f93801f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93801f6150 0x7f9380135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9380135980 con 0x7f9368003f30 2023-10-26T05:49:53.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801f65d0 con 0x7f93801f5790 2023-10-26T05:49:53.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9380159a80 con 0x7f93801f6150 2023-10-26T05:49:53.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93801576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93801576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53020/0 (socket says 172.21.15.71:53020) 2023-10-26T05:49:53.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 -- 172.21.15.71:0/4123598877 learned_addr learned my addr 172.21.15.71:0/4123598877 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:53.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937dd9b640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93801f5790 0x7f93801f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937ed9d640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93801f6150 0x7f9380135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 -- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93801f6150 msgr2=0x7f9380135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93801f6150 0x7f9380135e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 -- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93801f5790 msgr2=0x7f93801f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93801f5790 0x7f93801f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 -- 172.21.15.71:0/4123598877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380159d60 con 0x7f9368003f30 2023-10-26T05:49:53.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f937e59c640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93801576c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f937000d5a0 tx=0x7f937000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:53.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370012630 con 0x7f9368003f30 2023-10-26T05:49:53.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9370012790 con 0x7f9368003f30 2023-10-26T05:49:53.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938015a040 con 0x7f9368003f30 2023-10-26T05:49:53.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370012be0 con 0x7f9368003f30 2023-10-26T05:49:53.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.377+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938015a600 con 0x7f9368003f30 2023-10-26T05:49:53.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.381+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9380066b10 con 0x7f9368003f30 2023-10-26T05:49:53.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.381+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9370012d40 con 0x7f9368003f30 2023-10-26T05:49:53.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.381+0000 7f936effd640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9360066470 0x7f9360068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.381+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9370083a70 con 0x7f9368003f30 2023-10-26T05:49:53.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.381+0000 7f937dd9b640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9360066470 0x7f9360068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.381+0000 7f937dd9b640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9360066470 0x7f9360068930 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f93740ae000 tx=0x7f9374019ff0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:53.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.385+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9370051240 con 0x7f9368003f30 2023-10-26T05:49:53.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.593+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f93800721e0 con 0x7f9368003f30 2023-10-26T05:49:53.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.593+0000 7f936effd640 1 -- 172.21.15.71:0/4123598877 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9370054860 con 0x7f9368003f30 2023-10-26T05:49:53.600 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:53.600 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:49:53.601 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:49:53.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.601 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:49:53.601 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:49:53.601 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.602 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.602 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:49:53.602 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:49:53.602 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.603 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.603 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.603 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.603 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:49:53.603 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:49:53.603 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.604 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.604 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:49:53.604 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:49:53.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.605 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.605 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:49:53.605 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:49:53.605 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.606 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.606 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:49:53.606 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:49:53.606 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.607 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.607 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:49:53.607 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:49:53.607 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.608 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.608 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:49:53.608 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:49:53.608 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.608 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.609 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.609 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:49:53.609 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:49:53.609 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.609 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.610 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.610 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.611 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:49:53.611 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:49:53.611 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.611 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.611 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.612 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.612 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:49:53.613 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:49:53.613 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.613 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.613 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.613 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.614 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.614 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:49:53.615 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:49:53.615 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.615 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.615 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.615 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.615 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.616 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:49:53.616 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:49:53.617 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.617 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.617 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.617 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.617 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.618 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:49:53.618 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:49:53.618 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.618 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.619 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.619 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.620 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:49:53.620 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:49:53.620 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.621 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.621 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:49:53.621 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:49:53.622 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:49:53.622 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:49:53.622 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.623 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:49:53.623 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:49:53.624 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.624 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.624 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:49:53.624 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:49:53.625 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.625 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:49:53.625 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:49:53.626 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.626 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.626 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:49:53.626 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:49:53.626 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:49:53.626 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.627 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.627 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.627 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:49:53.627 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:49:53.627 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:49:53.628 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:49:53.628 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:49:53.628 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.629 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:49:53.629 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:49:53.629 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.629 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:49:53.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:49:53.630 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:49:53.630 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:49:53.630 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.631 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.631 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:49:53.631 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:49:53.631 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.631 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:49:53.632 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:49:53.632 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:49:53.632 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.633 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:49:53.633 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:49:53.633 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.633 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:49:53.633 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:49:53.634 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:49:53.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:53.634 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:49:53.635 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:49:53.635 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:49:53.635 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:49:53.635 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:49:53.635 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:49:53.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:53.636 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:49:53.636 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:49:53.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9360066470 msgr2=0x7f9360068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9360066470 0x7f9360068930 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f93740ae000 tx=0x7f9374019ff0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 msgr2=0x7f93801576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93801576c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f937000d5a0 tx=0x7f937000da60 comp rx=0 tx=0).stop 2023-10-26T05:49:53.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 shutdown_connections 2023-10-26T05:49:53.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93801f6150 0x7f9380135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93801f5790 0x7f93801f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9360066470 0x7f9360068930 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 --2- 172.21.15.71:0/4123598877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9368003f30 0x7f93801576c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 >> 172.21.15.71:0/4123598877 conn(0x7f9380069770 msgr2=0x7f938013fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:53.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 shutdown_connections 2023-10-26T05:49:53.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.597+0000 7f9384d5a640 1 -- 172.21.15.71:0/4123598877 wait complete. 2023-10-26T05:49:53.641 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- 172.21.15.71:0/2036055580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb815bbc0 msgr2=0x7f5eb8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/2036055580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb815bbc0 0x7f5eb8070690 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009900 tx=0x7f5eb001c8c0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- 172.21.15.71:0/2036055580 shutdown_connections 2023-10-26T05:49:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/2036055580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb815bbc0 0x7f5eb8070690 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/2036055580 >> v2:172.21.15.86:3300/0 conn(0x7f5eb813d420 0x7f5eb80ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- 172.21.15.71:0/2036055580 >> 172.21.15.71:0/2036055580 conn(0x7f5eb8069770 msgr2=0x7f5eb8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- 172.21.15.71:0/2036055580 shutdown_connections 2023-10-26T05:49:53.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- 172.21.15.71:0/2036055580 wait complete. 2023-10-26T05:49:53.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 Processor -- start 2023-10-26T05:49:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- start start 2023-10-26T05:49:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb813d420 0x7f5eb81265c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb813d800 0x7f5eb8126b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 0x7f5eb81294e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8128ca0 con 0x7f5eb815bbc0 2023-10-26T05:49:53.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8128090 con 0x7f5eb813d800 2023-10-26T05:49:53.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5ebcc55640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb8128370 con 0x7f5eb813d420 2023-10-26T05:49:53.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5eb6d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 0x7f5eb81294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5eb6d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 0x7f5eb81294e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53034/0 (socket says 172.21.15.71:53034) 2023-10-26T05:49:53.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5eb6d9d640 1 -- 172.21.15.71:0/1532857277 learned_addr learned my addr 172.21.15.71:0/1532857277 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:53.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5eb659c640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb813d420 0x7f5eb81265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.757+0000 7f5eb5d9b640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb813d800 0x7f5eb8126b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb6d9d640 1 -- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb813d420 msgr2=0x7f5eb81265c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb6d9d640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb813d420 0x7f5eb81265c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb6d9d640 1 -- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb813d800 msgr2=0x7f5eb8126b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb6d9d640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb813d800 0x7f5eb8126b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb6d9d640 1 -- 172.21.15.71:0/1532857277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0009560 con 0x7f5eb815bbc0 2023-10-26T05:49:53.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb6d9d640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 0x7f5eb81294e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009850 tx=0x7f5eb0005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:53.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb00bb070 con 0x7f5eb815bbc0 2023-10-26T05:49:53.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5eb00a57e0 con 0x7f5eb815bbc0 2023-10-26T05:49:53.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb8128650 con 0x7f5eb815bbc0 2023-10-26T05:49:53.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0003860 con 0x7f5eb815bbc0 2023-10-26T05:49:53.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb813c4e0 con 0x7f5eb815bbc0 2023-10-26T05:49:53.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb8066b10 con 0x7f5eb815bbc0 2023-10-26T05:49:53.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5eb00b4040 con 0x7f5eb815bbc0 2023-10-26T05:49:53.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ea6ffd640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:53.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5eb011dce0 con 0x7f5eb815bbc0 2023-10-26T05:49:53.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb659c640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:53.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.761+0000 7f5eb659c640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5eb815ac20 tx=0x7f5ea8009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:53.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.765+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5eb00eb4c0 con 0x7f5eb815bbc0 2023-10-26T05:49:53.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.905+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eb8066b10 con 0x7f5eb815bbc0 2023-10-26T05:49:53.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ea6ffd640 1 -- 172.21.15.71:0/1532857277 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f5eb00eeae0 con 0x7f5eb815bbc0 2023-10-26T05:49:53.913 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:53.913 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:53.236870+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:53.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 msgr2=0x7f5e98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5eb815ac20 tx=0x7f5ea8009500 comp rx=0 tx=0).stop 2023-10-26T05:49:53.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 msgr2=0x7f5eb81294e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:53.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 0x7f5eb81294e0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009850 tx=0x7f5eb0005d60 comp rx=0 tx=0).stop 2023-10-26T05:49:53.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 shutdown_connections 2023-10-26T05:49:53.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e98066470 0x7f5e98068930 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5eb815bbc0 0x7f5eb81294e0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5eb813d800 0x7f5eb8126b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 --2- 172.21.15.71:0/1532857277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5eb813d420 0x7f5eb81265c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:53.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 >> 172.21.15.71:0/1532857277 conn(0x7f5eb8069770 msgr2=0x7f5eb806a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:53.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 shutdown_connections 2023-10-26T05:49:53.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:53.909+0000 7f5ebcc55640 1 -- 172.21.15.71:0/1532857277 wait complete. 2023-10-26T05:49:53.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:49:53.931 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:49:54.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- 172.21.15.71:0/1452557571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 msgr2=0x7fb93815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/1452557571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb93815b3a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb93808a140 tx=0x7fb92c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- 172.21.15.71:0/1452557571 shutdown_connections 2023-10-26T05:49:54.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/1452557571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb93815b3a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/1452557571 >> v2:172.21.15.86:3300/0 conn(0x7fb9380d0470 0x7fb93806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- 172.21.15.71:0/1452557571 >> 172.21.15.71:0/1452557571 conn(0x7fb9380cdd80 msgr2=0x7fb9380ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:54.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- 172.21.15.71:0/1452557571 shutdown_connections 2023-10-26T05:49:54.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- 172.21.15.71:0/1452557571 wait complete. 2023-10-26T05:49:54.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 Processor -- start 2023-10-26T05:49:54.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- start start 2023-10-26T05:49:54.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9380cd880 0x7fb93814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9380d0470 0x7fb9381526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb9380b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815e350 con 0x7fb9380d0850 2023-10-26T05:49:54.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815d740 con 0x7fb9380cd880 2023-10-26T05:49:54.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.057+0000 7fb93ce1a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93815da20 con 0x7fb9380d0470 2023-10-26T05:49:54.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb9380b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb9380b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53050/0 (socket says 172.21.15.71:53050) 2023-10-26T05:49:54.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 -- 172.21.15.71:0/4098811678 learned_addr learned my addr 172.21.15.71:0/4098811678 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:54.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93659c640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9380cd880 0x7fb93814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb935d9b640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9380d0470 0x7fb9381526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 -- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9380d0470 msgr2=0x7fb9381526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9380d0470 0x7fb9381526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 -- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9380cd880 msgr2=0x7fb93814edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9380cd880 0x7fb93814edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 -- 172.21.15.71:0/4098811678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c009560 con 0x7fb9380d0850 2023-10-26T05:49:54.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93659c640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9380cd880 0x7fb93814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:54.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb935d9b640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9380d0470 0x7fb9381526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:54.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb936d9d640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb9380b17b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb93000c780 tx=0x7fb93000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:54.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930004b70 con 0x7fb9380d0850 2023-10-26T05:49:54.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb93000ce90 con 0x7fb9380d0850 2023-10-26T05:49:54.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93815dd00 con 0x7fb9380d0850 2023-10-26T05:49:54.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93000aab0 con 0x7fb9380d0850 2023-10-26T05:49:54.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93814f2f0 con 0x7fb9380d0850 2023-10-26T05:49:54.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938066b10 con 0x7fb9380d0850 2023-10-26T05:49:54.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb93000ac10 con 0x7fb9380d0850 2023-10-26T05:49:54.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb926ffd640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb918066470 0x7fb918068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb930083f50 con 0x7fb9380d0850 2023-10-26T05:49:54.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93659c640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb918066470 0x7fb918068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.061+0000 7fb93659c640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb918066470 0x7fb918068930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb928005fd0 tx=0x7fb928005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:54.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.065+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb930051790 con 0x7fb9380d0850 2023-10-26T05:49:54.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.217+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9380721e0 con 0x7fb9380d0850 2023-10-26T05:49:54.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb926ffd640 1 -- 172.21.15.71:0/4098811678 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb930054db0 con 0x7fb9380d0850 2023-10-26T05:49:54.225 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:49:54.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb918066470 msgr2=0x7fb918068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb918066470 0x7fb918068930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb928005fd0 tx=0x7fb928005e60 comp rx=0 tx=0).stop 2023-10-26T05:49:54.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 msgr2=0x7fb9380b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb9380b17b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb93000c780 tx=0x7fb93000cc40 comp rx=0 tx=0).stop 2023-10-26T05:49:54.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 shutdown_connections 2023-10-26T05:49:54.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb918066470 0x7fb918068930 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9380d0850 0x7fb9380b17b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9380d0470 0x7fb9381526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 --2- 172.21.15.71:0/4098811678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9380cd880 0x7fb93814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 >> 172.21.15.71:0/4098811678 conn(0x7fb9380cdd80 msgr2=0x7fb938154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:54.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 shutdown_connections 2023-10-26T05:49:54.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.221+0000 7fb93ce1a640 1 -- 172.21.15.71:0/4098811678 wait complete. 2023-10-26T05:49:54.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:49:54.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- 172.21.15.71:0/1593166774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a80c9c10 msgr2=0x7f15a80cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 --2- 172.21.15.71:0/1593166774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a80c9c10 0x7f15a80cbb40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c01c7f0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- 172.21.15.71:0/1593166774 shutdown_connections 2023-10-26T05:49:54.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 --2- 172.21.15.71:0/1593166774 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15a80cc080 0x7f15a806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 --2- 172.21.15.71:0/1593166774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a80c9c10 0x7f15a80cbb40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- 172.21.15.71:0/1593166774 >> 172.21.15.71:0/1593166774 conn(0x7f15a806fdc0 msgr2=0x7f15a80b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:54.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- 172.21.15.71:0/1593166774 shutdown_connections 2023-10-26T05:49:54.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- 172.21.15.71:0/1593166774 wait complete. 2023-10-26T05:49:54.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 Processor -- start 2023-10-26T05:49:54.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- start start 2023-10-26T05:49:54.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15a80c9c10 0x7f15a814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a80cc080 0x7f15a81f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 0x7f15a8151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a814dc30 con 0x7f15a8159330 2023-10-26T05:49:54.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81f8e90 con 0x7f15a80cc080 2023-10-26T05:49:54.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15afd48640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81500b0 con 0x7f15a80c9c10 2023-10-26T05:49:54.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15ae2e5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 0x7f15a8151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15ae2e5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 0x7f15a8151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53072/0 (socket says 172.21.15.71:53072) 2023-10-26T05:49:54.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15ae2e5640 1 -- 172.21.15.71:0/2672783614 learned_addr learned my addr 172.21.15.71:0/2672783614 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:54.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.349+0000 7f15adae4640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15a80c9c10 0x7f15a814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ad2e3640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a80cc080 0x7f15a81f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ae2e5640 1 -- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15a80c9c10 msgr2=0x7f15a814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ae2e5640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15a80c9c10 0x7f15a814d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ae2e5640 1 -- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a80cc080 msgr2=0x7f15a81f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ae2e5640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a80cc080 0x7f15a81f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ae2e5640 1 -- 172.21.15.71:0/2672783614 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c009560 con 0x7f15a8159330 2023-10-26T05:49:54.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15ae2e5640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 0x7f15a8151150 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f159800c970 tx=0x7f159800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:54.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598004220 con 0x7f15a8159330 2023-10-26T05:49:54.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1598004470 con 0x7f15a8159330 2023-10-26T05:49:54.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a8150390 con 0x7f15a8159330 2023-10-26T05:49:54.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598007c10 con 0x7f15a8159330 2023-10-26T05:49:54.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a8150950 con 0x7f15a8159330 2023-10-26T05:49:54.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a8066d00 con 0x7f15a8159330 2023-10-26T05:49:54.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1598003600 con 0x7f15a8159330 2023-10-26T05:49:54.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15a67fc640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1594066470 0x7f1594068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1598083c70 con 0x7f15a8159330 2023-10-26T05:49:54.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15adae4640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1594066470 0x7f1594068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.353+0000 7f15adae4640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1594066470 0x7f1594068930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f15a80ce250 tx=0x7f15a00094d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:54.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.357+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1598051440 con 0x7f15a8159330 2023-10-26T05:49:54.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.513+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f15a806d100 con 0x7f15a8159330 2023-10-26T05:49:54.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.513+0000 7f15a67fc640 1 -- 172.21.15.71:0/2672783614 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+65 (secure 0 0 0) 0x7f159801d090 con 0x7f15a8159330 2023-10-26T05:49:54.517 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:54.518 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:49:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:49:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:49:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:49:54.518 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:49:54.519 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:49:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1594066470 msgr2=0x7f1594068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1594066470 0x7f1594068930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f15a80ce250 tx=0x7f15a00094d0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 msgr2=0x7f15a8151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 0x7f15a8151150 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f159800c970 tx=0x7f159800ce30 comp rx=0 tx=0).stop 2023-10-26T05:49:54.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 shutdown_connections 2023-10-26T05:49:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1594066470 0x7f1594068930 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15a8159330 0x7f15a8151150 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15a80cc080 0x7f15a81f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 --2- 172.21.15.71:0/2672783614 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15a80c9c10 0x7f15a814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 >> 172.21.15.71:0/2672783614 conn(0x7f15a806fdc0 msgr2=0x7f15a8156270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:54.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 shutdown_connections 2023-10-26T05:49:54.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.517+0000 7f15afd48640 1 -- 172.21.15.71:0/2672783614 wait complete. 2023-10-26T05:49:54.536 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:49:54.537 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:49:54.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- 172.21.15.71:0/910542079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec0ce990 msgr2=0x7f62ec15cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- 172.21.15.71:0/910542079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec0ce990 0x7f62ec15cd90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0b1cd0 tx=0x7f62e401d440 comp rx=0 tx=0).stop 2023-10-26T05:49:54.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- 172.21.15.71:0/910542079 shutdown_connections 2023-10-26T05:49:54.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- 172.21.15.71:0/910542079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec0ce990 0x7f62ec15cd90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- 172.21.15.71:0/910542079 >> v2:172.21.15.86:3300/0 conn(0x7f62ec06edf0 0x7f62ec0cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- 172.21.15.71:0/910542079 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62ec159230 0x7f62ec0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- 172.21.15.71:0/910542079 >> 172.21.15.71:0/910542079 conn(0x7f62ec06fd90 msgr2=0x7f62ec0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:54.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- 172.21.15.71:0/910542079 shutdown_connections 2023-10-26T05:49:54.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- 172.21.15.71:0/910542079 wait complete. 2023-10-26T05:49:54.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 Processor -- start 2023-10-26T05:49:54.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- start start 2023-10-26T05:49:54.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec06edf0 0x7f62ec151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62ec0ce990 0x7f62ec157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 0x7f62ec1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec151cc0 con 0x7f62ec159230 2023-10-26T05:49:54.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b2740 con 0x7f62ec06edf0 2023-10-26T05:49:54.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62f1acb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0b2a20 con 0x7f62ec0ce990 2023-10-26T05:49:54.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 0x7f62ec1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 0x7f62ec1584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53082/0 (socket says 172.21.15.71:53082) 2023-10-26T05:49:54.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 -- 172.21.15.71:0/470997028 learned_addr learned my addr 172.21.15.71:0/470997028 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:54.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62eb7fe640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec06edf0 0x7f62ec151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62eaffd640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62ec0ce990 0x7f62ec157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 -- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62ec0ce990 msgr2=0x7f62ec157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62ec0ce990 0x7f62ec157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 -- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec06edf0 msgr2=0x7f62ec151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:54.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec06edf0 0x7f62ec151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:54.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 -- 172.21.15.71:0/470997028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40098e0 con 0x7f62ec159230 2023-10-26T05:49:54.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.653+0000 7f62ebfff640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 0x7f62ec1584b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f62e401ca20 tx=0x7f62e40a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:54.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4004410 con 0x7f62ec159230 2023-10-26T05:49:54.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f62e40a5a70 con 0x7f62ec159230 2023-10-26T05:49:54.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec0b2d00 con 0x7f62ec159230 2023-10-26T05:49:54.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40ad440 con 0x7f62ec159230 2023-10-26T05:49:54.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec0b32c0 con 0x7f62ec159230 2023-10-26T05:49:54.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec0b34b0 con 0x7f62ec159230 2023-10-26T05:49:54.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f62e40b4040 con 0x7f62ec159230 2023-10-26T05:49:54.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62d3fff640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f62d8066470 0x7f62d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:54.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f62e40a5e40 con 0x7f62ec159230 2023-10-26T05:49:54.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62eb7fe640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f62d8066470 0x7f62d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:54.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.657+0000 7f62eb7fe640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f62d8066470 0x7f62d8068930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005990 tx=0x7f62e0005900 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:54.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.661+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f62e40ec310 con 0x7f62ec159230 2023-10-26T05:49:54.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:54.813+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f62ec066d00 con 0x7f62ec159230 2023-10-26T05:49:55.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.245+0000 7f62d3fff640 1 -- 172.21.15.71:0/470997028 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f62e40b2020 con 0x7f62ec159230 2023-10-26T05:49:55.250 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:49:55.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f62d8066470 msgr2=0x7f62d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:55.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f62d8066470 0x7f62d8068930 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f62e0005990 tx=0x7f62e0005900 comp rx=0 tx=0).stop 2023-10-26T05:49:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 msgr2=0x7f62ec1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 0x7f62ec1584b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f62e401ca20 tx=0x7f62e40a5880 comp rx=0 tx=0).stop 2023-10-26T05:49:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 shutdown_connections 2023-10-26T05:49:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f62d8066470 0x7f62d8068930 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec159230 0x7f62ec1584b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f62ec0ce990 0x7f62ec157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 --2- 172.21.15.71:0/470997028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62ec06edf0 0x7f62ec151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 >> 172.21.15.71:0/470997028 conn(0x7f62ec06fd90 msgr2=0x7f62ec0af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:55.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 shutdown_connections 2023-10-26T05:49:55.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.249+0000 7f62f1acb640 1 -- 172.21.15.71:0/470997028 wait complete. 2023-10-26T05:49:55.277 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:49:55.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- 172.21.15.71:0/1759787505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c813c190 msgr2=0x7f31c80cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 --2- 172.21.15.71:0/1759787505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c813c190 0x7f31c80cb820 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009900 tx=0x7f31b801c850 comp rx=0 tx=0).stop 2023-10-26T05:49:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- 172.21.15.71:0/1759787505 shutdown_connections 2023-10-26T05:49:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 --2- 172.21.15.71:0/1759787505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c813c190 0x7f31c80cb820 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- 172.21.15.71:0/1759787505 >> 172.21.15.71:0/1759787505 conn(0x7f31c80aec60 msgr2=0x7f31c80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- 172.21.15.71:0/1759787505 shutdown_connections 2023-10-26T05:49:55.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- 172.21.15.71:0/1759787505 wait complete. 2023-10-26T05:49:55.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 Processor -- start 2023-10-26T05:49:55.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- start start 2023-10-26T05:49:55.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 0x7f31c81f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:55.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f31c813c190 0x7f31c81361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:55.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c813efd0 0x7f31c815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:55.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c81f4c90 con 0x7f31c80cbd60 2023-10-26T05:49:55.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8139810 con 0x7f31c813efd0 2023-10-26T05:49:55.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31ceb79640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8139af0 con 0x7f31c813c190 2023-10-26T05:49:55.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 0x7f31c81f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:55.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 0x7f31c81f4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53090/0 (socket says 172.21.15.71:53090) 2023-10-26T05:49:55.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 -- 172.21.15.71:0/2526915482 learned_addr learned my addr 172.21.15.71:0/2526915482 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:55.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31c7fff640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f31c813c190 0x7f31c81361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:55.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cd116640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c813efd0 0x7f31c815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:55.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 -- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f31c813c190 msgr2=0x7f31c81361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:55.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f31c813c190 0x7f31c81361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 -- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c813efd0 msgr2=0x7f31c815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:55.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c813efd0 0x7f31c815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:55.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 -- 172.21.15.71:0/2526915482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8009560 con 0x7f31c80cbd60 2023-10-26T05:49:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cd116640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c813efd0 0x7f31c815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31cc915640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 0x7f31c81f4750 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f31b8019460 tx=0x7f31b80a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8003800 con 0x7f31c80cbd60 2023-10-26T05:49:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31b8003960 con 0x7f31c80cbd60 2023-10-26T05:49:55.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.413+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8003d70 con 0x7f31c80cbd60 2023-10-26T05:49:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c8139dd0 con 0x7f31c80cbd60 2023-10-26T05:49:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c813a390 con 0x7f31c80cbd60 2023-10-26T05:49:55.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c8066b10 con 0x7f31c80cbd60 2023-10-26T05:49:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31b80b4040 con 0x7f31c80cbd60 2023-10-26T05:49:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31c57fa640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31b4066470 0x7f31b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:55.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f31b811db60 con 0x7f31c80cbd60 2023-10-26T05:49:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31c7fff640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31b4066470 0x7f31b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.417+0000 7f31c7fff640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31b4066470 0x7f31b4068930 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f31c8137160 tx=0x7f31bc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:55.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.421+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f31b80eb1c0 con 0x7f31c80cbd60 2023-10-26T05:49:55.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:55.565+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f31c80721e0 con 0x7f31c80cbd60 2023-10-26T05:49:56.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.257+0000 7f31c57fa640 1 -- 172.21.15.71:0/2526915482 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7f31b80ee7e0 con 0x7f31c80cbd60 2023-10-26T05:49:56.264 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:49:56.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31b4066470 msgr2=0x7f31b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:56.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31b4066470 0x7f31b4068930 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f31c8137160 tx=0x7f31bc009040 comp rx=0 tx=0).stop 2023-10-26T05:49:56.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 msgr2=0x7f31c81f4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:56.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 0x7f31c81f4750 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f31b8019460 tx=0x7f31b80a5720 comp rx=0 tx=0).stop 2023-10-26T05:49:56.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 shutdown_connections 2023-10-26T05:49:56.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31c813efd0 0x7f31c815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f31c813c190 0x7f31c81361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31b4066470 0x7f31b4068930 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 --2- 172.21.15.71:0/2526915482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31c80cbd60 0x7f31c81f4750 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 >> 172.21.15.71:0/2526915482 conn(0x7f31c80aec60 msgr2=0x7f31c8070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:56.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 shutdown_connections 2023-10-26T05:49:56.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.261+0000 7f31ceb79640 1 -- 172.21.15.71:0/2526915482 wait complete. 2023-10-26T05:49:56.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:49:56.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/2634184716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 msgr2=0x7fc6c406a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:56.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/2634184716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c406a440 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc01c850 comp rx=0 tx=0).stop 2023-10-26T05:49:56.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/2634184716 shutdown_connections 2023-10-26T05:49:56.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/2634184716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c406a440 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/2634184716 >> 172.21.15.71:0/2634184716 conn(0x7fc6c40aec60 msgr2=0x7fc6c40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:56.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/2634184716 shutdown_connections 2023-10-26T05:49:56.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/2634184716 wait complete. 2023-10-26T05:49:56.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 Processor -- start 2023-10-26T05:49:56.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- start start 2023-10-26T05:49:56.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c413c190 0x7fc6c413f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:56.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6c413e8c0 0x7fc6c4149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:56.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c413b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:56.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40b13c0 con 0x7fc6c413ed70 2023-10-26T05:49:56.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c41377f0 con 0x7fc6c413c190 2023-10-26T05:49:56.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4137ad0 con 0x7fc6c413e8c0 2023-10-26T05:49:56.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:56.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53096/0 (socket says 172.21.15.71:53096) 2023-10-26T05:49:56.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 -- 172.21.15.71:0/1070217550 learned_addr learned my addr 172.21.15.71:0/1070217550 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:56.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c259c640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c413c190 0x7fc6c413f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:56.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 -- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6c413e8c0 msgr2=0x7fc6c4149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:56.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6c413e8c0 0x7fc6c4149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 -- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c413c190 msgr2=0x7fc6c413f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:56.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c413c190 0x7fc6c413f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:56.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 -- 172.21.15.71:0/1070217550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc009560 con 0x7fc6c413ed70 2023-10-26T05:49:56.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c259c640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c413c190 0x7fc6c413f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:56.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c2d9d640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c413b160 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0b0040 tx=0x7fc6bc0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:56.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc003800 con 0x7fc6c413ed70 2023-10-26T05:49:56.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc6bc0a5900 con 0x7fc6c413ed70 2023-10-26T05:49:56.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c40b1610 con 0x7fc6c413ed70 2023-10-26T05:49:56.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc003cd0 con 0x7fc6c413ed70 2023-10-26T05:49:56.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.425+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c413fed0 con 0x7fc6c413ed70 2023-10-26T05:49:56.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.429+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c4066b10 con 0x7fc6c413ed70 2023-10-26T05:49:56.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.429+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc6bc0b4070 con 0x7fc6c413ed70 2023-10-26T05:49:56.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.429+0000 7fc6baffd640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc6a40664a0 0x7fc6a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:56.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.429+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc6bc11e620 con 0x7fc6c413ed70 2023-10-26T05:49:56.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.429+0000 7fc6c259c640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc6a40664a0 0x7fc6a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:56.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.429+0000 7fc6c259c640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc6a40664a0 0x7fc6a4068960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4007fd0 tx=0x7fc6b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:56.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.433+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc6bc0ebca0 con 0x7fc6c413ed70 2023-10-26T05:49:56.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:56.621+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc6c4137c10 con 0x7fc6c413ed70 2023-10-26T05:49:57.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.261+0000 7fc6baffd640 1 -- 172.21.15.71:0/1070217550 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc6c4137c10 con 0x7fc6c413ed70 2023-10-26T05:49:57.267 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 8 and data pool 9 2023-10-26T05:49:57.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc6a40664a0 msgr2=0x7fc6a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc6a40664a0 0x7fc6a4068960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4007fd0 tx=0x7fc6b4009040 comp rx=0 tx=0).stop 2023-10-26T05:49:57.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 msgr2=0x7fc6c413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c413b160 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0b0040 tx=0x7fc6bc0a5710 comp rx=0 tx=0).stop 2023-10-26T05:49:57.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 shutdown_connections 2023-10-26T05:49:57.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc6a40664a0 0x7fc6a4068960 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6c413ed70 0x7fc6c413b160 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc6c413e8c0 0x7fc6c4149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 --2- 172.21.15.71:0/1070217550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6c413c190 0x7fc6c413f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 >> 172.21.15.71:0/1070217550 conn(0x7fc6c40aec60 msgr2=0x7fc6c40ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:57.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 shutdown_connections 2023-10-26T05:49:57.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.265+0000 7fc6c88e8640 1 -- 172.21.15.71:0/1070217550 wait complete. 2023-10-26T05:49:57.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:57.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/2909781855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f140b1360 msgr2=0x7f3f140b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/2909781855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f140b1360 0x7f3f140b17d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009c80 tx=0x7f3f0c01cd50 comp rx=0 tx=0).stop 2023-10-26T05:49:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/2909781855 shutdown_connections 2023-10-26T05:49:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/2909781855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f140b1360 0x7f3f140b17d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/2909781855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f14070fe0 0x7f3f14071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/2909781855 >> 172.21.15.71:0/2909781855 conn(0x7f3f14069770 msgr2=0x7f3f14069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/2909781855 shutdown_connections 2023-10-26T05:49:57.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/2909781855 wait complete. 2023-10-26T05:49:57.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 Processor -- start 2023-10-26T05:49:57.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- start start 2023-10-26T05:49:57.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14070fe0 0x7f3f14159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f140b1360 0x7f3f1413abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 0x7f3f1415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f141598e0 con 0x7f3f14140cc0 2023-10-26T05:49:57.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14158cd0 con 0x7f3f14070fe0 2023-10-26T05:49:57.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14158fb0 con 0x7f3f140b1360 2023-10-26T05:49:57.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 0x7f3f1415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 0x7f3f1415b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53124/0 (socket says 172.21.15.71:53124) 2023-10-26T05:49:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 -- 172.21.15.71:0/3860575355 learned_addr learned my addr 172.21.15.71:0/3860575355 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f13fff640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f140b1360 0x7f3f1413abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f18a94640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14070fe0 0x7f3f14159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 -- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f140b1360 msgr2=0x7f3f1413abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f140b1360 0x7f3f1413abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 -- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14070fe0 msgr2=0x7f3f14159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14070fe0 0x7f3f14159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 -- 172.21.15.71:0/3860575355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0098e0 con 0x7f3f14140cc0 2023-10-26T05:49:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f19295640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 0x7f3f1415b2e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c01a980 tx=0x7f3f0c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0bc070 con 0x7f3f14140cc0 2023-10-26T05:49:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f0c003c80 con 0x7f3f14140cc0 2023-10-26T05:49:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f14159290 con 0x7f3f14140cc0 2023-10-26T05:49:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0ae830 con 0x7f3f14140cc0 2023-10-26T05:49:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.425+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1413b110 con 0x7f3f14140cc0 2023-10-26T05:49:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.429+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f14066b10 con 0x7f3f14140cc0 2023-10-26T05:49:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.429+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f0c0b4040 con 0x7f3f14140cc0 2023-10-26T05:49:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.429+0000 7f3f117fa640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ef8066470 0x7f3ef8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.429+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f0c11ec50 con 0x7f3f14140cc0 2023-10-26T05:49:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.429+0000 7f3f18a94640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ef8066470 0x7f3ef8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.429+0000 7f3f18a94640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ef8066470 0x7f3ef8068930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f080015e0 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.433+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f0c0ec1f0 con 0x7f3f14140cc0 2023-10-26T05:49:57.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.589+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f140721e0 con 0x7f3f14140cc0 2023-10-26T05:49:57.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.589+0000 7f3f117fa640 1 -- 172.21.15.71:0/3860575355 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f3f0c0ef810 con 0x7f3f14140cc0 2023-10-26T05:49:57.595 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:57.595 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:56.623900+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:57.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ef8066470 msgr2=0x7f3ef8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ef8066470 0x7f3ef8068930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3f080015e0 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2023-10-26T05:49:57.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 msgr2=0x7f3f1415b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 0x7f3f1415b2e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c01a980 tx=0x7f3f0c0093d0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 shutdown_connections 2023-10-26T05:49:57.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ef8066470 0x7f3ef8068930 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f14140cc0 0x7f3f1415b2e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f140b1360 0x7f3f1413abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 --2- 172.21.15.71:0/3860575355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f14070fe0 0x7f3f14159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 >> 172.21.15.71:0/3860575355 conn(0x7f3f14069770 msgr2=0x7f3f14070740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:57.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 shutdown_connections 2023-10-26T05:49:57.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.593+0000 7f3f1acf8640 1 -- 172.21.15.71:0/3860575355 wait complete. 2023-10-26T05:49:57.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:57.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.733+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1553492776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb08c0047d0 msgr2=0x7fb08c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.733+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1553492776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb08c0047d0 0x7fb08c004c50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c2e0 tx=0x7fb09c01cfb0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1553492776 shutdown_connections 2023-10-26T05:49:57.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1553492776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb08c0047d0 0x7fb08c004c50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1553492776 >> 172.21.15.71:0/1553492776 conn(0x7fb0a00cdd80 msgr2=0x7fb0a00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:57.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1553492776 shutdown_connections 2023-10-26T05:49:57.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1553492776 wait complete. 2023-10-26T05:49:57.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 Processor -- start 2023-10-26T05:49:57.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- start start 2023-10-26T05:49:57.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c0047d0 0x7fb0a00c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 0x7fb0a01578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0a00b6240 0x7fb0a00b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006a650 con 0x7fb0a00cae00 2023-10-26T05:49:57.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00c7620 con 0x7fb08c0047d0 2023-10-26T05:49:57.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00b6c00 con 0x7fb0a00b6240 2023-10-26T05:49:57.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 0x7fb0a01578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 0x7fb0a01578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53154/0 (socket says 172.21.15.71:53154) 2023-10-26T05:49:57.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 -- 172.21.15.71:0/1889594830 learned_addr learned my addr 172.21.15.71:0/1889594830 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:57.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a6553640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0a00b6240 0x7fb0a00b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5d52640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c0047d0 0x7fb0a00c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 -- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0a00b6240 msgr2=0x7fb0a00b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0a00b6240 0x7fb0a00b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 -- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c0047d0 msgr2=0x7fb0a00c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c0047d0 0x7fb0a00c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 -- 172.21.15.71:0/1889594830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00c69a0 con 0x7fb0a00cae00 2023-10-26T05:49:57.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a5551640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 0x7fb0a01578f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01cde0 tx=0x7fb09c006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:57.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0b1070 con 0x7fb0a00cae00 2023-10-26T05:49:57.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb09c003600 con 0x7fb0a00cae00 2023-10-26T05:49:57.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0b4040 con 0x7fb0a00cae00 2023-10-26T05:49:57.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a00c6c80 con 0x7fb0a00cae00 2023-10-26T05:49:57.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.737+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a00c7240 con 0x7fb0a00cae00 2023-10-26T05:49:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.741+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a0066b10 con 0x7fb0a00cae00 2023-10-26T05:49:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.741+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb09c0ad430 con 0x7fb0a00cae00 2023-10-26T05:49:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.741+0000 7fb0967fc640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0840664a0 0x7fb084068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.741+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb09c11da20 con 0x7fb0a00cae00 2023-10-26T05:49:57.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.741+0000 7fb0a5d52640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0840664a0 0x7fb084068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.741+0000 7fb0a5d52640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0840664a0 0x7fb084068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb0a01eaff0 tx=0x7fb0900060f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:57.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.745+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb09c0eae90 con 0x7fb0a00cae00 2023-10-26T05:49:57.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.901+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0a00c7760 con 0x7fb0a00cae00 2023-10-26T05:49:57.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0967fc640 1 -- 172.21.15.71:0/1889594830 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16494 (secure 0 0 0) 0x7fb09c115e60 con 0x7fb0a00cae00 2023-10-26T05:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:57.908 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:56.623900+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:57.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0840664a0 msgr2=0x7fb084068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0840664a0 0x7fb084068960 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb0a01eaff0 tx=0x7fb0900060f0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 msgr2=0x7fb0a01578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:57.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 0x7fb0a01578f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb09c01cde0 tx=0x7fb09c006150 comp rx=0 tx=0).stop 2023-10-26T05:49:57.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 shutdown_connections 2023-10-26T05:49:57.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0a00b6240 0x7fb0a00b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb0840664a0 0x7fb084068960 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a00cae00 0x7fb0a01578f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 --2- 172.21.15.71:0/1889594830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb08c0047d0 0x7fb0a00c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:57.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 >> 172.21.15.71:0/1889594830 conn(0x7fb0a00cdd80 msgr2=0x7fb0a00b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:57.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 shutdown_connections 2023-10-26T05:49:57.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:57.905+0000 7fb0a7fb6640 1 -- 172.21.15.71:0/1889594830 wait complete. 2023-10-26T05:49:57.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:49:58.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3597953965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b2370 msgr2=0x7fa5c80ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3597953965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b2370 0x7fa5c80ca780 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009c80 tx=0x7fa5c001f580 comp rx=0 tx=0).stop 2023-10-26T05:49:58.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3597953965 shutdown_connections 2023-10-26T05:49:58.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3597953965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b2370 0x7fa5c80ca780 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3597953965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c8141a40 0x7fa5c80c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3597953965 >> 172.21.15.71:0/3597953965 conn(0x7fa5c80aec60 msgr2=0x7fa5c80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:58.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3597953965 shutdown_connections 2023-10-26T05:49:58.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3597953965 wait complete. 2023-10-26T05:49:58.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 Processor -- start 2023-10-26T05:49:58.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- start start 2023-10-26T05:49:58.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 0x7fa5c8156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c80b2370 0x7fa5c81f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c8141a40 0x7fa5c813ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c813d0f0 con 0x7fa5c80b1230 2023-10-26T05:49:58.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c813dc70 con 0x7fa5c8141a40 2023-10-26T05:49:58.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c813d340 con 0x7fa5c80b2370 2023-10-26T05:49:58.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 0x7fa5c8156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 0x7fa5c8156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53172/0 (socket says 172.21.15.71:53172) 2023-10-26T05:49:58.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 -- 172.21.15.71:0/3506073017 learned_addr learned my addr 172.21.15.71:0/3506073017 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:58.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5c7fff640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c80b2370 0x7fa5c81f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 -- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c80b2370 msgr2=0x7fa5c81f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c80b2370 0x7fa5c81f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 -- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c8141a40 msgr2=0x7fa5c813ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c8141a40 0x7fa5c813ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 -- 172.21.15.71:0/3506073017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00098e0 con 0x7fa5c80b1230 2023-10-26T05:49:58.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5c7fff640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c80b2370 0x7fa5c81f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:49:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5ccd09640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 0x7fa5c8156e40 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00058d0 tx=0x7fa5b000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0009c30 con 0x7fa5c80b1230 2023-10-26T05:49:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa5b0009e80 con 0x7fa5c80b1230 2023-10-26T05:49:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c813d620 con 0x7fa5c80b1230 2023-10-26T05:49:58.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00106a0 con 0x7fa5c80b1230 2023-10-26T05:49:58.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.045+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c8157380 con 0x7fa5c80b1230 2023-10-26T05:49:58.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.049+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8066b10 con 0x7fa5c80b1230 2023-10-26T05:49:58.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.049+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa5b0010800 con 0x7fa5c80b1230 2023-10-26T05:49:58.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.049+0000 7fa5c57fa640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a8066470 0x7fa5a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.049+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5b0084300 con 0x7fa5c80b1230 2023-10-26T05:49:58.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.049+0000 7fa5c7fff640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a8066470 0x7fa5a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.049+0000 7fa5c7fff640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a8066470 0x7fa5a8068930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80aff70 tx=0x7fa5b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:58.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.053+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa5b0051810 con 0x7fa5c80b1230 2023-10-26T05:49:58.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.245+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa5c80721e0 con 0x7fa5c80b1230 2023-10-26T05:49:58.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.285+0000 7fa5c57fa640 1 -- 172.21.15.71:0/3506073017 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa5b0054e30 con 0x7fa5c80b1230 2023-10-26T05:49:58.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a8066470 msgr2=0x7fa5a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a8066470 0x7fa5a8068930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80aff70 tx=0x7fa5b8009040 comp rx=0 tx=0).stop 2023-10-26T05:49:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 msgr2=0x7fa5c8156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 0x7fa5c8156e40 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00058d0 tx=0x7fa5b000dde0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 shutdown_connections 2023-10-26T05:49:58.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5c8141a40 0x7fa5c813ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5c80b2370 0x7fa5c81f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a8066470 0x7fa5a8068930 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 --2- 172.21.15.71:0/3506073017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c80b1230 0x7fa5c8156e40 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 >> 172.21.15.71:0/3506073017 conn(0x7fa5c80aec60 msgr2=0x7fa5c80cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:58.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 shutdown_connections 2023-10-26T05:49:58.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.289+0000 7fa5cef6d640 1 -- 172.21.15.71:0/3506073017 wait complete. 2023-10-26T05:49:58.310 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:49:58.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:49:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- 172.21.15.71:0/4020585770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc141a40 msgr2=0x7f84bc0c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/4020585770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc141a40 0x7f84bc0c9060 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009840 tx=0x7f84ac01eff0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- 172.21.15.71:0/4020585770 shutdown_connections 2023-10-26T05:49:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/4020585770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84bc0b1230 0x7f84bc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/4020585770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc141a40 0x7f84bc0c9060 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- 172.21.15.71:0/4020585770 >> 172.21.15.71:0/4020585770 conn(0x7f84bc0aec60 msgr2=0x7f84bc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- 172.21.15.71:0/4020585770 shutdown_connections 2023-10-26T05:49:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- 172.21.15.71:0/4020585770 wait complete. 2023-10-26T05:49:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 Processor -- start 2023-10-26T05:49:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- start start 2023-10-26T05:49:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84bc0b1230 0x7f84bc1f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc0b21a0 0x7f84bc13f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 0x7f84bc13f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc140f50 con 0x7f84bc141a40 2023-10-26T05:49:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1f1610 con 0x7f84bc0b21a0 2023-10-26T05:49:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84c1a3a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1379b0 con 0x7f84bc0b1230 2023-10-26T05:49:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 0x7f84bc13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 0x7f84bc13f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53184/0 (socket says 172.21.15.71:53184) 2023-10-26T05:49:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 -- 172.21.15.71:0/2429221638 learned_addr learned my addr 172.21.15.71:0/2429221638 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84b3fff640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc0b21a0 0x7f84bc13f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84baffd640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84bc0b1230 0x7f84bc1f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 -- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84bc0b1230 msgr2=0x7f84bc1f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84bc0b1230 0x7f84bc1f10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 -- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc0b21a0 msgr2=0x7f84bc13f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc0b21a0 0x7f84bc13f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.433+0000 7f84bb7fe640 1 -- 172.21.15.71:0/2429221638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ac0094a0 con 0x7f84bc141a40 2023-10-26T05:49:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84bb7fe640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 0x7f84bc13f880 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f84b400cb40 tx=0x7f84b400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:58.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b400aa50 con 0x7f84bc141a40 2023-10-26T05:49:58.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84b400aca0 con 0x7f84bc141a40 2023-10-26T05:49:58.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc13feb0 con 0x7f84bc141a40 2023-10-26T05:49:58.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4012680 con 0x7f84bc141a40 2023-10-26T05:49:58.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc140470 con 0x7f84bc141a40 2023-10-26T05:49:58.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc066b10 con 0x7f84bc141a40 2023-10-26T05:49:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f84b401b400 con 0x7f84bc141a40 2023-10-26T05:49:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84b8ff9640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f849c066470 0x7f849c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84b4084ae0 con 0x7f84bc141a40 2023-10-26T05:49:58.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84baffd640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f849c066470 0x7f849c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:58.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.437+0000 7f84baffd640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f849c066470 0x7f849c068930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0b2000 tx=0x7f84ac01d830 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:58.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.441+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f84b4052080 con 0x7f84bc141a40 2023-10-26T05:49:58.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:58.625+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84bc0721e0 con 0x7f84bc141a40 2023-10-26T05:49:59.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.297+0000 7f84b8ff9640 1 -- 172.21.15.71:0/2429221638 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84b40556a0 con 0x7f84bc141a40 2023-10-26T05:49:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f849c066470 msgr2=0x7f849c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f849c066470 0x7f849c068930 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84ac0b2000 tx=0x7f84ac01d830 comp rx=0 tx=0).stop 2023-10-26T05:49:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 msgr2=0x7f84bc13f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 0x7f84bc13f880 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f84b400cb40 tx=0x7f84b400cc30 comp rx=0 tx=0).stop 2023-10-26T05:49:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 shutdown_connections 2023-10-26T05:49:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f849c066470 0x7f849c068930 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84bc141a40 0x7f84bc13f880 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84bc0b21a0 0x7f84bc13f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 --2- 172.21.15.71:0/2429221638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84bc0b1230 0x7f84bc1f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 >> 172.21.15.71:0/2429221638 conn(0x7f84bc0aec60 msgr2=0x7f84bc06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:59.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 shutdown_connections 2023-10-26T05:49:59.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.301+0000 7f84c1a3a640 1 -- 172.21.15.71:0/2429221638 wait complete. 2023-10-26T05:49:59.329 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:49:59.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:49:59.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- 172.21.15.71:0/4046459109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 msgr2=0x7fbbb815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 --2- 172.21.15.71:0/4046459109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 0x7fbbb815b3a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009bc0 tx=0x7fbbac01ca60 comp rx=0 tx=0).stop 2023-10-26T05:49:59.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- 172.21.15.71:0/4046459109 shutdown_connections 2023-10-26T05:49:59.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 --2- 172.21.15.71:0/4046459109 >> v2:172.21.15.86:3300/0 conn(0x7fbbb80cd880 0x7fbbb815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 --2- 172.21.15.71:0/4046459109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 0x7fbbb815b3a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- 172.21.15.71:0/4046459109 >> 172.21.15.71:0/4046459109 conn(0x7fbbb80cdd80 msgr2=0x7fbbb80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:59.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- 172.21.15.71:0/4046459109 shutdown_connections 2023-10-26T05:49:59.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- 172.21.15.71:0/4046459109 wait complete. 2023-10-26T05:49:59.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 Processor -- start 2023-10-26T05:49:59.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- start start 2023-10-26T05:49:59.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80cd880 0x7fbbb8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 0x7fbbb80ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 0x7fbbb8152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81526a0 con 0x7fbbb80d0470 2023-10-26T05:49:59.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81554c0 con 0x7fbbb80cd880 2023-10-26T05:49:59.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbf183640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81557a0 con 0x7fbbb80d0850 2023-10-26T05:49:59.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 0x7fbbb80ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 0x7fbbb80ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53194/0 (socket says 172.21.15.71:53194) 2023-10-26T05:49:59.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 -- 172.21.15.71:0/3303165100 learned_addr learned my addr 172.21.15.71:0/3303165100 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:59.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbcf1f640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80cd880 0x7fbbb8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbbd720640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 0x7fbbb8152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 -- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 msgr2=0x7fbbb8152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 0x7fbbb8152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 -- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80cd880 msgr2=0x7fbbb8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80cd880 0x7fbbb8152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 -- 172.21.15.71:0/3303165100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009820 con 0x7fbbb80d0470 2023-10-26T05:49:59.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb7fff640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 0x7fbbb80ca3e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009b10 tx=0x7fbbac009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:59.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac005240 con 0x7fbbb80d0470 2023-10-26T05:49:59.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbbac005490 con 0x7fbbb80d0470 2023-10-26T05:49:59.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.457+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac005860 con 0x7fbbb80d0470 2023-10-26T05:49:59.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb8155a80 con 0x7fbbb80d0470 2023-10-26T05:49:59.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb80ca920 con 0x7fbbb80d0470 2023-10-26T05:49:59.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb8155f30 con 0x7fbbb80d0470 2023-10-26T05:49:59.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbbac0b4070 con 0x7fbbb80d0470 2023-10-26T05:49:59.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbb57fa640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbba4066470 0x7fbba4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbac11ed20 con 0x7fbbb80d0470 2023-10-26T05:49:59.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbbcf1f640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbba4066470 0x7fbba4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.461+0000 7fbbbcf1f640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbba4066470 0x7fbba4068930 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbba80010f0 tx=0x7fbba8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:59.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.465+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbbac0ec1a0 con 0x7fbbb80d0470 2023-10-26T05:49:59.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.653+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbb8066b10 con 0x7fbbb80d0470 2023-10-26T05:49:59.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.653+0000 7fbbb57fa640 1 -- 172.21.15.71:0/3303165100 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7fbbac0b2090 con 0x7fbbb80d0470 2023-10-26T05:49:59.658 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:59.659 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:49:59.659 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:49:59.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbba4066470 msgr2=0x7fbba4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbba4066470 0x7fbba4068930 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbba80010f0 tx=0x7fbba8009040 comp rx=0 tx=0).stop 2023-10-26T05:49:59.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 msgr2=0x7fbbb80ca3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 0x7fbbb80ca3e0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009b10 tx=0x7fbbac009290 comp rx=0 tx=0).stop 2023-10-26T05:49:59.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 shutdown_connections 2023-10-26T05:49:59.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbbb80d0850 0x7fbbb8152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbba4066470 0x7fbba4068930 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb80d0470 0x7fbbb80ca3e0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 --2- 172.21.15.71:0/3303165100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb80cd880 0x7fbbb8152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 >> 172.21.15.71:0/3303165100 conn(0x7fbbb80cdd80 msgr2=0x7fbbb80b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:59.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 shutdown_connections 2023-10-26T05:49:59.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.657+0000 7fbbbf183640 1 -- 172.21.15.71:0/3303165100 wait complete. 2023-10-26T05:49:59.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:49:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3652025756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c159210 msgr2=0x7fbb9c0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3652025756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c159210 0x7fbb9c0ce8b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009900 tx=0x7fbb9401c7f0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3652025756 shutdown_connections 2023-10-26T05:49:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3652025756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb9c0cd410 0x7fbb9c0aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3652025756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c159210 0x7fbb9c0ce8b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3652025756 >> 172.21.15.71:0/3652025756 conn(0x7fbb9c0b0c80 msgr2=0x7fbb9c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3652025756 shutdown_connections 2023-10-26T05:49:59.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3652025756 wait complete. 2023-10-26T05:49:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 Processor -- start 2023-10-26T05:49:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- start start 2023-10-26T05:49:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 0x7fbb9c0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb9c0cd410 0x7fbb9c1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb9c159210 0x7fbb9c06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c158e20 con 0x7fbb9c0b0420 2023-10-26T05:49:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c06b110 con 0x7fbb9c0cd410 2023-10-26T05:49:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9c06b3f0 con 0x7fbb9c159210 2023-10-26T05:49:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 0x7fbb9c0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 0x7fbb9c0c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53212/0 (socket says 172.21.15.71:53212) 2023-10-26T05:49:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 -- 172.21.15.71:0/3635389105 learned_addr learned my addr 172.21.15.71:0/3635389105 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:49:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9b7fe640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb9c159210 0x7fbb9c06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 -- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb9c159210 msgr2=0x7fbb9c06c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb9c159210 0x7fbb9c06c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 -- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb9c0cd410 msgr2=0x7fbb9c1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb9c0cd410 0x7fbb9c1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 -- 172.21.15.71:0/3635389105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94009560 con 0x7fbb9c0b0420 2023-10-26T05:49:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb9affd640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 0x7fbb9c0c5c80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009850 tx=0x7fbb940a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb94003800 con 0x7fbb9c0b0420 2023-10-26T05:49:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb94003960 con 0x7fbb9c0b0420 2023-10-26T05:49:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9c06b6d0 con 0x7fbb9c0b0420 2023-10-26T05:49:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb94003d50 con 0x7fbb9c0b0420 2023-10-26T05:49:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.801+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c06bc90 con 0x7fbb9c0b0420 2023-10-26T05:49:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.805+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb9c066b10 con 0x7fbb9c0b0420 2023-10-26T05:49:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.805+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbb940b4070 con 0x7fbb9c0b0420 2023-10-26T05:49:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.805+0000 7fbb93fff640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb80066470 0x7fbb80068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:49:59.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.805+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb9411e7e0 con 0x7fbb9c0b0420 2023-10-26T05:49:59.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.805+0000 7fbb9a7fc640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb80066470 0x7fbb80068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:49:59.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.805+0000 7fbb9a7fc640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb80066470 0x7fbb80068930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0b1690 tx=0x7fbb8c007d20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:49:59.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.809+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbb940ebd20 con 0x7fbb9c0b0420 2023-10-26T05:49:59.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.965+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb9c066b10 con 0x7fbb9c0b0420 2023-10-26T05:49:59.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.965+0000 7fbb93fff640 1 -- 172.21.15.71:0/3635389105 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fbb940ef340 con 0x7fbb9c0b0420 2023-10-26T05:49:59.971 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:49:59.971 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":68,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:49:58.082507+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:49:59.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb80066470 msgr2=0x7fbb80068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb80066470 0x7fbb80068930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0b1690 tx=0x7fbb8c007d20 comp rx=0 tx=0).stop 2023-10-26T05:49:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 msgr2=0x7fbb9c0c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:49:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 0x7fbb9c0c5c80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009850 tx=0x7fbb940a56d0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 shutdown_connections 2023-10-26T05:49:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb9c159210 0x7fbb9c06c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb9c0cd410 0x7fbb9c1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb80066470 0x7fbb80068930 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 --2- 172.21.15.71:0/3635389105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c0b0420 0x7fbb9c0c5c80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:49:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 >> 172.21.15.71:0/3635389105 conn(0x7fbb9c0b0c80 msgr2=0x7fbb9c0b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:49:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 shutdown_connections 2023-10-26T05:49:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:49:59.969+0000 7fbba1a57640 1 -- 172.21.15.71:0/3635389105 wait complete. 2023-10-26T05:49:59.990 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:49:59.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:50:00.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- 172.21.15.71:0/1806068037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c06d340 msgr2=0x7fd38c0ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 --2- 172.21.15.71:0/1806068037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c06d340 0x7fd38c0ca8c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd374009c80 tx=0x7fd38c0cd140 comp rx=0 tx=0).stop 2023-10-26T05:50:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- 172.21.15.71:0/1806068037 shutdown_connections 2023-10-26T05:50:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 --2- 172.21.15.71:0/1806068037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd38c06d720 0x7fd38c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 --2- 172.21.15.71:0/1806068037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c06d340 0x7fd38c0ca8c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- 172.21.15.71:0/1806068037 >> 172.21.15.71:0/1806068037 conn(0x7fd38c070170 msgr2=0x7fd38c070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- 172.21.15.71:0/1806068037 shutdown_connections 2023-10-26T05:50:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- 172.21.15.71:0/1806068037 wait complete. 2023-10-26T05:50:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 Processor -- start 2023-10-26T05:50:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- start start 2023-10-26T05:50:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c06d340 0x7fd38c132ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd38c06d720 0x7fd38c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 0x7fd38c155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1508a0 con 0x7fd38c15c1e0 2023-10-26T05:50:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c151170 con 0x7fd38c06d340 2023-10-26T05:50:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd392546640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c150e90 con 0x7fd38c06d720 2023-10-26T05:50:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 0x7fd38c155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 0x7fd38c155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53232/0 (socket says 172.21.15.71:53232) 2023-10-26T05:50:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 -- 172.21.15.71:0/1485795944 learned_addr learned my addr 172.21.15.71:0/1485795944 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd38b7fe640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd38c06d720 0x7fd38c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd38bfff640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c06d340 0x7fd38c132ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 -- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd38c06d720 msgr2=0x7fd38c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd38c06d720 0x7fd38c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 -- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c06d340 msgr2=0x7fd38c132ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c06d340 0x7fd38c132ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 -- 172.21.15.71:0/1485795944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740098e0 con 0x7fd38c15c1e0 2023-10-26T05:50:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd38bfff640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c06d340 0x7fd38c132ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:50:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.113+0000 7fd390ae3640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 0x7fd38c155f30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b440 tx=0x7fd38400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384011830 con 0x7fd38c15c1e0 2023-10-26T05:50:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd384011aa0 con 0x7fd38c15c1e0 2023-10-26T05:50:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c15b340 con 0x7fd38c15c1e0 2023-10-26T05:50:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384010380 con 0x7fd38c15c1e0 2023-10-26T05:50:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c133400 con 0x7fd38c15c1e0 2023-10-26T05:50:00.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c15b580 con 0x7fd38c15c1e0 2023-10-26T05:50:00.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd3840104e0 con 0x7fd38c15c1e0 2023-10-26T05:50:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd388ff9640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd36c066470 0x7fd36c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3840841c0 con 0x7fd38c15c1e0 2023-10-26T05:50:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd38bfff640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd36c066470 0x7fd36c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.117+0000 7fd38bfff640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd36c066470 0x7fd36c068930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd3740af000 tx=0x7fd37401b430 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:00.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.121+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd38c15b580 con 0x7fd38c15c1e0 2023-10-26T05:50:00.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.257+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd38c15b580 con 0x7fd36c066470 2023-10-26T05:50:00.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.281+0000 7fd388ff9640 1 -- 172.21.15.71:0/1485795944 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284161 (secure 0 0 0) 0x7fd38c15b580 con 0x7fd36c066470 2023-10-26T05:50:00.287 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:50:00.289 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:50:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd36c066470 msgr2=0x7fd36c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd36c066470 0x7fd36c068930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd3740af000 tx=0x7fd37401b430 comp rx=0 tx=0).stop 2023-10-26T05:50:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 msgr2=0x7fd38c155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 0x7fd38c155f30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd38400b440 tx=0x7fd38400b900 comp rx=0 tx=0).stop 2023-10-26T05:50:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 shutdown_connections 2023-10-26T05:50:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd36c066470 0x7fd36c068930 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd38c15c1e0 0x7fd38c155f30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd38c06d720 0x7fd38c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 --2- 172.21.15.71:0/1485795944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd38c06d340 0x7fd38c132ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 >> 172.21.15.71:0/1485795944 conn(0x7fd38c070170 msgr2=0x7fd38c158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 shutdown_connections 2023-10-26T05:50:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.285+0000 7fd392546640 1 -- 172.21.15.71:0/1485795944 wait complete. 2023-10-26T05:50:00.304 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-10-26T05:49:59.464855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3911,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558588,"statfs":{"total":1600000000,"available":1595994324,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.210283"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225276+0000","last_change":"2023-10-26T05:49:57.114010+0000","last_active":"2023-10-26T05:49:58.225276+0000","last_peered":"2023-10-26T05:49:58.225276+0000","last_clean":"2023-10-26T05:49:58.225276+0000","last_became_active":"2023-10-26T05:49:57.113315+0000","last_became_peered":"2023-10-26T05:49:57.113315+0000","last_unstale":"2023-10-26T05:49:58.225276+0000","last_undegraded":"2023-10-26T05:49:58.225276+0000","last_fullsized":"2023-10-26T05:49:58.225276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:10.952396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716521+0000","last_change":"2023-10-26T05:49:57.126680+0000","last_active":"2023-10-26T05:49:58.716521+0000","last_peered":"2023-10-26T05:49:58.716521+0000","last_clean":"2023-10-26T05:49:58.716521+0000","last_became_active":"2023-10-26T05:49:57.124014+0000","last_became_peered":"2023-10-26T05:49:57.124014+0000","last_unstale":"2023-10-26T05:49:58.716521+0000","last_undegraded":"2023-10-26T05:49:58.716521+0000","last_fullsized":"2023-10-26T05:49:58.716521+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:33:56.377336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090212+0000","last_change":"2023-10-26T05:49:57.128061+0000","last_active":"2023-10-26T05:49:58.090212+0000","last_peered":"2023-10-26T05:49:58.090212+0000","last_clean":"2023-10-26T05:49:58.090212+0000","last_became_active":"2023-10-26T05:49:57.124229+0000","last_became_peered":"2023-10-26T05:49:57.124229+0000","last_unstale":"2023-10-26T05:49:58.090212+0000","last_undegraded":"2023-10-26T05:49:58.090212+0000","last_fullsized":"2023-10-26T05:49:58.090212+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:02.687883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316712+0000","last_change":"2023-10-26T05:49:57.124535+0000","last_active":"2023-10-26T05:49:58.316712+0000","last_peered":"2023-10-26T05:49:58.316712+0000","last_clean":"2023-10-26T05:49:58.316712+0000","last_became_active":"2023-10-26T05:49:57.116839+0000","last_became_peered":"2023-10-26T05:49:57.116839+0000","last_unstale":"2023-10-26T05:49:58.316712+0000","last_undegraded":"2023-10-26T05:49:58.316712+0000","last_fullsized":"2023-10-26T05:49:58.316712+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:46.958437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225666+0000","last_change":"2023-10-26T05:49:57.103915+0000","last_active":"2023-10-26T05:49:58.225666+0000","last_peered":"2023-10-26T05:49:58.225666+0000","last_clean":"2023-10-26T05:49:58.225666+0000","last_became_active":"2023-10-26T05:49:57.103326+0000","last_became_peered":"2023-10-26T05:49:57.103326+0000","last_unstale":"2023-10-26T05:49:58.225666+0000","last_undegraded":"2023-10-26T05:49:58.225666+0000","last_fullsized":"2023-10-26T05:49:58.225666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:12.957362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086092+0000","last_change":"2023-10-26T05:49:57.112223+0000","last_active":"2023-10-26T05:49:58.086092+0000","last_peered":"2023-10-26T05:49:58.086092+0000","last_clean":"2023-10-26T05:49:58.086092+0000","last_became_active":"2023-10-26T05:49:57.106259+0000","last_became_peered":"2023-10-26T05:49:57.106259+0000","last_unstale":"2023-10-26T05:49:58.086092+0000","last_undegraded":"2023-10-26T05:49:58.086092+0000","last_fullsized":"2023-10-26T05:49:58.086092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:09:43.591005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090913+0000","last_change":"2023-10-26T05:49:57.104754+0000","last_active":"2023-10-26T05:49:58.090913+0000","last_peered":"2023-10-26T05:49:58.090913+0000","last_clean":"2023-10-26T05:49:58.090913+0000","last_became_active":"2023-10-26T05:49:57.104120+0000","last_became_peered":"2023-10-26T05:49:57.104120+0000","last_unstale":"2023-10-26T05:49:58.090913+0000","last_undegraded":"2023-10-26T05:49:58.090913+0000","last_fullsized":"2023-10-26T05:49:58.090913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:06.675265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716288+0000","last_change":"2023-10-26T05:49:57.118257+0000","last_active":"2023-10-26T05:49:58.716288+0000","last_peered":"2023-10-26T05:49:58.716288+0000","last_clean":"2023-10-26T05:49:58.716288+0000","last_became_active":"2023-10-26T05:49:57.115846+0000","last_became_peered":"2023-10-26T05:49:57.115846+0000","last_unstale":"2023-10-26T05:49:58.716288+0000","last_undegraded":"2023-10-26T05:49:58.716288+0000","last_fullsized":"2023-10-26T05:49:58.716288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:49.437924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091523+0000","last_change":"2023-10-26T05:49:57.104843+0000","last_active":"2023-10-26T05:49:58.091523+0000","last_peered":"2023-10-26T05:49:58.091523+0000","last_clean":"2023-10-26T05:49:58.091523+0000","last_became_active":"2023-10-26T05:49:57.103311+0000","last_became_peered":"2023-10-26T05:49:57.103311+0000","last_unstale":"2023-10-26T05:49:58.091523+0000","last_undegraded":"2023-10-26T05:49:58.091523+0000","last_fullsized":"2023-10-26T05:49:58.091523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:02.687883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314177+0000","last_change":"2023-10-26T05:49:57.099713+0000","last_active":"2023-10-26T05:49:58.314177+0000","last_peered":"2023-10-26T05:49:58.314177+0000","last_clean":"2023-10-26T05:49:58.314177+0000","last_became_active":"2023-10-26T05:49:57.099106+0000","last_became_peered":"2023-10-26T05:49:57.099106+0000","last_unstale":"2023-10-26T05:49:58.314177+0000","last_undegraded":"2023-10-26T05:49:58.314177+0000","last_fullsized":"2023-10-26T05:49:58.314177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:57.669526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314371+0000","last_change":"2023-10-26T05:49:57.113364+0000","last_active":"2023-10-26T05:49:58.314371+0000","last_peered":"2023-10-26T05:49:58.314371+0000","last_clean":"2023-10-26T05:49:58.314371+0000","last_became_active":"2023-10-26T05:49:57.112667+0000","last_became_peered":"2023-10-26T05:49:57.112667+0000","last_unstale":"2023-10-26T05:49:58.314371+0000","last_undegraded":"2023-10-26T05:49:58.314371+0000","last_fullsized":"2023-10-26T05:49:58.314371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:56.622294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317646+0000","last_change":"2023-10-26T05:49:57.113240+0000","last_active":"2023-10-26T05:49:58.317646+0000","last_peered":"2023-10-26T05:49:58.317646+0000","last_clean":"2023-10-26T05:49:58.317646+0000","last_became_active":"2023-10-26T05:49:57.111127+0000","last_became_peered":"2023-10-26T05:49:57.111127+0000","last_unstale":"2023-10-26T05:49:58.317646+0000","last_undegraded":"2023-10-26T05:49:58.317646+0000","last_fullsized":"2023-10-26T05:49:58.317646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:30.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316728+0000","last_change":"2023-10-26T05:49:57.121558+0000","last_active":"2023-10-26T05:49:58.316728+0000","last_peered":"2023-10-26T05:49:58.316728+0000","last_clean":"2023-10-26T05:49:58.316728+0000","last_became_active":"2023-10-26T05:49:57.120371+0000","last_became_peered":"2023-10-26T05:49:57.120371+0000","last_unstale":"2023-10-26T05:49:58.316728+0000","last_undegraded":"2023-10-26T05:49:58.316728+0000","last_fullsized":"2023-10-26T05:49:58.316728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:57.669526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091848+0000","last_change":"2023-10-26T05:49:57.105511+0000","last_active":"2023-10-26T05:49:58.091848+0000","last_peered":"2023-10-26T05:49:58.091848+0000","last_clean":"2023-10-26T05:49:58.091848+0000","last_became_active":"2023-10-26T05:49:57.104184+0000","last_became_peered":"2023-10-26T05:49:57.104184+0000","last_unstale":"2023-10-26T05:49:58.091848+0000","last_undegraded":"2023-10-26T05:49:58.091848+0000","last_fullsized":"2023-10-26T05:49:58.091848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:16.970220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.093368+0000","last_change":"2023-10-26T05:49:57.098966+0000","last_active":"2023-10-26T05:49:58.093368+0000","last_peered":"2023-10-26T05:49:58.093368+0000","last_clean":"2023-10-26T05:49:58.093368+0000","last_became_active":"2023-10-26T05:49:57.098323+0000","last_became_peered":"2023-10-26T05:49:57.098323+0000","last_unstale":"2023-10-26T05:49:58.093368+0000","last_undegraded":"2023-10-26T05:49:58.093368+0000","last_fullsized":"2023-10-26T05:49:58.093368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:29:01.910496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086298+0000","last_change":"2023-10-26T05:49:57.110983+0000","last_active":"2023-10-26T05:49:58.086298+0000","last_peered":"2023-10-26T05:49:58.086298+0000","last_clean":"2023-10-26T05:49:58.086298+0000","last_became_active":"2023-10-26T05:49:57.104680+0000","last_became_peered":"2023-10-26T05:49:57.104680+0000","last_unstale":"2023-10-26T05:49:58.086298+0000","last_undegraded":"2023-10-26T05:49:58.086298+0000","last_fullsized":"2023-10-26T05:49:58.086298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:31:53.093895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716729+0000","last_change":"2023-10-26T05:49:57.126050+0000","last_active":"2023-10-26T05:49:58.716729+0000","last_peered":"2023-10-26T05:49:58.716729+0000","last_clean":"2023-10-26T05:49:58.716729+0000","last_became_active":"2023-10-26T05:49:57.122612+0000","last_became_peered":"2023-10-26T05:49:57.122612+0000","last_unstale":"2023-10-26T05:49:58.716729+0000","last_undegraded":"2023-10-26T05:49:58.716729+0000","last_fullsized":"2023-10-26T05:49:58.716729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:03.709697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090536+0000","last_change":"2023-10-26T05:49:57.128330+0000","last_active":"2023-10-26T05:49:58.090536+0000","last_peered":"2023-10-26T05:49:58.090536+0000","last_clean":"2023-10-26T05:49:58.090536+0000","last_became_active":"2023-10-26T05:49:57.125489+0000","last_became_peered":"2023-10-26T05:49:57.125489+0000","last_unstale":"2023-10-26T05:49:58.090536+0000","last_undegraded":"2023-10-26T05:49:58.090536+0000","last_fullsized":"2023-10-26T05:49:58.090536+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:29:01.910496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225539+0000","last_change":"2023-10-26T05:49:57.107668+0000","last_active":"2023-10-26T05:49:58.225539+0000","last_peered":"2023-10-26T05:49:58.225539+0000","last_clean":"2023-10-26T05:49:58.225539+0000","last_became_active":"2023-10-26T05:49:57.107054+0000","last_became_peered":"2023-10-26T05:49:57.107054+0000","last_unstale":"2023-10-26T05:49:58.225539+0000","last_undegraded":"2023-10-26T05:49:58.225539+0000","last_fullsized":"2023-10-26T05:49:58.225539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:39.383759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716668+0000","last_change":"2023-10-26T05:49:57.108547+0000","last_active":"2023-10-26T05:49:58.716668+0000","last_peered":"2023-10-26T05:49:58.716668+0000","last_clean":"2023-10-26T05:49:58.716668+0000","last_became_active":"2023-10-26T05:49:57.107926+0000","last_became_peered":"2023-10-26T05:49:57.107926+0000","last_unstale":"2023-10-26T05:49:58.716668+0000","last_undegraded":"2023-10-26T05:49:58.716668+0000","last_fullsized":"2023-10-26T05:49:58.716668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:37.339136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092098+0000","last_change":"2023-10-26T05:49:57.121068+0000","last_active":"2023-10-26T05:49:58.092098+0000","last_peered":"2023-10-26T05:49:58.092098+0000","last_clean":"2023-10-26T05:49:58.092098+0000","last_became_active":"2023-10-26T05:49:57.118564+0000","last_became_peered":"2023-10-26T05:49:57.118564+0000","last_unstale":"2023-10-26T05:49:58.092098+0000","last_undegraded":"2023-10-26T05:49:58.092098+0000","last_fullsized":"2023-10-26T05:49:58.092098+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:21.674188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716914+0000","last_change":"2023-10-26T05:49:57.125635+0000","last_active":"2023-10-26T05:49:58.716914+0000","last_peered":"2023-10-26T05:49:58.716914+0000","last_clean":"2023-10-26T05:49:58.716914+0000","last_became_active":"2023-10-26T05:49:57.121338+0000","last_became_peered":"2023-10-26T05:49:57.121338+0000","last_unstale":"2023-10-26T05:49:58.716914+0000","last_undegraded":"2023-10-26T05:49:58.716914+0000","last_fullsized":"2023-10-26T05:49:58.716914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:01:23.419975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090876+0000","last_change":"2023-10-26T05:49:57.128624+0000","last_active":"2023-10-26T05:49:58.090876+0000","last_peered":"2023-10-26T05:49:58.090876+0000","last_clean":"2023-10-26T05:49:58.090876+0000","last_became_active":"2023-10-26T05:49:57.127164+0000","last_became_peered":"2023-10-26T05:49:57.127164+0000","last_unstale":"2023-10-26T05:49:58.090876+0000","last_undegraded":"2023-10-26T05:49:58.090876+0000","last_fullsized":"2023-10-26T05:49:58.090876+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:06.231291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316637+0000","last_change":"2023-10-26T05:49:57.121100+0000","last_active":"2023-10-26T05:49:58.316637+0000","last_peered":"2023-10-26T05:49:58.316637+0000","last_clean":"2023-10-26T05:49:58.316637+0000","last_became_active":"2023-10-26T05:49:57.120303+0000","last_became_peered":"2023-10-26T05:49:57.120303+0000","last_unstale":"2023-10-26T05:49:58.316637+0000","last_undegraded":"2023-10-26T05:49:58.316637+0000","last_fullsized":"2023-10-26T05:49:58.316637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:02.439356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225979+0000","last_change":"2023-10-26T05:49:57.099801+0000","last_active":"2023-10-26T05:49:58.225979+0000","last_peered":"2023-10-26T05:49:58.225979+0000","last_clean":"2023-10-26T05:49:58.225979+0000","last_became_active":"2023-10-26T05:49:57.099153+0000","last_became_peered":"2023-10-26T05:49:57.099153+0000","last_unstale":"2023-10-26T05:49:58.225979+0000","last_undegraded":"2023-10-26T05:49:58.225979+0000","last_fullsized":"2023-10-26T05:49:58.225979+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:50:44.697497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317659+0000","last_change":"2023-10-26T05:49:57.108003+0000","last_active":"2023-10-26T05:49:58.317659+0000","last_peered":"2023-10-26T05:49:58.317659+0000","last_clean":"2023-10-26T05:49:58.317659+0000","last_became_active":"2023-10-26T05:49:57.106379+0000","last_became_peered":"2023-10-26T05:49:57.106379+0000","last_unstale":"2023-10-26T05:49:58.317659+0000","last_undegraded":"2023-10-26T05:49:58.317659+0000","last_fullsized":"2023-10-26T05:49:58.317659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:06.675265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086111+0000","last_change":"2023-10-26T05:49:57.098399+0000","last_active":"2023-10-26T05:49:58.086111+0000","last_peered":"2023-10-26T05:49:58.086111+0000","last_clean":"2023-10-26T05:49:58.086111+0000","last_became_active":"2023-10-26T05:49:57.097723+0000","last_became_peered":"2023-10-26T05:49:57.097723+0000","last_unstale":"2023-10-26T05:49:58.086111+0000","last_undegraded":"2023-10-26T05:49:58.086111+0000","last_fullsized":"2023-10-26T05:49:58.086111+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:18:33.139199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225587+0000","last_change":"2023-10-26T05:49:57.102815+0000","last_active":"2023-10-26T05:49:58.225587+0000","last_peered":"2023-10-26T05:49:58.225587+0000","last_clean":"2023-10-26T05:49:58.225587+0000","last_became_active":"2023-10-26T05:49:57.102169+0000","last_became_peered":"2023-10-26T05:49:57.102169+0000","last_unstale":"2023-10-26T05:49:58.225587+0000","last_undegraded":"2023-10-26T05:49:58.225587+0000","last_fullsized":"2023-10-26T05:49:58.225587+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:01.993526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716407+0000","last_change":"2023-10-26T05:49:57.123223+0000","last_active":"2023-10-26T05:49:58.716407+0000","last_peered":"2023-10-26T05:49:58.716407+0000","last_clean":"2023-10-26T05:49:58.716407+0000","last_became_active":"2023-10-26T05:49:57.122494+0000","last_became_peered":"2023-10-26T05:49:57.122494+0000","last_unstale":"2023-10-26T05:49:58.716407+0000","last_undegraded":"2023-10-26T05:49:58.716407+0000","last_fullsized":"2023-10-26T05:49:58.716407+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:02.613671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090333+0000","last_change":"2023-10-26T05:49:57.112588+0000","last_active":"2023-10-26T05:49:58.090333+0000","last_peered":"2023-10-26T05:49:58.090333+0000","last_clean":"2023-10-26T05:49:58.090333+0000","last_became_active":"2023-10-26T05:49:57.111972+0000","last_became_peered":"2023-10-26T05:49:57.111972+0000","last_unstale":"2023-10-26T05:49:58.090333+0000","last_undegraded":"2023-10-26T05:49:58.090333+0000","last_fullsized":"2023-10-26T05:49:58.090333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:12.865553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225417+0000","last_change":"2023-10-26T05:49:57.091899+0000","last_active":"2023-10-26T05:49:58.225417+0000","last_peered":"2023-10-26T05:49:58.225417+0000","last_clean":"2023-10-26T05:49:58.225417+0000","last_became_active":"2023-10-26T05:49:57.091278+0000","last_became_peered":"2023-10-26T05:49:57.091278+0000","last_unstale":"2023-10-26T05:49:58.225417+0000","last_undegraded":"2023-10-26T05:49:58.225417+0000","last_fullsized":"2023-10-26T05:49:58.225417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:31.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086259+0000","last_change":"2023-10-26T05:49:57.123592+0000","last_active":"2023-10-26T05:49:58.086259+0000","last_peered":"2023-10-26T05:49:58.086259+0000","last_clean":"2023-10-26T05:49:58.086259+0000","last_became_active":"2023-10-26T05:49:57.122977+0000","last_became_peered":"2023-10-26T05:49:57.122977+0000","last_unstale":"2023-10-26T05:49:58.086259+0000","last_undegraded":"2023-10-26T05:49:58.086259+0000","last_fullsized":"2023-10-26T05:49:58.086259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:22:40.506950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086112+0000","last_change":"2023-10-26T05:49:57.108669+0000","last_active":"2023-10-26T05:49:58.086112+0000","last_peered":"2023-10-26T05:49:58.086112+0000","last_clean":"2023-10-26T05:49:58.086112+0000","last_became_active":"2023-10-26T05:49:57.107485+0000","last_became_peered":"2023-10-26T05:49:57.107485+0000","last_unstale":"2023-10-26T05:49:58.086112+0000","last_undegraded":"2023-10-26T05:49:58.086112+0000","last_fullsized":"2023-10-26T05:49:58.086112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:37:39.035147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"67'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086183+0000","last_change":"2023-10-26T05:49:56.268406+0000","last_active":"2023-10-26T05:49:58.086183+0000","last_peered":"2023-10-26T05:49:58.086183+0000","last_clean":"2023-10-26T05:49:58.086183+0000","last_became_active":"2023-10-26T05:49:56.268181+0000","last_became_peered":"2023-10-26T05:49:56.268181+0000","last_unstale":"2023-10-26T05:49:58.086183+0000","last_undegraded":"2023-10-26T05:49:58.086183+0000","last_fullsized":"2023-10-26T05:49:58.086183+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:14.545042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314228+0000","last_change":"2023-10-26T05:49:57.116076+0000","last_active":"2023-10-26T05:49:58.314228+0000","last_peered":"2023-10-26T05:49:58.314228+0000","last_clean":"2023-10-26T05:49:58.314228+0000","last_became_active":"2023-10-26T05:49:57.115255+0000","last_became_peered":"2023-10-26T05:49:57.115255+0000","last_unstale":"2023-10-26T05:49:58.314228+0000","last_undegraded":"2023-10-26T05:49:58.314228+0000","last_fullsized":"2023-10-26T05:49:58.314228+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:55.471662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716992+0000","last_change":"2023-10-26T05:49:56.278925+0000","last_active":"2023-10-26T05:49:58.716992+0000","last_peered":"2023-10-26T05:49:58.716992+0000","last_clean":"2023-10-26T05:49:58.716992+0000","last_became_active":"2023-10-26T05:49:56.278699+0000","last_became_peered":"2023-10-26T05:49:56.278699+0000","last_unstale":"2023-10-26T05:49:58.716992+0000","last_undegraded":"2023-10-26T05:49:58.716992+0000","last_fullsized":"2023-10-26T05:49:58.716992+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:47.637831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086433+0000","last_change":"2023-10-26T05:49:57.111818+0000","last_active":"2023-10-26T05:49:58.086433+0000","last_peered":"2023-10-26T05:49:58.086433+0000","last_clean":"2023-10-26T05:49:58.086433+0000","last_became_active":"2023-10-26T05:49:57.105428+0000","last_became_peered":"2023-10-26T05:49:57.105428+0000","last_unstale":"2023-10-26T05:49:58.086433+0000","last_undegraded":"2023-10-26T05:49:58.086433+0000","last_fullsized":"2023-10-26T05:49:58.086433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:51.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086470+0000","last_change":"2023-10-26T05:49:56.284883+0000","last_active":"2023-10-26T05:49:58.086470+0000","last_peered":"2023-10-26T05:49:58.086470+0000","last_clean":"2023-10-26T05:49:58.086470+0000","last_became_active":"2023-10-26T05:49:56.283583+0000","last_became_peered":"2023-10-26T05:49:56.283583+0000","last_unstale":"2023-10-26T05:49:58.086470+0000","last_undegraded":"2023-10-26T05:49:58.086470+0000","last_fullsized":"2023-10-26T05:49:58.086470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:20.669487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086331+0000","last_change":"2023-10-26T05:49:57.105500+0000","last_active":"2023-10-26T05:49:58.086331+0000","last_peered":"2023-10-26T05:49:58.086331+0000","last_clean":"2023-10-26T05:49:58.086331+0000","last_became_active":"2023-10-26T05:49:57.104941+0000","last_became_peered":"2023-10-26T05:49:57.104941+0000","last_unstale":"2023-10-26T05:49:58.086331+0000","last_undegraded":"2023-10-26T05:49:58.086331+0000","last_fullsized":"2023-10-26T05:49:58.086331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:42.142447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318204+0000","last_change":"2023-10-26T05:49:56.297704+0000","last_active":"2023-10-26T05:49:58.318204+0000","last_peered":"2023-10-26T05:49:58.318204+0000","last_clean":"2023-10-26T05:49:58.318204+0000","last_became_active":"2023-10-26T05:49:56.297264+0000","last_became_peered":"2023-10-26T05:49:56.297264+0000","last_unstale":"2023-10-26T05:49:58.318204+0000","last_undegraded":"2023-10-26T05:49:58.318204+0000","last_fullsized":"2023-10-26T05:49:58.318204+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:39:48.744303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716654+0000","last_change":"2023-10-26T05:49:57.112683+0000","last_active":"2023-10-26T05:49:58.716654+0000","last_peered":"2023-10-26T05:49:58.716654+0000","last_clean":"2023-10-26T05:49:58.716654+0000","last_became_active":"2023-10-26T05:49:57.105528+0000","last_became_peered":"2023-10-26T05:49:57.105528+0000","last_unstale":"2023-10-26T05:49:58.716654+0000","last_undegraded":"2023-10-26T05:49:58.716654+0000","last_fullsized":"2023-10-26T05:49:58.716654+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:41:44.409022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092011+0000","last_change":"2023-10-26T05:49:56.292088+0000","last_active":"2023-10-26T05:49:58.092011+0000","last_peered":"2023-10-26T05:49:58.092011+0000","last_clean":"2023-10-26T05:49:58.092011+0000","last_became_active":"2023-10-26T05:49:56.288903+0000","last_became_peered":"2023-10-26T05:49:56.288903+0000","last_unstale":"2023-10-26T05:49:58.092011+0000","last_undegraded":"2023-10-26T05:49:58.092011+0000","last_fullsized":"2023-10-26T05:49:58.092011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:15:44.185767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717083+0000","last_change":"2023-10-26T05:49:57.124648+0000","last_active":"2023-10-26T05:49:58.717083+0000","last_peered":"2023-10-26T05:49:58.717083+0000","last_clean":"2023-10-26T05:49:58.717083+0000","last_became_active":"2023-10-26T05:49:57.119895+0000","last_became_peered":"2023-10-26T05:49:57.119895+0000","last_unstale":"2023-10-26T05:49:58.717083+0000","last_undegraded":"2023-10-26T05:49:58.717083+0000","last_fullsized":"2023-10-26T05:49:58.717083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:24.430848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086280+0000","last_change":"2023-10-26T05:49:56.283792+0000","last_active":"2023-10-26T05:49:58.086280+0000","last_peered":"2023-10-26T05:49:58.086280+0000","last_clean":"2023-10-26T05:49:58.086280+0000","last_became_active":"2023-10-26T05:49:56.283304+0000","last_became_peered":"2023-10-26T05:49:56.283304+0000","last_unstale":"2023-10-26T05:49:58.086280+0000","last_undegraded":"2023-10-26T05:49:58.086280+0000","last_fullsized":"2023-10-26T05:49:58.086280+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:11.860851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317090+0000","last_change":"2023-10-26T05:49:57.116455+0000","last_active":"2023-10-26T05:49:58.317090+0000","last_peered":"2023-10-26T05:49:58.317090+0000","last_clean":"2023-10-26T05:49:58.317090+0000","last_became_active":"2023-10-26T05:49:57.115777+0000","last_became_peered":"2023-10-26T05:49:57.115777+0000","last_unstale":"2023-10-26T05:49:58.317090+0000","last_undegraded":"2023-10-26T05:49:58.317090+0000","last_fullsized":"2023-10-26T05:49:58.317090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:55.471662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317517+0000","last_change":"2023-10-26T05:49:56.290456+0000","last_active":"2023-10-26T05:49:58.317517+0000","last_peered":"2023-10-26T05:49:58.317517+0000","last_clean":"2023-10-26T05:49:58.317517+0000","last_became_active":"2023-10-26T05:49:56.285667+0000","last_became_peered":"2023-10-26T05:49:56.285667+0000","last_unstale":"2023-10-26T05:49:58.317517+0000","last_undegraded":"2023-10-26T05:49:58.317517+0000","last_fullsized":"2023-10-26T05:49:58.317517+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:16.657920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716738+0000","last_change":"2023-10-26T05:49:57.106930+0000","last_active":"2023-10-26T05:49:58.716738+0000","last_peered":"2023-10-26T05:49:58.716738+0000","last_clean":"2023-10-26T05:49:58.716738+0000","last_became_active":"2023-10-26T05:49:57.105668+0000","last_became_peered":"2023-10-26T05:49:57.105668+0000","last_unstale":"2023-10-26T05:49:58.716738+0000","last_undegraded":"2023-10-26T05:49:58.716738+0000","last_fullsized":"2023-10-26T05:49:58.716738+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:06.803512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.093587+0000","last_change":"2023-10-26T05:49:56.284243+0000","last_active":"2023-10-26T05:49:58.093587+0000","last_peered":"2023-10-26T05:49:58.093587+0000","last_clean":"2023-10-26T05:49:58.093587+0000","last_became_active":"2023-10-26T05:49:56.283521+0000","last_became_peered":"2023-10-26T05:49:56.283521+0000","last_unstale":"2023-10-26T05:49:58.093587+0000","last_undegraded":"2023-10-26T05:49:58.093587+0000","last_fullsized":"2023-10-26T05:49:58.093587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:17.053962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091775+0000","last_change":"2023-10-26T05:49:57.097532+0000","last_active":"2023-10-26T05:49:58.091775+0000","last_peered":"2023-10-26T05:49:58.091775+0000","last_clean":"2023-10-26T05:49:58.091775+0000","last_became_active":"2023-10-26T05:49:57.095879+0000","last_became_peered":"2023-10-26T05:49:57.095879+0000","last_unstale":"2023-10-26T05:49:58.091775+0000","last_undegraded":"2023-10-26T05:49:58.091775+0000","last_fullsized":"2023-10-26T05:49:58.091775+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:12.865553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.093002+0000","last_change":"2023-10-26T05:49:56.299951+0000","last_active":"2023-10-26T05:49:58.093002+0000","last_peered":"2023-10-26T05:49:58.093002+0000","last_clean":"2023-10-26T05:49:58.093002+0000","last_became_active":"2023-10-26T05:49:56.298979+0000","last_became_peered":"2023-10-26T05:49:56.298979+0000","last_unstale":"2023-10-26T05:49:58.093002+0000","last_undegraded":"2023-10-26T05:49:58.093002+0000","last_fullsized":"2023-10-26T05:49:58.093002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:05.670564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092371+0000","last_change":"2023-10-26T05:49:57.120704+0000","last_active":"2023-10-26T05:49:58.092371+0000","last_peered":"2023-10-26T05:49:58.092371+0000","last_clean":"2023-10-26T05:49:58.092371+0000","last_became_active":"2023-10-26T05:49:57.117741+0000","last_became_peered":"2023-10-26T05:49:57.117741+0000","last_unstale":"2023-10-26T05:49:58.092371+0000","last_undegraded":"2023-10-26T05:49:58.092371+0000","last_fullsized":"2023-10-26T05:49:58.092371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:43.307761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717672+0000","last_change":"2023-10-26T05:49:56.283861+0000","last_active":"2023-10-26T05:49:58.717672+0000","last_peered":"2023-10-26T05:49:58.717672+0000","last_clean":"2023-10-26T05:49:58.717672+0000","last_became_active":"2023-10-26T05:49:56.283224+0000","last_became_peered":"2023-10-26T05:49:56.283224+0000","last_unstale":"2023-10-26T05:49:58.717672+0000","last_undegraded":"2023-10-26T05:49:58.717672+0000","last_fullsized":"2023-10-26T05:49:58.717672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:48.291789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314102+0000","last_change":"2023-10-26T05:49:57.091946+0000","last_active":"2023-10-26T05:49:58.314102+0000","last_peered":"2023-10-26T05:49:58.314102+0000","last_clean":"2023-10-26T05:49:58.314102+0000","last_became_active":"2023-10-26T05:49:57.091288+0000","last_became_peered":"2023-10-26T05:49:57.091288+0000","last_unstale":"2023-10-26T05:49:58.314102+0000","last_undegraded":"2023-10-26T05:49:58.314102+0000","last_fullsized":"2023-10-26T05:49:58.314102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:02.439356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"67'3","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091437+0000","last_change":"2023-10-26T05:49:56.294929+0000","last_active":"2023-10-26T05:49:58.091437+0000","last_peered":"2023-10-26T05:49:58.091437+0000","last_clean":"2023-10-26T05:49:58.091437+0000","last_became_active":"2023-10-26T05:49:56.292173+0000","last_became_peered":"2023-10-26T05:49:56.292173+0000","last_unstale":"2023-10-26T05:49:58.091437+0000","last_undegraded":"2023-10-26T05:49:58.091437+0000","last_fullsized":"2023-10-26T05:49:58.091437+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:02.325213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091200+0000","last_change":"2023-10-26T05:49:57.117581+0000","last_active":"2023-10-26T05:49:58.091200+0000","last_peered":"2023-10-26T05:49:58.091200+0000","last_clean":"2023-10-26T05:49:58.091200+0000","last_became_active":"2023-10-26T05:49:57.116797+0000","last_became_peered":"2023-10-26T05:49:57.116797+0000","last_unstale":"2023-10-26T05:49:58.091200+0000","last_undegraded":"2023-10-26T05:49:58.091200+0000","last_fullsized":"2023-10-26T05:49:58.091200+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:01.518600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"67'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092076+0000","last_change":"2023-10-26T05:49:56.288391+0000","last_active":"2023-10-26T05:49:58.092076+0000","last_peered":"2023-10-26T05:49:58.092076+0000","last_clean":"2023-10-26T05:49:58.092076+0000","last_became_active":"2023-10-26T05:49:56.287777+0000","last_became_peered":"2023-10-26T05:49:56.287777+0000","last_unstale":"2023-10-26T05:49:58.092076+0000","last_undegraded":"2023-10-26T05:49:58.092076+0000","last_fullsized":"2023-10-26T05:49:58.092076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:29.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317991+0000","last_change":"2023-10-26T05:49:57.108754+0000","last_active":"2023-10-26T05:49:58.317991+0000","last_peered":"2023-10-26T05:49:58.317991+0000","last_clean":"2023-10-26T05:49:58.317991+0000","last_became_active":"2023-10-26T05:49:57.107345+0000","last_became_peered":"2023-10-26T05:49:57.107345+0000","last_unstale":"2023-10-26T05:49:58.317991+0000","last_undegraded":"2023-10-26T05:49:58.317991+0000","last_fullsized":"2023-10-26T05:49:58.317991+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:02.687883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717074+0000","last_change":"2023-10-26T05:49:56.286965+0000","last_active":"2023-10-26T05:49:58.717074+0000","last_peered":"2023-10-26T05:49:58.717074+0000","last_clean":"2023-10-26T05:49:58.717074+0000","last_became_active":"2023-10-26T05:49:56.286828+0000","last_became_peered":"2023-10-26T05:49:56.286828+0000","last_unstale":"2023-10-26T05:49:58.717074+0000","last_undegraded":"2023-10-26T05:49:58.717074+0000","last_fullsized":"2023-10-26T05:49:58.717074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:22:39.502248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090865+0000","last_change":"2023-10-26T05:49:57.125683+0000","last_active":"2023-10-26T05:49:58.090865+0000","last_peered":"2023-10-26T05:49:58.090865+0000","last_clean":"2023-10-26T05:49:58.090865+0000","last_became_active":"2023-10-26T05:49:57.124428+0000","last_became_peered":"2023-10-26T05:49:57.124428+0000","last_unstale":"2023-10-26T05:49:58.090865+0000","last_undegraded":"2023-10-26T05:49:58.090865+0000","last_fullsized":"2023-10-26T05:49:58.090865+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:13.861096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316975+0000","last_change":"2023-10-26T05:49:56.299611+0000","last_active":"2023-10-26T05:49:58.316975+0000","last_peered":"2023-10-26T05:49:58.316975+0000","last_clean":"2023-10-26T05:49:58.316975+0000","last_became_active":"2023-10-26T05:49:56.298909+0000","last_became_peered":"2023-10-26T05:49:56.298909+0000","last_unstale":"2023-10-26T05:49:58.316975+0000","last_undegraded":"2023-10-26T05:49:58.316975+0000","last_fullsized":"2023-10-26T05:49:58.316975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:58.589719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091378+0000","last_change":"2023-10-26T05:49:57.122737+0000","last_active":"2023-10-26T05:49:58.091378+0000","last_peered":"2023-10-26T05:49:58.091378+0000","last_clean":"2023-10-26T05:49:58.091378+0000","last_became_active":"2023-10-26T05:49:57.122071+0000","last_became_peered":"2023-10-26T05:49:57.122071+0000","last_unstale":"2023-10-26T05:49:58.091378+0000","last_undegraded":"2023-10-26T05:49:58.091378+0000","last_fullsized":"2023-10-26T05:49:58.091378+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:15.549743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318194+0000","last_change":"2023-10-26T05:49:56.292821+0000","last_active":"2023-10-26T05:49:58.318194+0000","last_peered":"2023-10-26T05:49:58.318194+0000","last_clean":"2023-10-26T05:49:58.318194+0000","last_became_active":"2023-10-26T05:49:56.291455+0000","last_became_peered":"2023-10-26T05:49:56.291455+0000","last_unstale":"2023-10-26T05:49:58.318194+0000","last_undegraded":"2023-10-26T05:49:58.318194+0000","last_fullsized":"2023-10-26T05:49:58.318194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:03:15.199063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090434+0000","last_change":"2023-10-26T05:49:57.127709+0000","last_active":"2023-10-26T05:49:58.090434+0000","last_peered":"2023-10-26T05:49:58.090434+0000","last_clean":"2023-10-26T05:49:58.090434+0000","last_became_active":"2023-10-26T05:49:57.127101+0000","last_became_peered":"2023-10-26T05:49:57.127101+0000","last_unstale":"2023-10-26T05:49:58.090434+0000","last_undegraded":"2023-10-26T05:49:58.090434+0000","last_fullsized":"2023-10-26T05:49:58.090434+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:16.970220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"67'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091662+0000","last_change":"2023-10-26T05:49:56.295402+0000","last_active":"2023-10-26T05:49:58.091662+0000","last_peered":"2023-10-26T05:49:58.091662+0000","last_clean":"2023-10-26T05:49:58.091662+0000","last_became_active":"2023-10-26T05:49:56.294000+0000","last_became_peered":"2023-10-26T05:49:56.294000+0000","last_unstale":"2023-10-26T05:49:58.091662+0000","last_undegraded":"2023-10-26T05:49:58.091662+0000","last_fullsized":"2023-10-26T05:49:58.091662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:50.215466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091187+0000","last_change":"2023-10-26T05:49:57.125220+0000","last_active":"2023-10-26T05:49:58.091187+0000","last_peered":"2023-10-26T05:49:58.091187+0000","last_clean":"2023-10-26T05:49:58.091187+0000","last_became_active":"2023-10-26T05:49:57.120408+0000","last_became_peered":"2023-10-26T05:49:57.120408+0000","last_unstale":"2023-10-26T05:49:58.091187+0000","last_undegraded":"2023-10-26T05:49:58.091187+0000","last_fullsized":"2023-10-26T05:49:58.091187+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:02.283721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717244+0000","last_change":"2023-10-26T05:49:56.281450+0000","last_active":"2023-10-26T05:49:58.717244+0000","last_peered":"2023-10-26T05:49:58.717244+0000","last_clean":"2023-10-26T05:49:58.717244+0000","last_became_active":"2023-10-26T05:49:56.281258+0000","last_became_peered":"2023-10-26T05:49:56.281258+0000","last_unstale":"2023-10-26T05:49:58.717244+0000","last_undegraded":"2023-10-26T05:49:58.717244+0000","last_fullsized":"2023-10-26T05:49:58.717244+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:31:52.089193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317040+0000","last_change":"2023-10-26T05:49:57.127099+0000","last_active":"2023-10-26T05:49:58.317040+0000","last_peered":"2023-10-26T05:49:58.317040+0000","last_clean":"2023-10-26T05:49:58.317040+0000","last_became_active":"2023-10-26T05:49:57.118602+0000","last_became_peered":"2023-10-26T05:49:57.118602+0000","last_unstale":"2023-10-26T05:49:58.317040+0000","last_undegraded":"2023-10-26T05:49:58.317040+0000","last_fullsized":"2023-10-26T05:49:58.317040+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:03.329914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086512+0000","last_change":"2023-10-26T05:49:56.284544+0000","last_active":"2023-10-26T05:49:58.086512+0000","last_peered":"2023-10-26T05:49:58.086512+0000","last_clean":"2023-10-26T05:49:58.086512+0000","last_became_active":"2023-10-26T05:49:56.282775+0000","last_became_peered":"2023-10-26T05:49:56.282775+0000","last_unstale":"2023-10-26T05:49:58.086512+0000","last_undegraded":"2023-10-26T05:49:58.086512+0000","last_fullsized":"2023-10-26T05:49:58.086512+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:42.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717248+0000","last_change":"2023-10-26T05:49:57.122032+0000","last_active":"2023-10-26T05:49:58.717248+0000","last_peered":"2023-10-26T05:49:58.717248+0000","last_clean":"2023-10-26T05:49:58.717248+0000","last_became_active":"2023-10-26T05:49:57.117067+0000","last_became_peered":"2023-10-26T05:49:57.117067+0000","last_unstale":"2023-10-26T05:49:58.717248+0000","last_undegraded":"2023-10-26T05:49:58.717248+0000","last_fullsized":"2023-10-26T05:49:58.717248+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:53:41.032022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318352+0000","last_change":"2023-10-26T05:49:56.296801+0000","last_active":"2023-10-26T05:49:58.318352+0000","last_peered":"2023-10-26T05:49:58.318352+0000","last_clean":"2023-10-26T05:49:58.318352+0000","last_became_active":"2023-10-26T05:49:56.296122+0000","last_became_peered":"2023-10-26T05:49:56.296122+0000","last_unstale":"2023-10-26T05:49:58.318352+0000","last_undegraded":"2023-10-26T05:49:58.318352+0000","last_fullsized":"2023-10-26T05:49:58.318352+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:16:43.154863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092545+0000","last_change":"2023-10-26T05:49:57.122024+0000","last_active":"2023-10-26T05:49:58.092545+0000","last_peered":"2023-10-26T05:49:58.092545+0000","last_clean":"2023-10-26T05:49:58.092545+0000","last_became_active":"2023-10-26T05:49:57.119774+0000","last_became_peered":"2023-10-26T05:49:57.119774+0000","last_unstale":"2023-10-26T05:49:58.092545+0000","last_undegraded":"2023-10-26T05:49:58.092545+0000","last_fullsized":"2023-10-26T05:49:58.092545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:29.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086341+0000","last_change":"2023-10-26T05:49:56.284036+0000","last_active":"2023-10-26T05:49:58.086341+0000","last_peered":"2023-10-26T05:49:58.086341+0000","last_clean":"2023-10-26T05:49:58.086341+0000","last_became_active":"2023-10-26T05:49:56.283798+0000","last_became_peered":"2023-10-26T05:49:56.283798+0000","last_unstale":"2023-10-26T05:49:58.086341+0000","last_undegraded":"2023-10-26T05:49:58.086341+0000","last_fullsized":"2023-10-26T05:49:58.086341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.226590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317393+0000","last_change":"2023-10-26T05:49:57.125312+0000","last_active":"2023-10-26T05:49:58.317393+0000","last_peered":"2023-10-26T05:49:58.317393+0000","last_clean":"2023-10-26T05:49:58.317393+0000","last_became_active":"2023-10-26T05:49:57.117770+0000","last_became_peered":"2023-10-26T05:49:57.117770+0000","last_unstale":"2023-10-26T05:49:58.317393+0000","last_undegraded":"2023-10-26T05:49:58.317393+0000","last_fullsized":"2023-10-26T05:49:58.317393+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:51.220167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717013+0000","last_change":"2023-10-26T05:49:56.287347+0000","last_active":"2023-10-26T05:49:58.717013+0000","last_peered":"2023-10-26T05:49:58.717013+0000","last_clean":"2023-10-26T05:49:58.717013+0000","last_became_active":"2023-10-26T05:49:56.286313+0000","last_became_peered":"2023-10-26T05:49:56.286313+0000","last_unstale":"2023-10-26T05:49:58.717013+0000","last_undegraded":"2023-10-26T05:49:58.717013+0000","last_fullsized":"2023-10-26T05:49:58.717013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:50.857996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086091+0000","last_change":"2023-10-26T05:49:57.101807+0000","last_active":"2023-10-26T05:49:58.086091+0000","last_peered":"2023-10-26T05:49:58.086091+0000","last_clean":"2023-10-26T05:49:58.086091+0000","last_became_active":"2023-10-26T05:49:57.101089+0000","last_became_peered":"2023-10-26T05:49:57.101089+0000","last_unstale":"2023-10-26T05:49:58.086091+0000","last_undegraded":"2023-10-26T05:49:58.086091+0000","last_fullsized":"2023-10-26T05:49:58.086091+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:29.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314446+0000","last_change":"2023-10-26T05:49:56.279916+0000","last_active":"2023-10-26T05:49:58.314446+0000","last_peered":"2023-10-26T05:49:58.314446+0000","last_clean":"2023-10-26T05:49:58.314446+0000","last_became_active":"2023-10-26T05:49:56.279676+0000","last_became_peered":"2023-10-26T05:49:56.279676+0000","last_unstale":"2023-10-26T05:49:58.314446+0000","last_undegraded":"2023-10-26T05:49:58.314446+0000","last_fullsized":"2023-10-26T05:49:58.314446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.573964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086424+0000","last_change":"2023-10-26T05:49:57.111387+0000","last_active":"2023-10-26T05:49:58.086424+0000","last_peered":"2023-10-26T05:49:58.086424+0000","last_clean":"2023-10-26T05:49:58.086424+0000","last_became_active":"2023-10-26T05:49:57.110755+0000","last_became_peered":"2023-10-26T05:49:57.110755+0000","last_unstale":"2023-10-26T05:49:58.086424+0000","last_undegraded":"2023-10-26T05:49:58.086424+0000","last_fullsized":"2023-10-26T05:49:58.086424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:28.417446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318537+0000","last_change":"2023-10-26T05:49:56.287398+0000","last_active":"2023-10-26T05:49:58.318537+0000","last_peered":"2023-10-26T05:49:58.318537+0000","last_clean":"2023-10-26T05:49:58.318537+0000","last_became_active":"2023-10-26T05:49:56.285793+0000","last_became_peered":"2023-10-26T05:49:56.285793+0000","last_unstale":"2023-10-26T05:49:58.318537+0000","last_undegraded":"2023-10-26T05:49:58.318537+0000","last_fullsized":"2023-10-26T05:49:58.318537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:19:40.984838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316743+0000","last_change":"2023-10-26T05:49:57.105773+0000","last_active":"2023-10-26T05:49:58.316743+0000","last_peered":"2023-10-26T05:49:58.316743+0000","last_clean":"2023-10-26T05:49:58.316743+0000","last_became_active":"2023-10-26T05:49:57.104838+0000","last_became_peered":"2023-10-26T05:49:57.104838+0000","last_unstale":"2023-10-26T05:49:58.316743+0000","last_undegraded":"2023-10-26T05:49:58.316743+0000","last_fullsized":"2023-10-26T05:49:58.316743+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:15:45.190469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314141+0000","last_change":"2023-10-26T05:49:56.284411+0000","last_active":"2023-10-26T05:49:58.314141+0000","last_peered":"2023-10-26T05:49:58.314141+0000","last_clean":"2023-10-26T05:49:58.314141+0000","last_became_active":"2023-10-26T05:49:56.284223+0000","last_became_peered":"2023-10-26T05:49:56.284223+0000","last_unstale":"2023-10-26T05:49:58.314141+0000","last_undegraded":"2023-10-26T05:49:58.314141+0000","last_fullsized":"2023-10-26T05:49:58.314141+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:37.644982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317988+0000","last_change":"2023-10-26T05:49:57.111877+0000","last_active":"2023-10-26T05:49:58.317988+0000","last_peered":"2023-10-26T05:49:58.317988+0000","last_clean":"2023-10-26T05:49:58.317988+0000","last_became_active":"2023-10-26T05:49:57.110262+0000","last_became_peered":"2023-10-26T05:49:57.110262+0000","last_unstale":"2023-10-26T05:49:58.317988+0000","last_undegraded":"2023-10-26T05:49:58.317988+0000","last_fullsized":"2023-10-26T05:49:58.317988+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:15.549743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086367+0000","last_change":"2023-10-26T05:49:56.283662+0000","last_active":"2023-10-26T05:49:58.086367+0000","last_peered":"2023-10-26T05:49:58.086367+0000","last_clean":"2023-10-26T05:49:58.086367+0000","last_became_active":"2023-10-26T05:49:56.283003+0000","last_became_peered":"2023-10-26T05:49:56.283003+0000","last_unstale":"2023-10-26T05:49:58.086367+0000","last_undegraded":"2023-10-26T05:49:58.086367+0000","last_fullsized":"2023-10-26T05:49:58.086367+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:12.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314259+0000","last_change":"2023-10-26T05:49:57.097257+0000","last_active":"2023-10-26T05:49:58.314259+0000","last_peered":"2023-10-26T05:49:58.314259+0000","last_clean":"2023-10-26T05:49:58.314259+0000","last_became_active":"2023-10-26T05:49:57.091200+0000","last_became_peered":"2023-10-26T05:49:57.091200+0000","last_unstale":"2023-10-26T05:49:58.314259+0000","last_undegraded":"2023-10-26T05:49:58.314259+0000","last_fullsized":"2023-10-26T05:49:58.314259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:11:57.244632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316856+0000","last_change":"2023-10-26T05:49:56.289528+0000","last_active":"2023-10-26T05:49:58.316856+0000","last_peered":"2023-10-26T05:49:58.316856+0000","last_clean":"2023-10-26T05:49:58.316856+0000","last_became_active":"2023-10-26T05:49:56.288670+0000","last_became_peered":"2023-10-26T05:49:56.288670+0000","last_unstale":"2023-10-26T05:49:58.316856+0000","last_undegraded":"2023-10-26T05:49:58.316856+0000","last_fullsized":"2023-10-26T05:49:58.316856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.573964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086251+0000","last_change":"2023-10-26T05:49:57.121953+0000","last_active":"2023-10-26T05:49:58.086251+0000","last_peered":"2023-10-26T05:49:58.086251+0000","last_clean":"2023-10-26T05:49:58.086251+0000","last_became_active":"2023-10-26T05:49:57.121345+0000","last_became_peered":"2023-10-26T05:49:57.121345+0000","last_unstale":"2023-10-26T05:49:58.086251+0000","last_undegraded":"2023-10-26T05:49:58.086251+0000","last_fullsized":"2023-10-26T05:49:58.086251+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:48.642532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317562+0000","last_change":"2023-10-26T05:49:56.281089+0000","last_active":"2023-10-26T05:49:58.317562+0000","last_peered":"2023-10-26T05:49:58.317562+0000","last_clean":"2023-10-26T05:49:58.317562+0000","last_became_active":"2023-10-26T05:49:56.280344+0000","last_became_peered":"2023-10-26T05:49:56.280344+0000","last_unstale":"2023-10-26T05:49:58.317562+0000","last_undegraded":"2023-10-26T05:49:58.317562+0000","last_fullsized":"2023-10-26T05:49:58.317562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:00.295875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717499+0000","last_change":"2023-10-26T05:49:57.125175+0000","last_active":"2023-10-26T05:49:58.717499+0000","last_peered":"2023-10-26T05:49:58.717499+0000","last_clean":"2023-10-26T05:49:58.717499+0000","last_became_active":"2023-10-26T05:49:57.120634+0000","last_became_peered":"2023-10-26T05:49:57.120634+0000","last_unstale":"2023-10-26T05:49:58.717499+0000","last_undegraded":"2023-10-26T05:49:58.717499+0000","last_fullsized":"2023-10-26T05:49:58.717499+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:22:34.413050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314433+0000","last_change":"2023-10-26T05:49:56.280398+0000","last_active":"2023-10-26T05:49:58.314433+0000","last_peered":"2023-10-26T05:49:58.314433+0000","last_clean":"2023-10-26T05:49:58.314433+0000","last_became_active":"2023-10-26T05:49:56.280161+0000","last_became_peered":"2023-10-26T05:49:56.280161+0000","last_unstale":"2023-10-26T05:49:58.314433+0000","last_undegraded":"2023-10-26T05:49:58.314433+0000","last_fullsized":"2023-10-26T05:49:58.314433+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:41:22.376152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092708+0000","last_change":"2023-10-26T05:49:57.124273+0000","last_active":"2023-10-26T05:49:58.092708+0000","last_peered":"2023-10-26T05:49:58.092708+0000","last_clean":"2023-10-26T05:49:58.092708+0000","last_became_active":"2023-10-26T05:49:57.123585+0000","last_became_peered":"2023-10-26T05:49:57.123585+0000","last_unstale":"2023-10-26T05:49:58.092708+0000","last_undegraded":"2023-10-26T05:49:58.092708+0000","last_fullsized":"2023-10-26T05:49:58.092708+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:09:43.591005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092224+0000","last_change":"2023-10-26T05:49:56.292615+0000","last_active":"2023-10-26T05:49:58.092224+0000","last_peered":"2023-10-26T05:49:58.092224+0000","last_clean":"2023-10-26T05:49:58.092224+0000","last_became_active":"2023-10-26T05:49:56.291283+0000","last_became_peered":"2023-10-26T05:49:56.291283+0000","last_unstale":"2023-10-26T05:49:58.092224+0000","last_undegraded":"2023-10-26T05:49:58.092224+0000","last_fullsized":"2023-10-26T05:49:58.092224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:02.325213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314321+0000","last_change":"2023-10-26T05:49:57.098875+0000","last_active":"2023-10-26T05:49:58.314321+0000","last_peered":"2023-10-26T05:49:58.314321+0000","last_clean":"2023-10-26T05:49:58.314321+0000","last_became_active":"2023-10-26T05:49:57.098279+0000","last_became_peered":"2023-10-26T05:49:57.098279+0000","last_unstale":"2023-10-26T05:49:58.314321+0000","last_undegraded":"2023-10-26T05:49:58.314321+0000","last_fullsized":"2023-10-26T05:49:58.314321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:45.510140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314342+0000","last_change":"2023-10-26T05:49:56.280279+0000","last_active":"2023-10-26T05:49:58.314342+0000","last_peered":"2023-10-26T05:49:58.314342+0000","last_clean":"2023-10-26T05:49:58.314342+0000","last_became_active":"2023-10-26T05:49:56.279945+0000","last_became_peered":"2023-10-26T05:49:56.279945+0000","last_unstale":"2023-10-26T05:49:58.314342+0000","last_undegraded":"2023-10-26T05:49:58.314342+0000","last_fullsized":"2023-10-26T05:49:58.314342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:58.589719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716875+0000","last_change":"2023-10-26T05:49:57.116325+0000","last_active":"2023-10-26T05:49:58.716875+0000","last_peered":"2023-10-26T05:49:58.716875+0000","last_clean":"2023-10-26T05:49:58.716875+0000","last_became_active":"2023-10-26T05:49:57.111986+0000","last_became_peered":"2023-10-26T05:49:57.111986+0000","last_unstale":"2023-10-26T05:49:58.716875+0000","last_undegraded":"2023-10-26T05:49:58.716875+0000","last_fullsized":"2023-10-26T05:49:58.716875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:43.777929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717454+0000","last_change":"2023-10-26T05:49:56.288356+0000","last_active":"2023-10-26T05:49:58.717454+0000","last_peered":"2023-10-26T05:49:58.717454+0000","last_clean":"2023-10-26T05:49:58.717454+0000","last_became_active":"2023-10-26T05:49:56.288093+0000","last_became_peered":"2023-10-26T05:49:56.288093+0000","last_unstale":"2023-10-26T05:49:58.717454+0000","last_undegraded":"2023-10-26T05:49:58.717454+0000","last_fullsized":"2023-10-26T05:49:58.717454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:41.137745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717220+0000","last_change":"2023-10-26T05:49:57.114374+0000","last_active":"2023-10-26T05:49:58.717220+0000","last_peered":"2023-10-26T05:49:58.717220+0000","last_clean":"2023-10-26T05:49:58.717220+0000","last_became_active":"2023-10-26T05:49:57.113762+0000","last_became_peered":"2023-10-26T05:49:57.113762+0000","last_unstale":"2023-10-26T05:49:58.717220+0000","last_undegraded":"2023-10-26T05:49:58.717220+0000","last_fullsized":"2023-10-26T05:49:58.717220+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:45:49.439355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317309+0000","last_change":"2023-10-26T05:49:56.282808+0000","last_active":"2023-10-26T05:49:58.317309+0000","last_peered":"2023-10-26T05:49:58.317309+0000","last_clean":"2023-10-26T05:49:58.317309+0000","last_became_active":"2023-10-26T05:49:56.282196+0000","last_became_peered":"2023-10-26T05:49:56.282196+0000","last_unstale":"2023-10-26T05:49:58.317309+0000","last_undegraded":"2023-10-26T05:49:58.317309+0000","last_fullsized":"2023-10-26T05:49:58.317309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:26.957737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":209,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-26T05:49:57.105151+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:49:57.105151+0000","last_peered":"2023-10-26T05:49:57.105151+0000","last_clean":"2023-10-26T05:49:57.105151+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:49:57.105151+0000","last_undegraded":"2023-10-26T05:49:57.105151+0000","last_fullsized":"2023-10-26T05:49:57.105151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:31:55.971230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194391,"statfs":{"total":200000000,"available":199056957,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.216,"5min":1.347,"15min":1.347},"last":1.014},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82899999999999996,"15min":0.82999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.262,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.0369999999999999,"15min":1.0580000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.349,"5min":1.391,"15min":1.391},"last":0.877},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95999999999999996,"15min":0.96999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.93899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.123,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.093}]},{"osd":3,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88100000000000001,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1910000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.96499999999999997,"15min":0.97099999999999997},"min":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.26,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.232,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.75},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.93799999999999994,"15min":0.93500000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.131,"15min":1.1459999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.292,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.149},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.093,"15min":1.105},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3200000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.93100000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1200000000000001,"15min":1.125},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3979999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0469999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5469999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.94999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.80500000000000005},"min":{"1min":0.61499999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1879999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2090000000000001,"5min":1.3,"15min":1.3},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1779999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.84399999999999997,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.79700000000000004,"15min":0.78100000000000003},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.141},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.83899999999999997,"15min":0.82499999999999996},"min":{"1min":0.68200000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85299999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.873,"15min":0.86299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.04},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.95499999999999996,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.96999999999999997,"15min":0.96799999999999997},"min":{"1min":0.67200000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2609999999999999,"5min":1.325,"15min":1.325},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.98799999999999999,"15min":0.97599999999999998},"min":{"1min":0.75600000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.73099999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3220000000000001,"5min":1.554,"15min":1.554},"last":1.115},{"interface":"front","average":{"1min":1.008,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1200000000000001,"15min":1.1060000000000001},"min":{"1min":0.92200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.133,"15min":1.1279999999999999},"min":{"1min":0.96799999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.333,"5min":1.464,"15min":1.464},"last":1.008}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194967,"statfs":{"total":200000000,"available":199646833,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.107},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95499999999999996,"15min":0.94999999999999996},"min":{"1min":0.64400000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.90300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1100000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.081},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1779999999999999,"5min":1.329,"15min":1.329},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.92500000000000004,"15min":0.93700000000000006},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4219999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.127,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.002}]},{"osd":3,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.92000000000000004,"15min":0.92400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2270000000000001,"5min":1.548,"15min":1.548},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2569999999999999,"5min":1.587,"15min":1.587},"last":1.147}]},{"osd":4,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0109999999999999,"15min":1.024},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.995,"5min":1.0960000000000001,"15min":1.113},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.458,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.355}]},{"osd":6,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0249999999999999,"15min":1.0489999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3129999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0329999999999999,"15min":1.042},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.1859999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0860000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.163,"15min":1.1819999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5029999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2110000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":345,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194966,"statfs":{"total":200000000,"available":199645873,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.873,"15min":0.88},"min":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1299999999999999,"5min":1.502,"15min":1.502},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.206,"5min":1.375,"15min":1.375},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.88300000000000001,"15min":0.878},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.79300000000000004},"min":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0740000000000001,"5min":1.29,"15min":1.29},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.55500000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93500000000000005,"15min":0.94499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.998,"5min":0.94199999999999995,"15min":0.93200000000000005},"min":{"1min":0.70199999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.296},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.002,"15min":1},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.97699999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.503,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.012},{"interface":"front","average":{"1min":1.004,"5min":1.032,"15min":1.0369999999999999},"min":{"1min":0.67900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2270000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.379}]},{"osd":7,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.03,"15min":1.03},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.069},{"interface":"front","average":{"1min":1.087,"5min":1.1080000000000001,"15min":1.1120000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.526,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.2270000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.095,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.308}]},{"osd":1,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.63500000000000001,"15min":0.65300000000000002},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94199999999999995,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73599999999999999,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.57399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1040000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90700000000000003,"15min":0.92000000000000004},"min":{"1min":0.59299999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2609999999999999,"5min":1.3,"15min":1.3},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1259999999999999,"5min":1.478,"15min":1.478},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.89300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":1.01},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2969999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.006,"15min":1.012},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.214,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.61399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0349999999999999,"15min":1.0449999999999999},"min":{"1min":0.65200000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.206,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.89700000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80300000000000005,"5min":0.87,"15min":0.87},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.72799999999999998,"15min":0.70599999999999996},"min":{"1min":0.39200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.502},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.33900000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.88800000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.623}]},{"osd":4,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.05,"5min":0.85399999999999998,"15min":0.82099999999999995},"min":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":5.9550000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.91600000000000004,"15min":0.88600000000000001},"min":{"1min":0.65900000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.875,"15min":0.871},"min":{"1min":0.58199999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.242,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.304,"5min":0.98899999999999999,"15min":0.93700000000000006},"min":{"1min":0.56599999999999995,"5min":0.5,"15min":0.5},"max":{"1min":6.7729999999999997,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.878,"15min":0.876},"min":{"1min":0.67800000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1759999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.764,"5min":1.1040000000000001,"15min":0.99399999999999999},"min":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":15.352,"5min":15.352,"15min":15.352},"last":0.94399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.875,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90300000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96599999999999997,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0609999999999999,"5min":1.264,"15min":1.264},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.754,"15min":0.77000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.054,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.32000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86399999999999999,"15min":0.86099999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4330000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.044,"5min":0.95599999999999996,"15min":0.94099999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.96599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92800000000000005,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.373,"15min":1.373},"last":0.628}]},{"osd":6,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.96299999999999997,"15min":0.96899999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2589999999999999,"5min":1.837,"15min":1.837},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.296,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.194,"5min":1.113,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2430000000000003,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.0109999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":922,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199054923,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64200000000000002,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0800000000000001,"5min":1.177,"15min":1.177},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86399999999999999,"5min":1.048,"15min":1.048},"last":1.131}]},{"osd":4,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.84699999999999998,"15min":0.83399999999999996},"min":{"1min":0.56100000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.095},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89000000000000001,"15min":0.88300000000000001},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.054}]},{"osd":5,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91600000000000004,"15min":0.91700000000000004},"min":{"1min":0.64500000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.347,"5min":1.609,"15min":1.609},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.89600000000000002,"15min":0.89300000000000002},"min":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.119,"5min":1.208,"15min":1.208},"last":1.167}]},{"osd":6,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.90500000000000003,"15min":0.89500000000000002},"min":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2569999999999999,"5min":1.331,"15min":1.331},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95099999999999996,"15min":0.94299999999999995},"min":{"1min":0.66400000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.2749999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98899999999999999,"15min":0.99299999999999999},"min":{"1min":0.626,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.337,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.026},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.68200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.204,"5min":1.306,"15min":1.306},"last":1.3420000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T05:50:00.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:50:00.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 -- 172.21.15.71:0/3668610006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 msgr2=0x7f99fc15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 --2- 172.21.15.71:0/3668610006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 0x7f99fc15b4b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009840 tx=0x7f99e401c800 comp rx=0 tx=0).stop 2023-10-26T05:50:00.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 -- 172.21.15.71:0/3668610006 shutdown_connections 2023-10-26T05:50:00.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 --2- 172.21.15.71:0/3668610006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 0x7f99fc15b4b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 --2- 172.21.15.71:0/3668610006 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc13d420 0x7f99fc0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 -- 172.21.15.71:0/3668610006 >> 172.21.15.71:0/3668610006 conn(0x7f99fc069770 msgr2=0x7f99fc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 -- 172.21.15.71:0/3668610006 shutdown_connections 2023-10-26T05:50:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 -- 172.21.15.71:0/3668610006 wait complete. 2023-10-26T05:50:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 Processor -- start 2023-10-26T05:50:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.469+0000 7f9a01989640 1 -- start start 2023-10-26T05:50:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc13d420 0x7f99fc139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 0x7f99fc0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 0x7f99fc135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc135f80 con 0x7f99fc13dc20 2023-10-26T05:50:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0ae990 con 0x7f99fc13d800 2023-10-26T05:50:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc135850 con 0x7f99fc13d420 2023-10-26T05:50:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 0x7f99fc135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 0x7f99fc135310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53244/0 (socket says 172.21.15.71:53244) 2023-10-26T05:50:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 -- 172.21.15.71:0/2844850660 learned_addr learned my addr 172.21.15.71:0/2844850660 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99faffd640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc13d420 0x7f99fc139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f27fc640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 0x7f99fc0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 -- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc13d420 msgr2=0x7f99fc139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc13d420 0x7f99fc139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 -- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 msgr2=0x7f99fc0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 0x7f99fc0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 -- 172.21.15.71:0/2844850660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40094a0 con 0x7f99fc13dc20 2023-10-26T05:50:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99fb7fe640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 0x7f99fc135310 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c790 tx=0x7f99f400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4006d60 con 0x7f99fc13dc20 2023-10-26T05:50:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99f4003980 con 0x7f99fc13dc20 2023-10-26T05:50:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc135be0 con 0x7f99fc13dc20 2023-10-26T05:50:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4003d90 con 0x7f99fc13dc20 2023-10-26T05:50:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc139fd0 con 0x7f99fc13dc20 2023-10-26T05:50:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc066b10 con 0x7f99fc13dc20 2023-10-26T05:50:00.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f99f40052e0 con 0x7f99fc13dc20 2023-10-26T05:50:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f8ff9640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99dc0664a0 0x7f99dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99f4084660 con 0x7f99fc13dc20 2023-10-26T05:50:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.473+0000 7f99faffd640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99dc0664a0 0x7f99dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.477+0000 7f99faffd640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99dc0664a0 0x7f99dc068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f99fc06b280 tx=0x7f99ec00a370 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:00.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.477+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f99f4051c50 con 0x7f99fc13dc20 2023-10-26T05:50:00.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.625+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99fc135a40 con 0x7f99dc0664a0 2023-10-26T05:50:00.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.649+0000 7f99f8ff9640 1 -- 172.21.15.71:0/2844850660 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284161 (secure 0 0 0) 0x7f99fc135a40 con 0x7f99dc0664a0 2023-10-26T05:50:00.656 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:50:00.659 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:50:00.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99dc0664a0 msgr2=0x7f99dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99dc0664a0 0x7f99dc068960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f99fc06b280 tx=0x7f99ec00a370 comp rx=0 tx=0).stop 2023-10-26T05:50:00.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 msgr2=0x7f99fc135310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 0x7f99fc135310 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c790 tx=0x7f99f400cb90 comp rx=0 tx=0).stop 2023-10-26T05:50:00.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 shutdown_connections 2023-10-26T05:50:00.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99dc0664a0 0x7f99dc068960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99fc13dc20 0x7f99fc135310 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99fc13d800 0x7f99fc0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 --2- 172.21.15.71:0/2844850660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99fc13d420 0x7f99fc139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 >> 172.21.15.71:0/2844850660 conn(0x7f99fc069770 msgr2=0x7f99fc069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:00.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 shutdown_connections 2023-10-26T05:50:00.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.653+0000 7f9a01989640 1 -- 172.21.15.71:0/2844850660 wait complete. 2023-10-26T05:50:00.672 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2023-10-26T05:49:59.464855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3911,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558588,"statfs":{"total":1600000000,"available":1595994324,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.210283"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225276+0000","last_change":"2023-10-26T05:49:57.114010+0000","last_active":"2023-10-26T05:49:58.225276+0000","last_peered":"2023-10-26T05:49:58.225276+0000","last_clean":"2023-10-26T05:49:58.225276+0000","last_became_active":"2023-10-26T05:49:57.113315+0000","last_became_peered":"2023-10-26T05:49:57.113315+0000","last_unstale":"2023-10-26T05:49:58.225276+0000","last_undegraded":"2023-10-26T05:49:58.225276+0000","last_fullsized":"2023-10-26T05:49:58.225276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:10.952396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716521+0000","last_change":"2023-10-26T05:49:57.126680+0000","last_active":"2023-10-26T05:49:58.716521+0000","last_peered":"2023-10-26T05:49:58.716521+0000","last_clean":"2023-10-26T05:49:58.716521+0000","last_became_active":"2023-10-26T05:49:57.124014+0000","last_became_peered":"2023-10-26T05:49:57.124014+0000","last_unstale":"2023-10-26T05:49:58.716521+0000","last_undegraded":"2023-10-26T05:49:58.716521+0000","last_fullsized":"2023-10-26T05:49:58.716521+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:33:56.377336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090212+0000","last_change":"2023-10-26T05:49:57.128061+0000","last_active":"2023-10-26T05:49:58.090212+0000","last_peered":"2023-10-26T05:49:58.090212+0000","last_clean":"2023-10-26T05:49:58.090212+0000","last_became_active":"2023-10-26T05:49:57.124229+0000","last_became_peered":"2023-10-26T05:49:57.124229+0000","last_unstale":"2023-10-26T05:49:58.090212+0000","last_undegraded":"2023-10-26T05:49:58.090212+0000","last_fullsized":"2023-10-26T05:49:58.090212+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:02.687883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316712+0000","last_change":"2023-10-26T05:49:57.124535+0000","last_active":"2023-10-26T05:49:58.316712+0000","last_peered":"2023-10-26T05:49:58.316712+0000","last_clean":"2023-10-26T05:49:58.316712+0000","last_became_active":"2023-10-26T05:49:57.116839+0000","last_became_peered":"2023-10-26T05:49:57.116839+0000","last_unstale":"2023-10-26T05:49:58.316712+0000","last_undegraded":"2023-10-26T05:49:58.316712+0000","last_fullsized":"2023-10-26T05:49:58.316712+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:05:46.958437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225666+0000","last_change":"2023-10-26T05:49:57.103915+0000","last_active":"2023-10-26T05:49:58.225666+0000","last_peered":"2023-10-26T05:49:58.225666+0000","last_clean":"2023-10-26T05:49:58.225666+0000","last_became_active":"2023-10-26T05:49:57.103326+0000","last_became_peered":"2023-10-26T05:49:57.103326+0000","last_unstale":"2023-10-26T05:49:58.225666+0000","last_undegraded":"2023-10-26T05:49:58.225666+0000","last_fullsized":"2023-10-26T05:49:58.225666+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:12.957362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086092+0000","last_change":"2023-10-26T05:49:57.112223+0000","last_active":"2023-10-26T05:49:58.086092+0000","last_peered":"2023-10-26T05:49:58.086092+0000","last_clean":"2023-10-26T05:49:58.086092+0000","last_became_active":"2023-10-26T05:49:57.106259+0000","last_became_peered":"2023-10-26T05:49:57.106259+0000","last_unstale":"2023-10-26T05:49:58.086092+0000","last_undegraded":"2023-10-26T05:49:58.086092+0000","last_fullsized":"2023-10-26T05:49:58.086092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:09:43.591005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090913+0000","last_change":"2023-10-26T05:49:57.104754+0000","last_active":"2023-10-26T05:49:58.090913+0000","last_peered":"2023-10-26T05:49:58.090913+0000","last_clean":"2023-10-26T05:49:58.090913+0000","last_became_active":"2023-10-26T05:49:57.104120+0000","last_became_peered":"2023-10-26T05:49:57.104120+0000","last_unstale":"2023-10-26T05:49:58.090913+0000","last_undegraded":"2023-10-26T05:49:58.090913+0000","last_fullsized":"2023-10-26T05:49:58.090913+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:06.675265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716288+0000","last_change":"2023-10-26T05:49:57.118257+0000","last_active":"2023-10-26T05:49:58.716288+0000","last_peered":"2023-10-26T05:49:58.716288+0000","last_clean":"2023-10-26T05:49:58.716288+0000","last_became_active":"2023-10-26T05:49:57.115846+0000","last_became_peered":"2023-10-26T05:49:57.115846+0000","last_unstale":"2023-10-26T05:49:58.716288+0000","last_undegraded":"2023-10-26T05:49:58.716288+0000","last_fullsized":"2023-10-26T05:49:58.716288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:49.437924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091523+0000","last_change":"2023-10-26T05:49:57.104843+0000","last_active":"2023-10-26T05:49:58.091523+0000","last_peered":"2023-10-26T05:49:58.091523+0000","last_clean":"2023-10-26T05:49:58.091523+0000","last_became_active":"2023-10-26T05:49:57.103311+0000","last_became_peered":"2023-10-26T05:49:57.103311+0000","last_unstale":"2023-10-26T05:49:58.091523+0000","last_undegraded":"2023-10-26T05:49:58.091523+0000","last_fullsized":"2023-10-26T05:49:58.091523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:02.687883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314177+0000","last_change":"2023-10-26T05:49:57.099713+0000","last_active":"2023-10-26T05:49:58.314177+0000","last_peered":"2023-10-26T05:49:58.314177+0000","last_clean":"2023-10-26T05:49:58.314177+0000","last_became_active":"2023-10-26T05:49:57.099106+0000","last_became_peered":"2023-10-26T05:49:57.099106+0000","last_unstale":"2023-10-26T05:49:58.314177+0000","last_undegraded":"2023-10-26T05:49:58.314177+0000","last_fullsized":"2023-10-26T05:49:58.314177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:57.669526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314371+0000","last_change":"2023-10-26T05:49:57.113364+0000","last_active":"2023-10-26T05:49:58.314371+0000","last_peered":"2023-10-26T05:49:58.314371+0000","last_clean":"2023-10-26T05:49:58.314371+0000","last_became_active":"2023-10-26T05:49:57.112667+0000","last_became_peered":"2023-10-26T05:49:57.112667+0000","last_unstale":"2023-10-26T05:49:58.314371+0000","last_undegraded":"2023-10-26T05:49:58.314371+0000","last_fullsized":"2023-10-26T05:49:58.314371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:56.622294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317646+0000","last_change":"2023-10-26T05:49:57.113240+0000","last_active":"2023-10-26T05:49:58.317646+0000","last_peered":"2023-10-26T05:49:58.317646+0000","last_clean":"2023-10-26T05:49:58.317646+0000","last_became_active":"2023-10-26T05:49:57.111127+0000","last_became_peered":"2023-10-26T05:49:57.111127+0000","last_unstale":"2023-10-26T05:49:58.317646+0000","last_undegraded":"2023-10-26T05:49:58.317646+0000","last_fullsized":"2023-10-26T05:49:58.317646+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:30.974513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316728+0000","last_change":"2023-10-26T05:49:57.121558+0000","last_active":"2023-10-26T05:49:58.316728+0000","last_peered":"2023-10-26T05:49:58.316728+0000","last_clean":"2023-10-26T05:49:58.316728+0000","last_became_active":"2023-10-26T05:49:57.120371+0000","last_became_peered":"2023-10-26T05:49:57.120371+0000","last_unstale":"2023-10-26T05:49:58.316728+0000","last_undegraded":"2023-10-26T05:49:58.316728+0000","last_fullsized":"2023-10-26T05:49:58.316728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:53:57.669526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091848+0000","last_change":"2023-10-26T05:49:57.105511+0000","last_active":"2023-10-26T05:49:58.091848+0000","last_peered":"2023-10-26T05:49:58.091848+0000","last_clean":"2023-10-26T05:49:58.091848+0000","last_became_active":"2023-10-26T05:49:57.104184+0000","last_became_peered":"2023-10-26T05:49:57.104184+0000","last_unstale":"2023-10-26T05:49:58.091848+0000","last_undegraded":"2023-10-26T05:49:58.091848+0000","last_fullsized":"2023-10-26T05:49:58.091848+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:16.970220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.093368+0000","last_change":"2023-10-26T05:49:57.098966+0000","last_active":"2023-10-26T05:49:58.093368+0000","last_peered":"2023-10-26T05:49:58.093368+0000","last_clean":"2023-10-26T05:49:58.093368+0000","last_became_active":"2023-10-26T05:49:57.098323+0000","last_became_peered":"2023-10-26T05:49:57.098323+0000","last_unstale":"2023-10-26T05:49:58.093368+0000","last_undegraded":"2023-10-26T05:49:58.093368+0000","last_fullsized":"2023-10-26T05:49:58.093368+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:29:01.910496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086298+0000","last_change":"2023-10-26T05:49:57.110983+0000","last_active":"2023-10-26T05:49:58.086298+0000","last_peered":"2023-10-26T05:49:58.086298+0000","last_clean":"2023-10-26T05:49:58.086298+0000","last_became_active":"2023-10-26T05:49:57.104680+0000","last_became_peered":"2023-10-26T05:49:57.104680+0000","last_unstale":"2023-10-26T05:49:58.086298+0000","last_undegraded":"2023-10-26T05:49:58.086298+0000","last_fullsized":"2023-10-26T05:49:58.086298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:31:53.093895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716729+0000","last_change":"2023-10-26T05:49:57.126050+0000","last_active":"2023-10-26T05:49:58.716729+0000","last_peered":"2023-10-26T05:49:58.716729+0000","last_clean":"2023-10-26T05:49:58.716729+0000","last_became_active":"2023-10-26T05:49:57.122612+0000","last_became_peered":"2023-10-26T05:49:57.122612+0000","last_unstale":"2023-10-26T05:49:58.716729+0000","last_undegraded":"2023-10-26T05:49:58.716729+0000","last_fullsized":"2023-10-26T05:49:58.716729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:03.709697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090536+0000","last_change":"2023-10-26T05:49:57.128330+0000","last_active":"2023-10-26T05:49:58.090536+0000","last_peered":"2023-10-26T05:49:58.090536+0000","last_clean":"2023-10-26T05:49:58.090536+0000","last_became_active":"2023-10-26T05:49:57.125489+0000","last_became_peered":"2023-10-26T05:49:57.125489+0000","last_unstale":"2023-10-26T05:49:58.090536+0000","last_undegraded":"2023-10-26T05:49:58.090536+0000","last_fullsized":"2023-10-26T05:49:58.090536+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:29:01.910496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225539+0000","last_change":"2023-10-26T05:49:57.107668+0000","last_active":"2023-10-26T05:49:58.225539+0000","last_peered":"2023-10-26T05:49:58.225539+0000","last_clean":"2023-10-26T05:49:58.225539+0000","last_became_active":"2023-10-26T05:49:57.107054+0000","last_became_peered":"2023-10-26T05:49:57.107054+0000","last_unstale":"2023-10-26T05:49:58.225539+0000","last_undegraded":"2023-10-26T05:49:58.225539+0000","last_fullsized":"2023-10-26T05:49:58.225539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:39.383759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716668+0000","last_change":"2023-10-26T05:49:57.108547+0000","last_active":"2023-10-26T05:49:58.716668+0000","last_peered":"2023-10-26T05:49:58.716668+0000","last_clean":"2023-10-26T05:49:58.716668+0000","last_became_active":"2023-10-26T05:49:57.107926+0000","last_became_peered":"2023-10-26T05:49:57.107926+0000","last_unstale":"2023-10-26T05:49:58.716668+0000","last_undegraded":"2023-10-26T05:49:58.716668+0000","last_fullsized":"2023-10-26T05:49:58.716668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:37.339136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092098+0000","last_change":"2023-10-26T05:49:57.121068+0000","last_active":"2023-10-26T05:49:58.092098+0000","last_peered":"2023-10-26T05:49:58.092098+0000","last_clean":"2023-10-26T05:49:58.092098+0000","last_became_active":"2023-10-26T05:49:57.118564+0000","last_became_peered":"2023-10-26T05:49:57.118564+0000","last_unstale":"2023-10-26T05:49:58.092098+0000","last_undegraded":"2023-10-26T05:49:58.092098+0000","last_fullsized":"2023-10-26T05:49:58.092098+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:21.674188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716914+0000","last_change":"2023-10-26T05:49:57.125635+0000","last_active":"2023-10-26T05:49:58.716914+0000","last_peered":"2023-10-26T05:49:58.716914+0000","last_clean":"2023-10-26T05:49:58.716914+0000","last_became_active":"2023-10-26T05:49:57.121338+0000","last_became_peered":"2023-10-26T05:49:57.121338+0000","last_unstale":"2023-10-26T05:49:58.716914+0000","last_undegraded":"2023-10-26T05:49:58.716914+0000","last_fullsized":"2023-10-26T05:49:58.716914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:01:23.419975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090876+0000","last_change":"2023-10-26T05:49:57.128624+0000","last_active":"2023-10-26T05:49:58.090876+0000","last_peered":"2023-10-26T05:49:58.090876+0000","last_clean":"2023-10-26T05:49:58.090876+0000","last_became_active":"2023-10-26T05:49:57.127164+0000","last_became_peered":"2023-10-26T05:49:57.127164+0000","last_unstale":"2023-10-26T05:49:58.090876+0000","last_undegraded":"2023-10-26T05:49:58.090876+0000","last_fullsized":"2023-10-26T05:49:58.090876+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:06.231291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316637+0000","last_change":"2023-10-26T05:49:57.121100+0000","last_active":"2023-10-26T05:49:58.316637+0000","last_peered":"2023-10-26T05:49:58.316637+0000","last_clean":"2023-10-26T05:49:58.316637+0000","last_became_active":"2023-10-26T05:49:57.120303+0000","last_became_peered":"2023-10-26T05:49:57.120303+0000","last_unstale":"2023-10-26T05:49:58.316637+0000","last_undegraded":"2023-10-26T05:49:58.316637+0000","last_fullsized":"2023-10-26T05:49:58.316637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:02.439356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225979+0000","last_change":"2023-10-26T05:49:57.099801+0000","last_active":"2023-10-26T05:49:58.225979+0000","last_peered":"2023-10-26T05:49:58.225979+0000","last_clean":"2023-10-26T05:49:58.225979+0000","last_became_active":"2023-10-26T05:49:57.099153+0000","last_became_peered":"2023-10-26T05:49:57.099153+0000","last_unstale":"2023-10-26T05:49:58.225979+0000","last_undegraded":"2023-10-26T05:49:58.225979+0000","last_fullsized":"2023-10-26T05:49:58.225979+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:50:44.697497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317659+0000","last_change":"2023-10-26T05:49:57.108003+0000","last_active":"2023-10-26T05:49:58.317659+0000","last_peered":"2023-10-26T05:49:58.317659+0000","last_clean":"2023-10-26T05:49:58.317659+0000","last_became_active":"2023-10-26T05:49:57.106379+0000","last_became_peered":"2023-10-26T05:49:57.106379+0000","last_unstale":"2023-10-26T05:49:58.317659+0000","last_undegraded":"2023-10-26T05:49:58.317659+0000","last_fullsized":"2023-10-26T05:49:58.317659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:06.675265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086111+0000","last_change":"2023-10-26T05:49:57.098399+0000","last_active":"2023-10-26T05:49:58.086111+0000","last_peered":"2023-10-26T05:49:58.086111+0000","last_clean":"2023-10-26T05:49:58.086111+0000","last_became_active":"2023-10-26T05:49:57.097723+0000","last_became_peered":"2023-10-26T05:49:57.097723+0000","last_unstale":"2023-10-26T05:49:58.086111+0000","last_undegraded":"2023-10-26T05:49:58.086111+0000","last_fullsized":"2023-10-26T05:49:58.086111+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:18:33.139199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225587+0000","last_change":"2023-10-26T05:49:57.102815+0000","last_active":"2023-10-26T05:49:58.225587+0000","last_peered":"2023-10-26T05:49:58.225587+0000","last_clean":"2023-10-26T05:49:58.225587+0000","last_became_active":"2023-10-26T05:49:57.102169+0000","last_became_peered":"2023-10-26T05:49:57.102169+0000","last_unstale":"2023-10-26T05:49:58.225587+0000","last_undegraded":"2023-10-26T05:49:58.225587+0000","last_fullsized":"2023-10-26T05:49:58.225587+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:01.993526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716407+0000","last_change":"2023-10-26T05:49:57.123223+0000","last_active":"2023-10-26T05:49:58.716407+0000","last_peered":"2023-10-26T05:49:58.716407+0000","last_clean":"2023-10-26T05:49:58.716407+0000","last_became_active":"2023-10-26T05:49:57.122494+0000","last_became_peered":"2023-10-26T05:49:57.122494+0000","last_unstale":"2023-10-26T05:49:58.716407+0000","last_undegraded":"2023-10-26T05:49:58.716407+0000","last_fullsized":"2023-10-26T05:49:58.716407+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:02.613671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090333+0000","last_change":"2023-10-26T05:49:57.112588+0000","last_active":"2023-10-26T05:49:58.090333+0000","last_peered":"2023-10-26T05:49:58.090333+0000","last_clean":"2023-10-26T05:49:58.090333+0000","last_became_active":"2023-10-26T05:49:57.111972+0000","last_became_peered":"2023-10-26T05:49:57.111972+0000","last_unstale":"2023-10-26T05:49:58.090333+0000","last_undegraded":"2023-10-26T05:49:58.090333+0000","last_fullsized":"2023-10-26T05:49:58.090333+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:12.865553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.225417+0000","last_change":"2023-10-26T05:49:57.091899+0000","last_active":"2023-10-26T05:49:58.225417+0000","last_peered":"2023-10-26T05:49:58.225417+0000","last_clean":"2023-10-26T05:49:58.225417+0000","last_became_active":"2023-10-26T05:49:57.091278+0000","last_became_peered":"2023-10-26T05:49:57.091278+0000","last_unstale":"2023-10-26T05:49:58.225417+0000","last_undegraded":"2023-10-26T05:49:58.225417+0000","last_fullsized":"2023-10-26T05:49:58.225417+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:31.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086259+0000","last_change":"2023-10-26T05:49:57.123592+0000","last_active":"2023-10-26T05:49:58.086259+0000","last_peered":"2023-10-26T05:49:58.086259+0000","last_clean":"2023-10-26T05:49:58.086259+0000","last_became_active":"2023-10-26T05:49:57.122977+0000","last_became_peered":"2023-10-26T05:49:57.122977+0000","last_unstale":"2023-10-26T05:49:58.086259+0000","last_undegraded":"2023-10-26T05:49:58.086259+0000","last_fullsized":"2023-10-26T05:49:58.086259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:22:40.506950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086112+0000","last_change":"2023-10-26T05:49:57.108669+0000","last_active":"2023-10-26T05:49:58.086112+0000","last_peered":"2023-10-26T05:49:58.086112+0000","last_clean":"2023-10-26T05:49:58.086112+0000","last_became_active":"2023-10-26T05:49:57.107485+0000","last_became_peered":"2023-10-26T05:49:57.107485+0000","last_unstale":"2023-10-26T05:49:58.086112+0000","last_undegraded":"2023-10-26T05:49:58.086112+0000","last_fullsized":"2023-10-26T05:49:58.086112+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:37:39.035147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"67'4","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086183+0000","last_change":"2023-10-26T05:49:56.268406+0000","last_active":"2023-10-26T05:49:58.086183+0000","last_peered":"2023-10-26T05:49:58.086183+0000","last_clean":"2023-10-26T05:49:58.086183+0000","last_became_active":"2023-10-26T05:49:56.268181+0000","last_became_peered":"2023-10-26T05:49:56.268181+0000","last_unstale":"2023-10-26T05:49:58.086183+0000","last_undegraded":"2023-10-26T05:49:58.086183+0000","last_fullsized":"2023-10-26T05:49:58.086183+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:14.545042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314228+0000","last_change":"2023-10-26T05:49:57.116076+0000","last_active":"2023-10-26T05:49:58.314228+0000","last_peered":"2023-10-26T05:49:58.314228+0000","last_clean":"2023-10-26T05:49:58.314228+0000","last_became_active":"2023-10-26T05:49:57.115255+0000","last_became_peered":"2023-10-26T05:49:57.115255+0000","last_unstale":"2023-10-26T05:49:58.314228+0000","last_undegraded":"2023-10-26T05:49:58.314228+0000","last_fullsized":"2023-10-26T05:49:58.314228+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:55.471662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716992+0000","last_change":"2023-10-26T05:49:56.278925+0000","last_active":"2023-10-26T05:49:58.716992+0000","last_peered":"2023-10-26T05:49:58.716992+0000","last_clean":"2023-10-26T05:49:58.716992+0000","last_became_active":"2023-10-26T05:49:56.278699+0000","last_became_peered":"2023-10-26T05:49:56.278699+0000","last_unstale":"2023-10-26T05:49:58.716992+0000","last_undegraded":"2023-10-26T05:49:58.716992+0000","last_fullsized":"2023-10-26T05:49:58.716992+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:47.637831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086433+0000","last_change":"2023-10-26T05:49:57.111818+0000","last_active":"2023-10-26T05:49:58.086433+0000","last_peered":"2023-10-26T05:49:58.086433+0000","last_clean":"2023-10-26T05:49:58.086433+0000","last_became_active":"2023-10-26T05:49:57.105428+0000","last_became_peered":"2023-10-26T05:49:57.105428+0000","last_unstale":"2023-10-26T05:49:58.086433+0000","last_undegraded":"2023-10-26T05:49:58.086433+0000","last_fullsized":"2023-10-26T05:49:58.086433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:51.862697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086470+0000","last_change":"2023-10-26T05:49:56.284883+0000","last_active":"2023-10-26T05:49:58.086470+0000","last_peered":"2023-10-26T05:49:58.086470+0000","last_clean":"2023-10-26T05:49:58.086470+0000","last_became_active":"2023-10-26T05:49:56.283583+0000","last_became_peered":"2023-10-26T05:49:56.283583+0000","last_unstale":"2023-10-26T05:49:58.086470+0000","last_undegraded":"2023-10-26T05:49:58.086470+0000","last_fullsized":"2023-10-26T05:49:58.086470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:20.669487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086331+0000","last_change":"2023-10-26T05:49:57.105500+0000","last_active":"2023-10-26T05:49:58.086331+0000","last_peered":"2023-10-26T05:49:58.086331+0000","last_clean":"2023-10-26T05:49:58.086331+0000","last_became_active":"2023-10-26T05:49:57.104941+0000","last_became_peered":"2023-10-26T05:49:57.104941+0000","last_unstale":"2023-10-26T05:49:58.086331+0000","last_undegraded":"2023-10-26T05:49:58.086331+0000","last_fullsized":"2023-10-26T05:49:58.086331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:42.142447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318204+0000","last_change":"2023-10-26T05:49:56.297704+0000","last_active":"2023-10-26T05:49:58.318204+0000","last_peered":"2023-10-26T05:49:58.318204+0000","last_clean":"2023-10-26T05:49:58.318204+0000","last_became_active":"2023-10-26T05:49:56.297264+0000","last_became_peered":"2023-10-26T05:49:56.297264+0000","last_unstale":"2023-10-26T05:49:58.318204+0000","last_undegraded":"2023-10-26T05:49:58.318204+0000","last_fullsized":"2023-10-26T05:49:58.318204+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:39:48.744303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716654+0000","last_change":"2023-10-26T05:49:57.112683+0000","last_active":"2023-10-26T05:49:58.716654+0000","last_peered":"2023-10-26T05:49:58.716654+0000","last_clean":"2023-10-26T05:49:58.716654+0000","last_became_active":"2023-10-26T05:49:57.105528+0000","last_became_peered":"2023-10-26T05:49:57.105528+0000","last_unstale":"2023-10-26T05:49:58.716654+0000","last_undegraded":"2023-10-26T05:49:58.716654+0000","last_fullsized":"2023-10-26T05:49:58.716654+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:41:44.409022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092011+0000","last_change":"2023-10-26T05:49:56.292088+0000","last_active":"2023-10-26T05:49:58.092011+0000","last_peered":"2023-10-26T05:49:58.092011+0000","last_clean":"2023-10-26T05:49:58.092011+0000","last_became_active":"2023-10-26T05:49:56.288903+0000","last_became_peered":"2023-10-26T05:49:56.288903+0000","last_unstale":"2023-10-26T05:49:58.092011+0000","last_undegraded":"2023-10-26T05:49:58.092011+0000","last_fullsized":"2023-10-26T05:49:58.092011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:15:44.185767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717083+0000","last_change":"2023-10-26T05:49:57.124648+0000","last_active":"2023-10-26T05:49:58.717083+0000","last_peered":"2023-10-26T05:49:58.717083+0000","last_clean":"2023-10-26T05:49:58.717083+0000","last_became_active":"2023-10-26T05:49:57.119895+0000","last_became_peered":"2023-10-26T05:49:57.119895+0000","last_unstale":"2023-10-26T05:49:58.717083+0000","last_undegraded":"2023-10-26T05:49:58.717083+0000","last_fullsized":"2023-10-26T05:49:58.717083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:18:24.430848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086280+0000","last_change":"2023-10-26T05:49:56.283792+0000","last_active":"2023-10-26T05:49:58.086280+0000","last_peered":"2023-10-26T05:49:58.086280+0000","last_clean":"2023-10-26T05:49:58.086280+0000","last_became_active":"2023-10-26T05:49:56.283304+0000","last_became_peered":"2023-10-26T05:49:56.283304+0000","last_unstale":"2023-10-26T05:49:58.086280+0000","last_undegraded":"2023-10-26T05:49:58.086280+0000","last_fullsized":"2023-10-26T05:49:58.086280+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:11.860851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317090+0000","last_change":"2023-10-26T05:49:57.116455+0000","last_active":"2023-10-26T05:49:58.317090+0000","last_peered":"2023-10-26T05:49:58.317090+0000","last_clean":"2023-10-26T05:49:58.317090+0000","last_became_active":"2023-10-26T05:49:57.115777+0000","last_became_peered":"2023-10-26T05:49:57.115777+0000","last_unstale":"2023-10-26T05:49:58.317090+0000","last_undegraded":"2023-10-26T05:49:58.317090+0000","last_fullsized":"2023-10-26T05:49:58.317090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:18:55.471662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317517+0000","last_change":"2023-10-26T05:49:56.290456+0000","last_active":"2023-10-26T05:49:58.317517+0000","last_peered":"2023-10-26T05:49:58.317517+0000","last_clean":"2023-10-26T05:49:58.317517+0000","last_became_active":"2023-10-26T05:49:56.285667+0000","last_became_peered":"2023-10-26T05:49:56.285667+0000","last_unstale":"2023-10-26T05:49:58.317517+0000","last_undegraded":"2023-10-26T05:49:58.317517+0000","last_fullsized":"2023-10-26T05:49:58.317517+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:35:16.657920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716738+0000","last_change":"2023-10-26T05:49:57.106930+0000","last_active":"2023-10-26T05:49:58.716738+0000","last_peered":"2023-10-26T05:49:58.716738+0000","last_clean":"2023-10-26T05:49:58.716738+0000","last_became_active":"2023-10-26T05:49:57.105668+0000","last_became_peered":"2023-10-26T05:49:57.105668+0000","last_unstale":"2023-10-26T05:49:58.716738+0000","last_undegraded":"2023-10-26T05:49:58.716738+0000","last_fullsized":"2023-10-26T05:49:58.716738+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:06.803512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.093587+0000","last_change":"2023-10-26T05:49:56.284243+0000","last_active":"2023-10-26T05:49:58.093587+0000","last_peered":"2023-10-26T05:49:58.093587+0000","last_clean":"2023-10-26T05:49:58.093587+0000","last_became_active":"2023-10-26T05:49:56.283521+0000","last_became_peered":"2023-10-26T05:49:56.283521+0000","last_unstale":"2023-10-26T05:49:58.093587+0000","last_undegraded":"2023-10-26T05:49:58.093587+0000","last_fullsized":"2023-10-26T05:49:58.093587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:17.053962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091775+0000","last_change":"2023-10-26T05:49:57.097532+0000","last_active":"2023-10-26T05:49:58.091775+0000","last_peered":"2023-10-26T05:49:58.091775+0000","last_clean":"2023-10-26T05:49:58.091775+0000","last_became_active":"2023-10-26T05:49:57.095879+0000","last_became_peered":"2023-10-26T05:49:57.095879+0000","last_unstale":"2023-10-26T05:49:58.091775+0000","last_undegraded":"2023-10-26T05:49:58.091775+0000","last_fullsized":"2023-10-26T05:49:58.091775+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:12.865553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.093002+0000","last_change":"2023-10-26T05:49:56.299951+0000","last_active":"2023-10-26T05:49:58.093002+0000","last_peered":"2023-10-26T05:49:58.093002+0000","last_clean":"2023-10-26T05:49:58.093002+0000","last_became_active":"2023-10-26T05:49:56.298979+0000","last_became_peered":"2023-10-26T05:49:56.298979+0000","last_unstale":"2023-10-26T05:49:58.093002+0000","last_undegraded":"2023-10-26T05:49:58.093002+0000","last_fullsized":"2023-10-26T05:49:58.093002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:05.670564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092371+0000","last_change":"2023-10-26T05:49:57.120704+0000","last_active":"2023-10-26T05:49:58.092371+0000","last_peered":"2023-10-26T05:49:58.092371+0000","last_clean":"2023-10-26T05:49:58.092371+0000","last_became_active":"2023-10-26T05:49:57.117741+0000","last_became_peered":"2023-10-26T05:49:57.117741+0000","last_unstale":"2023-10-26T05:49:58.092371+0000","last_undegraded":"2023-10-26T05:49:58.092371+0000","last_fullsized":"2023-10-26T05:49:58.092371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:43.307761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717672+0000","last_change":"2023-10-26T05:49:56.283861+0000","last_active":"2023-10-26T05:49:58.717672+0000","last_peered":"2023-10-26T05:49:58.717672+0000","last_clean":"2023-10-26T05:49:58.717672+0000","last_became_active":"2023-10-26T05:49:56.283224+0000","last_became_peered":"2023-10-26T05:49:56.283224+0000","last_unstale":"2023-10-26T05:49:58.717672+0000","last_undegraded":"2023-10-26T05:49:58.717672+0000","last_fullsized":"2023-10-26T05:49:58.717672+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:48.291789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314102+0000","last_change":"2023-10-26T05:49:57.091946+0000","last_active":"2023-10-26T05:49:58.314102+0000","last_peered":"2023-10-26T05:49:58.314102+0000","last_clean":"2023-10-26T05:49:58.314102+0000","last_became_active":"2023-10-26T05:49:57.091288+0000","last_became_peered":"2023-10-26T05:49:57.091288+0000","last_unstale":"2023-10-26T05:49:58.314102+0000","last_undegraded":"2023-10-26T05:49:58.314102+0000","last_fullsized":"2023-10-26T05:49:58.314102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:23:02.439356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"67'3","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091437+0000","last_change":"2023-10-26T05:49:56.294929+0000","last_active":"2023-10-26T05:49:58.091437+0000","last_peered":"2023-10-26T05:49:58.091437+0000","last_clean":"2023-10-26T05:49:58.091437+0000","last_became_active":"2023-10-26T05:49:56.292173+0000","last_became_peered":"2023-10-26T05:49:56.292173+0000","last_unstale":"2023-10-26T05:49:58.091437+0000","last_undegraded":"2023-10-26T05:49:58.091437+0000","last_fullsized":"2023-10-26T05:49:58.091437+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:02.325213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091200+0000","last_change":"2023-10-26T05:49:57.117581+0000","last_active":"2023-10-26T05:49:58.091200+0000","last_peered":"2023-10-26T05:49:58.091200+0000","last_clean":"2023-10-26T05:49:58.091200+0000","last_became_active":"2023-10-26T05:49:57.116797+0000","last_became_peered":"2023-10-26T05:49:57.116797+0000","last_unstale":"2023-10-26T05:49:58.091200+0000","last_undegraded":"2023-10-26T05:49:58.091200+0000","last_fullsized":"2023-10-26T05:49:58.091200+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:02:01.518600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"67'6","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092076+0000","last_change":"2023-10-26T05:49:56.288391+0000","last_active":"2023-10-26T05:49:58.092076+0000","last_peered":"2023-10-26T05:49:58.092076+0000","last_clean":"2023-10-26T05:49:58.092076+0000","last_became_active":"2023-10-26T05:49:56.287777+0000","last_became_peered":"2023-10-26T05:49:56.287777+0000","last_unstale":"2023-10-26T05:49:58.092076+0000","last_undegraded":"2023-10-26T05:49:58.092076+0000","last_fullsized":"2023-10-26T05:49:58.092076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:29.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317991+0000","last_change":"2023-10-26T05:49:57.108754+0000","last_active":"2023-10-26T05:49:58.317991+0000","last_peered":"2023-10-26T05:49:58.317991+0000","last_clean":"2023-10-26T05:49:58.317991+0000","last_became_active":"2023-10-26T05:49:57.107345+0000","last_became_peered":"2023-10-26T05:49:57.107345+0000","last_unstale":"2023-10-26T05:49:58.317991+0000","last_undegraded":"2023-10-26T05:49:58.317991+0000","last_fullsized":"2023-10-26T05:49:58.317991+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:00:02.687883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717074+0000","last_change":"2023-10-26T05:49:56.286965+0000","last_active":"2023-10-26T05:49:58.717074+0000","last_peered":"2023-10-26T05:49:58.717074+0000","last_clean":"2023-10-26T05:49:58.717074+0000","last_became_active":"2023-10-26T05:49:56.286828+0000","last_became_peered":"2023-10-26T05:49:56.286828+0000","last_unstale":"2023-10-26T05:49:58.717074+0000","last_undegraded":"2023-10-26T05:49:58.717074+0000","last_fullsized":"2023-10-26T05:49:58.717074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:22:39.502248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090865+0000","last_change":"2023-10-26T05:49:57.125683+0000","last_active":"2023-10-26T05:49:58.090865+0000","last_peered":"2023-10-26T05:49:58.090865+0000","last_clean":"2023-10-26T05:49:58.090865+0000","last_became_active":"2023-10-26T05:49:57.124428+0000","last_became_peered":"2023-10-26T05:49:57.124428+0000","last_unstale":"2023-10-26T05:49:58.090865+0000","last_undegraded":"2023-10-26T05:49:58.090865+0000","last_fullsized":"2023-10-26T05:49:58.090865+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:13.861096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316975+0000","last_change":"2023-10-26T05:49:56.299611+0000","last_active":"2023-10-26T05:49:58.316975+0000","last_peered":"2023-10-26T05:49:58.316975+0000","last_clean":"2023-10-26T05:49:58.316975+0000","last_became_active":"2023-10-26T05:49:56.298909+0000","last_became_peered":"2023-10-26T05:49:56.298909+0000","last_unstale":"2023-10-26T05:49:58.316975+0000","last_undegraded":"2023-10-26T05:49:58.316975+0000","last_fullsized":"2023-10-26T05:49:58.316975+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:58.589719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091378+0000","last_change":"2023-10-26T05:49:57.122737+0000","last_active":"2023-10-26T05:49:58.091378+0000","last_peered":"2023-10-26T05:49:58.091378+0000","last_clean":"2023-10-26T05:49:58.091378+0000","last_became_active":"2023-10-26T05:49:57.122071+0000","last_became_peered":"2023-10-26T05:49:57.122071+0000","last_unstale":"2023-10-26T05:49:58.091378+0000","last_undegraded":"2023-10-26T05:49:58.091378+0000","last_fullsized":"2023-10-26T05:49:58.091378+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:15.549743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318194+0000","last_change":"2023-10-26T05:49:56.292821+0000","last_active":"2023-10-26T05:49:58.318194+0000","last_peered":"2023-10-26T05:49:58.318194+0000","last_clean":"2023-10-26T05:49:58.318194+0000","last_became_active":"2023-10-26T05:49:56.291455+0000","last_became_peered":"2023-10-26T05:49:56.291455+0000","last_unstale":"2023-10-26T05:49:58.318194+0000","last_undegraded":"2023-10-26T05:49:58.318194+0000","last_fullsized":"2023-10-26T05:49:58.318194+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:03:15.199063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.090434+0000","last_change":"2023-10-26T05:49:57.127709+0000","last_active":"2023-10-26T05:49:58.090434+0000","last_peered":"2023-10-26T05:49:58.090434+0000","last_clean":"2023-10-26T05:49:58.090434+0000","last_became_active":"2023-10-26T05:49:57.127101+0000","last_became_peered":"2023-10-26T05:49:57.127101+0000","last_unstale":"2023-10-26T05:49:58.090434+0000","last_undegraded":"2023-10-26T05:49:58.090434+0000","last_fullsized":"2023-10-26T05:49:58.090434+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:16.970220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"67'8","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091662+0000","last_change":"2023-10-26T05:49:56.295402+0000","last_active":"2023-10-26T05:49:58.091662+0000","last_peered":"2023-10-26T05:49:58.091662+0000","last_clean":"2023-10-26T05:49:58.091662+0000","last_became_active":"2023-10-26T05:49:56.294000+0000","last_became_peered":"2023-10-26T05:49:56.294000+0000","last_unstale":"2023-10-26T05:49:58.091662+0000","last_undegraded":"2023-10-26T05:49:58.091662+0000","last_fullsized":"2023-10-26T05:49:58.091662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:50.215466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.091187+0000","last_change":"2023-10-26T05:49:57.125220+0000","last_active":"2023-10-26T05:49:58.091187+0000","last_peered":"2023-10-26T05:49:58.091187+0000","last_clean":"2023-10-26T05:49:58.091187+0000","last_became_active":"2023-10-26T05:49:57.120408+0000","last_became_peered":"2023-10-26T05:49:57.120408+0000","last_unstale":"2023-10-26T05:49:58.091187+0000","last_undegraded":"2023-10-26T05:49:58.091187+0000","last_fullsized":"2023-10-26T05:49:58.091187+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:02.283721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717244+0000","last_change":"2023-10-26T05:49:56.281450+0000","last_active":"2023-10-26T05:49:58.717244+0000","last_peered":"2023-10-26T05:49:58.717244+0000","last_clean":"2023-10-26T05:49:58.717244+0000","last_became_active":"2023-10-26T05:49:56.281258+0000","last_became_peered":"2023-10-26T05:49:56.281258+0000","last_unstale":"2023-10-26T05:49:58.717244+0000","last_undegraded":"2023-10-26T05:49:58.717244+0000","last_fullsized":"2023-10-26T05:49:58.717244+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:31:52.089193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317040+0000","last_change":"2023-10-26T05:49:57.127099+0000","last_active":"2023-10-26T05:49:58.317040+0000","last_peered":"2023-10-26T05:49:58.317040+0000","last_clean":"2023-10-26T05:49:58.317040+0000","last_became_active":"2023-10-26T05:49:57.118602+0000","last_became_peered":"2023-10-26T05:49:57.118602+0000","last_unstale":"2023-10-26T05:49:58.317040+0000","last_undegraded":"2023-10-26T05:49:58.317040+0000","last_fullsized":"2023-10-26T05:49:58.317040+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:03.329914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086512+0000","last_change":"2023-10-26T05:49:56.284544+0000","last_active":"2023-10-26T05:49:58.086512+0000","last_peered":"2023-10-26T05:49:58.086512+0000","last_clean":"2023-10-26T05:49:58.086512+0000","last_became_active":"2023-10-26T05:49:56.282775+0000","last_became_peered":"2023-10-26T05:49:56.282775+0000","last_unstale":"2023-10-26T05:49:58.086512+0000","last_undegraded":"2023-10-26T05:49:58.086512+0000","last_fullsized":"2023-10-26T05:49:58.086512+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:34:42.303059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717248+0000","last_change":"2023-10-26T05:49:57.122032+0000","last_active":"2023-10-26T05:49:58.717248+0000","last_peered":"2023-10-26T05:49:58.717248+0000","last_clean":"2023-10-26T05:49:58.717248+0000","last_became_active":"2023-10-26T05:49:57.117067+0000","last_became_peered":"2023-10-26T05:49:57.117067+0000","last_unstale":"2023-10-26T05:49:58.717248+0000","last_undegraded":"2023-10-26T05:49:58.717248+0000","last_fullsized":"2023-10-26T05:49:58.717248+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:53:41.032022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"67'1","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318352+0000","last_change":"2023-10-26T05:49:56.296801+0000","last_active":"2023-10-26T05:49:58.318352+0000","last_peered":"2023-10-26T05:49:58.318352+0000","last_clean":"2023-10-26T05:49:58.318352+0000","last_became_active":"2023-10-26T05:49:56.296122+0000","last_became_peered":"2023-10-26T05:49:56.296122+0000","last_unstale":"2023-10-26T05:49:58.318352+0000","last_undegraded":"2023-10-26T05:49:58.318352+0000","last_fullsized":"2023-10-26T05:49:58.318352+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:16:43.154863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092545+0000","last_change":"2023-10-26T05:49:57.122024+0000","last_active":"2023-10-26T05:49:58.092545+0000","last_peered":"2023-10-26T05:49:58.092545+0000","last_clean":"2023-10-26T05:49:58.092545+0000","last_became_active":"2023-10-26T05:49:57.119774+0000","last_became_peered":"2023-10-26T05:49:57.119774+0000","last_unstale":"2023-10-26T05:49:58.092545+0000","last_undegraded":"2023-10-26T05:49:58.092545+0000","last_fullsized":"2023-10-26T05:49:58.092545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:29.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086341+0000","last_change":"2023-10-26T05:49:56.284036+0000","last_active":"2023-10-26T05:49:58.086341+0000","last_peered":"2023-10-26T05:49:58.086341+0000","last_clean":"2023-10-26T05:49:58.086341+0000","last_became_active":"2023-10-26T05:49:56.283798+0000","last_became_peered":"2023-10-26T05:49:56.283798+0000","last_unstale":"2023-10-26T05:49:58.086341+0000","last_undegraded":"2023-10-26T05:49:58.086341+0000","last_fullsized":"2023-10-26T05:49:58.086341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:05.226590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317393+0000","last_change":"2023-10-26T05:49:57.125312+0000","last_active":"2023-10-26T05:49:58.317393+0000","last_peered":"2023-10-26T05:49:58.317393+0000","last_clean":"2023-10-26T05:49:58.317393+0000","last_became_active":"2023-10-26T05:49:57.117770+0000","last_became_peered":"2023-10-26T05:49:57.117770+0000","last_unstale":"2023-10-26T05:49:58.317393+0000","last_undegraded":"2023-10-26T05:49:58.317393+0000","last_fullsized":"2023-10-26T05:49:58.317393+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:13:51.220167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717013+0000","last_change":"2023-10-26T05:49:56.287347+0000","last_active":"2023-10-26T05:49:58.717013+0000","last_peered":"2023-10-26T05:49:58.717013+0000","last_clean":"2023-10-26T05:49:58.717013+0000","last_became_active":"2023-10-26T05:49:56.286313+0000","last_became_peered":"2023-10-26T05:49:56.286313+0000","last_unstale":"2023-10-26T05:49:58.717013+0000","last_undegraded":"2023-10-26T05:49:58.717013+0000","last_fullsized":"2023-10-26T05:49:58.717013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:50.857996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086091+0000","last_change":"2023-10-26T05:49:57.101807+0000","last_active":"2023-10-26T05:49:58.086091+0000","last_peered":"2023-10-26T05:49:58.086091+0000","last_clean":"2023-10-26T05:49:58.086091+0000","last_became_active":"2023-10-26T05:49:57.101089+0000","last_became_peered":"2023-10-26T05:49:57.101089+0000","last_unstale":"2023-10-26T05:49:58.086091+0000","last_undegraded":"2023-10-26T05:49:58.086091+0000","last_fullsized":"2023-10-26T05:49:58.086091+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:51:29.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314446+0000","last_change":"2023-10-26T05:49:56.279916+0000","last_active":"2023-10-26T05:49:58.314446+0000","last_peered":"2023-10-26T05:49:58.314446+0000","last_clean":"2023-10-26T05:49:58.314446+0000","last_became_active":"2023-10-26T05:49:56.279676+0000","last_became_peered":"2023-10-26T05:49:56.279676+0000","last_unstale":"2023-10-26T05:49:58.314446+0000","last_undegraded":"2023-10-26T05:49:58.314446+0000","last_fullsized":"2023-10-26T05:49:58.314446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.573964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086424+0000","last_change":"2023-10-26T05:49:57.111387+0000","last_active":"2023-10-26T05:49:58.086424+0000","last_peered":"2023-10-26T05:49:58.086424+0000","last_clean":"2023-10-26T05:49:58.086424+0000","last_became_active":"2023-10-26T05:49:57.110755+0000","last_became_peered":"2023-10-26T05:49:57.110755+0000","last_unstale":"2023-10-26T05:49:58.086424+0000","last_undegraded":"2023-10-26T05:49:58.086424+0000","last_fullsized":"2023-10-26T05:49:58.086424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:49:28.417446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.318537+0000","last_change":"2023-10-26T05:49:56.287398+0000","last_active":"2023-10-26T05:49:58.318537+0000","last_peered":"2023-10-26T05:49:58.318537+0000","last_clean":"2023-10-26T05:49:58.318537+0000","last_became_active":"2023-10-26T05:49:56.285793+0000","last_became_peered":"2023-10-26T05:49:56.285793+0000","last_unstale":"2023-10-26T05:49:58.318537+0000","last_undegraded":"2023-10-26T05:49:58.318537+0000","last_fullsized":"2023-10-26T05:49:58.318537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:19:40.984838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316743+0000","last_change":"2023-10-26T05:49:57.105773+0000","last_active":"2023-10-26T05:49:58.316743+0000","last_peered":"2023-10-26T05:49:58.316743+0000","last_clean":"2023-10-26T05:49:58.316743+0000","last_became_active":"2023-10-26T05:49:57.104838+0000","last_became_peered":"2023-10-26T05:49:57.104838+0000","last_unstale":"2023-10-26T05:49:58.316743+0000","last_undegraded":"2023-10-26T05:49:58.316743+0000","last_fullsized":"2023-10-26T05:49:58.316743+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:15:45.190469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"67'2","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314141+0000","last_change":"2023-10-26T05:49:56.284411+0000","last_active":"2023-10-26T05:49:58.314141+0000","last_peered":"2023-10-26T05:49:58.314141+0000","last_clean":"2023-10-26T05:49:58.314141+0000","last_became_active":"2023-10-26T05:49:56.284223+0000","last_became_peered":"2023-10-26T05:49:56.284223+0000","last_unstale":"2023-10-26T05:49:58.314141+0000","last_undegraded":"2023-10-26T05:49:58.314141+0000","last_fullsized":"2023-10-26T05:49:58.314141+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:37.644982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317988+0000","last_change":"2023-10-26T05:49:57.111877+0000","last_active":"2023-10-26T05:49:58.317988+0000","last_peered":"2023-10-26T05:49:58.317988+0000","last_clean":"2023-10-26T05:49:58.317988+0000","last_became_active":"2023-10-26T05:49:57.110262+0000","last_became_peered":"2023-10-26T05:49:57.110262+0000","last_unstale":"2023-10-26T05:49:58.317988+0000","last_undegraded":"2023-10-26T05:49:58.317988+0000","last_fullsized":"2023-10-26T05:49:58.317988+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:15.549743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086367+0000","last_change":"2023-10-26T05:49:56.283662+0000","last_active":"2023-10-26T05:49:58.086367+0000","last_peered":"2023-10-26T05:49:58.086367+0000","last_clean":"2023-10-26T05:49:58.086367+0000","last_became_active":"2023-10-26T05:49:56.283003+0000","last_became_peered":"2023-10-26T05:49:56.283003+0000","last_unstale":"2023-10-26T05:49:58.086367+0000","last_undegraded":"2023-10-26T05:49:58.086367+0000","last_fullsized":"2023-10-26T05:49:58.086367+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:12.856394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314259+0000","last_change":"2023-10-26T05:49:57.097257+0000","last_active":"2023-10-26T05:49:58.314259+0000","last_peered":"2023-10-26T05:49:58.314259+0000","last_clean":"2023-10-26T05:49:58.314259+0000","last_became_active":"2023-10-26T05:49:57.091200+0000","last_became_peered":"2023-10-26T05:49:57.091200+0000","last_unstale":"2023-10-26T05:49:58.314259+0000","last_undegraded":"2023-10-26T05:49:58.314259+0000","last_fullsized":"2023-10-26T05:49:58.314259+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:11:57.244632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.316856+0000","last_change":"2023-10-26T05:49:56.289528+0000","last_active":"2023-10-26T05:49:58.316856+0000","last_peered":"2023-10-26T05:49:58.316856+0000","last_clean":"2023-10-26T05:49:58.316856+0000","last_became_active":"2023-10-26T05:49:56.288670+0000","last_became_peered":"2023-10-26T05:49:56.288670+0000","last_unstale":"2023-10-26T05:49:58.316856+0000","last_undegraded":"2023-10-26T05:49:58.316856+0000","last_fullsized":"2023-10-26T05:49:58.316856+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:19.573964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.086251+0000","last_change":"2023-10-26T05:49:57.121953+0000","last_active":"2023-10-26T05:49:58.086251+0000","last_peered":"2023-10-26T05:49:58.086251+0000","last_clean":"2023-10-26T05:49:58.086251+0000","last_became_active":"2023-10-26T05:49:57.121345+0000","last_became_peered":"2023-10-26T05:49:57.121345+0000","last_unstale":"2023-10-26T05:49:58.086251+0000","last_undegraded":"2023-10-26T05:49:58.086251+0000","last_fullsized":"2023-10-26T05:49:58.086251+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:58:48.642532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317562+0000","last_change":"2023-10-26T05:49:56.281089+0000","last_active":"2023-10-26T05:49:58.317562+0000","last_peered":"2023-10-26T05:49:58.317562+0000","last_clean":"2023-10-26T05:49:58.317562+0000","last_became_active":"2023-10-26T05:49:56.280344+0000","last_became_peered":"2023-10-26T05:49:56.280344+0000","last_unstale":"2023-10-26T05:49:58.317562+0000","last_undegraded":"2023-10-26T05:49:58.317562+0000","last_fullsized":"2023-10-26T05:49:58.317562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:39:00.295875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717499+0000","last_change":"2023-10-26T05:49:57.125175+0000","last_active":"2023-10-26T05:49:58.717499+0000","last_peered":"2023-10-26T05:49:58.717499+0000","last_clean":"2023-10-26T05:49:58.717499+0000","last_became_active":"2023-10-26T05:49:57.120634+0000","last_became_peered":"2023-10-26T05:49:57.120634+0000","last_unstale":"2023-10-26T05:49:58.717499+0000","last_undegraded":"2023-10-26T05:49:58.717499+0000","last_fullsized":"2023-10-26T05:49:58.717499+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:22:34.413050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314433+0000","last_change":"2023-10-26T05:49:56.280398+0000","last_active":"2023-10-26T05:49:58.314433+0000","last_peered":"2023-10-26T05:49:58.314433+0000","last_clean":"2023-10-26T05:49:58.314433+0000","last_became_active":"2023-10-26T05:49:56.280161+0000","last_became_peered":"2023-10-26T05:49:56.280161+0000","last_unstale":"2023-10-26T05:49:58.314433+0000","last_undegraded":"2023-10-26T05:49:58.314433+0000","last_fullsized":"2023-10-26T05:49:58.314433+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:41:22.376152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092708+0000","last_change":"2023-10-26T05:49:57.124273+0000","last_active":"2023-10-26T05:49:58.092708+0000","last_peered":"2023-10-26T05:49:58.092708+0000","last_clean":"2023-10-26T05:49:58.092708+0000","last_became_active":"2023-10-26T05:49:57.123585+0000","last_became_peered":"2023-10-26T05:49:57.123585+0000","last_unstale":"2023-10-26T05:49:58.092708+0000","last_undegraded":"2023-10-26T05:49:58.092708+0000","last_fullsized":"2023-10-26T05:49:58.092708+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:09:43.591005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.092224+0000","last_change":"2023-10-26T05:49:56.292615+0000","last_active":"2023-10-26T05:49:58.092224+0000","last_peered":"2023-10-26T05:49:58.092224+0000","last_clean":"2023-10-26T05:49:58.092224+0000","last_became_active":"2023-10-26T05:49:56.291283+0000","last_became_peered":"2023-10-26T05:49:56.291283+0000","last_unstale":"2023-10-26T05:49:58.092224+0000","last_undegraded":"2023-10-26T05:49:58.092224+0000","last_fullsized":"2023-10-26T05:49:58.092224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:02.325213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314321+0000","last_change":"2023-10-26T05:49:57.098875+0000","last_active":"2023-10-26T05:49:58.314321+0000","last_peered":"2023-10-26T05:49:58.314321+0000","last_clean":"2023-10-26T05:49:58.314321+0000","last_became_active":"2023-10-26T05:49:57.098279+0000","last_became_peered":"2023-10-26T05:49:57.098279+0000","last_unstale":"2023-10-26T05:49:58.314321+0000","last_undegraded":"2023-10-26T05:49:58.314321+0000","last_fullsized":"2023-10-26T05:49:58.314321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:03:45.510140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.314342+0000","last_change":"2023-10-26T05:49:56.280279+0000","last_active":"2023-10-26T05:49:58.314342+0000","last_peered":"2023-10-26T05:49:58.314342+0000","last_clean":"2023-10-26T05:49:58.314342+0000","last_became_active":"2023-10-26T05:49:56.279945+0000","last_became_peered":"2023-10-26T05:49:56.279945+0000","last_unstale":"2023-10-26T05:49:58.314342+0000","last_undegraded":"2023-10-26T05:49:58.314342+0000","last_fullsized":"2023-10-26T05:49:58.314342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:58.589719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.716875+0000","last_change":"2023-10-26T05:49:57.116325+0000","last_active":"2023-10-26T05:49:58.716875+0000","last_peered":"2023-10-26T05:49:58.716875+0000","last_clean":"2023-10-26T05:49:58.716875+0000","last_became_active":"2023-10-26T05:49:57.111986+0000","last_became_peered":"2023-10-26T05:49:57.111986+0000","last_unstale":"2023-10-26T05:49:58.716875+0000","last_undegraded":"2023-10-26T05:49:58.716875+0000","last_fullsized":"2023-10-26T05:49:58.716875+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:43.777929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717454+0000","last_change":"2023-10-26T05:49:56.288356+0000","last_active":"2023-10-26T05:49:58.717454+0000","last_peered":"2023-10-26T05:49:58.717454+0000","last_clean":"2023-10-26T05:49:58.717454+0000","last_became_active":"2023-10-26T05:49:56.288093+0000","last_became_peered":"2023-10-26T05:49:56.288093+0000","last_unstale":"2023-10-26T05:49:58.717454+0000","last_undegraded":"2023-10-26T05:49:58.717454+0000","last_fullsized":"2023-10-26T05:49:58.717454+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:41.137745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.717220+0000","last_change":"2023-10-26T05:49:57.114374+0000","last_active":"2023-10-26T05:49:58.717220+0000","last_peered":"2023-10-26T05:49:58.717220+0000","last_clean":"2023-10-26T05:49:58.717220+0000","last_became_active":"2023-10-26T05:49:57.113762+0000","last_became_peered":"2023-10-26T05:49:57.113762+0000","last_unstale":"2023-10-26T05:49:58.717220+0000","last_undegraded":"2023-10-26T05:49:58.717220+0000","last_fullsized":"2023-10-26T05:49:58.717220+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:56.249069+0000","last_clean_scrub_stamp":"2023-10-26T05:49:56.249069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:45:49.439355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-10-26T05:49:58.317309+0000","last_change":"2023-10-26T05:49:56.282808+0000","last_active":"2023-10-26T05:49:58.317309+0000","last_peered":"2023-10-26T05:49:58.317309+0000","last_clean":"2023-10-26T05:49:58.317309+0000","last_became_active":"2023-10-26T05:49:56.282196+0000","last_became_peered":"2023-10-26T05:49:56.282196+0000","last_unstale":"2023-10-26T05:49:58.317309+0000","last_undegraded":"2023-10-26T05:49:58.317309+0000","last_fullsized":"2023-10-26T05:49:58.317309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:49:55.244368+0000","last_clean_scrub_stamp":"2023-10-26T05:49:55.244368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:26.957737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":209,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-10-26T05:49:57.105151+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:49:57.105151+0000","last_peered":"2023-10-26T05:49:57.105151+0000","last_clean":"2023-10-26T05:49:57.105151+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:49:57.105151+0000","last_undegraded":"2023-10-26T05:49:57.105151+0000","last_fullsized":"2023-10-26T05:49:57.105151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:31:55.971230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":920,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194391,"statfs":{"total":200000000,"available":199056957,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.216,"5min":1.347,"15min":1.347},"last":1.014},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82899999999999996,"15min":0.82999999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.262,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":1.0369999999999999,"15min":1.0580000000000001},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.349,"5min":1.391,"15min":1.391},"last":0.877},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95999999999999996,"15min":0.96999999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.165,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.92600000000000005,"15min":0.93899999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.123,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.093}]},{"osd":3,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88100000000000001,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1910000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.96499999999999997,"15min":0.97099999999999997},"min":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.26,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.232,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.75},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.93799999999999994,"15min":0.93500000000000005},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.131,"15min":1.1459999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.292,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.149},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.093,"15min":1.105},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3200000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.93100000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1200000000000001,"15min":1.125},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3979999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.044,"15min":1.0469999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5469999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.94999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647485,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.81599999999999995,"15min":0.80500000000000005},"min":{"1min":0.61499999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1879999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.85299999999999998,"15min":0.85499999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2090000000000001,"5min":1.3,"15min":1.3},"last":0.79400000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.51100000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1779999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.84399999999999997,"15min":0.82399999999999995},"min":{"1min":0.58799999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.79700000000000004,"15min":0.78100000000000003},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.141},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.83899999999999997,"15min":0.82499999999999996},"min":{"1min":0.68200000000000005,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.85299999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.873,"15min":0.86299999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.04},{"interface":"front","average":{"1min":1.0529999999999999,"5min":0.95499999999999996,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.96099999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.96999999999999997,"15min":0.96799999999999997},"min":{"1min":0.67200000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2609999999999999,"5min":1.325,"15min":1.325},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.98799999999999999,"15min":0.97599999999999998},"min":{"1min":0.75600000000000001,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.73099999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.3220000000000001,"5min":1.554,"15min":1.554},"last":1.115},{"interface":"front","average":{"1min":1.008,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.74199999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1200000000000001,"15min":1.1060000000000001},"min":{"1min":0.92200000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.133,"15min":1.1279999999999999},"min":{"1min":0.96799999999999997,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.333,"5min":1.464,"15min":1.464},"last":1.008}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194967,"statfs":{"total":200000000,"available":199646833,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.95099999999999996,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.107},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.95499999999999996,"15min":0.94999999999999996},"min":{"1min":0.64400000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.90300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89200000000000002,"15min":0.90600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1100000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.081},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1779999999999999,"5min":1.329,"15min":1.329},"last":0.85299999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.92500000000000004,"15min":0.93700000000000006},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4219999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":0.93500000000000005},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.127,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.002}]},{"osd":3,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.92000000000000004,"15min":0.92400000000000004},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2270000000000001,"5min":1.548,"15min":1.548},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2569999999999999,"5min":1.587,"15min":1.587},"last":1.147}]},{"osd":4,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.0109999999999999,"15min":1.024},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2549999999999999,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.995,"5min":1.0960000000000001,"15min":1.113},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.458,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.355}]},{"osd":6,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":1.0249999999999999,"15min":1.0489999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3129999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0329999999999999,"15min":1.042},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.1859999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.0860000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.163,"15min":1.1819999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.5029999999999999,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2110000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":345,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194966,"statfs":{"total":200000000,"available":199645873,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84199999999999997,"15min":0.83999999999999997},"min":{"1min":0.52100000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.873,"15min":0.88},"min":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1299999999999999,"5min":1.502,"15min":1.502},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.80000000000000004,"15min":0.79600000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.206,"5min":1.375,"15min":1.375},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.88300000000000001,"15min":0.878},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.79300000000000004},"min":{"1min":0.52700000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0740000000000001,"5min":1.29,"15min":1.29},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.55500000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93500000000000005,"15min":0.94499999999999995},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.341,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.998,"5min":0.94199999999999995,"15min":0.93200000000000005},"min":{"1min":0.70199999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.88900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.64900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.296},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.002,"15min":1},"min":{"1min":0.64900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.97699999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.99099999999999999,"15min":0.99299999999999999},"min":{"1min":0.503,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.012},{"interface":"front","average":{"1min":1.004,"5min":1.032,"15min":1.0369999999999999},"min":{"1min":0.67900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2270000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.379}]},{"osd":7,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.03,"15min":1.03},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.069},{"interface":"front","average":{"1min":1.087,"5min":1.1080000000000001,"15min":1.1120000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.526,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":1.2270000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.27400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.27900000000000003,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.095,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.308}]},{"osd":1,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.63500000000000001,"15min":0.65300000000000002},"min":{"1min":0.315,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94199999999999995,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73599999999999999,"15min":0.74099999999999999},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.57399999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1040000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.90700000000000003,"15min":0.92000000000000004},"min":{"1min":0.59299999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2609999999999999,"5min":1.3,"15min":1.3},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1259999999999999,"5min":1.478,"15min":1.478},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.89300000000000002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":1.01},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2969999999999999,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.006,"15min":1.012},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.214,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:11 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.61399999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0349999999999999,"15min":1.0449999999999999},"min":{"1min":0.65200000000000002,"5min":0.628,"15min":0.628},"max":{"1min":1.206,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.89700000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.35299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80300000000000005,"5min":0.87,"15min":0.87},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.72799999999999998,"15min":0.70599999999999996},"min":{"1min":0.39200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":2.8809999999999998,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.502},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56299999999999994,"15min":0.55700000000000005},"min":{"1min":0.33900000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.88800000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.623}]},{"osd":4,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.81399999999999995,"15min":0.79500000000000004},"min":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.05,"5min":0.85399999999999998,"15min":0.82099999999999995},"min":{"1min":0.61899999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":5.9550000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.91600000000000004,"15min":0.88600000000000001},"min":{"1min":0.65900000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.8319999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.875,"15min":0.871},"min":{"1min":0.58199999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.242,"5min":1.4379999999999999,"15min":1.4379999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.304,"5min":0.98899999999999999,"15min":0.93700000000000006},"min":{"1min":0.56599999999999995,"5min":0.5,"15min":0.5},"max":{"1min":6.7729999999999997,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:49:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.878,"15min":0.876},"min":{"1min":0.67800000000000005,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1759999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.764,"5min":1.1040000000000001,"15min":0.99399999999999999},"min":{"1min":0.56799999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":15.352,"5min":15.352,"15min":15.352},"last":0.94399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":344,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194968,"statfs":{"total":200000000,"available":199647395,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.34000000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.875,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.90300000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63800000000000001,"15min":0.64200000000000002},"min":{"1min":0.372,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96599999999999997,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69599999999999995,"15min":0.70399999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0609999999999999,"5min":1.264,"15min":1.264},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.754,"15min":0.77000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.054,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.32000000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86399999999999999,"15min":0.86099999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.4330000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.044,"5min":0.95599999999999996,"15min":0.94099999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":3.7440000000000002,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.96599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.92800000000000005,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2929999999999999,"5min":1.373,"15min":1.373},"last":0.628}]},{"osd":6,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.96299999999999997,"15min":0.96899999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2589999999999999,"5min":1.837,"15min":1.837},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:49:14 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.296,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.194,"5min":1.113,"15min":1.099},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":5.2430000000000003,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.0109999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":922,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199054923,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69799999999999995,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86199999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.69199999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.64200000000000002,"15min":0.63100000000000001},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.59599999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0800000000000001,"5min":1.177,"15min":1.177},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60099999999999998,"15min":0.60499999999999998},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86399999999999999,"5min":1.048,"15min":1.048},"last":1.131}]},{"osd":4,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.84699999999999998,"15min":0.83399999999999996},"min":{"1min":0.56100000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.095},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89000000000000001,"15min":0.88300000000000001},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.054}]},{"osd":5,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91600000000000004,"15min":0.91700000000000004},"min":{"1min":0.64500000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.347,"5min":1.609,"15min":1.609},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.89600000000000002,"15min":0.89300000000000002},"min":{"1min":0.64700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.119,"5min":1.208,"15min":1.208},"last":1.167}]},{"osd":6,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.90500000000000003,"15min":0.89500000000000002},"min":{"1min":0.60799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2569999999999999,"5min":1.331,"15min":1.331},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.95099999999999996,"15min":0.94299999999999995},"min":{"1min":0.66400000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.2749999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:49:10 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98899999999999999,"15min":0.99299999999999999},"min":{"1min":0.626,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.337,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.026},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.68200000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.204,"5min":1.306,"15min":1.306},"last":1.3420000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T05:50:00.682 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:50:00.683 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:50:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/402034976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 msgr2=0x7f8d9806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/402034976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9806a000 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009c80 tx=0x7f8d9001cd40 comp rx=0 tx=0).stop 2023-10-26T05:50:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/402034976 shutdown_connections 2023-10-26T05:50:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/402034976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9806a000 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/402034976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d98071270 0x7f8d9815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/402034976 >> 172.21.15.71:0/402034976 conn(0x7f8d980aec60 msgr2=0x7f8d980af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:00.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/402034976 shutdown_connections 2023-10-26T05:50:00.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/402034976 wait complete. 2023-10-26T05:50:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 Processor -- start 2023-10-26T05:50:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- start start 2023-10-26T05:50:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d98071270 0x7f8d98144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9814e7b0 0x7f8d98145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9814ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d981509f0 con 0x7f8d9815c0a0 2023-10-26T05:50:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9814e020 con 0x7f8d9814e7b0 2023-10-26T05:50:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98150cb0 con 0x7f8d98071270 2023-10-26T05:50:00.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9814ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9814ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53256/0 (socket says 172.21.15.71:53256) 2023-10-26T05:50:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 -- 172.21.15.71:0/2282650157 learned_addr learned my addr 172.21.15.71:0/2282650157 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9659c640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d98071270 0x7f8d98144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d95d9b640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9814e7b0 0x7f8d98145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 -- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d98071270 msgr2=0x7f8d98144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d98071270 0x7f8d98144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 -- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9814e7b0 msgr2=0x7f8d98145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:00.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9814e7b0 0x7f8d98145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:00.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 -- 172.21.15.71:0/2282650157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d900098e0 con 0x7f8d9815c0a0 2023-10-26T05:50:00.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d96d9d640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9814ee90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8d9001d5e0 tx=0x7f8d900093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:00.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90004410 con 0x7f8d9815c0a0 2023-10-26T05:50:00.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d90004570 con 0x7f8d9815c0a0 2023-10-26T05:50:00.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d98146500 con 0x7f8d9815c0a0 2023-10-26T05:50:00.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d900a5ad0 con 0x7f8d9815c0a0 2023-10-26T05:50:00.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.833+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9814da60 con 0x7f8d9815c0a0 2023-10-26T05:50:00.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.837+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d98066b10 con 0x7f8d9815c0a0 2023-10-26T05:50:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.837+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d900b4040 con 0x7f8d9815c0a0 2023-10-26T05:50:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.837+0000 7f8d86ffd640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d78066470 0x7f8d78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:00.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.837+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d9011ec10 con 0x7f8d9815c0a0 2023-10-26T05:50:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.837+0000 7f8d9659c640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d78066470 0x7f8d78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.837+0000 7f8d9659c640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d78066470 0x7f8d78068930 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c003d60 tx=0x7f8d8c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:00.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:00.841+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d900ec000 con 0x7f8d9815c0a0 2023-10-26T05:50:01.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.045+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8d98066b10 con 0x7f8d9815c0a0 2023-10-26T05:50:01.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.049+0000 7f8d86ffd640 1 -- 172.21.15.71:0/2282650157 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8d900b2030 con 0x7f8d9815c0a0 2023-10-26T05:50:01.054 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:50:01.054 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:50:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:50:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:50:01.055 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:50:01.056 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:50:01.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d78066470 msgr2=0x7f8d78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d78066470 0x7f8d78068930 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c003d60 tx=0x7f8d8c009040 comp rx=0 tx=0).stop 2023-10-26T05:50:01.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 msgr2=0x7f8d9814ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9814ee90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8d9001d5e0 tx=0x7f8d900093d0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 shutdown_connections 2023-10-26T05:50:01.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d78066470 0x7f8d78068930 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d9815c0a0 0x7f8d9814ee90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9814e7b0 0x7f8d98145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 --2- 172.21.15.71:0/2282650157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d98071270 0x7f8d98144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 >> 172.21.15.71:0/2282650157 conn(0x7f8d980aec60 msgr2=0x7f8d980af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 shutdown_connections 2023-10-26T05:50:01.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.053+0000 7f8d9cedf640 1 -- 172.21.15.71:0/2282650157 wait complete. 2023-10-26T05:50:01.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:50:01.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 -- 172.21.15.71:0/3528117540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 msgr2=0x7f5ea80ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/3528117540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea80ac740 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009900 tx=0x7f5ea401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 -- 172.21.15.71:0/3528117540 shutdown_connections 2023-10-26T05:50:01.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/3528117540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea815bbc0 0x7f5ea8070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/3528117540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea80ac740 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 -- 172.21.15.71:0/3528117540 >> 172.21.15.71:0/3528117540 conn(0x7f5ea8069770 msgr2=0x7f5ea8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 -- 172.21.15.71:0/3528117540 shutdown_connections 2023-10-26T05:50:01.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.185+0000 7f5eb07ac640 1 -- 172.21.15.71:0/3528117540 wait complete. 2023-10-26T05:50:01.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 Processor -- start 2023-10-26T05:50:01.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- start start 2023-10-26T05:50:01.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea81f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea813d800 0x7f5ea8137c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea815bbc0 0x7f5ea8159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80a6600 con 0x7f5ea813d420 2023-10-26T05:50:01.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8138170 con 0x7f5ea815bbc0 2023-10-26T05:50:01.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80a68a0 con 0x7f5ea813d800 2023-10-26T05:50:01.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea81f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea81f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53286/0 (socket says 172.21.15.71:53286) 2023-10-26T05:50:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 -- 172.21.15.71:0/409694413 learned_addr learned my addr 172.21.15.71:0/409694413 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 -- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea813d800 msgr2=0x7f5ea8137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea813d800 0x7f5ea8137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 -- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea815bbc0 msgr2=0x7f5ea8159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea815bbc0 0x7f5ea8159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 -- 172.21.15.71:0/409694413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea4009560 con 0x7f5ea813d420 2023-10-26T05:50:01.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eae548640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea81f3c20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009850 tx=0x7f5ea40a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4005230 con 0x7f5ea813d420 2023-10-26T05:50:01.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ea4005480 con 0x7f5ea813d420 2023-10-26T05:50:01.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4004930 con 0x7f5ea813d420 2023-10-26T05:50:01.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea80a6b80 con 0x7f5ea813d420 2023-10-26T05:50:01.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea80a7140 con 0x7f5ea813d420 2023-10-26T05:50:01.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.189+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea8066b10 con 0x7f5ea813d420 2023-10-26T05:50:01.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.193+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ea40b4070 con 0x7f5ea813d420 2023-10-26T05:50:01.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.193+0000 7f5e9effd640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e8c066470 0x7f5e8c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.193+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ea411e710 con 0x7f5ea813d420 2023-10-26T05:50:01.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.193+0000 7f5eadd47640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e8c066470 0x7f5e8c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.193+0000 7f5eadd47640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e8c066470 0x7f5e8c068930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e98007fd0 tx=0x7f5e98007960 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.197+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ea40ebc20 con 0x7f5ea813d420 2023-10-26T05:50:01.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.385+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea80a7330 con 0x7f5ea813d420 2023-10-26T05:50:01.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.385+0000 7f5e9effd640 1 -- 172.21.15.71:0/409694413 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f5ea40ef240 con 0x7f5ea813d420 2023-10-26T05:50:01.391 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:50:01.391 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:50:01.392 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:50:01.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e8c066470 msgr2=0x7f5e8c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e8c066470 0x7f5e8c068930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5e98007fd0 tx=0x7f5e98007960 comp rx=0 tx=0).stop 2023-10-26T05:50:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 msgr2=0x7f5ea81f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea81f3c20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009850 tx=0x7f5ea40a5780 comp rx=0 tx=0).stop 2023-10-26T05:50:01.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 shutdown_connections 2023-10-26T05:50:01.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea815bbc0 0x7f5ea8159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea813d800 0x7f5ea8137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e8c066470 0x7f5e8c068930 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 --2- 172.21.15.71:0/409694413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea813d420 0x7f5ea81f3c20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 >> 172.21.15.71:0/409694413 conn(0x7f5ea8069770 msgr2=0x7f5ea813ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 shutdown_connections 2023-10-26T05:50:01.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.389+0000 7f5eb07ac640 1 -- 172.21.15.71:0/409694413 wait complete. 2023-10-26T05:50:01.811 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:49:56.624598+0000', 'modified': '2023-10-26T05:49:58.630775+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6264': {'gid': 6264, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/2212616479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 2212616479}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2212616479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:50:01.812 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:50:01.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T05:50:01.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- 172.21.15.71:0/1926551871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 msgr2=0x7f3ebc06a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/1926551871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc06a7e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0cdd20 tx=0x7f3eb001f090 comp rx=0 tx=0).stop 2023-10-26T05:50:01.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- 172.21.15.71:0/1926551871 shutdown_connections 2023-10-26T05:50:01.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/1926551871 >> v2:172.21.15.86:3300/0 conn(0x7f3ebc0cc7c0 0x7f3ebc0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/1926551871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc06a7e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- 172.21.15.71:0/1926551871 >> 172.21.15.71:0/1926551871 conn(0x7f3ebc0d12d0 msgr2=0x7f3ebc0d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- 172.21.15.71:0/1926551871 shutdown_connections 2023-10-26T05:50:01.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- 172.21.15.71:0/1926551871 wait complete. 2023-10-26T05:50:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 Processor -- start 2023-10-26T05:50:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- start start 2023-10-26T05:50:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ebc0cc7c0 0x7f3ebc15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc15b3b0 0x7f3ebc1e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc156b70 con 0x7f3ebc06a390 2023-10-26T05:50:01.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc155f60 con 0x7f3ebc15b3b0 2023-10-26T05:50:01.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec2a71640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc156240 con 0x7f3ebc0cc7c0 2023-10-26T05:50:01.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc14e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44718/0 (socket says 172.21.15.71:44718) 2023-10-26T05:50:01.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 -- 172.21.15.71:0/3104432643 learned_addr learned my addr 172.21.15.71:0/3104432643 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:01.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ebc0cc7c0 0x7f3ebc15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 -- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ebc0cc7c0 msgr2=0x7f3ebc15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ebc0cc7c0 0x7f3ebc15d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 -- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc15b3b0 msgr2=0x7f3ebc1e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec100e640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc15b3b0 0x7f3ebc1e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc15b3b0 0x7f3ebc1e5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 -- 172.21.15.71:0/3104432643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0009560 con 0x7f3ebc06a390 2023-10-26T05:50:01.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.937+0000 7f3ec080d640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc14e830 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3eac001ec0 tx=0x7f3eac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3eb97fa640 1 -- 172.21.15.71:0/3104432643 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac0054e0 con 0x7f3ebc06a390 2023-10-26T05:50:01.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3eb97fa640 1 -- 172.21.15.71:0/3104432643 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3eac003600 con 0x7f3ebc06a390 2023-10-26T05:50:01.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc156520 con 0x7f3ebc06a390 2023-10-26T05:50:01.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3eb97fa640 1 -- 172.21.15.71:0/3104432643 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac003a10 con 0x7f3ebc06a390 2023-10-26T05:50:01.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc14ed70 con 0x7f3ebc06a390 2023-10-26T05:50:01.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3eb97fa640 1 -- 172.21.15.71:0/3104432643 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3eac003b70 con 0x7f3ebc06a390 2023-10-26T05:50:01.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3eb97fa640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ea8066470 0x7f3ea8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3eb97fa640 1 -- 172.21.15.71:0/3104432643 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eac0835e0 con 0x7f3ebc06a390 2023-10-26T05:50:01.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ea8066470 0x7f3ea8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 Processor -- start 2023-10-26T05:50:01.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 -- start start 2023-10-26T05:50:01.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ea8066470 0x7f3ea8068930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00b2000 tx=0x7f3eb001d8d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 0x5620c8789a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c878a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c857ecc0 con 0x5620c87d42e0 2023-10-26T05:50:01.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c85e1920 con 0x5620c87d39c0 2023-10-26T05:50:01.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620c85dee50 con 0x5620c8789f80 2023-10-26T05:50:01.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ebbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 0x5620c8789a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ebbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 0x5620c8789a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44726/0 (socket says 172.21.15.71:44726) 2023-10-26T05:50:01.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.941+0000 7f3ebbfff640 1 -- 172.21.15.71:0/1242738281 learned_addr learned my addr 172.21.15.71:0/1242738281 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:01.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 -- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 msgr2=0x5620c878a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec100e640 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c878a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c878a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 -- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 msgr2=0x5620c87d3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87d3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 -- 172.21.15.71:0/1242738281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620c82d3890 con 0x5620c87d42e0 2023-10-26T05:50:01.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 0x5620c8789a40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3eb009b9c0 tx=0x7f3eb00b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e10c03fa3396787 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ea27fc640 1 -- 172.21.15.71:0/1242738281 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0005300 con 0x5620c87d42e0 2023-10-26T05:50:01.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ea27fc640 1 -- 172.21.15.71:0/1242738281 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3eb0005550 con 0x5620c87d42e0 2023-10-26T05:50:01.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ea27fc640 1 -- 172.21.15.71:0/1242738281 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0004930 con 0x5620c87d42e0 2023-10-26T05:50:01.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1242738281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 msgr2=0x5620c8789a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 0x5620c8789a40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3eb009b9c0 tx=0x7f3eb00b8000 comp rx=0 tx=0).stop 2023-10-26T05:50:01.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1242738281 shutdown_connections 2023-10-26T05:50:01.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c878a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d42e0 0x5620c8789a40 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1242738281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87d3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1242738281 >> 172.21.15.71:0/1242738281 conn(0x5620c87812f0 msgr2=0x5620c8788320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1242738281 shutdown_connections 2023-10-26T05:50:01.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1242738281 wait complete. 2023-10-26T05:50:01.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- start start 2023-10-26T05:50:01.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 Processor -- start 2023-10-26T05:50:01.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 0x5620c8794b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87d42e0 0x5620c8795610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c85da740 con 0x5620c8789f80 2023-10-26T05:50:01.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c8599d10 con 0x5620c87d39c0 2023-10-26T05:50:01.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620c85999b0 con 0x5620c87d42e0 2023-10-26T05:50:01.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 10 client.0 _ll_register_callbacks cb 0x5620c87756c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:50:01.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 0x5620c8794b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 0x5620c8794b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44732/0 (socket says 172.21.15.71:44732) 2023-10-26T05:50:01.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 -- 172.21.15.71:0/3142845368 learned_addr learned my addr 172.21.15.71:0/3142845368 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:01.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec100e640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87d42e0 0x5620c8795610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 -- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87d42e0 msgr2=0x5620c8795610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87d42e0 0x5620c8795610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 -- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 msgr2=0x5620c87950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87950d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 -- 172.21.15.71:0/3142845368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc06ae30 con 0x5620c8789f80 2023-10-26T05:50:01.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec080d640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 0x5620c8794b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00cde0 tx=0x7f3eac054720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1841ea4e3df440ba server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3e9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:50:01.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/3142845368 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac05c790 con 0x5620c8789f80 2023-10-26T05:50:01.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/3142845368 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eac050510 con 0x5620c8789f80 2023-10-26T05:50:01.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620c858e370 con 0x5620c8789f80 2023-10-26T05:50:01.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/3142845368 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eac0506d0 con 0x5620c8789f80 2023-10-26T05:50:01.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 10 client.6399 fetch_fsmap learned FSMap version 25 2023-10-26T05:50:01.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5620c859c460 con 0x5620c8789f80 2023-10-26T05:50:01.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/3142845368 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5620c858e370 con 0x5620c8789f80 2023-10-26T05:50:01.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/3142845368 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3eac053680 con 0x5620c8789f80 2023-10-26T05:50:01.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 10 client.6399 fetch_fsmap finished waiting for FSMap version 25 2023-10-26T05:50:01.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 10 client.6399 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:50:01.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 20 client.6399 populate_metadata read hostname 'smithi071' 2023-10-26T05:50:01.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87e2a50 0x5620c87e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 4 client.6399 mds_command: new command op to 6264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:50:01.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 --> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620c85742d0 con 0x5620c87e2a50 2023-10-26T05:50:01.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.945+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87e2a50 0x5620c87e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87e2a50 0x5620c87e4e40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a72e0f0f650801a8 server_cookie=d4dcefe216a16829 in_seq=0 out_seq=0 2023-10-26T05:50:01.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3e9b7fe640 10 client.6399 ms_handle_connect on v2:172.21.15.71:6834/2212616479 2023-10-26T05:50:01.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/3142845368 <== mds.0 v2:172.21.15.71:6834/2212616479 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5620c85742d0 con 0x5620c87e2a50 2023-10-26T05:50:01.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3e9b7fe640 10 client.6399 handle_command_reply: tid=0 2023-10-26T05:50:01.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 client.6399 shutdown 2023-10-26T05:50:01.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 msgr2=0x5620c8794b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 0x5620c8794b90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00cde0 tx=0x7f3eac054720 comp rx=0 tx=0).stop 2023-10-26T05:50:01.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 shutdown_connections 2023-10-26T05:50:01.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87e2a50 0x5620c87e4e40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87d42e0 0x5620c8795610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87d39c0 0x5620c87950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3e9b7fe640 0 client.6399 ms_handle_reset on v2:172.21.15.71:6834/2212616479 2023-10-26T05:50:01.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/3142845368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c8789f80 0x5620c8794b90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 >> 172.21.15.71:0/3142845368 conn(0x5620c87812f0 msgr2=0x5620c8788a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 shutdown_connections 2023-10-26T05:50:01.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/3142845368 wait complete. 2023-10-26T05:50:01.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.949+0000 7f3ec5bbb000 20 client.6399 trim_cache size 0 max 16384 2023-10-26T05:50:01.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.961+0000 7f3ec5bbb000 1 Processor -- start 2023-10-26T05:50:01.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.961+0000 7f3ec5bbb000 1 -- start start 2023-10-26T05:50:01.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87812f0 0x5620c882e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c8789f80 0x5620c882e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c8294940 con 0x5620c87d39c0 2023-10-26T05:50:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c85e4200 con 0x5620c8789f80 2023-10-26T05:50:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620c85994d0 con 0x5620c87812f0 2023-10-26T05:50:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44746/0 (socket says 172.21.15.71:44746) 2023-10-26T05:50:01.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/1304747751 learned_addr learned my addr 172.21.15.71:0/1304747751 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec080d640 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87812f0 0x5620c882e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87812f0 msgr2=0x5620c882e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87812f0 0x5620c882e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c8789f80 msgr2=0x5620c882e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c8789f80 0x5620c882e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/1304747751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620c8582760 con 0x5620c87d39c0 2023-10-26T05:50:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882ecd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400acc0 tx=0x7f3eb400adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4bc09ffd783cef server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ea2ffd640 1 -- 172.21.15.71:0/1304747751 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400fd90 con 0x5620c87d39c0 2023-10-26T05:50:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ea2ffd640 1 -- 172.21.15.71:0/1304747751 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3eb4004b30 con 0x5620c87d39c0 2023-10-26T05:50:01.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ea2ffd640 1 -- 172.21.15.71:0/1304747751 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4005020 con 0x5620c87d39c0 2023-10-26T05:50:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1304747751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 msgr2=0x5620c882ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882ecd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400acc0 tx=0x7f3eb400adb0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1304747751 shutdown_connections 2023-10-26T05:50:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882ecd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c8789f80 0x5620c882e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/1304747751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c87812f0 0x5620c882e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1304747751 >> 172.21.15.71:0/1304747751 conn(0x5620c878f830 msgr2=0x5620c87e5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1304747751 shutdown_connections 2023-10-26T05:50:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/1304747751 wait complete. 2023-10-26T05:50:01.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- start start 2023-10-26T05:50:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 Processor -- start 2023-10-26T05:50:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87812f0 0x5620c87d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c882e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c878ecd0 con 0x5620c87d39c0 2023-10-26T05:50:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620c82d41b0 con 0x5620c87812f0 2023-10-26T05:50:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5620c878cd90 con 0x5620c8789f80 2023-10-26T05:50:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec5bbb000 10 client.0 _ll_register_callbacks cb 0x5620c85d8ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:50:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44752/0 (socket says 172.21.15.71:44752) 2023-10-26T05:50:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/2146567492 learned_addr learned my addr 172.21.15.71:0/2146567492 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c882e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 msgr2=0x5620c882e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c882e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87812f0 msgr2=0x5620c87d2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:50:01.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec080d640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87812f0 0x5620c87d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87812f0 0x5620c87d2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 -- 172.21.15.71:0/2146567492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb400a8f0 con 0x5620c87d39c0 2023-10-26T05:50:01.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ebbfff640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c882e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:50:01.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3ec100e640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882e830 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400f7f0 tx=0x7f3eb400fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6406dbe26cabb0e server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:01.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3e9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:50:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.965+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/2146567492 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4005430 con 0x5620c87d39c0 2023-10-26T05:50:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620c878d950 con 0x5620c87d39c0 2023-10-26T05:50:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/2146567492 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3eb401f070 con 0x5620c87d39c0 2023-10-26T05:50:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/2146567492 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eb4005590 con 0x5620c87d39c0 2023-10-26T05:50:01.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 10 client.6405 fetch_fsmap learned FSMap version 25 2023-10-26T05:50:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5620c82d3cc0 con 0x5620c87d39c0 2023-10-26T05:50:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/2146567492 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5620c878d950 con 0x5620c87d39c0 2023-10-26T05:50:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/2146567492 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3eb40057e0 con 0x5620c87d39c0 2023-10-26T05:50:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 10 client.6405 fetch_fsmap finished waiting for FSMap version 25 2023-10-26T05:50:01.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 10 client.6405 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:50:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 20 client.6405 populate_metadata read hostname 'smithi071' 2023-10-26T05:50:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87d42e0 0x5620c883b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 4 client.6405 mds_command: new command op to 6264 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:50:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 --> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5620c85726f0 con 0x5620c87d42e0 2023-10-26T05:50:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec080d640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87d42e0 0x5620c883b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:01.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec080d640 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87d42e0 0x5620c883b100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a19efaeb126e68c server_cookie=1f6eb60d3d29ea4c in_seq=0 out_seq=0 2023-10-26T05:50:01.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 10 client.6405 ms_handle_connect on v2:172.21.15.71:6834/2212616479 2023-10-26T05:50:01.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 1 -- 172.21.15.71:0/2146567492 <== mds.0 v2:172.21.15.71:6834/2212616479 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5620c85726f0 con 0x5620c87d42e0 2023-10-26T05:50:01.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 10 client.6405 handle_command_reply: tid=0 2023-10-26T05:50:01.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 client.6405 shutdown 2023-10-26T05:50:01.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 msgr2=0x5620c882e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882e830 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400f7f0 tx=0x7f3eb400fdc0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 shutdown_connections 2023-10-26T05:50:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:6834/2212616479,v1:172.21.15.71:6835/2212616479] conn(0x5620c87d42e0 0x5620c883b100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5620c87d39c0 0x5620c882e830 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3e9b7fe640 0 client.6405 ms_handle_reset on v2:172.21.15.71:6834/2212616479 2023-10-26T05:50:01.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5620c8789f80 0x5620c882e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 --2- 172.21.15.71:0/2146567492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620c87812f0 0x5620c87d2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:01.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 >> 172.21.15.71:0/2146567492 conn(0x5620c878f830 msgr2=0x5620c87e6450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:01.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 shutdown_connections 2023-10-26T05:50:01.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 1 -- 172.21.15.71:0/2146567492 wait complete. 2023-10-26T05:50:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec5bbb000 20 client.6405 trim_cache size 0 max 16384 2023-10-26T05:50:01.997 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:50:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:50:01.997 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:50:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6264, 2023-10-26T05:50:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:50:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:50:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:50:01.998 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 4.7052088750000003, 2023-10-26T05:50:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 25, 2023-10-26T05:50:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 68, 2023-10-26T05:50:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 68, 2023-10-26T05:50:01.999 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 9.3955733610000003 2023-10-26T05:50:01.999 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:50:02.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ea8066470 msgr2=0x7f3ea8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ea8066470 0x7f3ea8068930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00b2000 tx=0x7f3eb001d8d0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 msgr2=0x7f3ebc14e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc14e830 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3eac001ec0 tx=0x7f3eac00c8e0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 shutdown_connections 2023-10-26T05:50:02.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ebc15b3b0 0x7f3ebc1e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ebc0cc7c0 0x7f3ebc15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ea8066470 0x7f3ea8068930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 --2- 172.21.15.71:0/3104432643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ebc06a390 0x7f3ebc14e830 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 >> 172.21.15.71:0/3104432643 conn(0x7f3ebc0d12d0 msgr2=0x7f3ebc15a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:02.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 shutdown_connections 2023-10-26T05:50:02.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:01.969+0000 7f3ec2a71640 1 -- 172.21.15.71:0/3104432643 wait complete. 2023-10-26T05:50:02.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:50:02.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1606106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240047d0 msgr2=0x7f6324004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 --2- 172.21.15.71:0/1606106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240047d0 0x7f6324004c50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f633000c700 tx=0x7f633001c650 comp rx=0 tx=0).stop 2023-10-26T05:50:02.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1606106830 shutdown_connections 2023-10-26T05:50:02.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 --2- 172.21.15.71:0/1606106830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240047d0 0x7f6324004c50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1606106830 >> 172.21.15.71:0/1606106830 conn(0x7f633c070650 msgr2=0x7f633c070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:02.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1606106830 shutdown_connections 2023-10-26T05:50:02.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1606106830 wait complete. 2023-10-26T05:50:02.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 Processor -- start 2023-10-26T05:50:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- start start 2023-10-26T05:50:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240047d0 0x7f633c1e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 0x7f633c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633c0c9680 0x7f633c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:02.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1e6030 con 0x7f633c155b20 2023-10-26T05:50:02.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1e5660 con 0x7f633c0c9680 2023-10-26T05:50:02.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c1e4a50 con 0x7f63240047d0 2023-10-26T05:50:02.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 0x7f633c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:02.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 0x7f633c0c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44768/0 (socket says 172.21.15.71:44768) 2023-10-26T05:50:02.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 -- 172.21.15.71:0/1014806724 learned_addr learned my addr 172.21.15.71:0/1014806724 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:02.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633b7fe640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633c0c9680 0x7f633c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:02.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633affd640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240047d0 0x7f633c1e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:02.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 -- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240047d0 msgr2=0x7f633c1e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240047d0 0x7f633c1e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 -- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633c0c9680 msgr2=0x7f633c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633c0c9680 0x7f633c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 -- 172.21.15.71:0/1014806724 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c0ccd30 con 0x7f633c155b20 2023-10-26T05:50:02.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f633a7fc640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 0x7f633c0c9140 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6330000c10 tx=0x7f6330004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:02.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300ba070 con 0x7f633c155b20 2023-10-26T05:50:02.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6330004e40 con 0x7f633c155b20 2023-10-26T05:50:02.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c1e4d30 con 0x7f633c155b20 2023-10-26T05:50:02.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330006130 con 0x7f633c155b20 2023-10-26T05:50:02.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.105+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c1e52f0 con 0x7f633c155b20 2023-10-26T05:50:02.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.109+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c066b10 con 0x7f633c155b20 2023-10-26T05:50:02.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.109+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63300b5470 con 0x7f633c155b20 2023-10-26T05:50:02.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.109+0000 7f6323fff640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6328066470 0x7f6328068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:02.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.109+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f633011e190 con 0x7f633c155b20 2023-10-26T05:50:02.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.109+0000 7f633affd640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6328066470 0x7f6328068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:02.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.109+0000 7f633affd640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6328066470 0x7f6328068930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f632c0043d0 tx=0x7f632c003980 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:02.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.113+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f63300eb610 con 0x7f633c155b20 2023-10-26T05:50:02.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.285+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f633c1e57d0 con 0x7f633c155b20 2023-10-26T05:50:02.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.285+0000 7f6323fff640 1 -- 172.21.15.71:0/1014806724 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f63300eec30 con 0x7f633c155b20 2023-10-26T05:50:02.290 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:50:02.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6328066470 msgr2=0x7f6328068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6328066470 0x7f6328068930 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f632c0043d0 tx=0x7f632c003980 comp rx=0 tx=0).stop 2023-10-26T05:50:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 msgr2=0x7f633c0c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 0x7f633c0c9140 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6330000c10 tx=0x7f6330004c50 comp rx=0 tx=0).stop 2023-10-26T05:50:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 shutdown_connections 2023-10-26T05:50:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f633c0c9680 0x7f633c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6328066470 0x7f6328068930 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f633c155b20 0x7f633c0c9140 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 --2- 172.21.15.71:0/1014806724 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240047d0 0x7f633c1e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 >> 172.21.15.71:0/1014806724 conn(0x7f633c070650 msgr2=0x7f633c1579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 shutdown_connections 2023-10-26T05:50:02.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:02.289+0000 7f63419a3640 1 -- 172.21.15.71:0/1014806724 wait complete. 2023-10-26T05:50:02.307 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:50:02.308 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:50:02.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:50:02.309 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:50:02.309 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:02.309 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:50:02.309 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:50:02.310 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:50:02.310 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:50:02.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.311 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:50:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:50:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:50:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:50:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:50:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.318 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:50:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:50:02.319 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:50:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:02.319 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:50:02.320 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:50:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:02.320 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:50:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:02.320 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:50:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42374sec preferred_lft 42374sec 2023-10-26T05:50:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:50:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.321 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:50:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:02.321 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:50:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.322 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:50:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:50:02.322 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:50:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.323 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:50:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:02.323 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:50:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.324 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:50:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:50:02.324 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:02.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.325 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:50:02.369 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:50:02.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.371 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:50:02.418 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:50:02.419 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:50:02.419 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:50:02.419 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:50:02.419 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:50:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:50:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:50:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:50:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:50:02.420 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:50:02.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.421 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:50:02.422 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:50:02.422 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:50:02.422 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:50:02.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:50:02.423 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:50:02.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.495 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:50:02.540 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:50:02.540 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:50:02.541 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:50:02.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.542 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:50:02.600 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:50:02.600 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:50:02.601 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:02.648 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:50:02.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:50:02.649 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:02.699 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:50:02.750 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:50:02.750 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:50:02.750 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:50:02.751 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:50:02.751 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:50:02.751 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:50:02.751 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:50:02.751 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:50:02.752 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:50:02.752 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:50:02.752 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:50:02.752 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:50:02.752 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:50:02.753 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:50:02.753 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:50:02.753 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:50:02.753 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:50:02.753 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:50:02.754 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:50:02.754 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:50:02.754 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:50:02.754 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:50:02.754 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:50:02.755 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:50:02.755 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:50:02.755 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:50:02.755 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:50:02.755 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:50:02.755 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:50:02.756 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:02.756 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:02.756 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:02.756 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:02.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.757 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:50:02.797 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:50:02.797 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:50:02.841 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:50:02.848 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:50:02.848 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:50:02.849 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:50:02.849 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:50:02.849 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:50:02.849 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:50:02.849 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:50:02.850 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:50:02.850 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:50:02.850 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:50:02.850 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:50:02.850 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:50:02.851 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:50:02.851 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:50:02.851 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:50:02.851 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:50:02.851 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:50:02.852 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:50:02.852 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:50:02.852 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:50:02.852 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:50:02.852 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:50:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:50:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:50:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:50:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:50:02.853 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:50:02.854 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:50:02.854 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:50:02.854 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:02.854 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:02.854 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:02.855 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:02.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:02.857 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:50:02.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:50:02.917+0000 7f58272b4580 -1 init, newargv = 0x55afb6fa44b0 newargc=15 2023-10-26T05:50:02.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[42323]: starting ceph client 2023-10-26T05:50:02.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[42323]: starting fuse 2023-10-26T05:50:03.222 INFO:tasks.ceph.mds.c.smithi071.stderr:2023-10-26T05:50:03.217+0000 7fdae5c9b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:50:04.006 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:50:04.015 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:50:04.016 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:50:04.016 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:50:04.016 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:50:04.016 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:50:04.017 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:50:04.017 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:50:04.017 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:50:04.017 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:50:04.017 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:50:04.017 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:50:04.018 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:50:04.018 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:50:04.018 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:50:04.018 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:50:04.018 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:50:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:50:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:50:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:50:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:50:04.019 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:50:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:50:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:50:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:50:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:50:04.020 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:50:04.021 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:50:04.021 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:50:04.021 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:50:04.021 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:04.021 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:50:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:50:04.022 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:04.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:04.025 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:50:04.072 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:50:04.072 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:50:04.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:50:04.073 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:50:04.074 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:50:04.074 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:50:04.074 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:50:04.075 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:04.075 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:50:04.076 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:50:04.076 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:50:04.076 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:50:04.077 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:04.077 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:50:04.078 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:50:04.078 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:50:04.079 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:50:04.079 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:04.079 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:50:04.080 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:50:04.080 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:50:04.081 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:50:04.081 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:50:04.081 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:50:04.082 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:50:04.082 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:50:04.083 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:04.083 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:50:04.083 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:50:04.195 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.42323.asok 2023-10-26T05:50:04.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:04.206 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42323.asok 2023-10-26T05:50:04.206 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:50:04.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.42323.asok status 2023-10-26T05:50:04.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:50:04.400 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:50:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:50:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:50:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:50:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:50:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:50:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:50:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "42323", 2023-10-26T05:50:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:50:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:50:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:50:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:50:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6414, 2023-10-26T05:50:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:50:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:50:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:50:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 6414 2023-10-26T05:50:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:50:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:50:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:50:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:50:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3637970198 2023-10-26T05:50:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:50:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:50:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:50:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:50:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:50:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3637970198 2023-10-26T05:50:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:50:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.6414 192.168.0.1:0/3637970198", 2023-10-26T05:50:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/3637970198", 2023-10-26T05:50:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:50:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 25, 2023-10-26T05:50:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 68, 2023-10-26T05:50:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:50:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:50:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:50:04.410 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:50:04.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:04.420 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:50:04.420 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:50:04.421 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:04.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:50:04.471 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:50:04.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:04.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:04.478 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:50:04.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:50:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:50:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:04.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:50:04.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- 172.21.15.71:0/3930436934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 msgr2=0x7f0c6c06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:04.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 --2- 172.21.15.71:0/3930436934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c06b450 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0acf90 tx=0x7f0c5401c7b0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- 172.21.15.71:0/3930436934 shutdown_connections 2023-10-26T05:50:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 --2- 172.21.15.71:0/3930436934 >> v2:172.21.15.86:3300/0 conn(0x7f0c6c06bf90 0x7f0c6c06c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 --2- 172.21.15.71:0/3930436934 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c06b450 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- 172.21.15.71:0/3930436934 >> 172.21.15.71:0/3930436934 conn(0x7f0c6c0cddc0 msgr2=0x7f0c6c06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- 172.21.15.71:0/3930436934 shutdown_connections 2023-10-26T05:50:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- 172.21.15.71:0/3930436934 wait complete. 2023-10-26T05:50:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 Processor -- start 2023-10-26T05:50:04.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- start start 2023-10-26T05:50:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c13d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c6c06bf90 0x7f0c6c13a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c158f00 0x7f0c6c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c151200 con 0x7f0c6c06b010 2023-10-26T05:50:04.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c13ac50 con 0x7f0c6c158f00 2023-10-26T05:50:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c711fd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c13dbf0 con 0x7f0c6c06bf90 2023-10-26T05:50:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c13d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c13d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44792/0 (socket says 172.21.15.71:44792) 2023-10-26T05:50:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 -- 172.21.15.71:0/1177355155 learned_addr learned my addr 172.21.15.71:0/1177355155 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:04.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6a59c640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c6c06bf90 0x7f0c6c13a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:04.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6b59e640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c158f00 0x7f0c6c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:04.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 -- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c6c06bf90 msgr2=0x7f0c6c13a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c6c06bf90 0x7f0c6c13a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 -- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c158f00 msgr2=0x7f0c6c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c158f00 0x7f0c6c155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.685+0000 7f0c6ad9d640 1 -- 172.21.15.71:0/1177355155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c540094a0 con 0x7f0c6c06b010 2023-10-26T05:50:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c6ad9d640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c13d6b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400cae0 tx=0x7f0c6400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64003b80 con 0x7f0c6c06b010 2023-10-26T05:50:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0c64003df0 con 0x7f0c6c06b010 2023-10-26T05:50:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400aab0 con 0x7f0c6c06b010 2023-10-26T05:50:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c154a60 con 0x7f0c6c06b010 2023-10-26T05:50:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c155020 con 0x7f0c6c06b010 2023-10-26T05:50:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c066d00 con 0x7f0c6c06b010 2023-10-26T05:50:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0c6400ac10 con 0x7f0c6c06b010 2023-10-26T05:50:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c637fe640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0c4c066470 0x7f0c4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c64083c50 con 0x7f0c6c06b010 2023-10-26T05:50:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c6a59c640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0c4c066470 0x7f0c4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.689+0000 7f0c6a59c640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0c4c066470 0x7f0c4c068930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0c540af000 tx=0x7f0c5401aff0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.693+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0c64089020 con 0x7f0c6c06b010 2023-10-26T05:50:04.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.885+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0c6c165c10 con 0x7f0c6c06b010 2023-10-26T05:50:04.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.885+0000 7f0c637fe640 1 -- 172.21.15.71:0/1177355155 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0c64051160 con 0x7f0c6c06b010 2023-10-26T05:50:04.891 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:50:04.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0c4c066470 msgr2=0x7f0c4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:04.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0c4c066470 0x7f0c4c068930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0c540af000 tx=0x7f0c5401aff0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 msgr2=0x7f0c6c13d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:04.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c13d6b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400cae0 tx=0x7f0c6400cbd0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 shutdown_connections 2023-10-26T05:50:04.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c6c158f00 0x7f0c6c155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0c6c06bf90 0x7f0c6c13a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0c4c066470 0x7f0c4c068930 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 --2- 172.21.15.71:0/1177355155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c06b010 0x7f0c6c13d6b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 >> 172.21.15.71:0/1177355155 conn(0x7f0c6c0cddc0 msgr2=0x7f0c6c15bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:04.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 shutdown_connections 2023-10-26T05:50:04.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:04.889+0000 7f0c711fd640 1 -- 172.21.15.71:0/1177355155 wait complete. 2023-10-26T05:50:04.910 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:50:04.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:50:04.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:50:04.911 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:50:04.911 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:04.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:50:04.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:50:04.913 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:50:04.913 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:50:04.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:04.914 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:50:04.922 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:50:04.923 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:50:04.923 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:50:04.923 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.924 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:50:04.924 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.924 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:50:04.924 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:04.925 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:50:04.925 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:50:04.925 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:04.925 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:50:04.926 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:50:04.926 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:04.926 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:50:04.926 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:04.927 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:50:04.927 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42366sec preferred_lft 42366sec 2023-10-26T05:50:04.927 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:50:04.928 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.928 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:50:04.928 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:04.929 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:50:04.929 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.929 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:50:04.929 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:50:04.930 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:50:04.930 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.930 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:50:04.930 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.931 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:50:04.931 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:04.931 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:50:04.931 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.932 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:50:04.932 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:04.932 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:50:04.932 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:50:04.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:04.934 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:50:04.993 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:50:04.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:04.994 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:50:05.050 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:50:05.051 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:50:05.051 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:50:05.051 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:50:05.051 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:50:05.051 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:50:05.052 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:50:05.052 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:50:05.052 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:50:05.052 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:50:05.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:05.053 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:50:05.054 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:50:05.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:50:05.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:50:05.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:50:05.056 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:50:05.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:05.129 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:50:05.175 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:50:05.175 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:50:05.176 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:50:05.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:05.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:50:05.230 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:50:05.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:50:05.231 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:05.278 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:50:05.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:50:05.279 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:05.327 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:50:05.375 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:50:05.375 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:50:05.375 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:50:05.376 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:50:05.376 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:50:05.376 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:50:05.376 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:50:05.376 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:50:05.377 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:50:05.377 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:50:05.377 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:50:05.377 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:50:05.377 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:50:05.378 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:50:05.378 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:50:05.378 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:50:05.378 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:50:05.378 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:50:05.379 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:50:05.379 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:50:05.379 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:50:05.379 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:50:05.379 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:50:05.380 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:50:05.380 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:50:05.380 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:50:05.380 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:50:05.380 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:50:05.381 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:50:05.381 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:05.381 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:05.381 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:05.381 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:05.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:05.383 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:50:05.430 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:50:05.431 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:50:05.476 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:50:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:50:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:50:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:50:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:50:05.482 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:50:05.482 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:50:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:50:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:50:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:50:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:50:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:50:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:50:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:50:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:50:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:50:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:50:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:50:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:50:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:50:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:50:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:50:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:50:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:50:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:50:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:50:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:50:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:50:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:50:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:50:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:05.488 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:05.488 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:05.488 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:05.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:05.489 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:50:05.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:50:05.539+0000 7f491ae71580 -1 init, newargv = 0x5626ce37f320 newargc=15 2023-10-26T05:50:05.544 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34414]: starting ceph client 2023-10-26T05:50:05.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34414]: starting fuse 2023-10-26T05:50:06.632 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:50:06.641 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:50:06.642 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:50:06.642 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:50:06.642 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:50:06.642 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:50:06.643 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:50:06.643 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:50:06.643 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:50:06.643 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:50:06.643 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:50:06.643 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:50:06.644 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:50:06.644 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:50:06.644 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:50:06.644 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:50:06.644 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:50:06.645 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:50:06.645 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:50:06.645 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:50:06.645 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:50:06.645 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:50:06.646 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:50:06.646 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:50:06.646 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:50:06.646 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:50:06.646 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:50:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:50:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:50:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:50:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:50:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:06.648 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:50:06.648 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:06.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:50:06.649 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:50:06.695 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:50:06.696 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:50:06.696 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:50:06.696 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:50:06.697 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:50:06.697 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:50:06.698 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:50:06.698 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:50:06.698 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:50:06.699 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:50:06.699 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:50:06.699 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:50:06.700 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:50:06.700 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:50:06.701 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:50:06.701 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:50:06.701 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:50:06.702 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:50:06.702 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:50:06.702 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:50:06.703 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:50:06.703 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:50:06.704 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:50:06.704 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:50:06.704 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:50:06.705 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:50:06.705 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:50:06.705 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:50:06.706 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:50:06.818 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.34414.asok 2023-10-26T05:50:06.822 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:06.829 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34414.asok 2023-10-26T05:50:06.830 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:50:06.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.34414.asok status 2023-10-26T05:50:06.913 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:50:07.013 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:50:07.014 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:50:07.014 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:50:07.014 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:50:07.014 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:50:07.014 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:50:07.015 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:50:07.015 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34414", 2023-10-26T05:50:07.015 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:50:07.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:50:07.015 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:50:07.016 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:50:07.016 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5108, 2023-10-26T05:50:07.016 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:50:07.016 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:50:07.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:50:07.017 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 5108 2023-10-26T05:50:07.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:50:07.017 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:50:07.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:50:07.017 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:50:07.018 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2274762366 2023-10-26T05:50:07.018 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:50:07.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:50:07.018 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:50:07.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:50:07.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:50:07.019 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2274762366 2023-10-26T05:50:07.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:50:07.019 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.5108 192.168.0.1:0/2274762366", 2023-10-26T05:50:07.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2274762366", 2023-10-26T05:50:07.020 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:50:07.020 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 25, 2023-10-26T05:50:07.020 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 68, 2023-10-26T05:50:07.021 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:50:07.021 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:50:07.021 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:50:07.021 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:50:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:07.033 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:50:07.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:50:07.034 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:07.122 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:50:07.124 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:50:07.124 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:07.131 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:07.131 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:50:07.132 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:50:07.176 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:50:07.177 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:07.183 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:50:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.313+0000 7f0fbb59c640 1 -- 172.21.15.71:0/1712085072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 msgr2=0x7f0fb40b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.313+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/1712085072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb40b1a70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009bf0 tx=0x7f0fac01cb10 comp rx=0 tx=0).stop 2023-10-26T05:50:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.313+0000 7f0fbb59c640 1 -- 172.21.15.71:0/1712085072 shutdown_connections 2023-10-26T05:50:07.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.313+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/1712085072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb40b1a70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.313+0000 7f0fbb59c640 1 -- 172.21.15.71:0/1712085072 >> 172.21.15.71:0/1712085072 conn(0x7f0fb4069770 msgr2=0x7f0fb4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- 172.21.15.71:0/1712085072 shutdown_connections 2023-10-26T05:50:07.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- 172.21.15.71:0/1712085072 wait complete. 2023-10-26T05:50:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 Processor -- start 2023-10-26T05:50:07.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- start start 2023-10-26T05:50:07.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb4070fe0 0x7f0fb4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:07.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:07.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb4140cc0 0x7f0fb4154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:07.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4139100 con 0x7f0fb4071470 2023-10-26T05:50:07.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415b9c0 con 0x7f0fb4140cc0 2023-10-26T05:50:07.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4137a40 con 0x7f0fb4070fe0 2023-10-26T05:50:07.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:07.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44806/0 (socket says 172.21.15.71:44806) 2023-10-26T05:50:07.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 -- 172.21.15.71:0/4027373989 learned_addr learned my addr 172.21.15.71:0/4027373989 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:07.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb9b39640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb4140cc0 0x7f0fb4154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:07.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb9338640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb4070fe0 0x7f0fb4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:07.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 -- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb4070fe0 msgr2=0x7f0fb4156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:07.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb4070fe0 0x7f0fb4156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 -- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb4140cc0 msgr2=0x7f0fb4154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:07.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb4140cc0 0x7f0fb4154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 -- 172.21.15.71:0/4027373989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009820 con 0x7f0fb4071470 2023-10-26T05:50:07.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb9b39640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb4140cc0 0x7f0fb4154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:50:07.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb9338640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb4070fe0 0x7f0fb4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:50:07.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fb8b37640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb415b480 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c490 tx=0x7f0fb0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:07.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003600 con 0x7f0fb4071470 2023-10-26T05:50:07.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0fb0003780 con 0x7f0fb4071470 2023-10-26T05:50:07.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb4155f20 con 0x7f0fb4071470 2023-10-26T05:50:07.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003bd0 con 0x7f0fb4071470 2023-10-26T05:50:07.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41564e0 con 0x7f0fb4071470 2023-10-26T05:50:07.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.317+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb4066b10 con 0x7f0fb4071470 2023-10-26T05:50:07.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.321+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0fb0003d30 con 0x7f0fb4071470 2023-10-26T05:50:07.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.321+0000 7f0fa9ffb640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:07.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.321+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fb0083120 con 0x7f0fb4071470 2023-10-26T05:50:07.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.321+0000 7f0fb9338640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:07.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.321+0000 7f0fb9338640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4001640 tx=0x7f0fa4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:07.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.325+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0fb00505b0 con 0x7f0fb4071470 2023-10-26T05:50:07.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.521+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fb40721e0 con 0x7f0fb4071470 2023-10-26T05:50:07.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.521+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/4027373989 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f0fb0053bd0 con 0x7f0fb4071470 2023-10-26T05:50:07.527 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:50:07.528 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:50:07.528 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:50:07.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 msgr2=0x7f0f98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:07.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4001640 tx=0x7f0fa4009040 comp rx=0 tx=0).stop 2023-10-26T05:50:07.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 msgr2=0x7f0fb415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:07.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb415b480 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0fb000c490 tx=0x7f0fb0001e80 comp rx=0 tx=0).stop 2023-10-26T05:50:07.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 shutdown_connections 2023-10-26T05:50:07.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb4140cc0 0x7f0fb4154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4071470 0x7f0fb415b480 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 --2- 172.21.15.71:0/4027373989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb4070fe0 0x7f0fb4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 >> 172.21.15.71:0/4027373989 conn(0x7f0fb4069770 msgr2=0x7f0fb40ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 shutdown_connections 2023-10-26T05:50:07.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:07.525+0000 7f0fbb59c640 1 -- 172.21.15.71:0/4027373989 wait complete. 2023-10-26T05:50:07.547 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:50:07.683 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":6264,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.401567418000001,"mdsmap_epoch":25,"osdmap_epoch":68,"osdmap_epoch_barrier":68,"uptime":15.091937250999999} 2023-10-26T05:50:07.684 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 6264, "mdsmap_epoch": 25, "osdmap_epoch": 68, "osdmap_epoch_barrier": 68, "rank_uptime": 10.401567418, "state": "up:active", "uptime": 15.091937251, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:50:07.685 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:50:07.812 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:50:07.813 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:50:07.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T05:50:07.940 INFO:teuthology.orchestra.run.smithi071.stdout:{"mon_tick_interval":"5"} 2023-10-26T05:50:07.941 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T05:50:07.941 INFO:tasks.cephfs.test_full:pool capacity 557028928, 531MB should be enough to fill it 2023-10-26T05:50:07.942 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:50:07.942 DEBUG:teuthology.orchestra.run.smithi071:> import time 2023-10-26T05:50:07.942 DEBUG:teuthology.orchestra.run.smithi071:> import datetime 2023-10-26T05:50:07.943 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:50:07.943 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:50:07.943 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.944 DEBUG:teuthology.orchestra.run.smithi071:> # Write some buffered data through before going full, all should be well 2023-10-26T05:50:07.944 DEBUG:teuthology.orchestra.run.smithi071:> print("writing some data through which we expect to succeed") 2023-10-26T05:50:07.944 DEBUG:teuthology.orchestra.run.smithi071:> bytes = 0 2023-10-26T05:50:07.945 DEBUG:teuthology.orchestra.run.smithi071:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T05:50:07.945 DEBUG:teuthology.orchestra.run.smithi071:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T05:50:07.945 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-26T05:50:07.946 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:50:07.946 DEBUG:teuthology.orchestra.run.smithi071:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T05:50:07.946 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.947 DEBUG:teuthology.orchestra.run.smithi071:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T05:50:07.947 DEBUG:teuthology.orchestra.run.smithi071:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T05:50:07.947 DEBUG:teuthology.orchestra.run.smithi071:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T05:50:07.948 DEBUG:teuthology.orchestra.run.smithi071:> # from write 2023-10-26T05:50:07.948 DEBUG:teuthology.orchestra.run.smithi071:> full = False 2023-10-26T05:50:07.948 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.949 DEBUG:teuthology.orchestra.run.smithi071:> for n in range(0, int(531 * 0.9)): 2023-10-26T05:50:07.949 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T05:50:07.949 DEBUG:teuthology.orchestra.run.smithi071:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-26T05:50:07.950 DEBUG:teuthology.orchestra.run.smithi071:> print("done writing {0} bytes".format(bytes)) 2023-10-26T05:50:07.950 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.950 DEBUG:teuthology.orchestra.run.smithi071:> # OK, now we should sneak in under the full condition 2023-10-26T05:50:07.951 DEBUG:teuthology.orchestra.run.smithi071:> # due to the time it takes the OSDs to report to the 2023-10-26T05:50:07.951 DEBUG:teuthology.orchestra.run.smithi071:> # mons, and get a successful fsync on our full-making data 2023-10-26T05:50:07.951 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:50:07.952 DEBUG:teuthology.orchestra.run.smithi071:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-26T05:50:07.952 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.952 DEBUG:teuthology.orchestra.run.smithi071:> # buffered write, add more dirty data to the buffer 2023-10-26T05:50:07.953 DEBUG:teuthology.orchestra.run.smithi071:> print("starting buffered write") 2023-10-26T05:50:07.953 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:50:07.953 DEBUG:teuthology.orchestra.run.smithi071:> for n in range(0, int(531 * 0.2)): 2023-10-26T05:50:07.954 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T05:50:07.954 DEBUG:teuthology.orchestra.run.smithi071:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T05:50:07.955 DEBUG:teuthology.orchestra.run.smithi071:> time.sleep(15.0) 2023-10-26T05:50:07.955 DEBUG:teuthology.orchestra.run.smithi071:> except OSError: 2023-10-26T05:50:07.955 DEBUG:teuthology.orchestra.run.smithi071:> pass; 2023-10-26T05:50:07.956 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.956 DEBUG:teuthology.orchestra.run.smithi071:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-26T05:50:07.956 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.957 DEBUG:teuthology.orchestra.run.smithi071:> # Wait long enough for a background flush that should fail 2023-10-26T05:50:07.957 DEBUG:teuthology.orchestra.run.smithi071:> time.sleep(30) 2023-10-26T05:50:07.957 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.958 DEBUG:teuthology.orchestra.run.smithi071:> if True: 2023-10-26T05:50:07.958 DEBUG:teuthology.orchestra.run.smithi071:> # ...and check that the failed background flush is reflected in fclose 2023-10-26T05:50:07.958 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:50:07.959 DEBUG:teuthology.orchestra.run.smithi071:> os.close(f) 2023-10-26T05:50:07.959 DEBUG:teuthology.orchestra.run.smithi071:> except OSError: 2023-10-26T05:50:07.959 DEBUG:teuthology.orchestra.run.smithi071:> print("close() returned an error as expected") 2023-10-26T05:50:07.960 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:50:07.960 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("close() failed to raise error") 2023-10-26T05:50:07.960 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:50:07.961 DEBUG:teuthology.orchestra.run.smithi071:> # The kernel cephfs client does not raise errors on fclose 2023-10-26T05:50:07.961 DEBUG:teuthology.orchestra.run.smithi071:> os.close(f) 2023-10-26T05:50:07.961 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:50:07.962 DEBUG:teuthology.orchestra.run.smithi071:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T05:50:07.962 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:50:13.239 INFO:teuthology.orchestra.run.smithi071.stdout:writing some data through which we expect to succeed 2023-10-26T05:50:13.240 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T05:50:13.240 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-26T05:50:13.240 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-26T05:50:13.241 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-26T05:50:13.241 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-26T05:50:13.241 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-26T05:50:13.241 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-26T05:50:13.242 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-26T05:50:13.242 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-26T05:50:13.242 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-26T05:50:13.242 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-26T05:50:13.242 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-26T05:50:13.243 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-26T05:50:13.243 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-26T05:50:13.243 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-26T05:50:13.243 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-26T05:50:13.243 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-26T05:50:13.244 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-26T05:50:13.244 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-26T05:50:13.244 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-26T05:50:13.244 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-26T05:50:13.244 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-26T05:50:13.245 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-26T05:50:13.245 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-26T05:50:13.245 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-26T05:50:13.245 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-26T05:50:13.245 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-26T05:50:13.246 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-26T05:50:13.246 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-26T05:50:13.246 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-10-26T05:50:13.246 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-10-26T05:50:13.247 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-10-26T05:50:13.247 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-10-26T05:50:13.247 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-10-26T05:50:13.247 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-10-26T05:50:13.247 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-10-26T05:50:13.248 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-10-26T05:50:13.248 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-10-26T05:50:13.248 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-10-26T05:50:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-10-26T05:50:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-10-26T05:50:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-10-26T05:50:13.249 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-10-26T05:50:13.250 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-10-26T05:50:13.250 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-10-26T05:50:13.250 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-10-26T05:50:13.250 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-10-26T05:50:13.250 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-10-26T05:50:13.251 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-10-26T05:50:13.251 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-10-26T05:50:13.251 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-10-26T05:50:13.251 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-10-26T05:50:13.251 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-10-26T05:50:13.252 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-10-26T05:50:13.252 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-10-26T05:50:13.252 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-10-26T05:50:13.252 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-10-26T05:50:13.252 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-10-26T05:50:13.253 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-10-26T05:50:13.253 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-10-26T05:50:13.253 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-10-26T05:50:13.253 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-10-26T05:50:13.253 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-10-26T05:50:13.254 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-10-26T05:50:13.254 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-10-26T05:50:13.254 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-10-26T05:50:13.254 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-10-26T05:50:13.254 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-10-26T05:50:13.255 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-10-26T05:50:13.255 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-10-26T05:50:13.255 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-10-26T05:50:13.255 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-10-26T05:50:13.255 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-10-26T05:50:13.256 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-10-26T05:50:13.256 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-10-26T05:50:13.256 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-10-26T05:50:13.256 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-10-26T05:50:13.257 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-10-26T05:50:13.257 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-10-26T05:50:13.257 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-10-26T05:50:13.257 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-10-26T05:50:13.258 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-10-26T05:50:13.258 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-10-26T05:50:13.258 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-10-26T05:50:13.259 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-10-26T05:50:13.259 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-10-26T05:50:13.259 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-10-26T05:50:13.259 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-10-26T05:50:13.259 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-10-26T05:50:13.260 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-10-26T05:50:13.260 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-10-26T05:50:13.260 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-10-26T05:50:13.260 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-10-26T05:50:13.260 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-10-26T05:50:13.261 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-10-26T05:50:13.261 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-10-26T05:50:13.261 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-10-26T05:50:13.261 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-10-26T05:50:13.261 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-10-26T05:50:13.261 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-10-26T05:50:13.262 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-10-26T05:50:13.262 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-10-26T05:50:13.262 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-10-26T05:50:13.262 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-10-26T05:50:13.262 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-10-26T05:50:13.263 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-10-26T05:50:13.263 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-10-26T05:50:13.263 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-10-26T05:50:13.263 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-10-26T05:50:13.263 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-10-26T05:50:13.264 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-10-26T05:50:13.264 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-10-26T05:50:13.264 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-10-26T05:50:13.264 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-10-26T05:50:13.264 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-10-26T05:50:13.265 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-10-26T05:50:13.265 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-10-26T05:50:13.265 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-10-26T05:50:13.265 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-10-26T05:50:13.266 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-10-26T05:50:13.266 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-10-26T05:50:13.266 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-10-26T05:50:13.266 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-10-26T05:50:13.266 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-10-26T05:50:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-10-26T05:50:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-10-26T05:50:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-10-26T05:50:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-10-26T05:50:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-10-26T05:50:13.268 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-10-26T05:50:13.268 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-10-26T05:50:13.268 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-10-26T05:50:13.268 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-10-26T05:50:13.268 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-10-26T05:50:13.269 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-10-26T05:50:13.269 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-10-26T05:50:13.269 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-10-26T05:50:13.269 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-10-26T05:50:13.270 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-10-26T05:50:13.270 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-10-26T05:50:13.270 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-10-26T05:50:13.270 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-10-26T05:50:13.270 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-10-26T05:50:13.271 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-10-26T05:50:13.271 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-10-26T05:50:13.271 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-10-26T05:50:13.271 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-10-26T05:50:13.271 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-10-26T05:50:13.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-10-26T05:50:13.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-10-26T05:50:13.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-10-26T05:50:13.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-10-26T05:50:13.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-10-26T05:50:13.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-10-26T05:50:13.273 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-10-26T05:50:13.575 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-10-26T05:50:13.576 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-10-26T05:50:13.576 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-10-26T05:50:13.576 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-10-26T05:50:13.576 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-10-26T05:50:13.577 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-10-26T05:50:13.577 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-10-26T05:50:13.577 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-10-26T05:50:13.577 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-10-26T05:50:13.577 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-10-26T05:50:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-10-26T05:50:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-10-26T05:50:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-10-26T05:50:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-10-26T05:50:13.578 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-10-26T05:50:13.579 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-10-26T05:50:13.579 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-10-26T05:50:13.579 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-10-26T05:50:13.579 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-10-26T05:50:13.579 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-10-26T05:50:13.580 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-10-26T05:50:13.580 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-10-26T05:50:13.580 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-10-26T05:50:13.580 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-10-26T05:50:13.580 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-10-26T05:50:13.581 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-10-26T05:50:13.581 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-10-26T05:50:13.581 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-10-26T05:50:13.581 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-10-26T05:50:13.581 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-10-26T05:50:13.582 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-10-26T05:50:13.582 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-10-26T05:50:13.582 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-10-26T05:50:13.582 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-10-26T05:50:13.582 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-10-26T05:50:13.583 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-10-26T05:50:13.583 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-10-26T05:50:13.583 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-10-26T05:50:13.583 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-10-26T05:50:13.583 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-10-26T05:50:13.583 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-10-26T05:50:13.584 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-10-26T05:50:13.584 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-10-26T05:50:13.584 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-10-26T05:50:13.584 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-10-26T05:50:13.584 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-10-26T05:50:13.585 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-10-26T05:50:13.585 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-10-26T05:50:13.585 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-10-26T05:50:13.585 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-10-26T05:50:13.585 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-10-26T05:50:13.586 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-10-26T05:50:13.586 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-10-26T05:50:13.586 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-10-26T05:50:13.586 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-10-26T05:50:13.586 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-10-26T05:50:13.587 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-10-26T05:50:13.587 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-10-26T05:50:13.587 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-10-26T05:50:13.587 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-10-26T05:50:13.587 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-10-26T05:50:13.587 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-10-26T05:50:13.588 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-10-26T05:50:13.588 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-10-26T05:50:13.588 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-10-26T05:50:13.588 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-10-26T05:50:13.588 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-10-26T05:50:13.589 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-10-26T05:50:13.589 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-10-26T05:50:13.589 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-10-26T05:50:13.589 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-10-26T05:50:13.589 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-10-26T05:50:13.589 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-10-26T05:50:13.590 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-10-26T05:50:13.590 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-10-26T05:50:13.590 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-10-26T05:50:13.590 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-10-26T05:50:13.590 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-10-26T05:50:13.591 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-10-26T05:50:13.591 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-10-26T05:50:13.591 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-10-26T05:50:13.591 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-10-26T05:50:13.591 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-10-26T05:50:13.592 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-10-26T05:50:13.592 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-10-26T05:50:13.592 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-10-26T05:50:13.592 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-10-26T05:50:13.592 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-10-26T05:50:13.593 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-10-26T05:50:13.593 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-10-26T05:50:13.593 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-10-26T05:50:13.593 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-10-26T05:50:13.593 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-10-26T05:50:13.594 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-10-26T05:50:13.594 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-10-26T05:50:13.594 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-10-26T05:50:13.594 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-10-26T05:50:13.594 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-10-26T05:50:13.595 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-10-26T05:50:13.595 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-10-26T05:50:13.595 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-10-26T05:50:13.595 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-10-26T05:50:13.595 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-10-26T05:50:13.596 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-10-26T05:50:13.596 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-10-26T05:50:13.596 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-10-26T05:50:13.596 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-10-26T05:50:13.596 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-10-26T05:50:13.596 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-10-26T05:50:13.597 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-10-26T05:50:13.597 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-10-26T05:50:13.597 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-10-26T05:50:13.597 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-10-26T05:50:13.597 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-10-26T05:50:13.598 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-10-26T05:50:13.598 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-10-26T05:50:13.598 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-10-26T05:50:13.598 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-10-26T05:50:13.598 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-10-26T05:50:13.599 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-10-26T05:50:13.599 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-10-26T05:50:13.599 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-10-26T05:50:13.599 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-10-26T05:50:13.599 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-10-26T05:50:13.599 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-10-26T05:50:13.600 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-10-26T05:50:13.600 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-10-26T05:50:13.600 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-10-26T05:50:13.600 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-10-26T05:50:13.600 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-10-26T05:50:13.601 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-10-26T05:50:13.601 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-10-26T05:50:13.601 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-10-26T05:50:13.601 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-10-26T05:50:13.601 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-10-26T05:50:13.602 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-10-26T05:50:13.602 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-10-26T05:50:13.602 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-10-26T05:50:13.602 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-10-26T05:50:13.602 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-10-26T05:50:13.602 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-10-26T05:50:13.603 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-10-26T05:50:13.603 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-10-26T05:50:13.603 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-10-26T05:50:13.603 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-10-26T05:50:13.603 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-10-26T05:50:13.604 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-10-26T05:50:13.604 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-10-26T05:50:13.604 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-10-26T05:50:13.604 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-10-26T05:50:13.604 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-10-26T05:50:13.605 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-10-26T05:50:13.605 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-10-26T05:50:13.605 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-10-26T05:50:13.841 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-10-26T05:50:13.841 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-10-26T05:50:13.841 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-10-26T05:50:13.841 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-10-26T05:50:13.842 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-10-26T05:50:13.842 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-10-26T05:50:13.842 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-10-26T05:50:13.842 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-10-26T05:50:13.842 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-10-26T05:50:13.843 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-10-26T05:50:13.843 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-10-26T05:50:13.843 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-10-26T05:50:13.843 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-10-26T05:50:13.843 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-10-26T05:50:13.844 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-10-26T05:50:13.844 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-10-26T05:50:13.844 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-10-26T05:50:13.844 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-10-26T05:50:13.844 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-10-26T05:50:13.845 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-10-26T05:50:13.845 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-10-26T05:50:13.845 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-10-26T05:50:13.845 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-10-26T05:50:13.846 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-10-26T05:50:13.846 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-10-26T05:50:13.846 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-10-26T05:50:13.846 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-10-26T05:50:13.846 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-10-26T05:50:13.847 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-10-26T05:50:13.847 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-10-26T05:50:13.847 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-10-26T05:50:13.847 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-10-26T05:50:13.847 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-10-26T05:50:13.848 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-10-26T05:50:13.848 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-10-26T05:50:13.848 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-10-26T05:50:13.848 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-10-26T05:50:13.848 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-10-26T05:50:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-10-26T05:50:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-10-26T05:50:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-10-26T05:50:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-10-26T05:50:13.849 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-10-26T05:50:13.850 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-10-26T05:50:13.850 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-10-26T05:50:13.850 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-10-26T05:50:13.850 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-10-26T05:50:13.851 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-10-26T05:50:13.851 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-10-26T05:50:13.851 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-10-26T05:50:13.851 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-10-26T05:50:13.851 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-10-26T05:50:13.851 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-10-26T05:50:13.852 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-10-26T05:50:13.852 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-10-26T05:50:13.852 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-10-26T05:50:13.852 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-10-26T05:50:13.852 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-10-26T05:50:13.853 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-10-26T05:50:13.853 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-10-26T05:50:13.853 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-10-26T05:50:13.853 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-10-26T05:50:13.853 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-10-26T05:50:13.854 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-10-26T05:50:13.854 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-10-26T05:50:13.854 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-10-26T05:50:13.854 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-10-26T05:50:13.854 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-10-26T05:50:13.855 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-10-26T05:50:13.855 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-10-26T05:50:13.855 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-10-26T05:50:13.855 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-10-26T05:50:13.855 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-10-26T05:50:13.856 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-10-26T05:50:13.856 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-10-26T05:50:13.856 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-10-26T05:50:13.856 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-10-26T05:50:13.856 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-10-26T05:50:13.856 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-10-26T05:50:13.857 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-10-26T05:50:13.857 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-10-26T05:50:13.857 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-10-26T05:50:13.857 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-10-26T05:50:13.857 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-10-26T05:50:13.858 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-10-26T05:50:13.858 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-10-26T05:50:13.858 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-10-26T05:50:13.858 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-10-26T05:50:13.858 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-10-26T05:50:13.859 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-10-26T05:50:13.859 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-10-26T05:50:13.859 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-10-26T05:50:13.859 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-10-26T05:50:13.859 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-10-26T05:50:13.860 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-10-26T05:50:13.860 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-10-26T05:50:13.860 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-10-26T05:50:13.860 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-10-26T05:50:13.860 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-10-26T05:50:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-10-26T05:50:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-10-26T05:50:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-10-26T05:50:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-10-26T05:50:13.861 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-10-26T05:50:13.862 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-10-26T05:50:13.862 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-10-26T05:50:13.862 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-10-26T05:50:13.862 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-10-26T05:50:13.862 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-10-26T05:50:13.862 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-10-26T05:50:13.863 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-10-26T05:50:13.863 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-10-26T05:50:13.863 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-10-26T05:50:13.863 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-10-26T05:50:13.864 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-10-26T05:50:13.864 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-10-26T05:50:13.864 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-10-26T05:50:13.864 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-10-26T05:50:13.864 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-10-26T05:50:13.865 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-10-26T05:50:13.865 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-10-26T05:50:13.865 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-10-26T05:50:13.865 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-10-26T05:50:13.865 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-10-26T05:50:13.866 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-10-26T05:50:13.866 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-10-26T05:50:13.866 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-10-26T05:50:13.866 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-10-26T05:50:13.866 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-10-26T05:50:13.866 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-10-26T05:50:13.867 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-10-26T05:50:13.867 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-10-26T05:50:13.867 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-10-26T05:50:13.867 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-10-26T05:50:13.867 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-10-26T05:50:13.868 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-10-26T05:50:13.868 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-10-26T05:50:13.868 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-10-26T05:50:13.868 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-10-26T05:50:13.868 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-10-26T05:50:13.869 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-10-26T05:50:13.869 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-10-26T05:50:13.869 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-10-26T05:50:13.869 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-10-26T05:50:13.869 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-10-26T05:50:13.869 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-10-26T05:50:13.870 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-10-26T05:50:13.870 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-10-26T05:50:13.870 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-10-26T05:50:13.870 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-10-26T05:50:13.870 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-10-26T05:50:13.871 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-10-26T05:50:13.871 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-10-26T05:50:13.871 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-10-26T05:50:15.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:50:15.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:50:15.186 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:50:15.181+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-10-26T05:50:20.323 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:50:20.317+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T05:50:45.160 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:50:45.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:50:59.004 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-10-26T05:50:59.004 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-10-26T05:50:59.004 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-10-26T05:50:59.004 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-10-26T05:50:59.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-10-26T05:50:59.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-10-26T05:50:59.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-10-26T05:50:59.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-10-26T05:50:59.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-10-26T05:50:59.006 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-10-26T05:50:59.006 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-10-26T05:50:59.006 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-10-26T05:50:59.006 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-10-26T05:50:59.006 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-10-26T05:50:59.007 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-10-26T05:50:59.007 INFO:teuthology.orchestra.run.smithi071.stdout:done writing 500695040 bytes 2023-10-26T05:50:59.007 INFO:teuthology.orchestra.run.smithi071.stdout:successfully fsync'ed prior to getting full state reported 2023-10-26T05:50:59.007 INFO:teuthology.orchestra.run.smithi071.stdout:starting buffered write 2023-10-26T05:50:59.007 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:50:59.008 INFO:teuthology.orchestra.run.smithi071.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-26T05:50:59.008 INFO:teuthology.orchestra.run.smithi071.stdout:close() returned an error as expected 2023-10-26T05:50:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:50:59.019 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:50:59.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:50:59.078 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:50:59.120 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:50:59.120 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:50:59.121 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:50:59.127 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:50:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3604802204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 msgr2=0x7fc2a40ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3604802204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a40ac740 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fc294009900 tx=0x7fc29401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3604802204 shutdown_connections 2023-10-26T05:50:59.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3604802204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a413d800 0x7fc2a415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3604802204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a40ac740 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3604802204 >> 172.21.15.71:0/3604802204 conn(0x7fc2a4069770 msgr2=0x7fc2a4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3604802204 shutdown_connections 2023-10-26T05:50:59.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3604802204 wait complete. 2023-10-26T05:50:59.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 Processor -- start 2023-10-26T05:50:59.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- start start 2023-10-26T05:50:59.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a413d800 0x7fc2a4150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a413dc20 0x7fc2a41390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a41f1570 con 0x7fc2a413d420 2023-10-26T05:50:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4151ae0 con 0x7fc2a413d800 2023-10-26T05:50:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a41f1780 con 0x7fc2a413dc20 2023-10-26T05:50:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a415a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44722/0 (socket says 172.21.15.71:44722) 2023-10-26T05:50:59.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 -- 172.21.15.71:0/3298470311 learned_addr learned my addr 172.21.15.71:0/3298470311 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:59.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a27fc640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a413d800 0x7fc2a4150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a37fe640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a413dc20 0x7fc2a41390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 -- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a413dc20 msgr2=0x7fc2a41390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a413dc20 0x7fc2a41390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 -- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a413d800 msgr2=0x7fc2a4150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a413d800 0x7fc2a4150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 -- 172.21.15.71:0/3298470311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294009560 con 0x7fc2a413d420 2023-10-26T05:50:59.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a2ffd640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a415a4e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc2940b0040 tx=0x7fc2940a66f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:59.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2940bc070 con 0x7fc2a413d420 2023-10-26T05:50:59.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc2940041b0 con 0x7fc2a413d420 2023-10-26T05:50:59.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294005030 con 0x7fc2a413d420 2023-10-26T05:50:59.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a41f1a60 con 0x7fc2a413d420 2023-10-26T05:50:59.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.305+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a41f2020 con 0x7fc2a413d420 2023-10-26T05:50:59.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a4066b10 con 0x7fc2a413d420 2023-10-26T05:50:59.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc294005190 con 0x7fc2a413d420 2023-10-26T05:50:59.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc28b7fe640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc290066470 0x7fc290068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2940aa070 con 0x7fc2a413d420 2023-10-26T05:50:59.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fc29006dfa0 con 0x7fc2a413d420 2023-10-26T05:50:59.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc2a27fc640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc290066470 0x7fc290068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.309+0000 7fc2a27fc640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc290066470 0x7fc290068930 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc2980058d0 tx=0x7fc298005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:59.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.313+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc294120e60 con 0x7fc2a413d420 2023-10-26T05:50:59.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.461+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2a40721e0 con 0x7fc2a413d420 2023-10-26T05:50:59.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc28b7fe640 1 -- 172.21.15.71:0/3298470311 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16539 (secure 0 0 0) 0x7fc2940ec000 con 0x7fc2a413d420 2023-10-26T05:50:59.470 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:50:59.470 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":73,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:50:20.321590+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:50:59.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc290066470 msgr2=0x7fc290068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc290066470 0x7fc290068930 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc2980058d0 tx=0x7fc298005860 comp rx=0 tx=0).stop 2023-10-26T05:50:59.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 msgr2=0x7fc2a415a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a415a4e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc2940b0040 tx=0x7fc2940a66f0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 shutdown_connections 2023-10-26T05:50:59.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc2a413dc20 0x7fc2a41390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2a413d800 0x7fc2a4150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc290066470 0x7fc290068930 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 --2- 172.21.15.71:0/3298470311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2a413d420 0x7fc2a415a4e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 >> 172.21.15.71:0/3298470311 conn(0x7fc2a4069770 msgr2=0x7fc2a413ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:59.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 shutdown_connections 2023-10-26T05:50:59.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.465+0000 7fc2a9428640 1 -- 172.21.15.71:0/3298470311 wait complete. 2023-10-26T05:50:59.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:49:54.856291+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:49:55.604098+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:50:59.488 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:50:59.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.597+0000 7fa9caf50640 1 -- 172.21.15.71:0/3371555632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40d0470 msgr2=0x7fa9c406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.597+0000 7fa9caf50640 1 --2- 172.21.15.71:0/3371555632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40d0470 0x7fa9c406aa70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009c80 tx=0x7fa9bc01cc80 comp rx=0 tx=0).stop 2023-10-26T05:50:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.597+0000 7fa9caf50640 1 -- 172.21.15.71:0/3371555632 shutdown_connections 2023-10-26T05:50:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.597+0000 7fa9caf50640 1 --2- 172.21.15.71:0/3371555632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40d0470 0x7fa9c406aa70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.597+0000 7fa9caf50640 1 -- 172.21.15.71:0/3371555632 >> 172.21.15.71:0/3371555632 conn(0x7fa9c40cdd80 msgr2=0x7fa9c40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.597+0000 7fa9caf50640 1 -- 172.21.15.71:0/3371555632 shutdown_connections 2023-10-26T05:50:59.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- 172.21.15.71:0/3371555632 wait complete. 2023-10-26T05:50:59.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 Processor -- start 2023-10-26T05:50:59.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- start start 2023-10-26T05:50:59.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 0x7fa9c40c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9c40d0470 0x7fa9c414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c40d0850 0x7fa9c40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c406a650 con 0x7fa9c40cd880 2023-10-26T05:50:59.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40ca5d0 con 0x7fa9c40d0850 2023-10-26T05:50:59.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c40c99c0 con 0x7fa9c40d0470 2023-10-26T05:50:59.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 0x7fa9c40c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 0x7fa9c40c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44738/0 (socket says 172.21.15.71:44738) 2023-10-26T05:50:59.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 -- 172.21.15.71:0/653222309 learned_addr learned my addr 172.21.15.71:0/653222309 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:59.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c3fff640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9c40d0470 0x7fa9c414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c94ed640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c40d0850 0x7fa9c40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 -- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9c40d0470 msgr2=0x7fa9c414cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9c40d0470 0x7fa9c414cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 -- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c40d0850 msgr2=0x7fa9c40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c40d0850 0x7fa9c40c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 -- 172.21.15.71:0/653222309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0098e0 con 0x7fa9c40cd880 2023-10-26T05:50:59.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c3fff640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9c40d0470 0x7fa9c414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:50:59.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c8cec640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 0x7fa9c40c90c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc000f90 tx=0x7fa9bc0a5bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:59.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc004410 con 0x7fa9c40cd880 2023-10-26T05:50:59.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa9bc004570 con 0x7fa9c40cd880 2023-10-26T05:50:59.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c40c9ca0 con 0x7fa9c40cd880 2023-10-26T05:50:59.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0ad5b0 con 0x7fa9c40cd880 2023-10-26T05:50:59.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c40ca260 con 0x7fa9c40cd880 2023-10-26T05:50:59.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.601+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c4066b10 con 0x7fa9c40cd880 2023-10-26T05:50:59.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.605+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa9bc0b4040 con 0x7fa9c40cd880 2023-10-26T05:50:59.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.605+0000 7fa9c17fa640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9a4066470 0x7fa9a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.605+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9bc11f030 con 0x7fa9c40cd880 2023-10-26T05:50:59.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.605+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fa9a406dfa0 con 0x7fa9c40cd880 2023-10-26T05:50:59.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.605+0000 7fa9c3fff640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9a4066470 0x7fa9a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.605+0000 7fa9c3fff640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9a4066470 0x7fa9a4068930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40b6790 tx=0x7fa9ac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:59.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.609+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa9bc0b2020 con 0x7fa9c40cd880 2023-10-26T05:50:59.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.749+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9c4066b10 con 0x7fa9c40cd880 2023-10-26T05:50:59.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.749+0000 7fa9c17fa640 1 -- 172.21.15.71:0/653222309 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9bc0ec4b0 con 0x7fa9c40cd880 2023-10-26T05:50:59.756 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:50:59.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9a4066470 msgr2=0x7fa9a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9a4066470 0x7fa9a4068930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40b6790 tx=0x7fa9ac008040 comp rx=0 tx=0).stop 2023-10-26T05:50:59.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 msgr2=0x7fa9c40c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 0x7fa9c40c90c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc000f90 tx=0x7fa9bc0a5bd0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 shutdown_connections 2023-10-26T05:50:59.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9c40d0850 0x7fa9c40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9c40d0470 0x7fa9c414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9a4066470 0x7fa9a4068930 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 --2- 172.21.15.71:0/653222309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9c40cd880 0x7fa9c40c90c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 >> 172.21.15.71:0/653222309 conn(0x7fa9c40cdd80 msgr2=0x7fa9c40cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:59.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 shutdown_connections 2023-10-26T05:50:59.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.753+0000 7fa9caf50640 1 -- 172.21.15.71:0/653222309 wait complete. 2023-10-26T05:50:59.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:50:59.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 -- 172.21.15.71:0/3876635535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 msgr2=0x7f4b90003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 --2- 172.21.15.71:0/3876635535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4b90003da0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400c260 tx=0x7f4b9401cf90 comp rx=0 tx=0).stop 2023-10-26T05:50:59.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 -- 172.21.15.71:0/3876635535 shutdown_connections 2023-10-26T05:50:59.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 --2- 172.21.15.71:0/3876635535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4b90003da0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 -- 172.21.15.71:0/3876635535 >> 172.21.15.71:0/3876635535 conn(0x7f4ba40aec60 msgr2=0x7f4ba40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:50:59.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 -- 172.21.15.71:0/3876635535 shutdown_connections 2023-10-26T05:50:59.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 -- 172.21.15.71:0/3876635535 wait complete. 2023-10-26T05:50:59.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 Processor -- start 2023-10-26T05:50:59.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.897+0000 7f4baa916640 1 -- start start 2023-10-26T05:50:59.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4ba4137590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba41359e0 0x7f4ba415c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ba4138660 0x7f4ba4138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4137170 con 0x7f4b90003980 2023-10-26T05:50:59.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4139020 con 0x7f4ba41359e0 2023-10-26T05:50:59.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4135e60 con 0x7f4ba4138660 2023-10-26T05:50:59.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4ba4137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4ba4137590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44766/0 (socket says 172.21.15.71:44766) 2023-10-26T05:50:59.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 -- 172.21.15.71:0/2983125682 learned_addr learned my addr 172.21.15.71:0/2983125682 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:50:59.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba37fe640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba41359e0 0x7f4ba415c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba8eb3640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ba4138660 0x7f4ba4138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 -- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ba4138660 msgr2=0x7f4ba4138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ba4138660 0x7f4ba4138ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 -- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba41359e0 msgr2=0x7f4ba415c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:50:59.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba41359e0 0x7f4ba415c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:50:59.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 -- 172.21.15.71:0/2983125682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba4136210 con 0x7f4b90003980 2023-10-26T05:50:59.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba3fff640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4ba4137590 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4b9401cdc0 tx=0x7f4b94001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:59.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b940b3070 con 0x7f4b90003980 2023-10-26T05:50:59.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b940017c0 con 0x7f4b90003980 2023-10-26T05:50:59.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba41364f0 con 0x7f4b90003980 2023-10-26T05:50:59.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94006420 con 0x7f4b90003980 2023-10-26T05:50:59.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba4136ab0 con 0x7f4b90003980 2023-10-26T05:50:59.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba4066b10 con 0x7f4b90003980 2023-10-26T05:50:59.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b940ab020 con 0x7f4b90003980 2023-10-26T05:50:59.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b840664a0 0x7f4b84068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:50:59.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b9411e440 con 0x7f4b90003980 2023-10-26T05:50:59.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f4b8406dfd0 con 0x7f4b90003980 2023-10-26T05:50:59.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba37fe640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b840664a0 0x7f4b84068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:50:59.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.901+0000 7f4ba37fe640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b840664a0 0x7f4b84068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40ca420 tx=0x7f4b8c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:50:59.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:50:59.905+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4b940eb8a0 con 0x7f4b90003980 2023-10-26T05:51:00.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.045+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ba40721e0 con 0x7f4b90003980 2023-10-26T05:51:00.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.049+0000 7f4ba0ff9640 1 -- 172.21.15.71:0/2983125682 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b940eeec0 con 0x7f4b90003980 2023-10-26T05:51:00.053 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:51:00.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b840664a0 msgr2=0x7f4b84068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b840664a0 0x7f4b84068960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40ca420 tx=0x7f4b8c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 msgr2=0x7f4ba4137590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4ba4137590 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f4b9401cdc0 tx=0x7f4b94001510 comp rx=0 tx=0).stop 2023-10-26T05:51:00.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 shutdown_connections 2023-10-26T05:51:00.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ba4138660 0x7f4ba4138ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ba41359e0 0x7f4ba415c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b840664a0 0x7f4b84068960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 --2- 172.21.15.71:0/2983125682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b90003980 0x7f4ba4137590 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 >> 172.21.15.71:0/2983125682 conn(0x7f4ba40aec60 msgr2=0x7f4ba40ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:00.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 shutdown_connections 2023-10-26T05:51:00.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.053+0000 7f4baa916640 1 -- 172.21.15.71:0/2983125682 wait complete. 2023-10-26T05:51:00.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:51:00.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 -- 172.21.15.71:0/1639701891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc59c0047d0 msgr2=0x7fc59c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/1639701891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc59c0047d0 0x7fc59c004c50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c700 tx=0x7fc59801c660 comp rx=0 tx=0).stop 2023-10-26T05:51:00.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 -- 172.21.15.71:0/1639701891 shutdown_connections 2023-10-26T05:51:00.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/1639701891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc59c0047d0 0x7fc59c004c50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 -- 172.21.15.71:0/1639701891 >> 172.21.15.71:0/1639701891 conn(0x7fc5b00ce070 msgr2=0x7fc5b00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:00.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 -- 172.21.15.71:0/1639701891 shutdown_connections 2023-10-26T05:51:00.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.209+0000 7fc5b4efa640 1 -- 172.21.15.71:0/1639701891 wait complete. 2023-10-26T05:51:00.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 Processor -- start 2023-10-26T05:51:00.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- start start 2023-10-26T05:51:00.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c0047d0 0x7fc5b015bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 0x7fc5b0159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5b0149ac0 0x7fc5b015a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b015a9a0 con 0x7fc5b00c7910 2023-10-26T05:51:00.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00c7e40 con 0x7fc59c0047d0 2023-10-26T05:51:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b0159440 con 0x7fc5b0149ac0 2023-10-26T05:51:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 0x7fc5b0159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 0x7fc5b0159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44786/0 (socket says 172.21.15.71:44786) 2023-10-26T05:51:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 -- 172.21.15.71:0/2451673877 learned_addr learned my addr 172.21.15.71:0/2451673877 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:00.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5ae59c640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c0047d0 0x7fc5b015bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5aed9d640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5b0149ac0 0x7fc5b015a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 -- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5b0149ac0 msgr2=0x7fc5b015a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5b0149ac0 0x7fc5b015a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 -- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c0047d0 msgr2=0x7fc5b015bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c0047d0 0x7fc5b015bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 -- 172.21.15.71:0/2451673877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0159720 con 0x7fc5b00c7910 2023-10-26T05:51:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5ae59c640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c0047d0 0x7fc5b015bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5add9b640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 0x7fc5b0159f20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc59801c490 tx=0x7fc598004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598006330 con 0x7fc5b00c7910 2023-10-26T05:51:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc598004e60 con 0x7fc5b00c7910 2023-10-26T05:51:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b0159a00 con 0x7fc5b00c7910 2023-10-26T05:51:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5980a96c0 con 0x7fc5b00c7910 2023-10-26T05:51:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b015e850 con 0x7fc5b00c7910 2023-10-26T05:51:00.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b0159b90 con 0x7fc5b00c7910 2023-10-26T05:51:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc5980a9820 con 0x7fc5b00c7910 2023-10-26T05:51:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc590066470 0x7fc590068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc59811f740 con 0x7fc5b00c7910 2023-10-26T05:51:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.213+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fc59006dfa0 con 0x7fc5b00c7910 2023-10-26T05:51:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.217+0000 7fc5ae59c640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc590066470 0x7fc590068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.217+0000 7fc5ae59c640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc590066470 0x7fc590068930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001b30 tx=0x7fc5a00014e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:00.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.217+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc5980ae060 con 0x7fc5b00c7910 2023-10-26T05:51:00.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.373+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5b00b6280 con 0x7fc5b00c7910 2023-10-26T05:51:00.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.373+0000 7fc5a6ffd640 1 -- 172.21.15.71:0/2451673877 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5980ecc50 con 0x7fc5b00c7910 2023-10-26T05:51:00.379 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:51:00.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc590066470 msgr2=0x7fc590068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc590066470 0x7fc590068930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001b30 tx=0x7fc5a00014e0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 msgr2=0x7fc5b0159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 0x7fc5b0159f20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc59801c490 tx=0x7fc598004c70 comp rx=0 tx=0).stop 2023-10-26T05:51:00.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 shutdown_connections 2023-10-26T05:51:00.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5b0149ac0 0x7fc5b015a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc590066470 0x7fc590068930 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5b00c7910 0x7fc5b0159f20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 --2- 172.21.15.71:0/2451673877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c0047d0 0x7fc5b015bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 >> 172.21.15.71:0/2451673877 conn(0x7fc5b00ce070 msgr2=0x7fc5b014ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:00.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 shutdown_connections 2023-10-26T05:51:00.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.377+0000 7fc5b4efa640 1 -- 172.21.15.71:0/2451673877 wait complete. 2023-10-26T05:51:00.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:51:00.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 -- 172.21.15.71:0/1388007177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 msgr2=0x7f72080b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 --2- 172.21.15.71:0/1388007177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f72080b1300 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009900 tx=0x7f71fc01c7b0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 -- 172.21.15.71:0/1388007177 shutdown_connections 2023-10-26T05:51:00.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 --2- 172.21.15.71:0/1388007177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f72080b1300 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 --2- 172.21.15.71:0/1388007177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 0x7f7208159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 -- 172.21.15.71:0/1388007177 >> 172.21.15.71:0/1388007177 conn(0x7f720806b8c0 msgr2=0x7f720806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:00.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 -- 172.21.15.71:0/1388007177 shutdown_connections 2023-10-26T05:51:00.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 -- 172.21.15.71:0/1388007177 wait complete. 2023-10-26T05:51:00.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 Processor -- start 2023-10-26T05:51:00.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.521+0000 7f720c89b640 1 -- start start 2023-10-26T05:51:00.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 0x7f720813e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f7208162110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720815ccf0 0x7f7208162650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7208163890 con 0x7f72080cabd0 2023-10-26T05:51:00.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720815ba60 con 0x7f720815ccf0 2023-10-26T05:51:00.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7208162c80 con 0x7f72080ca7f0 2023-10-26T05:51:00.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f7208162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f7208162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44810/0 (socket says 172.21.15.71:44810) 2023-10-26T05:51:00.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 -- 172.21.15.71:0/3452357003 learned_addr learned my addr 172.21.15.71:0/3452357003 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:00.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720659c640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 0x7f720813e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7206d9d640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720815ccf0 0x7f7208162650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 -- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 msgr2=0x7f720813e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 0x7f720813e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 -- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720815ccf0 msgr2=0x7f7208162650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720815ccf0 0x7f7208162650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 -- 172.21.15.71:0/3452357003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009560 con 0x7f72080cabd0 2023-10-26T05:51:00.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720659c640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 0x7f720813e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:00.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7206d9d640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720815ccf0 0x7f7208162650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:00.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f7205d9b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f7208162110 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f71fc01ae60 tx=0x7f71fc0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:00.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004380 con 0x7f72080cabd0 2023-10-26T05:51:00.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f71fc005020 con 0x7f72080cabd0 2023-10-26T05:51:00.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7208162f60 con 0x7f72080cabd0 2023-10-26T05:51:00.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc005430 con 0x7f72080cabd0 2023-10-26T05:51:00.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7208163520 con 0x7f72080cabd0 2023-10-26T05:51:00.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208066b10 con 0x7f72080cabd0 2023-10-26T05:51:00.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f71fc0b4040 con 0x7f72080cabd0 2023-10-26T05:51:00.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71fc11dda0 con 0x7f72080cabd0 2023-10-26T05:51:00.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f71e806dfa0 con 0x7f72080cabd0 2023-10-26T05:51:00.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.525+0000 7f720659c640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.529+0000 7f720659c640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005fd0 tx=0x7f71f8005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:00.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.529+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f71fc0eb220 con 0x7f72080cabd0 2023-10-26T05:51:00.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.685+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f720806c7d0 con 0x7f72080cabd0 2023-10-26T05:51:00.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.685+0000 7f71f6ffd640 1 -- 172.21.15.71:0/3452357003 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) v1 ==== 80+0+0 (secure 0 0 0) 0x7f71fc0b2020 con 0x7f72080cabd0 2023-10-26T05:51:00.691 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:51:00.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 msgr2=0x7f71e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f71f8005fd0 tx=0x7f71f8005e60 comp rx=0 tx=0).stop 2023-10-26T05:51:00.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 msgr2=0x7f7208162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f7208162110 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f71fc01ae60 tx=0x7f71fc0a57d0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 shutdown_connections 2023-10-26T05:51:00.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f720815ccf0 0x7f7208162650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72080cabd0 0x7f7208162110 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 --2- 172.21.15.71:0/3452357003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72080ca7f0 0x7f720813e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 >> 172.21.15.71:0/3452357003 conn(0x7f720806b8c0 msgr2=0x7f7208141c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:00.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 shutdown_connections 2023-10-26T05:51:00.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.689+0000 7f720c89b640 1 -- 172.21.15.71:0/3452357003 wait complete. 2023-10-26T05:51:00.708 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:51:00.708 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:00.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:00.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:00.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[42323]: fuse finished with error 0 and tester_r 0 2023-10-26T05:51:00.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:00.763 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:51:00.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:00.764 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:51:00.770 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:00.778 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:51:00.778 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:00.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:00.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- 172.21.15.71:0/3699217425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb864003980 msgr2=0x7fb864003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 --2- 172.21.15.71:0/3699217425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb864003980 0x7fb864003da0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb874009b60 tx=0x7fb87401d1d0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87d9a9640 1 -- 172.21.15.71:0/3699217425 reap_dead start 2023-10-26T05:51:00.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- 172.21.15.71:0/3699217425 shutdown_connections 2023-10-26T05:51:00.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- 172.21.15.71:0/3699217425 >> 172.21.15.71:0/3699217425 conn(0x7fb878070170 msgr2=0x7fb878070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:00.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- 172.21.15.71:0/3699217425 shutdown_connections 2023-10-26T05:51:00.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- 172.21.15.71:0/3699217425 wait complete. 2023-10-26T05:51:00.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 Processor -- start 2023-10-26T05:51:00.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- start start 2023-10-26T05:51:00.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb864003980 0x7fb8780ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 0x7fb878131570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb878157cd0 0x7fb878162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8780ad960 con 0x7fb878157850 2023-10-26T05:51:00.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb878135dc0 con 0x7fb878157cd0 2023-10-26T05:51:00.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87f40c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb878135490 con 0x7fb864003980 2023-10-26T05:51:00.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 0x7fb878131570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 0x7fb878131570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44828/0 (socket says 172.21.15.71:44828) 2023-10-26T05:51:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 -- 172.21.15.71:0/1665149196 learned_addr learned my addr 172.21.15.71:0/1665149196 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 -- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb864003980 msgr2=0x7fb8780ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb864003980 0x7fb8780ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 -- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb878157cd0 msgr2=0x7fb878162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb878157cd0 0x7fb878162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 -- 172.21.15.71:0/1665149196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878135770 con 0x7fb878157850 2023-10-26T05:51:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb87c9a7640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 0x7fb878131570 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb86800d980 tx=0x7fb86800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.925+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868009bd0 con 0x7fb878157850 2023-10-26T05:51:00.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb868009e20 con 0x7fb878157850 2023-10-26T05:51:00.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb878135a50 con 0x7fb878157850 2023-10-26T05:51:00.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868004290 con 0x7fb878157850 2023-10-26T05:51:00.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb878132010 con 0x7fb878157850 2023-10-26T05:51:00.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb878066d00 con 0x7fb878157850 2023-10-26T05:51:00.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb868003600 con 0x7fb878157850 2023-10-26T05:51:00.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb86dffb640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb85c0664a0 0x7fb85c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:00.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb868084420 con 0x7fb878157850 2023-10-26T05:51:00.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fb85c06dfd0 con 0x7fb878157850 2023-10-26T05:51:00.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb87d1a8640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb85c0664a0 0x7fb85c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.929+0000 7fb87d1a8640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb85c0664a0 0x7fb85c068960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8740ae000 tx=0x7fb864005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:00.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:00.933+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb86801f060 con 0x7fb878157850 2023-10-26T05:51:01.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.085+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb878135f00 con 0x7fb878157850 2023-10-26T05:51:01.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb86dffb640 1 -- 172.21.15.71:0/1665149196 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16539 (secure 0 0 0) 0x7fb8680518a0 con 0x7fb878157850 2023-10-26T05:51:01.093 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:01.094 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":73,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:50:20.321590+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:01.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb85c0664a0 msgr2=0x7fb85c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb85c0664a0 0x7fb85c068960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8740ae000 tx=0x7fb864005800 comp rx=0 tx=0).stop 2023-10-26T05:51:01.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 msgr2=0x7fb878131570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 0x7fb878131570 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb86800d980 tx=0x7fb86800de40 comp rx=0 tx=0).stop 2023-10-26T05:51:01.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 shutdown_connections 2023-10-26T05:51:01.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb878157cd0 0x7fb878162300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb85c0664a0 0x7fb85c068960 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb878157850 0x7fb878131570 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 --2- 172.21.15.71:0/1665149196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb864003980 0x7fb8780ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 >> 172.21.15.71:0/1665149196 conn(0x7fb878070170 msgr2=0x7fb8780b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:01.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 shutdown_connections 2023-10-26T05:51:01.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.089+0000 7fb87f40c640 1 -- 172.21.15.71:0/1665149196 wait complete. 2023-10-26T05:51:01.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:49:54.856291+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:49:55.604098+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:01.113 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:01.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 -- 172.21.15.71:0/545545948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 msgr2=0x7ffbc4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/545545948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbc4004c50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800bbb0 tx=0x7ffbc801cd90 comp rx=0 tx=0).stop 2023-10-26T05:51:01.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 -- 172.21.15.71:0/545545948 shutdown_connections 2023-10-26T05:51:01.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/545545948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbc4004c50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 -- 172.21.15.71:0/545545948 >> 172.21.15.71:0/545545948 conn(0x7ffbd80aec60 msgr2=0x7ffbd80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:01.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 -- 172.21.15.71:0/545545948 shutdown_connections 2023-10-26T05:51:01.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.233+0000 7ffbdfd67640 1 -- 172.21.15.71:0/545545948 wait complete. 2023-10-26T05:51:01.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 Processor -- start 2023-10-26T05:51:01.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- start start 2023-10-26T05:51:01.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbd8157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd8155610 0x7ffbd81359b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8157690 0x7ffbd8135ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8157ba0 con 0x7ffbc40047d0 2023-10-26T05:51:01.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81f6590 con 0x7ffbd8157690 2023-10-26T05:51:01.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd81f6870 con 0x7ffbd8155610 2023-10-26T05:51:01.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbd8157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbd8157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44838/0 (socket says 172.21.15.71:44838) 2023-10-26T05:51:01.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 -- 172.21.15.71:0/4268908989 learned_addr learned my addr 172.21.15.71:0/4268908989 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:01.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbde304640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8157690 0x7ffbd8135ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdd302640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd8155610 0x7ffbd81359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 -- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd8155610 msgr2=0x7ffbd81359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd8155610 0x7ffbd81359b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 -- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8157690 msgr2=0x7ffbd8135ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8157690 0x7ffbd8135ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 -- 172.21.15.71:0/4268908989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd81f6b50 con 0x7ffbc40047d0 2023-10-26T05:51:01.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdd302640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd8155610 0x7ffbd81359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:01.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbddb03640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbd8157150 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400d580 tx=0x7ffbd400da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:01.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4006360 con 0x7ffbc40047d0 2023-10-26T05:51:01.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffbd4003620 con 0x7ffbc40047d0 2023-10-26T05:51:01.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40039f0 con 0x7ffbc40047d0 2023-10-26T05:51:01.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd8157e80 con 0x7ffbc40047d0 2023-10-26T05:51:01.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd81f7240 con 0x7ffbc40047d0 2023-10-26T05:51:01.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd8066b10 con 0x7ffbc40047d0 2023-10-26T05:51:01.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.237+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffbd4003b50 con 0x7ffbc40047d0 2023-10-26T05:51:01.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.241+0000 7ffbce7fc640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffbbc066470 0x7ffbbc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.241+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbd40844a0 con 0x7ffbc40047d0 2023-10-26T05:51:01.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.241+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7ffbbc06dfa0 con 0x7ffbc40047d0 2023-10-26T05:51:01.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.241+0000 7ffbdd302640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffbbc066470 0x7ffbbc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.241+0000 7ffbdd302640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffbbc066470 0x7ffbbc068930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80b0000 tx=0x7ffbc8019b30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:01.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.245+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffbd4014060 con 0x7ffbc40047d0 2023-10-26T05:51:01.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.445+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbd80721e0 con 0x7ffbc40047d0 2023-10-26T05:51:01.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.445+0000 7ffbce7fc640 1 -- 172.21.15.71:0/4268908989 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbd40519b0 con 0x7ffbc40047d0 2023-10-26T05:51:01.452 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:01.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffbbc066470 msgr2=0x7ffbbc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffbbc066470 0x7ffbbc068930 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80b0000 tx=0x7ffbc8019b30 comp rx=0 tx=0).stop 2023-10-26T05:51:01.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 msgr2=0x7ffbd8157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbd8157150 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400d580 tx=0x7ffbd400da40 comp rx=0 tx=0).stop 2023-10-26T05:51:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 shutdown_connections 2023-10-26T05:51:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd8157690 0x7ffbd8135ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd8155610 0x7ffbd81359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffbbc066470 0x7ffbbc068930 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 --2- 172.21.15.71:0/4268908989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffbc40047d0 0x7ffbd8157150 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 >> 172.21.15.71:0/4268908989 conn(0x7ffbd80aec60 msgr2=0x7ffbd80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 shutdown_connections 2023-10-26T05:51:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.449+0000 7ffbdfd67640 1 -- 172.21.15.71:0/4268908989 wait complete. 2023-10-26T05:51:01.470 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:51:01.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 -- 172.21.15.71:0/3369735298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 msgr2=0x7f64e40699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/3369735298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40699f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f64e40cc300 tx=0x7f64d401c7a0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 -- 172.21.15.71:0/3369735298 shutdown_connections 2023-10-26T05:51:01.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/3369735298 >> v2:172.21.15.86:3300/0 conn(0x7f64e415f100 0x7f64e415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/3369735298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40699f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 -- 172.21.15.71:0/3369735298 >> 172.21.15.71:0/3369735298 conn(0x7f64e40d1440 msgr2=0x7f64e40d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:01.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 -- 172.21.15.71:0/3369735298 shutdown_connections 2023-10-26T05:51:01.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.581+0000 7f64e8dd8640 1 -- 172.21.15.71:0/3369735298 wait complete. 2023-10-26T05:51:01.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 Processor -- start 2023-10-26T05:51:01.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- start start 2023-10-26T05:51:01.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64e4152000 0x7f64e4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e415f100 0x7f64e415bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e414e550 con 0x7f64e40cc810 2023-10-26T05:51:01.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e414e9e0 con 0x7f64e415f100 2023-10-26T05:51:01.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e414d940 con 0x7f64e4152000 2023-10-26T05:51:01.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44860/0 (socket says 172.21.15.71:44860) 2023-10-26T05:51:01.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 -- 172.21.15.71:0/527438653 learned_addr learned my addr 172.21.15.71:0/527438653 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:01.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e2d9d640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e415f100 0x7f64e415bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e1d9b640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64e4152000 0x7f64e4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 -- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64e4152000 msgr2=0x7f64e4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64e4152000 0x7f64e4155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 -- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e415f100 msgr2=0x7f64e415bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e415f100 0x7f64e415bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 -- 172.21.15.71:0/527438653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4009560 con 0x7f64e40cc810 2023-10-26T05:51:01.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e2d9d640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e415f100 0x7f64e415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:01.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e259c640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40c3d50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f64d40b0040 tx=0x7f64d40a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:01.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4005230 con 0x7f64e40cc810 2023-10-26T05:51:01.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f64d40054a0 con 0x7f64e40cc810 2023-10-26T05:51:01.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e414dc20 con 0x7f64e40cc810 2023-10-26T05:51:01.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4004930 con 0x7f64e40cc810 2023-10-26T05:51:01.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e414e1e0 con 0x7f64e40cc810 2023-10-26T05:51:01.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e4066b10 con 0x7f64e40cc810 2023-10-26T05:51:01.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.585+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f64d40b4070 con 0x7f64e40cc810 2023-10-26T05:51:01.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.589+0000 7f64d2ffd640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f64c40664a0 0x7f64c4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.589+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64d411e550 con 0x7f64e40cc810 2023-10-26T05:51:01.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.589+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f64c406dfd0 con 0x7f64e40cc810 2023-10-26T05:51:01.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.589+0000 7f64e1d9b640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f64c40664a0 0x7f64c4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.589+0000 7f64e1d9b640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f64c40664a0 0x7f64c4068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64e41f7d60 tx=0x7f64d8005bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:01.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.593+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f64d40eb940 con 0x7f64e40cc810 2023-10-26T05:51:01.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.741+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64e4066b10 con 0x7f64e40cc810 2023-10-26T05:51:01.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64d2ffd640 1 -- 172.21.15.71:0/527438653 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64d40eef60 con 0x7f64e40cc810 2023-10-26T05:51:01.749 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:51:01.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f64c40664a0 msgr2=0x7f64c4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f64c40664a0 0x7f64c4068960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f64e41f7d60 tx=0x7f64d8005bd0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 msgr2=0x7f64e40c3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40c3d50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f64d40b0040 tx=0x7f64d40a5840 comp rx=0 tx=0).stop 2023-10-26T05:51:01.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 shutdown_connections 2023-10-26T05:51:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64e415f100 0x7f64e415bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64e4152000 0x7f64e4155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f64c40664a0 0x7f64c4068960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 --2- 172.21.15.71:0/527438653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64e40cc810 0x7f64e40c3d50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 >> 172.21.15.71:0/527438653 conn(0x7f64e40d1440 msgr2=0x7f64e4151780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:01.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 shutdown_connections 2023-10-26T05:51:01.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.745+0000 7f64e8dd8640 1 -- 172.21.15.71:0/527438653 wait complete. 2023-10-26T05:51:01.768 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:51:01.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/3928266877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab813c220 msgr2=0x7faab815cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 --2- 172.21.15.71:0/3928266877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab813c220 0x7faab815cce0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7faab806c770 tx=0x7faaa801c7e0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/3928266877 shutdown_connections 2023-10-26T05:51:01.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 --2- 172.21.15.71:0/3928266877 >> v2:172.21.15.86:3300/0 conn(0x7faab815d220 0x7faab80cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 --2- 172.21.15.71:0/3928266877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab813c220 0x7faab815cce0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/3928266877 >> 172.21.15.71:0/3928266877 conn(0x7faab80aec60 msgr2=0x7faab80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:01.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/3928266877 shutdown_connections 2023-10-26T05:51:01.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/3928266877 wait complete. 2023-10-26T05:51:01.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 Processor -- start 2023-10-26T05:51:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- start start 2023-10-26T05:51:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80cbd60 0x7faab81356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faab815d220 0x7faab8151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 0x7faab814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab8150bd0 con 0x7faab814f420 2023-10-26T05:51:01.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80ae860 con 0x7faab80cbd60 2023-10-26T05:51:01.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab814ffc0 con 0x7faab815d220 2023-10-26T05:51:01.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 0x7faab814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 0x7faab814f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60604/0 (socket says 172.21.15.71:60604) 2023-10-26T05:51:01.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 -- 172.21.15.71:0/4049104010 learned_addr learned my addr 172.21.15.71:0/4049104010 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:01.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faab77fe640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faab815d220 0x7faab8151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faab7fff640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80cbd60 0x7faab81356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 -- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faab815d220 msgr2=0x7faab8151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faab815d220 0x7faab8151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 -- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80cbd60 msgr2=0x7faab81356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80cbd60 0x7faab81356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 -- 172.21.15.71:0/4049104010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8009560 con 0x7faab814f420 2023-10-26T05:51:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabcdf6640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 0x7faab814f8d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faab000c770 tx=0x7faab000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0003b80 con 0x7faab814f420 2023-10-26T05:51:01.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faab0003ce0 con 0x7faab814f420 2023-10-26T05:51:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab81502a0 con 0x7faab814f420 2023-10-26T05:51:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000aab0 con 0x7faab814f420 2023-10-26T05:51:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.897+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab8150860 con 0x7faab814f420 2023-10-26T05:51:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab8066b10 con 0x7faab814f420 2023-10-26T05:51:01.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faab000ac10 con 0x7faab814f420 2023-10-26T05:51:01.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faab4ff9640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faaa4066470 0x7faaa4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:01.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faab0084500 con 0x7faab814f420 2023-10-26T05:51:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7faaa406dfa0 con 0x7faab814f420 2023-10-26T05:51:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faab7fff640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faaa4066470 0x7faaa4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.901+0000 7faab7fff640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faaa4066470 0x7faaa4068930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faaa80af000 tx=0x7faaa801b020 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:01.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:01.905+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faab0051af0 con 0x7faab814f420 2023-10-26T05:51:02.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.057+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faab80721e0 con 0x7faab814f420 2023-10-26T05:51:02.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faab4ff9640 1 -- 172.21.15.71:0/4049104010 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7faab0055110 con 0x7faab814f420 2023-10-26T05:51:02.066 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:51:02.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faaa4066470 msgr2=0x7faaa4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faaa4066470 0x7faaa4068930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faaa80af000 tx=0x7faaa801b020 comp rx=0 tx=0).stop 2023-10-26T05:51:02.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 msgr2=0x7faab814f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 0x7faab814f8d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7faab000c770 tx=0x7faab000cc30 comp rx=0 tx=0).stop 2023-10-26T05:51:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 shutdown_connections 2023-10-26T05:51:02.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faaa4066470 0x7faaa4068930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faab814f420 0x7faab814f8d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faab815d220 0x7faab8151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 --2- 172.21.15.71:0/4049104010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faab80cbd60 0x7faab81356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 >> 172.21.15.71:0/4049104010 conn(0x7faab80aec60 msgr2=0x7faab815baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:02.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 shutdown_connections 2023-10-26T05:51:02.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.061+0000 7faabe859640 1 -- 172.21.15.71:0/4049104010 wait complete. 2023-10-26T05:51:02.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:51:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 -- 172.21.15.71:0/163652723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8840039b0 msgr2=0x7ff884003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/163652723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8840039b0 0x7ff884003dd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff8880013e0 tx=0x7ff88801d300 comp rx=0 tx=0).stop 2023-10-26T05:51:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff8977fe640 1 -- 172.21.15.71:0/163652723 reap_dead start 2023-10-26T05:51:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 -- 172.21.15.71:0/163652723 shutdown_connections 2023-10-26T05:51:02.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 -- 172.21.15.71:0/163652723 >> 172.21.15.71:0/163652723 conn(0x7ff89806b8c0 msgr2=0x7ff89806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:02.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 -- 172.21.15.71:0/163652723 shutdown_connections 2023-10-26T05:51:02.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 -- 172.21.15.71:0/163652723 wait complete. 2023-10-26T05:51:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 Processor -- start 2023-10-26T05:51:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.201+0000 7ff89d5c4640 1 -- start start 2023-10-26T05:51:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8840039b0 0x7ff89814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff898147950 0x7ff898156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 0x7ff8980cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980cbac0 con 0x7ff8980cb100 2023-10-26T05:51:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898155330 con 0x7ff898147950 2023-10-26T05:51:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff898154a40 con 0x7ff8840039b0 2023-10-26T05:51:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 0x7ff8980cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 0x7ff8980cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60624/0 (socket says 172.21.15.71:60624) 2023-10-26T05:51:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 -- 172.21.15.71:0/3677529060 learned_addr learned my addr 172.21.15.71:0/3677529060 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff896ffd640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8840039b0 0x7ff89814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8967fc640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff898147950 0x7ff898156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 -- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8840039b0 msgr2=0x7ff89814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8840039b0 0x7ff89814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 -- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff898147950 msgr2=0x7ff898156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff898147950 0x7ff898156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 -- 172.21.15.71:0/3677529060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898157000 con 0x7ff8980cb100 2023-10-26T05:51:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff8977fe640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 0x7ff8980cb580 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff898071a60 tx=0x7ff89000f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890005220 con 0x7ff8980cb100 2023-10-26T05:51:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff890005490 con 0x7ff8980cb100 2023-10-26T05:51:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89814b530 con 0x7ff8980cb100 2023-10-26T05:51:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890004a10 con 0x7ff8980cb100 2023-10-26T05:51:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff898154fc0 con 0x7ff8980cb100 2023-10-26T05:51:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff898066b10 con 0x7ff8980cb100 2023-10-26T05:51:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff890004b70 con 0x7ff8980cb100 2023-10-26T05:51:02.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8780664a0 0x7ff878068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff890084450 con 0x7ff8980cb100 2023-10-26T05:51:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7ff87806dfd0 con 0x7ff8980cb100 2023-10-26T05:51:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.205+0000 7ff896ffd640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8780664a0 0x7ff878068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.209+0000 7ff896ffd640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8780664a0 0x7ff878068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ad000 tx=0x7ff88801a0a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:02.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.209+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff890020060 con 0x7ff8980cb100 2023-10-26T05:51:02.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff89806c7d0 con 0x7ff8980cb100 2023-10-26T05:51:02.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff88f7fe640 1 -- 172.21.15.71:0/3677529060 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff890051980 con 0x7ff8980cb100 2023-10-26T05:51:02.382 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:51:02.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8780664a0 msgr2=0x7ff878068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8780664a0 0x7ff878068960 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ad000 tx=0x7ff88801a0a0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 msgr2=0x7ff8980cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 0x7ff8980cb580 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff898071a60 tx=0x7ff89000f960 comp rx=0 tx=0).stop 2023-10-26T05:51:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 shutdown_connections 2023-10-26T05:51:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff8780664a0 0x7ff878068960 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8980cb100 0x7ff8980cb580 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff898147950 0x7ff898156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 --2- 172.21.15.71:0/3677529060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8840039b0 0x7ff89814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 >> 172.21.15.71:0/3677529060 conn(0x7ff89806b8c0 msgr2=0x7ff8980b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.377+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 shutdown_connections 2023-10-26T05:51:02.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.381+0000 7ff89d5c4640 1 -- 172.21.15.71:0/3677529060 wait complete. 2023-10-26T05:51:02.400 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:51:02.401 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:02.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:02.455 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:02.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34414]: fuse finished with error 0 and tester_r 0 2023-10-26T05:51:02.456 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:02.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:02.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:51:02.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:02.465 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:51:02.472 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:51:02.472 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:02.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:51:02.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:51:02.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:02.479 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:51:02.524 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:51:02.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:51:02.526 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:51:02.526 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:02.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:51:02.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:51:02.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:02.536 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:51:02.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:51:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:51:02.583 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:02.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/1413913651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f14070fe0 msgr2=0x7f2f14071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 --2- 172.21.15.71:0/1413913651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f14070fe0 0x7f2f14071470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009930 tx=0x7f2efc01c890 comp rx=0 tx=0).stop 2023-10-26T05:51:02.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/1413913651 shutdown_connections 2023-10-26T05:51:02.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 --2- 172.21.15.71:0/1413913651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f14070fe0 0x7f2f14071470 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/1413913651 >> 172.21.15.71:0/1413913651 conn(0x7f2f14069770 msgr2=0x7f2f14069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:02.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/1413913651 shutdown_connections 2023-10-26T05:51:02.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/1413913651 wait complete. 2023-10-26T05:51:02.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 Processor -- start 2023-10-26T05:51:02.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- start start 2023-10-26T05:51:02.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f14070fe0 0x7f2f14156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 0x7f2f14138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f14140cc0 0x7f2f14157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f141572d0 con 0x7f2f140b1360 2023-10-26T05:51:02.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14158fe0 con 0x7f2f14070fe0 2023-10-26T05:51:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f14137710 con 0x7f2f14140cc0 2023-10-26T05:51:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 0x7f2f14138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 0x7f2f14138120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60644/0 (socket says 172.21.15.71:60644) 2023-10-26T05:51:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 -- 172.21.15.71:0/395829045 learned_addr learned my addr 172.21.15.71:0/395829045 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f137fe640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f14070fe0 0x7f2f14156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f13fff640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f14140cc0 0x7f2f14157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 -- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f14140cc0 msgr2=0x7f2f14157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f14140cc0 0x7f2f14157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 -- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f14070fe0 msgr2=0x7f2f14156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f14070fe0 0x7f2f14156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 -- 172.21.15.71:0/395829045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc009560 con 0x7f2f140b1360 2023-10-26T05:51:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f12ffd640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 0x7f2f14138120 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009850 tx=0x7f2efc0a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc005220 con 0x7f2f140b1360 2023-10-26T05:51:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2efc005470 con 0x7f2f140b1360 2023-10-26T05:51:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f141587d0 con 0x7f2f140b1360 2023-10-26T05:51:02.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc004930 con 0x7f2f140b1360 2023-10-26T05:51:02.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.709+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f141399a0 con 0x7f2f140b1360 2023-10-26T05:51:02.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f14066b10 con 0x7f2f140b1360 2023-10-26T05:51:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2efc0b4070 con 0x7f2f140b1360 2023-10-26T05:51:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f0bfff640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ef4066470 0x7f2ef4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:02.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2efc11e000 con 0x7f2f140b1360 2023-10-26T05:51:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f2ef406dfa0 con 0x7f2f140b1360 2023-10-26T05:51:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f137fe640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ef4066470 0x7f2ef4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.713+0000 7f2f137fe640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ef4066470 0x7f2ef4068930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f14138f30 tx=0x7f2f0c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:02.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.717+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2efc0eb480 con 0x7f2f140b1360 2023-10-26T05:51:02.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.905+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f140721e0 con 0x7f2f140b1360 2023-10-26T05:51:02.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.905+0000 7f2f0bfff640 1 -- 172.21.15.71:0/395829045 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f2efc0b2020 con 0x7f2f140b1360 2023-10-26T05:51:02.911 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:02.912 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:51:02.912 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:51:02.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ef4066470 msgr2=0x7f2ef4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ef4066470 0x7f2ef4068930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f14138f30 tx=0x7f2f0c009040 comp rx=0 tx=0).stop 2023-10-26T05:51:02.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 msgr2=0x7f2f14138120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:02.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 0x7f2f14138120 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009850 tx=0x7f2efc0a5780 comp rx=0 tx=0).stop 2023-10-26T05:51:02.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 shutdown_connections 2023-10-26T05:51:02.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2f14140cc0 0x7f2f14157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ef4066470 0x7f2ef4068930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f140b1360 0x7f2f14138120 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 --2- 172.21.15.71:0/395829045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f14070fe0 0x7f2f14156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:02.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 >> 172.21.15.71:0/395829045 conn(0x7f2f14069770 msgr2=0x7f2f140b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:02.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 shutdown_connections 2023-10-26T05:51:02.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:02.909+0000 7f2f19b88640 1 -- 172.21.15.71:0/395829045 wait complete. 2023-10-26T05:51:02.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/175547048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f04140cc0 msgr2=0x7f7f0415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/175547048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f04140cc0 0x7f7f0415c760 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009c80 tx=0x7f7ef401ccd0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/175547048 shutdown_connections 2023-10-26T05:51:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/175547048 >> v2:172.21.15.86:3300/0 conn(0x7f7f04070fe0 0x7f7f04071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/175547048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f04140cc0 0x7f7f0415c760 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/175547048 >> 172.21.15.71:0/175547048 conn(0x7f7f04069770 msgr2=0x7f7f04069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/175547048 shutdown_connections 2023-10-26T05:51:03.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.045+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/175547048 wait complete. 2023-10-26T05:51:03.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 Processor -- start 2023-10-26T05:51:03.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 -- start start 2023-10-26T05:51:03.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f04070fe0 0x7f7f0415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 0x7f7f0415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f04140cc0 0x7f7f04136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0415ac40 con 0x7f7f040b0040 2023-10-26T05:51:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0415d4d0 con 0x7f7f04070fe0 2023-10-26T05:51:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0415a270 con 0x7f7f04140cc0 2023-10-26T05:51:03.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 0x7f7f0415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 0x7f7f0415a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60662/0 (socket says 172.21.15.71:60662) 2023-10-26T05:51:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 -- 172.21.15.71:0/2437362334 learned_addr learned my addr 172.21.15.71:0/2437362334 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0884a640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f04140cc0 0x7f7f04136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f03fff640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f04070fe0 0x7f7f0415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 -- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f04140cc0 msgr2=0x7f7f04136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f04140cc0 0x7f7f04136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 -- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f04070fe0 msgr2=0x7f7f0415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f04070fe0 0x7f7f0415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 -- 172.21.15.71:0/2437362334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef40098e0 con 0x7f7f040b0040 2023-10-26T05:51:03.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f037fe640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 0x7f7f0415a700 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c9a0 tx=0x7f7eec00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec007b50 con 0x7f7f040b0040 2023-10-26T05:51:03.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7eec007dc0 con 0x7f7f040b0040 2023-10-26T05:51:03.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f041373f0 con 0x7f7f040b0040 2023-10-26T05:51:03.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec005280 con 0x7f7f040b0040 2023-10-26T05:51:03.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.049+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0415d1f0 con 0x7f7f040b0040 2023-10-26T05:51:03.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f04066b10 con 0x7f7f040b0040 2023-10-26T05:51:03.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7eec0053e0 con 0x7f7f040b0040 2023-10-26T05:51:03.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f00ff9640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ee4066470 0x7f7ee4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7eec084360 con 0x7f7f040b0040 2023-10-26T05:51:03.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7f7ee406dfa0 con 0x7f7f040b0040 2023-10-26T05:51:03.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f03fff640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ee4066470 0x7f7ee4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.053+0000 7f7f03fff640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ee4066470 0x7f7ee4068930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40af000 tx=0x7f7ef401b510 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.057+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7eec01d020 con 0x7f7f040b0040 2023-10-26T05:51:03.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.205+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f040721e0 con 0x7f7f040b0040 2023-10-26T05:51:03.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f00ff9640 1 -- 172.21.15.71:0/2437362334 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16539 (secure 0 0 0) 0x7f7eec055300 con 0x7f7f040b0040 2023-10-26T05:51:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:03.214 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":73,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:50:20.321590+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:03.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ee4066470 msgr2=0x7f7ee4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ee4066470 0x7f7ee4068930 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40af000 tx=0x7f7ef401b510 comp rx=0 tx=0).stop 2023-10-26T05:51:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 msgr2=0x7f7f0415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 0x7f7f0415a700 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00c9a0 tx=0x7f7eec00ce60 comp rx=0 tx=0).stop 2023-10-26T05:51:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 shutdown_connections 2023-10-26T05:51:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f04140cc0 0x7f7f04136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ee4066470 0x7f7ee4068930 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f040b0040 0x7f7f0415a700 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 --2- 172.21.15.71:0/2437362334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f04070fe0 0x7f7f0415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 >> 172.21.15.71:0/2437362334 conn(0x7f7f04069770 msgr2=0x7f7f040b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 shutdown_connections 2023-10-26T05:51:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.209+0000 7f7f0a2ad640 1 -- 172.21.15.71:0/2437362334 wait complete. 2023-10-26T05:51:03.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:49:54.856291+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:49:55.604098+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:03.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:03.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/544992112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 msgr2=0x7fe58806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 --2- 172.21.15.71:0/544992112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe58806a000 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe5880cce60 tx=0x7fe58401c7a0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/544992112 shutdown_connections 2023-10-26T05:51:03.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 --2- 172.21.15.71:0/544992112 >> v2:172.21.15.86:3300/0 conn(0x7fe5880b1080 0x7fe5880ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 --2- 172.21.15.71:0/544992112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe58806a000 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/544992112 >> 172.21.15.71:0/544992112 conn(0x7fe5880aec60 msgr2=0x7fe5880af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/544992112 shutdown_connections 2023-10-26T05:51:03.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/544992112 wait complete. 2023-10-26T05:51:03.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 Processor -- start 2023-10-26T05:51:03.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- start start 2023-10-26T05:51:03.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe58806a540 0x7fe58813fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5880b1080 0x7fe588143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe588148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588147a10 con 0x7fe588157920 2023-10-26T05:51:03.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe588147cf0 con 0x7fe5880b1080 2023-10-26T05:51:03.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe588147fd0 con 0x7fe58806a540 2023-10-26T05:51:03.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe588148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe588148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60682/0 (socket says 172.21.15.71:60682) 2023-10-26T05:51:03.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 -- 172.21.15.71:0/1724509098 learned_addr learned my addr 172.21.15.71:0/1724509098 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:03.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58e34f640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe58806a540 0x7fe58813fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58db4e640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5880b1080 0x7fe588143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 -- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe58806a540 msgr2=0x7fe58813fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe58806a540 0x7fe58813fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 -- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5880b1080 msgr2=0x7fe588143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5880b1080 0x7fe588143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 -- 172.21.15.71:0/1724509098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe584009560 con 0x7fe588157920 2023-10-26T05:51:03.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe58eb50640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe588148ab0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe58000b970 tx=0x7fe58000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000cb40 con 0x7fe588157920 2023-10-26T05:51:03.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe58000cdb0 con 0x7fe588157920 2023-10-26T05:51:03.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5881482b0 con 0x7fe588157920 2023-10-26T05:51:03.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5800146a0 con 0x7fe588157920 2023-10-26T05:51:03.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.357+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588143c40 con 0x7fe588157920 2023-10-26T05:51:03.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588066b10 con 0x7fe588157920 2023-10-26T05:51:03.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe58001d3b0 con 0x7fe588157920 2023-10-26T05:51:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe57effd640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe56c066470 0x7fe56c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe580084010 con 0x7fe588157920 2023-10-26T05:51:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7fe56c06dfa0 con 0x7fe588157920 2023-10-26T05:51:03.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe58e34f640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe56c066470 0x7fe56c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.361+0000 7fe58e34f640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe56c066470 0x7fe56c068930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe5840af000 tx=0x7fe58401afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.365+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe580019060 con 0x7fe588157920 2023-10-26T05:51:03.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5880721e0 con 0x7fe588157920 2023-10-26T05:51:03.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe57effd640 1 -- 172.21.15.71:0/1724509098 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe580051490 con 0x7fe588157920 2023-10-26T05:51:03.525 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe56c066470 msgr2=0x7fe56c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe56c066470 0x7fe56c068930 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe5840af000 tx=0x7fe58401afe0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 msgr2=0x7fe588148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe588148ab0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe58000b970 tx=0x7fe58000be30 comp rx=0 tx=0).stop 2023-10-26T05:51:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 shutdown_connections 2023-10-26T05:51:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe56c066470 0x7fe56c068930 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe588157920 0x7fe588148ab0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5880b1080 0x7fe588143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 --2- 172.21.15.71:0/1724509098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe58806a540 0x7fe58813fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 >> 172.21.15.71:0/1724509098 conn(0x7fe5880aec60 msgr2=0x7fe58814d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 shutdown_connections 2023-10-26T05:51:03.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.521+0000 7fe5905b3640 1 -- 172.21.15.71:0/1724509098 wait complete. 2023-10-26T05:51:03.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:51:03.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- 172.21.15.71:0/669917500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 msgr2=0x7faa5c15ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 --2- 172.21.15.71:0/669917500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c15ccc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0cdec0 tx=0x7faa4c01c830 comp rx=0 tx=0).stop 2023-10-26T05:51:03.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- 172.21.15.71:0/669917500 shutdown_connections 2023-10-26T05:51:03.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 --2- 172.21.15.71:0/669917500 >> v2:172.21.15.86:3300/0 conn(0x7faa5c15d200 0x7faa5c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 --2- 172.21.15.71:0/669917500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c15ccc0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- 172.21.15.71:0/669917500 >> 172.21.15.71:0/669917500 conn(0x7faa5c0aec60 msgr2=0x7faa5c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- 172.21.15.71:0/669917500 shutdown_connections 2023-10-26T05:51:03.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- 172.21.15.71:0/669917500 wait complete. 2023-10-26T05:51:03.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 Processor -- start 2023-10-26T05:51:03.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- start start 2023-10-26T05:51:03.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa5c0cbcd0 0x7faa5c13b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa5c15d200 0x7faa5c137580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c152560 con 0x7faa5c13c220 2023-10-26T05:51:03.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c152840 con 0x7faa5c0cbcd0 2023-10-26T05:51:03.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa625e4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c152b20 con 0x7faa5c15d200 2023-10-26T05:51:03.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60704/0 (socket says 172.21.15.71:60704) 2023-10-26T05:51:03.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 -- 172.21.15.71:0/2501335720 learned_addr learned my addr 172.21.15.71:0/2501335720 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:03.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5bfff640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa5c0cbcd0 0x7faa5c13b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa60b81640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa5c15d200 0x7faa5c137580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 -- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa5c15d200 msgr2=0x7faa5c137580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa5c15d200 0x7faa5c137580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 -- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa5c0cbcd0 msgr2=0x7faa5c13b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa5c0cbcd0 0x7faa5c13b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 -- 172.21.15.71:0/2501335720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c009560 con 0x7faa5c13c220 2023-10-26T05:51:03.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa60b81640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa5c15d200 0x7faa5c137580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:03.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5bfff640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa5c0cbcd0 0x7faa5c13b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:03.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa5b7fe640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c158280 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faa5000c9f0 tx=0x7faa5000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.661+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa50007bb0 con 0x7faa5c13c220 2023-10-26T05:51:03.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa50007e00 con 0x7faa5c13c220 2023-10-26T05:51:03.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c152e00 con 0x7faa5c13c220 2023-10-26T05:51:03.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5000f6b0 con 0x7faa5c13c220 2023-10-26T05:51:03.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c152210 con 0x7faa5c13c220 2023-10-26T05:51:03.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c066b10 con 0x7faa5c13c220 2023-10-26T05:51:03.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faa5000f810 con 0x7faa5c13c220 2023-10-26T05:51:03.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa58ff9640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa480664a0 0x7faa48068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa50084680 con 0x7faa5c13c220 2023-10-26T05:51:03.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7faa4806dfd0 con 0x7faa5c13c220 2023-10-26T05:51:03.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa5bfff640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa480664a0 0x7faa48068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.665+0000 7faa5bfff640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa480664a0 0x7faa48068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0af000 tx=0x7faa4c01b070 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.669+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faa50051b00 con 0x7faa5c13c220 2023-10-26T05:51:03.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.817+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa5c0721e0 con 0x7faa5c13c220 2023-10-26T05:51:03.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa58ff9640 1 -- 172.21.15.71:0/2501335720 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa5001d0a0 con 0x7faa5c13c220 2023-10-26T05:51:03.824 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:51:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa480664a0 msgr2=0x7faa48068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa480664a0 0x7faa48068960 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0af000 tx=0x7faa4c01b070 comp rx=0 tx=0).stop 2023-10-26T05:51:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 msgr2=0x7faa5c158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c158280 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faa5000c9f0 tx=0x7faa5000ceb0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 shutdown_connections 2023-10-26T05:51:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa5c15d200 0x7faa5c137580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa480664a0 0x7faa48068960 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5c13c220 0x7faa5c158280 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 --2- 172.21.15.71:0/2501335720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa5c0cbcd0 0x7faa5c13b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 >> 172.21.15.71:0/2501335720 conn(0x7faa5c0aec60 msgr2=0x7faa5c0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 shutdown_connections 2023-10-26T05:51:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.821+0000 7faa625e4640 1 -- 172.21.15.71:0/2501335720 wait complete. 2023-10-26T05:51:03.842 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:51:03.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 -- 172.21.15.71:0/4099112004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec0cbe00 msgr2=0x7ff0ec13e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 --2- 172.21.15.71:0/4099112004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec0cbe00 0x7ff0ec13e850 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009900 tx=0x7ff0e001c850 comp rx=0 tx=0).stop 2023-10-26T05:51:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 -- 172.21.15.71:0/4099112004 shutdown_connections 2023-10-26T05:51:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 --2- 172.21.15.71:0/4099112004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec0cbe00 0x7ff0ec13e850 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 --2- 172.21.15.71:0/4099112004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec13c120 0x7ff0ec0cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 -- 172.21.15.71:0/4099112004 >> 172.21.15.71:0/4099112004 conn(0x7ff0ec0aec60 msgr2=0x7ff0ec0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.953+0000 7ff0f3036640 1 -- 172.21.15.71:0/4099112004 shutdown_connections 2023-10-26T05:51:03.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- 172.21.15.71:0/4099112004 wait complete. 2023-10-26T05:51:03.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 Processor -- start 2023-10-26T05:51:03.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- start start 2023-10-26T05:51:03.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0ec0cbe00 0x7ff0ec1358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 0x7ff0ec13e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec15a3f0 0x7ff0ec13d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec14ab20 con 0x7ff0ec13c120 2023-10-26T05:51:03.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec06f6e0 con 0x7ff0ec0cbe00 2023-10-26T05:51:03.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ec14a840 con 0x7ff0ec15a3f0 2023-10-26T05:51:03.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 0x7ff0ec13e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 0x7ff0ec13e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60720/0 (socket says 172.21.15.71:60720) 2023-10-26T05:51:03.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 -- 172.21.15.71:0/2591392316 learned_addr learned my addr 172.21.15.71:0/2591392316 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:03.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f0dd2640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0ec0cbe00 0x7ff0ec1358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f15d3640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec15a3f0 0x7ff0ec13d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 -- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec15a3f0 msgr2=0x7ff0ec13d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec15a3f0 0x7ff0ec13d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 -- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0ec0cbe00 msgr2=0x7ff0ec1358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0ec0cbe00 0x7ff0ec1358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 -- 172.21.15.71:0/2591392316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0009560 con 0x7ff0ec13c120 2023-10-26T05:51:03.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f15d3640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec15a3f0 0x7ff0ec13d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0ebfff640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 0x7ff0ec13e510 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009850 tx=0x7ff0e00a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0005230 con 0x7ff0ec13c120 2023-10-26T05:51:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff0e0005480 con 0x7ff0ec13c120 2023-10-26T05:51:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ec06f980 con 0x7ff0ec13c120 2023-10-26T05:51:03.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0004930 con 0x7ff0ec13c120 2023-10-26T05:51:03.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ec13ea50 con 0x7ff0ec13c120 2023-10-26T05:51:03.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.957+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ec066b10 con 0x7ff0ec13c120 2023-10-26T05:51:03.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.961+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff0e00b4070 con 0x7ff0ec13c120 2023-10-26T05:51:03.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.961+0000 7ff0e97fa640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0d8066470 0x7ff0d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:03.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.961+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0e011dbf0 con 0x7ff0ec13c120 2023-10-26T05:51:03.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.961+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=74}) v3 -- 0x7ff0d806dfa0 con 0x7ff0ec13c120 2023-10-26T05:51:03.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.961+0000 7ff0f0dd2640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0d8066470 0x7ff0d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:03.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.961+0000 7ff0f0dd2640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0d8066470 0x7ff0d8068930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0058d0 tx=0x7ff0dc005e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:03.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:03.965+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff0e00eb070 con 0x7ff0ec13c120 2023-10-26T05:51:04.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0ec06f540 con 0x7ff0ec13c120 2023-10-26T05:51:04.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0e97fa640 1 -- 172.21.15.71:0/2591392316 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0ec06f540 con 0x7ff0ec13c120 2023-10-26T05:51:04.118 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:51:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0d8066470 msgr2=0x7ff0d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0d8066470 0x7ff0d8068930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0058d0 tx=0x7ff0dc005e30 comp rx=0 tx=0).stop 2023-10-26T05:51:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 msgr2=0x7ff0ec13e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 0x7ff0ec13e510 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009850 tx=0x7ff0e00a57e0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 shutdown_connections 2023-10-26T05:51:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0ec15a3f0 0x7ff0ec13d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff0d8066470 0x7ff0d8068930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0ec13c120 0x7ff0ec13e510 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 --2- 172.21.15.71:0/2591392316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0ec0cbe00 0x7ff0ec1358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.113+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 >> 172.21.15.71:0/2591392316 conn(0x7ff0ec0aec60 msgr2=0x7ff0ec14bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:04.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.117+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 shutdown_connections 2023-10-26T05:51:04.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.117+0000 7ff0f3036640 1 -- 172.21.15.71:0/2591392316 wait complete. 2023-10-26T05:51:04.136 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:04.186 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:51:04.181+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-10-26T05:51:04.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.253+0000 7f013f1c1640 1 -- 172.21.15.71:0/1091331810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0120004d10 msgr2=0x7f0120005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.253+0000 7f013f1c1640 1 --2- 172.21.15.71:0/1091331810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0120004d10 0x7f0120005190 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0130009e30 tx=0x7f013001d150 comp rx=0 tx=0).stop 2023-10-26T05:51:04.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 -- 172.21.15.71:0/1091331810 reap_dead start 2023-10-26T05:51:04.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/1091331810 shutdown_connections 2023-10-26T05:51:04.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/1091331810 >> 172.21.15.71:0/1091331810 conn(0x7f0138069770 msgr2=0x7f0138069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:04.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/1091331810 shutdown_connections 2023-10-26T05:51:04.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/1091331810 wait complete. 2023-10-26T05:51:04.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 Processor -- start 2023-10-26T05:51:04.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- start start 2023-10-26T05:51:04.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01381f1190 0x7f01380aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01381342e0 0x7f0138134cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 0x7f01381f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138135680 con 0x7f0138135200 2023-10-26T05:51:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380abcd0 con 0x7f01381342e0 2023-10-26T05:51:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380aae20 con 0x7f01381f1190 2023-10-26T05:51:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 0x7f01381f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 0x7f01381f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60738/0 (socket says 172.21.15.71:60738) 2023-10-26T05:51:04.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 -- 172.21.15.71:0/774566359 learned_addr learned my addr 172.21.15.71:0/774566359 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013cf5d640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01381f1190 0x7f01380aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f0137fff640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01381342e0 0x7f0138134cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 -- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01381f1190 msgr2=0x7f01380aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01381f1190 0x7f01380aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 -- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01381342e0 msgr2=0x7f0138134cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01381342e0 0x7f0138134cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 -- 172.21.15.71:0/774566359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0138141800 con 0x7f0138135200 2023-10-26T05:51:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013d75e640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 0x7f01381f4e40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0130002240 tx=0x7f01300ada10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01300adc90 con 0x7f0138135200 2023-10-26T05:51:04.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0130004a50 con 0x7f0138135200 2023-10-26T05:51:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01380ab0c0 con 0x7f0138135200 2023-10-26T05:51:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130004e00 con 0x7f0138135200 2023-10-26T05:51:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01380ab680 con 0x7f0138135200 2023-10-26T05:51:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0138066b10 con 0x7f0138135200 2023-10-26T05:51:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.257+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f01300b3070 con 0x7f0138135200 2023-10-26T05:51:04.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.261+0000 7f01357fa640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0124066470 0x7f0124068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.261+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01300a5e40 con 0x7f0138135200 2023-10-26T05:51:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.261+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f012406dfa0 con 0x7f0138135200 2023-10-26T05:51:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.261+0000 7f013cf5d640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0124066470 0x7f0124068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.261+0000 7f013cf5d640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0124066470 0x7f0124068930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0128006fd0 tx=0x7f0128006b10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:04.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.265+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f01300b1070 con 0x7f0138135200 2023-10-26T05:51:04.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.441+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01381412b0 con 0x7f0138135200 2023-10-26T05:51:04.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.441+0000 7f01357fa640 1 -- 172.21.15.71:0/774566359 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f01300ec1a0 con 0x7f0138135200 2023-10-26T05:51:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:04.447 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:51:04.447 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:51:04.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.441+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0124066470 msgr2=0x7f0124068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.441+0000 7f013f1c1640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0124066470 0x7f0124068930 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0128006fd0 tx=0x7f0128006b10 comp rx=0 tx=0).stop 2023-10-26T05:51:04.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.441+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 msgr2=0x7f01381f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.441+0000 7f013f1c1640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 0x7f01381f4e40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0130002240 tx=0x7f01300ada10 comp rx=0 tx=0).stop 2023-10-26T05:51:04.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 shutdown_connections 2023-10-26T05:51:04.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0124066470 0x7f0124068930 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0138135200 0x7f01381f4e40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01381342e0 0x7f0138134cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 --2- 172.21.15.71:0/774566359 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f01381f1190 0x7f01380aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 >> 172.21.15.71:0/774566359 conn(0x7f0138069770 msgr2=0x7f013813c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 shutdown_connections 2023-10-26T05:51:04.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.445+0000 7f013f1c1640 1 -- 172.21.15.71:0/774566359 wait complete. 2023-10-26T05:51:04.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 -- 172.21.15.71:0/1309601300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45e4003980 msgr2=0x7f45e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/1309601300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45e4003980 0x7f45e4003da0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f45f4001da0 tx=0x7f45f401cad0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 -- 172.21.15.71:0/1309601300 shutdown_connections 2023-10-26T05:51:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/1309601300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45e4003980 0x7f45e4003da0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 -- 172.21.15.71:0/1309601300 >> 172.21.15.71:0/1309601300 conn(0x7f45f80cdd80 msgr2=0x7f45f80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:04.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 -- 172.21.15.71:0/1309601300 shutdown_connections 2023-10-26T05:51:04.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.581+0000 7f45ff3b4640 1 -- 172.21.15.71:0/1309601300 wait complete. 2023-10-26T05:51:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 Processor -- start 2023-10-26T05:51:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- start start 2023-10-26T05:51:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45e4003980 0x7f45f815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f815a340 0x7f45f81e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 0x7f45f81e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f806a650 con 0x7f45f81e8800 2023-10-26T05:51:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f80c7e70 con 0x7f45f815a340 2023-10-26T05:51:04.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f81e91c0 con 0x7f45e4003980 2023-10-26T05:51:04.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 0x7f45f81e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 0x7f45f81e8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60754/0 (socket says 172.21.15.71:60754) 2023-10-26T05:51:04.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 -- 172.21.15.71:0/4127196911 learned_addr learned my addr 172.21.15.71:0/4127196911 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:04.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd150640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45e4003980 0x7f45f815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fc94f640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f815a340 0x7f45f81e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 -- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45e4003980 msgr2=0x7f45f815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45e4003980 0x7f45f815ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 -- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f815a340 msgr2=0x7f45f81e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f815a340 0x7f45f81e82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 -- 172.21.15.71:0/4127196911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80cb1e0 con 0x7f45f81e8800 2023-10-26T05:51:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd951640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 0x7f45f81e8c80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f45f80d0d10 tx=0x7f45f000d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0005240 con 0x7f45f81e8800 2023-10-26T05:51:04.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f45f00053c0 con 0x7f45f81e8800 2023-10-26T05:51:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f80c7b40 con 0x7f45f81e8800 2023-10-26T05:51:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0003860 con 0x7f45f81e8800 2023-10-26T05:51:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f80c5110 con 0x7f45f81e8800 2023-10-26T05:51:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8066b10 con 0x7f45f81e8800 2023-10-26T05:51:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45f00039c0 con 0x7f45f81e8800 2023-10-26T05:51:04.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45dc066470 0x7f45dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f0083090 con 0x7f45f81e8800 2023-10-26T05:51:04.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f45dc06dfa0 con 0x7f45f81e8800 2023-10-26T05:51:04.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.585+0000 7f45fd150640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45dc066470 0x7f45dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.589+0000 7f45fd150640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45dc066470 0x7f45dc068930 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45f40a9000 tx=0x7f45f4019870 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:04.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.589+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f45f0019060 con 0x7f45f81e8800 2023-10-26T05:51:04.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.733+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f80721e0 con 0x7f45f81e8800 2023-10-26T05:51:04.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45edffb640 1 -- 172.21.15.71:0/4127196911 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16525 (secure 0 0 0) 0x7f45f0050510 con 0x7f45f81e8800 2023-10-26T05:51:04.740 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:04.741 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":74,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:04.184788+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45dc066470 msgr2=0x7f45dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45dc066470 0x7f45dc068930 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45f40a9000 tx=0x7f45f4019870 comp rx=0 tx=0).stop 2023-10-26T05:51:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 msgr2=0x7f45f81e8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 0x7f45f81e8c80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f45f80d0d10 tx=0x7f45f000d960 comp rx=0 tx=0).stop 2023-10-26T05:51:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 shutdown_connections 2023-10-26T05:51:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45dc066470 0x7f45dc068930 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45f81e8800 0x7f45f81e8c80 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45f815a340 0x7f45f81e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 --2- 172.21.15.71:0/4127196911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45e4003980 0x7f45f815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 >> 172.21.15.71:0/4127196911 conn(0x7f45f80cdd80 msgr2=0x7f45f80b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 shutdown_connections 2023-10-26T05:51:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.737+0000 7f45ff3b4640 1 -- 172.21.15.71:0/4127196911 wait complete. 2023-10-26T05:51:04.758 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:51:04.759 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:04.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 -- 172.21.15.71:0/2104292797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 msgr2=0x7f7afc15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 --2- 172.21.15.71:0/2104292797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15b3a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7afc08a3b0 tx=0x7f7aec01c730 comp rx=0 tx=0).stop 2023-10-26T05:51:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 -- 172.21.15.71:0/2104292797 shutdown_connections 2023-10-26T05:51:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 --2- 172.21.15.71:0/2104292797 >> v2:172.21.15.86:3300/0 conn(0x7f7afc0cd880 0x7f7afc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 --2- 172.21.15.71:0/2104292797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15b3a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 -- 172.21.15.71:0/2104292797 >> 172.21.15.71:0/2104292797 conn(0x7f7afc0cdd80 msgr2=0x7f7afc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:04.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 -- 172.21.15.71:0/2104292797 shutdown_connections 2023-10-26T05:51:04.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.865+0000 7f7b04337640 1 -- 172.21.15.71:0/2104292797 wait complete. 2023-10-26T05:51:04.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 Processor -- start 2023-10-26T05:51:04.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- start start 2023-10-26T05:51:04.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc0cd880 0x7f7afc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc0d0470 0x7f7afc151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1512b0 con 0x7f7afc0d0850 2023-10-26T05:51:04.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc1f5c10 con 0x7f7afc0cd880 2023-10-26T05:51:04.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc151490 con 0x7f7afc0d0470 2023-10-26T05:51:04.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60784/0 (socket says 172.21.15.71:60784) 2023-10-26T05:51:04.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 -- 172.21.15.71:0/1198755673 learned_addr learned my addr 172.21.15.71:0/1198755673 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:04.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b018d2640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc0d0470 0x7f7afc151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b020d3640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc0cd880 0x7f7afc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 -- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc0d0470 msgr2=0x7f7afc151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc0d0470 0x7f7afc151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 -- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc0cd880 msgr2=0x7f7afc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:04.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc0cd880 0x7f7afc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:04.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 -- 172.21.15.71:0/1198755673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec0094a0 con 0x7f7afc0d0850 2023-10-26T05:51:04.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b028d4640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15a470 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7af400ccb0 tx=0x7f7af400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:04.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004b30 con 0x7f7afc0d0850 2023-10-26T05:51:04.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7af4004da0 con 0x7f7afc0d0850 2023-10-26T05:51:04.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc151770 con 0x7f7afc0d0850 2023-10-26T05:51:04.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4005280 con 0x7f7afc0d0850 2023-10-26T05:51:04.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc152aa0 con 0x7f7afc0d0850 2023-10-26T05:51:04.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.869+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc066b10 con 0x7f7afc0d0850 2023-10-26T05:51:04.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.873+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7af40053e0 con 0x7f7afc0d0850 2023-10-26T05:51:04.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.873+0000 7f7af2ffd640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ae0066470 0x7f7ae0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:04.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.873+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7af40847a0 con 0x7f7afc0d0850 2023-10-26T05:51:04.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.873+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7f7ae006dfa0 con 0x7f7afc0d0850 2023-10-26T05:51:04.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.873+0000 7f7b020d3640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ae0066470 0x7f7ae0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:04.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.873+0000 7f7b020d3640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ae0066470 0x7f7ae0068930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7af8007fd0 tx=0x7f7af8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:04.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:04.877+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7af401d070 con 0x7f7afc0d0850 2023-10-26T05:51:05.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.049+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7afc151c90 con 0x7f7afc0d0850 2023-10-26T05:51:05.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.049+0000 7f7af2ffd640 1 -- 172.21.15.71:0/1198755673 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f7afc151c90 con 0x7f7afc0d0850 2023-10-26T05:51:05.055 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:05.055 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:51:05.055 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:51:05.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ae0066470 msgr2=0x7f7ae0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ae0066470 0x7f7ae0068930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7af8007fd0 tx=0x7f7af8009040 comp rx=0 tx=0).stop 2023-10-26T05:51:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 msgr2=0x7f7afc15a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15a470 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7af400ccb0 tx=0x7f7af400a620 comp rx=0 tx=0).stop 2023-10-26T05:51:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 shutdown_connections 2023-10-26T05:51:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7ae0066470 0x7f7ae0068930 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7afc0d0850 0x7f7afc15a470 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7afc0d0470 0x7f7afc151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 --2- 172.21.15.71:0/1198755673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7afc0cd880 0x7f7afc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 >> 172.21.15.71:0/1198755673 conn(0x7f7afc0cdd80 msgr2=0x7f7afc15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 shutdown_connections 2023-10-26T05:51:05.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.053+0000 7f7b04337640 1 -- 172.21.15.71:0/1198755673 wait complete. 2023-10-26T05:51:05.071 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/233349496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9500b2890 msgr2=0x7fc9500caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 --2- 172.21.15.71:0/233349496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9500b2890 0x7fc9500caad0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc95006f190 tx=0x7fc94401c880 comp rx=0 tx=0).stop 2023-10-26T05:51:05.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/233349496 shutdown_connections 2023-10-26T05:51:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 --2- 172.21.15.71:0/233349496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9500b2890 0x7fc9500caad0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 --2- 172.21.15.71:0/233349496 >> v2:172.21.15.86:3300/0 conn(0x7fc95013c2f0 0x7fc9500b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/233349496 >> 172.21.15.71:0/233349496 conn(0x7fc950069770 msgr2=0x7fc950069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/233349496 shutdown_connections 2023-10-26T05:51:05.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/233349496 wait complete. 2023-10-26T05:51:05.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 Processor -- start 2023-10-26T05:51:05.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- start start 2023-10-26T05:51:05.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9500b2890 0x7fc9500add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9500cb010 0x7fc950156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 0x7fc950157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc950157060 con 0x7fc95013c2f0 2023-10-26T05:51:05.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95013b520 con 0x7fc9500cb010 2023-10-26T05:51:05.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95013a2c0 con 0x7fc9500b2890 2023-10-26T05:51:05.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 0x7fc950157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 0x7fc950157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60800/0 (socket says 172.21.15.71:60800) 2023-10-26T05:51:05.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 -- 172.21.15.71:0/1187720381 learned_addr learned my addr 172.21.15.71:0/1187720381 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:05.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc954d71640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9500b2890 0x7fc9500add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc94ffff640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9500cb010 0x7fc950156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 -- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9500b2890 msgr2=0x7fc9500add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9500b2890 0x7fc9500add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 -- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9500cb010 msgr2=0x7fc950156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9500cb010 0x7fc950156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944009560 con 0x7fc95013c2f0 2023-10-26T05:51:05.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc954d71640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9500b2890 0x7fc9500add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:05.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc94ffff640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9500cb010 0x7fc950156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:05.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc955572640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 0x7fc950157ae0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c780 tx=0x7fc94800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:05.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948003ba0 con 0x7fc95013c2f0 2023-10-26T05:51:05.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc94800ce90 con 0x7fc95013c2f0 2023-10-26T05:51:05.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95013a5a0 con 0x7fc95013c2f0 2023-10-26T05:51:05.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800a850 con 0x7fc95013c2f0 2023-10-26T05:51:05.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.189+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95013ab60 con 0x7fc95013c2f0 2023-10-26T05:51:05.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950066b10 con 0x7fc95013c2f0 2023-10-26T05:51:05.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc94800a9b0 con 0x7fc95013c2f0 2023-10-26T05:51:05.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc94d7fa640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc93c066470 0x7fc93c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc948083de0 con 0x7fc95013c2f0 2023-10-26T05:51:05.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=75}) v3 -- 0x7fc93c06dfa0 con 0x7fc95013c2f0 2023-10-26T05:51:05.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc954d71640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc93c066470 0x7fc93c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.193+0000 7fc954d71640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc93c066470 0x7fc93c068930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc940005fd0 tx=0x7fc940005de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:05.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.197+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc9480512f0 con 0x7fc95013c2f0 2023-10-26T05:51:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.205+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 7 ==== osd_map(75..75 src has 1..75) v4 ==== 1484+0+0 (secure 0 0 0) 0x7fc948059e40 con 0x7fc95013c2f0 2023-10-26T05:51:05.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.205+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=76}) v3 -- 0x7fc93c06f150 con 0x7fc95013c2f0 2023-10-26T05:51:05.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.385+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9500721e0 con 0x7fc95013c2f0 2023-10-26T05:51:05.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc94d7fa640 1 -- 172.21.15.71:0/1187720381 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7fc948054910 con 0x7fc95013c2f0 2023-10-26T05:51:05.393 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:05.393 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":25,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:49:56.624598+0000","modified":"2023-10-26T05:49:58.630775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2212616479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2212616479},{"type":"v1","addr":"172.21.15.71:6835","nonce":2212616479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-26T05:51:05.393 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 25 2023-10-26T05:51:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc93c066470 msgr2=0x7fc93c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc93c066470 0x7fc93c068930 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc940005fd0 tx=0x7fc940005de0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 msgr2=0x7fc950157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 0x7fc950157ae0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c780 tx=0x7fc94800cc40 comp rx=0 tx=0).stop 2023-10-26T05:51:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 shutdown_connections 2023-10-26T05:51:05.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc93c066470 0x7fc93c068930 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc95013c2f0 0x7fc950157ae0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9500cb010 0x7fc950156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 --2- 172.21.15.71:0/1187720381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9500b2890 0x7fc9500add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 >> 172.21.15.71:0/1187720381 conn(0x7fc950069770 msgr2=0x7fc950135810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:05.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 shutdown_connections 2023-10-26T05:51:05.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.389+0000 7fc956fd5640 1 -- 172.21.15.71:0/1187720381 wait complete. 2023-10-26T05:51:05.413 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/1685954606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd4141a70 msgr2=0x7f1cd40cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/1685954606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd4141a70 0x7f1cd40cd670 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009900 tx=0x7f1cc401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/1685954606 shutdown_connections 2023-10-26T05:51:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/1685954606 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd40cdbb0 0x7f1cd415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/1685954606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd4141a70 0x7f1cd40cd670 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/1685954606 >> 172.21.15.71:0/1685954606 conn(0x7f1cd40afeb0 msgr2=0x7f1cd40b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/1685954606 shutdown_connections 2023-10-26T05:51:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/1685954606 wait complete. 2023-10-26T05:51:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 Processor -- start 2023-10-26T05:51:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.533+0000 7f1cd8f6b640 1 -- start start 2023-10-26T05:51:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 0x7f1cd4138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd4141a70 0x7f1cd4139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd415c150 0x7f1cd415d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd4139930 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd413a300 con 0x7f1cd415c150 2023-10-26T05:51:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd4161e20 con 0x7f1cd4141a70 2023-10-26T05:51:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 0x7f1cd4138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 0x7f1cd4138590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60826/0 (socket says 172.21.15.71:60826) 2023-10-26T05:51:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 -- 172.21.15.71:0/2112249980 learned_addr learned my addr 172.21.15.71:0/2112249980 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccbfff640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd4141a70 0x7f1cd4139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 -- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd4141a70 msgr2=0x7f1cd4139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd2d9d640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd415c150 0x7f1cd415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd4141a70 0x7f1cd4139dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 -- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd415c150 msgr2=0x7f1cd415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd415c150 0x7f1cd415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 -- 172.21.15.71:0/2112249980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4009560 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccbfff640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd4141a70 0x7f1cd4139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd259c640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 0x7f1cd4138590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009850 tx=0x7f1cc4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4004430 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1cc4005030 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd415dd70 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4005420 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd41623a0 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd4066d00 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1cc40b4070 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccaffd640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cb4066470 0x7f1cb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cc411e3d0 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.537+0000 7f1ccbfff640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cb4066470 0x7f1cb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.541+0000 7f1ccbfff640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cb4066470 0x7f1cb4068930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005fd0 tx=0x7f1cbc005d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.541+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1cc40eb850 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.701+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cd4159fb0 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.701+0000 7f1ccaffd640 1 -- 172.21.15.71:0/2112249980 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f1cc40eee70 con 0x7f1cd40cdbb0 2023-10-26T05:51:05.709 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:05.709 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":75,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:05.200621+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:49:54.856291+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-26T05:49:55.604098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:05.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cb4066470 msgr2=0x7f1cb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cb4066470 0x7f1cb4068930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005fd0 tx=0x7f1cbc005d40 comp rx=0 tx=0).stop 2023-10-26T05:51:05.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 msgr2=0x7f1cd4138590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 0x7f1cd4138590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009850 tx=0x7f1cc4004040 comp rx=0 tx=0).stop 2023-10-26T05:51:05.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 shutdown_connections 2023-10-26T05:51:05.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cd415c150 0x7f1cd415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cd4141a70 0x7f1cd4139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cb4066470 0x7f1cb4068930 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 --2- 172.21.15.71:0/2112249980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cd40cdbb0 0x7f1cd4138590 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 >> 172.21.15.71:0/2112249980 conn(0x7f1cd40afeb0 msgr2=0x7f1cd415a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 shutdown_connections 2023-10-26T05:51:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.705+0000 7f1cd8f6b640 1 -- 172.21.15.71:0/2112249980 wait complete. 2023-10-26T05:51:05.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:51:05.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/3105467370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac0cabd0 msgr2=0x7ff3ac0b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 --2- 172.21.15.71:0/3105467370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac0cabd0 0x7ff3ac0b1300 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0b0820 tx=0x7ff3a001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/3105467370 shutdown_connections 2023-10-26T05:51:05.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 --2- 172.21.15.71:0/3105467370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac0cabd0 0x7ff3ac0b1300 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 --2- 172.21.15.71:0/3105467370 >> v2:172.21.15.86:3300/0 conn(0x7ff3ac0ca7f0 0x7ff3ac159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/3105467370 >> 172.21.15.71:0/3105467370 conn(0x7ff3ac06b8c0 msgr2=0x7ff3ac06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/3105467370 shutdown_connections 2023-10-26T05:51:05.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/3105467370 wait complete. 2023-10-26T05:51:05.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 Processor -- start 2023-10-26T05:51:05.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- start start 2023-10-26T05:51:05.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0ca7f0 0x7ff3ac131e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ac0cabd0 0x7ff3ac1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 0x7ff3ac1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac138210 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac159b70 con 0x7ff3ac0ca7f0 2023-10-26T05:51:05.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac159e50 con 0x7ff3ac0cabd0 2023-10-26T05:51:05.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 0x7ff3ac1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 0x7ff3ac1586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60836/0 (socket says 172.21.15.71:60836) 2023-10-26T05:51:05.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 -- 172.21.15.71:0/2354797328 learned_addr learned my addr 172.21.15.71:0/2354797328 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:05.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3aa59c640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ac0cabd0 0x7ff3ac1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3aad9d640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0ca7f0 0x7ff3ac131e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 -- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ac0cabd0 msgr2=0x7ff3ac1581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ac0cabd0 0x7ff3ac1581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 -- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0ca7f0 msgr2=0x7ff3ac131e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:05.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0ca7f0 0x7ff3ac131e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:05.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 -- 172.21.15.71:0/2354797328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a0009560 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3aa59c640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ac0cabd0 0x7ff3ac1581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:05.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3aad9d640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0ca7f0 0x7ff3ac131e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:05.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3ab59e640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 0x7ff3ac1586f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c730 tx=0x7ff3a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:05.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4003b80 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3a4003ce0 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac15a940 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400aa50 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.845+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac15a130 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.849+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac066b10 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.849+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff3a400abb0 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.849+0000 7ff3937fe640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff398066470 0x7ff398068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:05.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.849+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3a4083da0 con 0x7ff3ac15ccf0 2023-10-26T05:51:05.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.849+0000 7ff3aad9d640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff398066470 0x7ff398068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:05.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.849+0000 7ff3aad9d640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff398066470 0x7ff398068930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005fd0 tx=0x7ff39c005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:05.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:05.853+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff3a4051220 con 0x7ff3ac15ccf0 2023-10-26T05:51:06.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.025+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff3ac15a790 con 0x7ff3ac15ccf0 2023-10-26T05:51:06.030 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:51:06.025+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:51:06.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.029+0000 7ff3937fe640 1 -- 172.21.15.71:0/2354797328 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff3a400ae60 con 0x7ff3ac15ccf0 2023-10-26T05:51:06.033 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:51:06.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff398066470 msgr2=0x7ff398068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:06.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff398066470 0x7ff398068930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005fd0 tx=0x7ff39c005e60 comp rx=0 tx=0).stop 2023-10-26T05:51:06.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 msgr2=0x7ff3ac1586f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:06.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 0x7ff3ac1586f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c730 tx=0x7ff3a400cbf0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 shutdown_connections 2023-10-26T05:51:06.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff398066470 0x7ff398068930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ac15ccf0 0x7ff3ac1586f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ac0cabd0 0x7ff3ac1581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 --2- 172.21.15.71:0/2354797328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3ac0ca7f0 0x7ff3ac131e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 >> 172.21.15.71:0/2354797328 conn(0x7ff3ac06b8c0 msgr2=0x7ff3ac0ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 shutdown_connections 2023-10-26T05:51:06.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.033+0000 7ff3b1176640 1 -- 172.21.15.71:0/2354797328 wait complete. 2023-10-26T05:51:06.041 INFO:tasks.ceph.mds.c.smithi071.stderr: -7138> 2023-10-26T05:50:03.217+0000 7fdae5c9b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:51:06.055 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:51:06.071 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:51:06.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3aa18c3640 1 -- 172.21.15.71:0/1980532437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a84004d10 msgr2=0x7f3a84005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:06.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/1980532437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a84004d10 0x7f3a84005190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009ee0 tx=0x7f3a9401d610 comp rx=0 tx=0).stop 2023-10-26T05:51:06.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3a9b7fe640 1 -- 172.21.15.71:0/1980532437 reap_dead start 2023-10-26T05:51:06.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3aa18c3640 1 -- 172.21.15.71:0/1980532437 shutdown_connections 2023-10-26T05:51:06.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3aa18c3640 1 -- 172.21.15.71:0/1980532437 >> 172.21.15.71:0/1980532437 conn(0x7f3a9c0aec60 msgr2=0x7f3a9c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:06.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3aa18c3640 1 -- 172.21.15.71:0/1980532437 shutdown_connections 2023-10-26T05:51:06.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.181+0000 7f3aa18c3640 1 -- 172.21.15.71:0/1980532437 wait complete. 2023-10-26T05:51:06.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 Processor -- start 2023-10-26T05:51:06.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- start start 2023-10-26T05:51:06.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84004d10 0x7f3a9c13b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:06.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 0x7f3a9c1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:06.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a9c0b22a0 0x7f3a9c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:06.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c13b5b0 con 0x7f3a9c138660 2023-10-26T05:51:06.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c13a2f0 con 0x7f3a9c0b22a0 2023-10-26T05:51:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c13a590 con 0x7f3a84004d10 2023-10-26T05:51:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 0x7f3a9c1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 0x7f3a9c1501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60868/0 (socket says 172.21.15.71:60868) 2023-10-26T05:51:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 -- 172.21.15.71:0/3577468753 learned_addr learned my addr 172.21.15.71:0/3577468753 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9affd640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84004d10 0x7f3a9c13b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9b7fe640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a9c0b22a0 0x7f3a9c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:06.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 -- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84004d10 msgr2=0x7f3a9c13b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:06.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84004d10 0x7f3a9c13b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 -- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a9c0b22a0 msgr2=0x7f3a9c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:06.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a9c0b22a0 0x7f3a9c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:06.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 -- 172.21.15.71:0/3577468753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c13a870 con 0x7f3a9c138660 2023-10-26T05:51:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a9a7fc640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 0x7f3a9c1501e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e700 tx=0x7f3a9000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90004460 con 0x7f3a9c138660 2023-10-26T05:51:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a9000ee10 con 0x7f3a9c138660 2023-10-26T05:51:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c13ab50 con 0x7f3a9c138660 2023-10-26T05:51:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000caa0 con 0x7f3a9c138660 2023-10-26T05:51:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c138bd0 con 0x7f3a9c138660 2023-10-26T05:51:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.185+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c066b10 con 0x7f3a9c138660 2023-10-26T05:51:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.189+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3a9000cc00 con 0x7f3a9c138660 2023-10-26T05:51:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.189+0000 7f3a83fff640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3a880664a0 0x7f3a88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.189+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3a90084900 con 0x7f3a9c138660 2023-10-26T05:51:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.189+0000 7f3a9affd640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3a880664a0 0x7f3a88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:06.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.189+0000 7f3a9affd640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3a880664a0 0x7f3a88068960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c001600 tx=0x7f3a8c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:06.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.193+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3a90051d20 con 0x7f3a9c138660 2023-10-26T05:51:06.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:06.381+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a9c0721e0 con 0x7f3a9c138660 2023-10-26T05:51:07.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3a83fff640 1 -- 172.21.15.71:0/3577468753 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a900150a0 con 0x7f3a9c138660 2023-10-26T05:51:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3a880664a0 msgr2=0x7f3a88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3a880664a0 0x7f3a88068960 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c001600 tx=0x7f3a8c009040 comp rx=0 tx=0).stop 2023-10-26T05:51:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 msgr2=0x7f3a9c1501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 0x7f3a9c1501e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e700 tx=0x7f3a9000ebc0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 shutdown_connections 2023-10-26T05:51:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a9c0b22a0 0x7f3a9c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3a880664a0 0x7f3a88068960 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a9c138660 0x7f3a9c1501e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 --2- 172.21.15.71:0/3577468753 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a84004d10 0x7f3a9c13b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 >> 172.21.15.71:0/3577468753 conn(0x7f3a9c0aec60 msgr2=0x7f3a9c136b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:07.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 shutdown_connections 2023-10-26T05:51:07.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.033+0000 7f3aa18c3640 1 -- 172.21.15.71:0/3577468753 wait complete. 2023-10-26T05:51:07.063 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:51:07.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 -- 172.21.15.71:0/3687119518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 msgr2=0x7efdf8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:07.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 --2- 172.21.15.71:0/3687119518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efdf8003dd0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00bb10 tx=0x7efdfc01cd30 comp rx=0 tx=0).stop 2023-10-26T05:51:07.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe12349640 1 -- 172.21.15.71:0/3687119518 reap_dead start 2023-10-26T05:51:07.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 -- 172.21.15.71:0/3687119518 shutdown_connections 2023-10-26T05:51:07.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 -- 172.21.15.71:0/3687119518 >> 172.21.15.71:0/3687119518 conn(0x7efe0c0d13f0 msgr2=0x7efe0c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:07.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 -- 172.21.15.71:0/3687119518 shutdown_connections 2023-10-26T05:51:07.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 -- 172.21.15.71:0/3687119518 wait complete. 2023-10-26T05:51:07.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 Processor -- start 2023-10-26T05:51:07.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.237+0000 7efe13dac640 1 -- start start 2023-10-26T05:51:07.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efe0c15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:07.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0c15bcb0 0x7efe0c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:07.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0c156930 0x7efe0c1e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:07.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1591d0 con 0x7efdf80039b0 2023-10-26T05:51:07.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c0c50c0 con 0x7efe0c156930 2023-10-26T05:51:07.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c0c5360 con 0x7efe0c15bcb0 2023-10-26T05:51:07.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efe0c15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:07.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efe0c15f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60884/0 (socket says 172.21.15.71:60884) 2023-10-26T05:51:07.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 -- 172.21.15.71:0/3183172064 learned_addr learned my addr 172.21.15.71:0/3183172064 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:07.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11347640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0c15bcb0 0x7efe0c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:07.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 -- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0c15bcb0 msgr2=0x7efe0c1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:07.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0c15bcb0 0x7efe0c1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 -- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0c156930 msgr2=0x7efe0c1e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0c156930 0x7efe0c1e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 -- 172.21.15.71:0/3183172064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0c5640 con 0x7efdf80039b0 2023-10-26T05:51:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11b48640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efe0c15f490 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ba60 tx=0x7efdfc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0be070 con 0x7efdf80039b0 2023-10-26T05:51:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efdfc004460 con 0x7efdf80039b0 2023-10-26T05:51:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c0c5920 con 0x7efdf80039b0 2023-10-26T05:51:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0052a0 con 0x7efdf80039b0 2023-10-26T05:51:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c15f9d0 con 0x7efdf80039b0 2023-10-26T05:51:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c066b10 con 0x7efdf80039b0 2023-10-26T05:51:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efdfc005400 con 0x7efdf80039b0 2023-10-26T05:51:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe027fc640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efdf0066470 0x7efdf0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efdfc11e030 con 0x7efdf80039b0 2023-10-26T05:51:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11347640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efdf0066470 0x7efdf0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.241+0000 7efe11347640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efdf0066470 0x7efdf0068930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0d1130 tx=0x7efe08006b30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.245+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efdfc0eb3c0 con 0x7efdf80039b0 2023-10-26T05:51:07.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:07.389+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efe0c0721e0 con 0x7efdf80039b0 2023-10-26T05:51:08.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.225+0000 7efe027fc640 1 -- 172.21.15.71:0/3183172064 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7efdfc0ab020 con 0x7efdf80039b0 2023-10-26T05:51:08.229 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:51:08.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efdf0066470 msgr2=0x7efdf0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:08.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efdf0066470 0x7efdf0068930 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0d1130 tx=0x7efe08006b30 comp rx=0 tx=0).stop 2023-10-26T05:51:08.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 msgr2=0x7efe0c15f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:08.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efe0c15f490 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ba60 tx=0x7efdfc0040c0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 shutdown_connections 2023-10-26T05:51:08.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe0c156930 0x7efe0c1e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efe0c15bcb0 0x7efe0c1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efdf0066470 0x7efdf0068930 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 --2- 172.21.15.71:0/3183172064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efdf80039b0 0x7efe0c15f490 secure :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00ba60 tx=0x7efdfc0040c0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 >> 172.21.15.71:0/3183172064 conn(0x7efe0c0d13f0 msgr2=0x7efe0c0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:08.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 shutdown_connections 2023-10-26T05:51:08.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.229+0000 7efe13dac640 1 -- 172.21.15.71:0/3183172064 wait complete. 2023-10-26T05:51:08.255 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:51:08.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 -- 172.21.15.71:0/1130461211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015bbb0 msgr2=0x7f5dd0070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:08.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 --2- 172.21.15.71:0/1130461211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015bbb0 0x7f5dd0070370 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00710a0 tx=0x7f5db801c810 comp rx=0 tx=0).stop 2023-10-26T05:51:08.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 -- 172.21.15.71:0/1130461211 shutdown_connections 2023-10-26T05:51:08.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 --2- 172.21.15.71:0/1130461211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015bbb0 0x7f5dd0070370 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 --2- 172.21.15.71:0/1130461211 >> v2:172.21.15.86:3300/0 conn(0x7f5dd015b7d0 0x7f5dd00acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 -- 172.21.15.71:0/1130461211 >> 172.21.15.71:0/1130461211 conn(0x7f5dd00af9e0 msgr2=0x7f5dd00afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:08.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 -- 172.21.15.71:0/1130461211 shutdown_connections 2023-10-26T05:51:08.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 -- 172.21.15.71:0/1130461211 wait complete. 2023-10-26T05:51:08.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 Processor -- start 2023-10-26T05:51:08.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.385+0000 7f5dd5692640 1 -- start start 2023-10-26T05:51:08.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd006d460 0x7f5dd01400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:08.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 0x7f5dd0152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:08.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd015bbb0 0x7f5dd01525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:08.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0153830 con 0x7f5dd015b7d0 2023-10-26T05:51:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0152c50 con 0x7f5dd006d460 2023-10-26T05:51:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0152f30 con 0x7f5dd015bbb0 2023-10-26T05:51:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 0x7f5dd0152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 0x7f5dd0152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60916/0 (socket says 172.21.15.71:60916) 2023-10-26T05:51:08.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 -- 172.21.15.71:0/2791356107 learned_addr learned my addr 172.21.15.71:0/2791356107 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd006d460 0x7f5dd01400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dcf7fe640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd015bbb0 0x7f5dd01525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 -- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd015bbb0 msgr2=0x7f5dd01525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd015bbb0 0x7f5dd01525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 -- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd006d460 msgr2=0x7f5dd01400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:08.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd006d460 0x7f5dd01400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:08.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 -- 172.21.15.71:0/2791356107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8009560 con 0x7f5dd015b7d0 2023-10-26T05:51:08.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd006d460 0x7f5dd01400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:08.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dcf7fe640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd015bbb0 0x7f5dd01525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:08.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc7fff640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 0x7f5dd0152090 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b0040 tx=0x7f5db80a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:08.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80042c0 con 0x7f5dd015b7d0 2023-10-26T05:51:08.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5db8004510 con 0x7f5dd015b7d0 2023-10-26T05:51:08.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8003840 con 0x7f5dd015b7d0 2023-10-26T05:51:08.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd0153210 con 0x7f5dd015b7d0 2023-10-26T05:51:08.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd015ad40 con 0x7f5dd015b7d0 2023-10-26T05:51:08.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd00717f0 con 0x7f5dd015b7d0 2023-10-26T05:51:08.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5db80b4070 con 0x7f5dd015b7d0 2023-10-26T05:51:08.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc77fe640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:08.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.389+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f5db811d710 con 0x7f5dd015b7d0 2023-10-26T05:51:08.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.393+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:08.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.393+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000a8b0 tx=0x7f5dc0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:08.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.397+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5db80ead00 con 0x7f5dd015b7d0 2023-10-26T05:51:08.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:08.537+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5dd00ccd70 con 0x7f5dd015b7d0 2023-10-26T05:51:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.237+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2791356107 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5db80ee320 con 0x7f5dd015b7d0 2023-10-26T05:51:09.242 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:51:09.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 msgr2=0x7f5db0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000a8b0 tx=0x7f5dc0008040 comp rx=0 tx=0).stop 2023-10-26T05:51:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 msgr2=0x7f5dd0152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 0x7f5dd0152090 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b0040 tx=0x7f5db80a57e0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 shutdown_connections 2023-10-26T05:51:09.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd015bbb0 0x7f5dd01525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd015b7d0 0x7f5dd0152090 secure :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5db80b0040 tx=0x7f5db80a57e0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 --2- 172.21.15.71:0/2791356107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd006d460 0x7f5dd01400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 >> 172.21.15.71:0/2791356107 conn(0x7f5dd00af9e0 msgr2=0x7f5dd0157300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:09.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 shutdown_connections 2023-10-26T05:51:09.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.241+0000 7f5dd5692640 1 -- 172.21.15.71:0/2791356107 wait complete. 2023-10-26T05:51:09.271 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:51:09.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- 172.21.15.71:0/1185303773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a8144bc0 msgr2=0x7f98a815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 --2- 172.21.15.71:0/1185303773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a8144bc0 0x7f98a815f910 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009840 tx=0x7f98a001c790 comp rx=0 tx=0).stop 2023-10-26T05:51:09.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- 172.21.15.71:0/1185303773 shutdown_connections 2023-10-26T05:51:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 --2- 172.21.15.71:0/1185303773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a8144bc0 0x7f98a815f910 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- 172.21.15.71:0/1185303773 >> 172.21.15.71:0/1185303773 conn(0x7f98a80afce0 msgr2=0x7f98a80b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- 172.21.15.71:0/1185303773 shutdown_connections 2023-10-26T05:51:09.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- 172.21.15.71:0/1185303773 wait complete. 2023-10-26T05:51:09.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 Processor -- start 2023-10-26T05:51:09.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- start start 2023-10-26T05:51:09.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a806eb20 0x7f98a81393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8144bc0 0x7f98a8141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 0x7f98a8141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8139910 con 0x7f98a815b8e0 2023-10-26T05:51:09.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8142880 con 0x7f98a8144bc0 2023-10-26T05:51:09.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98aee83640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8141c70 con 0x7f98a806eb20 2023-10-26T05:51:09.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 0x7f98a8141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 0x7f98a8141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60926/0 (socket says 172.21.15.71:60926) 2023-10-26T05:51:09.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 -- 172.21.15.71:0/3737910700 learned_addr learned my addr 172.21.15.71:0/3737910700 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:09.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98a7fff640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8144bc0 0x7f98a8141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98acc1f640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a806eb20 0x7f98a81393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 -- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a806eb20 msgr2=0x7f98a81393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a806eb20 0x7f98a81393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 -- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8144bc0 msgr2=0x7f98a8141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8144bc0 0x7f98a8141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 -- 172.21.15.71:0/3737910700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00094a0 con 0x7f98a815b8e0 2023-10-26T05:51:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98a7fff640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8144bc0 0x7f98a8141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98ad420640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 0x7f98a8141640 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f989800c780 tx=0x7f989800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.385+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898004b70 con 0x7f98a815b8e0 2023-10-26T05:51:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f989800ce90 con 0x7f98a815b8e0 2023-10-26T05:51:09.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a8141f50 con 0x7f98a815b8e0 2023-10-26T05:51:09.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989800a850 con 0x7f98a815b8e0 2023-10-26T05:51:09.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a8142510 con 0x7f98a815b8e0 2023-10-26T05:51:09.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a8066b10 con 0x7f98a815b8e0 2023-10-26T05:51:09.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f989800a9b0 con 0x7f98a815b8e0 2023-10-26T05:51:09.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98a57fa640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9894066470 0x7f9894068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9898083eb0 con 0x7f98a815b8e0 2023-10-26T05:51:09.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98acc1f640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9894066470 0x7f9894068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.389+0000 7f98acc1f640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9894066470 0x7f9894068930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f98a00af000 tx=0x7f98a001afd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:09.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.393+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98980516c0 con 0x7f98a815b8e0 2023-10-26T05:51:09.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.541+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f98a81429f0 con 0x7f98a815b8e0 2023-10-26T05:51:09.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.549+0000 7f98a57fa640 1 -- 172.21.15.71:0/3737910700 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v24) v1 ==== 71+0+0 (secure 0 0 0) 0x7f9898054ce0 con 0x7f98a815b8e0 2023-10-26T05:51:09.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9894066470 msgr2=0x7f9894068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9894066470 0x7f9894068930 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f98a00af000 tx=0x7f98a001afd0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 msgr2=0x7f98a8141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 0x7f98a8141640 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f989800c780 tx=0x7f989800cc40 comp rx=0 tx=0).stop 2023-10-26T05:51:09.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 shutdown_connections 2023-10-26T05:51:09.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9894066470 0x7f9894068930 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98a815b8e0 0x7f98a8141640 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98a8144bc0 0x7f98a8141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 --2- 172.21.15.71:0/3737910700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98a806eb20 0x7f98a81393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 >> 172.21.15.71:0/3737910700 conn(0x7f98a80afce0 msgr2=0x7f98a80adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:09.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 shutdown_connections 2023-10-26T05:51:09.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.553+0000 7f98aee83640 1 -- 172.21.15.71:0/3737910700 wait complete. 2023-10-26T05:51:09.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-10-26T05:51:09.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 -- 172.21.15.71:0/2542415243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 msgr2=0x7f449415b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 --2- 172.21.15.71:0/2542415243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 0x7f449415b4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4484009900 tx=0x7f448401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 -- 172.21.15.71:0/2542415243 shutdown_connections 2023-10-26T05:51:09.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 --2- 172.21.15.71:0/2542415243 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 0x7f449415b4b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 --2- 172.21.15.71:0/2542415243 >> v2:172.21.15.86:3300/0 conn(0x7f449413d420 0x7f44940ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 -- 172.21.15.71:0/2542415243 >> 172.21.15.71:0/2542415243 conn(0x7f4494069770 msgr2=0x7f4494069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:09.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 -- 172.21.15.71:0/2542415243 shutdown_connections 2023-10-26T05:51:09.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 -- 172.21.15.71:0/2542415243 wait complete. 2023-10-26T05:51:09.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 Processor -- start 2023-10-26T05:51:09.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 -- start start 2023-10-26T05:51:09.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449413d420 0x7f4494130280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.697+0000 7f449c01f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 0x7f44941f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 0x7f4494131900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494130860 con 0x7f449413dc20 2023-10-26T05:51:09.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4494130b40 con 0x7f449413d420 2023-10-26T05:51:09.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4494130e20 con 0x7f449413d800 2023-10-26T05:51:09.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 0x7f4494131900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 0x7f4494131900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60942/0 (socket says 172.21.15.71:60942) 2023-10-26T05:51:09.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 -- 172.21.15.71:0/707312506 learned_addr learned my addr 172.21.15.71:0/707312506 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:09.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f44995ba640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 0x7f44941f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f4499dbb640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449413d420 0x7f4494130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 -- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 msgr2=0x7f44941f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 0x7f44941f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 -- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449413d420 msgr2=0x7f4494130280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:09.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449413d420 0x7f4494130280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:09.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 -- 172.21.15.71:0/707312506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009560 con 0x7f449413dc20 2023-10-26T05:51:09.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449a5bc640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 0x7f4494131900 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c730 tx=0x7f448c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:09.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c003b80 con 0x7f449413dc20 2023-10-26T05:51:09.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f448c003ce0 con 0x7f449413dc20 2023-10-26T05:51:09.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4494131100 con 0x7f449413dc20 2023-10-26T05:51:09.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00aa50 con 0x7f449413dc20 2023-10-26T05:51:09.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44941f2be0 con 0x7f449413dc20 2023-10-26T05:51:09.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4494066b10 con 0x7f449413dc20 2023-10-26T05:51:09.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f448c00abb0 con 0x7f449413dc20 2023-10-26T05:51:09.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f448a7fc640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4478066470 0x7f4478068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:09.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f448c083940 con 0x7f449413dc20 2023-10-26T05:51:09.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.701+0000 7f4499dbb640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4478066470 0x7f4478068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:09.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.705+0000 7f4499dbb640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4478066470 0x7f4478068930 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4490009be0 tx=0x7f44900091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:09.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.705+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f448c051170 con 0x7f449413dc20 2023-10-26T05:51:09.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:09.917+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f44940721e0 con 0x7f449413dc20 2023-10-26T05:51:10.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.249+0000 7f448a7fc640 1 -- 172.21.15.71:0/707312506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v213) v1 ==== 134+0+0 (secure 0 0 0) 0x7f448c00ae60 con 0x7f449413dc20 2023-10-26T05:51:10.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4478066470 msgr2=0x7f4478068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4478066470 0x7f4478068930 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4490009be0 tx=0x7f44900091e0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 msgr2=0x7f4494131900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 0x7f4494131900 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c730 tx=0x7f448c00cbf0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 shutdown_connections 2023-10-26T05:51:10.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4478066470 0x7f4478068930 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f449413dc20 0x7f4494131900 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f449413d800 0x7f44941f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 --2- 172.21.15.71:0/707312506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449413d420 0x7f4494130280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 >> 172.21.15.71:0/707312506 conn(0x7f4494069770 msgr2=0x7f449413c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:10.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 shutdown_connections 2023-10-26T05:51:10.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.253+0000 7f449c01f640 1 -- 172.21.15.71:0/707312506 wait complete. 2023-10-26T05:51:10.277 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-26T05:51:10.278 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2023-10-26T05:51:10.278 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T05:51:10.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- 172.21.15.71:0/103864948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40cd340 msgr2=0x7f98d415b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 --2- 172.21.15.71:0/103864948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40cd340 0x7f98d415b8e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009bf0 tx=0x7f98cc01cb50 comp rx=0 tx=0).stop 2023-10-26T05:51:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- 172.21.15.71:0/103864948 shutdown_connections 2023-10-26T05:51:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 --2- 172.21.15.71:0/103864948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40cd340 0x7f98d415b8e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- 172.21.15.71:0/103864948 >> 172.21.15.71:0/103864948 conn(0x7f98d406fed0 msgr2=0x7f98d40702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- 172.21.15.71:0/103864948 shutdown_connections 2023-10-26T05:51:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- 172.21.15.71:0/103864948 wait complete. 2023-10-26T05:51:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 Processor -- start 2023-10-26T05:51:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- start start 2023-10-26T05:51:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d40aaff0 0x7f98d4152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 0x7f98d41543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d40cd340 0x7f98d4146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4141740 con 0x7f98d40ccf60 2023-10-26T05:51:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4145190 con 0x7f98d40aaff0 2023-10-26T05:51:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98dacdc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4145470 con 0x7f98d40cd340 2023-10-26T05:51:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 0x7f98d41543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 0x7f98d41543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60952/0 (socket says 172.21.15.71:60952) 2023-10-26T05:51:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 -- 172.21.15.71:0/4190121332 learned_addr learned my addr 172.21.15.71:0/4190121332 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d8a78640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d40aaff0 0x7f98d4152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d9279640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d40cd340 0x7f98d4146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 -- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d40cd340 msgr2=0x7f98d4146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d40cd340 0x7f98d4146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 -- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d40aaff0 msgr2=0x7f98d4152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d40aaff0 0x7f98d4152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d3fff640 1 -- 172.21.15.71:0/4190121332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009820 con 0x7f98d40ccf60 2023-10-26T05:51:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d9279640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d40cd340 0x7f98d4146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:51:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.401+0000 7f98d8a78640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d40aaff0 0x7f98d4152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:51:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d3fff640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 0x7f98d41543c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c3c0 tx=0x7f98c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00052d0 con 0x7f98d40ccf60 2023-10-26T05:51:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98c0003600 con 0x7f98d40ccf60 2023-10-26T05:51:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d4145750 con 0x7f98d40ccf60 2023-10-26T05:51:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0003a10 con 0x7f98d40ccf60 2023-10-26T05:51:10.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d4145d10 con 0x7f98d40ccf60 2023-10-26T05:51:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d4145f30 con 0x7f98d40ccf60 2023-10-26T05:51:10.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f98c0003b70 con 0x7f98d40ccf60 2023-10-26T05:51:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d17fa640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98b8066470 0x7f98b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f98c0083290 con 0x7f98d40ccf60 2023-10-26T05:51:10.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d8a78640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98b8066470 0x7f98b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.405+0000 7f98d8a78640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98b8066470 0x7f98b8068930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f98c8007fd0 tx=0x7f98c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:10.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.409+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98c0050aa0 con 0x7f98d40ccf60 2023-10-26T05:51:10.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.565+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98d4066d00 con 0x7f98d40ccf60 2023-10-26T05:51:10.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.565+0000 7f98d17fa640 1 -- 172.21.15.71:0/4190121332 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12902 (secure 0 0 0) 0x7f98c00540c0 con 0x7f98d40ccf60 2023-10-26T05:51:10.571 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:10.572 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":78,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:09.227385+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/2212616479":"2023-10-27T05:51:06.028002+0000","172.21.15.71:6834/2212616479":"2023-10-27T05:51:06.028002+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:10.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98b8066470 msgr2=0x7f98b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98b8066470 0x7f98b8068930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f98c8007fd0 tx=0x7f98c8009040 comp rx=0 tx=0).stop 2023-10-26T05:51:10.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 msgr2=0x7f98d41543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 0x7f98d41543c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f98c000c3c0 tx=0x7f98c000c880 comp rx=0 tx=0).stop 2023-10-26T05:51:10.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 shutdown_connections 2023-10-26T05:51:10.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d40cd340 0x7f98d4146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98b8066470 0x7f98b8068930 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ccf60 0x7f98d41543c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 --2- 172.21.15.71:0/4190121332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d40aaff0 0x7f98d4152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 >> 172.21.15.71:0/4190121332 conn(0x7f98d406fed0 msgr2=0x7f98d415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 shutdown_connections 2023-10-26T05:51:10.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.569+0000 7f98dacdc640 1 -- 172.21.15.71:0/4190121332 wait complete. 2023-10-26T05:51:10.591 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:10.592 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:10.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/3721321777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 msgr2=0x7f6e00003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/3721321777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e00003da0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c260 tx=0x7f6e0801cf50 comp rx=0 tx=0).stop 2023-10-26T05:51:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1c89c640 1 -- 172.21.15.71:0/3721321777 reap_dead start 2023-10-26T05:51:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/3721321777 shutdown_connections 2023-10-26T05:51:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/3721321777 >> 172.21.15.71:0/3721321777 conn(0x7f6e180aec60 msgr2=0x7f6e180af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:10.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/3721321777 shutdown_connections 2023-10-26T05:51:10.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/3721321777 wait complete. 2023-10-26T05:51:10.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 Processor -- start 2023-10-26T05:51:10.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- start start 2023-10-26T05:51:10.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e18147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e181475a0 0x7f6e18153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1814f430 0x7f6e18146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1814e390 con 0x7f6e00003980 2023-10-26T05:51:10.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1814e670 con 0x7f6e181475a0 2023-10-26T05:51:10.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1e2ff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1814e950 con 0x7f6e1814f430 2023-10-26T05:51:10.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e18147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e18147aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60970/0 (socket says 172.21.15.71:60970) 2023-10-26T05:51:10.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 -- 172.21.15.71:0/74437430 learned_addr learned my addr 172.21.15.71:0/74437430 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:10.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e1c89c640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1814f430 0x7f6e18146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e177fe640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e181475a0 0x7f6e18153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 -- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1814f430 msgr2=0x7f6e18146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1814f430 0x7f6e18146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 -- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e181475a0 msgr2=0x7f6e18153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e181475a0 0x7f6e18153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 -- 172.21.15.71:0/74437430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1814ec30 con 0x7f6e00003980 2023-10-26T05:51:10.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.721+0000 7f6e17fff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e18147aa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6e08001940 tx=0x7f6e080066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:10.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e080b1070 con 0x7f6e00003980 2023-10-26T05:51:10.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e08003600 con 0x7f6e00003980 2023-10-26T05:51:10.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1814ef10 con 0x7f6e00003980 2023-10-26T05:51:10.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e080b4040 con 0x7f6e00003980 2023-10-26T05:51:10.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e18154640 con 0x7f6e00003980 2023-10-26T05:51:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e18066b10 con 0x7f6e00003980 2023-10-26T05:51:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e080039d0 con 0x7f6e00003980 2023-10-26T05:51:10.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e14ff9640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e04066470 0x7f6e04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6e0811e1c0 con 0x7f6e00003980 2023-10-26T05:51:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e177fe640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e04066470 0x7f6e04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.725+0000 7f6e177fe640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e04066470 0x7f6e04068930 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005b70 tx=0x7f6e0c005a80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.729+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e080eb9d0 con 0x7f6e00003980 2023-10-26T05:51:10.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e1814f0a0 con 0x7f6e00003980 2023-10-26T05:51:10.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e14ff9640 1 -- 172.21.15.71:0/74437430 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e080eeff0 con 0x7f6e00003980 2023-10-26T05:51:10.890 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e04066470 msgr2=0x7f6e04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e04066470 0x7f6e04068930 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c005b70 tx=0x7f6e0c005a80 comp rx=0 tx=0).stop 2023-10-26T05:51:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 msgr2=0x7f6e18147aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e18147aa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6e08001940 tx=0x7f6e080066c0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 shutdown_connections 2023-10-26T05:51:10.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1814f430 0x7f6e18146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e181475a0 0x7f6e18153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e04066470 0x7f6e04068930 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 --2- 172.21.15.71:0/74437430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e00003980 0x7f6e18147aa0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:10.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 >> 172.21.15.71:0/74437430 conn(0x7f6e180aec60 msgr2=0x7f6e18156200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:10.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 shutdown_connections 2023-10-26T05:51:10.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:10.885+0000 7f6e1e2ff640 1 -- 172.21.15.71:0/74437430 wait complete. 2023-10-26T05:51:10.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-26T05:51:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/2623565561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc158870 msgr2=0x7f07bc159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 --2- 172.21.15.71:0/2623565561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc158870 0x7f07bc159730 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009bc0 tx=0x7f07b401cac0 comp rx=0 tx=0).stop 2023-10-26T05:51:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/2623565561 shutdown_connections 2023-10-26T05:51:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 --2- 172.21.15.71:0/2623565561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc158870 0x7f07bc159730 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 --2- 172.21.15.71:0/2623565561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07bc06c720 0x7f07bc06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/2623565561 >> 172.21.15.71:0/2623565561 conn(0x7f07bc0cd830 msgr2=0x7f07bc0cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/2623565561 shutdown_connections 2023-10-26T05:51:11.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/2623565561 wait complete. 2023-10-26T05:51:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 Processor -- start 2023-10-26T05:51:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- start start 2023-10-26T05:51:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07bc06c720 0x7f07bc1344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07bc158870 0x7f07bc148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 0x7f07bc1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc134a10 con 0x7f07bc1492f0 2023-10-26T05:51:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1497a0 con 0x7f07bc158870 2023-10-26T05:51:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc14f490 con 0x7f07bc06c720 2023-10-26T05:51:11.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 0x7f07bc1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:11.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 0x7f07bc1546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60982/0 (socket says 172.21.15.71:60982) 2023-10-26T05:51:11.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 -- 172.21.15.71:0/1471280289 learned_addr learned my addr 172.21.15.71:0/1471280289 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:11.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c0ec0640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07bc06c720 0x7f07bc1344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07bbfff640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07bc158870 0x7f07bc148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 -- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07bc06c720 msgr2=0x7f07bc1344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07bc06c720 0x7f07bc1344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:11.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 -- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07bc158870 msgr2=0x7f07bc148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07bc158870 0x7f07bc148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 -- 172.21.15.71:0/1471280289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009820 con 0x7f07bc1492f0 2023-10-26T05:51:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07bbfff640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07bc158870 0x7f07bc148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c16c1640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 0x7f07bc1546d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009bc0 tx=0x7f07b4009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:11.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4004240 con 0x7f07bc1492f0 2023-10-26T05:51:11.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07b4004490 con 0x7f07bc1492f0 2023-10-26T05:51:11.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc149a30 con 0x7f07bc1492f0 2023-10-26T05:51:11.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4004860 con 0x7f07bc1492f0 2023-10-26T05:51:11.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.033+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc149ff0 con 0x7f07bc1492f0 2023-10-26T05:51:11.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.037+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc066b10 con 0x7f07bc1492f0 2023-10-26T05:51:11.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.037+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07b40b4040 con 0x7f07bc1492f0 2023-10-26T05:51:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.037+0000 7f07b97fa640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a0066470 0x7f07a0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.037+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07b411eb40 con 0x7f07bc1492f0 2023-10-26T05:51:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.037+0000 7f07c0ec0640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a0066470 0x7f07a0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.037+0000 7f07c0ec0640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a0066470 0x7f07a0068930 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07b0000f70 tx=0x7f07b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:11.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.041+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07b40ec2c0 con 0x7f07bc1492f0 2023-10-26T05:51:11.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:11.253+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f07bc066d00 con 0x7f07bc1492f0 2023-10-26T05:51:12.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.257+0000 7f07b97fa640 1 -- 172.21.15.71:0/1471280289 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v215) v1 ==== 136+0+0 (secure 0 0 0) 0x7f07b40ef8e0 con 0x7f07bc1492f0 2023-10-26T05:51:12.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a0066470 msgr2=0x7f07a0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a0066470 0x7f07a0068930 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07b0000f70 tx=0x7f07b0009040 comp rx=0 tx=0).stop 2023-10-26T05:51:12.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 msgr2=0x7f07bc1546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 0x7f07bc1546d0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009bc0 tx=0x7f07b4009280 comp rx=0 tx=0).stop 2023-10-26T05:51:12.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 shutdown_connections 2023-10-26T05:51:12.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a0066470 0x7f07a0068930 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07bc1492f0 0x7f07bc1546d0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07bc158870 0x7f07bc148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 --2- 172.21.15.71:0/1471280289 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07bc06c720 0x7f07bc1344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 >> 172.21.15.71:0/1471280289 conn(0x7f07bc0cd830 msgr2=0x7f07bc1576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:12.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 shutdown_connections 2023-10-26T05:51:12.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.261+0000 7f07c3124640 1 -- 172.21.15.71:0/1471280289 wait complete. 2023-10-26T05:51:12.289 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T05:51:12.429 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_objectstore":"memstore"} 2023-10-26T05:51:12.429 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T05:51:12.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:51:12.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/1216154462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd2c003fe0 msgr2=0x7fbd2c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 --2- 172.21.15.71:0/1216154462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd2c003fe0 0x7fbd2c0047d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400bb10 tx=0x7fbd3401cd80 comp rx=0 tx=0).stop 2023-10-26T05:51:12.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd42d9d640 1 -- 172.21.15.71:0/1216154462 reap_dead start 2023-10-26T05:51:12.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/1216154462 shutdown_connections 2023-10-26T05:51:12.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/1216154462 >> 172.21.15.71:0/1216154462 conn(0x7fbd44069770 msgr2=0x7fbd44069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:12.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/1216154462 shutdown_connections 2023-10-26T05:51:12.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/1216154462 wait complete. 2023-10-26T05:51:12.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 Processor -- start 2023-10-26T05:51:12.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- start start 2023-10-26T05:51:12.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd2c003fe0 0x7fbd4413ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 0x7fbd441425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd44158540 0x7fbd441589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44158f30 con 0x7fbd44140bf0 2023-10-26T05:51:12.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd441352b0 con 0x7fbd2c003fe0 2023-10-26T05:51:12.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd441f2f20 con 0x7fbd44158540 2023-10-26T05:51:12.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 0x7fbd441425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 0x7fbd441425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60512/0 (socket says 172.21.15.71:60512) 2023-10-26T05:51:12.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 -- 172.21.15.71:0/3312244582 learned_addr learned my addr 172.21.15.71:0/3312244582 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:12.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd4259c640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd2c003fe0 0x7fbd4413ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd42d9d640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd44158540 0x7fbd441589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 -- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd44158540 msgr2=0x7fbd441589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd44158540 0x7fbd441589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 -- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd2c003fe0 msgr2=0x7fbd4413ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd2c003fe0 0x7fbd4413ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 -- 172.21.15.71:0/3312244582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd441f3170 con 0x7fbd44140bf0 2023-10-26T05:51:12.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd4259c640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd2c003fe0 0x7fbd4413ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:12.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd41d9b640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 0x7fbd441425c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fbd3401cbb0 tx=0x7fbd340040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:12.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340be070 con 0x7fbd44140bf0 2023-10-26T05:51:12.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd340044a0 con 0x7fbd44140bf0 2023-10-26T05:51:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340052a0 con 0x7fbd44140bf0 2023-10-26T05:51:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd441f3450 con 0x7fbd44140bf0 2023-10-26T05:51:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.553+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd441f3a10 con 0x7fbd44140bf0 2023-10-26T05:51:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.557+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd44066b10 con 0x7fbd44140bf0 2023-10-26T05:51:12.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.557+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbd34005400 con 0x7fbd44140bf0 2023-10-26T05:51:12.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.557+0000 7fbd32ffd640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd24066470 0x7fbd24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.557+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbd3411dd60 con 0x7fbd44140bf0 2023-10-26T05:51:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.557+0000 7fbd4259c640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd24066470 0x7fbd24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.557+0000 7fbd4259c640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd24066470 0x7fbd24068930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e950 tx=0x7fbd38001440 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:12.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.561+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbd340eb560 con 0x7fbd44140bf0 2023-10-26T05:51:12.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.709+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fbd440721e0 con 0x7fbd44140bf0 2023-10-26T05:51:12.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.709+0000 7fbd32ffd640 1 -- 172.21.15.71:0/3312244582 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v24) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fbd340ab080 con 0x7fbd44140bf0 2023-10-26T05:51:12.716 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:51:12.716 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:51:12.716 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:51:12.716 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:51:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:51:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:51:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:51:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:51:12.717 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:51:12.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:51:12.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:51:12.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:51:12.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:51:12.718 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:51:12.719 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:51:12.719 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:51:12.719 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:51:12.719 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:51:12.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd24066470 msgr2=0x7fbd24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd24066470 0x7fbd24068930 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e950 tx=0x7fbd38001440 comp rx=0 tx=0).stop 2023-10-26T05:51:12.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 msgr2=0x7fbd441425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 0x7fbd441425c0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fbd3401cbb0 tx=0x7fbd340040e0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 shutdown_connections 2023-10-26T05:51:12.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd44158540 0x7fbd441589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd24066470 0x7fbd24068930 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd44140bf0 0x7fbd441425c0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 --2- 172.21.15.71:0/3312244582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd2c003fe0 0x7fbd4413ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 >> 172.21.15.71:0/3312244582 conn(0x7fbd44069770 msgr2=0x7fbd4415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:12.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 shutdown_connections 2023-10-26T05:51:12.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.713+0000 7fbd48e62640 1 -- 172.21.15.71:0/3312244582 wait complete. 2023-10-26T05:51:12.736 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:51:12.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1340671650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 msgr2=0x7f07c80ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1340671650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 0x7f07c80ca6d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009900 tx=0x7f07c001c890 comp rx=0 tx=0).stop 2023-10-26T05:51:12.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1340671650 shutdown_connections 2023-10-26T05:51:12.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1340671650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 0x7f07c80ca6d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1340671650 >> v2:172.21.15.86:3300/0 conn(0x7f07c815bb90 0x7f07c80b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1340671650 >> 172.21.15.71:0/1340671650 conn(0x7f07c80aec60 msgr2=0x7f07c80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:12.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1340671650 shutdown_connections 2023-10-26T05:51:12.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.857+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1340671650 wait complete. 2023-10-26T05:51:12.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 Processor -- start 2023-10-26T05:51:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- start start 2023-10-26T05:51:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8070dc0 0x7f07c81380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 0x7f07c8148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 0x7f07c80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8157290 con 0x7f07c815bb90 2023-10-26T05:51:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8156d20 con 0x7f07c8070dc0 2023-10-26T05:51:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c81375e0 con 0x7f07c815bf70 2023-10-26T05:51:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 0x7f07c8148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 0x7f07c8148a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60520/0 (socket says 172.21.15.71:60520) 2023-10-26T05:51:12.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 -- 172.21.15.71:0/1598454286 learned_addr learned my addr 172.21.15.71:0/1598454286 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:12.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c6ffd640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8070dc0 0x7f07c81380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c77fe640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 0x7f07c80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 -- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 msgr2=0x7f07c80afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 0x7f07c80afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 -- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8070dc0 msgr2=0x7f07c81380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8070dc0 0x7f07c81380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 -- 172.21.15.71:0/1598454286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0009560 con 0x7f07c815bb90 2023-10-26T05:51:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c6ffd640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8070dc0 0x7f07c81380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:51:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c67fc640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 0x7f07c8148a60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c4d0 tx=0x7f07b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:12.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b00052d0 con 0x7f07c815bb90 2023-10-26T05:51:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07b0005430 con 0x7f07c815bb90 2023-10-26T05:51:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c81378c0 con 0x7f07c815bb90 2023-10-26T05:51:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b0012890 con 0x7f07c815bb90 2023-10-26T05:51:12.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c8138780 con 0x7f07c815bb90 2023-10-26T05:51:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c8066b10 con 0x7f07c815bb90 2023-10-26T05:51:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07b0003600 con 0x7f07c815bb90 2023-10-26T05:51:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07bf7fe640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a8066470 0x7f07a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07b0083690 con 0x7f07c815bb90 2023-10-26T05:51:12.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.861+0000 7f07c6ffd640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a8066470 0x7f07a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:12.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.865+0000 7f07c6ffd640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a8066470 0x7f07a8068930 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07c80cca80 tx=0x7f07b8009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:12.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:12.865+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07b0050e00 con 0x7f07c815bb90 2023-10-26T05:51:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.009+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f07c8071220 con 0x7f07c815bb90 2023-10-26T05:51:13.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.013+0000 7f07bf7fe640 1 -- 172.21.15.71:0/1598454286 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v25)=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f07c8071220 con 0x7f07c815bb90 2023-10-26T05:51:13.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a8066470 msgr2=0x7f07a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a8066470 0x7f07a8068930 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07c80cca80 tx=0x7f07b8009500 comp rx=0 tx=0).stop 2023-10-26T05:51:13.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 msgr2=0x7f07c8148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 0x7f07c8148a60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c4d0 tx=0x7f07b000c990 comp rx=0 tx=0).stop 2023-10-26T05:51:13.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 shutdown_connections 2023-10-26T05:51:13.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07c815bf70 0x7f07c80afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07a8066470 0x7f07a8068930 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c815bb90 0x7f07c8148a60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 --2- 172.21.15.71:0/1598454286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c8070dc0 0x7f07c81380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 >> 172.21.15.71:0/1598454286 conn(0x7f07c80aec60 msgr2=0x7f07c80ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:13.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 shutdown_connections 2023-10-26T05:51:13.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.017+0000 7f07cd7b7640 1 -- 172.21.15.71:0/1598454286 wait complete. 2023-10-26T05:51:13.037 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:51:13.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:51:13.081 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:51:13.136 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:51:13.137 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:51:13.137 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:13.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:51:13.143 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:51:13.143 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:13.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:51:13.149 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:51:13.150 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:13.190 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:51:13.191 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:51:13.191 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:13.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:51:13.197 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:51:13.197 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 -- 172.21.15.71:0/1833825854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 msgr2=0x7f75d00b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1833825854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d00b04b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f75d00afd40 tx=0x7f75c801c870 comp rx=0 tx=0).stop 2023-10-26T05:51:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 -- 172.21.15.71:0/1833825854 shutdown_connections 2023-10-26T05:51:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1833825854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d00b04b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1833825854 >> v2:172.21.15.86:3300/0 conn(0x7f75d0070fe0 0x7f75d0071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1833825854 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d0140cc0 0x7f75d015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 -- 172.21.15.71:0/1833825854 >> 172.21.15.71:0/1833825854 conn(0x7f75d0069770 msgr2=0x7f75d0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:13.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 -- 172.21.15.71:0/1833825854 shutdown_connections 2023-10-26T05:51:13.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.353+0000 7f75d70d5640 1 -- 172.21.15.71:0/1833825854 wait complete. 2023-10-26T05:51:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 Processor -- start 2023-10-26T05:51:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- start start 2023-10-26T05:51:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d0070fe0 0x7f75d0139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d0161c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d0140cc0 0x7f75d0069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d0163840 con 0x7f75d00b0040 2023-10-26T05:51:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d015aa70 con 0x7f75d0070fe0 2023-10-26T05:51:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d015a1d0 con 0x7f75d0140cc0 2023-10-26T05:51:13.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d0161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d0161c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60552/0 (socket says 172.21.15.71:60552) 2023-10-26T05:51:13.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 -- 172.21.15.71:0/1514402766 learned_addr learned my addr 172.21.15.71:0/1514402766 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:13.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d5672640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d0140cc0 0x7f75d0069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 -- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d0140cc0 msgr2=0x7f75d0069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d0140cc0 0x7f75d0069b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d4e71640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d0070fe0 0x7f75d0139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 -- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d0070fe0 msgr2=0x7f75d0139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d0070fe0 0x7f75d0139a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 -- 172.21.15.71:0/1514402766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8009560 con 0x7f75d00b0040 2023-10-26T05:51:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cffff640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d0161c90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c970 tx=0x7f75b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8007bb0 con 0x7f75d00b0040 2023-10-26T05:51:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75b8007e00 con 0x7f75d00b0040 2023-10-26T05:51:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d015a4b0 con 0x7f75d00b0040 2023-10-26T05:51:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8005280 con 0x7f75d00b0040 2023-10-26T05:51:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d0139f90 con 0x7f75d00b0040 2023-10-26T05:51:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.357+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d0066b10 con 0x7f75d00b0040 2023-10-26T05:51:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.361+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f75b80053e0 con 0x7f75d00b0040 2023-10-26T05:51:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.361+0000 7f75cd7fa640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75b00664a0 0x7f75b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.361+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f75b8083f80 con 0x7f75d00b0040 2023-10-26T05:51:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.361+0000 7f75d4e71640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75b00664a0 0x7f75b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.361+0000 7f75d4e71640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75b00664a0 0x7f75b0068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f75c0005c50 tx=0x7f75c0005f50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.365+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f75b80b5090 con 0x7f75d00b0040 2023-10-26T05:51:13.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.557+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75d01635a0 con 0x7f75d00b0040 2023-10-26T05:51:13.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.557+0000 7f75cd7fa640 1 -- 172.21.15.71:0/1514402766 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3690 (secure 0 0 0) 0x7f75d01635a0 con 0x7f75d00b0040 2023-10-26T05:51:13.563 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:13.563 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5030,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/683921990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":683921990},{"type":"v1","addr":"172.21.15.86:6835","nonce":683921990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5033,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/174975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":174975565},{"type":"v1","addr":"172.21.15.86:6837","nonce":174975565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6261,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/999895073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":999895073},{"type":"v1","addr":"172.21.15.71:6833","nonce":999895073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":6576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3996916672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3996916672},{"type":"v1","addr":"172.21.15.71:6835","nonce":3996916672}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-10-26T05:51:13.563 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 27 2023-10-26T05:51:13.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75b00664a0 msgr2=0x7f75b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75b00664a0 0x7f75b0068960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f75c0005c50 tx=0x7f75c0005f50 comp rx=0 tx=0).stop 2023-10-26T05:51:13.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 msgr2=0x7f75d0161c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d0161c90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c970 tx=0x7f75b800ce30 comp rx=0 tx=0).stop 2023-10-26T05:51:13.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 shutdown_connections 2023-10-26T05:51:13.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75d0140cc0 0x7f75d0069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75b00664a0 0x7f75b0068960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d00b0040 0x7f75d0161c90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 --2- 172.21.15.71:0/1514402766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d0070fe0 0x7f75d0139a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 >> 172.21.15.71:0/1514402766 conn(0x7f75d0069770 msgr2=0x7f75d00b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:13.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 shutdown_connections 2023-10-26T05:51:13.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.561+0000 7f75d70d5640 1 -- 172.21.15.71:0/1514402766 wait complete. 2023-10-26T05:51:13.582 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:51:13.583 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:51:13.583 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:51:13.583 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:51:13.583 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:51:13.584 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:51:13.584 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:51:13.584 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:51:13.584 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:51:13.585 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:51:13.585 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:51:13.585 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:51:13.585 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:51:13.586 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:51:13.586 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:51:13.586 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:51:13.594 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:51:13.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:51:13.598 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:51:13.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:51:13.600 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:51:13.600 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:51:13.602 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:51:13.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:51:13.641 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:51:13.641 DEBUG:teuthology.parallel:result is None 2023-10-26T05:51:13.642 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:51:13.643 DEBUG:teuthology.parallel:result is None 2023-10-26T05:51:13.647 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:51:13.648 DEBUG:teuthology.parallel:result is None 2023-10-26T05:51:13.649 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:51:13.649 DEBUG:teuthology.parallel:result is None 2023-10-26T05:51:13.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:51:13.701 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:51:13.702 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:51:13.716 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:51:13.723 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:51:13.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/2059790875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cbc004d10 msgr2=0x7f5cbc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/2059790875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cbc004d10 0x7f5cbc005190 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009ee0 tx=0x7f5ccc01d5d0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 -- 172.21.15.71:0/2059790875 reap_dead start 2023-10-26T05:51:13.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/2059790875 shutdown_connections 2023-10-26T05:51:13.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/2059790875 >> 172.21.15.71:0/2059790875 conn(0x7f5cd40d12d0 msgr2=0x7f5cd40d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:13.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/2059790875 shutdown_connections 2023-10-26T05:51:13.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/2059790875 wait complete. 2023-10-26T05:51:13.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 Processor -- start 2023-10-26T05:51:13.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- start start 2023-10-26T05:51:13.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cd414eb50 0x7f5cd415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd40cad60 0x7f5cd415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 0x7f5cd40c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd415ae40 con 0x7f5cd415a9c0 2023-10-26T05:51:13.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd415c980 con 0x7f5cd40cad60 2023-10-26T05:51:13.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd415d7c0 con 0x7f5cd414eb50 2023-10-26T05:51:13.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 0x7f5cd40c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 0x7f5cd40c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60578/0 (socket says 172.21.15.71:60578) 2023-10-26T05:51:13.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 -- 172.21.15.71:0/912003201 learned_addr learned my addr 172.21.15.71:0/912003201 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:13.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd37fe640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd40cad60 0x7f5cd415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd3fff640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cd414eb50 0x7f5cd415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 -- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cd414eb50 msgr2=0x7f5cd415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cd414eb50 0x7f5cd415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 -- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd40cad60 msgr2=0x7f5cd415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:13.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd40cad60 0x7f5cd415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:13.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 -- 172.21.15.71:0/912003201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd415dc50 con 0x7f5cd415a9c0 2023-10-26T05:51:13.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd8d57640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 0x7f5cd40c83a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009ee0 tx=0x7f5ccc009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:13.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0b6070 con 0x7f5cd415a9c0 2023-10-26T05:51:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5ccc009200 con 0x7f5cd415a9c0 2023-10-26T05:51:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd415cbb0 con 0x7f5cd415a9c0 2023-10-26T05:51:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0a9be0 con 0x7f5cd415a9c0 2023-10-26T05:51:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.765+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd415d170 con 0x7f5cd415a9c0 2023-10-26T05:51:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.769+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd4066b10 con 0x7f5cd415a9c0 2023-10-26T05:51:13.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.769+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5ccc0a9d40 con 0x7f5cd415a9c0 2023-10-26T05:51:13.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.769+0000 7f5cd0ff9640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5cc0066470 0x7f5cc0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:13.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.769+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5ccc11ebf0 con 0x7f5cd415a9c0 2023-10-26T05:51:13.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.769+0000 7f5cd3fff640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5cc0066470 0x7f5cc0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:13.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.769+0000 7f5cd3fff640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5cc0066470 0x7f5cc0068930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4003680 tx=0x7f5cc4006730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:13.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.773+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5ccc0ec570 con 0x7f5cd415a9c0 2023-10-26T05:51:13.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:13.925+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5cd415ae40 con 0x7f5cd415a9c0 2023-10-26T05:51:14.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cd0ff9640 1 -- 172.21.15.71:0/912003201 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5ccc0c0080 con 0x7f5cd415a9c0 2023-10-26T05:51:14.285 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:51:14.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5cc0066470 msgr2=0x7f5cc0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5cc0066470 0x7f5cc0068930 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4003680 tx=0x7f5cc4006730 comp rx=0 tx=0).stop 2023-10-26T05:51:14.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 msgr2=0x7f5cd40c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 0x7f5cd40c83a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009ee0 tx=0x7f5ccc009480 comp rx=0 tx=0).stop 2023-10-26T05:51:14.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 shutdown_connections 2023-10-26T05:51:14.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5cc0066470 0x7f5cc0068930 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cd415a9c0 0x7f5cd40c83a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd40cad60 0x7f5cd415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 --2- 172.21.15.71:0/912003201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cd414eb50 0x7f5cd415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 >> 172.21.15.71:0/912003201 conn(0x7f5cd40d12d0 msgr2=0x7f5cd40b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:14.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 shutdown_connections 2023-10-26T05:51:14.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.281+0000 7f5cda7ba640 1 -- 172.21.15.71:0/912003201 wait complete. 2023-10-26T05:51:14.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:51:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- 172.21.15.71:0/2081028113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57ac004d10 msgr2=0x7f57ac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 --2- 172.21.15.71:0/2081028113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57ac004d10 0x7f57ac005190 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009ee0 tx=0x7f57b801d230 comp rx=0 tx=0).stop 2023-10-26T05:51:14.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c5da0640 1 -- 172.21.15.71:0/2081028113 reap_dead start 2023-10-26T05:51:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- 172.21.15.71:0/2081028113 shutdown_connections 2023-10-26T05:51:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- 172.21.15.71:0/2081028113 >> 172.21.15.71:0/2081028113 conn(0x7f57c0069770 msgr2=0x7f57c0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- 172.21.15.71:0/2081028113 shutdown_connections 2023-10-26T05:51:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- 172.21.15.71:0/2081028113 wait complete. 2023-10-26T05:51:14.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 Processor -- start 2023-10-26T05:51:14.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- start start 2023-10-26T05:51:14.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57ac004d10 0x7f57c01596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 0x7f57c013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57c0139940 0x7f57c0139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0140800 con 0x7f57c015bcb0 2023-10-26T05:51:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c01401f0 con 0x7f57ac004d10 2023-10-26T05:51:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c7803640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c015c250 con 0x7f57c0139940 2023-10-26T05:51:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.429+0000 7f57c4d9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 0x7f57c013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 0x7f57c013f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60600/0 (socket says 172.21.15.71:60600) 2023-10-26T05:51:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 -- 172.21.15.71:0/567437751 learned_addr learned my addr 172.21.15.71:0/567437751 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c559f640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57ac004d10 0x7f57c01596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c5da0640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57c0139940 0x7f57c0139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 -- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57c0139940 msgr2=0x7f57c0139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57c0139940 0x7f57c0139dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 -- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57ac004d10 msgr2=0x7f57c01596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57ac004d10 0x7f57c01596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 -- 172.21.15.71:0/567437751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c0140490 con 0x7f57c015bcb0 2023-10-26T05:51:14.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c559f640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57ac004d10 0x7f57c01596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:51:14.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c4d9e640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 0x7f57c013f040 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f57bc003f20 tx=0x7f57bc00ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:14.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc005220 con 0x7f57c015bcb0 2023-10-26T05:51:14.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57bc015070 con 0x7f57c015bcb0 2023-10-26T05:51:14.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c015c530 con 0x7f57c015bcb0 2023-10-26T05:51:14.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc005600 con 0x7f57c015bcb0 2023-10-26T05:51:14.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c013f580 con 0x7f57c015bcb0 2023-10-26T05:51:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c0066b10 con 0x7f57c015bcb0 2023-10-26T05:51:14.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57bc017030 con 0x7f57c015bcb0 2023-10-26T05:51:14.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57b5ffb640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f57a4066470 0x7f57a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f57bc084380 con 0x7f57c015bcb0 2023-10-26T05:51:14.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c559f640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f57a4066470 0x7f57a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.433+0000 7f57c559f640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f57a4066470 0x7f57a4068930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000fc0 tx=0x7f57b00067b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:14.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.437+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f57bc0555e0 con 0x7f57c015bcb0 2023-10-26T05:51:14.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f57c00721e0 con 0x7f57c015bcb0 2023-10-26T05:51:14.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57b5ffb640 1 -- 172.21.15.71:0/567437751 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f57bc005760 con 0x7f57c015bcb0 2023-10-26T05:51:14.634 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:14.634 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:51:14.635 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:51:14.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.635 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:51:14.635 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:51:14.635 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.635 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.636 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:51:14.636 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:51:14.636 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.636 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.637 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.637 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:51:14.637 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:51:14.637 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.637 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.638 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:51:14.638 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:51:14.638 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.638 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.639 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:51:14.639 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:51:14.639 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.639 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.639 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:51:14.640 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:51:14.640 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.641 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:51:14.641 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:51:14.641 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.641 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.641 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:51:14.642 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:51:14.642 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.642 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.643 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:51:14.643 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:51:14.643 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.643 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.643 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.644 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.644 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.644 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:51:14.645 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:51:14.645 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.645 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.645 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.645 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:51:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:51:14.646 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.647 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.647 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.647 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.648 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:51:14.648 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:51:14.648 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.648 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.648 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.649 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.651 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:51:14.651 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:51:14.651 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.651 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.652 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.652 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.652 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.652 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:51:14.653 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:51:14.653 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.653 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.653 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.653 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.654 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:51:14.654 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:51:14.655 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.655 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.655 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.655 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.655 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.656 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:51:14.656 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:51:14.656 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.656 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:51:14.657 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:51:14.657 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.657 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.657 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.658 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:51:14.658 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:51:14.658 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.658 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:51:14.659 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:51:14.659 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.659 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:51:14.660 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:51:14.660 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.660 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.661 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:51:14.661 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:51:14.661 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:51:14.662 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.662 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:51:14.662 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:51:14.663 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:51:14.663 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:51:14.664 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:51:14.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.664 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:51:14.665 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:51:14.665 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.665 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:51:14.665 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:51:14.665 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:51:14.666 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:51:14.666 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.666 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:51:14.667 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:51:14.667 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.667 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:51:14.667 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:51:14.667 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:51:14.667 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.668 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.668 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:51:14.668 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:51:14.669 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.669 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:51:14.669 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:51:14.669 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.669 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:51:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:51:14.670 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:51:14.671 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:51:14.671 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:51:14.671 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:51:14.671 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.671 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:14.672 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:51:14.672 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:51:14.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f57a4066470 msgr2=0x7f57a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f57a4066470 0x7f57a4068930 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f57b0000fc0 tx=0x7f57b00067b0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 msgr2=0x7f57c013f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 0x7f57c013f040 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f57bc003f20 tx=0x7f57bc00ba80 comp rx=0 tx=0).stop 2023-10-26T05:51:14.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 shutdown_connections 2023-10-26T05:51:14.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57c0139940 0x7f57c0139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f57a4066470 0x7f57a4068930 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57c015bcb0 0x7f57c013f040 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 --2- 172.21.15.71:0/567437751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57ac004d10 0x7f57c01596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 >> 172.21.15.71:0/567437751 conn(0x7f57c0069770 msgr2=0x7f57c0158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:14.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 shutdown_connections 2023-10-26T05:51:14.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.629+0000 7f57c7803640 1 -- 172.21.15.71:0/567437751 wait complete. 2023-10-26T05:51:14.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:14.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 -- 172.21.15.71:0/651755005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 msgr2=0x7f495c06f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 --2- 172.21.15.71:0/651755005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c06f640 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f495c0afd40 tx=0x7f495401caa0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 -- 172.21.15.71:0/651755005 shutdown_connections 2023-10-26T05:51:14.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 --2- 172.21.15.71:0/651755005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c06f640 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 --2- 172.21.15.71:0/651755005 >> v2:172.21.15.86:3300/0 conn(0x7f495c0b2890 0x7f495c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 --2- 172.21.15.71:0/651755005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f495c13c2f0 0x7f495c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 -- 172.21.15.71:0/651755005 >> 172.21.15.71:0/651755005 conn(0x7f495c069770 msgr2=0x7f495c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:14.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 -- 172.21.15.71:0/651755005 shutdown_connections 2023-10-26T05:51:14.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 -- 172.21.15.71:0/651755005 wait complete. 2023-10-26T05:51:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 Processor -- start 2023-10-26T05:51:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.785+0000 7f4960c40640 1 -- start start 2023-10-26T05:51:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f495c0b2890 0x7f495c12e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c138a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f495c13c2f0 0x7f495c12fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c157d20 con 0x7f495c0cb010 2023-10-26T05:51:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c157430 con 0x7f495c13c2f0 2023-10-26T05:51:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c132810 con 0x7f495c0b2890 2023-10-26T05:51:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c138a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c138a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60632/0 (socket says 172.21.15.71:60632) 2023-10-26T05:51:14.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 -- 172.21.15.71:0/24950675 learned_addr learned my addr 172.21.15.71:0/24950675 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:14.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f495ad9d640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f495c13c2f0 0x7f495c12fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f495a59c640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f495c0b2890 0x7f495c12e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 -- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f495c0b2890 msgr2=0x7f495c12e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f495c0b2890 0x7f495c12e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 -- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f495c13c2f0 msgr2=0x7f495c12fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f495c13c2f0 0x7f495c12fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 -- 172.21.15.71:0/24950675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540098e0 con 0x7f495c0cb010 2023-10-26T05:51:14.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f495ad9d640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f495c13c2f0 0x7f495c12fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:51:14.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4959d9b640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c138a20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9f0 tx=0x7f495000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:14.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495000ccc0 con 0x7f495c0cb010 2023-10-26T05:51:14.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4950010430 con 0x7f495c0cb010 2023-10-26T05:51:14.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c1576d0 con 0x7f495c0cb010 2023-10-26T05:51:14.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950010800 con 0x7f495c0cb010 2023-10-26T05:51:14.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c138f60 con 0x7f495c0cb010 2023-10-26T05:51:14.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c066b10 con 0x7f495c0cb010 2023-10-26T05:51:14.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4950010960 con 0x7f495c0cb010 2023-10-26T05:51:14.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f494affd640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f493c066470 0x7f493c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:14.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4950014070 con 0x7f495c0cb010 2023-10-26T05:51:14.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.789+0000 7f495a59c640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f493c066470 0x7f493c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:14.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.793+0000 7f495a59c640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f493c066470 0x7f493c068930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f494c009ad0 tx=0x7f494c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:14.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.793+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4950052780 con 0x7f495c0cb010 2023-10-26T05:51:14.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.937+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f495c139150 con 0x7f495c0cb010 2023-10-26T05:51:14.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.937+0000 7f494affd640 1 -- 172.21.15.71:0/24950675 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f495c139150 con 0x7f495c0cb010 2023-10-26T05:51:14.943 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:14.943 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":82,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:14.298197+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:14.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f493c066470 msgr2=0x7f493c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f493c066470 0x7f493c068930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f494c009ad0 tx=0x7f494c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 msgr2=0x7f495c138a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:14.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c138a20 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f495000e9f0 tx=0x7f495000eeb0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 shutdown_connections 2023-10-26T05:51:14.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f495c13c2f0 0x7f495c12fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f493c066470 0x7f493c068930 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f495c0cb010 0x7f495c138a20 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 --2- 172.21.15.71:0/24950675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f495c0b2890 0x7f495c12e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:14.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 >> 172.21.15.71:0/24950675 conn(0x7f495c069770 msgr2=0x7f495c138040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:14.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 shutdown_connections 2023-10-26T05:51:14.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:14.941+0000 7f4960c40640 1 -- 172.21.15.71:0/24950675 wait complete. 2023-10-26T05:51:14.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:14.962 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:15.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 -- 172.21.15.71:0/164462587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f707815c720 msgr2=0x7f70780b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 --2- 172.21.15.71:0/164462587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f707815c720 0x7f70780b0560 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7078072f50 tx=0x7f706801c7e0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 -- 172.21.15.71:0/164462587 shutdown_connections 2023-10-26T05:51:15.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 --2- 172.21.15.71:0/164462587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707815d390 0x7f7078159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 --2- 172.21.15.71:0/164462587 >> v2:172.21.15.86:3300/0 conn(0x7f70780cede0 0x7f707815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 --2- 172.21.15.71:0/164462587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f707815c720 0x7f70780b0560 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 -- 172.21.15.71:0/164462587 >> 172.21.15.71:0/164462587 conn(0x7f70780b0f20 msgr2=0x7f70780b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:15.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 -- 172.21.15.71:0/164462587 shutdown_connections 2023-10-26T05:51:15.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.077+0000 7f707e30b640 1 -- 172.21.15.71:0/164462587 wait complete. 2023-10-26T05:51:15.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 Processor -- start 2023-10-26T05:51:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- start start 2023-10-26T05:51:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 0x7f7078153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707815c720 0x7f707814bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707815d390 0x7f70780ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70781545b0 con 0x7f70780cede0 2023-10-26T05:51:15.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078154890 con 0x7f707815d390 2023-10-26T05:51:15.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7078154b70 con 0x7f707815c720 2023-10-26T05:51:15.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 0x7f7078153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 0x7f7078153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60652/0 (socket says 172.21.15.71:60652) 2023-10-26T05:51:15.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 -- 172.21.15.71:0/2874929566 learned_addr learned my addr 172.21.15.71:0/2874929566 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:15.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 -- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707815c720 msgr2=0x7f707814bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707815c720 0x7f707814bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 -- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707815d390 msgr2=0x7f70780ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707c8a8640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707815d390 0x7f70780ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707815d390 0x7f70780ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 -- 172.21.15.71:0/2874929566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068009560 con 0x7f70780cede0 2023-10-26T05:51:15.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7077fff640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 0x7f7078153f20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f70680b0040 tx=0x7f70680a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:15.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068005230 con 0x7f70780cede0 2023-10-26T05:51:15.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f70680054a0 con 0x7f70780cede0 2023-10-26T05:51:15.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7078154e50 con 0x7f70780cede0 2023-10-26T05:51:15.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068004930 con 0x7f70780cede0 2023-10-26T05:51:15.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707814c310 con 0x7f70780cede0 2023-10-26T05:51:15.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.081+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7078155370 con 0x7f70780cede0 2023-10-26T05:51:15.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.085+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f70680b4070 con 0x7f70780cede0 2023-10-26T05:51:15.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.085+0000 7f7074ff9640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7064066470 0x7f7064068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.085+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f706811e240 con 0x7f70780cede0 2023-10-26T05:51:15.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.085+0000 7f70777fe640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7064066470 0x7f7064068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.085+0000 7f70777fe640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7064066470 0x7f7064068930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7078156ec0 tx=0x7f706c006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:15.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.089+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7078155370 con 0x7f70780cede0 2023-10-26T05:51:15.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.241+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7078155370 con 0x7f70780cede0 2023-10-26T05:51:15.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f7074ff9640 1 -- 172.21.15.71:0/2874929566 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70680eba20 con 0x7f70780cede0 2023-10-26T05:51:15.248 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:15.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7064066470 msgr2=0x7f7064068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7064066470 0x7f7064068930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7078156ec0 tx=0x7f706c006ca0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 msgr2=0x7f7078153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 0x7f7078153f20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f70680b0040 tx=0x7f70680a5840 comp rx=0 tx=0).stop 2023-10-26T05:51:15.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 shutdown_connections 2023-10-26T05:51:15.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707815d390 0x7f70780ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707815c720 0x7f707814bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7064066470 0x7f7064068930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 --2- 172.21.15.71:0/2874929566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70780cede0 0x7f7078153f20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 >> 172.21.15.71:0/2874929566 conn(0x7f70780b0f20 msgr2=0x7f7078158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:15.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 shutdown_connections 2023-10-26T05:51:15.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.245+0000 7f707e30b640 1 -- 172.21.15.71:0/2874929566 wait complete. 2023-10-26T05:51:15.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:51:15.279 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:51:15.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:51:15.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/2064516828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a8003980 msgr2=0x7fb0a8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/2064516828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0a8003980 0x7fb0a8003da0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009ee0 tx=0x7fb0ac01d5a0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c1028640 1 -- 172.21.15.71:0/2064516828 reap_dead start 2023-10-26T05:51:15.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/2064516828 shutdown_connections 2023-10-26T05:51:15.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/2064516828 >> 172.21.15.71:0/2064516828 conn(0x7fb0bc0cdd80 msgr2=0x7fb0bc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:15.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/2064516828 shutdown_connections 2023-10-26T05:51:15.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.381+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/2064516828 wait complete. 2023-10-26T05:51:15.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 Processor -- start 2023-10-26T05:51:15.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- start start 2023-10-26T05:51:15.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8003980 0x7fb0bc164e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 0x7fb0bc159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0bc162f70 0x7fb0bc0b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc1640a0 con 0x7fb0bc159df0 2023-10-26T05:51:15.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc164380 con 0x7fb0a8003980 2023-10-26T05:51:15.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc164660 con 0x7fb0bc162f70 2023-10-26T05:51:15.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 0x7fb0bc159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 0x7fb0bc159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60664/0 (socket says 172.21.15.71:60664) 2023-10-26T05:51:15.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 -- 172.21.15.71:0/1570222938 learned_addr learned my addr 172.21.15.71:0/1570222938 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:15.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 -- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0bc162f70 msgr2=0x7fb0bc0b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c1028640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0bc162f70 0x7fb0bc0b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c0827640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8003980 0x7fb0bc164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0bc162f70 0x7fb0bc0b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 -- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8003980 msgr2=0x7fb0bc164e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8003980 0x7fb0bc164e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 -- 172.21.15.71:0/1570222938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc164940 con 0x7fb0bc159df0 2023-10-26T05:51:15.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0bbfff640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 0x7fb0bc159550 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c10 tx=0x7fb0b400dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:15.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40062e0 con 0x7fb0bc159df0 2023-10-26T05:51:15.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb0b400b830 con 0x7fb0bc159df0 2023-10-26T05:51:15.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc159a90 con 0x7fb0bc159df0 2023-10-26T05:51:15.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4006530 con 0x7fb0bc159df0 2023-10-26T05:51:15.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc1635d0 con 0x7fb0bc159df0 2023-10-26T05:51:15.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.385+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc066b10 con 0x7fb0bc159df0 2023-10-26T05:51:15.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.389+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb0b400bc00 con 0x7fb0bc159df0 2023-10-26T05:51:15.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.389+0000 7fb0b97fa640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb09c0664a0 0x7fb09c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.389+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb0b4084800 con 0x7fb0bc159df0 2023-10-26T05:51:15.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.389+0000 7fb0c0827640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb09c0664a0 0x7fb09c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.389+0000 7fb0c0827640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb09c0664a0 0x7fb09c068960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0af000 tx=0x7fb0ac01a340 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:15.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.393+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0b4051f50 con 0x7fb0bc159df0 2023-10-26T05:51:15.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb0bc1641e0 con 0x7fb0bc159df0 2023-10-26T05:51:15.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0b97fa640 1 -- 172.21.15.71:0/1570222938 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb0bc1641e0 con 0x7fb0bc159df0 2023-10-26T05:51:15.553 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:15.553 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:51:15.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:51:15.554 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:51:15.554 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:51:15.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:15.554 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:51:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb09c0664a0 msgr2=0x7fb09c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb09c0664a0 0x7fb09c068960 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0af000 tx=0x7fb0ac01a340 comp rx=0 tx=0).stop 2023-10-26T05:51:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 msgr2=0x7fb0bc159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 0x7fb0bc159550 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4000c10 tx=0x7fb0b400dd80 comp rx=0 tx=0).stop 2023-10-26T05:51:15.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 shutdown_connections 2023-10-26T05:51:15.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb0bc162f70 0x7fb0bc0b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb09c0664a0 0x7fb09c068960 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0bc159df0 0x7fb0bc159550 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 --2- 172.21.15.71:0/1570222938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0a8003980 0x7fb0bc164e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 >> 172.21.15.71:0/1570222938 conn(0x7fb0bc0cdd80 msgr2=0x7fb0bc0ca3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:15.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 shutdown_connections 2023-10-26T05:51:15.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.549+0000 7fb0c2a8b640 1 -- 172.21.15.71:0/1570222938 wait complete. 2023-10-26T05:51:15.572 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:51:15.572 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:51:15.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- 172.21.15.71:0/436385616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2338070a90 msgr2=0x7f23380b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 --2- 172.21.15.71:0/436385616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2338070a90 0x7f23380b28c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2328009900 tx=0x7f232801c890 comp rx=0 tx=0).stop 2023-10-26T05:51:15.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- 172.21.15.71:0/436385616 shutdown_connections 2023-10-26T05:51:15.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 --2- 172.21.15.71:0/436385616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2338070a90 0x7f23380b28c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 --2- 172.21.15.71:0/436385616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23380cd7d0 0x7f2338070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- 172.21.15.71:0/436385616 >> 172.21.15.71:0/436385616 conn(0x7f23380afe40 msgr2=0x7f23380b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:15.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- 172.21.15.71:0/436385616 shutdown_connections 2023-10-26T05:51:15.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- 172.21.15.71:0/436385616 wait complete. 2023-10-26T05:51:15.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 Processor -- start 2023-10-26T05:51:15.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- start start 2023-10-26T05:51:15.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2338070a90 0x7f233815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 0x7f23380cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2338144d90 0x7f23381577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338140e60 con 0x7f23380cd7d0 2023-10-26T05:51:15.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233813ff90 con 0x7f2338070a90 2023-10-26T05:51:15.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f233dd7d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233815b710 con 0x7f2338144d90 2023-10-26T05:51:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f2336ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 0x7f23380cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f2336ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 0x7f23380cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60684/0 (socket says 172.21.15.71:60684) 2023-10-26T05:51:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.697+0000 7f2336ffd640 1 -- 172.21.15.71:0/2578980627 learned_addr learned my addr 172.21.15.71:0/2578980627 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2337fff640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2338144d90 0x7f23381577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f23377fe640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2338070a90 0x7f233815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2336ffd640 1 -- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2338144d90 msgr2=0x7f23381577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2336ffd640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2338144d90 0x7f23381577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2336ffd640 1 -- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2338070a90 msgr2=0x7f233815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2336ffd640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2338070a90 0x7f233815b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:15.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2336ffd640 1 -- 172.21.15.71:0/2578980627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328009560 con 0x7f23380cd7d0 2023-10-26T05:51:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f23377fe640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2338070a90 0x7f233815b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f2336ffd640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 0x7f23380cabd0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f232000d950 tx=0x7f232000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2320009c30 con 0x7f23380cd7d0 2023-10-26T05:51:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2320009e80 con 0x7f23380cd7d0 2023-10-26T05:51:15.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23381401e0 con 0x7f23380cd7d0 2023-10-26T05:51:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23200106c0 con 0x7f23380cd7d0 2023-10-26T05:51:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23381407a0 con 0x7f23380cd7d0 2023-10-26T05:51:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338066d00 con 0x7f23380cd7d0 2023-10-26T05:51:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2320010820 con 0x7f23380cd7d0 2023-10-26T05:51:15.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f232ffff640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2318066470 0x7f2318068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2320083f20 con 0x7f23380cd7d0 2023-10-26T05:51:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f23377fe640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2318066470 0x7f2318068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:15.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.701+0000 7f23377fe640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2318066470 0x7f2318068930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2330007fd0 tx=0x7f2330009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:15.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.705+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2320051780 con 0x7f23380cd7d0 2023-10-26T05:51:15.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:15.849+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f23380add60 con 0x7f23380cd7d0 2023-10-26T05:51:16.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.285+0000 7f232ffff640 1 -- 172.21.15.71:0/2578980627 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f23200170b0 con 0x7f23380cd7d0 2023-10-26T05:51:16.293 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:51:16.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.289+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2318066470 msgr2=0x7f2318068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:16.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.289+0000 7f233dd7d640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2318066470 0x7f2318068930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2330007fd0 tx=0x7f2330009040 comp rx=0 tx=0).stop 2023-10-26T05:51:16.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.289+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 msgr2=0x7f23380cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:16.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.289+0000 7f233dd7d640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 0x7f23380cabd0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f232000d950 tx=0x7f232000de10 comp rx=0 tx=0).stop 2023-10-26T05:51:16.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 shutdown_connections 2023-10-26T05:51:16.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2338144d90 0x7f23381577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2318066470 0x7f2318068930 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23380cd7d0 0x7f23380cabd0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 --2- 172.21.15.71:0/2578980627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2338070a90 0x7f233815b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 >> 172.21.15.71:0/2578980627 conn(0x7f23380afe40 msgr2=0x7f2338065550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:16.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 shutdown_connections 2023-10-26T05:51:16.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.293+0000 7f233dd7d640 1 -- 172.21.15.71:0/2578980627 wait complete. 2023-10-26T05:51:16.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:51:16.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- 172.21.15.71:0/2961157638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda40c9720 msgr2=0x7fdda415cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:16.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 --2- 172.21.15.71:0/2961157638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda40c9720 0x7fdda415cd00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fdda40b4ec0 tx=0x7fdd8c01c8a0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- 172.21.15.71:0/2961157638 shutdown_connections 2023-10-26T05:51:16.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 --2- 172.21.15.71:0/2961157638 >> v2:172.21.15.86:3300/0 conn(0x7fdda415a8d0 0x7fdda415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 --2- 172.21.15.71:0/2961157638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda40c9720 0x7fdda415cd00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- 172.21.15.71:0/2961157638 >> 172.21.15.71:0/2961157638 conn(0x7fdda40cbb50 msgr2=0x7fdda40cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:16.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- 172.21.15.71:0/2961157638 shutdown_connections 2023-10-26T05:51:16.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- 172.21.15.71:0/2961157638 wait complete. 2023-10-26T05:51:16.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 Processor -- start 2023-10-26T05:51:16.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- start start 2023-10-26T05:51:16.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdda40b4580 0x7fdda4152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:16.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40c9720 0x7fdda40c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:16.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 0x7fdda40c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:16.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40c37b0 con 0x7fdda415a8d0 2023-10-26T05:51:16.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40c3a90 con 0x7fdda40c9720 2023-10-26T05:51:16.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda9f60640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40c3d70 con 0x7fdda40b4580 2023-10-26T05:51:16.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 0x7fdda40c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:16.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 0x7fdda40c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60690/0 (socket says 172.21.15.71:60690) 2023-10-26T05:51:16.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.449+0000 7fdda3fff640 1 -- 172.21.15.71:0/1760109860 learned_addr learned my addr 172.21.15.71:0/1760109860 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:16.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda2ffd640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40c9720 0x7fdda40c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:16.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda37fe640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdda40b4580 0x7fdda4152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:16.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda3fff640 1 -- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdda40b4580 msgr2=0x7fdda4152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:16.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda3fff640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdda40b4580 0x7fdda4152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda3fff640 1 -- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40c9720 msgr2=0x7fdda40c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:16.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda3fff640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40c9720 0x7fdda40c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:16.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda3fff640 1 -- 172.21.15.71:0/1760109860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009560 con 0x7fdda415a8d0 2023-10-26T05:51:16.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda3fff640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 0x7fdda40c51f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d660 tx=0x7fdd9400db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:16.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94005220 con 0x7fdda415a8d0 2023-10-26T05:51:16.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd94005470 con 0x7fdda415a8d0 2023-10-26T05:51:16.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda40c4050 con 0x7fdda415a8d0 2023-10-26T05:51:16.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94009a80 con 0x7fdda415a8d0 2023-10-26T05:51:16.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda4152bb0 con 0x7fdda415a8d0 2023-10-26T05:51:16.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda4066d00 con 0x7fdda415a8d0 2023-10-26T05:51:16.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdd94009be0 con 0x7fdda415a8d0 2023-10-26T05:51:16.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdd9bfff640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd84066470 0x7fdd84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:16.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fdd94083d50 con 0x7fdda415a8d0 2023-10-26T05:51:16.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda37fe640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd84066470 0x7fdd84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:16.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.453+0000 7fdda37fe640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd84066470 0x7fdd84068930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0012f0 tx=0x7fdd9c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:16.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.457+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdd94087030 con 0x7fdda415a8d0 2023-10-26T05:51:16.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:16.613+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdda40c4500 con 0x7fdda415a8d0 2023-10-26T05:51:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.293+0000 7fdd9bfff640 1 -- 172.21.15.71:0/1760109860 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdd940200d0 con 0x7fdda415a8d0 2023-10-26T05:51:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:51:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.293+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd84066470 msgr2=0x7fdd84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.293+0000 7fdda9f60640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd84066470 0x7fdd84068930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0012f0 tx=0x7fdd9c009040 comp rx=0 tx=0).stop 2023-10-26T05:51:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.293+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 msgr2=0x7fdda40c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.293+0000 7fdda9f60640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 0x7fdda40c51f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400d660 tx=0x7fdd9400db20 comp rx=0 tx=0).stop 2023-10-26T05:51:17.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 shutdown_connections 2023-10-26T05:51:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd84066470 0x7fdd84068930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdda415a8d0 0x7fdda40c51f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40c9720 0x7fdda40c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 --2- 172.21.15.71:0/1760109860 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdda40b4580 0x7fdda4152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 >> 172.21.15.71:0/1760109860 conn(0x7fdda40cbb50 msgr2=0x7fdda415c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:17.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 shutdown_connections 2023-10-26T05:51:17.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.297+0000 7fdda9f60640 1 -- 172.21.15.71:0/1760109860 wait complete. 2023-10-26T05:51:17.328 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:51:17.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- 172.21.15.71:0/2903510916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 msgr2=0x7f104815d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:17.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 --2- 172.21.15.71:0/2903510916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f104815d3b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f103c009840 tx=0x7f103c01c800 comp rx=0 tx=0).stop 2023-10-26T05:51:17.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- 172.21.15.71:0/2903510916 shutdown_connections 2023-10-26T05:51:17.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 --2- 172.21.15.71:0/2903510916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f104815d3b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 --2- 172.21.15.71:0/2903510916 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1048159380 0x7f10480b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- 172.21.15.71:0/2903510916 >> 172.21.15.71:0/2903510916 conn(0x7f10480ca230 msgr2=0x7f10480ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:17.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- 172.21.15.71:0/2903510916 shutdown_connections 2023-10-26T05:51:17.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- 172.21.15.71:0/2903510916 wait complete. 2023-10-26T05:51:17.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 Processor -- start 2023-10-26T05:51:17.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- start start 2023-10-26T05:51:17.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1048159380 0x7f104814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:17.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f1048157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:17.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f104815d8f0 0x7f10480b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:17.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104814f920 con 0x7f104815cf30 2023-10-26T05:51:17.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104806acf0 con 0x7f1048159380 2023-10-26T05:51:17.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f104e11f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1048069c50 con 0x7f104815d8f0 2023-10-26T05:51:17.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f1048157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:17.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f1048157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60708/0 (socket says 172.21.15.71:60708) 2023-10-26T05:51:17.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 -- 172.21.15.71:0/1293862222 learned_addr learned my addr 172.21.15.71:0/1293862222 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:17.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 -- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f104815d8f0 msgr2=0x7f10480b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:17.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1047fff640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f104815d8f0 0x7f10480b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:17.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f10477fe640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1048159380 0x7f104814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:17.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f104815d8f0 0x7f10480b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 -- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1048159380 msgr2=0x7f104814f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:17.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1048159380 0x7f104814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:17.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 -- 172.21.15.71:0/1293862222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0094a0 con 0x7f104815cf30 2023-10-26T05:51:17.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.453+0000 7f1046ffd640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f1048157980 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f103c01a430 tx=0x7f103c0a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:17.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0bc070 con 0x7f104815cf30 2023-10-26T05:51:17.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f103c004130 con 0x7f104815cf30 2023-10-26T05:51:17.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1048069f30 con 0x7f104815cf30 2023-10-26T05:51:17.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0044e0 con 0x7f104815cf30 2023-10-26T05:51:17.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104806a4f0 con 0x7f104815cf30 2023-10-26T05:51:17.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048066d00 con 0x7f104815cf30 2023-10-26T05:51:17.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f103c0b4040 con 0x7f104815cf30 2023-10-26T05:51:17.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f102ffff640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1034066470 0x7f1034068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:17.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f103c11efe0 con 0x7f104815cf30 2023-10-26T05:51:17.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f10477fe640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1034066470 0x7f1034068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:17.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.457+0000 7f10477fe640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1034066470 0x7f1034068930 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f10400058d0 tx=0x7f1040005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:17.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.461+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f103c0ec510 con 0x7f104815cf30 2023-10-26T05:51:17.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:17.653+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f104815a8b0 con 0x7f104815cf30 2023-10-26T05:51:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.365+0000 7f102ffff640 1 -- 172.21.15.71:0/1293862222 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v30) v1 ==== 174+0+0 (secure 0 0 0) 0x7f103c0b2020 con 0x7f104815cf30 2023-10-26T05:51:18.369 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 10 and data pool 11 2023-10-26T05:51:18.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1034066470 msgr2=0x7f1034068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1034066470 0x7f1034068930 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f10400058d0 tx=0x7f1040005860 comp rx=0 tx=0).stop 2023-10-26T05:51:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 msgr2=0x7f1048157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f1048157980 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f103c01a430 tx=0x7f103c0a5790 comp rx=0 tx=0).stop 2023-10-26T05:51:18.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 shutdown_connections 2023-10-26T05:51:18.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f104815d8f0 0x7f10480b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1034066470 0x7f1034068930 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f104815cf30 0x7f1048157980 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 --2- 172.21.15.71:0/1293862222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1048159380 0x7f104814f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 >> 172.21.15.71:0/1293862222 conn(0x7f10480ca230 msgr2=0x7f1048155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:18.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 shutdown_connections 2023-10-26T05:51:18.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.369+0000 7f104e11f640 1 -- 172.21.15.71:0/1293862222 wait complete. 2023-10-26T05:51:18.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:18.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 -- 172.21.15.71:0/1109876059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c013c2f0 msgr2=0x7f48c00b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 --2- 172.21.15.71:0/1109876059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c013c2f0 0x7f48c00b2350 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f48c00b32d0 tx=0x7f48bc01c5e0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 -- 172.21.15.71:0/1109876059 shutdown_connections 2023-10-26T05:51:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 --2- 172.21.15.71:0/1109876059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48c00b2890 0x7f48c00caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 --2- 172.21.15.71:0/1109876059 >> v2:172.21.15.86:3300/0 conn(0x7f48c00cb010 0x7f48c006f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 --2- 172.21.15.71:0/1109876059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c013c2f0 0x7f48c00b2350 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 -- 172.21.15.71:0/1109876059 >> 172.21.15.71:0/1109876059 conn(0x7f48c0069770 msgr2=0x7f48c0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 -- 172.21.15.71:0/1109876059 shutdown_connections 2023-10-26T05:51:18.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.509+0000 7f48c72ab640 1 -- 172.21.15.71:0/1109876059 wait complete. 2023-10-26T05:51:18.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 Processor -- start 2023-10-26T05:51:18.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- start start 2023-10-26T05:51:18.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48c00b2890 0x7f48c01f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 0x7f48c01540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c013c2f0 0x7f48c0137670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00aaa80 con 0x7f48c00cb010 2023-10-26T05:51:18.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0154620 con 0x7f48c013c2f0 2023-10-26T05:51:18.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0155160 con 0x7f48c00b2890 2023-10-26T05:51:18.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 0x7f48c01540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:18.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 0x7f48c01540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60724/0 (socket says 172.21.15.71:60724) 2023-10-26T05:51:18.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 -- 172.21.15.71:0/2913707551 learned_addr learned my addr 172.21.15.71:0/2913707551 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:18.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c5047640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48c00b2890 0x7f48c01f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:18.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c5848640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c013c2f0 0x7f48c0137670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:18.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 -- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48c00b2890 msgr2=0x7f48c01f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48c00b2890 0x7f48c01f8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 -- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c013c2f0 msgr2=0x7f48c0137670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c013c2f0 0x7f48c0137670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 -- 172.21.15.71:0/2913707551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc009560 con 0x7f48c00cb010 2023-10-26T05:51:18.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c4846640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 0x7f48c01540e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f48b000d750 tx=0x7f48b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:18.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0009c30 con 0x7f48c00cb010 2023-10-26T05:51:18.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48b0009e80 con 0x7f48c00cb010 2023-10-26T05:51:18.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c0137bb0 con 0x7f48c00cb010 2023-10-26T05:51:18.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0005280 con 0x7f48c00cb010 2023-10-26T05:51:18.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c0154ba0 con 0x7f48c00cb010 2023-10-26T05:51:18.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c0066b10 con 0x7f48c00cb010 2023-10-26T05:51:18.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48b00053e0 con 0x7f48c00cb010 2023-10-26T05:51:18.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48b5ffb640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48a40664a0 0x7f48a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.513+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48b0083fc0 con 0x7f48c00cb010 2023-10-26T05:51:18.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.517+0000 7f48c5047640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48a40664a0 0x7f48a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:18.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.517+0000 7f48c5047640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48a40664a0 0x7f48a4068960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ae000 tx=0x7f48bc01ae20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:18.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.521+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48b00b5090 con 0x7f48c00cb010 2023-10-26T05:51:18.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.673+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48c00aa8e0 con 0x7f48c00cb010 2023-10-26T05:51:18.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48b5ffb640 1 -- 172.21.15.71:0/2913707551 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f48c00aa8e0 con 0x7f48c00cb010 2023-10-26T05:51:18.682 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:18.683 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":85,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:17.657973+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48a40664a0 msgr2=0x7f48a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48a40664a0 0x7f48a4068960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ae000 tx=0x7f48bc01ae20 comp rx=0 tx=0).stop 2023-10-26T05:51:18.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 msgr2=0x7f48c01540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 0x7f48c01540e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f48b000d750 tx=0x7f48b000dc10 comp rx=0 tx=0).stop 2023-10-26T05:51:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 shutdown_connections 2023-10-26T05:51:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48c013c2f0 0x7f48c0137670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48a40664a0 0x7f48a4068960 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48c00cb010 0x7f48c01540e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 --2- 172.21.15.71:0/2913707551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48c00b2890 0x7f48c01f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.677+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 >> 172.21.15.71:0/2913707551 conn(0x7f48c0069770 msgr2=0x7f48c0137fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.681+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 shutdown_connections 2023-10-26T05:51:18.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.681+0000 7f48c72ab640 1 -- 172.21.15.71:0/2913707551 wait complete. 2023-10-26T05:51:18.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:18.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 -- 172.21.15.71:0/3661273151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 msgr2=0x7f1e0406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/3661273151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e0406aa70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df401c6b0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 -- 172.21.15.71:0/3661273151 shutdown_connections 2023-10-26T05:51:18.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/3661273151 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e040d0850 0x7f1e0415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/3661273151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e0406aa70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 -- 172.21.15.71:0/3661273151 >> 172.21.15.71:0/3661273151 conn(0x7f1e040cdd80 msgr2=0x7f1e040ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:18.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 -- 172.21.15.71:0/3661273151 shutdown_connections 2023-10-26T05:51:18.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.809+0000 7f1e0aebf640 1 -- 172.21.15.71:0/3661273151 wait complete. 2023-10-26T05:51:18.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 Processor -- start 2023-10-26T05:51:18.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- start start 2023-10-26T05:51:18.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e040cd880 0x7f1e04157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e040c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e040d0850 0x7f1e0415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e04156b10 con 0x7f1e040d0470 2023-10-26T05:51:18.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040c7c70 con 0x7f1e040d0850 2023-10-26T05:51:18.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e041596b0 con 0x7f1e040cd880 2023-10-26T05:51:18.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e040c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:18.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e040c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60746/0 (socket says 172.21.15.71:60746) 2023-10-26T05:51:18.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 -- 172.21.15.71:0/1275006135 learned_addr learned my addr 172.21.15.71:0/1275006135 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:18.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 -- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e040cd880 msgr2=0x7f1e04157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e040cd880 0x7f1e04157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 -- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e040d0850 msgr2=0x7f1e0415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e040d0850 0x7f1e0415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 -- 172.21.15.71:0/1275006135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40094a0 con 0x7f1e040d0470 2023-10-26T05:51:18.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e03fff640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e040c7730 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1df80058d0 tx=0x7f1df800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:18.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8009c30 con 0x7f1e040d0470 2023-10-26T05:51:18.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1df8009e80 con 0x7f1e040d0470 2023-10-26T05:51:18.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e04159990 con 0x7f1e040d0470 2023-10-26T05:51:18.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df80106a0 con 0x7f1e040d0470 2023-10-26T05:51:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e04158fa0 con 0x7f1e040d0470 2023-10-26T05:51:18.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.813+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04066b10 con 0x7f1e040d0470 2023-10-26T05:51:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.817+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1df8010800 con 0x7f1e040d0470 2023-10-26T05:51:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.817+0000 7f1e017fa640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1df0066470 0x7f1df0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:18.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.817+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1df8084740 con 0x7f1e040d0470 2023-10-26T05:51:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.817+0000 7f1e08c5b640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1df0066470 0x7f1df0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.817+0000 7f1e08c5b640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1df0066470 0x7f1df0068930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1df40af000 tx=0x7f1df4019450 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:18.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.817+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1df8051ca0 con 0x7f1e040d0470 2023-10-26T05:51:18.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.961+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e04156c50 con 0x7f1e040d0470 2023-10-26T05:51:18.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e017fa640 1 -- 172.21.15.71:0/1275006135 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f1e04156c50 con 0x7f1e040d0470 2023-10-26T05:51:18.969 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:18.969 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":85,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:17.657973+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:18.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1df0066470 msgr2=0x7f1df0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1df0066470 0x7f1df0068930 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1df40af000 tx=0x7f1df4019450 comp rx=0 tx=0).stop 2023-10-26T05:51:18.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 msgr2=0x7f1e040c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:18.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e040c7730 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1df80058d0 tx=0x7f1df800dde0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 shutdown_connections 2023-10-26T05:51:18.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e040d0850 0x7f1e0415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1df0066470 0x7f1df0068930 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e040d0470 0x7f1e040c7730 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 --2- 172.21.15.71:0/1275006135 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e040cd880 0x7f1e04157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:18.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 >> 172.21.15.71:0/1275006135 conn(0x7f1e040cdd80 msgr2=0x7f1e04157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:18.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 shutdown_connections 2023-10-26T05:51:18.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:18.965+0000 7f1e0aebf640 1 -- 172.21.15.71:0/1275006135 wait complete. 2023-10-26T05:51:18.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:51:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- 172.21.15.71:0/1228423139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 msgr2=0x7fe730158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 --2- 172.21.15.71:0/1228423139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe730158680 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009930 tx=0x7fe72c01c7c0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- 172.21.15.71:0/1228423139 shutdown_connections 2023-10-26T05:51:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 --2- 172.21.15.71:0/1228423139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7300b0f70 0x7fe7300ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 --2- 172.21.15.71:0/1228423139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe730158680 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- 172.21.15.71:0/1228423139 >> 172.21.15.71:0/1228423139 conn(0x7fe7300afe40 msgr2=0x7fe7300b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- 172.21.15.71:0/1228423139 shutdown_connections 2023-10-26T05:51:19.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- 172.21.15.71:0/1228423139 wait complete. 2023-10-26T05:51:19.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 Processor -- start 2023-10-26T05:51:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- start start 2023-10-26T05:51:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe730070550 0x7fe7301597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7300b0f70 0x7fe730158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe7301f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730159d10 con 0x7fe730141370 2023-10-26T05:51:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730159410 con 0x7fe7300b0f70 2023-10-26T05:51:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe737512640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73013df60 con 0x7fe730070550 2023-10-26T05:51:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe7301f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe7301f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60772/0 (socket says 172.21.15.71:60772) 2023-10-26T05:51:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 -- 172.21.15.71:0/3010448363 learned_addr learned my addr 172.21.15.71:0/3010448363 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 -- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe730070550 msgr2=0x7fe7301597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe734aad640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7300b0f70 0x7fe730158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe7352ae640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe730070550 0x7fe7301597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe730070550 0x7fe7301597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 -- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7300b0f70 msgr2=0x7fe730158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7300b0f70 0x7fe730158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.101+0000 7fe735aaf640 1 -- 172.21.15.71:0/3010448363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c009560 con 0x7fe730141370 2023-10-26T05:51:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe735aaf640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe7301f7ca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe72800c7b0 tx=0x7fe72800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe728004b70 con 0x7fe730141370 2023-10-26T05:51:19.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe728004dc0 con 0x7fe730141370 2023-10-26T05:51:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe730159190 con 0x7fe730141370 2023-10-26T05:51:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72800aab0 con 0x7fe730141370 2023-10-26T05:51:19.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7301f81e0 con 0x7fe730141370 2023-10-26T05:51:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73013e0f0 con 0x7fe730141370 2023-10-26T05:51:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe72800ac10 con 0x7fe730141370 2023-10-26T05:51:19.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe725ffb640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7140664a0 0x7fe714068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe728084520 con 0x7fe730141370 2023-10-26T05:51:19.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe7352ae640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7140664a0 0x7fe714068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.105+0000 7fe7352ae640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7140664a0 0x7fe714068960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0af000 tx=0x7fe72c01b000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:19.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.109+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe728051a30 con 0x7fe730141370 2023-10-26T05:51:19.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.293+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe730066d00 con 0x7fe730141370 2023-10-26T05:51:19.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.377+0000 7fe725ffb640 1 -- 172.21.15.71:0/3010448363 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe728055050 con 0x7fe730141370 2023-10-26T05:51:19.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7140664a0 msgr2=0x7fe714068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7140664a0 0x7fe714068960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0af000 tx=0x7fe72c01b000 comp rx=0 tx=0).stop 2023-10-26T05:51:19.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 msgr2=0x7fe7301f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe7301f7ca0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe72800c7b0 tx=0x7fe72800cc70 comp rx=0 tx=0).stop 2023-10-26T05:51:19.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 shutdown_connections 2023-10-26T05:51:19.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7140664a0 0x7fe714068960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe730141370 0x7fe7301f7ca0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7300b0f70 0x7fe730158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 --2- 172.21.15.71:0/3010448363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe730070550 0x7fe7301597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 >> 172.21.15.71:0/3010448363 conn(0x7fe7300afe40 msgr2=0x7fe730140900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:19.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 shutdown_connections 2023-10-26T05:51:19.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.381+0000 7fe737512640 1 -- 172.21.15.71:0/3010448363 wait complete. 2023-10-26T05:51:19.403 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:51:19.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:51:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 -- 172.21.15.71:0/3988049384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9680cbd60 msgr2=0x7fe968069dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 --2- 172.21.15.71:0/3988049384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9680cbd60 0x7fe968069dc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe960009bc0 tx=0x7fe96001cac0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 -- 172.21.15.71:0/3988049384 shutdown_connections 2023-10-26T05:51:19.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 --2- 172.21.15.71:0/3988049384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9680cbd60 0x7fe968069dc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 --2- 172.21.15.71:0/3988049384 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe96815d3f0 0x7fe9680cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 -- 172.21.15.71:0/3988049384 >> 172.21.15.71:0/3988049384 conn(0x7fe9680aec60 msgr2=0x7fe9680af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:19.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 -- 172.21.15.71:0/3988049384 shutdown_connections 2023-10-26T05:51:19.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.513+0000 7fe96efa1640 1 -- 172.21.15.71:0/3988049384 wait complete. 2023-10-26T05:51:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 Processor -- start 2023-10-26T05:51:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- start start 2023-10-26T05:51:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9680cbd60 0x7fe968152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 0x7fe96814b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe96815d3f0 0x7fe96814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe968152e80 con 0x7fe96813c220 2023-10-26T05:51:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96814bc40 con 0x7fe96815d3f0 2023-10-26T05:51:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96814bea0 con 0x7fe9680cbd60 2023-10-26T05:51:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 0x7fe96814b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 0x7fe96814b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60788/0 (socket says 172.21.15.71:60788) 2023-10-26T05:51:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 -- 172.21.15.71:0/843840734 learned_addr learned my addr 172.21.15.71:0/843840734 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96cd3d640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9680cbd60 0x7fe968152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96d53e640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe96815d3f0 0x7fe96814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 -- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9680cbd60 msgr2=0x7fe968152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9680cbd60 0x7fe968152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 -- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe96815d3f0 msgr2=0x7fe96814d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe96815d3f0 0x7fe96814d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 -- 172.21.15.71:0/843840734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe960009820 con 0x7fe96813c220 2023-10-26T05:51:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96cd3d640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9680cbd60 0x7fe968152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe967fff640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 0x7fe96814b700 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c970 tx=0x7fe95c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c007bb0 con 0x7fe96813c220 2023-10-26T05:51:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe95c007e00 con 0x7fe96813c220 2023-10-26T05:51:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe96806b100 con 0x7fe96813c220 2023-10-26T05:51:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95c005280 con 0x7fe96813c220 2023-10-26T05:51:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96814c720 con 0x7fe96813c220 2023-10-26T05:51:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe968066b10 con 0x7fe96813c220 2023-10-26T05:51:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.517+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe95c0053e0 con 0x7fe96813c220 2023-10-26T05:51:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.521+0000 7fe9657fa640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe954066470 0x7fe954068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.521+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe95c084350 con 0x7fe96813c220 2023-10-26T05:51:19.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.521+0000 7fe96cd3d640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe954066470 0x7fe954068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.521+0000 7fe96cd3d640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe954066470 0x7fe954068930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011f0 tx=0x7fe958009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:19.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.525+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe95c0517d0 con 0x7fe96813c220 2023-10-26T05:51:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:19.697+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe9680b0c50 con 0x7fe96813c220 2023-10-26T05:51:20.286 INFO:tasks.ceph.mds.c.smithi071.stderr:2023-10-26T05:51:20.280+0000 7f5daf8b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:51:20.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.388+0000 7fe9657fa640 1 -- 172.21.15.71:0/843840734 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe9680b0c50 con 0x7fe96813c220 2023-10-26T05:51:20.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe954066470 msgr2=0x7fe954068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe954066470 0x7fe954068930 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011f0 tx=0x7fe958009040 comp rx=0 tx=0).stop 2023-10-26T05:51:20.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 msgr2=0x7fe96814b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 0x7fe96814b700 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00c970 tx=0x7fe95c00ce30 comp rx=0 tx=0).stop 2023-10-26T05:51:20.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 shutdown_connections 2023-10-26T05:51:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe96815d3f0 0x7fe96814d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe954066470 0x7fe954068930 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe96813c220 0x7fe96814b700 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 --2- 172.21.15.71:0/843840734 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9680cbd60 0x7fe968152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 >> 172.21.15.71:0/843840734 conn(0x7fe9680aec60 msgr2=0x7fe968150c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:20.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 shutdown_connections 2023-10-26T05:51:20.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.392+0000 7fe96efa1640 1 -- 172.21.15.71:0/843840734 wait complete. 2023-10-26T05:51:20.416 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:51:20.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:20.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- 172.21.15.71:0/634006722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec13d420 msgr2=0x7f4cec0ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/634006722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec13d420 0x7f4cec0ac740 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4cec06db30 tx=0x7f4ce801c840 comp rx=0 tx=0).stop 2023-10-26T05:51:20.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- 172.21.15.71:0/634006722 shutdown_connections 2023-10-26T05:51:20.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/634006722 >> v2:172.21.15.86:3300/0 conn(0x7f4cec15bbc0 0x7f4cec070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/634006722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec13d420 0x7f4cec0ac740 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- 172.21.15.71:0/634006722 >> 172.21.15.71:0/634006722 conn(0x7f4cec069770 msgr2=0x7f4cec069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:20.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- 172.21.15.71:0/634006722 shutdown_connections 2023-10-26T05:51:20.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- 172.21.15.71:0/634006722 wait complete. 2023-10-26T05:51:20.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 Processor -- start 2023-10-26T05:51:20.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- start start 2023-10-26T05:51:20.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cec13d420 0x7f4cec13a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cec13d800 0x7f4cec0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 0x7f4cec0ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec14fc70 con 0x7f4cec15bbc0 2023-10-26T05:51:20.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec14ff50 con 0x7f4cec13d420 2023-10-26T05:51:20.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf3a28640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec150230 con 0x7f4cec13d800 2023-10-26T05:51:20.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 0x7f4cec0ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 0x7f4cec0ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60812/0 (socket says 172.21.15.71:60812) 2023-10-26T05:51:20.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 -- 172.21.15.71:0/2445915838 learned_addr learned my addr 172.21.15.71:0/2445915838 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:20.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf17c4640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cec13d420 0x7f4cec13a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf0fc3640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cec13d800 0x7f4cec0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 -- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cec13d800 msgr2=0x7f4cec0ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cec13d800 0x7f4cec0ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 -- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cec13d420 msgr2=0x7f4cec13a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cec13d420 0x7f4cec13a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf1fc5640 1 -- 172.21.15.71:0/2445915838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8009560 con 0x7f4cec15bbc0 2023-10-26T05:51:20.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf17c4640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cec13d420 0x7f4cec13a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:20.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.536+0000 7f4cf0fc3640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cec13d800 0x7f4cec0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:20.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4cf1fc5640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 0x7f4cec0ae5c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b970 tx=0x7f4ce400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:20.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400cb40 con 0x7f4cec15bbc0 2023-10-26T05:51:20.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4ce400cdb0 con 0x7f4cec15bbc0 2023-10-26T05:51:20.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec150510 con 0x7f4cec15bbc0 2023-10-26T05:51:20.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce40146a0 con 0x7f4cec15bbc0 2023-10-26T05:51:20.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec13aba0 con 0x7f4cec15bbc0 2023-10-26T05:51:20.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cec066b10 con 0x7f4cec15bbc0 2023-10-26T05:51:20.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ce401d3b0 con 0x7f4cec15bbc0 2023-10-26T05:51:20.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4ce1ffb640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cd0066470 0x7f4cd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ce4084010 con 0x7f4cec15bbc0 2023-10-26T05:51:20.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4cf17c4640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cd0066470 0x7f4cd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.540+0000 7f4cf17c4640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cd0066470 0x7f4cd0068930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80af000 tx=0x7f4ce801b080 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:20.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.544+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ce4051490 con 0x7f4cec15bbc0 2023-10-26T05:51:20.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.712+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cec0721e0 con 0x7f4cec15bbc0 2023-10-26T05:51:20.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4ce1ffb640 1 -- 172.21.15.71:0/2445915838 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f4ce4054ab0 con 0x7f4cec15bbc0 2023-10-26T05:51:20.721 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:20.721 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:20.722 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:20.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cd0066470 msgr2=0x7f4cd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cd0066470 0x7f4cd0068930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80af000 tx=0x7f4ce801b080 comp rx=0 tx=0).stop 2023-10-26T05:51:20.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 msgr2=0x7f4cec0ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 0x7f4cec0ae5c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b970 tx=0x7f4ce400be30 comp rx=0 tx=0).stop 2023-10-26T05:51:20.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 shutdown_connections 2023-10-26T05:51:20.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4cd0066470 0x7f4cd0068930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4cec15bbc0 0x7f4cec0ae5c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4cec13d800 0x7f4cec0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 --2- 172.21.15.71:0/2445915838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cec13d420 0x7f4cec13a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 >> 172.21.15.71:0/2445915838 conn(0x7f4cec069770 msgr2=0x7f4cec15a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:20.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 shutdown_connections 2023-10-26T05:51:20.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.716+0000 7f4cf3a28640 1 -- 172.21.15.71:0/2445915838 wait complete. 2023-10-26T05:51:20.739 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:20.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/3515270317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29680047d0 msgr2=0x7f2968004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 --2- 172.21.15.71:0/3515270317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29680047d0 0x7f2968004c50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f29740016b0 tx=0x7f297401c780 comp rx=0 tx=0).stop 2023-10-26T05:51:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/3515270317 shutdown_connections 2023-10-26T05:51:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 --2- 172.21.15.71:0/3515270317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29680047d0 0x7f2968004c50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/3515270317 >> 172.21.15.71:0/3515270317 conn(0x7f2980069770 msgr2=0x7f2980069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/3515270317 shutdown_connections 2023-10-26T05:51:20.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/3515270317 wait complete. 2023-10-26T05:51:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 Processor -- start 2023-10-26T05:51:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- start start 2023-10-26T05:51:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29680047d0 0x7f29800b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 0x7f2980139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29801f1da0 0x7f29801f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b1b40 con 0x7f2980139650 2023-10-26T05:51:20.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298013a010 con 0x7f29680047d0 2023-10-26T05:51:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29800b0f30 con 0x7f29801f1da0 2023-10-26T05:51:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 0x7f2980139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 0x7f2980139ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60822/0 (socket says 172.21.15.71:60822) 2023-10-26T05:51:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 -- 172.21.15.71:0/312119045 learned_addr learned my addr 172.21.15.71:0/312119045 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297ffff640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29680047d0 0x7f29800b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298491c640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29801f1da0 0x7f29801f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 -- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29801f1da0 msgr2=0x7f29801f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29801f1da0 0x7f29801f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 -- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29680047d0 msgr2=0x7f29800b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29680047d0 0x7f29800b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 -- 172.21.15.71:0/312119045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29800b1210 con 0x7f2980139650 2023-10-26T05:51:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298491c640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29801f1da0 0x7f29801f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297ffff640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29680047d0 0x7f29800b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297f7fe640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 0x7f2980139ad0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f29740016b0 tx=0x7f2974003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740a9040 con 0x7f2980139650 2023-10-26T05:51:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2974006150 con 0x7f2980139650 2023-10-26T05:51:20.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29800b14f0 con 0x7f2980139650 2023-10-26T05:51:20.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29740a57e0 con 0x7f2980139650 2023-10-26T05:51:20.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29801f2760 con 0x7f2980139650 2023-10-26T05:51:20.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.848+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2980066b10 con 0x7f2980139650 2023-10-26T05:51:20.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.852+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29740b5020 con 0x7f2980139650 2023-10-26T05:51:20.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.852+0000 7f297cff9640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f296c066470 0x7f296c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:20.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.852+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29740aa070 con 0x7f2980139650 2023-10-26T05:51:20.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.852+0000 7f297ffff640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f296c066470 0x7f296c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.852+0000 7f297ffff640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f296c066470 0x7f296c068930 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2970005fd0 tx=0x7f2970005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:20.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:20.856+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29740eb1d0 con 0x7f2980139650 2023-10-26T05:51:21.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.020+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29800721e0 con 0x7f2980139650 2023-10-26T05:51:21.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.020+0000 7f297cff9640 1 -- 172.21.15.71:0/312119045 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16484 (secure 0 0 0) 0x7f29740bd020 con 0x7f2980139650 2023-10-26T05:51:21.028 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:21.028 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":86,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:19.326479+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:21.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f296c066470 msgr2=0x7f296c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f296c066470 0x7f296c068930 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2970005fd0 tx=0x7f2970005e60 comp rx=0 tx=0).stop 2023-10-26T05:51:21.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 msgr2=0x7f2980139ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 0x7f2980139ad0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f29740016b0 tx=0x7f2974003c20 comp rx=0 tx=0).stop 2023-10-26T05:51:21.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 shutdown_connections 2023-10-26T05:51:21.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29801f1da0 0x7f29801f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f296c066470 0x7f296c068930 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980139650 0x7f2980139ad0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 --2- 172.21.15.71:0/312119045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29680047d0 0x7f29800b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 >> 172.21.15.71:0/312119045 conn(0x7f2980069770 msgr2=0x7f298013e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:21.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 shutdown_connections 2023-10-26T05:51:21.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.024+0000 7f298637f640 1 -- 172.21.15.71:0/312119045 wait complete. 2023-10-26T05:51:21.045 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:51:21.046 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:51:21.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- 172.21.15.71:0/2275149621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 msgr2=0x7fe514005d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2275149621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe514005d60 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe51001f670 comp rx=0 tx=0).stop 2023-10-26T05:51:21.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- 172.21.15.71:0/2275149621 shutdown_connections 2023-10-26T05:51:21.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2275149621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe514005d60 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- 172.21.15.71:0/2275149621 >> 172.21.15.71:0/2275149621 conn(0x7fe528069770 msgr2=0x7fe528069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:21.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- 172.21.15.71:0/2275149621 shutdown_connections 2023-10-26T05:51:21.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- 172.21.15.71:0/2275149621 wait complete. 2023-10-26T05:51:21.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 Processor -- start 2023-10-26T05:51:21.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- start start 2023-10-26T05:51:21.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe5281e1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5280a86e0 0x7fe5280b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5280b2250 0x7fe52812b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280b2700 con 0x7fe5140058e0 2023-10-26T05:51:21.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280a8b60 con 0x7fe5280b2250 2023-10-26T05:51:21.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52e78c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe528129340 con 0x7fe5280a86e0 2023-10-26T05:51:21.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe5281e1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe5281e1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60858/0 (socket says 172.21.15.71:60858) 2023-10-26T05:51:21.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 -- 172.21.15.71:0/2471317256 learned_addr learned my addr 172.21.15.71:0/2471317256 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52cd29640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5280b2250 0x7fe52812b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe51ffff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5280a86e0 0x7fe5280b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 -- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5280a86e0 msgr2=0x7fe5280b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5280a86e0 0x7fe5280b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 -- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5280b2250 msgr2=0x7fe52812b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5280b2250 0x7fe52812b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 -- 172.21.15.71:0/2471317256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528129620 con 0x7fe5140058e0 2023-10-26T05:51:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe52cd29640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5280b2250 0x7fe52812b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:21.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe51ffff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5280a86e0 0x7fe5280b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.156+0000 7fe527fff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe5281e1370 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe518003da0 tx=0x7fe518001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518001cf0 con 0x7fe5140058e0 2023-10-26T05:51:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe518011640 con 0x7fe5140058e0 2023-10-26T05:51:21.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe528129900 con 0x7fe5140058e0 2023-10-26T05:51:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518011a30 con 0x7fe5140058e0 2023-10-26T05:51:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe528129ec0 con 0x7fe5140058e0 2023-10-26T05:51:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528066b10 con 0x7fe5140058e0 2023-10-26T05:51:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe518011b90 con 0x7fe5140058e0 2023-10-26T05:51:21.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe5257fa640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe5080664a0 0x7fe508068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe518084a40 con 0x7fe5140058e0 2023-10-26T05:51:21.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe51ffff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe5080664a0 0x7fe508068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.160+0000 7fe51ffff640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe5080664a0 0x7fe508068960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe5100b0000 tx=0x7fe51001c410 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:21.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.164+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe518086e60 con 0x7fe5140058e0 2023-10-26T05:51:21.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.304+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5280721e0 con 0x7fe5080664a0 2023-10-26T05:51:21.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.328+0000 7fe5257fa640 1 -- 172.21.15.71:0/2471317256 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283246 (secure 0 0 0) 0x7fe5280721e0 con 0x7fe5080664a0 2023-10-26T05:51:21.336 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:21.337 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:51:21.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe5080664a0 msgr2=0x7fe508068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe5080664a0 0x7fe508068960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe5100b0000 tx=0x7fe51001c410 comp rx=0 tx=0).stop 2023-10-26T05:51:21.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 msgr2=0x7fe5281e1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe5281e1370 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe518003da0 tx=0x7fe518001640 comp rx=0 tx=0).stop 2023-10-26T05:51:21.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 shutdown_connections 2023-10-26T05:51:21.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5280b2250 0x7fe52812b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5280a86e0 0x7fe5280b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe5080664a0 0x7fe508068960 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 --2- 172.21.15.71:0/2471317256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5140058e0 0x7fe5281e1370 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 >> 172.21.15.71:0/2471317256 conn(0x7fe528069770 msgr2=0x7fe5280ab7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:21.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 shutdown_connections 2023-10-26T05:51:21.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.332+0000 7fe52e78c640 1 -- 172.21.15.71:0/2471317256 wait complete. 2023-10-26T05:51:21.353 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2023-10-26T05:51:19.491227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4698,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557801,"statfs":{"total":1600000000,"available":1595188828,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001138"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336101+0000","last_change":"2023-10-26T05:51:18.367188+0000","last_active":"2023-10-26T05:51:19.336101+0000","last_peered":"2023-10-26T05:51:19.336101+0000","last_clean":"2023-10-26T05:51:19.336101+0000","last_became_active":"2023-10-26T05:51:18.365723+0000","last_became_peered":"2023-10-26T05:51:18.365723+0000","last_unstale":"2023-10-26T05:51:19.336101+0000","last_undegraded":"2023-10-26T05:51:19.336101+0000","last_fullsized":"2023-10-26T05:51:19.336101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:25:49.104673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.371423+0000","last_change":"2023-10-26T05:51:18.371277+0000","last_active":"2023-10-26T05:51:18.371423+0000","last_peered":"2023-10-26T05:51:18.371423+0000","last_clean":"2023-10-26T05:51:18.371423+0000","last_became_active":"2023-10-26T05:51:18.370596+0000","last_became_peered":"2023-10-26T05:51:18.370596+0000","last_unstale":"2023-10-26T05:51:18.371423+0000","last_undegraded":"2023-10-26T05:51:18.371423+0000","last_fullsized":"2023-10-26T05:51:18.371423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:27.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357987+0000","last_change":"2023-10-26T05:51:18.357440+0000","last_active":"2023-10-26T05:51:18.357987+0000","last_peered":"2023-10-26T05:51:18.357987+0000","last_clean":"2023-10-26T05:51:18.357987+0000","last_became_active":"2023-10-26T05:51:18.356753+0000","last_became_peered":"2023-10-26T05:51:18.356753+0000","last_unstale":"2023-10-26T05:51:18.357987+0000","last_undegraded":"2023-10-26T05:51:18.357987+0000","last_fullsized":"2023-10-26T05:51:18.357987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:42:56.874532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.364262+0000","last_change":"2023-10-26T05:51:18.364102+0000","last_active":"2023-10-26T05:51:18.364262+0000","last_peered":"2023-10-26T05:51:18.364262+0000","last_clean":"2023-10-26T05:51:18.364262+0000","last_became_active":"2023-10-26T05:51:18.362772+0000","last_became_peered":"2023-10-26T05:51:18.362772+0000","last_unstale":"2023-10-26T05:51:18.364262+0000","last_undegraded":"2023-10-26T05:51:18.364262+0000","last_fullsized":"2023-10-26T05:51:18.364262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:57:58.380157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.355110+0000","last_change":"2023-10-26T05:51:18.354949+0000","last_active":"2023-10-26T05:51:18.355110+0000","last_peered":"2023-10-26T05:51:18.355110+0000","last_clean":"2023-10-26T05:51:18.355110+0000","last_became_active":"2023-10-26T05:51:18.354308+0000","last_became_peered":"2023-10-26T05:51:18.354308+0000","last_unstale":"2023-10-26T05:51:18.355110+0000","last_undegraded":"2023-10-26T05:51:18.355110+0000","last_fullsized":"2023-10-26T05:51:18.355110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:48:27.716287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.376401+0000","last_change":"2023-10-26T05:51:18.376278+0000","last_active":"2023-10-26T05:51:18.376401+0000","last_peered":"2023-10-26T05:51:18.376401+0000","last_clean":"2023-10-26T05:51:18.376401+0000","last_became_active":"2023-10-26T05:51:18.372962+0000","last_became_peered":"2023-10-26T05:51:18.372962+0000","last_unstale":"2023-10-26T05:51:18.376401+0000","last_undegraded":"2023-10-26T05:51:18.376401+0000","last_fullsized":"2023-10-26T05:51:18.376401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:16.515008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441288+0000","last_change":"2023-10-26T05:51:18.360414+0000","last_active":"2023-10-26T05:51:19.441288+0000","last_peered":"2023-10-26T05:51:19.441288+0000","last_clean":"2023-10-26T05:51:19.441288+0000","last_became_active":"2023-10-26T05:51:18.359444+0000","last_became_peered":"2023-10-26T05:51:18.359444+0000","last_unstale":"2023-10-26T05:51:19.441288+0000","last_undegraded":"2023-10-26T05:51:19.441288+0000","last_fullsized":"2023-10-26T05:51:19.441288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:41.440408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.371016+0000","last_change":"2023-10-26T05:51:18.370646+0000","last_active":"2023-10-26T05:51:18.371016+0000","last_peered":"2023-10-26T05:51:18.371016+0000","last_clean":"2023-10-26T05:51:18.371016+0000","last_became_active":"2023-10-26T05:51:18.368578+0000","last_became_peered":"2023-10-26T05:51:18.368578+0000","last_unstale":"2023-10-26T05:51:18.371016+0000","last_undegraded":"2023-10-26T05:51:18.371016+0000","last_fullsized":"2023-10-26T05:51:18.371016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:04.818950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.354889+0000","last_change":"2023-10-26T05:51:18.354639+0000","last_active":"2023-10-26T05:51:18.354889+0000","last_peered":"2023-10-26T05:51:18.354889+0000","last_clean":"2023-10-26T05:51:18.354889+0000","last_became_active":"2023-10-26T05:51:18.354022+0000","last_became_peered":"2023-10-26T05:51:18.354022+0000","last_unstale":"2023-10-26T05:51:18.354889+0000","last_undegraded":"2023-10-26T05:51:18.354889+0000","last_fullsized":"2023-10-26T05:51:18.354889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:21:24.370935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337356+0000","last_change":"2023-10-26T05:51:18.339285+0000","last_active":"2023-10-26T05:51:19.337356+0000","last_peered":"2023-10-26T05:51:19.337356+0000","last_clean":"2023-10-26T05:51:19.337356+0000","last_became_active":"2023-10-26T05:51:18.338736+0000","last_became_peered":"2023-10-26T05:51:18.338736+0000","last_unstale":"2023-10-26T05:51:19.337356+0000","last_undegraded":"2023-10-26T05:51:19.337356+0000","last_fullsized":"2023-10-26T05:51:19.337356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:07:54.940812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336623+0000","last_change":"2023-10-26T05:51:18.355190+0000","last_active":"2023-10-26T05:51:19.336623+0000","last_peered":"2023-10-26T05:51:19.336623+0000","last_clean":"2023-10-26T05:51:19.336623+0000","last_became_active":"2023-10-26T05:51:18.354517+0000","last_became_peered":"2023-10-26T05:51:18.354517+0000","last_unstale":"2023-10-26T05:51:19.336623+0000","last_undegraded":"2023-10-26T05:51:19.336623+0000","last_fullsized":"2023-10-26T05:51:19.336623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:07.797423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.339847+0000","last_change":"2023-10-26T05:51:18.339221+0000","last_active":"2023-10-26T05:51:18.339847+0000","last_peered":"2023-10-26T05:51:18.339847+0000","last_clean":"2023-10-26T05:51:18.339847+0000","last_became_active":"2023-10-26T05:51:18.337707+0000","last_became_peered":"2023-10-26T05:51:18.337707+0000","last_unstale":"2023-10-26T05:51:18.339847+0000","last_undegraded":"2023-10-26T05:51:18.339847+0000","last_fullsized":"2023-10-26T05:51:18.339847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:23.654693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332089+0000","last_change":"2023-10-26T05:51:18.369316+0000","last_active":"2023-10-26T05:51:19.332089+0000","last_peered":"2023-10-26T05:51:19.332089+0000","last_clean":"2023-10-26T05:51:19.332089+0000","last_became_active":"2023-10-26T05:51:18.368284+0000","last_became_peered":"2023-10-26T05:51:18.368284+0000","last_unstale":"2023-10-26T05:51:19.332089+0000","last_undegraded":"2023-10-26T05:51:19.332089+0000","last_fullsized":"2023-10-26T05:51:19.332089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:32.679461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.354843+0000","last_change":"2023-10-26T05:51:18.353408+0000","last_active":"2023-10-26T05:51:18.354843+0000","last_peered":"2023-10-26T05:51:18.354843+0000","last_clean":"2023-10-26T05:51:18.354843+0000","last_became_active":"2023-10-26T05:51:18.352684+0000","last_became_peered":"2023-10-26T05:51:18.352684+0000","last_unstale":"2023-10-26T05:51:18.354843+0000","last_undegraded":"2023-10-26T05:51:18.354843+0000","last_fullsized":"2023-10-26T05:51:18.354843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:37:36.590764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331528+0000","last_change":"2023-10-26T05:51:18.345200+0000","last_active":"2023-10-26T05:51:19.331528+0000","last_peered":"2023-10-26T05:51:19.331528+0000","last_clean":"2023-10-26T05:51:19.331528+0000","last_became_active":"2023-10-26T05:51:18.344552+0000","last_became_peered":"2023-10-26T05:51:18.344552+0000","last_unstale":"2023-10-26T05:51:19.331528+0000","last_undegraded":"2023-10-26T05:51:19.331528+0000","last_fullsized":"2023-10-26T05:51:19.331528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:44:49.548875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336453+0000","last_change":"2023-10-26T05:51:18.375093+0000","last_active":"2023-10-26T05:51:19.336453+0000","last_peered":"2023-10-26T05:51:19.336453+0000","last_clean":"2023-10-26T05:51:19.336453+0000","last_became_active":"2023-10-26T05:51:18.374324+0000","last_became_peered":"2023-10-26T05:51:18.374324+0000","last_unstale":"2023-10-26T05:51:19.336453+0000","last_undegraded":"2023-10-26T05:51:19.336453+0000","last_fullsized":"2023-10-26T05:51:19.336453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:41.440408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357903+0000","last_change":"2023-10-26T05:51:18.357104+0000","last_active":"2023-10-26T05:51:18.357903+0000","last_peered":"2023-10-26T05:51:18.357903+0000","last_clean":"2023-10-26T05:51:18.357903+0000","last_became_active":"2023-10-26T05:51:18.355589+0000","last_became_peered":"2023-10-26T05:51:18.355589+0000","last_unstale":"2023-10-26T05:51:18.357903+0000","last_undegraded":"2023-10-26T05:51:18.357903+0000","last_fullsized":"2023-10-26T05:51:18.357903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:55:27.272312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.369398+0000","last_change":"2023-10-26T05:51:18.369226+0000","last_active":"2023-10-26T05:51:18.369398+0000","last_peered":"2023-10-26T05:51:18.369398+0000","last_clean":"2023-10-26T05:51:18.369398+0000","last_became_active":"2023-10-26T05:51:18.368593+0000","last_became_peered":"2023-10-26T05:51:18.368593+0000","last_unstale":"2023-10-26T05:51:18.369398+0000","last_undegraded":"2023-10-26T05:51:18.369398+0000","last_fullsized":"2023-10-26T05:51:18.369398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:05.450043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.368030+0000","last_change":"2023-10-26T05:51:18.367654+0000","last_active":"2023-10-26T05:51:18.368030+0000","last_peered":"2023-10-26T05:51:18.368030+0000","last_clean":"2023-10-26T05:51:18.368030+0000","last_became_active":"2023-10-26T05:51:18.365301+0000","last_became_peered":"2023-10-26T05:51:18.365301+0000","last_unstale":"2023-10-26T05:51:18.368030+0000","last_undegraded":"2023-10-26T05:51:18.368030+0000","last_fullsized":"2023-10-26T05:51:18.368030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:57:58.380157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331611+0000","last_change":"2023-10-26T05:51:18.357483+0000","last_active":"2023-10-26T05:51:19.331611+0000","last_peered":"2023-10-26T05:51:19.331611+0000","last_clean":"2023-10-26T05:51:19.331611+0000","last_became_active":"2023-10-26T05:51:18.356877+0000","last_became_peered":"2023-10-26T05:51:18.356877+0000","last_unstale":"2023-10-26T05:51:19.331611+0000","last_undegraded":"2023-10-26T05:51:19.331611+0000","last_fullsized":"2023-10-26T05:51:19.331611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:00:27.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336710+0000","last_change":"2023-10-26T05:51:18.352441+0000","last_active":"2023-10-26T05:51:19.336710+0000","last_peered":"2023-10-26T05:51:19.336710+0000","last_clean":"2023-10-26T05:51:19.336710+0000","last_became_active":"2023-10-26T05:51:18.351797+0000","last_became_peered":"2023-10-26T05:51:18.351797+0000","last_unstale":"2023-10-26T05:51:19.336710+0000","last_undegraded":"2023-10-26T05:51:19.336710+0000","last_fullsized":"2023-10-26T05:51:19.336710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:47.574397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335875+0000","last_change":"2023-10-26T05:51:18.341218+0000","last_active":"2023-10-26T05:51:19.335875+0000","last_peered":"2023-10-26T05:51:19.335875+0000","last_clean":"2023-10-26T05:51:19.335875+0000","last_became_active":"2023-10-26T05:51:18.340721+0000","last_became_peered":"2023-10-26T05:51:18.340721+0000","last_unstale":"2023-10-26T05:51:19.335875+0000","last_undegraded":"2023-10-26T05:51:19.335875+0000","last_fullsized":"2023-10-26T05:51:19.335875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:23.878227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337256+0000","last_change":"2023-10-26T05:51:18.362537+0000","last_active":"2023-10-26T05:51:19.337256+0000","last_peered":"2023-10-26T05:51:19.337256+0000","last_clean":"2023-10-26T05:51:19.337256+0000","last_became_active":"2023-10-26T05:51:18.361869+0000","last_became_peered":"2023-10-26T05:51:18.361869+0000","last_unstale":"2023-10-26T05:51:19.337256+0000","last_undegraded":"2023-10-26T05:51:19.337256+0000","last_fullsized":"2023-10-26T05:51:19.337256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:49.311401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335228+0000","last_change":"2023-10-26T05:51:18.366512+0000","last_active":"2023-10-26T05:51:19.335228+0000","last_peered":"2023-10-26T05:51:19.335228+0000","last_clean":"2023-10-26T05:51:19.335228+0000","last_became_active":"2023-10-26T05:51:18.363896+0000","last_became_peered":"2023-10-26T05:51:18.363896+0000","last_unstale":"2023-10-26T05:51:19.335228+0000","last_undegraded":"2023-10-26T05:51:19.335228+0000","last_fullsized":"2023-10-26T05:51:19.335228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:57.308958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337631+0000","last_change":"2023-10-26T05:51:18.360206+0000","last_active":"2023-10-26T05:51:19.337631+0000","last_peered":"2023-10-26T05:51:19.337631+0000","last_clean":"2023-10-26T05:51:19.337631+0000","last_became_active":"2023-10-26T05:51:18.359418+0000","last_became_peered":"2023-10-26T05:51:18.359418+0000","last_unstale":"2023-10-26T05:51:19.337631+0000","last_undegraded":"2023-10-26T05:51:19.337631+0000","last_fullsized":"2023-10-26T05:51:19.337631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:10.510054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441433+0000","last_change":"2023-10-26T05:51:18.374920+0000","last_active":"2023-10-26T05:51:19.441433+0000","last_peered":"2023-10-26T05:51:19.441433+0000","last_clean":"2023-10-26T05:51:19.441433+0000","last_became_active":"2023-10-26T05:51:18.374377+0000","last_became_peered":"2023-10-26T05:51:18.374377+0000","last_unstale":"2023-10-26T05:51:19.441433+0000","last_undegraded":"2023-10-26T05:51:19.441433+0000","last_fullsized":"2023-10-26T05:51:19.441433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:07:54.940812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353590+0000","last_change":"2023-10-26T05:51:18.353179+0000","last_active":"2023-10-26T05:51:18.353590+0000","last_peered":"2023-10-26T05:51:18.353590+0000","last_clean":"2023-10-26T05:51:18.353590+0000","last_became_active":"2023-10-26T05:51:18.352601+0000","last_became_peered":"2023-10-26T05:51:18.352601+0000","last_unstale":"2023-10-26T05:51:18.353590+0000","last_undegraded":"2023-10-26T05:51:18.353590+0000","last_fullsized":"2023-10-26T05:51:18.353590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:30:57.049868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335605+0000","last_change":"2023-10-26T05:51:18.337400+0000","last_active":"2023-10-26T05:51:19.335605+0000","last_peered":"2023-10-26T05:51:19.335605+0000","last_clean":"2023-10-26T05:51:19.335605+0000","last_became_active":"2023-10-26T05:51:18.336799+0000","last_became_peered":"2023-10-26T05:51:18.336799+0000","last_unstale":"2023-10-26T05:51:19.335605+0000","last_undegraded":"2023-10-26T05:51:19.335605+0000","last_fullsized":"2023-10-26T05:51:19.335605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:55.156799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353509+0000","last_change":"2023-10-26T05:51:18.352948+0000","last_active":"2023-10-26T05:51:18.353509+0000","last_peered":"2023-10-26T05:51:18.353509+0000","last_clean":"2023-10-26T05:51:18.353509+0000","last_became_active":"2023-10-26T05:51:18.352356+0000","last_became_peered":"2023-10-26T05:51:18.352356+0000","last_unstale":"2023-10-26T05:51:18.353509+0000","last_undegraded":"2023-10-26T05:51:18.353509+0000","last_fullsized":"2023-10-26T05:51:18.353509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:04:18.025028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441680+0000","last_change":"2023-10-26T05:51:18.364361+0000","last_active":"2023-10-26T05:51:19.441680+0000","last_peered":"2023-10-26T05:51:19.441680+0000","last_clean":"2023-10-26T05:51:19.441680+0000","last_became_active":"2023-10-26T05:51:18.363759+0000","last_became_peered":"2023-10-26T05:51:18.363759+0000","last_unstale":"2023-10-26T05:51:19.441680+0000","last_undegraded":"2023-10-26T05:51:19.441680+0000","last_fullsized":"2023-10-26T05:51:19.441680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:48:48.160737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441781+0000","last_change":"2023-10-26T05:51:18.363247+0000","last_active":"2023-10-26T05:51:19.441781+0000","last_peered":"2023-10-26T05:51:19.441781+0000","last_clean":"2023-10-26T05:51:19.441781+0000","last_became_active":"2023-10-26T05:51:18.362641+0000","last_became_peered":"2023-10-26T05:51:18.362641+0000","last_unstale":"2023-10-26T05:51:19.441781+0000","last_undegraded":"2023-10-26T05:51:19.441781+0000","last_fullsized":"2023-10-26T05:51:19.441781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:24:38.265270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.368001+0000","last_change":"2023-10-26T05:51:18.366448+0000","last_active":"2023-10-26T05:51:18.368001+0000","last_peered":"2023-10-26T05:51:18.368001+0000","last_clean":"2023-10-26T05:51:18.368001+0000","last_became_active":"2023-10-26T05:51:18.365782+0000","last_became_peered":"2023-10-26T05:51:18.365782+0000","last_unstale":"2023-10-26T05:51:18.368001+0000","last_undegraded":"2023-10-26T05:51:18.368001+0000","last_fullsized":"2023-10-26T05:51:18.368001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:05.450043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336202+0000","last_change":"2023-10-26T05:51:18.366920+0000","last_active":"2023-10-26T05:51:19.336202+0000","last_peered":"2023-10-26T05:51:19.336202+0000","last_clean":"2023-10-26T05:51:19.336202+0000","last_became_active":"2023-10-26T05:51:18.364803+0000","last_became_peered":"2023-10-26T05:51:18.364803+0000","last_unstale":"2023-10-26T05:51:19.336202+0000","last_undegraded":"2023-10-26T05:51:19.336202+0000","last_fullsized":"2023-10-26T05:51:19.336202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:02.233918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337501+0000","last_change":"2023-10-26T05:51:17.326442+0000","last_active":"2023-10-26T05:51:19.337501+0000","last_peered":"2023-10-26T05:51:19.337501+0000","last_clean":"2023-10-26T05:51:19.337501+0000","last_became_active":"2023-10-26T05:51:17.325660+0000","last_became_peered":"2023-10-26T05:51:17.325660+0000","last_unstale":"2023-10-26T05:51:19.337501+0000","last_undegraded":"2023-10-26T05:51:19.337501+0000","last_fullsized":"2023-10-26T05:51:19.337501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:40.428460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.361971+0000","last_change":"2023-10-26T05:51:18.361801+0000","last_active":"2023-10-26T05:51:18.361971+0000","last_peered":"2023-10-26T05:51:18.361971+0000","last_clean":"2023-10-26T05:51:18.361971+0000","last_became_active":"2023-10-26T05:51:18.361023+0000","last_became_peered":"2023-10-26T05:51:18.361023+0000","last_unstale":"2023-10-26T05:51:18.361971+0000","last_undegraded":"2023-10-26T05:51:18.361971+0000","last_fullsized":"2023-10-26T05:51:18.361971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:52.015534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332155+0000","last_change":"2023-10-26T05:51:17.315511+0000","last_active":"2023-10-26T05:51:19.332155+0000","last_peered":"2023-10-26T05:51:19.332155+0000","last_clean":"2023-10-26T05:51:19.332155+0000","last_became_active":"2023-10-26T05:51:17.315173+0000","last_became_peered":"2023-10-26T05:51:17.315173+0000","last_unstale":"2023-10-26T05:51:19.332155+0000","last_undegraded":"2023-10-26T05:51:19.332155+0000","last_fullsized":"2023-10-26T05:51:19.332155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:55.249322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.376079+0000","last_change":"2023-10-26T05:51:18.375881+0000","last_active":"2023-10-26T05:51:18.376079+0000","last_peered":"2023-10-26T05:51:18.376079+0000","last_clean":"2023-10-26T05:51:18.376079+0000","last_became_active":"2023-10-26T05:51:18.371860+0000","last_became_peered":"2023-10-26T05:51:18.371860+0000","last_unstale":"2023-10-26T05:51:18.376079+0000","last_undegraded":"2023-10-26T05:51:18.376079+0000","last_fullsized":"2023-10-26T05:51:18.376079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:19:45.471869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.338072+0000","last_change":"2023-10-26T05:51:17.327271+0000","last_active":"2023-10-26T05:51:19.338072+0000","last_peered":"2023-10-26T05:51:19.338072+0000","last_clean":"2023-10-26T05:51:19.338072+0000","last_became_active":"2023-10-26T05:51:17.325095+0000","last_became_peered":"2023-10-26T05:51:17.325095+0000","last_unstale":"2023-10-26T05:51:19.338072+0000","last_undegraded":"2023-10-26T05:51:19.338072+0000","last_fullsized":"2023-10-26T05:51:19.338072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:39:21.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336226+0000","last_change":"2023-10-26T05:51:18.355735+0000","last_active":"2023-10-26T05:51:19.336226+0000","last_peered":"2023-10-26T05:51:19.336226+0000","last_clean":"2023-10-26T05:51:19.336226+0000","last_became_active":"2023-10-26T05:51:18.355193+0000","last_became_peered":"2023-10-26T05:51:18.355193+0000","last_unstale":"2023-10-26T05:51:19.336226+0000","last_undegraded":"2023-10-26T05:51:19.336226+0000","last_fullsized":"2023-10-26T05:51:19.336226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:55:38.473204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.383405+0000","last_change":"2023-10-26T05:51:17.337841+0000","last_active":"2023-10-26T05:51:18.383405+0000","last_peered":"2023-10-26T05:51:18.383405+0000","last_clean":"2023-10-26T05:51:18.383405+0000","last_became_active":"2023-10-26T05:51:17.337304+0000","last_became_peered":"2023-10-26T05:51:17.337304+0000","last_unstale":"2023-10-26T05:51:18.383405+0000","last_undegraded":"2023-10-26T05:51:18.383405+0000","last_fullsized":"2023-10-26T05:51:18.383405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:41:02.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441375+0000","last_change":"2023-10-26T05:51:18.365437+0000","last_active":"2023-10-26T05:51:19.441375+0000","last_peered":"2023-10-26T05:51:19.441375+0000","last_clean":"2023-10-26T05:51:19.441375+0000","last_became_active":"2023-10-26T05:51:18.363033+0000","last_became_peered":"2023-10-26T05:51:18.363033+0000","last_unstale":"2023-10-26T05:51:19.441375+0000","last_undegraded":"2023-10-26T05:51:19.441375+0000","last_fullsized":"2023-10-26T05:51:19.441375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:47.574397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.378110+0000","last_change":"2023-10-26T05:51:17.319266+0000","last_active":"2023-10-26T05:51:18.378110+0000","last_peered":"2023-10-26T05:51:18.378110+0000","last_clean":"2023-10-26T05:51:18.378110+0000","last_became_active":"2023-10-26T05:51:17.318903+0000","last_became_peered":"2023-10-26T05:51:17.318903+0000","last_unstale":"2023-10-26T05:51:18.378110+0000","last_undegraded":"2023-10-26T05:51:18.378110+0000","last_fullsized":"2023-10-26T05:51:18.378110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:07.206116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336223+0000","last_change":"2023-10-26T05:51:18.352565+0000","last_active":"2023-10-26T05:51:19.336223+0000","last_peered":"2023-10-26T05:51:19.336223+0000","last_clean":"2023-10-26T05:51:19.336223+0000","last_became_active":"2023-10-26T05:51:18.351931+0000","last_became_peered":"2023-10-26T05:51:18.351931+0000","last_unstale":"2023-10-26T05:51:19.336223+0000","last_undegraded":"2023-10-26T05:51:19.336223+0000","last_fullsized":"2023-10-26T05:51:19.336223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:00:04.009796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.387711+0000","last_change":"2023-10-26T05:51:17.330054+0000","last_active":"2023-10-26T05:51:18.387711+0000","last_peered":"2023-10-26T05:51:18.387711+0000","last_clean":"2023-10-26T05:51:18.387711+0000","last_became_active":"2023-10-26T05:51:17.329449+0000","last_became_peered":"2023-10-26T05:51:17.329449+0000","last_unstale":"2023-10-26T05:51:18.387711+0000","last_undegraded":"2023-10-26T05:51:18.387711+0000","last_fullsized":"2023-10-26T05:51:18.387711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:55:01.061094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.337887+0000","last_change":"2023-10-26T05:51:18.337654+0000","last_active":"2023-10-26T05:51:18.337887+0000","last_peered":"2023-10-26T05:51:18.337887+0000","last_clean":"2023-10-26T05:51:18.337887+0000","last_became_active":"2023-10-26T05:51:18.337020+0000","last_became_peered":"2023-10-26T05:51:18.337020+0000","last_unstale":"2023-10-26T05:51:18.337887+0000","last_undegraded":"2023-10-26T05:51:18.337887+0000","last_fullsized":"2023-10-26T05:51:18.337887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:38.011241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.331017+0000","last_change":"2023-10-26T05:51:17.318531+0000","last_active":"2023-10-26T05:51:18.331017+0000","last_peered":"2023-10-26T05:51:18.331017+0000","last_clean":"2023-10-26T05:51:18.331017+0000","last_became_active":"2023-10-26T05:51:17.318144+0000","last_became_peered":"2023-10-26T05:51:17.318144+0000","last_unstale":"2023-10-26T05:51:18.331017+0000","last_undegraded":"2023-10-26T05:51:18.331017+0000","last_fullsized":"2023-10-26T05:51:18.331017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:43.449048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331962+0000","last_change":"2023-10-26T05:51:18.359072+0000","last_active":"2023-10-26T05:51:19.331962+0000","last_peered":"2023-10-26T05:51:19.331962+0000","last_clean":"2023-10-26T05:51:19.331962+0000","last_became_active":"2023-10-26T05:51:18.358411+0000","last_became_peered":"2023-10-26T05:51:18.358411+0000","last_unstale":"2023-10-26T05:51:19.331962+0000","last_undegraded":"2023-10-26T05:51:19.331962+0000","last_fullsized":"2023-10-26T05:51:19.331962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:49.131461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.333643+0000","last_change":"2023-10-26T05:51:17.317027+0000","last_active":"2023-10-26T05:51:18.333643+0000","last_peered":"2023-10-26T05:51:18.333643+0000","last_clean":"2023-10-26T05:51:18.333643+0000","last_became_active":"2023-10-26T05:51:17.316340+0000","last_became_peered":"2023-10-26T05:51:17.316340+0000","last_unstale":"2023-10-26T05:51:18.333643+0000","last_undegraded":"2023-10-26T05:51:18.333643+0000","last_fullsized":"2023-10-26T05:51:18.333643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:43:33.890168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441634+0000","last_change":"2023-10-26T05:51:18.355463+0000","last_active":"2023-10-26T05:51:19.441634+0000","last_peered":"2023-10-26T05:51:19.441634+0000","last_clean":"2023-10-26T05:51:19.441634+0000","last_became_active":"2023-10-26T05:51:18.354880+0000","last_became_peered":"2023-10-26T05:51:18.354880+0000","last_unstale":"2023-10-26T05:51:19.441634+0000","last_undegraded":"2023-10-26T05:51:19.441634+0000","last_fullsized":"2023-10-26T05:51:19.441634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:10.510054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336817+0000","last_change":"2023-10-26T05:51:17.319515+0000","last_active":"2023-10-26T05:51:19.336817+0000","last_peered":"2023-10-26T05:51:19.336817+0000","last_clean":"2023-10-26T05:51:19.336817+0000","last_became_active":"2023-10-26T05:51:17.318899+0000","last_became_peered":"2023-10-26T05:51:17.318899+0000","last_unstale":"2023-10-26T05:51:19.336817+0000","last_undegraded":"2023-10-26T05:51:19.336817+0000","last_fullsized":"2023-10-26T05:51:19.336817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:25:48.030863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336927+0000","last_change":"2023-10-26T05:51:18.373896+0000","last_active":"2023-10-26T05:51:19.336927+0000","last_peered":"2023-10-26T05:51:19.336927+0000","last_clean":"2023-10-26T05:51:19.336927+0000","last_became_active":"2023-10-26T05:51:18.373282+0000","last_became_peered":"2023-10-26T05:51:18.373282+0000","last_unstale":"2023-10-26T05:51:19.336927+0000","last_undegraded":"2023-10-26T05:51:19.336927+0000","last_fullsized":"2023-10-26T05:51:19.336927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:00.843086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.345023+0000","last_change":"2023-10-26T05:51:17.324960+0000","last_active":"2023-10-26T05:51:18.345023+0000","last_peered":"2023-10-26T05:51:18.345023+0000","last_clean":"2023-10-26T05:51:18.345023+0000","last_became_active":"2023-10-26T05:51:17.324347+0000","last_became_peered":"2023-10-26T05:51:17.324347+0000","last_unstale":"2023-10-26T05:51:18.345023+0000","last_undegraded":"2023-10-26T05:51:18.345023+0000","last_fullsized":"2023-10-26T05:51:18.345023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:43.449048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.370672+0000","last_change":"2023-10-26T05:51:18.370529+0000","last_active":"2023-10-26T05:51:18.370672+0000","last_peered":"2023-10-26T05:51:18.370672+0000","last_clean":"2023-10-26T05:51:18.370672+0000","last_became_active":"2023-10-26T05:51:18.368687+0000","last_became_peered":"2023-10-26T05:51:18.368687+0000","last_unstale":"2023-10-26T05:51:18.370672+0000","last_undegraded":"2023-10-26T05:51:18.370672+0000","last_fullsized":"2023-10-26T05:51:18.370672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:23.654693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.380378+0000","last_change":"2023-10-26T05:51:17.328324+0000","last_active":"2023-10-26T05:51:18.380378+0000","last_peered":"2023-10-26T05:51:18.380378+0000","last_clean":"2023-10-26T05:51:18.380378+0000","last_became_active":"2023-10-26T05:51:17.328052+0000","last_became_peered":"2023-10-26T05:51:17.328052+0000","last_unstale":"2023-10-26T05:51:18.380378+0000","last_undegraded":"2023-10-26T05:51:18.380378+0000","last_fullsized":"2023-10-26T05:51:18.380378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:24.164396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.337178+0000","last_change":"2023-10-26T05:51:18.337053+0000","last_active":"2023-10-26T05:51:18.337178+0000","last_peered":"2023-10-26T05:51:18.337178+0000","last_clean":"2023-10-26T05:51:18.337178+0000","last_became_active":"2023-10-26T05:51:18.336423+0000","last_became_peered":"2023-10-26T05:51:18.336423+0000","last_unstale":"2023-10-26T05:51:18.337178+0000","last_undegraded":"2023-10-26T05:51:18.337178+0000","last_fullsized":"2023-10-26T05:51:18.337178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:21:33.906574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.381999+0000","last_change":"2023-10-26T05:51:17.315688+0000","last_active":"2023-10-26T05:51:18.381999+0000","last_peered":"2023-10-26T05:51:18.381999+0000","last_clean":"2023-10-26T05:51:18.381999+0000","last_became_active":"2023-10-26T05:51:17.315127+0000","last_became_peered":"2023-10-26T05:51:17.315127+0000","last_unstale":"2023-10-26T05:51:18.381999+0000","last_undegraded":"2023-10-26T05:51:18.381999+0000","last_fullsized":"2023-10-26T05:51:18.381999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:19:44.459921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337231+0000","last_change":"2023-10-26T05:51:18.367616+0000","last_active":"2023-10-26T05:51:19.337231+0000","last_peered":"2023-10-26T05:51:19.337231+0000","last_clean":"2023-10-26T05:51:19.337231+0000","last_became_active":"2023-10-26T05:51:18.366564+0000","last_became_peered":"2023-10-26T05:51:18.366564+0000","last_unstale":"2023-10-26T05:51:19.337231+0000","last_undegraded":"2023-10-26T05:51:19.337231+0000","last_fullsized":"2023-10-26T05:51:19.337231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:44.582270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.380106+0000","last_change":"2023-10-26T05:51:17.317567+0000","last_active":"2023-10-26T05:51:18.380106+0000","last_peered":"2023-10-26T05:51:18.380106+0000","last_clean":"2023-10-26T05:51:18.380106+0000","last_became_active":"2023-10-26T05:51:17.317197+0000","last_became_peered":"2023-10-26T05:51:17.317197+0000","last_unstale":"2023-10-26T05:51:18.380106+0000","last_undegraded":"2023-10-26T05:51:18.380106+0000","last_fullsized":"2023-10-26T05:51:18.380106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:16.406408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.443278+0000","last_change":"2023-10-26T05:51:18.357275+0000","last_active":"2023-10-26T05:51:19.443278+0000","last_peered":"2023-10-26T05:51:19.443278+0000","last_clean":"2023-10-26T05:51:19.443278+0000","last_became_active":"2023-10-26T05:51:18.356457+0000","last_became_peered":"2023-10-26T05:51:18.356457+0000","last_unstale":"2023-10-26T05:51:19.443278+0000","last_undegraded":"2023-10-26T05:51:19.443278+0000","last_fullsized":"2023-10-26T05:51:19.443278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:16:45.591903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.346977+0000","last_change":"2023-10-26T05:51:17.327090+0000","last_active":"2023-10-26T05:51:18.346977+0000","last_peered":"2023-10-26T05:51:18.346977+0000","last_clean":"2023-10-26T05:51:18.346977+0000","last_became_active":"2023-10-26T05:51:17.326080+0000","last_became_peered":"2023-10-26T05:51:17.326080+0000","last_unstale":"2023-10-26T05:51:18.346977+0000","last_undegraded":"2023-10-26T05:51:18.346977+0000","last_fullsized":"2023-10-26T05:51:18.346977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:07.209178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332372+0000","last_change":"2023-10-26T05:51:18.369459+0000","last_active":"2023-10-26T05:51:19.332372+0000","last_peered":"2023-10-26T05:51:19.332372+0000","last_clean":"2023-10-26T05:51:19.332372+0000","last_became_active":"2023-10-26T05:51:18.368982+0000","last_became_peered":"2023-10-26T05:51:18.368982+0000","last_unstale":"2023-10-26T05:51:19.332372+0000","last_undegraded":"2023-10-26T05:51:19.332372+0000","last_fullsized":"2023-10-26T05:51:19.332372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:05.819900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.442289+0000","last_change":"2023-10-26T05:51:17.319285+0000","last_active":"2023-10-26T05:51:19.442289+0000","last_peered":"2023-10-26T05:51:19.442289+0000","last_clean":"2023-10-26T05:51:19.442289+0000","last_became_active":"2023-10-26T05:51:17.317741+0000","last_became_peered":"2023-10-26T05:51:17.317741+0000","last_unstale":"2023-10-26T05:51:19.442289+0000","last_undegraded":"2023-10-26T05:51:19.442289+0000","last_fullsized":"2023-10-26T05:51:19.442289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:25:48.030863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357859+0000","last_change":"2023-10-26T05:51:18.356747+0000","last_active":"2023-10-26T05:51:18.357859+0000","last_peered":"2023-10-26T05:51:18.357859+0000","last_clean":"2023-10-26T05:51:18.357859+0000","last_became_active":"2023-10-26T05:51:18.354766+0000","last_became_peered":"2023-10-26T05:51:18.354766+0000","last_unstale":"2023-10-26T05:51:18.357859+0000","last_undegraded":"2023-10-26T05:51:18.357859+0000","last_fullsized":"2023-10-26T05:51:18.357859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:00:23.324742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335883+0000","last_change":"2023-10-26T05:51:17.328932+0000","last_active":"2023-10-26T05:51:19.335883+0000","last_peered":"2023-10-26T05:51:19.335883+0000","last_clean":"2023-10-26T05:51:19.335883+0000","last_became_active":"2023-10-26T05:51:17.327451+0000","last_became_peered":"2023-10-26T05:51:17.327451+0000","last_unstale":"2023-10-26T05:51:19.335883+0000","last_undegraded":"2023-10-26T05:51:19.335883+0000","last_fullsized":"2023-10-26T05:51:19.335883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:57.442408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337838+0000","last_change":"2023-10-26T05:51:18.338377+0000","last_active":"2023-10-26T05:51:19.337838+0000","last_peered":"2023-10-26T05:51:19.337838+0000","last_clean":"2023-10-26T05:51:19.337838+0000","last_became_active":"2023-10-26T05:51:18.337873+0000","last_became_peered":"2023-10-26T05:51:18.337873+0000","last_unstale":"2023-10-26T05:51:19.337838+0000","last_undegraded":"2023-10-26T05:51:19.337838+0000","last_fullsized":"2023-10-26T05:51:19.337838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:48:48.160737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.384867+0000","last_change":"2023-10-26T05:51:17.326903+0000","last_active":"2023-10-26T05:51:18.384867+0000","last_peered":"2023-10-26T05:51:18.384867+0000","last_clean":"2023-10-26T05:51:18.384867+0000","last_became_active":"2023-10-26T05:51:17.322566+0000","last_became_peered":"2023-10-26T05:51:17.322566+0000","last_unstale":"2023-10-26T05:51:18.384867+0000","last_undegraded":"2023-10-26T05:51:18.384867+0000","last_fullsized":"2023-10-26T05:51:18.384867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:23:54.442123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336590+0000","last_change":"2023-10-26T05:51:18.367218+0000","last_active":"2023-10-26T05:51:19.336590+0000","last_peered":"2023-10-26T05:51:19.336590+0000","last_clean":"2023-10-26T05:51:19.336590+0000","last_became_active":"2023-10-26T05:51:18.365867+0000","last_became_peered":"2023-10-26T05:51:18.365867+0000","last_unstale":"2023-10-26T05:51:19.336590+0000","last_undegraded":"2023-10-26T05:51:19.336590+0000","last_fullsized":"2023-10-26T05:51:19.336590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:07:05.937272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.338196+0000","last_change":"2023-10-26T05:51:17.327760+0000","last_active":"2023-10-26T05:51:19.338196+0000","last_peered":"2023-10-26T05:51:19.338196+0000","last_clean":"2023-10-26T05:51:19.338196+0000","last_became_active":"2023-10-26T05:51:17.326336+0000","last_became_peered":"2023-10-26T05:51:17.326336+0000","last_unstale":"2023-10-26T05:51:19.338196+0000","last_undegraded":"2023-10-26T05:51:19.338196+0000","last_fullsized":"2023-10-26T05:51:19.338196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:51:54.919933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.339891+0000","last_change":"2023-10-26T05:51:18.339675+0000","last_active":"2023-10-26T05:51:18.339891+0000","last_peered":"2023-10-26T05:51:18.339891+0000","last_clean":"2023-10-26T05:51:18.339891+0000","last_became_active":"2023-10-26T05:51:18.338553+0000","last_became_peered":"2023-10-26T05:51:18.338553+0000","last_unstale":"2023-10-26T05:51:18.339891+0000","last_undegraded":"2023-10-26T05:51:18.339891+0000","last_fullsized":"2023-10-26T05:51:18.339891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:27.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.390289+0000","last_change":"2023-10-26T05:51:17.326689+0000","last_active":"2023-10-26T05:51:18.390289+0000","last_peered":"2023-10-26T05:51:18.390289+0000","last_clean":"2023-10-26T05:51:18.390289+0000","last_became_active":"2023-10-26T05:51:17.325254+0000","last_became_peered":"2023-10-26T05:51:17.325254+0000","last_unstale":"2023-10-26T05:51:18.390289+0000","last_undegraded":"2023-10-26T05:51:18.390289+0000","last_fullsized":"2023-10-26T05:51:18.390289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:45.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.337125+0000","last_change":"2023-10-26T05:51:18.336932+0000","last_active":"2023-10-26T05:51:18.337125+0000","last_peered":"2023-10-26T05:51:18.337125+0000","last_clean":"2023-10-26T05:51:18.337125+0000","last_became_active":"2023-10-26T05:51:18.336291+0000","last_became_peered":"2023-10-26T05:51:18.336291+0000","last_unstale":"2023-10-26T05:51:18.337125+0000","last_undegraded":"2023-10-26T05:51:18.337125+0000","last_fullsized":"2023-10-26T05:51:18.337125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:15:12.261188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335974+0000","last_change":"2023-10-26T05:51:17.321687+0000","last_active":"2023-10-26T05:51:19.335974+0000","last_peered":"2023-10-26T05:51:19.335974+0000","last_clean":"2023-10-26T05:51:19.335974+0000","last_became_active":"2023-10-26T05:51:17.321081+0000","last_became_peered":"2023-10-26T05:51:17.321081+0000","last_unstale":"2023-10-26T05:51:19.335974+0000","last_undegraded":"2023-10-26T05:51:19.335974+0000","last_fullsized":"2023-10-26T05:51:19.335974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:37.624729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353632+0000","last_change":"2023-10-26T05:51:18.353361+0000","last_active":"2023-10-26T05:51:18.353632+0000","last_peered":"2023-10-26T05:51:18.353632+0000","last_clean":"2023-10-26T05:51:18.353632+0000","last_became_active":"2023-10-26T05:51:18.351796+0000","last_became_peered":"2023-10-26T05:51:18.351796+0000","last_unstale":"2023-10-26T05:51:18.353632+0000","last_undegraded":"2023-10-26T05:51:18.353632+0000","last_fullsized":"2023-10-26T05:51:18.353632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:01:29.970218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.346340+0000","last_change":"2023-10-26T05:51:17.329785+0000","last_active":"2023-10-26T05:51:18.346340+0000","last_peered":"2023-10-26T05:51:18.346340+0000","last_clean":"2023-10-26T05:51:18.346340+0000","last_became_active":"2023-10-26T05:51:17.328680+0000","last_became_peered":"2023-10-26T05:51:17.328680+0000","last_unstale":"2023-10-26T05:51:18.346340+0000","last_undegraded":"2023-10-26T05:51:18.346340+0000","last_fullsized":"2023-10-26T05:51:18.346340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:19.867242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.372067+0000","last_change":"2023-10-26T05:51:18.371994+0000","last_active":"2023-10-26T05:51:18.372067+0000","last_peered":"2023-10-26T05:51:18.372067+0000","last_clean":"2023-10-26T05:51:18.372067+0000","last_became_active":"2023-10-26T05:51:18.371711+0000","last_became_peered":"2023-10-26T05:51:18.371711+0000","last_unstale":"2023-10-26T05:51:18.372067+0000","last_undegraded":"2023-10-26T05:51:18.372067+0000","last_fullsized":"2023-10-26T05:51:18.372067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:23.190182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.443503+0000","last_change":"2023-10-26T05:51:17.322173+0000","last_active":"2023-10-26T05:51:19.443503+0000","last_peered":"2023-10-26T05:51:19.443503+0000","last_clean":"2023-10-26T05:51:19.443503+0000","last_became_active":"2023-10-26T05:51:17.321475+0000","last_became_peered":"2023-10-26T05:51:17.321475+0000","last_unstale":"2023-10-26T05:51:19.443503+0000","last_undegraded":"2023-10-26T05:51:19.443503+0000","last_fullsized":"2023-10-26T05:51:19.443503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:32.813796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.368371+0000","last_change":"2023-10-26T05:51:18.368150+0000","last_active":"2023-10-26T05:51:18.368371+0000","last_peered":"2023-10-26T05:51:18.368371+0000","last_clean":"2023-10-26T05:51:18.368371+0000","last_became_active":"2023-10-26T05:51:18.367535+0000","last_became_peered":"2023-10-26T05:51:18.367535+0000","last_unstale":"2023-10-26T05:51:18.368371+0000","last_undegraded":"2023-10-26T05:51:18.368371+0000","last_fullsized":"2023-10-26T05:51:18.368371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:17:29.737714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331747+0000","last_change":"2023-10-26T05:51:17.314808+0000","last_active":"2023-10-26T05:51:19.331747+0000","last_peered":"2023-10-26T05:51:19.331747+0000","last_clean":"2023-10-26T05:51:19.331747+0000","last_became_active":"2023-10-26T05:51:17.314531+0000","last_became_peered":"2023-10-26T05:51:17.314531+0000","last_unstale":"2023-10-26T05:51:19.331747+0000","last_undegraded":"2023-10-26T05:51:19.331747+0000","last_fullsized":"2023-10-26T05:51:19.331747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:55:37.461256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332522+0000","last_change":"2023-10-26T05:51:18.352442+0000","last_active":"2023-10-26T05:51:19.332522+0000","last_peered":"2023-10-26T05:51:19.332522+0000","last_clean":"2023-10-26T05:51:19.332522+0000","last_became_active":"2023-10-26T05:51:18.350609+0000","last_became_peered":"2023-10-26T05:51:18.350609+0000","last_unstale":"2023-10-26T05:51:19.332522+0000","last_undegraded":"2023-10-26T05:51:19.332522+0000","last_fullsized":"2023-10-26T05:51:19.332522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:33:55.719578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.385794+0000","last_change":"2023-10-26T05:51:17.320906+0000","last_active":"2023-10-26T05:51:18.385794+0000","last_peered":"2023-10-26T05:51:18.385794+0000","last_clean":"2023-10-26T05:51:18.385794+0000","last_became_active":"2023-10-26T05:51:17.320278+0000","last_became_peered":"2023-10-26T05:51:17.320278+0000","last_unstale":"2023-10-26T05:51:18.385794+0000","last_undegraded":"2023-10-26T05:51:18.385794+0000","last_fullsized":"2023-10-26T05:51:18.385794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:47:09.468428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441951+0000","last_change":"2023-10-26T05:51:18.360066+0000","last_active":"2023-10-26T05:51:19.441951+0000","last_peered":"2023-10-26T05:51:19.441951+0000","last_clean":"2023-10-26T05:51:19.441951+0000","last_became_active":"2023-10-26T05:51:18.358597+0000","last_became_peered":"2023-10-26T05:51:18.358597+0000","last_unstale":"2023-10-26T05:51:19.441951+0000","last_undegraded":"2023-10-26T05:51:19.441951+0000","last_fullsized":"2023-10-26T05:51:19.441951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:23.837622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441791+0000","last_change":"2023-10-26T05:51:17.329032+0000","last_active":"2023-10-26T05:51:19.441791+0000","last_peered":"2023-10-26T05:51:19.441791+0000","last_clean":"2023-10-26T05:51:19.441791+0000","last_became_active":"2023-10-26T05:51:17.327671+0000","last_became_peered":"2023-10-26T05:51:17.327671+0000","last_unstale":"2023-10-26T05:51:19.441791+0000","last_undegraded":"2023-10-26T05:51:19.441791+0000","last_fullsized":"2023-10-26T05:51:19.441791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:00:02.997848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336589+0000","last_change":"2023-10-26T05:51:18.364132+0000","last_active":"2023-10-26T05:51:19.336589+0000","last_peered":"2023-10-26T05:51:19.336589+0000","last_clean":"2023-10-26T05:51:19.336589+0000","last_became_active":"2023-10-26T05:51:18.363581+0000","last_became_peered":"2023-10-26T05:51:18.363581+0000","last_unstale":"2023-10-26T05:51:19.336589+0000","last_undegraded":"2023-10-26T05:51:19.336589+0000","last_fullsized":"2023-10-26T05:51:19.336589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:56.261271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.333122+0000","last_change":"2023-10-26T05:51:17.313890+0000","last_active":"2023-10-26T05:51:18.333122+0000","last_peered":"2023-10-26T05:51:18.333122+0000","last_clean":"2023-10-26T05:51:18.333122+0000","last_became_active":"2023-10-26T05:51:17.313404+0000","last_became_peered":"2023-10-26T05:51:17.313404+0000","last_unstale":"2023-10-26T05:51:18.333122+0000","last_undegraded":"2023-10-26T05:51:18.333122+0000","last_fullsized":"2023-10-26T05:51:18.333122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:21.939569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336907+0000","last_change":"2023-10-26T05:51:18.359775+0000","last_active":"2023-10-26T05:51:19.336907+0000","last_peered":"2023-10-26T05:51:19.336907+0000","last_clean":"2023-10-26T05:51:19.336907+0000","last_became_active":"2023-10-26T05:51:18.359173+0000","last_became_peered":"2023-10-26T05:51:18.359173+0000","last_unstale":"2023-10-26T05:51:19.336907+0000","last_undegraded":"2023-10-26T05:51:19.336907+0000","last_fullsized":"2023-10-26T05:51:19.336907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:55:03.859917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331670+0000","last_change":"2023-10-26T05:51:17.322437+0000","last_active":"2023-10-26T05:51:19.331670+0000","last_peered":"2023-10-26T05:51:19.331670+0000","last_clean":"2023-10-26T05:51:19.331670+0000","last_became_active":"2023-10-26T05:51:17.322193+0000","last_became_peered":"2023-10-26T05:51:17.322193+0000","last_unstale":"2023-10-26T05:51:19.331670+0000","last_undegraded":"2023-10-26T05:51:19.331670+0000","last_fullsized":"2023-10-26T05:51:19.331670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:31.583859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357792+0000","last_change":"2023-10-26T05:51:18.356268+0000","last_active":"2023-10-26T05:51:18.357792+0000","last_peered":"2023-10-26T05:51:18.357792+0000","last_clean":"2023-10-26T05:51:18.357792+0000","last_became_active":"2023-10-26T05:51:18.353894+0000","last_became_peered":"2023-10-26T05:51:18.353894+0000","last_unstale":"2023-10-26T05:51:18.357792+0000","last_undegraded":"2023-10-26T05:51:18.357792+0000","last_fullsized":"2023-10-26T05:51:18.357792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:22.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.332445+0000","last_change":"2023-10-26T05:51:17.325260+0000","last_active":"2023-10-26T05:51:18.332445+0000","last_peered":"2023-10-26T05:51:18.332445+0000","last_clean":"2023-10-26T05:51:18.332445+0000","last_became_active":"2023-10-26T05:51:17.324982+0000","last_became_peered":"2023-10-26T05:51:17.324982+0000","last_unstale":"2023-10-26T05:51:18.332445+0000","last_undegraded":"2023-10-26T05:51:18.332445+0000","last_fullsized":"2023-10-26T05:51:18.332445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:01:22.716955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.375685+0000","last_change":"2023-10-26T05:51:18.375484+0000","last_active":"2023-10-26T05:51:18.375685+0000","last_peered":"2023-10-26T05:51:18.375685+0000","last_clean":"2023-10-26T05:51:18.375685+0000","last_became_active":"2023-10-26T05:51:18.370738+0000","last_became_peered":"2023-10-26T05:51:18.370738+0000","last_unstale":"2023-10-26T05:51:18.375685+0000","last_undegraded":"2023-10-26T05:51:18.375685+0000","last_fullsized":"2023-10-26T05:51:18.375685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:24.750718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.330715+0000","last_change":"2023-10-26T05:51:17.318911+0000","last_active":"2023-10-26T05:51:18.330715+0000","last_peered":"2023-10-26T05:51:18.330715+0000","last_clean":"2023-10-26T05:51:18.330715+0000","last_became_active":"2023-10-26T05:51:17.318573+0000","last_became_peered":"2023-10-26T05:51:17.318573+0000","last_unstale":"2023-10-26T05:51:18.330715+0000","last_undegraded":"2023-10-26T05:51:18.330715+0000","last_fullsized":"2023-10-26T05:51:18.330715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.807002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.363676+0000","last_change":"2023-10-26T05:51:18.363499+0000","last_active":"2023-10-26T05:51:18.363676+0000","last_peered":"2023-10-26T05:51:18.363676+0000","last_clean":"2023-10-26T05:51:18.363676+0000","last_became_active":"2023-10-26T05:51:18.359528+0000","last_became_peered":"2023-10-26T05:51:18.359528+0000","last_unstale":"2023-10-26T05:51:18.363676+0000","last_undegraded":"2023-10-26T05:51:18.363676+0000","last_fullsized":"2023-10-26T05:51:18.363676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:24.750718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336082+0000","last_change":"2023-10-26T05:51:17.329311+0000","last_active":"2023-10-26T05:51:19.336082+0000","last_peered":"2023-10-26T05:51:19.336082+0000","last_clean":"2023-10-26T05:51:19.336082+0000","last_became_active":"2023-10-26T05:51:17.328313+0000","last_became_peered":"2023-10-26T05:51:17.328313+0000","last_unstale":"2023-10-26T05:51:19.336082+0000","last_undegraded":"2023-10-26T05:51:19.336082+0000","last_fullsized":"2023-10-26T05:51:19.336082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:10.305515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.371729+0000","last_change":"2023-10-26T05:51:18.371545+0000","last_active":"2023-10-26T05:51:18.371729+0000","last_peered":"2023-10-26T05:51:18.371729+0000","last_clean":"2023-10-26T05:51:18.371729+0000","last_became_active":"2023-10-26T05:51:18.370056+0000","last_became_peered":"2023-10-26T05:51:18.370056+0000","last_unstale":"2023-10-26T05:51:18.371729+0000","last_undegraded":"2023-10-26T05:51:18.371729+0000","last_fullsized":"2023-10-26T05:51:18.371729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:17.418357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.346672+0000","last_change":"2023-10-26T05:51:17.329413+0000","last_active":"2023-10-26T05:51:18.346672+0000","last_peered":"2023-10-26T05:51:18.346672+0000","last_clean":"2023-10-26T05:51:18.346672+0000","last_became_active":"2023-10-26T05:51:17.326291+0000","last_became_peered":"2023-10-26T05:51:17.326291+0000","last_unstale":"2023-10-26T05:51:18.346672+0000","last_undegraded":"2023-10-26T05:51:18.346672+0000","last_fullsized":"2023-10-26T05:51:18.346672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:08:39.974332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.362150+0000","last_change":"2023-10-26T05:51:18.361949+0000","last_active":"2023-10-26T05:51:18.362150+0000","last_peered":"2023-10-26T05:51:18.362150+0000","last_clean":"2023-10-26T05:51:18.362150+0000","last_became_active":"2023-10-26T05:51:18.361432+0000","last_became_peered":"2023-10-26T05:51:18.361432+0000","last_unstale":"2023-10-26T05:51:18.362150+0000","last_undegraded":"2023-10-26T05:51:18.362150+0000","last_fullsized":"2023-10-26T05:51:18.362150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:26.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.387463+0000","last_change":"2023-10-26T05:51:17.336865+0000","last_active":"2023-10-26T05:51:18.387463+0000","last_peered":"2023-10-26T05:51:18.387463+0000","last_clean":"2023-10-26T05:51:18.387463+0000","last_became_active":"2023-10-26T05:51:17.336144+0000","last_became_peered":"2023-10-26T05:51:17.336144+0000","last_unstale":"2023-10-26T05:51:18.387463+0000","last_undegraded":"2023-10-26T05:51:18.387463+0000","last_fullsized":"2023-10-26T05:51:18.387463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:06.330605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353537+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:51:18.353537+0000","last_peered":"2023-10-26T05:51:18.353537+0000","last_clean":"2023-10-26T05:51:18.353537+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:51:18.353537+0000","last_undegraded":"2023-10-26T05:51:18.353537+0000","last_fullsized":"2023-10-26T05:51:18.353537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:57:23.631505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1019,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194292,"statfs":{"total":200000000,"available":198955814,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.86199999999999999},"min":{"1min":0.58799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.623},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.502,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.286,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.218,"5min":1.391,"15min":1.391},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.73499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.3340000000000001,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.81499999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1639999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.91800000000000004,"15min":0.93600000000000005},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1859999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.97599999999999998,"15min":0.97399999999999998},"min":{"1min":0.69199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.242,"5min":1.375,"15min":1.375},"last":1.143}]},{"osd":4,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.63200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2909999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.95299999999999996,"15min":0.93999999999999995},"min":{"1min":0.748,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2190000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0669999999999999,"15min":1.125},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2370000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.046,"15min":1.089},"min":{"1min":0.69899999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.361,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.411}]},{"osd":6,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.077,"15min":1.111},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.413,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.274},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0509999999999999,"15min":1.0489999999999999},"min":{"1min":0.79700000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3280000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.097,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.87,"15min":0.85999999999999999},"min":{"1min":0.63200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.298,"5min":1.3,"15min":1.3},"last":0.91600000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.79400000000000004,"15min":0.76200000000000001},"min":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1160000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86099999999999999,"15min":0.82999999999999996},"min":{"1min":0.63600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2190000000000001,"5min":1.405,"15min":1.405},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.83399999999999996,"15min":0.79300000000000004},"min":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.141,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.82699999999999996},"min":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.089,"5min":1.23,"15min":1.23},"last":1.04}]},{"osd":3,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":0.91400000000000003,"15min":0.877},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.069,"5min":0.98299999999999998,"15min":0.94799999999999995},"min":{"1min":0.81499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.1080000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.95199999999999996,"15min":0.96199999999999997},"min":{"1min":0.57899999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2390000000000001,"5min":1.325,"15min":1.325},"last":1.429},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.238}]},{"osd":5,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0509999999999999,"15min":1.0529999999999999},"min":{"1min":0.78400000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.234,"5min":1.554,"15min":1.554},"last":1.155},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0049999999999999},"min":{"1min":0.68300000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.1799999999999999,"5min":1.27,"15min":1.27},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.127,"15min":1.109},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.121,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.256,"5min":1.464,"15min":1.464},"last":1.2829999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.01},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97099999999999997,"15min":0.95499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.90000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2130000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.85799999999999998,"15min":0.85699999999999998},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.92100000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.89300000000000002,"15min":0.92700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2450000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92300000000000004,"15min":0.93300000000000005},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2709999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92000000000000004,"15min":0.92400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.409,"5min":1.548,"15min":1.548},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98199999999999998,"15min":1.006},"min":{"1min":0.73399999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4139999999999999,"5min":1.587,"15min":1.587},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.96899999999999997,"15min":1.01},"min":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.075,"15min":1.1060000000000001},"min":{"1min":0.67300000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.262,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.97499999999999998,"15min":1.032},"min":{"1min":0.65500000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2290000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.048},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.008,"15min":1.034},"min":{"1min":0.63100000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.624}]},{"osd":7,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.056,"15min":1.077},"min":{"1min":0.66800000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.534,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.149},{"interface":"front","average":{"1min":1.071,"5min":1.1279999999999999,"15min":1.1699999999999999},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546728,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85399999999999998,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.872,"5min":0.86499999999999999,"15min":0.877},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.502,"15min":1.502},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1819999999999999,"5min":1.375,"15min":1.375},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.91600000000000004,"15min":0.88900000000000001},"min":{"1min":0.755,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.282,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.501,"15min":0.501},"max":{"1min":1.1910000000000001,"5min":1.29,"15min":1.29},"last":0.623},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.85999999999999999},"min":{"1min":0.57599999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.226,"5min":1.373,"15min":1.373},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.66500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96499999999999997,"15min":0.93999999999999995},"min":{"1min":0.73199999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3300000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.874},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":0.999},"min":{"1min":0.64200000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2470000000000001,"5min":1.673,"15min":1.673},"last":1.0169999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0069999999999999,"15min":0.998},"min":{"1min":0.69499999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.383,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.999,"5min":1.032,"15min":1.0369999999999999},"min":{"1min":0.72899999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4690000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.034},"min":{"1min":0.83599999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4330000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.119,"15min":1.115},"min":{"1min":0.95199999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.403,"5min":1.728,"15min":1.728},"last":0.94099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546686,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.95099999999999996,"5min":1.155,"15min":1.155},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.073,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64200000000000002,"15min":0.65600000000000003},"min":{"1min":0.54300000000000004,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.038,"5min":1.085,"15min":1.085},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.57999999999999996,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.278,"5min":1.367,"15min":1.367},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.236,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.85899999999999999,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.92900000000000005,"15min":0.92700000000000005},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.27,"5min":1.3,"15min":1.3},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":0.96599999999999997},"min":{"1min":0.71899999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.248,"5min":1.478,"15min":1.478},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.90000000000000002},"min":{"1min":0.64300000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3100000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.96899999999999997,"15min":1.004},"min":{"1min":0.66200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.745},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0589999999999999,"15min":1.03},"min":{"1min":0.71399999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2330000000000001,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.94099999999999995,"15min":0.91500000000000004},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.383,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.038,"15min":1.046},"min":{"1min":0.68899999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.4019999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.88800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":444,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194868,"statfs":{"total":200000000,"available":199545186,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.42399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.317},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.64200000000000002,"15min":0.58199999999999996},"min":{"1min":0.50800000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.012,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.54900000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2090000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.58399999999999996,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0740000000000001,"5min":1.294,"15min":1.294},"last":0.377}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.77700000000000002,"15min":0.72299999999999998},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.113,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.5},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.56799999999999995,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.27800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.86499999999999999,"15min":0.81200000000000006},"min":{"1min":0.78200000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.224,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2310000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94799999999999995,"15min":0.89700000000000002},"min":{"1min":0.77700000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.353,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.92400000000000004,"15min":0.88800000000000001},"min":{"1min":0.77600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4370000000000001,"5min":1.653,"15min":1.653},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.95399999999999996,"15min":0.90400000000000003},"min":{"1min":0.88500000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.526,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.05,"15min":0.95699999999999996},"min":{"1min":0.72899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.492,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.91300000000000003,"15min":0.88700000000000001},"min":{"1min":0.84899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.21,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.159,"15min":1.0129999999999999},"min":{"1min":0.91800000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":15.352,"15min":15.352},"last":0.86699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":444,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194867,"statfs":{"total":200000000,"available":199544570,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.311,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.751}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0209999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.72999999999999998,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0860000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.165,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0700000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.56899999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1379999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96899999999999997,"15min":0.94499999999999995},"min":{"1min":0.75900000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.286,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.73899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.262,"5min":1.492,"15min":1.492},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.208,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.208}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.208,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.72599999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.23,"5min":1.837,"15min":1.837},"last":1.0489999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.01,"15min":1.028},"min":{"1min":0.71299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.248,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.248},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.095,"15min":1.093},"min":{"1min":0.64900000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.1759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1019,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194293,"statfs":{"total":200000000,"available":198956430,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0189999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0329999999999999,"5min":1.173,"15min":1.173},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.177},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.97599999999999998,"5min":1.409,"15min":1.409},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.67500000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.877,"15min":0.84399999999999997},"min":{"1min":0.67900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.88900000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.24,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.95699999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.92100000000000004},"min":{"1min":0.627,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.28,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.22,"5min":1.27,"15min":1.27},"last":1.111}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92000000000000004,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.02,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.764,"15min":1.764},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98799999999999999,"15min":0.99299999999999999},"min":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5780000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.288,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.0589999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:51:21.360 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:51:21.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/2030239534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 msgr2=0x7f1fac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 --2- 172.21.15.71:0/2030239534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fac003dd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00bb10 tx=0x7f1fbc01cd80 comp rx=0 tx=0).stop 2023-10-26T05:51:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc8926640 1 -- 172.21.15.71:0/2030239534 reap_dead start 2023-10-26T05:51:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/2030239534 shutdown_connections 2023-10-26T05:51:21.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/2030239534 >> 172.21.15.71:0/2030239534 conn(0x7f1fc4069770 msgr2=0x7f1fc4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/2030239534 shutdown_connections 2023-10-26T05:51:21.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/2030239534 wait complete. 2023-10-26T05:51:21.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 Processor -- start 2023-10-26T05:51:21.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- start start 2023-10-26T05:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fc4150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc4158830 0x7f1fc413fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc4134a90 0x7f1fc41360b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc4140190 con 0x7f1fac0039b0 2023-10-26T05:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc413aa70 con 0x7f1fc4134a90 2023-10-26T05:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc413ad50 con 0x7f1fc4158830 2023-10-26T05:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fc4150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fc4150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60882/0 (socket says 172.21.15.71:60882) 2023-10-26T05:51:21.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 -- 172.21.15.71:0/4148781356 learned_addr learned my addr 172.21.15.71:0/4148781356 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc37fe640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc4158830 0x7f1fc413fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 -- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc4158830 msgr2=0x7f1fc413fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc4158830 0x7f1fc413fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc8926640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc4134a90 0x7f1fc41360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 -- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc4134a90 msgr2=0x7f1fc41360b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc4134a90 0x7f1fc41360b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 -- 172.21.15.71:0/4148781356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc413d3c0 con 0x7f1fac0039b0 2023-10-26T05:51:21.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc37fe640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc4158830 0x7f1fc413fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:21.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc8926640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc4134a90 0x7f1fc41360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:21.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc3fff640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fc4150280 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00ba60 tx=0x7f1fbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:21.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0be070 con 0x7f1fac0039b0 2023-10-26T05:51:21.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1fbc0a6c50 con 0x7f1fac0039b0 2023-10-26T05:51:21.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc4135ae0 con 0x7f1fac0039b0 2023-10-26T05:51:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc005040 con 0x7f1fac0039b0 2023-10-26T05:51:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.520+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc4134f40 con 0x7f1fac0039b0 2023-10-26T05:51:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.524+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1fbc0051a0 con 0x7f1fac0039b0 2023-10-26T05:51:21.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.524+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc4066b10 con 0x7f1fac0039b0 2023-10-26T05:51:21.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.524+0000 7f1fc0ff9640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1fb00664a0 0x7f1fb0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.524+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fbc11e580 con 0x7f1fac0039b0 2023-10-26T05:51:21.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.524+0000 7f1fc37fe640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1fb00664a0 0x7f1fb0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.524+0000 7f1fc37fe640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1fb00664a0 0x7f1fb0068960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40ad090 tx=0x7f1fb80069c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:21.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.528+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1fbc0eb8e0 con 0x7f1fac0039b0 2023-10-26T05:51:21.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.680+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1fc413abe0 con 0x7f1fb00664a0 2023-10-26T05:51:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.700+0000 7f1fc0ff9640 1 -- 172.21.15.71:0/4148781356 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283246 (secure 0 0 0) 0x7f1fc413abe0 con 0x7f1fb00664a0 2023-10-26T05:51:21.708 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:21.711 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:51:21.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1fb00664a0 msgr2=0x7f1fb0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1fb00664a0 0x7f1fb0068960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1fc40ad090 tx=0x7f1fb80069c0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 msgr2=0x7f1fc4150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fc4150280 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc00ba60 tx=0x7f1fbc005880 comp rx=0 tx=0).stop 2023-10-26T05:51:21.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 shutdown_connections 2023-10-26T05:51:21.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc4134a90 0x7f1fc41360b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc4158830 0x7f1fc413fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1fb00664a0 0x7f1fb0068960 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 --2- 172.21.15.71:0/4148781356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1fac0039b0 0x7f1fc4150280 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 >> 172.21.15.71:0/4148781356 conn(0x7f1fc4069770 msgr2=0x7f1fc4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:21.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 shutdown_connections 2023-10-26T05:51:21.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.704+0000 7f1fca389640 1 -- 172.21.15.71:0/4148781356 wait complete. 2023-10-26T05:51:21.730 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2023-10-26T05:51:21.491992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":93,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4698,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557801,"statfs":{"total":1600000000,"available":1595188828,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.197566"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336101+0000","last_change":"2023-10-26T05:51:18.367188+0000","last_active":"2023-10-26T05:51:19.336101+0000","last_peered":"2023-10-26T05:51:19.336101+0000","last_clean":"2023-10-26T05:51:19.336101+0000","last_became_active":"2023-10-26T05:51:18.365723+0000","last_became_peered":"2023-10-26T05:51:18.365723+0000","last_unstale":"2023-10-26T05:51:19.336101+0000","last_undegraded":"2023-10-26T05:51:19.336101+0000","last_fullsized":"2023-10-26T05:51:19.336101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:25:49.104673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.371423+0000","last_change":"2023-10-26T05:51:18.371277+0000","last_active":"2023-10-26T05:51:18.371423+0000","last_peered":"2023-10-26T05:51:18.371423+0000","last_clean":"2023-10-26T05:51:18.371423+0000","last_became_active":"2023-10-26T05:51:18.370596+0000","last_became_peered":"2023-10-26T05:51:18.370596+0000","last_unstale":"2023-10-26T05:51:18.371423+0000","last_undegraded":"2023-10-26T05:51:18.371423+0000","last_fullsized":"2023-10-26T05:51:18.371423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:27.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357987+0000","last_change":"2023-10-26T05:51:18.357440+0000","last_active":"2023-10-26T05:51:18.357987+0000","last_peered":"2023-10-26T05:51:18.357987+0000","last_clean":"2023-10-26T05:51:18.357987+0000","last_became_active":"2023-10-26T05:51:18.356753+0000","last_became_peered":"2023-10-26T05:51:18.356753+0000","last_unstale":"2023-10-26T05:51:18.357987+0000","last_undegraded":"2023-10-26T05:51:18.357987+0000","last_fullsized":"2023-10-26T05:51:18.357987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:42:56.874532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.364262+0000","last_change":"2023-10-26T05:51:18.364102+0000","last_active":"2023-10-26T05:51:18.364262+0000","last_peered":"2023-10-26T05:51:18.364262+0000","last_clean":"2023-10-26T05:51:18.364262+0000","last_became_active":"2023-10-26T05:51:18.362772+0000","last_became_peered":"2023-10-26T05:51:18.362772+0000","last_unstale":"2023-10-26T05:51:18.364262+0000","last_undegraded":"2023-10-26T05:51:18.364262+0000","last_fullsized":"2023-10-26T05:51:18.364262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:57:58.380157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.355110+0000","last_change":"2023-10-26T05:51:18.354949+0000","last_active":"2023-10-26T05:51:18.355110+0000","last_peered":"2023-10-26T05:51:18.355110+0000","last_clean":"2023-10-26T05:51:18.355110+0000","last_became_active":"2023-10-26T05:51:18.354308+0000","last_became_peered":"2023-10-26T05:51:18.354308+0000","last_unstale":"2023-10-26T05:51:18.355110+0000","last_undegraded":"2023-10-26T05:51:18.355110+0000","last_fullsized":"2023-10-26T05:51:18.355110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:48:27.716287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.376401+0000","last_change":"2023-10-26T05:51:18.376278+0000","last_active":"2023-10-26T05:51:18.376401+0000","last_peered":"2023-10-26T05:51:18.376401+0000","last_clean":"2023-10-26T05:51:18.376401+0000","last_became_active":"2023-10-26T05:51:18.372962+0000","last_became_peered":"2023-10-26T05:51:18.372962+0000","last_unstale":"2023-10-26T05:51:18.376401+0000","last_undegraded":"2023-10-26T05:51:18.376401+0000","last_fullsized":"2023-10-26T05:51:18.376401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:16.515008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441288+0000","last_change":"2023-10-26T05:51:18.360414+0000","last_active":"2023-10-26T05:51:19.441288+0000","last_peered":"2023-10-26T05:51:19.441288+0000","last_clean":"2023-10-26T05:51:19.441288+0000","last_became_active":"2023-10-26T05:51:18.359444+0000","last_became_peered":"2023-10-26T05:51:18.359444+0000","last_unstale":"2023-10-26T05:51:19.441288+0000","last_undegraded":"2023-10-26T05:51:19.441288+0000","last_fullsized":"2023-10-26T05:51:19.441288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:41.440408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.371016+0000","last_change":"2023-10-26T05:51:18.370646+0000","last_active":"2023-10-26T05:51:18.371016+0000","last_peered":"2023-10-26T05:51:18.371016+0000","last_clean":"2023-10-26T05:51:18.371016+0000","last_became_active":"2023-10-26T05:51:18.368578+0000","last_became_peered":"2023-10-26T05:51:18.368578+0000","last_unstale":"2023-10-26T05:51:18.371016+0000","last_undegraded":"2023-10-26T05:51:18.371016+0000","last_fullsized":"2023-10-26T05:51:18.371016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:04.818950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.354889+0000","last_change":"2023-10-26T05:51:18.354639+0000","last_active":"2023-10-26T05:51:18.354889+0000","last_peered":"2023-10-26T05:51:18.354889+0000","last_clean":"2023-10-26T05:51:18.354889+0000","last_became_active":"2023-10-26T05:51:18.354022+0000","last_became_peered":"2023-10-26T05:51:18.354022+0000","last_unstale":"2023-10-26T05:51:18.354889+0000","last_undegraded":"2023-10-26T05:51:18.354889+0000","last_fullsized":"2023-10-26T05:51:18.354889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:21:24.370935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337356+0000","last_change":"2023-10-26T05:51:18.339285+0000","last_active":"2023-10-26T05:51:19.337356+0000","last_peered":"2023-10-26T05:51:19.337356+0000","last_clean":"2023-10-26T05:51:19.337356+0000","last_became_active":"2023-10-26T05:51:18.338736+0000","last_became_peered":"2023-10-26T05:51:18.338736+0000","last_unstale":"2023-10-26T05:51:19.337356+0000","last_undegraded":"2023-10-26T05:51:19.337356+0000","last_fullsized":"2023-10-26T05:51:19.337356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:07:54.940812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336623+0000","last_change":"2023-10-26T05:51:18.355190+0000","last_active":"2023-10-26T05:51:19.336623+0000","last_peered":"2023-10-26T05:51:19.336623+0000","last_clean":"2023-10-26T05:51:19.336623+0000","last_became_active":"2023-10-26T05:51:18.354517+0000","last_became_peered":"2023-10-26T05:51:18.354517+0000","last_unstale":"2023-10-26T05:51:19.336623+0000","last_undegraded":"2023-10-26T05:51:19.336623+0000","last_fullsized":"2023-10-26T05:51:19.336623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:07.797423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.339847+0000","last_change":"2023-10-26T05:51:18.339221+0000","last_active":"2023-10-26T05:51:18.339847+0000","last_peered":"2023-10-26T05:51:18.339847+0000","last_clean":"2023-10-26T05:51:18.339847+0000","last_became_active":"2023-10-26T05:51:18.337707+0000","last_became_peered":"2023-10-26T05:51:18.337707+0000","last_unstale":"2023-10-26T05:51:18.339847+0000","last_undegraded":"2023-10-26T05:51:18.339847+0000","last_fullsized":"2023-10-26T05:51:18.339847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:23.654693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332089+0000","last_change":"2023-10-26T05:51:18.369316+0000","last_active":"2023-10-26T05:51:19.332089+0000","last_peered":"2023-10-26T05:51:19.332089+0000","last_clean":"2023-10-26T05:51:19.332089+0000","last_became_active":"2023-10-26T05:51:18.368284+0000","last_became_peered":"2023-10-26T05:51:18.368284+0000","last_unstale":"2023-10-26T05:51:19.332089+0000","last_undegraded":"2023-10-26T05:51:19.332089+0000","last_fullsized":"2023-10-26T05:51:19.332089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:32.679461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.354843+0000","last_change":"2023-10-26T05:51:18.353408+0000","last_active":"2023-10-26T05:51:18.354843+0000","last_peered":"2023-10-26T05:51:18.354843+0000","last_clean":"2023-10-26T05:51:18.354843+0000","last_became_active":"2023-10-26T05:51:18.352684+0000","last_became_peered":"2023-10-26T05:51:18.352684+0000","last_unstale":"2023-10-26T05:51:18.354843+0000","last_undegraded":"2023-10-26T05:51:18.354843+0000","last_fullsized":"2023-10-26T05:51:18.354843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:37:36.590764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331528+0000","last_change":"2023-10-26T05:51:18.345200+0000","last_active":"2023-10-26T05:51:19.331528+0000","last_peered":"2023-10-26T05:51:19.331528+0000","last_clean":"2023-10-26T05:51:19.331528+0000","last_became_active":"2023-10-26T05:51:18.344552+0000","last_became_peered":"2023-10-26T05:51:18.344552+0000","last_unstale":"2023-10-26T05:51:19.331528+0000","last_undegraded":"2023-10-26T05:51:19.331528+0000","last_fullsized":"2023-10-26T05:51:19.331528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:44:49.548875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336453+0000","last_change":"2023-10-26T05:51:18.375093+0000","last_active":"2023-10-26T05:51:19.336453+0000","last_peered":"2023-10-26T05:51:19.336453+0000","last_clean":"2023-10-26T05:51:19.336453+0000","last_became_active":"2023-10-26T05:51:18.374324+0000","last_became_peered":"2023-10-26T05:51:18.374324+0000","last_unstale":"2023-10-26T05:51:19.336453+0000","last_undegraded":"2023-10-26T05:51:19.336453+0000","last_fullsized":"2023-10-26T05:51:19.336453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:41.440408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357903+0000","last_change":"2023-10-26T05:51:18.357104+0000","last_active":"2023-10-26T05:51:18.357903+0000","last_peered":"2023-10-26T05:51:18.357903+0000","last_clean":"2023-10-26T05:51:18.357903+0000","last_became_active":"2023-10-26T05:51:18.355589+0000","last_became_peered":"2023-10-26T05:51:18.355589+0000","last_unstale":"2023-10-26T05:51:18.357903+0000","last_undegraded":"2023-10-26T05:51:18.357903+0000","last_fullsized":"2023-10-26T05:51:18.357903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:55:27.272312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.369398+0000","last_change":"2023-10-26T05:51:18.369226+0000","last_active":"2023-10-26T05:51:18.369398+0000","last_peered":"2023-10-26T05:51:18.369398+0000","last_clean":"2023-10-26T05:51:18.369398+0000","last_became_active":"2023-10-26T05:51:18.368593+0000","last_became_peered":"2023-10-26T05:51:18.368593+0000","last_unstale":"2023-10-26T05:51:18.369398+0000","last_undegraded":"2023-10-26T05:51:18.369398+0000","last_fullsized":"2023-10-26T05:51:18.369398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:05.450043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.368030+0000","last_change":"2023-10-26T05:51:18.367654+0000","last_active":"2023-10-26T05:51:18.368030+0000","last_peered":"2023-10-26T05:51:18.368030+0000","last_clean":"2023-10-26T05:51:18.368030+0000","last_became_active":"2023-10-26T05:51:18.365301+0000","last_became_peered":"2023-10-26T05:51:18.365301+0000","last_unstale":"2023-10-26T05:51:18.368030+0000","last_undegraded":"2023-10-26T05:51:18.368030+0000","last_fullsized":"2023-10-26T05:51:18.368030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:57:58.380157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331611+0000","last_change":"2023-10-26T05:51:18.357483+0000","last_active":"2023-10-26T05:51:19.331611+0000","last_peered":"2023-10-26T05:51:19.331611+0000","last_clean":"2023-10-26T05:51:19.331611+0000","last_became_active":"2023-10-26T05:51:18.356877+0000","last_became_peered":"2023-10-26T05:51:18.356877+0000","last_unstale":"2023-10-26T05:51:19.331611+0000","last_undegraded":"2023-10-26T05:51:19.331611+0000","last_fullsized":"2023-10-26T05:51:19.331611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:00:27.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336710+0000","last_change":"2023-10-26T05:51:18.352441+0000","last_active":"2023-10-26T05:51:19.336710+0000","last_peered":"2023-10-26T05:51:19.336710+0000","last_clean":"2023-10-26T05:51:19.336710+0000","last_became_active":"2023-10-26T05:51:18.351797+0000","last_became_peered":"2023-10-26T05:51:18.351797+0000","last_unstale":"2023-10-26T05:51:19.336710+0000","last_undegraded":"2023-10-26T05:51:19.336710+0000","last_fullsized":"2023-10-26T05:51:19.336710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:47.574397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335875+0000","last_change":"2023-10-26T05:51:18.341218+0000","last_active":"2023-10-26T05:51:19.335875+0000","last_peered":"2023-10-26T05:51:19.335875+0000","last_clean":"2023-10-26T05:51:19.335875+0000","last_became_active":"2023-10-26T05:51:18.340721+0000","last_became_peered":"2023-10-26T05:51:18.340721+0000","last_unstale":"2023-10-26T05:51:19.335875+0000","last_undegraded":"2023-10-26T05:51:19.335875+0000","last_fullsized":"2023-10-26T05:51:19.335875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:23.878227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337256+0000","last_change":"2023-10-26T05:51:18.362537+0000","last_active":"2023-10-26T05:51:19.337256+0000","last_peered":"2023-10-26T05:51:19.337256+0000","last_clean":"2023-10-26T05:51:19.337256+0000","last_became_active":"2023-10-26T05:51:18.361869+0000","last_became_peered":"2023-10-26T05:51:18.361869+0000","last_unstale":"2023-10-26T05:51:19.337256+0000","last_undegraded":"2023-10-26T05:51:19.337256+0000","last_fullsized":"2023-10-26T05:51:19.337256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:59:49.311401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335228+0000","last_change":"2023-10-26T05:51:18.366512+0000","last_active":"2023-10-26T05:51:19.335228+0000","last_peered":"2023-10-26T05:51:19.335228+0000","last_clean":"2023-10-26T05:51:19.335228+0000","last_became_active":"2023-10-26T05:51:18.363896+0000","last_became_peered":"2023-10-26T05:51:18.363896+0000","last_unstale":"2023-10-26T05:51:19.335228+0000","last_undegraded":"2023-10-26T05:51:19.335228+0000","last_fullsized":"2023-10-26T05:51:19.335228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:05:57.308958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337631+0000","last_change":"2023-10-26T05:51:18.360206+0000","last_active":"2023-10-26T05:51:19.337631+0000","last_peered":"2023-10-26T05:51:19.337631+0000","last_clean":"2023-10-26T05:51:19.337631+0000","last_became_active":"2023-10-26T05:51:18.359418+0000","last_became_peered":"2023-10-26T05:51:18.359418+0000","last_unstale":"2023-10-26T05:51:19.337631+0000","last_undegraded":"2023-10-26T05:51:19.337631+0000","last_fullsized":"2023-10-26T05:51:19.337631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:10.510054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441433+0000","last_change":"2023-10-26T05:51:18.374920+0000","last_active":"2023-10-26T05:51:19.441433+0000","last_peered":"2023-10-26T05:51:19.441433+0000","last_clean":"2023-10-26T05:51:19.441433+0000","last_became_active":"2023-10-26T05:51:18.374377+0000","last_became_peered":"2023-10-26T05:51:18.374377+0000","last_unstale":"2023-10-26T05:51:19.441433+0000","last_undegraded":"2023-10-26T05:51:19.441433+0000","last_fullsized":"2023-10-26T05:51:19.441433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:07:54.940812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353590+0000","last_change":"2023-10-26T05:51:18.353179+0000","last_active":"2023-10-26T05:51:18.353590+0000","last_peered":"2023-10-26T05:51:18.353590+0000","last_clean":"2023-10-26T05:51:18.353590+0000","last_became_active":"2023-10-26T05:51:18.352601+0000","last_became_peered":"2023-10-26T05:51:18.352601+0000","last_unstale":"2023-10-26T05:51:18.353590+0000","last_undegraded":"2023-10-26T05:51:18.353590+0000","last_fullsized":"2023-10-26T05:51:18.353590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:30:57.049868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335605+0000","last_change":"2023-10-26T05:51:18.337400+0000","last_active":"2023-10-26T05:51:19.335605+0000","last_peered":"2023-10-26T05:51:19.335605+0000","last_clean":"2023-10-26T05:51:19.335605+0000","last_became_active":"2023-10-26T05:51:18.336799+0000","last_became_peered":"2023-10-26T05:51:18.336799+0000","last_unstale":"2023-10-26T05:51:19.335605+0000","last_undegraded":"2023-10-26T05:51:19.335605+0000","last_fullsized":"2023-10-26T05:51:19.335605+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:55.156799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353509+0000","last_change":"2023-10-26T05:51:18.352948+0000","last_active":"2023-10-26T05:51:18.353509+0000","last_peered":"2023-10-26T05:51:18.353509+0000","last_clean":"2023-10-26T05:51:18.353509+0000","last_became_active":"2023-10-26T05:51:18.352356+0000","last_became_peered":"2023-10-26T05:51:18.352356+0000","last_unstale":"2023-10-26T05:51:18.353509+0000","last_undegraded":"2023-10-26T05:51:18.353509+0000","last_fullsized":"2023-10-26T05:51:18.353509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:04:18.025028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441680+0000","last_change":"2023-10-26T05:51:18.364361+0000","last_active":"2023-10-26T05:51:19.441680+0000","last_peered":"2023-10-26T05:51:19.441680+0000","last_clean":"2023-10-26T05:51:19.441680+0000","last_became_active":"2023-10-26T05:51:18.363759+0000","last_became_peered":"2023-10-26T05:51:18.363759+0000","last_unstale":"2023-10-26T05:51:19.441680+0000","last_undegraded":"2023-10-26T05:51:19.441680+0000","last_fullsized":"2023-10-26T05:51:19.441680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:48:48.160737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441781+0000","last_change":"2023-10-26T05:51:18.363247+0000","last_active":"2023-10-26T05:51:19.441781+0000","last_peered":"2023-10-26T05:51:19.441781+0000","last_clean":"2023-10-26T05:51:19.441781+0000","last_became_active":"2023-10-26T05:51:18.362641+0000","last_became_peered":"2023-10-26T05:51:18.362641+0000","last_unstale":"2023-10-26T05:51:19.441781+0000","last_undegraded":"2023-10-26T05:51:19.441781+0000","last_fullsized":"2023-10-26T05:51:19.441781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:24:38.265270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.368001+0000","last_change":"2023-10-26T05:51:18.366448+0000","last_active":"2023-10-26T05:51:18.368001+0000","last_peered":"2023-10-26T05:51:18.368001+0000","last_clean":"2023-10-26T05:51:18.368001+0000","last_became_active":"2023-10-26T05:51:18.365782+0000","last_became_peered":"2023-10-26T05:51:18.365782+0000","last_unstale":"2023-10-26T05:51:18.368001+0000","last_undegraded":"2023-10-26T05:51:18.368001+0000","last_fullsized":"2023-10-26T05:51:18.368001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:05.450043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336202+0000","last_change":"2023-10-26T05:51:18.366920+0000","last_active":"2023-10-26T05:51:19.336202+0000","last_peered":"2023-10-26T05:51:19.336202+0000","last_clean":"2023-10-26T05:51:19.336202+0000","last_became_active":"2023-10-26T05:51:18.364803+0000","last_became_peered":"2023-10-26T05:51:18.364803+0000","last_unstale":"2023-10-26T05:51:19.336202+0000","last_undegraded":"2023-10-26T05:51:19.336202+0000","last_fullsized":"2023-10-26T05:51:19.336202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:02.233918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"85'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337501+0000","last_change":"2023-10-26T05:51:17.326442+0000","last_active":"2023-10-26T05:51:19.337501+0000","last_peered":"2023-10-26T05:51:19.337501+0000","last_clean":"2023-10-26T05:51:19.337501+0000","last_became_active":"2023-10-26T05:51:17.325660+0000","last_became_peered":"2023-10-26T05:51:17.325660+0000","last_unstale":"2023-10-26T05:51:19.337501+0000","last_undegraded":"2023-10-26T05:51:19.337501+0000","last_fullsized":"2023-10-26T05:51:19.337501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:40.428460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.361971+0000","last_change":"2023-10-26T05:51:18.361801+0000","last_active":"2023-10-26T05:51:18.361971+0000","last_peered":"2023-10-26T05:51:18.361971+0000","last_clean":"2023-10-26T05:51:18.361971+0000","last_became_active":"2023-10-26T05:51:18.361023+0000","last_became_peered":"2023-10-26T05:51:18.361023+0000","last_unstale":"2023-10-26T05:51:18.361971+0000","last_undegraded":"2023-10-26T05:51:18.361971+0000","last_fullsized":"2023-10-26T05:51:18.361971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:24:52.015534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"85'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332155+0000","last_change":"2023-10-26T05:51:17.315511+0000","last_active":"2023-10-26T05:51:19.332155+0000","last_peered":"2023-10-26T05:51:19.332155+0000","last_clean":"2023-10-26T05:51:19.332155+0000","last_became_active":"2023-10-26T05:51:17.315173+0000","last_became_peered":"2023-10-26T05:51:17.315173+0000","last_unstale":"2023-10-26T05:51:19.332155+0000","last_undegraded":"2023-10-26T05:51:19.332155+0000","last_fullsized":"2023-10-26T05:51:19.332155+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:55.249322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.376079+0000","last_change":"2023-10-26T05:51:18.375881+0000","last_active":"2023-10-26T05:51:18.376079+0000","last_peered":"2023-10-26T05:51:18.376079+0000","last_clean":"2023-10-26T05:51:18.376079+0000","last_became_active":"2023-10-26T05:51:18.371860+0000","last_became_peered":"2023-10-26T05:51:18.371860+0000","last_unstale":"2023-10-26T05:51:18.376079+0000","last_undegraded":"2023-10-26T05:51:18.376079+0000","last_fullsized":"2023-10-26T05:51:18.376079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:19:45.471869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"85'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.338072+0000","last_change":"2023-10-26T05:51:17.327271+0000","last_active":"2023-10-26T05:51:19.338072+0000","last_peered":"2023-10-26T05:51:19.338072+0000","last_clean":"2023-10-26T05:51:19.338072+0000","last_became_active":"2023-10-26T05:51:17.325095+0000","last_became_peered":"2023-10-26T05:51:17.325095+0000","last_unstale":"2023-10-26T05:51:19.338072+0000","last_undegraded":"2023-10-26T05:51:19.338072+0000","last_fullsized":"2023-10-26T05:51:19.338072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:39:21.244767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336226+0000","last_change":"2023-10-26T05:51:18.355735+0000","last_active":"2023-10-26T05:51:19.336226+0000","last_peered":"2023-10-26T05:51:19.336226+0000","last_clean":"2023-10-26T05:51:19.336226+0000","last_became_active":"2023-10-26T05:51:18.355193+0000","last_became_peered":"2023-10-26T05:51:18.355193+0000","last_unstale":"2023-10-26T05:51:19.336226+0000","last_undegraded":"2023-10-26T05:51:19.336226+0000","last_fullsized":"2023-10-26T05:51:19.336226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:55:38.473204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.383405+0000","last_change":"2023-10-26T05:51:17.337841+0000","last_active":"2023-10-26T05:51:18.383405+0000","last_peered":"2023-10-26T05:51:18.383405+0000","last_clean":"2023-10-26T05:51:18.383405+0000","last_became_active":"2023-10-26T05:51:17.337304+0000","last_became_peered":"2023-10-26T05:51:17.337304+0000","last_unstale":"2023-10-26T05:51:18.383405+0000","last_undegraded":"2023-10-26T05:51:18.383405+0000","last_fullsized":"2023-10-26T05:51:18.383405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:41:02.561751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441375+0000","last_change":"2023-10-26T05:51:18.365437+0000","last_active":"2023-10-26T05:51:19.441375+0000","last_peered":"2023-10-26T05:51:19.441375+0000","last_clean":"2023-10-26T05:51:19.441375+0000","last_became_active":"2023-10-26T05:51:18.363033+0000","last_became_peered":"2023-10-26T05:51:18.363033+0000","last_unstale":"2023-10-26T05:51:19.441375+0000","last_undegraded":"2023-10-26T05:51:19.441375+0000","last_fullsized":"2023-10-26T05:51:19.441375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:47.574397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.378110+0000","last_change":"2023-10-26T05:51:17.319266+0000","last_active":"2023-10-26T05:51:18.378110+0000","last_peered":"2023-10-26T05:51:18.378110+0000","last_clean":"2023-10-26T05:51:18.378110+0000","last_became_active":"2023-10-26T05:51:17.318903+0000","last_became_peered":"2023-10-26T05:51:17.318903+0000","last_unstale":"2023-10-26T05:51:18.378110+0000","last_undegraded":"2023-10-26T05:51:18.378110+0000","last_fullsized":"2023-10-26T05:51:18.378110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:53:07.206116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336223+0000","last_change":"2023-10-26T05:51:18.352565+0000","last_active":"2023-10-26T05:51:19.336223+0000","last_peered":"2023-10-26T05:51:19.336223+0000","last_clean":"2023-10-26T05:51:19.336223+0000","last_became_active":"2023-10-26T05:51:18.351931+0000","last_became_peered":"2023-10-26T05:51:18.351931+0000","last_unstale":"2023-10-26T05:51:19.336223+0000","last_undegraded":"2023-10-26T05:51:19.336223+0000","last_fullsized":"2023-10-26T05:51:19.336223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:00:04.009796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.387711+0000","last_change":"2023-10-26T05:51:17.330054+0000","last_active":"2023-10-26T05:51:18.387711+0000","last_peered":"2023-10-26T05:51:18.387711+0000","last_clean":"2023-10-26T05:51:18.387711+0000","last_became_active":"2023-10-26T05:51:17.329449+0000","last_became_peered":"2023-10-26T05:51:17.329449+0000","last_unstale":"2023-10-26T05:51:18.387711+0000","last_undegraded":"2023-10-26T05:51:18.387711+0000","last_fullsized":"2023-10-26T05:51:18.387711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:55:01.061094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.337887+0000","last_change":"2023-10-26T05:51:18.337654+0000","last_active":"2023-10-26T05:51:18.337887+0000","last_peered":"2023-10-26T05:51:18.337887+0000","last_clean":"2023-10-26T05:51:18.337887+0000","last_became_active":"2023-10-26T05:51:18.337020+0000","last_became_peered":"2023-10-26T05:51:18.337020+0000","last_unstale":"2023-10-26T05:51:18.337887+0000","last_undegraded":"2023-10-26T05:51:18.337887+0000","last_fullsized":"2023-10-26T05:51:18.337887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:38.011241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.331017+0000","last_change":"2023-10-26T05:51:17.318531+0000","last_active":"2023-10-26T05:51:18.331017+0000","last_peered":"2023-10-26T05:51:18.331017+0000","last_clean":"2023-10-26T05:51:18.331017+0000","last_became_active":"2023-10-26T05:51:17.318144+0000","last_became_peered":"2023-10-26T05:51:17.318144+0000","last_unstale":"2023-10-26T05:51:18.331017+0000","last_undegraded":"2023-10-26T05:51:18.331017+0000","last_fullsized":"2023-10-26T05:51:18.331017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:43.449048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331962+0000","last_change":"2023-10-26T05:51:18.359072+0000","last_active":"2023-10-26T05:51:19.331962+0000","last_peered":"2023-10-26T05:51:19.331962+0000","last_clean":"2023-10-26T05:51:19.331962+0000","last_became_active":"2023-10-26T05:51:18.358411+0000","last_became_peered":"2023-10-26T05:51:18.358411+0000","last_unstale":"2023-10-26T05:51:19.331962+0000","last_undegraded":"2023-10-26T05:51:19.331962+0000","last_fullsized":"2023-10-26T05:51:19.331962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:49.131461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.333643+0000","last_change":"2023-10-26T05:51:17.317027+0000","last_active":"2023-10-26T05:51:18.333643+0000","last_peered":"2023-10-26T05:51:18.333643+0000","last_clean":"2023-10-26T05:51:18.333643+0000","last_became_active":"2023-10-26T05:51:17.316340+0000","last_became_peered":"2023-10-26T05:51:17.316340+0000","last_unstale":"2023-10-26T05:51:18.333643+0000","last_undegraded":"2023-10-26T05:51:18.333643+0000","last_fullsized":"2023-10-26T05:51:18.333643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:43:33.890168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441634+0000","last_change":"2023-10-26T05:51:18.355463+0000","last_active":"2023-10-26T05:51:19.441634+0000","last_peered":"2023-10-26T05:51:19.441634+0000","last_clean":"2023-10-26T05:51:19.441634+0000","last_became_active":"2023-10-26T05:51:18.354880+0000","last_became_peered":"2023-10-26T05:51:18.354880+0000","last_unstale":"2023-10-26T05:51:19.441634+0000","last_undegraded":"2023-10-26T05:51:19.441634+0000","last_fullsized":"2023-10-26T05:51:19.441634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:10.510054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336817+0000","last_change":"2023-10-26T05:51:17.319515+0000","last_active":"2023-10-26T05:51:19.336817+0000","last_peered":"2023-10-26T05:51:19.336817+0000","last_clean":"2023-10-26T05:51:19.336817+0000","last_became_active":"2023-10-26T05:51:17.318899+0000","last_became_peered":"2023-10-26T05:51:17.318899+0000","last_unstale":"2023-10-26T05:51:19.336817+0000","last_undegraded":"2023-10-26T05:51:19.336817+0000","last_fullsized":"2023-10-26T05:51:19.336817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:25:48.030863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336927+0000","last_change":"2023-10-26T05:51:18.373896+0000","last_active":"2023-10-26T05:51:19.336927+0000","last_peered":"2023-10-26T05:51:19.336927+0000","last_clean":"2023-10-26T05:51:19.336927+0000","last_became_active":"2023-10-26T05:51:18.373282+0000","last_became_peered":"2023-10-26T05:51:18.373282+0000","last_unstale":"2023-10-26T05:51:19.336927+0000","last_undegraded":"2023-10-26T05:51:19.336927+0000","last_fullsized":"2023-10-26T05:51:19.336927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:00.843086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.345023+0000","last_change":"2023-10-26T05:51:17.324960+0000","last_active":"2023-10-26T05:51:18.345023+0000","last_peered":"2023-10-26T05:51:18.345023+0000","last_clean":"2023-10-26T05:51:18.345023+0000","last_became_active":"2023-10-26T05:51:17.324347+0000","last_became_peered":"2023-10-26T05:51:17.324347+0000","last_unstale":"2023-10-26T05:51:18.345023+0000","last_undegraded":"2023-10-26T05:51:18.345023+0000","last_fullsized":"2023-10-26T05:51:18.345023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:43.449048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.370672+0000","last_change":"2023-10-26T05:51:18.370529+0000","last_active":"2023-10-26T05:51:18.370672+0000","last_peered":"2023-10-26T05:51:18.370672+0000","last_clean":"2023-10-26T05:51:18.370672+0000","last_became_active":"2023-10-26T05:51:18.368687+0000","last_became_peered":"2023-10-26T05:51:18.368687+0000","last_unstale":"2023-10-26T05:51:18.370672+0000","last_undegraded":"2023-10-26T05:51:18.370672+0000","last_fullsized":"2023-10-26T05:51:18.370672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:23.654693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"85'3","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.380378+0000","last_change":"2023-10-26T05:51:17.328324+0000","last_active":"2023-10-26T05:51:18.380378+0000","last_peered":"2023-10-26T05:51:18.380378+0000","last_clean":"2023-10-26T05:51:18.380378+0000","last_became_active":"2023-10-26T05:51:17.328052+0000","last_became_peered":"2023-10-26T05:51:17.328052+0000","last_unstale":"2023-10-26T05:51:18.380378+0000","last_undegraded":"2023-10-26T05:51:18.380378+0000","last_fullsized":"2023-10-26T05:51:18.380378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:24.164396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.337178+0000","last_change":"2023-10-26T05:51:18.337053+0000","last_active":"2023-10-26T05:51:18.337178+0000","last_peered":"2023-10-26T05:51:18.337178+0000","last_clean":"2023-10-26T05:51:18.337178+0000","last_became_active":"2023-10-26T05:51:18.336423+0000","last_became_peered":"2023-10-26T05:51:18.336423+0000","last_unstale":"2023-10-26T05:51:18.337178+0000","last_undegraded":"2023-10-26T05:51:18.337178+0000","last_fullsized":"2023-10-26T05:51:18.337178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:21:33.906574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"85'6","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.381999+0000","last_change":"2023-10-26T05:51:17.315688+0000","last_active":"2023-10-26T05:51:18.381999+0000","last_peered":"2023-10-26T05:51:18.381999+0000","last_clean":"2023-10-26T05:51:18.381999+0000","last_became_active":"2023-10-26T05:51:17.315127+0000","last_became_peered":"2023-10-26T05:51:17.315127+0000","last_unstale":"2023-10-26T05:51:18.381999+0000","last_undegraded":"2023-10-26T05:51:18.381999+0000","last_fullsized":"2023-10-26T05:51:18.381999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:19:44.459921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337231+0000","last_change":"2023-10-26T05:51:18.367616+0000","last_active":"2023-10-26T05:51:19.337231+0000","last_peered":"2023-10-26T05:51:19.337231+0000","last_clean":"2023-10-26T05:51:19.337231+0000","last_became_active":"2023-10-26T05:51:18.366564+0000","last_became_peered":"2023-10-26T05:51:18.366564+0000","last_unstale":"2023-10-26T05:51:19.337231+0000","last_undegraded":"2023-10-26T05:51:19.337231+0000","last_fullsized":"2023-10-26T05:51:19.337231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:56:44.582270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.380106+0000","last_change":"2023-10-26T05:51:17.317567+0000","last_active":"2023-10-26T05:51:18.380106+0000","last_peered":"2023-10-26T05:51:18.380106+0000","last_clean":"2023-10-26T05:51:18.380106+0000","last_became_active":"2023-10-26T05:51:17.317197+0000","last_became_peered":"2023-10-26T05:51:17.317197+0000","last_unstale":"2023-10-26T05:51:18.380106+0000","last_undegraded":"2023-10-26T05:51:18.380106+0000","last_fullsized":"2023-10-26T05:51:18.380106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:16.406408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.443278+0000","last_change":"2023-10-26T05:51:18.357275+0000","last_active":"2023-10-26T05:51:19.443278+0000","last_peered":"2023-10-26T05:51:19.443278+0000","last_clean":"2023-10-26T05:51:19.443278+0000","last_became_active":"2023-10-26T05:51:18.356457+0000","last_became_peered":"2023-10-26T05:51:18.356457+0000","last_unstale":"2023-10-26T05:51:19.443278+0000","last_undegraded":"2023-10-26T05:51:19.443278+0000","last_fullsized":"2023-10-26T05:51:19.443278+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:16:45.591903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.346977+0000","last_change":"2023-10-26T05:51:17.327090+0000","last_active":"2023-10-26T05:51:18.346977+0000","last_peered":"2023-10-26T05:51:18.346977+0000","last_clean":"2023-10-26T05:51:18.346977+0000","last_became_active":"2023-10-26T05:51:17.326080+0000","last_became_peered":"2023-10-26T05:51:17.326080+0000","last_unstale":"2023-10-26T05:51:18.346977+0000","last_undegraded":"2023-10-26T05:51:18.346977+0000","last_fullsized":"2023-10-26T05:51:18.346977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:07.209178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332372+0000","last_change":"2023-10-26T05:51:18.369459+0000","last_active":"2023-10-26T05:51:19.332372+0000","last_peered":"2023-10-26T05:51:19.332372+0000","last_clean":"2023-10-26T05:51:19.332372+0000","last_became_active":"2023-10-26T05:51:18.368982+0000","last_became_peered":"2023-10-26T05:51:18.368982+0000","last_unstale":"2023-10-26T05:51:19.332372+0000","last_undegraded":"2023-10-26T05:51:19.332372+0000","last_fullsized":"2023-10-26T05:51:19.332372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:05.819900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.442289+0000","last_change":"2023-10-26T05:51:17.319285+0000","last_active":"2023-10-26T05:51:19.442289+0000","last_peered":"2023-10-26T05:51:19.442289+0000","last_clean":"2023-10-26T05:51:19.442289+0000","last_became_active":"2023-10-26T05:51:17.317741+0000","last_became_peered":"2023-10-26T05:51:17.317741+0000","last_unstale":"2023-10-26T05:51:19.442289+0000","last_undegraded":"2023-10-26T05:51:19.442289+0000","last_fullsized":"2023-10-26T05:51:19.442289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:25:48.030863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357859+0000","last_change":"2023-10-26T05:51:18.356747+0000","last_active":"2023-10-26T05:51:18.357859+0000","last_peered":"2023-10-26T05:51:18.357859+0000","last_clean":"2023-10-26T05:51:18.357859+0000","last_became_active":"2023-10-26T05:51:18.354766+0000","last_became_peered":"2023-10-26T05:51:18.354766+0000","last_unstale":"2023-10-26T05:51:18.357859+0000","last_undegraded":"2023-10-26T05:51:18.357859+0000","last_fullsized":"2023-10-26T05:51:18.357859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:00:23.324742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"85'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335883+0000","last_change":"2023-10-26T05:51:17.328932+0000","last_active":"2023-10-26T05:51:19.335883+0000","last_peered":"2023-10-26T05:51:19.335883+0000","last_clean":"2023-10-26T05:51:19.335883+0000","last_became_active":"2023-10-26T05:51:17.327451+0000","last_became_peered":"2023-10-26T05:51:17.327451+0000","last_unstale":"2023-10-26T05:51:19.335883+0000","last_undegraded":"2023-10-26T05:51:19.335883+0000","last_fullsized":"2023-10-26T05:51:19.335883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:57.442408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.337838+0000","last_change":"2023-10-26T05:51:18.338377+0000","last_active":"2023-10-26T05:51:19.337838+0000","last_peered":"2023-10-26T05:51:19.337838+0000","last_clean":"2023-10-26T05:51:19.337838+0000","last_became_active":"2023-10-26T05:51:18.337873+0000","last_became_peered":"2023-10-26T05:51:18.337873+0000","last_unstale":"2023-10-26T05:51:19.337838+0000","last_undegraded":"2023-10-26T05:51:19.337838+0000","last_fullsized":"2023-10-26T05:51:19.337838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:48:48.160737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.384867+0000","last_change":"2023-10-26T05:51:17.326903+0000","last_active":"2023-10-26T05:51:18.384867+0000","last_peered":"2023-10-26T05:51:18.384867+0000","last_clean":"2023-10-26T05:51:18.384867+0000","last_became_active":"2023-10-26T05:51:17.322566+0000","last_became_peered":"2023-10-26T05:51:17.322566+0000","last_unstale":"2023-10-26T05:51:18.384867+0000","last_undegraded":"2023-10-26T05:51:18.384867+0000","last_fullsized":"2023-10-26T05:51:18.384867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:23:54.442123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336590+0000","last_change":"2023-10-26T05:51:18.367218+0000","last_active":"2023-10-26T05:51:19.336590+0000","last_peered":"2023-10-26T05:51:19.336590+0000","last_clean":"2023-10-26T05:51:19.336590+0000","last_became_active":"2023-10-26T05:51:18.365867+0000","last_became_peered":"2023-10-26T05:51:18.365867+0000","last_unstale":"2023-10-26T05:51:19.336590+0000","last_undegraded":"2023-10-26T05:51:19.336590+0000","last_fullsized":"2023-10-26T05:51:19.336590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:07:05.937272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"85'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.338196+0000","last_change":"2023-10-26T05:51:17.327760+0000","last_active":"2023-10-26T05:51:19.338196+0000","last_peered":"2023-10-26T05:51:19.338196+0000","last_clean":"2023-10-26T05:51:19.338196+0000","last_became_active":"2023-10-26T05:51:17.326336+0000","last_became_peered":"2023-10-26T05:51:17.326336+0000","last_unstale":"2023-10-26T05:51:19.338196+0000","last_undegraded":"2023-10-26T05:51:19.338196+0000","last_fullsized":"2023-10-26T05:51:19.338196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:51:54.919933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.339891+0000","last_change":"2023-10-26T05:51:18.339675+0000","last_active":"2023-10-26T05:51:18.339891+0000","last_peered":"2023-10-26T05:51:18.339891+0000","last_clean":"2023-10-26T05:51:18.339891+0000","last_became_active":"2023-10-26T05:51:18.338553+0000","last_became_peered":"2023-10-26T05:51:18.338553+0000","last_unstale":"2023-10-26T05:51:18.339891+0000","last_undegraded":"2023-10-26T05:51:18.339891+0000","last_fullsized":"2023-10-26T05:51:18.339891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:32:27.844533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"85'1","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.390289+0000","last_change":"2023-10-26T05:51:17.326689+0000","last_active":"2023-10-26T05:51:18.390289+0000","last_peered":"2023-10-26T05:51:18.390289+0000","last_clean":"2023-10-26T05:51:18.390289+0000","last_became_active":"2023-10-26T05:51:17.325254+0000","last_became_peered":"2023-10-26T05:51:17.325254+0000","last_unstale":"2023-10-26T05:51:18.390289+0000","last_undegraded":"2023-10-26T05:51:18.390289+0000","last_fullsized":"2023-10-26T05:51:18.390289+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:45.347809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.337125+0000","last_change":"2023-10-26T05:51:18.336932+0000","last_active":"2023-10-26T05:51:18.337125+0000","last_peered":"2023-10-26T05:51:18.337125+0000","last_clean":"2023-10-26T05:51:18.337125+0000","last_became_active":"2023-10-26T05:51:18.336291+0000","last_became_peered":"2023-10-26T05:51:18.336291+0000","last_unstale":"2023-10-26T05:51:18.337125+0000","last_undegraded":"2023-10-26T05:51:18.337125+0000","last_fullsized":"2023-10-26T05:51:18.337125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:15:12.261188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.335974+0000","last_change":"2023-10-26T05:51:17.321687+0000","last_active":"2023-10-26T05:51:19.335974+0000","last_peered":"2023-10-26T05:51:19.335974+0000","last_clean":"2023-10-26T05:51:19.335974+0000","last_became_active":"2023-10-26T05:51:17.321081+0000","last_became_peered":"2023-10-26T05:51:17.321081+0000","last_unstale":"2023-10-26T05:51:19.335974+0000","last_undegraded":"2023-10-26T05:51:19.335974+0000","last_fullsized":"2023-10-26T05:51:19.335974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:37.624729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353632+0000","last_change":"2023-10-26T05:51:18.353361+0000","last_active":"2023-10-26T05:51:18.353632+0000","last_peered":"2023-10-26T05:51:18.353632+0000","last_clean":"2023-10-26T05:51:18.353632+0000","last_became_active":"2023-10-26T05:51:18.351796+0000","last_became_peered":"2023-10-26T05:51:18.351796+0000","last_unstale":"2023-10-26T05:51:18.353632+0000","last_undegraded":"2023-10-26T05:51:18.353632+0000","last_fullsized":"2023-10-26T05:51:18.353632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:01:29.970218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.346340+0000","last_change":"2023-10-26T05:51:17.329785+0000","last_active":"2023-10-26T05:51:18.346340+0000","last_peered":"2023-10-26T05:51:18.346340+0000","last_clean":"2023-10-26T05:51:18.346340+0000","last_became_active":"2023-10-26T05:51:17.328680+0000","last_became_peered":"2023-10-26T05:51:17.328680+0000","last_unstale":"2023-10-26T05:51:18.346340+0000","last_undegraded":"2023-10-26T05:51:18.346340+0000","last_fullsized":"2023-10-26T05:51:18.346340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:19.867242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.372067+0000","last_change":"2023-10-26T05:51:18.371994+0000","last_active":"2023-10-26T05:51:18.372067+0000","last_peered":"2023-10-26T05:51:18.372067+0000","last_clean":"2023-10-26T05:51:18.372067+0000","last_became_active":"2023-10-26T05:51:18.371711+0000","last_became_peered":"2023-10-26T05:51:18.371711+0000","last_unstale":"2023-10-26T05:51:18.372067+0000","last_undegraded":"2023-10-26T05:51:18.372067+0000","last_fullsized":"2023-10-26T05:51:18.372067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:23.190182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.443503+0000","last_change":"2023-10-26T05:51:17.322173+0000","last_active":"2023-10-26T05:51:19.443503+0000","last_peered":"2023-10-26T05:51:19.443503+0000","last_clean":"2023-10-26T05:51:19.443503+0000","last_became_active":"2023-10-26T05:51:17.321475+0000","last_became_peered":"2023-10-26T05:51:17.321475+0000","last_unstale":"2023-10-26T05:51:19.443503+0000","last_undegraded":"2023-10-26T05:51:19.443503+0000","last_fullsized":"2023-10-26T05:51:19.443503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:32.813796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.368371+0000","last_change":"2023-10-26T05:51:18.368150+0000","last_active":"2023-10-26T05:51:18.368371+0000","last_peered":"2023-10-26T05:51:18.368371+0000","last_clean":"2023-10-26T05:51:18.368371+0000","last_became_active":"2023-10-26T05:51:18.367535+0000","last_became_peered":"2023-10-26T05:51:18.367535+0000","last_unstale":"2023-10-26T05:51:18.368371+0000","last_undegraded":"2023-10-26T05:51:18.368371+0000","last_fullsized":"2023-10-26T05:51:18.368371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:17:29.737714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331747+0000","last_change":"2023-10-26T05:51:17.314808+0000","last_active":"2023-10-26T05:51:19.331747+0000","last_peered":"2023-10-26T05:51:19.331747+0000","last_clean":"2023-10-26T05:51:19.331747+0000","last_became_active":"2023-10-26T05:51:17.314531+0000","last_became_peered":"2023-10-26T05:51:17.314531+0000","last_unstale":"2023-10-26T05:51:19.331747+0000","last_undegraded":"2023-10-26T05:51:19.331747+0000","last_fullsized":"2023-10-26T05:51:19.331747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:55:37.461256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.332522+0000","last_change":"2023-10-26T05:51:18.352442+0000","last_active":"2023-10-26T05:51:19.332522+0000","last_peered":"2023-10-26T05:51:19.332522+0000","last_clean":"2023-10-26T05:51:19.332522+0000","last_became_active":"2023-10-26T05:51:18.350609+0000","last_became_peered":"2023-10-26T05:51:18.350609+0000","last_unstale":"2023-10-26T05:51:19.332522+0000","last_undegraded":"2023-10-26T05:51:19.332522+0000","last_fullsized":"2023-10-26T05:51:19.332522+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:33:55.719578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.385794+0000","last_change":"2023-10-26T05:51:17.320906+0000","last_active":"2023-10-26T05:51:18.385794+0000","last_peered":"2023-10-26T05:51:18.385794+0000","last_clean":"2023-10-26T05:51:18.385794+0000","last_became_active":"2023-10-26T05:51:17.320278+0000","last_became_peered":"2023-10-26T05:51:17.320278+0000","last_unstale":"2023-10-26T05:51:18.385794+0000","last_undegraded":"2023-10-26T05:51:18.385794+0000","last_fullsized":"2023-10-26T05:51:18.385794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:47:09.468428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441951+0000","last_change":"2023-10-26T05:51:18.360066+0000","last_active":"2023-10-26T05:51:19.441951+0000","last_peered":"2023-10-26T05:51:19.441951+0000","last_clean":"2023-10-26T05:51:19.441951+0000","last_became_active":"2023-10-26T05:51:18.358597+0000","last_became_peered":"2023-10-26T05:51:18.358597+0000","last_unstale":"2023-10-26T05:51:19.441951+0000","last_undegraded":"2023-10-26T05:51:19.441951+0000","last_fullsized":"2023-10-26T05:51:19.441951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:34:23.837622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.441791+0000","last_change":"2023-10-26T05:51:17.329032+0000","last_active":"2023-10-26T05:51:19.441791+0000","last_peered":"2023-10-26T05:51:19.441791+0000","last_clean":"2023-10-26T05:51:19.441791+0000","last_became_active":"2023-10-26T05:51:17.327671+0000","last_became_peered":"2023-10-26T05:51:17.327671+0000","last_unstale":"2023-10-26T05:51:19.441791+0000","last_undegraded":"2023-10-26T05:51:19.441791+0000","last_fullsized":"2023-10-26T05:51:19.441791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:00:02.997848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336589+0000","last_change":"2023-10-26T05:51:18.364132+0000","last_active":"2023-10-26T05:51:19.336589+0000","last_peered":"2023-10-26T05:51:19.336589+0000","last_clean":"2023-10-26T05:51:19.336589+0000","last_became_active":"2023-10-26T05:51:18.363581+0000","last_became_peered":"2023-10-26T05:51:18.363581+0000","last_unstale":"2023-10-26T05:51:19.336589+0000","last_undegraded":"2023-10-26T05:51:19.336589+0000","last_fullsized":"2023-10-26T05:51:19.336589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:56.261271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.333122+0000","last_change":"2023-10-26T05:51:17.313890+0000","last_active":"2023-10-26T05:51:18.333122+0000","last_peered":"2023-10-26T05:51:18.333122+0000","last_clean":"2023-10-26T05:51:18.333122+0000","last_became_active":"2023-10-26T05:51:17.313404+0000","last_became_peered":"2023-10-26T05:51:17.313404+0000","last_unstale":"2023-10-26T05:51:18.333122+0000","last_undegraded":"2023-10-26T05:51:18.333122+0000","last_fullsized":"2023-10-26T05:51:18.333122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:21.939569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336907+0000","last_change":"2023-10-26T05:51:18.359775+0000","last_active":"2023-10-26T05:51:19.336907+0000","last_peered":"2023-10-26T05:51:19.336907+0000","last_clean":"2023-10-26T05:51:19.336907+0000","last_became_active":"2023-10-26T05:51:18.359173+0000","last_became_peered":"2023-10-26T05:51:18.359173+0000","last_unstale":"2023-10-26T05:51:19.336907+0000","last_undegraded":"2023-10-26T05:51:19.336907+0000","last_fullsized":"2023-10-26T05:51:19.336907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:55:03.859917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.331670+0000","last_change":"2023-10-26T05:51:17.322437+0000","last_active":"2023-10-26T05:51:19.331670+0000","last_peered":"2023-10-26T05:51:19.331670+0000","last_clean":"2023-10-26T05:51:19.331670+0000","last_became_active":"2023-10-26T05:51:17.322193+0000","last_became_peered":"2023-10-26T05:51:17.322193+0000","last_unstale":"2023-10-26T05:51:19.331670+0000","last_undegraded":"2023-10-26T05:51:19.331670+0000","last_fullsized":"2023-10-26T05:51:19.331670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:01:31.583859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.357792+0000","last_change":"2023-10-26T05:51:18.356268+0000","last_active":"2023-10-26T05:51:18.357792+0000","last_peered":"2023-10-26T05:51:18.357792+0000","last_clean":"2023-10-26T05:51:18.357792+0000","last_became_active":"2023-10-26T05:51:18.353894+0000","last_became_peered":"2023-10-26T05:51:18.353894+0000","last_unstale":"2023-10-26T05:51:18.357792+0000","last_undegraded":"2023-10-26T05:51:18.357792+0000","last_fullsized":"2023-10-26T05:51:18.357792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:03:22.559621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.332445+0000","last_change":"2023-10-26T05:51:17.325260+0000","last_active":"2023-10-26T05:51:18.332445+0000","last_peered":"2023-10-26T05:51:18.332445+0000","last_clean":"2023-10-26T05:51:18.332445+0000","last_became_active":"2023-10-26T05:51:17.324982+0000","last_became_peered":"2023-10-26T05:51:17.324982+0000","last_unstale":"2023-10-26T05:51:18.332445+0000","last_undegraded":"2023-10-26T05:51:18.332445+0000","last_fullsized":"2023-10-26T05:51:18.332445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:01:22.716955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.375685+0000","last_change":"2023-10-26T05:51:18.375484+0000","last_active":"2023-10-26T05:51:18.375685+0000","last_peered":"2023-10-26T05:51:18.375685+0000","last_clean":"2023-10-26T05:51:18.375685+0000","last_became_active":"2023-10-26T05:51:18.370738+0000","last_became_peered":"2023-10-26T05:51:18.370738+0000","last_unstale":"2023-10-26T05:51:18.375685+0000","last_undegraded":"2023-10-26T05:51:18.375685+0000","last_fullsized":"2023-10-26T05:51:18.375685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:24.750718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.330715+0000","last_change":"2023-10-26T05:51:17.318911+0000","last_active":"2023-10-26T05:51:18.330715+0000","last_peered":"2023-10-26T05:51:18.330715+0000","last_clean":"2023-10-26T05:51:18.330715+0000","last_became_active":"2023-10-26T05:51:17.318573+0000","last_became_peered":"2023-10-26T05:51:17.318573+0000","last_unstale":"2023-10-26T05:51:18.330715+0000","last_undegraded":"2023-10-26T05:51:18.330715+0000","last_fullsized":"2023-10-26T05:51:18.330715+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:03.807002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.363676+0000","last_change":"2023-10-26T05:51:18.363499+0000","last_active":"2023-10-26T05:51:18.363676+0000","last_peered":"2023-10-26T05:51:18.363676+0000","last_clean":"2023-10-26T05:51:18.363676+0000","last_became_active":"2023-10-26T05:51:18.359528+0000","last_became_peered":"2023-10-26T05:51:18.359528+0000","last_unstale":"2023-10-26T05:51:18.363676+0000","last_undegraded":"2023-10-26T05:51:18.363676+0000","last_fullsized":"2023-10-26T05:51:18.363676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:29:24.750718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-26T05:51:19.336082+0000","last_change":"2023-10-26T05:51:17.329311+0000","last_active":"2023-10-26T05:51:19.336082+0000","last_peered":"2023-10-26T05:51:19.336082+0000","last_clean":"2023-10-26T05:51:19.336082+0000","last_became_active":"2023-10-26T05:51:17.328313+0000","last_became_peered":"2023-10-26T05:51:17.328313+0000","last_unstale":"2023-10-26T05:51:19.336082+0000","last_undegraded":"2023-10-26T05:51:19.336082+0000","last_fullsized":"2023-10-26T05:51:19.336082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:33:10.305515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.371729+0000","last_change":"2023-10-26T05:51:18.371545+0000","last_active":"2023-10-26T05:51:18.371729+0000","last_peered":"2023-10-26T05:51:18.371729+0000","last_clean":"2023-10-26T05:51:18.371729+0000","last_became_active":"2023-10-26T05:51:18.370056+0000","last_became_peered":"2023-10-26T05:51:18.370056+0000","last_unstale":"2023-10-26T05:51:18.371729+0000","last_undegraded":"2023-10-26T05:51:18.371729+0000","last_fullsized":"2023-10-26T05:51:18.371729+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:17.418357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.346672+0000","last_change":"2023-10-26T05:51:17.329413+0000","last_active":"2023-10-26T05:51:18.346672+0000","last_peered":"2023-10-26T05:51:18.346672+0000","last_clean":"2023-10-26T05:51:18.346672+0000","last_became_active":"2023-10-26T05:51:17.326291+0000","last_became_peered":"2023-10-26T05:51:17.326291+0000","last_unstale":"2023-10-26T05:51:18.346672+0000","last_undegraded":"2023-10-26T05:51:18.346672+0000","last_fullsized":"2023-10-26T05:51:18.346672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:08:39.974332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.362150+0000","last_change":"2023-10-26T05:51:18.361949+0000","last_active":"2023-10-26T05:51:18.362150+0000","last_peered":"2023-10-26T05:51:18.362150+0000","last_clean":"2023-10-26T05:51:18.362150+0000","last_became_active":"2023-10-26T05:51:18.361432+0000","last_became_peered":"2023-10-26T05:51:18.361432+0000","last_unstale":"2023-10-26T05:51:18.362150+0000","last_undegraded":"2023-10-26T05:51:18.362150+0000","last_fullsized":"2023-10-26T05:51:18.362150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:17.290090+0000","last_clean_scrub_stamp":"2023-10-26T05:51:17.290090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:45:26.003905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"85'2","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.387463+0000","last_change":"2023-10-26T05:51:17.336865+0000","last_active":"2023-10-26T05:51:18.387463+0000","last_peered":"2023-10-26T05:51:18.387463+0000","last_clean":"2023-10-26T05:51:18.387463+0000","last_became_active":"2023-10-26T05:51:17.336144+0000","last_became_peered":"2023-10-26T05:51:17.336144+0000","last_unstale":"2023-10-26T05:51:18.387463+0000","last_undegraded":"2023-10-26T05:51:18.387463+0000","last_fullsized":"2023-10-26T05:51:18.387463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:51:16.278142+0000","last_clean_scrub_stamp":"2023-10-26T05:51:16.278142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:06.330605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-10-26T05:51:18.353537+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:51:18.353537+0000","last_peered":"2023-10-26T05:51:18.353537+0000","last_clean":"2023-10-26T05:51:18.353537+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:51:18.353537+0000","last_undegraded":"2023-10-26T05:51:18.353537+0000","last_fullsized":"2023-10-26T05:51:18.353537+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:57:23.631505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1019,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194292,"statfs":{"total":200000000,"available":198955814,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.86199999999999999},"min":{"1min":0.58799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.623},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.502,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.286,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.67300000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.218,"5min":1.391,"15min":1.391},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.73499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.3340000000000001,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.81499999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1639999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.91800000000000004,"15min":0.93600000000000005},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1859999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.04,"5min":0.97599999999999998,"15min":0.97399999999999998},"min":{"1min":0.69199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.242,"5min":1.375,"15min":1.375},"last":1.143}]},{"osd":4,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.63200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2909999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.95299999999999996,"15min":0.93999999999999995},"min":{"1min":0.748,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2190000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0669999999999999,"15min":1.125},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2370000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.046,"15min":1.089},"min":{"1min":0.69899999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.361,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.411}]},{"osd":6,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.077,"15min":1.111},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.413,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.274},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0509999999999999,"15min":1.0489999999999999},"min":{"1min":0.79700000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3280000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1970000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.097,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.87,"15min":0.85999999999999999},"min":{"1min":0.63200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.298,"5min":1.3,"15min":1.3},"last":0.91600000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.79400000000000004,"15min":0.76200000000000001},"min":{"1min":0.57699999999999996,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1160000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.86099999999999999,"15min":0.82999999999999996},"min":{"1min":0.63600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2190000000000001,"5min":1.405,"15min":1.405},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.83399999999999996,"15min":0.79300000000000004},"min":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.141,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.82699999999999996},"min":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.089,"5min":1.23,"15min":1.23},"last":1.04}]},{"osd":3,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":0.91400000000000003,"15min":0.877},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.069,"5min":0.98299999999999998,"15min":0.94799999999999995},"min":{"1min":0.81499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.1080000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.95199999999999996,"15min":0.96199999999999997},"min":{"1min":0.57899999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.2390000000000001,"5min":1.325,"15min":1.325},"last":1.429},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.98699999999999999,"15min":0.97499999999999998},"min":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":1.238}]},{"osd":5,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.0509999999999999,"15min":1.0529999999999999},"min":{"1min":0.78400000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.234,"5min":1.554,"15min":1.554},"last":1.155},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":1.0049999999999999},"min":{"1min":0.68300000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.1799999999999999,"5min":1.27,"15min":1.27},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:50:16 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.127,"15min":1.109},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.4339999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.069,"5min":1.121,"15min":1.1240000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.256,"5min":1.464,"15min":1.464},"last":1.2829999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.01},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97099999999999997,"15min":0.95499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.86199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.90000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2130000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.85799999999999998,"15min":0.85699999999999998},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.92100000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.89300000000000002,"15min":0.92700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2450000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92300000000000004,"15min":0.93300000000000005},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2709999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92000000000000004,"15min":0.92400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.409,"5min":1.548,"15min":1.548},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98199999999999998,"15min":1.006},"min":{"1min":0.73399999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4139999999999999,"5min":1.587,"15min":1.587},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.96899999999999997,"15min":1.01},"min":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.075,"15min":1.1060000000000001},"min":{"1min":0.67300000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.262,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0900000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.97499999999999998,"15min":1.032},"min":{"1min":0.65500000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2290000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.048},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.008,"15min":1.034},"min":{"1min":0.63100000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.624}]},{"osd":7,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.056,"15min":1.077},"min":{"1min":0.66800000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.534,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.149},{"interface":"front","average":{"1min":1.071,"5min":1.1279999999999999,"15min":1.1699999999999999},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546728,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85399999999999998,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.872,"5min":0.86499999999999999,"15min":0.877},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.502,"15min":1.502},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1819999999999999,"5min":1.375,"15min":1.375},"last":0.46899999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.91600000000000004,"15min":0.88900000000000001},"min":{"1min":0.755,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.282,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.501,"15min":0.501},"max":{"1min":1.1910000000000001,"5min":1.29,"15min":1.29},"last":0.623},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.85999999999999999},"min":{"1min":0.57599999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.226,"5min":1.373,"15min":1.373},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.66500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96499999999999997,"15min":0.93999999999999995},"min":{"1min":0.73199999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3300000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.874},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":0.999},"min":{"1min":0.64200000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2470000000000001,"5min":1.673,"15min":1.673},"last":1.0169999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0069999999999999,"15min":0.998},"min":{"1min":0.69499999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.383,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.999,"5min":1.032,"15min":1.0369999999999999},"min":{"1min":0.72899999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4690000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.034},"min":{"1min":0.83599999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4330000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.157,"5min":1.119,"15min":1.115},"min":{"1min":0.95199999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.403,"5min":1.728,"15min":1.728},"last":0.94099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":442,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194869,"statfs":{"total":200000000,"available":199546686,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.95099999999999996,"5min":1.155,"15min":1.155},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.073,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64200000000000002,"15min":0.65600000000000003},"min":{"1min":0.54300000000000004,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.038,"5min":1.085,"15min":1.085},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.57999999999999996,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.278,"5min":1.367,"15min":1.367},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.236,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.85899999999999999,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.92900000000000005,"15min":0.92700000000000005},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.27,"5min":1.3,"15min":1.3},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":0.96599999999999997},"min":{"1min":0.71899999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.248,"5min":1.478,"15min":1.478},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.90000000000000002},"min":{"1min":0.64300000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3100000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.96899999999999997,"15min":1.004},"min":{"1min":0.66200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.745},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0589999999999999,"15min":1.03},"min":{"1min":0.71399999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2330000000000001,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.94099999999999995,"15min":0.91500000000000004},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.383,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.038,"15min":1.046},"min":{"1min":0.68899999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.4019999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.88800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":444,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194868,"statfs":{"total":200000000,"available":199545186,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.42399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.317},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.64200000000000002,"15min":0.58199999999999996},"min":{"1min":0.50800000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.012,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.46200000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.54900000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2090000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.58399999999999996,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0740000000000001,"5min":1.294,"15min":1.294},"last":0.377}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.77700000000000002,"15min":0.72299999999999998},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.113,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.5},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.56799999999999995,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.27800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.86499999999999999,"15min":0.81200000000000006},"min":{"1min":0.78200000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.224,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2310000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94799999999999995,"15min":0.89700000000000002},"min":{"1min":0.77700000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.353,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.92400000000000004,"15min":0.88800000000000001},"min":{"1min":0.77600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4370000000000001,"5min":1.653,"15min":1.653},"last":0.84599999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.95399999999999996,"15min":0.90400000000000003},"min":{"1min":0.88500000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.526,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.05,"15min":0.95699999999999996},"min":{"1min":0.72899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.492,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.91300000000000003,"15min":0.88700000000000001},"min":{"1min":0.84899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.21,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.159,"15min":1.0129999999999999},"min":{"1min":0.91800000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":15.352,"15min":15.352},"last":0.86699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":444,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194867,"statfs":{"total":200000000,"available":199544570,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.311,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.751}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0209999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.72999999999999998,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0860000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.84699999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.165,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0700000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.56899999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1379999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96899999999999997,"15min":0.94499999999999995},"min":{"1min":0.75900000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.286,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.73899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.262,"5min":1.492,"15min":1.492},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.208,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.208}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.208,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.72599999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.23,"5min":1.837,"15min":1.837},"last":1.0489999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.01,"15min":1.028},"min":{"1min":0.71299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.248,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.248},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.095,"15min":1.093},"min":{"1min":0.64900000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.1759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1019,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194293,"statfs":{"total":200000000,"available":198956430,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0189999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0329999999999999,"5min":1.173,"15min":1.173},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.177},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.97599999999999998,"5min":1.409,"15min":1.409},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.67500000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.877,"15min":0.84399999999999997},"min":{"1min":0.67900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.88900000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.24,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.95699999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.92100000000000004},"min":{"1min":0.627,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.28,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.22,"5min":1.27,"15min":1.27},"last":1.111}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92000000000000004,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.02,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.764,"15min":1.764},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98799999999999999,"15min":0.99299999999999999},"min":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5780000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.288,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.0589999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:51:21.737 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:51:21.737 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:51:21.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc589ffb640 1 -- 172.21.15.71:0/888788195 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5840ad850 con 0x7fc58c06e950 2023-10-26T05:51:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 -- 172.21.15.71:0/888788195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c06e950 msgr2=0x7fc58c0b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 --2- 172.21.15.71:0/888788195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c06e950 0x7fc58c0b1150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc584009c80 tx=0x7fc58401cd50 comp rx=0 tx=0).stop 2023-10-26T05:51:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 -- 172.21.15.71:0/888788195 shutdown_connections 2023-10-26T05:51:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 --2- 172.21.15.71:0/888788195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c06e950 0x7fc58c0b1150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 --2- 172.21.15.71:0/888788195 >> v2:172.21.15.86:3300/0 conn(0x7fc58c0cd600 0x7fc58c06e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 -- 172.21.15.71:0/888788195 >> 172.21.15.71:0/888788195 conn(0x7fc58c0caa30 msgr2=0x7fc58c0cae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:21.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 -- 172.21.15.71:0/888788195 shutdown_connections 2023-10-26T05:51:21.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 -- 172.21.15.71:0/888788195 wait complete. 2023-10-26T05:51:21.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 Processor -- start 2023-10-26T05:51:21.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.896+0000 7fc591359640 1 -- start start 2023-10-26T05:51:21.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 0x7fc58c136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc58c0cd600 0x7fc58c137540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c144090 0x7fc58c0aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c137370 con 0x7fc58c06e950 2023-10-26T05:51:21.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c1390a0 con 0x7fc58c0cd600 2023-10-26T05:51:21.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc58c138000 con 0x7fc58c144090 2023-10-26T05:51:21.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 0x7fc58c136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 0x7fc58c136e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32836/0 (socket says 172.21.15.71:32836) 2023-10-26T05:51:21.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 -- 172.21.15.71:0/1959546805 learned_addr learned my addr 172.21.15.71:0/1959546805 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:21.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58b7fe640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c144090 0x7fc58c0aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58a7fc640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc58c0cd600 0x7fc58c137540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 -- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c144090 msgr2=0x7fc58c0aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c144090 0x7fc58c0aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 -- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc58c0cd600 msgr2=0x7fc58c137540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:21.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc58c0cd600 0x7fc58c137540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:21.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 -- 172.21.15.71:0/1959546805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5800094a0 con 0x7fc58c06e950 2023-10-26T05:51:21.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc58affd640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 0x7fc58c136e30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c370 tx=0x7fc57c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:21.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0052d0 con 0x7fc58c06e950 2023-10-26T05:51:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc57c007020 con 0x7fc58c06e950 2023-10-26T05:51:21.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5840098e0 con 0x7fc58c06e950 2023-10-26T05:51:21.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c007430 con 0x7fc58c06e950 2023-10-26T05:51:21.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58c1385c0 con 0x7fc58c06e950 2023-10-26T05:51:21.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58c066ae0 con 0x7fc58c06e950 2023-10-26T05:51:21.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc57c0067e0 con 0x7fc58c06e950 2023-10-26T05:51:21.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc5737fe640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc5780664a0 0x7fc578068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:21.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.900+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc57c083440 con 0x7fc58c06e950 2023-10-26T05:51:21.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.904+0000 7fc58a7fc640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc5780664a0 0x7fc578068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:21.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.904+0000 7fc58a7fc640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc5780664a0 0x7fc578068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc58c15e610 tx=0x7fc5800091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:21.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:21.904+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc57c0508c0 con 0x7fc58c06e950 2023-10-26T05:51:22.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.100+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc58c0721e0 con 0x7fc58c06e950 2023-10-26T05:51:22.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.104+0000 7fc5737fe640 1 -- 172.21.15.71:0/1959546805 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc57c053ee0 con 0x7fc58c06e950 2023-10-26T05:51:22.111 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:51:22.112 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:51:22.112 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:51:22.112 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:51:22.112 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:51:22.113 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:51:22.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc5780664a0 msgr2=0x7fc578068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc5780664a0 0x7fc578068960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc58c15e610 tx=0x7fc5800091e0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 msgr2=0x7fc58c136e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 0x7fc58c136e30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c370 tx=0x7fc57c00c830 comp rx=0 tx=0).stop 2023-10-26T05:51:22.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 shutdown_connections 2023-10-26T05:51:22.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc58c144090 0x7fc58c0aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc58c0cd600 0x7fc58c137540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc5780664a0 0x7fc578068960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 --2- 172.21.15.71:0/1959546805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc58c06e950 0x7fc58c136e30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 >> 172.21.15.71:0/1959546805 conn(0x7fc58c0caa30 msgr2=0x7fc58c139eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:22.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 shutdown_connections 2023-10-26T05:51:22.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.108+0000 7fc591359640 1 -- 172.21.15.71:0/1959546805 wait complete. 2023-10-26T05:51:22.130 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:22.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 -- 172.21.15.71:0/3801205240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813dc20 msgr2=0x7f7a0815b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 --2- 172.21.15.71:0/3801205240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813dc20 0x7f7a0815b9f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f7a0806db30 tx=0x7f7a0001c8a0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 -- 172.21.15.71:0/3801205240 shutdown_connections 2023-10-26T05:51:22.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 --2- 172.21.15.71:0/3801205240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a0813d800 0x7f7a0815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 --2- 172.21.15.71:0/3801205240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813dc20 0x7f7a0815b9f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 --2- 172.21.15.71:0/3801205240 >> v2:172.21.15.86:3300/0 conn(0x7f7a0813d420 0x7f7a080ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 -- 172.21.15.71:0/3801205240 >> 172.21.15.71:0/3801205240 conn(0x7f7a08069770 msgr2=0x7f7a08069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:22.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 -- 172.21.15.71:0/3801205240 shutdown_connections 2023-10-26T05:51:22.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 -- 172.21.15.71:0/3801205240 wait complete. 2023-10-26T05:51:22.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 Processor -- start 2023-10-26T05:51:22.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.244+0000 7f7a1043b640 1 -- start start 2023-10-26T05:51:22.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 0x7f7a0814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:22.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a0813d800 0x7f7a08138b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:22.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a0813dc20 0x7f7a0814d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:22.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a081356b0 con 0x7f7a0813d420 2023-10-26T05:51:22.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0814e230 con 0x7f7a0813d800 2023-10-26T05:51:22.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0813cf20 con 0x7f7a0813dc20 2023-10-26T05:51:22.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 0x7f7a0814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:22.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 0x7f7a0814d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32846/0 (socket says 172.21.15.71:32846) 2023-10-26T05:51:22.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 -- 172.21.15.71:0/2168879996 learned_addr learned my addr 172.21.15.71:0/2168879996 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:22.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e9d8640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a0813dc20 0x7f7a0814d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:22.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0d9d6640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a0813d800 0x7f7a08138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:22.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 -- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a0813dc20 msgr2=0x7f7a0814d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a0813dc20 0x7f7a0814d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 -- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a0813d800 msgr2=0x7f7a08138b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a0813d800 0x7f7a08138b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 -- 172.21.15.71:0/2168879996 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00009560 con 0x7f7a0813d420 2023-10-26T05:51:22.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0e1d7640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 0x7f7a0814d9e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400cc30 tx=0x7f7a0400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:22.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400f650 con 0x7f7a0813d420 2023-10-26T05:51:22.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7a0400f8a0 con 0x7f7a0813d420 2023-10-26T05:51:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0813c680 con 0x7f7a0813d420 2023-10-26T05:51:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0400fcb0 con 0x7f7a0813d420 2023-10-26T05:51:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0813cc40 con 0x7f7a0813d420 2023-10-26T05:51:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08066b10 con 0x7f7a0813d420 2023-10-26T05:51:22.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7a04018400 con 0x7f7a0813d420 2023-10-26T05:51:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f79feffd640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79ec0664a0 0x7f79ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a04084980 con 0x7f7a0813d420 2023-10-26T05:51:22.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.248+0000 7f7a0d9d6640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79ec0664a0 0x7f79ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:22.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.252+0000 7f7a0d9d6640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79ec0664a0 0x7f79ec068960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a080aa900 tx=0x7f79f8009110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:22.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.252+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7a04051f40 con 0x7f7a0813d420 2023-10-26T05:51:22.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.440+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a080721e0 con 0x7f7a0813d420 2023-10-26T05:51:22.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.440+0000 7f79feffd640 1 -- 172.21.15.71:0/2168879996 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f7a04055560 con 0x7f7a0813d420 2023-10-26T05:51:22.446 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:22.447 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:22.447 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:22.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.440+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79ec0664a0 msgr2=0x7f79ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.440+0000 7f7a1043b640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79ec0664a0 0x7f79ec068960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a080aa900 tx=0x7f79f8009110 comp rx=0 tx=0).stop 2023-10-26T05:51:22.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.440+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 msgr2=0x7f7a0814d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.440+0000 7f7a1043b640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 0x7f7a0814d9e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400cc30 tx=0x7f7a0400a620 comp rx=0 tx=0).stop 2023-10-26T05:51:22.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 shutdown_connections 2023-10-26T05:51:22.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a0813dc20 0x7f7a0814d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a0813d800 0x7f7a08138b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79ec0664a0 0x7f79ec068960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 --2- 172.21.15.71:0/2168879996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a0813d420 0x7f7a0814d9e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 >> 172.21.15.71:0/2168879996 conn(0x7f7a08069770 msgr2=0x7f7a080716c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 shutdown_connections 2023-10-26T05:51:22.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.444+0000 7f7a1043b640 1 -- 172.21.15.71:0/2168879996 wait complete. 2023-10-26T05:51:22.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:51:17.658710+0000', 'modified': '2023-10-26T05:51:19.700419+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6657}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6657': {'gid': 6657, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6833/2132957941', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 2132957941}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 2132957941}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:51:22.867 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:51:22.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T05:51:22.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 -- 172.21.15.71:0/1498491269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 msgr2=0x7f39a006f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 --2- 172.21.15.71:0/1498491269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 0x7f39a006f6f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3998009bc0 tx=0x7f399801cac0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 -- 172.21.15.71:0/1498491269 shutdown_connections 2023-10-26T05:51:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 --2- 172.21.15.71:0/1498491269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 0x7f39a006f6f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:22.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 --2- 172.21.15.71:0/1498491269 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a013ae90 0x7f39a00cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 -- 172.21.15.71:0/1498491269 >> 172.21.15.71:0/1498491269 conn(0x7f39a006b8c0 msgr2=0x7f39a006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 -- 172.21.15.71:0/1498491269 shutdown_connections 2023-10-26T05:51:23.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 -- 172.21.15.71:0/1498491269 wait complete. 2023-10-26T05:51:23.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 Processor -- start 2023-10-26T05:51:23.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.992+0000 7f39a7200640 1 -- start start 2023-10-26T05:51:23.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 0x7f39a01390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 0x7f39a0152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a0159060 0x7f39a013a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a0151bc0 con 0x7f39a013ae90 2023-10-26T05:51:23.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a0150fb0 con 0x7f39a013de80 2023-10-26T05:51:23.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a0151290 con 0x7f39a0159060 2023-10-26T05:51:23.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 0x7f39a01390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 0x7f39a01390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32852/0 (socket says 172.21.15.71:32852) 2023-10-26T05:51:23.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 -- 172.21.15.71:0/636342406 learned_addr learned my addr 172.21.15.71:0/636342406 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a579d640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a0159060 0x7f39a013a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399ffff640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 0x7f39a0152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 -- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a0159060 msgr2=0x7f39a013a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a0159060 0x7f39a013a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 -- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 msgr2=0x7f39a0152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 0x7f39a0152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 -- 172.21.15.71:0/636342406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3998009820 con 0x7f39a013ae90 2023-10-26T05:51:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a579d640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a0159060 0x7f39a013a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:51:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 0x7f39a01390d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f399000e7c0 tx=0x7f399000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399d7fa640 1 -- 172.21.15.71:0/636342406 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000ccc0 con 0x7f39a013ae90 2023-10-26T05:51:23.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399d7fa640 1 -- 172.21.15.71:0/636342406 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f399000eed0 con 0x7f39a013ae90 2023-10-26T05:51:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a0151570 con 0x7f39a013ae90 2023-10-26T05:51:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399d7fa640 1 -- 172.21.15.71:0/636342406 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990010690 con 0x7f39a013ae90 2023-10-26T05:51:23.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a0139610 con 0x7f39a013ae90 2023-10-26T05:51:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399d7fa640 1 -- 172.21.15.71:0/636342406 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39900107f0 con 0x7f39a013ae90 2023-10-26T05:51:23.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399d7fa640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f398c066470 0x7f398c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399d7fa640 1 -- 172.21.15.71:0/636342406 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3990014070 con 0x7f39a013ae90 2023-10-26T05:51:23.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:22.996+0000 7f399ffff640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f398c066470 0x7f398c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 Processor -- start 2023-10-26T05:51:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- start start 2023-10-26T05:51:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f399ffff640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f398c066470 0x7f398c068930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39a0157020 tx=0x7f3994009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e122cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e11e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e11e2f80 0x5602e11e3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e0fd7cc0 con 0x5602e122c9c0 2023-10-26T05:51:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e103a920 con 0x5602e122d2e0 2023-10-26T05:51:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602e1037e50 con 0x5602e11e2f80 2023-10-26T05:51:23.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e122cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e122cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32862/0 (socket says 172.21.15.71:32862) 2023-10-26T05:51:23.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 -- 172.21.15.71:0/103321940 learned_addr learned my addr 172.21.15.71:0/103321940 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:23.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e11e2f80 0x5602e11e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f399ffff640 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e11e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 -- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e11e2f80 msgr2=0x5602e11e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e11e2f80 0x5602e11e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 -- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 msgr2=0x5602e11e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e11e2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 -- 172.21.15.71:0/103321940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e0d2c890 con 0x5602e122c9c0 2023-10-26T05:51:23.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e11e2f80 0x5602e11e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:23.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e122cda0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f39900096e0 tx=0x7f3990056220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5d981d23e684bb server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39867fc640 1 -- 172.21.15.71:0/103321940 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399005de20 con 0x5602e122c9c0 2023-10-26T05:51:23.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39867fc640 1 -- 172.21.15.71:0/103321940 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f399001fdf0 con 0x5602e122c9c0 2023-10-26T05:51:23.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39867fc640 1 -- 172.21.15.71:0/103321940 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990052230 con 0x5602e122c9c0 2023-10-26T05:51:23.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- 172.21.15.71:0/103321940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 msgr2=0x5602e122cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e122cda0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f39900096e0 tx=0x7f3990056220 comp rx=0 tx=0).stop 2023-10-26T05:51:23.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- 172.21.15.71:0/103321940 shutdown_connections 2023-10-26T05:51:23.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e11e2f80 0x5602e11e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e11e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- 172.21.15.71:0/103321940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e122cda0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- 172.21.15.71:0/103321940 >> 172.21.15.71:0/103321940 conn(0x5602e11da2f0 msgr2=0x5602e11e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- 172.21.15.71:0/103321940 shutdown_connections 2023-10-26T05:51:23.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- 172.21.15.71:0/103321940 wait complete. 2023-10-26T05:51:23.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- start start 2023-10-26T05:51:23.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 Processor -- start 2023-10-26T05:51:23.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e122c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e11e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 0x5602e11e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e1033740 con 0x5602e122d2e0 2023-10-26T05:51:23.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e0ff2d10 con 0x5602e11e2f80 2023-10-26T05:51:23.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602e0ff29b0 con 0x5602e122c9c0 2023-10-26T05:51:23.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39aa34a000 10 client.0 _ll_register_callbacks cb 0x5602e11ce6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:51:23.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 0x5602e11e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 0x5602e11e8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32870/0 (socket says 172.21.15.71:32870) 2023-10-26T05:51:23.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 -- 172.21.15.71:0/4167507063 learned_addr learned my addr 172.21.15.71:0/4167507063 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:23.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e122c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f399ffff640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e11e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 -- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 msgr2=0x5602e11e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e11e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 -- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 msgr2=0x5602e122c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e122c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f39a579d640 1 -- 172.21.15.71:0/4167507063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a0139610 con 0x5602e122d2e0 2023-10-26T05:51:23.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.000+0000 7f399ffff640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e11e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:23.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39a579d640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 0x5602e11e8860 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3998009bc0 tx=0x7f39980092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6d8729e3d44103 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:51:23.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 1 -- 172.21.15.71:0/4167507063 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3998003b80 con 0x5602e122d2e0 2023-10-26T05:51:23.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 1 -- 172.21.15.71:0/4167507063 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3998003ce0 con 0x5602e122d2e0 2023-10-26T05:51:23.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602e0fe7370 con 0x5602e122d2e0 2023-10-26T05:51:23.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 1 -- 172.21.15.71:0/4167507063 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3998003ea0 con 0x5602e122d2e0 2023-10-26T05:51:23.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 10 client.6801 fetch_fsmap learned FSMap version 32 2023-10-26T05:51:23.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5602e0ff5460 con 0x5602e122d2e0 2023-10-26T05:51:23.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 1 -- 172.21.15.71:0/4167507063 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5602e0fe7370 con 0x5602e122d2e0 2023-10-26T05:51:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 1 -- 172.21.15.71:0/4167507063 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3998004130 con 0x5602e122d2e0 2023-10-26T05:51:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 10 client.6801 fetch_fsmap finished waiting for FSMap version 32 2023-10-26T05:51:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 10 client.6801 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:51:23.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 20 client.6801 populate_metadata read hostname 'smithi071' 2023-10-26T05:51:23.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e11ef410 0x5602e123ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 4 client.6801 mds_command: new command op to 6657 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:51:23.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 --> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602e0fcd2d0 con 0x5602e11ef410 2023-10-26T05:51:23.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e11ef410 0x5602e123ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e11ef410 0x5602e123ea70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6e703c475bd42a server_cookie=28002c072694fb5a in_seq=0 out_seq=0 2023-10-26T05:51:23.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 10 client.6801 ms_handle_connect on v2:172.21.15.71:6832/2132957941 2023-10-26T05:51:23.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 1 -- 172.21.15.71:0/4167507063 <== mds.0 v2:172.21.15.71:6832/2132957941 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5602e0fcd2d0 con 0x5602e11ef410 2023-10-26T05:51:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 10 client.6801 handle_command_reply: tid=0 2023-10-26T05:51:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 client.6801 shutdown 2023-10-26T05:51:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 msgr2=0x5602e11e8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 0x5602e11e8860 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3998009bc0 tx=0x7f39980092c0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 shutdown_connections 2023-10-26T05:51:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e11ef410 0x5602e123ea70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122d2e0 0x5602e11e8860 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e11e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 --2- 172.21.15.71:0/4167507063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e122c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f397f7fe640 0 client.6801 ms_handle_reset on v2:172.21.15.71:6832/2132957941 2023-10-26T05:51:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 >> 172.21.15.71:0/4167507063 conn(0x5602e11da2f0 msgr2=0x5602e11e1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 shutdown_connections 2023-10-26T05:51:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 1 -- 172.21.15.71:0/4167507063 wait complete. 2023-10-26T05:51:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.004+0000 7f39aa34a000 20 client.6801 trim_cache size 0 max 16384 2023-10-26T05:51:23.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 Processor -- start 2023-10-26T05:51:23.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- start start 2023-10-26T05:51:23.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 0x5602e122b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e122b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e123ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e0ced940 con 0x5602e11e2f80 2023-10-26T05:51:23.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e1037740 con 0x5602e122d2e0 2023-10-26T05:51:23.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602e10325d0 con 0x5602e122c9c0 2023-10-26T05:51:23.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 0x5602e122b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 0x5602e122b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32876/0 (socket says 172.21.15.71:32876) 2023-10-26T05:51:23.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 -- 172.21.15.71:0/1544350251 learned_addr learned my addr 172.21.15.71:0/1544350251 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:23.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 -- 172.21.15.71:0/1544350251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 msgr2=0x5602e122b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e122b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 -- 172.21.15.71:0/1544350251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 msgr2=0x5602e123ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e123ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 -- 172.21.15.71:0/1544350251 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602e0fdb760 con 0x5602e11e2f80 2023-10-26T05:51:23.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 0x5602e122b0e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f39900563d0 tx=0x7f3990009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8940fe8bef0dd28 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f3986ffd640 1 -- 172.21.15.71:0/1544350251 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990071070 con 0x5602e11e2f80 2023-10-26T05:51:23.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f3986ffd640 1 -- 172.21.15.71:0/1544350251 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f399005de20 con 0x5602e11e2f80 2023-10-26T05:51:23.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f3986ffd640 1 -- 172.21.15.71:0/1544350251 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3990053ec0 con 0x5602e11e2f80 2023-10-26T05:51:23.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- 172.21.15.71:0/1544350251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 msgr2=0x5602e122b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 0x5602e122b0e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f39900563d0 tx=0x7f3990009670 comp rx=0 tx=0).stop 2023-10-26T05:51:23.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- 172.21.15.71:0/1544350251 shutdown_connections 2023-10-26T05:51:23.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e122d2e0 0x5602e123ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122c9c0 0x5602e122b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1544350251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e11e2f80 0x5602e122b0e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- 172.21.15.71:0/1544350251 >> 172.21.15.71:0/1544350251 conn(0x5602e11da2f0 msgr2=0x5602e12299c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- 172.21.15.71:0/1544350251 shutdown_connections 2023-10-26T05:51:23.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- 172.21.15.71:0/1544350251 wait complete. 2023-10-26T05:51:23.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 -- start start 2023-10-26T05:51:23.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.020+0000 7f39aa34a000 1 Processor -- start 2023-10-26T05:51:23.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e1240080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e1250640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122d2e0 0x5602e1250b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e102da40 con 0x5602e122c9c0 2023-10-26T05:51:23.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602e10388b0 con 0x5602e11e2f80 2023-10-26T05:51:23.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602e11e5d90 con 0x5602e122d2e0 2023-10-26T05:51:23.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 10 client.0 _ll_register_callbacks cb 0x5602e1031ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:51:23.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e1250640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e1250640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32880/0 (socket says 172.21.15.71:32880) 2023-10-26T05:51:23.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 -- 172.21.15.71:0/1783595914 learned_addr learned my addr 172.21.15.71:0/1783595914 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:23.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 -- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122d2e0 msgr2=0x5602e1250b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122d2e0 0x5602e1250b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 -- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 msgr2=0x5602e1240080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:51:23.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e1240080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e1240080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 -- 172.21.15.71:0/1783595914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602e0d2c890 con 0x5602e122c9c0 2023-10-26T05:51:23.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f399ffff640 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e1250640 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3994001540 tx=0x7f3994001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b117633a2b21bd server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:51:23.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 1 -- 172.21.15.71:0/1783595914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3994003a90 con 0x5602e122c9c0 2023-10-26T05:51:23.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 1 -- 172.21.15.71:0/1783595914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3994003bf0 con 0x5602e122c9c0 2023-10-26T05:51:23.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602e1239c60 con 0x5602e122c9c0 2023-10-26T05:51:23.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 1 -- 172.21.15.71:0/1783595914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3994003db0 con 0x5602e122c9c0 2023-10-26T05:51:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 10 client.6807 fetch_fsmap learned FSMap version 32 2023-10-26T05:51:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5602e0d2ccc0 con 0x5602e122c9c0 2023-10-26T05:51:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 1 -- 172.21.15.71:0/1783595914 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5602e1239c60 con 0x5602e122c9c0 2023-10-26T05:51:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 1 -- 172.21.15.71:0/1783595914 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3994013420 con 0x5602e122c9c0 2023-10-26T05:51:23.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 10 client.6807 fetch_fsmap finished waiting for FSMap version 32 2023-10-26T05:51:23.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 10 client.6807 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:51:23.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 20 client.6807 populate_metadata read hostname 'smithi071' 2023-10-26T05:51:23.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e1293c80 0x5602e1294060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 4 client.6807 mds_command: new command op to 6657 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:51:23.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 --> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5602e0fcb6f0 con 0x5602e1293c80 2023-10-26T05:51:23.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e1293c80 0x5602e1294060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 10 client.6807 ms_handle_connect on v2:172.21.15.71:6832/2132957941 2023-10-26T05:51:23.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a4f9c640 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e1293c80 0x5602e1294060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6baa0b93234df48 server_cookie=1dac3a10357f9a5c in_seq=0 out_seq=0 2023-10-26T05:51:23.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 1 -- 172.21.15.71:0/1783595914 <== mds.0 v2:172.21.15.71:6832/2132957941 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5602e0fcb6f0 con 0x5602e1293c80 2023-10-26T05:51:23.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 10 client.6807 handle_command_reply: tid=0 2023-10-26T05:51:23.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 client.6807 shutdown 2023-10-26T05:51:23.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 msgr2=0x5602e1250640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e1250640 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3994001540 tx=0x7f3994001130 comp rx=0 tx=0).stop 2023-10-26T05:51:23.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 shutdown_connections 2023-10-26T05:51:23.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:6832/2132957941,v1:172.21.15.71:6833/2132957941] conn(0x5602e1293c80 0x5602e1294060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5602e122d2e0 0x5602e1250b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602e122c9c0 0x5602e1250640 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 --2- 172.21.15.71:0/1783595914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5602e11e2f80 0x5602e1240080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f397f7fe640 0 client.6807 ms_handle_reset on v2:172.21.15.71:6832/2132957941 2023-10-26T05:51:23.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 >> 172.21.15.71:0/1783595914 conn(0x5602e11da2f0 msgr2=0x5602e122a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 shutdown_connections 2023-10-26T05:51:23.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 1 -- 172.21.15.71:0/1783595914 wait complete. 2023-10-26T05:51:23.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39aa34a000 20 client.6807 trim_cache size 0 max 16384 2023-10-26T05:51:23.072 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:51:23.073 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:51:23.073 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:51:23.073 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6657, 2023-10-26T05:51:23.073 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:51:23.074 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:51:23.074 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:51:23.074 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 4.6602783490000004, 2023-10-26T05:51:23.074 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 32, 2023-10-26T05:51:23.074 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 86, 2023-10-26T05:51:23.075 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 86, 2023-10-26T05:51:23.075 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 9.3277021409999996 2023-10-26T05:51:23.075 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:51:23.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f398c066470 msgr2=0x7f398c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f398c066470 0x7f398c068930 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39a0157020 tx=0x7f3994009040 comp rx=0 tx=0).stop 2023-10-26T05:51:23.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 msgr2=0x7f39a01390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 0x7f39a01390d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f399000e7c0 tx=0x7f399000ec80 comp rx=0 tx=0).stop 2023-10-26T05:51:23.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 shutdown_connections 2023-10-26T05:51:23.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39a0159060 0x7f39a013a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39a013de80 0x7f39a0152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f398c066470 0x7f398c068930 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 --2- 172.21.15.71:0/636342406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39a013ae90 0x7f39a01390d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 >> 172.21.15.71:0/636342406 conn(0x7f39a006b8c0 msgr2=0x7f39a0156230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 shutdown_connections 2023-10-26T05:51:23.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.024+0000 7f39a7200640 1 -- 172.21.15.71:0/636342406 wait complete. 2023-10-26T05:51:23.080 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:51:23.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/3028698001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 msgr2=0x7f72bc158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- 172.21.15.71:0/3028698001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc158620 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0718f0 tx=0x7f72ac01c7e0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/3028698001 shutdown_connections 2023-10-26T05:51:23.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- 172.21.15.71:0/3028698001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc158620 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- 172.21.15.71:0/3028698001 >> v2:172.21.15.86:3300/0 conn(0x7f72bc0cdbb0 0x7f72bc15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- 172.21.15.71:0/3028698001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 0x7f72bc0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/3028698001 >> 172.21.15.71:0/3028698001 conn(0x7f72bc0afeb0 msgr2=0x7f72bc0b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/3028698001 shutdown_connections 2023-10-26T05:51:23.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/3028698001 wait complete. 2023-10-26T05:51:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 Processor -- start 2023-10-26T05:51:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- start start 2023-10-26T05:51:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72bc0cdbb0 0x7f72bc13a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 0x7f72bc155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc161bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1f4770 con 0x7f72bc15c150 2023-10-26T05:51:23.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc155d10 con 0x7f72bc0cdbb0 2023-10-26T05:51:23.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc13a850 con 0x7f72bc141a70 2023-10-26T05:51:23.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc161bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc161bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32908/0 (socket says 172.21.15.71:32908) 2023-10-26T05:51:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 -- 172.21.15.71:0/911077529 learned_addr learned my addr 172.21.15.71:0/911077529 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72bbfff640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 0x7f72bc155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c0e38640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72bc0cdbb0 0x7f72bc13a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 -- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 msgr2=0x7f72bc155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 0x7f72bc155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 -- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72bc0cdbb0 msgr2=0x7f72bc13a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72bc0cdbb0 0x7f72bc13a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 -- 172.21.15.71:0/911077529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac009560 con 0x7f72bc15c150 2023-10-26T05:51:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72bbfff640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 0x7f72bc155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:23.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c1639640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc161bf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009850 tx=0x7f72ac0a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac003800 con 0x7f72bc15c150 2023-10-26T05:51:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f72ac0a5850 con 0x7f72bc15c150 2023-10-26T05:51:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc157000 con 0x7f72bc15c150 2023-10-26T05:51:23.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac003d10 con 0x7f72bc15c150 2023-10-26T05:51:23.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc156240 con 0x7f72bc15c150 2023-10-26T05:51:23.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.192+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc066d00 con 0x7f72bc15c150 2023-10-26T05:51:23.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.196+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f72ac0b4070 con 0x7f72bc15c150 2023-10-26T05:51:23.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.196+0000 7f72b97fa640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f72a8066470 0x7f72a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:23.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.196+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72ac11eba0 con 0x7f72bc15c150 2023-10-26T05:51:23.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.196+0000 7f72c0e38640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f72a8066470 0x7f72a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:23.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.196+0000 7f72c0e38640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f72a8066470 0x7f72a8068930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f72b4008cc0 tx=0x7f72b4008230 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:23.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.200+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f72ac0ec0b0 con 0x7f72bc15c150 2023-10-26T05:51:23.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.380+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72bc0cafb0 con 0x7f72bc15c150 2023-10-26T05:51:23.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.380+0000 7f72b97fa640 1 -- 172.21.15.71:0/911077529 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f72ac0ef6d0 con 0x7f72bc15c150 2023-10-26T05:51:23.389 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:51:23.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f72a8066470 msgr2=0x7f72a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f72a8066470 0x7f72a8068930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f72b4008cc0 tx=0x7f72b4008230 comp rx=0 tx=0).stop 2023-10-26T05:51:23.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 msgr2=0x7f72bc161bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:23.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc161bf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009850 tx=0x7f72ac0a5660 comp rx=0 tx=0).stop 2023-10-26T05:51:23.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 shutdown_connections 2023-10-26T05:51:23.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f72a8066470 0x7f72a8068930 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72bc15c150 0x7f72bc161bf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f72bc141a70 0x7f72bc155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 --2- 172.21.15.71:0/911077529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72bc0cdbb0 0x7f72bc13a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:23.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 >> 172.21.15.71:0/911077529 conn(0x7f72bc0afeb0 msgr2=0x7f72bc158190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:23.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 shutdown_connections 2023-10-26T05:51:23.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:23.384+0000 7f72c309c640 1 -- 172.21.15.71:0/911077529 wait complete. 2023-10-26T05:51:23.405 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:51:23.405 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:51:23.406 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:51:23.406 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:51:23.406 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:23.407 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:51:23.408 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:51:23.408 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:51:23.408 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:51:23.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.410 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:51:23.416 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:51:23.416 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:51:23.416 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:51:23.417 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.417 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:51:23.417 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.417 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:51:23.417 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:23.418 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:51:23.418 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:51:23.418 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:23.418 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:51:23.418 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:51:23.419 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:23.419 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:51:23.419 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:23.419 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:51:23.419 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42293sec preferred_lft 42293sec 2023-10-26T05:51:23.420 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:51:23.420 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.420 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:51:23.420 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:23.420 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:51:23.421 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.421 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:51:23.421 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:23.421 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:51:23.421 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.422 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:51:23.422 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.422 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:51:23.422 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:23.422 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:51:23.423 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.423 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:51:23.423 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:23.423 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:51:23.423 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:23.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.424 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:51:23.470 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:51:23.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.472 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:51:23.519 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:51:23.519 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:51:23.519 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:51:23.520 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:51:23.520 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:51:23.520 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:51:23.520 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:51:23.520 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:51:23.521 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:51:23.521 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:51:23.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.522 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:51:23.522 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:51:23.523 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:51:23.523 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:51:23.523 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:51:23.524 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:51:23.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.586 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:51:23.633 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:51:23.633 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:51:23.633 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:51:23.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:51:23.688 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:51:23.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:51:23.689 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:23.737 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:51:23.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:51:23.738 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:23.784 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:51:23.833 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:51:23.834 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:51:23.834 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:51:23.834 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:51:23.834 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:51:23.835 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:51:23.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:51:23.835 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:51:23.835 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:51:23.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:51:23.836 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:51:23.836 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:51:23.836 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:51:23.836 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:51:23.836 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:51:23.837 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:51:23.837 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:51:23.837 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:51:23.837 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:51:23.837 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:51:23.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:51:23.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:51:23.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:51:23.838 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:51:23.838 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:51:23.839 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:51:23.839 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:51:23.839 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:51:23.839 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:51:23.839 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:23.840 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:23.840 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:23.840 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:23.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.842 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:51:23.890 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:51:23.890 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:51:23.934 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:51:23.941 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:51:23.941 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:51:23.942 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:51:23.942 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:51:23.942 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:51:23.942 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:51:23.943 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:51:23.943 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:51:23.943 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:51:23.943 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:51:23.943 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:51:23.944 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:51:23.944 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:51:23.944 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:51:23.944 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:51:23.944 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:51:23.945 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:51:23.945 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:51:23.945 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:51:23.945 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:51:23.946 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:51:23.946 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:51:23.946 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:51:23.946 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:51:23.946 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:51:23.947 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:51:23.947 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:51:23.947 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:51:23.947 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:51:23.948 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:23.948 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:23.948 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:23.948 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:23.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:23.950 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:51:24.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:51:24.012+0000 7fd60f264580 -1 init, newargv = 0x557a764242e0 newargc=15 2023-10-26T05:51:24.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[43962]: starting ceph client 2023-10-26T05:51:24.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[43962]: starting fuse 2023-10-26T05:51:25.102 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:51:25.112 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:51:25.112 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:51:25.112 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:51:25.112 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:51:25.113 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:51:25.113 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:51:25.113 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:51:25.113 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:51:25.114 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:51:25.114 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:51:25.114 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:51:25.114 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:51:25.115 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:51:25.115 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:51:25.115 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:51:25.115 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:51:25.115 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:51:25.116 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:51:25.116 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:51:25.116 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:51:25.116 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:51:25.117 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:51:25.117 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:51:25.117 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:51:25.117 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:51:25.118 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:51:25.118 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:51:25.118 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:51:25.118 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:51:25.119 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:25.119 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:51:25.119 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:25.119 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:51:25.120 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:25.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:25.121 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:51:25.173 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:51:25.174 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:51:25.174 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:51:25.175 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:51:25.175 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:51:25.175 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:51:25.176 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:51:25.176 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:25.177 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:51:25.177 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:51:25.178 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:51:25.178 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:51:25.179 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:25.179 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:51:25.180 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:51:25.180 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:51:25.180 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:51:25.181 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:25.181 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:51:25.182 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:51:25.182 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:51:25.183 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:51:25.183 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:51:25.183 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:51:25.184 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:51:25.184 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:51:25.185 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:25.185 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:51:25.186 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:51:25.298 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.43962.asok 2023-10-26T05:51:25.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:25.310 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43962.asok 2023-10-26T05:51:25.310 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:25.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.43962.asok status 2023-10-26T05:51:25.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:25.503 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:51:25.504 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:51:25.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:51:25.504 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:51:25.504 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:51:25.505 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:51:25.505 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:51:25.505 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "43962", 2023-10-26T05:51:25.505 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:51:25.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:25.506 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:51:25.506 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:51:25.506 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 6816, 2023-10-26T05:51:25.506 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:51:25.506 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:51:25.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:51:25.507 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 6816 2023-10-26T05:51:25.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:25.507 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:51:25.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:51:25.507 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:51:25.508 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1045081847 2023-10-26T05:51:25.508 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:51:25.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:25.508 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:51:25.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:51:25.509 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:51:25.509 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1045081847 2023-10-26T05:51:25.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:51:25.509 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.6816 192.168.0.1:0/1045081847", 2023-10-26T05:51:25.509 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1045081847", 2023-10-26T05:51:25.510 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:51:25.510 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 32, 2023-10-26T05:51:25.510 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 86, 2023-10-26T05:51:25.510 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:51:25.511 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:51:25.511 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:51:25.511 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:51:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:25.523 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:25.523 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:51:25.524 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:25.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:25.572 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:51:25.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:25.579 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:25.579 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:25.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:25.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:25.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:25.631 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:51:25.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/1261245191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd414e300 msgr2=0x7f0bd4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:25.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/1261245191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd414e300 0x7f0bd4157a00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0bd415bfd0 tx=0x7f0bbc01c810 comp rx=0 tx=0).stop 2023-10-26T05:51:25.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/1261245191 shutdown_connections 2023-10-26T05:51:25.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/1261245191 >> v2:172.21.15.86:3300/0 conn(0x7f0bd4157f40 0x7f0bd41583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/1261245191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd414e300 0x7f0bd4157a00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/1261245191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd406d730 0x7f0bd40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/1261245191 >> 172.21.15.71:0/1261245191 conn(0x7f0bd4069770 msgr2=0x7f0bd4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:25.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/1261245191 shutdown_connections 2023-10-26T05:51:25.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/1261245191 wait complete. 2023-10-26T05:51:25.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 Processor -- start 2023-10-26T05:51:25.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- start start 2023-10-26T05:51:25.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd406d730 0x7f0bd41f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:25.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd414e300 0x7f0bd41494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:25.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 0x7f0bd4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:25.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd414ec50 con 0x7f0bd4157f40 2023-10-26T05:51:25.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd414a600 con 0x7f0bd406d730 2023-10-26T05:51:25.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4149d60 con 0x7f0bd414e300 2023-10-26T05:51:25.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 0x7f0bd4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:25.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 0x7f0bd4069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32930/0 (socket says 172.21.15.71:32930) 2023-10-26T05:51:25.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 -- 172.21.15.71:0/3485911363 learned_addr learned my addr 172.21.15.71:0/3485911363 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:25.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd3fff640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd406d730 0x7f0bd41f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:25.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bcbfff640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd414e300 0x7f0bd41494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:25.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 -- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd414e300 msgr2=0x7f0bd41494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:25.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd414e300 0x7f0bd41494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 -- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd406d730 msgr2=0x7f0bd41f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:25.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd406d730 0x7f0bd41f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 -- 172.21.15.71:0/3485911363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc009560 con 0x7f0bd4157f40 2023-10-26T05:51:25.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bcbfff640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd414e300 0x7f0bd41494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:25.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd8e6d640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 0x7f0bd4069db0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00e970 tx=0x7f0bcc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:25.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004d30 con 0x7f0bd4157f40 2023-10-26T05:51:25.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bcc00cb20 con 0x7f0bd4157f40 2023-10-26T05:51:25.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd414a040 con 0x7f0bd4157f40 2023-10-26T05:51:25.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc005020 con 0x7f0bd4157f40 2023-10-26T05:51:25.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.792+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd41499e0 con 0x7f0bd4157f40 2023-10-26T05:51:25.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.796+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4066b10 con 0x7f0bd4157f40 2023-10-26T05:51:25.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.796+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0bcc005180 con 0x7f0bd4157f40 2023-10-26T05:51:25.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.796+0000 7f0bd17fa640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bb4066470 0x7f0bb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:25.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.796+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bcc084cb0 con 0x7f0bd4157f40 2023-10-26T05:51:25.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.796+0000 7f0bd3fff640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bb4066470 0x7f0bb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:25.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.796+0000 7f0bd3fff640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bb4066470 0x7f0bb4068930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005fd0 tx=0x7f0bc4005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:25.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.800+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0bcc052210 con 0x7f0bd4157f40 2023-10-26T05:51:25.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0bd40721e0 con 0x7f0bd4157f40 2023-10-26T05:51:25.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bd17fa640 1 -- 172.21.15.71:0/3485911363 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0bcc01f020 con 0x7f0bd4157f40 2023-10-26T05:51:25.994 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:51:25.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bb4066470 msgr2=0x7f0bb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:25.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bb4066470 0x7f0bb4068930 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4005fd0 tx=0x7f0bc4005e60 comp rx=0 tx=0).stop 2023-10-26T05:51:25.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 msgr2=0x7f0bd4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:25.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 0x7f0bd4069db0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00e970 tx=0x7f0bcc00ee30 comp rx=0 tx=0).stop 2023-10-26T05:51:25.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 shutdown_connections 2023-10-26T05:51:25.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bb4066470 0x7f0bb4068930 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd4157f40 0x7f0bd4069db0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0bd414e300 0x7f0bd41494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 --2- 172.21.15.71:0/3485911363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0bd406d730 0x7f0bd41f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:25.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 >> 172.21.15.71:0/3485911363 conn(0x7f0bd4069770 msgr2=0x7f0bd406f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:25.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 shutdown_connections 2023-10-26T05:51:25.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:25.988+0000 7f0bda8d0640 1 -- 172.21.15.71:0/3485911363 wait complete. 2023-10-26T05:51:26.013 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:51:26.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:51:26.014 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:51:26.014 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:51:26.014 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:26.015 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:51:26.015 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:51:26.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:51:26.016 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:51:26.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.016 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:51:26.024 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:51:26.024 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:51:26.024 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:51:26.024 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.025 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:51:26.025 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.025 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:51:26.025 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:26.025 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:51:26.026 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:51:26.026 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:26.026 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:51:26.026 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:51:26.026 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:26.026 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:51:26.027 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:26.027 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:51:26.028 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42285sec preferred_lft 42285sec 2023-10-26T05:51:26.028 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:51:26.028 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.028 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:51:26.028 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:26.029 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:51:26.029 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.029 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:51:26.029 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:51:26.029 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:51:26.030 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.030 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:51:26.030 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.031 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:51:26.031 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:26.031 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:51:26.031 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.031 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:51:26.031 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:51:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:51:26.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.033 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:51:26.082 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:51:26.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.083 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:51:26.134 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:51:26.135 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:51:26.135 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:51:26.135 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:51:26.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:51:26.136 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:51:26.136 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:51:26.136 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:51:26.136 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:51:26.136 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:51:26.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.137 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:51:26.138 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:51:26.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:51:26.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:51:26.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:51:26.139 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:51:26.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.215 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:51:26.263 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:51:26.263 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:51:26.263 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:51:26.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:51:26.319 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:51:26.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:51:26.320 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:26.368 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:51:26.368 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:51:26.368 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:26.415 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:51:26.465 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:51:26.465 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:51:26.466 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:51:26.466 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:51:26.466 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:51:26.466 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:51:26.467 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:51:26.467 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:51:26.467 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:51:26.467 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:51:26.467 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:51:26.468 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:51:26.468 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:51:26.468 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:51:26.468 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:51:26.469 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:51:26.469 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:51:26.469 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:51:26.469 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:51:26.469 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:51:26.470 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:51:26.470 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:51:26.470 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:51:26.470 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:51:26.471 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:51:26.471 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:51:26.471 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:51:26.471 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:51:26.471 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:51:26.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:26.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:26.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:26.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:26.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.474 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:51:26.518 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:51:26.519 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:51:26.564 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:51:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:51:26.570 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:51:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:51:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:51:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:51:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:51:26.571 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:51:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:51:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:51:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:51:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:51:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:51:26.572 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:51:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:51:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:51:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:51:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:51:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:51:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:51:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:51:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:51:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:51:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:51:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:51:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:51:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:51:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:51:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:51:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:51:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:26.576 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:26.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:26.578 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:51:26.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:51:26.646+0000 7fb39e460580 -1 init, newargv = 0x55fa0c36cf70 newargc=15 2023-10-26T05:51:26.647 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34587]: starting ceph client 2023-10-26T05:51:26.653 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34587]: starting fuse 2023-10-26T05:51:27.728 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:51:27.736 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:51:27.737 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:51:27.737 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:51:27.737 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:51:27.737 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:51:27.738 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:51:27.738 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:51:27.738 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:51:27.738 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:51:27.738 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:51:27.739 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:51:27.739 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:51:27.739 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:51:27.739 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:51:27.739 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:51:27.740 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:51:27.740 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:51:27.740 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:51:27.740 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:51:27.740 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:51:27.741 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:51:27.741 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:51:27.741 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:51:27.741 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:51:27.741 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:51:27.742 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:51:27.742 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:51:27.742 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:51:27.742 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:51:27.742 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:27.743 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:51:27.745 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:27.745 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:51:27.746 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:27.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:27.746 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:51:27.783 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:51:27.783 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:51:27.784 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:51:27.784 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:51:27.785 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:51:27.785 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:51:27.785 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:51:27.786 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:51:27.786 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:51:27.786 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:51:27.787 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:51:27.787 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:51:27.788 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:51:27.788 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:51:27.788 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:51:27.789 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:51:27.789 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:51:27.789 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:51:27.790 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:51:27.790 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:51:27.791 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:51:27.791 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:51:27.791 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:51:27.792 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:51:27.792 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:51:27.793 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:51:27.793 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:51:27.793 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:51:27.794 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:51:27.906 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.34587.asok 2023-10-26T05:51:27.909 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:27.917 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34587.asok 2023-10-26T05:51:27.918 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:27.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.34587.asok status 2023-10-26T05:51:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:28.101 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:51:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:51:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:51:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:51:28.102 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:51:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:51:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:51:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34587", 2023-10-26T05:51:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:51:28.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:51:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:51:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:51:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5638, 2023-10-26T05:51:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:51:28.104 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:51:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:51:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 5638 2023-10-26T05:51:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:51:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:51:28.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:51:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:51:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 899513911 2023-10-26T05:51:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:51:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:51:28.106 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:51:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:51:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:51:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 899513911 2023-10-26T05:51:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:51:28.107 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.5638 192.168.0.1:0/899513911", 2023-10-26T05:51:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/899513911", 2023-10-26T05:51:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:51:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 32, 2023-10-26T05:51:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 86, 2023-10-26T05:51:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:51:28.108 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:51:28.109 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:51:28.109 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:51:28.114 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:28.121 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:28.122 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:51:28.122 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:28.166 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:28.167 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:51:28.168 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:28.174 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:28.175 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:28.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:28.223 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:28.224 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:28.230 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:28.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 -- 172.21.15.71:0/3431338752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c15b9f0 msgr2=0x7f014c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:28.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 --2- 172.21.15.71:0/3431338752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c15b9f0 0x7f014c070690 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f0144009c80 tx=0x7f014401cd40 comp rx=0 tx=0).stop 2023-10-26T05:51:28.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 -- 172.21.15.71:0/3431338752 shutdown_connections 2023-10-26T05:51:28.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 --2- 172.21.15.71:0/3431338752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c15b9f0 0x7f014c070690 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 --2- 172.21.15.71:0/3431338752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f014c0ac740 0x7f014c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 -- 172.21.15.71:0/3431338752 >> 172.21.15.71:0/3431338752 conn(0x7f014c069770 msgr2=0x7f014c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:28.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 -- 172.21.15.71:0/3431338752 shutdown_connections 2023-10-26T05:51:28.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 -- 172.21.15.71:0/3431338752 wait complete. 2023-10-26T05:51:28.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 Processor -- start 2023-10-26T05:51:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.360+0000 7f0153ae2640 1 -- start start 2023-10-26T05:51:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f014c0ac740 0x7f014c13a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 0x7f014c1513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f014c15b9f0 0x7f014c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:28.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c137810 con 0x7f014c13d420 2023-10-26T05:51:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c1530e0 con 0x7f014c0ac740 2023-10-26T05:51:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c1524f0 con 0x7f014c15b9f0 2023-10-26T05:51:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 0x7f014c1513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 0x7f014c1513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32946/0 (socket says 172.21.15.71:32946) 2023-10-26T05:51:28.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 -- 172.21.15.71:0/1599743901 learned_addr learned my addr 172.21.15.71:0/1599743901 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015187e640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f014c0ac740 0x7f014c13a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015207f640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f014c15b9f0 0x7f014c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 -- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f014c15b9f0 msgr2=0x7f014c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:28.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f014c15b9f0 0x7f014c0accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 -- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f014c0ac740 msgr2=0x7f014c13a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:28.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f014c0ac740 0x7f014c13a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 -- 172.21.15.71:0/1599743901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440098e0 con 0x7f014c13d420 2023-10-26T05:51:28.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f015107d640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 0x7f014c1513c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f013c00dbd0 tx=0x7f013c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:28.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c005220 con 0x7f014c13d420 2023-10-26T05:51:28.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f013c00ded0 con 0x7f014c13d420 2023-10-26T05:51:28.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00ae50 con 0x7f014c13d420 2023-10-26T05:51:28.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c1527b0 con 0x7f014c13d420 2023-10-26T05:51:28.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c152d70 con 0x7f014c13d420 2023-10-26T05:51:28.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c066b10 con 0x7f014c13d420 2023-10-26T05:51:28.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f013c0047b0 con 0x7f014c13d420 2023-10-26T05:51:28.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f01427fc640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0130066470 0x7f0130068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:28.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.364+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f013c0846c0 con 0x7f014c13d420 2023-10-26T05:51:28.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.368+0000 7f015187e640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0130066470 0x7f0130068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:28.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.368+0000 7f015187e640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0130066470 0x7f0130068930 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0148004e30 tx=0x7f0148009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:28.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.372+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f013c051b40 con 0x7f014c13d420 2023-10-26T05:51:28.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.548+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f014c0721e0 con 0x7f014c13d420 2023-10-26T05:51:28.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f01427fc640 1 -- 172.21.15.71:0/1599743901 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f013c055160 con 0x7f014c13d420 2023-10-26T05:51:28.556 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:28.557 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:28.557 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:28.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0130066470 msgr2=0x7f0130068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:28.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0130066470 0x7f0130068930 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0148004e30 tx=0x7f0148009040 comp rx=0 tx=0).stop 2023-10-26T05:51:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 msgr2=0x7f014c1513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 0x7f014c1513c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f013c00dbd0 tx=0x7f013c00a620 comp rx=0 tx=0).stop 2023-10-26T05:51:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 shutdown_connections 2023-10-26T05:51:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f014c15b9f0 0x7f014c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0130066470 0x7f0130068930 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f014c13d420 0x7f014c1513c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 --2- 172.21.15.71:0/1599743901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f014c0ac740 0x7f014c13a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 >> 172.21.15.71:0/1599743901 conn(0x7f014c069770 msgr2=0x7f014c15a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 shutdown_connections 2023-10-26T05:51:28.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:28.552+0000 7f0153ae2640 1 -- 172.21.15.71:0/1599743901 wait complete. 2023-10-26T05:51:28.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:51:28.692 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":6657,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.309598867,"mdsmap_epoch":32,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":14.977025643999999} 2023-10-26T05:51:28.693 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 6657, "mdsmap_epoch": 32, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 10.309598867, "state": "up:active", "uptime": 14.977025644, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:51:28.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:51:28.810 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:51:28.811 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:51:28.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T05:51:28.936 INFO:teuthology.orchestra.run.smithi071.stdout:{"mon_tick_interval":"5"} 2023-10-26T05:51:28.937 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T05:51:28.937 INFO:tasks.cephfs.test_full:pool capacity 557028928, 531MB should be enough to fill it 2023-10-26T05:51:28.937 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:51:28.938 DEBUG:teuthology.orchestra.run.smithi071:> import time 2023-10-26T05:51:28.938 DEBUG:teuthology.orchestra.run.smithi071:> import datetime 2023-10-26T05:51:28.939 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:51:28.939 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:51:28.939 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.940 DEBUG:teuthology.orchestra.run.smithi071:> # Write some buffered data through before going full, all should be well 2023-10-26T05:51:28.940 DEBUG:teuthology.orchestra.run.smithi071:> print("writing some data through which we expect to succeed") 2023-10-26T05:51:28.940 DEBUG:teuthology.orchestra.run.smithi071:> bytes = 0 2023-10-26T05:51:28.941 DEBUG:teuthology.orchestra.run.smithi071:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T05:51:28.941 DEBUG:teuthology.orchestra.run.smithi071:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T05:51:28.941 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-26T05:51:28.942 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:51:28.942 DEBUG:teuthology.orchestra.run.smithi071:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T05:51:28.942 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.943 DEBUG:teuthology.orchestra.run.smithi071:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T05:51:28.943 DEBUG:teuthology.orchestra.run.smithi071:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T05:51:28.943 DEBUG:teuthology.orchestra.run.smithi071:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T05:51:28.944 DEBUG:teuthology.orchestra.run.smithi071:> # from write 2023-10-26T05:51:28.944 DEBUG:teuthology.orchestra.run.smithi071:> full = False 2023-10-26T05:51:28.944 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.945 DEBUG:teuthology.orchestra.run.smithi071:> for n in range(0, int(531 * 1.1)): 2023-10-26T05:51:28.945 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:51:28.945 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T05:51:28.946 DEBUG:teuthology.orchestra.run.smithi071:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-26T05:51:28.946 DEBUG:teuthology.orchestra.run.smithi071:> except OSError as e: 2023-10-26T05:51:28.946 DEBUG:teuthology.orchestra.run.smithi071:> if True: 2023-10-26T05:51:28.947 DEBUG:teuthology.orchestra.run.smithi071:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-26T05:51:28.947 DEBUG:teuthology.orchestra.run.smithi071:> raise 2023-10-26T05:51:28.947 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:51:28.948 DEBUG:teuthology.orchestra.run.smithi071:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T05:51:28.948 DEBUG:teuthology.orchestra.run.smithi071:> full = True 2023-10-26T05:51:28.948 DEBUG:teuthology.orchestra.run.smithi071:> break 2023-10-26T05:51:28.949 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.949 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:51:28.949 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:51:28.950 DEBUG:teuthology.orchestra.run.smithi071:> print("fsync'"'"'ed successfully") 2023-10-26T05:51:28.950 DEBUG:teuthology.orchestra.run.smithi071:> except OSError as e: 2023-10-26T05:51:28.950 DEBUG:teuthology.orchestra.run.smithi071:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T05:51:28.951 DEBUG:teuthology.orchestra.run.smithi071:> full = True 2023-10-26T05:51:28.951 DEBUG:teuthology.orchestra.run.smithi071:> break 2023-10-26T05:51:28.951 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:51:28.952 DEBUG:teuthology.orchestra.run.smithi071:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T05:51:28.952 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.952 DEBUG:teuthology.orchestra.run.smithi071:> if n > 531 * 0.9: 2023-10-26T05:51:28.953 DEBUG:teuthology.orchestra.run.smithi071:> # Be cautious in the last region where we expect to hit 2023-10-26T05:51:28.953 DEBUG:teuthology.orchestra.run.smithi071:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-26T05:51:28.953 DEBUG:teuthology.orchestra.run.smithi071:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T05:51:28.954 DEBUG:teuthology.orchestra.run.smithi071:> time.sleep(15.0) 2023-10-26T05:51:28.954 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.954 DEBUG:teuthology.orchestra.run.smithi071:> if not full: 2023-10-26T05:51:28.955 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-26T05:51:28.955 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.955 DEBUG:teuthology.orchestra.run.smithi071:> # close() should not raise an error because we already caught it in 2023-10-26T05:51:28.956 DEBUG:teuthology.orchestra.run.smithi071:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-26T05:51:28.956 DEBUG:teuthology.orchestra.run.smithi071:> # since then because all IOs got cancelled on the full flag. 2023-10-26T05:51:28.956 DEBUG:teuthology.orchestra.run.smithi071:> print("calling close") 2023-10-26T05:51:28.957 DEBUG:teuthology.orchestra.run.smithi071:> os.close(f) 2023-10-26T05:51:28.957 DEBUG:teuthology.orchestra.run.smithi071:> print("close() did not raise error") 2023-10-26T05:51:28.957 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:51:28.958 DEBUG:teuthology.orchestra.run.smithi071:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T05:51:28.958 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:51:34.483 INFO:teuthology.orchestra.run.smithi071.stdout:writing some data through which we expect to succeed 2023-10-26T05:51:34.484 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T05:51:34.484 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.484 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 1.00 MB 2023-10-26T05:51:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 2.00 MB 2023-10-26T05:51:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.486 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.486 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 3.00 MB 2023-10-26T05:51:34.486 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.486 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.486 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 4.00 MB 2023-10-26T05:51:34.487 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.487 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.487 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 5.00 MB 2023-10-26T05:51:34.487 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.487 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.488 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 6.00 MB 2023-10-26T05:51:34.488 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.488 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.488 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 7.00 MB 2023-10-26T05:51:34.488 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 8.00 MB 2023-10-26T05:51:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 9.00 MB 2023-10-26T05:51:34.489 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.490 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.490 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 10.00 MB 2023-10-26T05:51:34.490 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.490 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.490 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 11.00 MB 2023-10-26T05:51:34.491 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.491 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.491 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 12.00 MB 2023-10-26T05:51:34.491 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.491 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 13.00 MB 2023-10-26T05:51:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 14.00 MB 2023-10-26T05:51:34.492 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.493 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.493 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 15.00 MB 2023-10-26T05:51:34.493 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.493 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.493 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 16.00 MB 2023-10-26T05:51:34.494 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.494 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.494 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 17.00 MB 2023-10-26T05:51:34.494 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.494 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.495 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 18.00 MB 2023-10-26T05:51:34.495 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.495 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.495 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 19.00 MB 2023-10-26T05:51:34.495 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.496 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.496 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 20.00 MB 2023-10-26T05:51:34.496 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.496 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.496 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 21.00 MB 2023-10-26T05:51:34.497 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.497 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.497 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 22.00 MB 2023-10-26T05:51:34.497 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.497 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.498 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 23.00 MB 2023-10-26T05:51:34.498 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.498 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.498 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 24.00 MB 2023-10-26T05:51:34.499 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.499 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.499 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 25.00 MB 2023-10-26T05:51:34.499 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.499 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.500 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 26.00 MB 2023-10-26T05:51:34.500 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.500 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.500 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 27.00 MB 2023-10-26T05:51:34.500 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.501 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.501 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 28.00 MB 2023-10-26T05:51:34.501 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.501 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.502 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 29.00 MB 2023-10-26T05:51:34.502 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.502 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.502 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 30.00 MB 2023-10-26T05:51:34.502 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.503 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.503 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 31.00 MB 2023-10-26T05:51:34.503 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.503 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.503 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 32.00 MB 2023-10-26T05:51:34.504 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.504 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.504 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 33.00 MB 2023-10-26T05:51:34.504 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.504 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.505 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 34.00 MB 2023-10-26T05:51:34.505 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.505 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.505 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 35.00 MB 2023-10-26T05:51:34.505 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.506 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.506 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 36.00 MB 2023-10-26T05:51:34.506 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.506 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.506 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 37.00 MB 2023-10-26T05:51:34.507 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.507 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.507 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 38.00 MB 2023-10-26T05:51:34.507 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 39.00 MB 2023-10-26T05:51:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.508 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.509 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 40.00 MB 2023-10-26T05:51:34.509 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.509 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.509 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 41.00 MB 2023-10-26T05:51:34.509 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 42.00 MB 2023-10-26T05:51:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 43.00 MB 2023-10-26T05:51:34.511 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.511 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.511 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 44.00 MB 2023-10-26T05:51:34.511 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.511 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.512 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 45.00 MB 2023-10-26T05:51:34.512 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.512 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.512 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 46.00 MB 2023-10-26T05:51:34.512 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.513 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.513 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 47.00 MB 2023-10-26T05:51:34.513 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.513 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.513 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 48.00 MB 2023-10-26T05:51:34.514 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.514 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.514 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 49.00 MB 2023-10-26T05:51:34.514 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.514 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 50.00 MB 2023-10-26T05:51:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 51.00 MB 2023-10-26T05:51:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.515 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.516 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 52.00 MB 2023-10-26T05:51:34.516 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.516 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.516 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 53.00 MB 2023-10-26T05:51:34.516 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.517 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.517 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 54.00 MB 2023-10-26T05:51:34.517 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.517 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.517 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 55.00 MB 2023-10-26T05:51:34.518 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.518 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.518 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 56.00 MB 2023-10-26T05:51:34.518 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.518 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.519 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 57.00 MB 2023-10-26T05:51:34.519 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.519 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.519 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 58.00 MB 2023-10-26T05:51:34.519 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.519 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.520 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 59.00 MB 2023-10-26T05:51:34.520 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.520 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.520 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 60.00 MB 2023-10-26T05:51:34.520 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.521 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.521 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 61.00 MB 2023-10-26T05:51:34.521 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.521 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.521 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 62.00 MB 2023-10-26T05:51:34.522 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.522 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.522 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 63.00 MB 2023-10-26T05:51:34.522 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.522 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.523 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 64.00 MB 2023-10-26T05:51:34.523 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.523 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.523 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 65.00 MB 2023-10-26T05:51:34.523 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.524 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.524 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 66.00 MB 2023-10-26T05:51:34.524 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.524 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.524 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 67.00 MB 2023-10-26T05:51:34.524 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.525 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.525 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 68.00 MB 2023-10-26T05:51:34.525 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.525 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.525 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 69.00 MB 2023-10-26T05:51:34.526 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.526 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.526 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 70.00 MB 2023-10-26T05:51:34.526 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.526 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.527 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 71.00 MB 2023-10-26T05:51:34.527 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.527 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.527 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 72.00 MB 2023-10-26T05:51:34.527 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.528 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.528 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 73.00 MB 2023-10-26T05:51:34.528 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.528 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.528 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 74.00 MB 2023-10-26T05:51:34.529 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.529 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.529 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 75.00 MB 2023-10-26T05:51:34.529 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.529 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.530 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 76.00 MB 2023-10-26T05:51:34.530 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.530 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.530 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 77.00 MB 2023-10-26T05:51:34.530 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.530 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.531 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 78.00 MB 2023-10-26T05:51:34.531 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.531 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.531 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 79.00 MB 2023-10-26T05:51:34.532 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:34.532 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:34.532 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 80.00 MB 2023-10-26T05:51:35.272 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.272 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.272 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 81.00 MB 2023-10-26T05:51:35.273 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.273 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.273 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 82.00 MB 2023-10-26T05:51:35.273 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.273 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.274 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 83.00 MB 2023-10-26T05:51:35.274 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.274 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.274 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 84.00 MB 2023-10-26T05:51:35.274 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.275 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.275 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 85.00 MB 2023-10-26T05:51:35.275 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.275 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.275 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 86.00 MB 2023-10-26T05:51:35.276 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.276 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.276 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 87.00 MB 2023-10-26T05:51:35.276 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.276 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.277 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 88.00 MB 2023-10-26T05:51:35.277 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.277 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.277 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 89.00 MB 2023-10-26T05:51:35.277 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.278 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.278 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 90.00 MB 2023-10-26T05:51:35.278 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.278 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.278 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 91.00 MB 2023-10-26T05:51:35.279 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.279 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.279 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 92.00 MB 2023-10-26T05:51:35.279 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.279 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.280 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 93.00 MB 2023-10-26T05:51:35.280 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.280 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.280 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 94.00 MB 2023-10-26T05:51:35.280 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.281 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.281 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 95.00 MB 2023-10-26T05:51:35.281 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.281 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.281 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 96.00 MB 2023-10-26T05:51:35.282 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.282 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.282 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 97.00 MB 2023-10-26T05:51:35.282 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.282 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.283 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 98.00 MB 2023-10-26T05:51:35.283 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.283 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.283 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 99.00 MB 2023-10-26T05:51:35.283 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.284 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.284 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 100.00 MB 2023-10-26T05:51:35.284 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.284 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.284 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 101.00 MB 2023-10-26T05:51:35.284 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.285 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.285 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 102.00 MB 2023-10-26T05:51:35.285 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.285 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.285 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 103.00 MB 2023-10-26T05:51:35.286 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.286 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.286 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 104.00 MB 2023-10-26T05:51:35.286 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.286 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.287 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 105.00 MB 2023-10-26T05:51:35.287 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.287 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.287 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 106.00 MB 2023-10-26T05:51:35.287 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.288 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.288 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 107.00 MB 2023-10-26T05:51:35.288 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.288 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.288 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 108.00 MB 2023-10-26T05:51:35.289 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.289 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.289 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 109.00 MB 2023-10-26T05:51:35.289 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.289 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.290 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 110.00 MB 2023-10-26T05:51:35.290 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.290 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.290 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 111.00 MB 2023-10-26T05:51:35.290 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.291 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.291 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 112.00 MB 2023-10-26T05:51:35.291 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.291 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.291 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 113.00 MB 2023-10-26T05:51:35.292 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.292 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.292 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 114.00 MB 2023-10-26T05:51:35.292 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.292 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.293 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 115.00 MB 2023-10-26T05:51:35.293 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.293 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.293 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 116.00 MB 2023-10-26T05:51:35.293 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.294 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.294 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 117.00 MB 2023-10-26T05:51:35.294 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.294 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.294 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 118.00 MB 2023-10-26T05:51:35.295 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.295 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.295 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 119.00 MB 2023-10-26T05:51:35.295 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.295 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.296 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 120.00 MB 2023-10-26T05:51:35.296 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.296 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.296 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 121.00 MB 2023-10-26T05:51:35.296 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.296 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.297 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 122.00 MB 2023-10-26T05:51:35.297 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.297 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.297 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 123.00 MB 2023-10-26T05:51:35.297 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.298 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.298 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 124.00 MB 2023-10-26T05:51:35.298 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.298 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.298 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 125.00 MB 2023-10-26T05:51:35.299 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.299 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.299 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 126.00 MB 2023-10-26T05:51:35.299 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.299 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.300 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 127.00 MB 2023-10-26T05:51:35.300 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.300 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.300 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 128.00 MB 2023-10-26T05:51:35.300 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.301 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.301 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 129.00 MB 2023-10-26T05:51:35.301 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.301 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.301 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 130.00 MB 2023-10-26T05:51:35.302 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.302 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.302 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 131.00 MB 2023-10-26T05:51:35.302 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.302 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.303 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 132.00 MB 2023-10-26T05:51:35.303 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.303 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.303 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 133.00 MB 2023-10-26T05:51:35.303 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.303 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.304 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 134.00 MB 2023-10-26T05:51:35.304 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.304 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.304 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 135.00 MB 2023-10-26T05:51:35.304 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.305 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.305 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 136.00 MB 2023-10-26T05:51:35.305 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.305 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.305 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 137.00 MB 2023-10-26T05:51:35.306 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.306 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.306 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 138.00 MB 2023-10-26T05:51:35.306 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.306 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.307 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 139.00 MB 2023-10-26T05:51:35.307 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.307 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.307 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 140.00 MB 2023-10-26T05:51:35.307 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.308 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.308 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 141.00 MB 2023-10-26T05:51:35.308 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.308 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.308 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 142.00 MB 2023-10-26T05:51:35.309 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.309 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.309 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 143.00 MB 2023-10-26T05:51:35.309 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.309 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.309 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 144.00 MB 2023-10-26T05:51:35.310 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.310 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.310 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 145.00 MB 2023-10-26T05:51:35.310 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.311 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.311 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 146.00 MB 2023-10-26T05:51:35.311 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.312 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.312 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 147.00 MB 2023-10-26T05:51:35.312 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.312 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.313 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 148.00 MB 2023-10-26T05:51:35.313 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.313 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.313 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 149.00 MB 2023-10-26T05:51:35.313 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.314 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.314 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 150.00 MB 2023-10-26T05:51:35.314 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.314 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.314 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 151.00 MB 2023-10-26T05:51:35.315 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.315 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.315 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 152.00 MB 2023-10-26T05:51:35.315 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.315 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.316 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 153.00 MB 2023-10-26T05:51:35.316 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.316 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.316 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 154.00 MB 2023-10-26T05:51:35.316 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.316 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.317 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 155.00 MB 2023-10-26T05:51:35.317 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.317 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.317 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 156.00 MB 2023-10-26T05:51:35.317 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.318 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.318 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 157.00 MB 2023-10-26T05:51:35.319 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.319 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.319 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 158.00 MB 2023-10-26T05:51:35.319 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.319 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.320 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 159.00 MB 2023-10-26T05:51:35.320 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.320 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.320 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 160.00 MB 2023-10-26T05:51:35.320 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.976 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.977 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 161.00 MB 2023-10-26T05:51:35.977 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.977 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.977 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 162.00 MB 2023-10-26T05:51:35.978 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.978 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.978 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 163.00 MB 2023-10-26T05:51:35.978 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.978 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.979 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 164.00 MB 2023-10-26T05:51:35.979 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.979 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.979 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 165.00 MB 2023-10-26T05:51:35.979 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.980 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.980 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 166.00 MB 2023-10-26T05:51:35.980 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.980 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.980 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 167.00 MB 2023-10-26T05:51:35.981 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.981 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.981 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 168.00 MB 2023-10-26T05:51:35.981 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.981 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.982 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 169.00 MB 2023-10-26T05:51:35.982 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.982 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.982 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 170.00 MB 2023-10-26T05:51:35.982 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.984 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.984 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 171.00 MB 2023-10-26T05:51:35.984 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.984 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.985 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 172.00 MB 2023-10-26T05:51:35.985 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.985 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.985 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 173.00 MB 2023-10-26T05:51:35.985 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.985 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.986 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 174.00 MB 2023-10-26T05:51:35.986 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.986 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.986 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 175.00 MB 2023-10-26T05:51:35.987 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.987 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.987 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 176.00 MB 2023-10-26T05:51:35.987 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.987 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.988 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 177.00 MB 2023-10-26T05:51:35.988 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.988 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.988 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 178.00 MB 2023-10-26T05:51:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 179.00 MB 2023-10-26T05:51:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.989 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 180.00 MB 2023-10-26T05:51:35.990 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.990 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.990 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 181.00 MB 2023-10-26T05:51:35.990 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.990 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.991 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 182.00 MB 2023-10-26T05:51:35.991 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.991 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.991 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 183.00 MB 2023-10-26T05:51:35.991 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.992 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.992 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 184.00 MB 2023-10-26T05:51:35.992 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.992 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.992 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 185.00 MB 2023-10-26T05:51:35.993 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.993 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.993 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 186.00 MB 2023-10-26T05:51:35.993 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.993 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.994 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 187.00 MB 2023-10-26T05:51:35.994 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.994 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.994 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 188.00 MB 2023-10-26T05:51:35.994 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.995 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.995 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 189.00 MB 2023-10-26T05:51:35.995 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.995 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.995 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 190.00 MB 2023-10-26T05:51:35.996 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.996 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.996 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 191.00 MB 2023-10-26T05:51:35.996 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.996 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.996 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 192.00 MB 2023-10-26T05:51:35.997 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.997 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.997 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 193.00 MB 2023-10-26T05:51:35.997 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.997 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.998 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 194.00 MB 2023-10-26T05:51:35.998 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.998 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.998 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 195.00 MB 2023-10-26T05:51:35.998 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.999 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.999 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 196.00 MB 2023-10-26T05:51:35.999 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:35.999 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:35.999 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 197.00 MB 2023-10-26T05:51:36.000 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.000 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.000 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 198.00 MB 2023-10-26T05:51:36.000 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.000 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.001 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 199.00 MB 2023-10-26T05:51:36.001 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.001 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.001 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 200.00 MB 2023-10-26T05:51:36.001 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.002 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.002 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 201.00 MB 2023-10-26T05:51:36.002 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.002 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.002 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 202.00 MB 2023-10-26T05:51:36.003 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.003 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.003 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 203.00 MB 2023-10-26T05:51:36.003 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.003 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.003 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 204.00 MB 2023-10-26T05:51:36.004 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.004 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.004 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 205.00 MB 2023-10-26T05:51:36.004 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.004 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.005 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 206.00 MB 2023-10-26T05:51:36.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.005 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.005 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 207.00 MB 2023-10-26T05:51:36.005 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.006 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.006 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 208.00 MB 2023-10-26T05:51:36.006 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.006 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.006 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 209.00 MB 2023-10-26T05:51:36.007 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.007 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.007 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 210.00 MB 2023-10-26T05:51:36.007 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.007 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.007 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 211.00 MB 2023-10-26T05:51:36.008 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.008 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.008 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 212.00 MB 2023-10-26T05:51:36.008 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.008 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.009 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 213.00 MB 2023-10-26T05:51:36.009 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.009 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.009 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 214.00 MB 2023-10-26T05:51:36.009 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.010 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.010 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 215.00 MB 2023-10-26T05:51:36.010 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.010 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.010 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 216.00 MB 2023-10-26T05:51:36.011 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.011 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.011 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 217.00 MB 2023-10-26T05:51:36.011 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.011 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.011 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 218.00 MB 2023-10-26T05:51:36.012 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.012 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.012 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 219.00 MB 2023-10-26T05:51:36.012 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 220.00 MB 2023-10-26T05:51:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.013 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 221.00 MB 2023-10-26T05:51:36.014 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.014 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.014 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 222.00 MB 2023-10-26T05:51:36.014 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.014 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.015 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 223.00 MB 2023-10-26T05:51:36.015 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.015 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.015 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 224.00 MB 2023-10-26T05:51:36.015 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.015 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.016 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 225.00 MB 2023-10-26T05:51:36.016 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.016 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.016 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 226.00 MB 2023-10-26T05:51:36.016 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.017 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.017 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 227.00 MB 2023-10-26T05:51:36.017 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.017 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.017 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 228.00 MB 2023-10-26T05:51:36.018 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.018 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.018 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 229.00 MB 2023-10-26T05:51:36.018 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.019 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.019 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 230.00 MB 2023-10-26T05:51:36.019 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.019 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.019 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 231.00 MB 2023-10-26T05:51:36.020 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.020 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.020 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 232.00 MB 2023-10-26T05:51:36.020 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.020 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.021 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 233.00 MB 2023-10-26T05:51:36.021 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.021 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.021 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 234.00 MB 2023-10-26T05:51:36.021 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.021 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.022 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 235.00 MB 2023-10-26T05:51:36.022 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.022 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.022 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 236.00 MB 2023-10-26T05:51:36.022 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.023 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.023 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 237.00 MB 2023-10-26T05:51:36.023 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.023 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.023 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 238.00 MB 2023-10-26T05:51:36.024 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.024 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.024 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 239.00 MB 2023-10-26T05:51:36.024 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.024 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.024 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 240.00 MB 2023-10-26T05:51:36.025 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.025 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.654 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 241.00 MB 2023-10-26T05:51:36.654 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.655 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.655 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 242.00 MB 2023-10-26T05:51:36.655 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.655 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.655 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 243.00 MB 2023-10-26T05:51:36.656 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.656 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.656 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 244.00 MB 2023-10-26T05:51:36.656 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.656 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.656 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 245.00 MB 2023-10-26T05:51:36.657 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.657 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.657 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 246.00 MB 2023-10-26T05:51:36.657 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.657 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.658 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 247.00 MB 2023-10-26T05:51:36.658 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.658 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.658 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 248.00 MB 2023-10-26T05:51:36.658 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.659 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.659 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 249.00 MB 2023-10-26T05:51:36.659 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.659 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.659 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 250.00 MB 2023-10-26T05:51:36.659 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.660 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.660 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 251.00 MB 2023-10-26T05:51:36.660 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.660 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.660 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 252.00 MB 2023-10-26T05:51:36.661 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.661 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.661 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 253.00 MB 2023-10-26T05:51:36.661 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.661 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.662 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 254.00 MB 2023-10-26T05:51:36.662 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.662 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.662 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 255.00 MB 2023-10-26T05:51:36.662 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.663 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.663 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 256.00 MB 2023-10-26T05:51:36.663 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.663 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.664 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 257.00 MB 2023-10-26T05:51:36.664 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.664 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.664 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 258.00 MB 2023-10-26T05:51:36.664 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.665 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.665 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 259.00 MB 2023-10-26T05:51:36.665 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.665 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.665 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 260.00 MB 2023-10-26T05:51:36.666 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.666 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.666 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 261.00 MB 2023-10-26T05:51:36.666 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.666 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.667 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 262.00 MB 2023-10-26T05:51:36.667 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.667 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.667 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 263.00 MB 2023-10-26T05:51:36.668 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.668 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.668 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 264.00 MB 2023-10-26T05:51:36.668 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.668 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.669 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 265.00 MB 2023-10-26T05:51:36.669 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.669 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.669 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 266.00 MB 2023-10-26T05:51:36.669 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.670 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.670 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 267.00 MB 2023-10-26T05:51:36.670 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 268.00 MB 2023-10-26T05:51:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 269.00 MB 2023-10-26T05:51:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.672 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.672 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 270.00 MB 2023-10-26T05:51:36.672 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.672 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.672 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 271.00 MB 2023-10-26T05:51:36.673 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.673 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.673 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 272.00 MB 2023-10-26T05:51:36.673 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.673 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.674 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 273.00 MB 2023-10-26T05:51:36.674 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.674 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.675 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 274.00 MB 2023-10-26T05:51:36.675 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.675 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.675 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 275.00 MB 2023-10-26T05:51:36.675 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.676 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.676 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 276.00 MB 2023-10-26T05:51:36.676 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.676 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.676 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 277.00 MB 2023-10-26T05:51:36.676 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.677 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.677 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 278.00 MB 2023-10-26T05:51:36.677 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.677 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.677 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 279.00 MB 2023-10-26T05:51:36.678 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.678 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.678 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 280.00 MB 2023-10-26T05:51:36.678 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.678 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.679 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 281.00 MB 2023-10-26T05:51:36.679 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.679 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.679 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 282.00 MB 2023-10-26T05:51:36.679 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.679 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.680 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 283.00 MB 2023-10-26T05:51:36.680 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.680 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.680 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 284.00 MB 2023-10-26T05:51:36.680 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.681 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.681 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 285.00 MB 2023-10-26T05:51:36.681 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.681 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.681 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 286.00 MB 2023-10-26T05:51:36.682 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.682 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.682 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 287.00 MB 2023-10-26T05:51:36.682 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.682 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.682 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 288.00 MB 2023-10-26T05:51:36.683 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.683 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.683 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 289.00 MB 2023-10-26T05:51:36.683 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.683 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.684 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 290.00 MB 2023-10-26T05:51:36.684 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.684 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.685 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 291.00 MB 2023-10-26T05:51:36.685 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.685 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.685 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 292.00 MB 2023-10-26T05:51:36.685 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.686 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.686 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 293.00 MB 2023-10-26T05:51:36.686 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.686 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.686 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 294.00 MB 2023-10-26T05:51:36.687 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.687 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.687 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 295.00 MB 2023-10-26T05:51:36.687 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.687 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 296.00 MB 2023-10-26T05:51:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 297.00 MB 2023-10-26T05:51:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.689 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.689 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 298.00 MB 2023-10-26T05:51:36.689 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.689 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.689 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 299.00 MB 2023-10-26T05:51:36.689 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.690 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.690 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 300.00 MB 2023-10-26T05:51:36.690 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.690 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 301.00 MB 2023-10-26T05:51:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 302.00 MB 2023-10-26T05:51:36.691 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.692 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.692 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 303.00 MB 2023-10-26T05:51:36.692 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.692 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.692 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 304.00 MB 2023-10-26T05:51:36.692 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.693 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.693 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 305.00 MB 2023-10-26T05:51:36.693 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.693 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.693 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 306.00 MB 2023-10-26T05:51:36.694 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.694 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.694 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 307.00 MB 2023-10-26T05:51:36.694 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.694 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.695 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 308.00 MB 2023-10-26T05:51:36.695 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.695 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.695 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 309.00 MB 2023-10-26T05:51:36.696 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.696 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.696 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 310.00 MB 2023-10-26T05:51:36.696 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.696 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.697 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 311.00 MB 2023-10-26T05:51:36.697 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.697 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.697 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 312.00 MB 2023-10-26T05:51:36.697 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.697 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.698 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 313.00 MB 2023-10-26T05:51:36.698 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.698 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.698 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 314.00 MB 2023-10-26T05:51:36.698 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.699 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.699 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 315.00 MB 2023-10-26T05:51:36.699 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.699 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.699 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 316.00 MB 2023-10-26T05:51:36.700 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.700 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.700 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 317.00 MB 2023-10-26T05:51:36.700 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.700 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.701 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 318.00 MB 2023-10-26T05:51:36.701 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.701 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.701 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 319.00 MB 2023-10-26T05:51:36.701 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.701 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.702 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 320.00 MB 2023-10-26T05:51:36.702 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:36.702 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:36.702 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 321.00 MB 2023-10-26T05:51:37.343 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.343 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.344 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 322.00 MB 2023-10-26T05:51:37.344 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.344 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.344 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 323.00 MB 2023-10-26T05:51:37.344 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.344 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.345 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 324.00 MB 2023-10-26T05:51:37.345 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.345 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.345 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 325.00 MB 2023-10-26T05:51:37.345 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.346 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.346 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 326.00 MB 2023-10-26T05:51:37.346 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.346 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.346 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 327.00 MB 2023-10-26T05:51:37.347 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.347 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.347 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 328.00 MB 2023-10-26T05:51:37.347 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.347 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.348 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 329.00 MB 2023-10-26T05:51:37.348 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.348 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.348 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 330.00 MB 2023-10-26T05:51:37.348 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.349 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.349 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 331.00 MB 2023-10-26T05:51:37.349 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.349 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 332.00 MB 2023-10-26T05:51:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 333.00 MB 2023-10-26T05:51:37.350 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 334.00 MB 2023-10-26T05:51:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 335.00 MB 2023-10-26T05:51:37.352 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.352 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.352 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 336.00 MB 2023-10-26T05:51:37.352 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.352 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.353 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 337.00 MB 2023-10-26T05:51:37.353 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.353 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.353 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 338.00 MB 2023-10-26T05:51:37.353 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.354 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.354 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 339.00 MB 2023-10-26T05:51:37.354 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.354 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.354 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 340.00 MB 2023-10-26T05:51:37.354 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.355 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.355 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 341.00 MB 2023-10-26T05:51:37.355 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.355 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.355 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 342.00 MB 2023-10-26T05:51:37.356 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.356 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.356 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 343.00 MB 2023-10-26T05:51:37.356 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.356 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.357 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 344.00 MB 2023-10-26T05:51:37.357 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.357 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.357 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 345.00 MB 2023-10-26T05:51:37.357 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.358 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.358 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 346.00 MB 2023-10-26T05:51:37.358 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.358 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.358 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 347.00 MB 2023-10-26T05:51:37.359 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.359 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.359 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 348.00 MB 2023-10-26T05:51:37.359 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.359 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.360 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 349.00 MB 2023-10-26T05:51:37.360 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.360 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.360 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 350.00 MB 2023-10-26T05:51:37.360 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.361 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.361 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 351.00 MB 2023-10-26T05:51:37.361 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.361 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.361 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 352.00 MB 2023-10-26T05:51:37.362 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.362 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.362 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 353.00 MB 2023-10-26T05:51:37.362 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.362 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.363 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 354.00 MB 2023-10-26T05:51:37.363 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.363 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.363 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 355.00 MB 2023-10-26T05:51:37.363 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.363 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.364 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 356.00 MB 2023-10-26T05:51:37.364 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.364 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.364 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 357.00 MB 2023-10-26T05:51:37.364 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.365 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.365 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 358.00 MB 2023-10-26T05:51:37.365 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.365 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.365 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 359.00 MB 2023-10-26T05:51:37.366 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.366 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.366 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 360.00 MB 2023-10-26T05:51:37.366 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.366 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.366 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 361.00 MB 2023-10-26T05:51:37.367 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.367 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.367 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 362.00 MB 2023-10-26T05:51:37.367 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.367 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.368 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 363.00 MB 2023-10-26T05:51:37.368 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.368 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.368 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 364.00 MB 2023-10-26T05:51:37.368 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.369 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.369 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 365.00 MB 2023-10-26T05:51:37.369 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.369 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.369 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 366.00 MB 2023-10-26T05:51:37.370 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.370 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.370 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 367.00 MB 2023-10-26T05:51:37.370 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.371 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.371 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 368.00 MB 2023-10-26T05:51:37.371 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.371 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.372 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 369.00 MB 2023-10-26T05:51:37.372 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.372 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.372 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 370.00 MB 2023-10-26T05:51:37.373 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.373 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.373 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 371.00 MB 2023-10-26T05:51:37.374 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.374 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.374 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 372.00 MB 2023-10-26T05:51:37.374 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.375 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.375 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 373.00 MB 2023-10-26T05:51:37.375 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.375 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.375 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 374.00 MB 2023-10-26T05:51:37.375 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.376 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.376 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 375.00 MB 2023-10-26T05:51:37.376 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.376 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.376 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 376.00 MB 2023-10-26T05:51:37.376 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.377 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.377 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 377.00 MB 2023-10-26T05:51:37.377 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.377 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.377 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 378.00 MB 2023-10-26T05:51:37.378 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.378 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.378 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 379.00 MB 2023-10-26T05:51:37.378 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.378 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.379 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 380.00 MB 2023-10-26T05:51:37.379 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.379 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.379 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 381.00 MB 2023-10-26T05:51:37.379 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.379 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.380 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 382.00 MB 2023-10-26T05:51:37.380 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.380 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.380 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 383.00 MB 2023-10-26T05:51:37.380 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.381 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.381 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 384.00 MB 2023-10-26T05:51:37.381 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.381 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.381 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 385.00 MB 2023-10-26T05:51:37.382 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.382 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.382 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 386.00 MB 2023-10-26T05:51:37.382 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.382 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.383 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 387.00 MB 2023-10-26T05:51:37.383 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.383 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.383 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 388.00 MB 2023-10-26T05:51:37.383 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.385 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.385 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 389.00 MB 2023-10-26T05:51:37.385 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.385 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.386 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 390.00 MB 2023-10-26T05:51:37.386 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.386 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.386 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 391.00 MB 2023-10-26T05:51:37.386 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.387 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.387 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 392.00 MB 2023-10-26T05:51:37.387 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.387 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.387 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 393.00 MB 2023-10-26T05:51:37.388 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.388 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.388 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 394.00 MB 2023-10-26T05:51:37.388 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.388 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.388 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 395.00 MB 2023-10-26T05:51:37.389 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.389 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.389 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 396.00 MB 2023-10-26T05:51:37.389 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.389 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 397.00 MB 2023-10-26T05:51:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 398.00 MB 2023-10-26T05:51:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.391 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.391 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 399.00 MB 2023-10-26T05:51:37.391 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.391 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.391 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 400.00 MB 2023-10-26T05:51:37.392 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:37.392 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:37.392 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 401.00 MB 2023-10-26T05:51:41.467 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:51:41.460+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check failed: 3 full osd(s) (OSD_FULL) 2023-10-26T05:51:45.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:51:45.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:51:47.124 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:51:47.120+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T05:51:53.134 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.134 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.135 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 402.00 MB 2023-10-26T05:51:53.135 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.135 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.135 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 403.00 MB 2023-10-26T05:51:53.135 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.136 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.136 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 404.00 MB 2023-10-26T05:51:53.137 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.137 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.137 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 405.00 MB 2023-10-26T05:51:53.137 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.138 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.138 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 406.00 MB 2023-10-26T05:51:53.138 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.138 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.139 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 407.00 MB 2023-10-26T05:51:53.139 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.139 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.139 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 408.00 MB 2023-10-26T05:51:53.139 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.140 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.140 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 409.00 MB 2023-10-26T05:51:53.140 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.140 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.140 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 410.00 MB 2023-10-26T05:51:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 411.00 MB 2023-10-26T05:51:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 412.00 MB 2023-10-26T05:51:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 413.00 MB 2023-10-26T05:51:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 414.00 MB 2023-10-26T05:51:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 415.00 MB 2023-10-26T05:51:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 416.00 MB 2023-10-26T05:51:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 417.00 MB 2023-10-26T05:51:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 418.00 MB 2023-10-26T05:51:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 419.00 MB 2023-10-26T05:51:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 420.00 MB 2023-10-26T05:51:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 421.00 MB 2023-10-26T05:51:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 422.00 MB 2023-10-26T05:51:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.148 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.148 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 423.00 MB 2023-10-26T05:51:53.148 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.148 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.149 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 424.00 MB 2023-10-26T05:51:53.149 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.149 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.149 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 425.00 MB 2023-10-26T05:51:53.149 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.150 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.150 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 426.00 MB 2023-10-26T05:51:53.150 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.150 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.151 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 427.00 MB 2023-10-26T05:51:53.151 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.151 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.152 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 428.00 MB 2023-10-26T05:51:53.152 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.152 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.152 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 429.00 MB 2023-10-26T05:51:53.153 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.153 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.153 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 430.00 MB 2023-10-26T05:51:53.153 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.153 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.154 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 431.00 MB 2023-10-26T05:51:53.154 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.154 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.154 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 432.00 MB 2023-10-26T05:51:53.154 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.155 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.155 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 433.00 MB 2023-10-26T05:51:53.155 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.155 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.155 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 434.00 MB 2023-10-26T05:51:53.156 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.156 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.156 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 435.00 MB 2023-10-26T05:51:53.156 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.156 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.157 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 436.00 MB 2023-10-26T05:51:53.157 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.157 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.157 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 437.00 MB 2023-10-26T05:51:53.157 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.157 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.158 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 438.00 MB 2023-10-26T05:51:53.158 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.158 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.158 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 439.00 MB 2023-10-26T05:51:53.158 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.159 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.159 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 440.00 MB 2023-10-26T05:51:53.159 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.159 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.160 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 441.00 MB 2023-10-26T05:51:53.160 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.160 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.160 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 442.00 MB 2023-10-26T05:51:53.160 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 443.00 MB 2023-10-26T05:51:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.161 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 444.00 MB 2023-10-26T05:51:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 445.00 MB 2023-10-26T05:51:53.162 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 446.00 MB 2023-10-26T05:51:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 447.00 MB 2023-10-26T05:51:53.163 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 448.00 MB 2023-10-26T05:51:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 449.00 MB 2023-10-26T05:51:53.164 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.165 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.165 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 450.00 MB 2023-10-26T05:51:53.165 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.165 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.165 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 451.00 MB 2023-10-26T05:51:53.166 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.166 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.166 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 452.00 MB 2023-10-26T05:51:53.167 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.167 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.167 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 453.00 MB 2023-10-26T05:51:53.167 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.168 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.168 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 454.00 MB 2023-10-26T05:51:53.168 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.169 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.169 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 455.00 MB 2023-10-26T05:51:53.169 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.169 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.170 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 456.00 MB 2023-10-26T05:51:53.170 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.171 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.171 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 457.00 MB 2023-10-26T05:51:53.173 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.174 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.174 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 458.00 MB 2023-10-26T05:51:53.174 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.175 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.175 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 459.00 MB 2023-10-26T05:51:53.175 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.176 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.176 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 460.00 MB 2023-10-26T05:51:53.176 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.177 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.177 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 461.00 MB 2023-10-26T05:51:53.177 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.178 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.178 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 462.00 MB 2023-10-26T05:51:53.178 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.178 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.179 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 463.00 MB 2023-10-26T05:51:53.179 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.179 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.179 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 464.00 MB 2023-10-26T05:51:53.180 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.180 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.180 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 465.00 MB 2023-10-26T05:51:53.180 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.180 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.180 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 466.00 MB 2023-10-26T05:51:53.181 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.181 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.181 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 467.00 MB 2023-10-26T05:51:53.181 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.181 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.182 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 468.00 MB 2023-10-26T05:51:53.182 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.182 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.182 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 469.00 MB 2023-10-26T05:51:53.182 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.183 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.183 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 470.00 MB 2023-10-26T05:51:53.183 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.183 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.184 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 471.00 MB 2023-10-26T05:51:53.184 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.184 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.184 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 472.00 MB 2023-10-26T05:51:53.184 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.184 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.185 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 473.00 MB 2023-10-26T05:51:53.185 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.185 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.185 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 474.00 MB 2023-10-26T05:51:53.185 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.186 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.186 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 475.00 MB 2023-10-26T05:51:53.186 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.186 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.186 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 476.00 MB 2023-10-26T05:51:53.187 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.187 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.187 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 477.00 MB 2023-10-26T05:51:53.187 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.187 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.188 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 478.00 MB 2023-10-26T05:51:53.188 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.188 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:51:53.188 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 479.00 MB 2023-10-26T05:51:53.188 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:51:53.189 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:51:53.189 INFO:teuthology.orchestra.run.smithi071.stdout:Reached fullness after 480.00 MB 2023-10-26T05:51:53.189 INFO:teuthology.orchestra.run.smithi071.stdout:calling close 2023-10-26T05:51:53.189 INFO:teuthology.orchestra.run.smithi071.stdout:close() did not raise error 2023-10-26T05:51:53.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:53.191 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:51:53.192 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:51:53.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:51:53.296 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:51:53.296 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:51:53.297 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:53.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- 172.21.15.71:0/1935810369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000039b0 msgr2=0x7f8600003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 --2- 172.21.15.71:0/1935810369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86000039b0 0x7f8600003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f861000c290 tx=0x7f861001cfc0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a93b640 1 -- 172.21.15.71:0/1935810369 reap_dead start 2023-10-26T05:51:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- 172.21.15.71:0/1935810369 shutdown_connections 2023-10-26T05:51:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- 172.21.15.71:0/1935810369 >> 172.21.15.71:0/1935810369 conn(0x7f8614069770 msgr2=0x7f8614069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- 172.21.15.71:0/1935810369 shutdown_connections 2023-10-26T05:51:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- 172.21.15.71:0/1935810369 wait complete. 2023-10-26T05:51:53.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 Processor -- start 2023-10-26T05:51:53.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- start start 2023-10-26T05:51:53.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 0x7f86140b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86141379b0 0x7f8614164310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8614164850 0x7f8614069b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8614163ef0 con 0x7f86141373c0 2023-10-26T05:51:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86141377a0 con 0x7f8614164850 2023-10-26T05:51:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861c39e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8614163270 con 0x7f86141379b0 2023-10-26T05:51:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 0x7f86140b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 0x7f86140b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57596/0 (socket says 172.21.15.71:57596) 2023-10-26T05:51:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 -- 172.21.15.71:0/816961685 learned_addr learned my addr 172.21.15.71:0/816961685 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 -- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86141379b0 msgr2=0x7f8614164310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86141379b0 0x7f8614164310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 -- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8614164850 msgr2=0x7f8614069b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a93b640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8614164850 0x7f8614069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8614164850 0x7f8614069b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 -- 172.21.15.71:0/816961685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8614163550 con 0x7f86141373c0 2023-10-26T05:51:53.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.472+0000 7f861a13a640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 0x7f86140b0ae0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f861001cdf0 tx=0x7f8610000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:53.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86100b3070 con 0x7f86141373c0 2023-10-26T05:51:53.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8610001680 con 0x7f86141373c0 2023-10-26T05:51:53.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86100a5e10 con 0x7f86141373c0 2023-10-26T05:51:53.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8614163830 con 0x7f86141373c0 2023-10-26T05:51:53.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86140b1020 con 0x7f86141373c0 2023-10-26T05:51:53.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8614164160 con 0x7f86141373c0 2023-10-26T05:51:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86100ab020 con 0x7f86141373c0 2023-10-26T05:51:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f8066470 0x7f85f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f861011da70 con 0x7f86141373c0 2023-10-26T05:51:53.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f85f806dfa0 con 0x7f86141373c0 2023-10-26T05:51:53.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.476+0000 7f8619939640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f8066470 0x7f85f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.480+0000 7f8619939640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f8066470 0x7f85f8068930 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8604005bb0 tx=0x7f8604005aa0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:53.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.480+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86100eb060 con 0x7f86141373c0 2023-10-26T05:51:53.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.636+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8614066b10 con 0x7f86141373c0 2023-10-26T05:51:53.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f860affd640 1 -- 172.21.15.71:0/816961685 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16527 (secure 0 0 0) 0x7f86100bd020 con 0x7f86141373c0 2023-10-26T05:51:53.646 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:53.646 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":88,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:43.124800+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f8066470 msgr2=0x7f85f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f8066470 0x7f85f8068930 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8604005bb0 tx=0x7f8604005aa0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 msgr2=0x7f86140b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 0x7f86140b0ae0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f861001cdf0 tx=0x7f8610000fb0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 shutdown_connections 2023-10-26T05:51:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8614164850 0x7f8614069b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86141379b0 0x7f8614164310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f8066470 0x7f85f8068930 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 --2- 172.21.15.71:0/816961685 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86141373c0 0x7f86140b0ae0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 >> 172.21.15.71:0/816961685 conn(0x7f8614069770 msgr2=0x7f86140ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 shutdown_connections 2023-10-26T05:51:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.640+0000 7f861c39e640 1 -- 172.21.15.71:0/816961685 wait complete. 2023-10-26T05:51:53.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:51:15.888674+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:51:16.655596+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:53.666 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:53.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- 172.21.15.71:0/501324240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 msgr2=0x7fa20c15bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 --2- 172.21.15.71:0/501324240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c15bab0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa20c08a140 tx=0x7fa20401cc60 comp rx=0 tx=0).stop 2023-10-26T05:51:53.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- 172.21.15.71:0/501324240 shutdown_connections 2023-10-26T05:51:53.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 --2- 172.21.15.71:0/501324240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c15bab0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 --2- 172.21.15.71:0/501324240 >> v2:172.21.15.86:3300/0 conn(0x7fa20c0d0470 0x7fa20c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- 172.21.15.71:0/501324240 >> 172.21.15.71:0/501324240 conn(0x7fa20c0cdd80 msgr2=0x7fa20c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:53.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- 172.21.15.71:0/501324240 shutdown_connections 2023-10-26T05:51:53.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- 172.21.15.71:0/501324240 wait complete. 2023-10-26T05:51:53.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 Processor -- start 2023-10-26T05:51:53.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- start start 2023-10-26T05:51:53.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa20c0d0470 0x7fa20c15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20c0d0850 0x7fa20c0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15da70 con 0x7fa20c0cd880 2023-10-26T05:51:53.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15cdf0 con 0x7fa20c0d0850 2023-10-26T05:51:53.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa210a23640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c15d0d0 con 0x7fa20c0d0470 2023-10-26T05:51:53.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57618/0 (socket says 172.21.15.71:57618) 2023-10-26T05:51:53.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.788+0000 7fa20a59c640 1 -- 172.21.15.71:0/861241322 learned_addr learned my addr 172.21.15.71:0/861241322 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:53.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa209d9b640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa20c0d0470 0x7fa20c15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20ad9d640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20c0d0850 0x7fa20c0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20a59c640 1 -- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa20c0d0470 msgr2=0x7fa20c15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20a59c640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa20c0d0470 0x7fa20c15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20a59c640 1 -- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20c0d0850 msgr2=0x7fa20c0cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20a59c640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20c0d0850 0x7fa20c0cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20a59c640 1 -- 172.21.15.71:0/861241322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2040098e0 con 0x7fa20c0cd880 2023-10-26T05:51:53.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20ad9d640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20c0d0850 0x7fa20c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:53.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa20a59c640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c150f60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b550 tx=0x7fa1fc00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:53.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc010870 con 0x7fa20c0cd880 2023-10-26T05:51:53.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1fc010ae0 con 0x7fa20c0cd880 2023-10-26T05:51:53.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00f3c0 con 0x7fa20c0cd880 2023-10-26T05:51:53.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c15d3b0 con 0x7fa20c0cd880 2023-10-26T05:51:53.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c15de90 con 0x7fa20c0cd880 2023-10-26T05:51:53.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c066b10 con 0x7fa20c0cd880 2023-10-26T05:51:53.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa1fc00f520 con 0x7fa20c0cd880 2023-10-26T05:51:53.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1ec066470 0x7fa1ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:53.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1fc0840b0 con 0x7fa20c0cd880 2023-10-26T05:51:53.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fa1ec06dfa0 con 0x7fa20c0cd880 2023-10-26T05:51:53.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa209d9b640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1ec066470 0x7fa1ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:53.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.792+0000 7fa209d9b640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1ec066470 0x7fa1ec068930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa20c1f8030 tx=0x7fa200008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:53.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.796+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa1fc01e060 con 0x7fa20c0cd880 2023-10-26T05:51:53.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa20c0721e0 con 0x7fa20c0cd880 2023-10-26T05:51:53.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa1faffd640 1 -- 172.21.15.71:0/861241322 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1fc014020 con 0x7fa20c0cd880 2023-10-26T05:51:53.958 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:53.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1ec066470 msgr2=0x7fa1ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1ec066470 0x7fa1ec068930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa20c1f8030 tx=0x7fa200008040 comp rx=0 tx=0).stop 2023-10-26T05:51:53.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 msgr2=0x7fa20c150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:53.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c150f60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b550 tx=0x7fa1fc00ba10 comp rx=0 tx=0).stop 2023-10-26T05:51:53.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 shutdown_connections 2023-10-26T05:51:53.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20c0d0850 0x7fa20c0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa20c0d0470 0x7fa20c15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1ec066470 0x7fa1ec068930 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 --2- 172.21.15.71:0/861241322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c0cd880 0x7fa20c150f60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 >> 172.21.15.71:0/861241322 conn(0x7fa20c0cdd80 msgr2=0x7fa20c154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 shutdown_connections 2023-10-26T05:51:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:53.952+0000 7fa210a23640 1 -- 172.21.15.71:0/861241322 wait complete. 2023-10-26T05:51:53.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:51:54.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 -- 172.21.15.71:0/2214185838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 msgr2=0x7fb3ec15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 --2- 172.21.15.71:0/2214185838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec15b3a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec08a140 tx=0x7fb3e001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 -- 172.21.15.71:0/2214185838 shutdown_connections 2023-10-26T05:51:54.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 --2- 172.21.15.71:0/2214185838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec15b3a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 --2- 172.21.15.71:0/2214185838 >> v2:172.21.15.86:3300/0 conn(0x7fb3ec0d0470 0x7fb3ec06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 -- 172.21.15.71:0/2214185838 >> 172.21.15.71:0/2214185838 conn(0x7fb3ec0cdd80 msgr2=0x7fb3ec0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:54.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 -- 172.21.15.71:0/2214185838 shutdown_connections 2023-10-26T05:51:54.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.096+0000 7fb3f2850640 1 -- 172.21.15.71:0/2214185838 wait complete. 2023-10-26T05:51:54.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 Processor -- start 2023-10-26T05:51:54.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 -- start start 2023-10-26T05:51:54.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3ec0cd880 0x7fb3ec0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ec0d0470 0x7fb3ec159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec159b40 con 0x7fb3ec0d0850 2023-10-26T05:51:54.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec158f30 con 0x7fb3ec0cd880 2023-10-26T05:51:54.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec159210 con 0x7fb3ec0d0470 2023-10-26T05:51:54.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57636/0 (socket says 172.21.15.71:57636) 2023-10-26T05:51:54.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 -- 172.21.15.71:0/499144767 learned_addr learned my addr 172.21.15.71:0/499144767 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:54.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3eb7fe640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ec0d0470 0x7fb3ec159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3ebfff640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3ec0cd880 0x7fb3ec0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 -- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ec0d0470 msgr2=0x7fb3ec159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ec0d0470 0x7fb3ec159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 -- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3ec0cd880 msgr2=0x7fb3ec0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3ec0cd880 0x7fb3ec0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 -- 172.21.15.71:0/499144767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0009560 con 0x7fb3ec0d0850 2023-10-26T05:51:54.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f0ded640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec0c4e30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c3c0 tx=0x7fb3e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4011790 con 0x7fb3ec0d0850 2023-10-26T05:51:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb3e40119e0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ec1594f0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4011df0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.100+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ec15a510 con 0x7fb3ec0d0850 2023-10-26T05:51:54.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ec066b10 con 0x7fb3ec0d0850 2023-10-26T05:51:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb3e401c460 con 0x7fb3ec0d0850 2023-10-26T05:51:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3e8ff9640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb3d80664a0 0x7fb3d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3e4083fb0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fb3d806dfd0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3ebfff640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb3d80664a0 0x7fb3d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.104+0000 7fb3ebfff640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb3d80664a0 0x7fb3d8068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc005c50 tx=0x7fb3dc008230 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:54.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.108+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb3e40b5090 con 0x7fb3ec0d0850 2023-10-26T05:51:54.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.264+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3ec0721e0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.264+0000 7fb3e8ff9640 1 -- 172.21.15.71:0/499144767 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3e40513a0 con 0x7fb3ec0d0850 2023-10-26T05:51:54.272 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:51:54.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb3d80664a0 msgr2=0x7fb3d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb3d80664a0 0x7fb3d8068960 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc005c50 tx=0x7fb3dc008230 comp rx=0 tx=0).stop 2023-10-26T05:51:54.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 msgr2=0x7fb3ec0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec0c4e30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb3e400c3c0 tx=0x7fb3e400c880 comp rx=0 tx=0).stop 2023-10-26T05:51:54.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 shutdown_connections 2023-10-26T05:51:54.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb3d80664a0 0x7fb3d8068960 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3ec0d0850 0x7fb3ec0c4e30 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3ec0d0470 0x7fb3ec159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 --2- 172.21.15.71:0/499144767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3ec0cd880 0x7fb3ec0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 >> 172.21.15.71:0/499144767 conn(0x7fb3ec0cdd80 msgr2=0x7fb3ec0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:54.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 shutdown_connections 2023-10-26T05:51:54.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.268+0000 7fb3f2850640 1 -- 172.21.15.71:0/499144767 wait complete. 2023-10-26T05:51:54.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:51:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/169983091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 msgr2=0x7f99e4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 --2- 172.21.15.71:0/169983091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99e4004c50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009b10 tx=0x7f99e001d100 comp rx=0 tx=0).stop 2023-10-26T05:51:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f77fe640 1 -- 172.21.15.71:0/169983091 reap_dead start 2023-10-26T05:51:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/169983091 shutdown_connections 2023-10-26T05:51:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/169983091 >> 172.21.15.71:0/169983091 conn(0x7f99f80b0f20 msgr2=0x7f99f80b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:54.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/169983091 shutdown_connections 2023-10-26T05:51:54.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/169983091 wait complete. 2023-10-26T05:51:54.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 Processor -- start 2023-10-26T05:51:54.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- start start 2023-10-26T05:51:54.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99f80c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f81532c0 0x7f99f814bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f80c63c0 0x7f99f81520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80c8be0 con 0x7f99e40047d0 2023-10-26T05:51:54.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f8151600 con 0x7f99f80c63c0 2023-10-26T05:51:54.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f80c8620 con 0x7f99f81532c0 2023-10-26T05:51:54.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99f80c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99f80c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57650/0 (socket says 172.21.15.71:57650) 2023-10-26T05:51:54.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 -- 172.21.15.71:0/3097347350 learned_addr learned my addr 172.21.15.71:0/3097347350 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:54.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f77fe640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f80c63c0 0x7f99f81520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99effff640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f81532c0 0x7f99f814bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 -- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f81532c0 msgr2=0x7f99f814bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f81532c0 0x7f99f814bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 -- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f80c63c0 msgr2=0x7f99f81520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f80c63c0 0x7f99f81520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 -- 172.21.15.71:0/3097347350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f80c8900 con 0x7f99e40047d0 2023-10-26T05:51:54.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f77fe640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f80c63c0 0x7f99f81520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:54.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99f6ffd640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99f80c7f90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f99e800d7f0 tx=0x7f99e800dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:54.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80052e0 con 0x7f99e40047d0 2023-10-26T05:51:54.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99e8005440 con 0x7f99e40047d0 2023-10-26T05:51:54.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800b900 con 0x7f99e40047d0 2023-10-26T05:51:54.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f81518a0 con 0x7f99e40047d0 2023-10-26T05:51:54.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.412+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f8153740 con 0x7f99e40047d0 2023-10-26T05:51:54.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f8151dc0 con 0x7f99e40047d0 2023-10-26T05:51:54.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f99e800ba60 con 0x7f99e40047d0 2023-10-26T05:51:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99ef7fe640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99d8066470 0x7f99d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99e8083800 con 0x7f99e40047d0 2023-10-26T05:51:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99effff640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99d8066470 0x7f99d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f99d806dfa0 con 0x7f99e40047d0 2023-10-26T05:51:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.416+0000 7f99effff640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99d8066470 0x7f99d8068930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ad000 tx=0x7f99e0019ea0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:54.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.420+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f99f8151dc0 con 0x7f99e40047d0 2023-10-26T05:51:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.572+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99f8151dc0 con 0x7f99e40047d0 2023-10-26T05:51:54.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.572+0000 7f99ef7fe640 1 -- 172.21.15.71:0/3097347350 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99f8151dc0 con 0x7f99e40047d0 2023-10-26T05:51:54.580 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:51:54.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99d8066470 msgr2=0x7f99d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99d8066470 0x7f99d8068930 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f99e00ad000 tx=0x7f99e0019ea0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 msgr2=0x7f99f80c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99f80c7f90 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f99e800d7f0 tx=0x7f99e800dcb0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 shutdown_connections 2023-10-26T05:51:54.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f80c63c0 0x7f99f81520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f81532c0 0x7f99f814bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99d8066470 0x7f99d8068930 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 --2- 172.21.15.71:0/3097347350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99e40047d0 0x7f99f80c7f90 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 >> 172.21.15.71:0/3097347350 conn(0x7f99f80b0f20 msgr2=0x7f99f80ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:54.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 shutdown_connections 2023-10-26T05:51:54.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.576+0000 7f99fd522640 1 -- 172.21.15.71:0/3097347350 wait complete. 2023-10-26T05:51:54.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:51:54.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 -- 172.21.15.71:0/2464742855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ac4004d10 msgr2=0x7f9ac4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 --2- 172.21.15.71:0/2464742855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ac4004d10 0x7f9ac4005190 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009ee0 tx=0x7f9ad001d5d0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9adeaf7640 1 -- 172.21.15.71:0/2464742855 reap_dead start 2023-10-26T05:51:54.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 -- 172.21.15.71:0/2464742855 shutdown_connections 2023-10-26T05:51:54.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 -- 172.21.15.71:0/2464742855 >> 172.21.15.71:0/2464742855 conn(0x7f9ad80b0c80 msgr2=0x7f9ad80b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:54.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 -- 172.21.15.71:0/2464742855 shutdown_connections 2023-10-26T05:51:54.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 -- 172.21.15.71:0/2464742855 wait complete. 2023-10-26T05:51:54.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 Processor -- start 2023-10-26T05:51:54.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.716+0000 7f9ae055a640 1 -- start start 2023-10-26T05:51:54.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ac4004d10 0x7f9ad80c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 0x7f9ad81eefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad8143f50 0x7f9ad81443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8144910 con 0x7f9ad8151390 2023-10-26T05:51:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80c9380 con 0x7f9ad8143f50 2023-10-26T05:51:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad81ef500 con 0x7f9ac4004d10 2023-10-26T05:51:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 0x7f9ad81eefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 0x7f9ad81eefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57674/0 (socket says 172.21.15.71:57674) 2023-10-26T05:51:54.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 -- 172.21.15.71:0/1602240201 learned_addr learned my addr 172.21.15.71:0/1602240201 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:54.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9adeaf7640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad8143f50 0x7f9ad81443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ade2f6640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ac4004d10 0x7f9ad80c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 -- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ac4004d10 msgr2=0x7f9ad80c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ac4004d10 0x7f9ad80c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 -- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad8143f50 msgr2=0x7f9ad81443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad8143f50 0x7f9ad81443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 -- 172.21.15.71:0/1602240201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8153870 con 0x7f9ad8151390 2023-10-26T05:51:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9addaf5640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 0x7f9ad81eefc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400f6a0 tx=0x7f9ad400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400a9c0 con 0x7f9ad8151390 2023-10-26T05:51:54.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9ad400ac10 con 0x7f9ad8151390 2023-10-26T05:51:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8153b50 con 0x7f9ad8151390 2023-10-26T05:51:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4012470 con 0x7f9ad8151390 2023-10-26T05:51:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad8154110 con 0x7f9ad8151390 2023-10-26T05:51:54.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad8066b10 con 0x7f9ad8151390 2023-10-26T05:51:54.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9ad40125d0 con 0x7f9ad8151390 2023-10-26T05:51:54.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9abc066470 0x7f9abc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:54.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ad4087750 con 0x7f9ad8151390 2023-10-26T05:51:54.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f9abc06dfa0 con 0x7f9ad8151390 2023-10-26T05:51:54.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.720+0000 7f9ade2f6640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9abc066470 0x7f9abc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:54.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.724+0000 7f9ade2f6640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9abc066470 0x7f9abc068930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80065b0 tx=0x7f9ac8006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:54.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.724+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9ad4054bd0 con 0x7f9ad8151390 2023-10-26T05:51:54.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.884+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ad806a7b0 con 0x7f9ad8151390 2023-10-26T05:51:54.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9aceffd640 1 -- 172.21.15.71:0/1602240201 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9ad40581f0 con 0x7f9ad8151390 2023-10-26T05:51:54.893 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:51:54.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9abc066470 msgr2=0x7f9abc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9abc066470 0x7f9abc068930 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80065b0 tx=0x7f9ac8006af0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 msgr2=0x7f9ad81eefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:54.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 0x7f9ad81eefc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400f6a0 tx=0x7f9ad400fb60 comp rx=0 tx=0).stop 2023-10-26T05:51:54.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 shutdown_connections 2023-10-26T05:51:54.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ad8143f50 0x7f9ad81443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9abc066470 0x7f9abc068930 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ad8151390 0x7f9ad81eefc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 --2- 172.21.15.71:0/1602240201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ac4004d10 0x7f9ad80c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:54.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 >> 172.21.15.71:0/1602240201 conn(0x7f9ad80b0c80 msgr2=0x7f9ad81581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:54.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 shutdown_connections 2023-10-26T05:51:54.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54.888+0000 7f9ae055a640 1 -- 172.21.15.71:0/1602240201 wait complete. 2023-10-26T05:51:54.911 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:51:54.911 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:54.911 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:51:54.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:54.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[43962]: fuse finished with error 0 and tester_r 0 2023-10-26T05:51:54.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:54.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:51:54.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:54.962 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:51:54.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:54.978 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:51:54.978 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:54.985 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:55.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 -- 172.21.15.71:0/2855459093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480d0470 msgr2=0x7fed4806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 --2- 172.21.15.71:0/2855459093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480d0470 0x7fed4806aa70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fed38009930 tx=0x7fed3801c800 comp rx=0 tx=0).stop 2023-10-26T05:51:55.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 -- 172.21.15.71:0/2855459093 shutdown_connections 2023-10-26T05:51:55.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 --2- 172.21.15.71:0/2855459093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480cd880 0x7fed4815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 --2- 172.21.15.71:0/2855459093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480d0470 0x7fed4806aa70 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 -- 172.21.15.71:0/2855459093 >> 172.21.15.71:0/2855459093 conn(0x7fed480cdd80 msgr2=0x7fed480ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:55.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 -- 172.21.15.71:0/2855459093 shutdown_connections 2023-10-26T05:51:55.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.132+0000 7fed4e933640 1 -- 172.21.15.71:0/2855459093 wait complete. 2023-10-26T05:51:55.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 Processor -- start 2023-10-26T05:51:55.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- start start 2023-10-26T05:51:55.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 0x7fed481567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed480d0470 0x7fed4814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480d0850 0x7fed48154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480c6ac0 con 0x7fed480cd880 2023-10-26T05:51:55.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4814bc80 con 0x7fed480d0470 2023-10-26T05:51:55.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed480cc200 con 0x7fed480d0850 2023-10-26T05:51:55.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 0x7fed481567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 0x7fed481567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57678/0 (socket says 172.21.15.71:57678) 2023-10-26T05:51:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 -- 172.21.15.71:0/3156353650 learned_addr learned my addr 172.21.15.71:0/3156353650 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4ced0640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480d0850 0x7fed48154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed477fe640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed480d0470 0x7fed4814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 -- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480d0850 msgr2=0x7fed48154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480d0850 0x7fed48154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 -- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed480d0470 msgr2=0x7fed4814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed480d0470 0x7fed4814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 -- 172.21.15.71:0/3156353650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38009560 con 0x7fed480cd880 2023-10-26T05:51:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed477fe640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed480d0470 0x7fed4814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4ced0640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480d0850 0x7fed48154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed47fff640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 0x7fed481567f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fed3801a430 tx=0x7fed38004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed380ba070 con 0x7fed480cd880 2023-10-26T05:51:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fed38005030 con 0x7fed480cd880 2023-10-26T05:51:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4814bf30 con 0x7fed480cd880 2023-10-26T05:51:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38005420 con 0x7fed480cd880 2023-10-26T05:51:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed480c6750 con 0x7fed480cd880 2023-10-26T05:51:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed48066b10 con 0x7fed480cd880 2023-10-26T05:51:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.136+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fed380b4040 con 0x7fed480cd880 2023-10-26T05:51:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.140+0000 7fed44ff9640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed34066470 0x7fed34068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.140+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed3811e170 con 0x7fed480cd880 2023-10-26T05:51:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.140+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fed3406dfa0 con 0x7fed480cd880 2023-10-26T05:51:55.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.140+0000 7fed477fe640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed34066470 0x7fed34068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.140+0000 7fed477fe640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed34066470 0x7fed34068930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fed480b0a90 tx=0x7fed3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:55.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.140+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fed380eb660 con 0x7fed480cd880 2023-10-26T05:51:55.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.296+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed480721e0 con 0x7fed480cd880 2023-10-26T05:51:55.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.300+0000 7fed44ff9640 1 -- 172.21.15.71:0/3156353650 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16527 (secure 0 0 0) 0x7fed380b2070 con 0x7fed480cd880 2023-10-26T05:51:55.307 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:55.307 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":88,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:43.124800+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:55.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.300+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed34066470 msgr2=0x7fed34068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.300+0000 7fed4e933640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed34066470 0x7fed34068930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fed480b0a90 tx=0x7fed3c009040 comp rx=0 tx=0).stop 2023-10-26T05:51:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.300+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 msgr2=0x7fed481567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.300+0000 7fed4e933640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 0x7fed481567f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fed3801a430 tx=0x7fed38004600 comp rx=0 tx=0).stop 2023-10-26T05:51:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 shutdown_connections 2023-10-26T05:51:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fed480d0850 0x7fed48154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed480d0470 0x7fed4814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fed34066470 0x7fed34068930 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 --2- 172.21.15.71:0/3156353650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed480cd880 0x7fed481567f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 >> 172.21.15.71:0/3156353650 conn(0x7fed480cdd80 msgr2=0x7fed480cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 shutdown_connections 2023-10-26T05:51:55.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.304+0000 7fed4e933640 1 -- 172.21.15.71:0/3156353650 wait complete. 2023-10-26T05:51:55.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:51:15.888674+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:51:16.655596+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:55.326 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:55.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- 172.21.15.71:0/623208499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c141370 msgr2=0x7fca6c158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 --2- 172.21.15.71:0/623208499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c141370 0x7fca6c158680 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fca540098d0 tx=0x7fca5401e950 comp rx=0 tx=0).stop 2023-10-26T05:51:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- 172.21.15.71:0/623208499 shutdown_connections 2023-10-26T05:51:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 --2- 172.21.15.71:0/623208499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca6c0b0f70 0x7fca6c0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 --2- 172.21.15.71:0/623208499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c141370 0x7fca6c158680 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- 172.21.15.71:0/623208499 >> 172.21.15.71:0/623208499 conn(0x7fca6c0afe40 msgr2=0x7fca6c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- 172.21.15.71:0/623208499 shutdown_connections 2023-10-26T05:51:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- 172.21.15.71:0/623208499 wait complete. 2023-10-26T05:51:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 Processor -- start 2023-10-26T05:51:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- start start 2023-10-26T05:51:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca6c070550 0x7fca6c13cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c0b0f70 0x7fca6c0c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c141370 0x7fca6c1f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0c5fe0 con 0x7fca6c0b0f70 2023-10-26T05:51:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c0c4de0 con 0x7fca6c141370 2023-10-26T05:51:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.464+0000 7fca70b0d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c06c060 con 0x7fca6c070550 2023-10-26T05:51:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca6ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c141370 0x7fca6c1f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca6ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c141370 0x7fca6c1f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:46116/0 (socket says 172.21.15.71:46116) 2023-10-26T05:51:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca6ad9d640 1 -- 172.21.15.71:0/1988511285 learned_addr learned my addr 172.21.15.71:0/1988511285 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c0b0f70 0x7fca6c0c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca6a59c640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca6c070550 0x7fca6c13cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 -- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca6c070550 msgr2=0x7fca6c13cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca6c070550 0x7fca6c13cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 -- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c141370 msgr2=0x7fca6c1f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c141370 0x7fca6c1f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 -- 172.21.15.71:0/1988511285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540094a0 con 0x7fca6c0b0f70 2023-10-26T05:51:55.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca61d9b640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c0b0f70 0x7fca6c0c48a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00d750 tx=0x7fca5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:55.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c009bd0 con 0x7fca6c0b0f70 2023-10-26T05:51:55.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca5c009e20 con 0x7fca6c0b0f70 2023-10-26T05:51:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c06c900 con 0x7fca6c0b0f70 2023-10-26T05:51:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c06c2b0 con 0x7fca6c0b0f70 2023-10-26T05:51:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c005280 con 0x7fca6c0b0f70 2023-10-26T05:51:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca50005260 con 0x7fca6c0b0f70 2023-10-26T05:51:55.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca5c0053e0 con 0x7fca6c0b0f70 2023-10-26T05:51:55.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.468+0000 7fca637fe640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca4c0664a0 0x7fca4c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.472+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca5c083e50 con 0x7fca6c0b0f70 2023-10-26T05:51:55.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.472+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fca4c06dfd0 con 0x7fca6c0b0f70 2023-10-26T05:51:55.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.472+0000 7fca6a59c640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca4c0664a0 0x7fca4c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.472+0000 7fca6a59c640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca4c0664a0 0x7fca4c068960 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca540b1000 tx=0x7fca5401d190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:55.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.476+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fca5c020060 con 0x7fca6c0b0f70 2023-10-26T05:51:55.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.640+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca50005a90 con 0x7fca6c0b0f70 2023-10-26T05:51:55.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.640+0000 7fca637fe640 1 -- 172.21.15.71:0/1988511285 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca5c085030 con 0x7fca6c0b0f70 2023-10-26T05:51:55.647 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:55.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca4c0664a0 msgr2=0x7fca4c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca4c0664a0 0x7fca4c068960 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fca540b1000 tx=0x7fca5401d190 comp rx=0 tx=0).stop 2023-10-26T05:51:55.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c0b0f70 msgr2=0x7fca6c0c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c0b0f70 0x7fca6c0c48a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00d750 tx=0x7fca5c00dc10 comp rx=0 tx=0).stop 2023-10-26T05:51:55.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 shutdown_connections 2023-10-26T05:51:55.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6c141370 0x7fca6c1f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca4c0664a0 0x7fca4c068960 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca6c0b0f70 0x7fca6c0c48a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 --2- 172.21.15.71:0/1988511285 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca6c070550 0x7fca6c13cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 >> 172.21.15.71:0/1988511285 conn(0x7fca6c0afe40 msgr2=0x7fca6c159860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:55.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 shutdown_connections 2023-10-26T05:51:55.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.644+0000 7fca70b0d640 1 -- 172.21.15.71:0/1988511285 wait complete. 2023-10-26T05:51:55.666 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:51:55.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 -- 172.21.15.71:0/3992587832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c01422e0 msgr2=0x7fe3c013eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/3992587832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c01422e0 0x7fe3c013eda0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009840 tx=0x7fe3b001c790 comp rx=0 tx=0).stop 2023-10-26T05:51:55.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 -- 172.21.15.71:0/3992587832 shutdown_connections 2023-10-26T05:51:55.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/3992587832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c01422e0 0x7fe3c013eda0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 -- 172.21.15.71:0/3992587832 >> 172.21.15.71:0/3992587832 conn(0x7fe3c00aec60 msgr2=0x7fe3c00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:55.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 -- 172.21.15.71:0/3992587832 shutdown_connections 2023-10-26T05:51:55.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 -- 172.21.15.71:0/3992587832 wait complete. 2023-10-26T05:51:55.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 Processor -- start 2023-10-26T05:51:55.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.784+0000 7fe3c6de0640 1 -- start start 2023-10-26T05:51:55.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 0x7fe3c01f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3c013f2e0 0x7fe3c013a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c01422e0 0x7fe3c0138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c013ae70 con 0x7fe3c013e9c0 2023-10-26T05:51:55.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c013b150 con 0x7fe3c01422e0 2023-10-26T05:51:55.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c013b430 con 0x7fe3c013f2e0 2023-10-26T05:51:55.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 0x7fe3c01f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 0x7fe3c01f8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57710/0 (socket says 172.21.15.71:57710) 2023-10-26T05:51:55.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 -- 172.21.15.71:0/2981938757 learned_addr learned my addr 172.21.15.71:0/2981938757 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:55.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c537d640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c01422e0 0x7fe3c0138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b7fff640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3c013f2e0 0x7fe3c013a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 -- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3c013f2e0 msgr2=0x7fe3c013a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3c013f2e0 0x7fe3c013a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 -- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c01422e0 msgr2=0x7fe3c0138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c01422e0 0x7fe3c0138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 -- 172.21.15.71:0/2981938757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00094a0 con 0x7fe3c013e9c0 2023-10-26T05:51:55.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c4b7c640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 0x7fe3c01f8d40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00c480 tx=0x7fe3bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:55.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc012830 con 0x7fe3c013e9c0 2023-10-26T05:51:55.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c01392d0 con 0x7fe3c013e9c0 2023-10-26T05:51:55.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3bc012a80 con 0x7fe3c013e9c0 2023-10-26T05:51:55.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc011380 con 0x7fe3c013e9c0 2023-10-26T05:51:55.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c0138970 con 0x7fe3c013e9c0 2023-10-26T05:51:55.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c0139920 con 0x7fe3c013e9c0 2023-10-26T05:51:55.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe3bc0114e0 con 0x7fe3c013e9c0 2023-10-26T05:51:55.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3ac066470 0x7fe3ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:55.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3bc083ff0 con 0x7fe3c013e9c0 2023-10-26T05:51:55.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fe3ac06dfa0 con 0x7fe3c013e9c0 2023-10-26T05:51:55.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.788+0000 7fe3b7fff640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3ac066470 0x7fe3ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.792+0000 7fe3b7fff640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3ac066470 0x7fe3ac068930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00af000 tx=0x7fe3b001afd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:55.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.792+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe3bc051480 con 0x7fe3c013e9c0 2023-10-26T05:51:55.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.944+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3c0066b10 con 0x7fe3c013e9c0 2023-10-26T05:51:55.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.944+0000 7fe3b57fa640 1 -- 172.21.15.71:0/2981938757 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3bc054aa0 con 0x7fe3c013e9c0 2023-10-26T05:51:55.952 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:51:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3ac066470 msgr2=0x7fe3ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3ac066470 0x7fe3ac068930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00af000 tx=0x7fe3b001afd0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 msgr2=0x7fe3c01f8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:55.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 0x7fe3c01f8d40 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00c480 tx=0x7fe3bc00c940 comp rx=0 tx=0).stop 2023-10-26T05:51:55.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 shutdown_connections 2023-10-26T05:51:55.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3c01422e0 0x7fe3c0138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3c013f2e0 0x7fe3c013a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3ac066470 0x7fe3ac068930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 --2- 172.21.15.71:0/2981938757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3c013e9c0 0x7fe3c01f8d40 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 >> 172.21.15.71:0/2981938757 conn(0x7fe3c00aec60 msgr2=0x7fe3c0158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 shutdown_connections 2023-10-26T05:51:55.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:55.948+0000 7fe3c6de0640 1 -- 172.21.15.71:0/2981938757 wait complete. 2023-10-26T05:51:55.971 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:51:56.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 -- 172.21.15.71:0/2149959681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe414e7b0 msgr2=0x7fbfe4157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 --2- 172.21.15.71:0/2149959681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe414e7b0 0x7fbfe4157db0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4070830 tx=0x7fbfd401c860 comp rx=0 tx=0).stop 2023-10-26T05:51:56.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 -- 172.21.15.71:0/2149959681 shutdown_connections 2023-10-26T05:51:56.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 --2- 172.21.15.71:0/2149959681 >> v2:172.21.15.86:3300/0 conn(0x7fbfe4071270 0x7fbfe415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 --2- 172.21.15.71:0/2149959681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe414e7b0 0x7fbfe4157db0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 -- 172.21.15.71:0/2149959681 >> 172.21.15.71:0/2149959681 conn(0x7fbfe40aec60 msgr2=0x7fbfe40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:56.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 -- 172.21.15.71:0/2149959681 shutdown_connections 2023-10-26T05:51:56.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.092+0000 7fbfe9349640 1 -- 172.21.15.71:0/2149959681 wait complete. 2023-10-26T05:51:56.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 Processor -- start 2023-10-26T05:51:56.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- start start 2023-10-26T05:51:56.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfe4071270 0x7fbfe41f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfe414e7b0 0x7fbfe40c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 0x7fbfe40ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe40ad6b0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe414ef30 con 0x7fbfe414e7b0 2023-10-26T05:51:56.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe413dff0 con 0x7fbfe4071270 2023-10-26T05:51:56.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 0x7fbfe40ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 0x7fbfe40ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57724/0 (socket says 172.21.15.71:57724) 2023-10-26T05:51:56.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 -- 172.21.15.71:0/1910962704 learned_addr learned my addr 172.21.15.71:0/1910962704 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:56.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe2ffd640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfe4071270 0x7fbfe41f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe27fc640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfe414e7b0 0x7fbfe40c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 -- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfe4071270 msgr2=0x7fbfe41f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfe4071270 0x7fbfe41f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 -- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfe414e7b0 msgr2=0x7fbfe40c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfe414e7b0 0x7fbfe40c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 -- 172.21.15.71:0/1910962704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4009560 con 0x7fbfe415c0a0 2023-10-26T05:51:56.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe2ffd640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfe4071270 0x7fbfe41f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:56.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe27fc640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfe414e7b0 0x7fbfe40c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:56.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe37fe640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 0x7fbfe40ca4e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b970 tx=0x7fbfdc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:56.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00cb40 con 0x7fbfe415c0a0 2023-10-26T05:51:56.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe40ad950 con 0x7fbfe415c0a0 2023-10-26T05:51:56.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfdc00cd90 con 0x7fbfe415c0a0 2023-10-26T05:51:56.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0146a0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe40adc30 con 0x7fbfe415c0a0 2023-10-26T05:51:56.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.096+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe4066b10 con 0x7fbfe415c0a0 2023-10-26T05:51:56.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.100+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbfdc01d3b0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.100+0000 7fbfcb7fe640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd0066470 0x7fbfd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.100+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfdc0841d0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.100+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fbfd006dfa0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.100+0000 7fbfe2ffd640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd0066470 0x7fbfd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.100+0000 7fbfe2ffd640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd0066470 0x7fbfd0068930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40af000 tx=0x7fbfd401b0a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:56.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.104+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbfdc051530 con 0x7fbfe415c0a0 2023-10-26T05:51:56.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.248+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbfe40cdcd0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.248+0000 7fbfcb7fe640 1 -- 172.21.15.71:0/1910962704 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbfdc0190a0 con 0x7fbfe415c0a0 2023-10-26T05:51:56.256 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:51:56.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd0066470 msgr2=0x7fbfd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd0066470 0x7fbfd0068930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40af000 tx=0x7fbfd401b0a0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 msgr2=0x7fbfe40ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 0x7fbfe40ca4e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b970 tx=0x7fbfdc00be30 comp rx=0 tx=0).stop 2023-10-26T05:51:56.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 shutdown_connections 2023-10-26T05:51:56.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd0066470 0x7fbfd0068930 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe415c0a0 0x7fbfe40ca4e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfe414e7b0 0x7fbfe40c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 --2- 172.21.15.71:0/1910962704 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfe4071270 0x7fbfe41f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 >> 172.21.15.71:0/1910962704 conn(0x7fbfe40aec60 msgr2=0x7fbfe41510a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:56.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 shutdown_connections 2023-10-26T05:51:56.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.252+0000 7fbfe9349640 1 -- 172.21.15.71:0/1910962704 wait complete. 2023-10-26T05:51:56.273 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:51:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 -- 172.21.15.71:0/877070456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd93013de80 msgr2=0x7fd93006f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 --2- 172.21.15.71:0/877070456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd93013de80 0x7fd93006f6f0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd928009c80 tx=0x7fd92801ccc0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 -- 172.21.15.71:0/877070456 shutdown_connections 2023-10-26T05:51:56.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 --2- 172.21.15.71:0/877070456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd93013de80 0x7fd93006f6f0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 --2- 172.21.15.71:0/877070456 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 0x7fd9300cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 -- 172.21.15.71:0/877070456 >> 172.21.15.71:0/877070456 conn(0x7fd93006b8c0 msgr2=0x7fd93006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 -- 172.21.15.71:0/877070456 shutdown_connections 2023-10-26T05:51:56.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 -- 172.21.15.71:0/877070456 wait complete. 2023-10-26T05:51:56.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 Processor -- start 2023-10-26T05:51:56.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.388+0000 7fd934aef640 1 -- start start 2023-10-26T05:51:56.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 0x7fd930133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd93013de80 0x7fd9300b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 0x7fd93014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930134430 con 0x7fd930159060 2023-10-26T05:51:56.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93013dbc0 con 0x7fd93013de80 2023-10-26T05:51:56.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93013cfb0 con 0x7fd93013ae90 2023-10-26T05:51:56.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 0x7fd93014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 0x7fd93014de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57736/0 (socket says 172.21.15.71:57736) 2023-10-26T05:51:56.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 -- 172.21.15.71:0/2013419457 learned_addr learned my addr 172.21.15.71:0/2013419457 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:56.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92e59c640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 0x7fd930133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92dd9b640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd93013de80 0x7fd9300b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 -- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 msgr2=0x7fd930133ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 0x7fd930133ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 -- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd93013de80 msgr2=0x7fd9300b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd93013de80 0x7fd9300b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 -- 172.21.15.71:0/2013419457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280098e0 con 0x7fd930159060 2023-10-26T05:51:56.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92dd9b640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd93013de80 0x7fd9300b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:56.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92e59c640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 0x7fd930133ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:56.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92ed9d640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 0x7fd93014de90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd928009bd0 tx=0x7fd928009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:56.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928004e60 con 0x7fd930159060 2023-10-26T05:51:56.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd928004000 con 0x7fd930159060 2023-10-26T05:51:56.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93013d290 con 0x7fd930159060 2023-10-26T05:51:56.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280043d0 con 0x7fd930159060 2023-10-26T05:51:56.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93013d850 con 0x7fd930159060 2023-10-26T05:51:56.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930066b10 con 0x7fd930159060 2023-10-26T05:51:56.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9280b4040 con 0x7fd930159060 2023-10-26T05:51:56.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd910066470 0x7fd910068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd92811e300 con 0x7fd930159060 2023-10-26T05:51:56.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7fd91006dfa0 con 0x7fd930159060 2023-10-26T05:51:56.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.392+0000 7fd92e59c640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd910066470 0x7fd910068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.396+0000 7fd92e59c640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd910066470 0x7fd910068930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd920009aa0 tx=0x7fd920005d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:56.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.396+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9280b2020 con 0x7fd930159060 2023-10-26T05:51:56.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.556+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd93006c7d0 con 0x7fd930159060 2023-10-26T05:51:56.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.556+0000 7fd91effd640 1 -- 172.21.15.71:0/2013419457 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd9280bc070 con 0x7fd930159060 2023-10-26T05:51:56.563 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:51:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd910066470 msgr2=0x7fd910068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd910066470 0x7fd910068930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd920009aa0 tx=0x7fd920005d30 comp rx=0 tx=0).stop 2023-10-26T05:51:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 msgr2=0x7fd93014de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 0x7fd93014de90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd928009bd0 tx=0x7fd928009390 comp rx=0 tx=0).stop 2023-10-26T05:51:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 shutdown_connections 2023-10-26T05:51:56.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd910066470 0x7fd910068930 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd930159060 0x7fd93014de90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd93013de80 0x7fd9300b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 --2- 172.21.15.71:0/2013419457 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd93013ae90 0x7fd930133ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 >> 172.21.15.71:0/2013419457 conn(0x7fd93006b8c0 msgr2=0x7fd930156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:56.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 shutdown_connections 2023-10-26T05:51:56.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.560+0000 7fd934aef640 1 -- 172.21.15.71:0/2013419457 wait complete. 2023-10-26T05:51:56.580 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:51:56.581 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:51:56.581 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:51:56.635 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:51:56.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34587]: fuse finished with error 0 and tester_r 0 2023-10-26T05:51:56.636 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:51:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:56.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:51:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:51:56.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:51:56.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:56.646 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:51:56.654 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:51:56.654 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:56.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:51:56.662 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:51:56.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:56.664 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:51:56.708 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:51:56.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:51:56.710 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:51:56.711 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:51:56.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:51:56.717 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:51:56.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:51:56.718 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:51:56.762 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:51:56.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:51:56.764 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:56.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/838419281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 msgr2=0x7f09f415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/838419281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f415c760 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009930 tx=0x7f09dc01c850 comp rx=0 tx=0).stop 2023-10-26T05:51:56.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/838419281 shutdown_connections 2023-10-26T05:51:56.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/838419281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f415c760 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/838419281 >> 172.21.15.71:0/838419281 conn(0x7f09f4069770 msgr2=0x7f09f4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:56.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/838419281 shutdown_connections 2023-10-26T05:51:56.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/838419281 wait complete. 2023-10-26T05:51:56.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 Processor -- start 2023-10-26T05:51:56.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- start start 2023-10-26T05:51:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09f4070fe0 0x7f09f412d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f40b2370 0x7f09f41391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f413f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4158d30 con 0x7f09f4140cc0 2023-10-26T05:51:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4163800 con 0x7f09f40b2370 2023-10-26T05:51:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4163ae0 con 0x7f09f4070fe0 2023-10-26T05:51:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f413f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f413f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57756/0 (socket says 172.21.15.71:57756) 2023-10-26T05:51:56.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 -- 172.21.15.71:0/282746106 learned_addr learned my addr 172.21.15.71:0/282746106 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09eaffd640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f40b2370 0x7f09f41391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f37fe640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09f4070fe0 0x7f09f412d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 -- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09f4070fe0 msgr2=0x7f09f412d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09f4070fe0 0x7f09f412d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 -- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f40b2370 msgr2=0x7f09f41391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f40b2370 0x7f09f41391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:56.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 -- 172.21.15.71:0/282746106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc009560 con 0x7f09f4140cc0 2023-10-26T05:51:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f3fff640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f413f3d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c780 tx=0x7f09ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec003d90 con 0x7f09f4140cc0 2023-10-26T05:51:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09ec00ce90 con 0x7f09f4140cc0 2023-10-26T05:51:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f4163dc0 con 0x7f09f4140cc0 2023-10-26T05:51:56.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0052d0 con 0x7f09f4140cc0 2023-10-26T05:51:56.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.888+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f4139720 con 0x7f09f4140cc0 2023-10-26T05:51:56.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4066b10 con 0x7f09f4140cc0 2023-10-26T05:51:56.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f09ec005430 con 0x7f09f4140cc0 2023-10-26T05:51:56.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f0ff9640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09d8066470 0x7f09d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:56.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09ec083480 con 0x7f09f4140cc0 2023-10-26T05:51:56.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=89}) v3 -- 0x7f09d806dfa0 con 0x7f09f4140cc0 2023-10-26T05:51:56.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f37fe640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09d8066470 0x7f09d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:56.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.892+0000 7f09f37fe640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09d8066470 0x7f09d8068930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0af000 tx=0x7f09dc01b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:56.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:56.896+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f09ec050990 con 0x7f09f4140cc0 2023-10-26T05:51:57.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.080+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09f4158e70 con 0x7f09f4140cc0 2023-10-26T05:51:57.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f0ff9640 1 -- 172.21.15.71:0/282746106 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f09f4158e70 con 0x7f09f4140cc0 2023-10-26T05:51:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:57.089 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:57.089 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:57.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09d8066470 msgr2=0x7f09d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09d8066470 0x7f09d8068930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0af000 tx=0x7f09dc01b090 comp rx=0 tx=0).stop 2023-10-26T05:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 msgr2=0x7f09f413f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f413f3d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f09ec00c780 tx=0x7f09ec00cc40 comp rx=0 tx=0).stop 2023-10-26T05:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 shutdown_connections 2023-10-26T05:51:57.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09d8066470 0x7f09d8068930 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09f4140cc0 0x7f09f413f3d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f40b2370 0x7f09f41391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 --2- 172.21.15.71:0/282746106 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f09f4070fe0 0x7f09f412d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 >> 172.21.15.71:0/282746106 conn(0x7f09f4069770 msgr2=0x7f09f415a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:57.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 shutdown_connections 2023-10-26T05:51:57.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.084+0000 7f09f9e1e640 1 -- 172.21.15.71:0/282746106 wait complete. 2023-10-26T05:51:57.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:57.167 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:51:57.160+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-10-26T05:51:57.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f9906b38640 1 -- 172.21.15.71:0/1170507589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 msgr2=0x7f98ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f9906b38640 1 --2- 172.21.15.71:0/1170507589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f98ec003dd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006fb0 tx=0x7f98f001cfa0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f99050d5640 1 -- 172.21.15.71:0/1170507589 reap_dead start 2023-10-26T05:51:57.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f9906b38640 1 -- 172.21.15.71:0/1170507589 shutdown_connections 2023-10-26T05:51:57.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f9906b38640 1 -- 172.21.15.71:0/1170507589 >> 172.21.15.71:0/1170507589 conn(0x7f9900069770 msgr2=0x7f9900069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:57.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f9906b38640 1 -- 172.21.15.71:0/1170507589 shutdown_connections 2023-10-26T05:51:57.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.224+0000 7f9906b38640 1 -- 172.21.15.71:0/1170507589 wait complete. 2023-10-26T05:51:57.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 Processor -- start 2023-10-26T05:51:57.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- start start 2023-10-26T05:51:57.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f990013a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99000b17f0 0x7f99001375e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99001387c0 0x7f99000adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990013a850 con 0x7f98ec0039b0 2023-10-26T05:51:57.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000b1c70 con 0x7f99001387c0 2023-10-26T05:51:57.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990013aa40 con 0x7f99000b17f0 2023-10-26T05:51:57.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f990013a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f990013a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57768/0 (socket says 172.21.15.71:57768) 2023-10-26T05:51:57.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 -- 172.21.15.71:0/348904277 learned_addr learned my addr 172.21.15.71:0/348904277 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:57.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 -- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99000b17f0 msgr2=0x7f99001375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99000b17f0 0x7f99001375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 -- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99001387c0 msgr2=0x7f99000adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99001387c0 0x7f99000adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 -- 172.21.15.71:0/348904277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990013ad20 con 0x7f98ec0039b0 2023-10-26T05:51:57.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f99048d4640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f990013a310 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f98f001cdd0 tx=0x7f98f0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:57.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00b3070 con 0x7f98ec0039b0 2023-10-26T05:51:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98f0001600 con 0x7f98ec0039b0 2023-10-26T05:51:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990013d5d0 con 0x7f98ec0039b0 2023-10-26T05:51:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0006440 con 0x7f98ec0039b0 2023-10-26T05:51:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990013b000 con 0x7f98ec0039b0 2023-10-26T05:51:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9900066b10 con 0x7f98ec0039b0 2023-10-26T05:51:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.228+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f98f00ab020 con 0x7f98ec0039b0 2023-10-26T05:51:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.232+0000 7f98fd7fa640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e00664a0 0x7f98e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.232+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98f011d4d0 con 0x7f98ec0039b0 2023-10-26T05:51:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.232+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f98e006dfd0 con 0x7f98ec0039b0 2023-10-26T05:51:57.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.232+0000 7f98fffff640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e00664a0 0x7f98e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.232+0000 7f98fffff640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e00664a0 0x7f98e0068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006fd0 tx=0x7f98e8006a10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:57.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.236+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98f00ea9e0 con 0x7f98ec0039b0 2023-10-26T05:51:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.380+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f990013d430 con 0x7f98ec0039b0 2023-10-26T05:51:57.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f98fd7fa640 1 -- 172.21.15.71:0/348904277 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16520 (secure 0 0 0) 0x7f990013d430 con 0x7f98ec0039b0 2023-10-26T05:51:57.418 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:57.418 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":89,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:57.165584+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e00664a0 msgr2=0x7f98e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e00664a0 0x7f98e0068960 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98e8006fd0 tx=0x7f98e8006a10 comp rx=0 tx=0).stop 2023-10-26T05:51:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 msgr2=0x7f990013a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f990013a310 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f98f001cdd0 tx=0x7f98f0000ee0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 shutdown_connections 2023-10-26T05:51:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99001387c0 0x7f99000adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99000b17f0 0x7f99001375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e00664a0 0x7f98e0068960 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 --2- 172.21.15.71:0/348904277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98ec0039b0 0x7f990013a310 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 >> 172.21.15.71:0/348904277 conn(0x7f9900069770 msgr2=0x7f99000b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 shutdown_connections 2023-10-26T05:51:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.384+0000 7f9906b38640 1 -- 172.21.15.71:0/348904277 wait complete. 2023-10-26T05:51:57.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:51:15.888674+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:51:16.655596+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:51:57.423 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:51:57.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 -- 172.21.15.71:0/1687115740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c13ed90 msgr2=0x7faf8c069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 --2- 172.21.15.71:0/1687115740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c13ed90 0x7faf8c069c00 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7faf84009bc0 tx=0x7faf8401cb50 comp rx=0 tx=0).stop 2023-10-26T05:51:57.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 -- 172.21.15.71:0/1687115740 shutdown_connections 2023-10-26T05:51:57.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 --2- 172.21.15.71:0/1687115740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c13ed90 0x7faf8c069c00 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 --2- 172.21.15.71:0/1687115740 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf8c15a3f0 0x7faf8c13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 -- 172.21.15.71:0/1687115740 >> 172.21.15.71:0/1687115740 conn(0x7faf8c0aec60 msgr2=0x7faf8c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 -- 172.21.15.71:0/1687115740 shutdown_connections 2023-10-26T05:51:57.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 -- 172.21.15.71:0/1687115740 wait complete. 2023-10-26T05:51:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 Processor -- start 2023-10-26T05:51:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.540+0000 7faf93113640 1 -- start start 2023-10-26T05:51:57.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf8c13c120 0x7faf8c136e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf8c13ed90 0x7faf8c1343f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 0x7faf8c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c134930 con 0x7faf8c15a3f0 2023-10-26T05:51:57.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c1572f0 con 0x7faf8c13c120 2023-10-26T05:51:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c1575d0 con 0x7faf8c13ed90 2023-10-26T05:51:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 0x7faf8c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 0x7faf8c159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57788/0 (socket says 172.21.15.71:57788) 2023-10-26T05:51:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 -- 172.21.15.71:0/2128915151 learned_addr learned my addr 172.21.15.71:0/2128915151 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf90eaf640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf8c13c120 0x7faf8c136e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 -- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf8c13ed90 msgr2=0x7faf8c1343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf8c13ed90 0x7faf8c1343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 -- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf8c13c120 msgr2=0x7faf8c136e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf8c13c120 0x7faf8c136e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 -- 172.21.15.71:0/2128915151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf84009820 con 0x7faf8c15a3f0 2023-10-26T05:51:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf916b0640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 0x7faf8c159b20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7faf84009b10 tx=0x7faf840a5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:57.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84004d10 con 0x7faf8c15a3f0 2023-10-26T05:51:57.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faf84003fe0 con 0x7faf8c15a3f0 2023-10-26T05:51:57.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c153120 con 0x7faf8c15a3f0 2023-10-26T05:51:57.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84004370 con 0x7faf8c15a3f0 2023-10-26T05:51:57.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c135d40 con 0x7faf8c15a3f0 2023-10-26T05:51:57.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c066b10 con 0x7faf8c15a3f0 2023-10-26T05:51:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf840b4040 con 0x7faf8c15a3f0 2023-10-26T05:51:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf78066470 0x7faf78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf8411dfd0 con 0x7faf8c15a3f0 2023-10-26T05:51:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7faf7806dfa0 con 0x7faf8c15a3f0 2023-10-26T05:51:57.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.544+0000 7faf90eaf640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf78066470 0x7faf78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.548+0000 7faf90eaf640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf78066470 0x7faf78068930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf7c000f70 tx=0x7faf7c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:57.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.548+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faf840b2020 con 0x7faf8c15a3f0 2023-10-26T05:51:57.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf8c159840 con 0x7faf8c15a3f0 2023-10-26T05:51:57.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf897fa640 1 -- 172.21.15.71:0/2128915151 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf8c159840 con 0x7faf8c15a3f0 2023-10-26T05:51:57.710 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:51:57.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf78066470 msgr2=0x7faf78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf78066470 0x7faf78068930 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faf7c000f70 tx=0x7faf7c009040 comp rx=0 tx=0).stop 2023-10-26T05:51:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 msgr2=0x7faf8c159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 0x7faf8c159b20 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7faf84009b10 tx=0x7faf840a5b80 comp rx=0 tx=0).stop 2023-10-26T05:51:57.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 shutdown_connections 2023-10-26T05:51:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faf78066470 0x7faf78068930 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf8c15a3f0 0x7faf8c159b20 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf8c13ed90 0x7faf8c1343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 --2- 172.21.15.71:0/2128915151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf8c13c120 0x7faf8c136e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 >> 172.21.15.71:0/2128915151 conn(0x7faf8c0aec60 msgr2=0x7faf8c13c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 shutdown_connections 2023-10-26T05:51:57.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.704+0000 7faf93113640 1 -- 172.21.15.71:0/2128915151 wait complete. 2023-10-26T05:51:57.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:51:57.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/4192599434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c13c220 msgr2=0x7f1a8c15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 --2- 172.21.15.71:0/4192599434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c13c220 0x7f1a8c15cce0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009840 tx=0x7f1a7c01c700 comp rx=0 tx=0).stop 2023-10-26T05:51:57.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/4192599434 shutdown_connections 2023-10-26T05:51:57.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 --2- 172.21.15.71:0/4192599434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c13c220 0x7f1a8c15cce0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/4192599434 >> 172.21.15.71:0/4192599434 conn(0x7f1a8c0aec60 msgr2=0x7f1a8c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:57.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/4192599434 shutdown_connections 2023-10-26T05:51:57.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/4192599434 wait complete. 2023-10-26T05:51:57.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 Processor -- start 2023-10-26T05:51:57.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- start start 2023-10-26T05:51:57.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 0x7f1a8c133a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a8c13c220 0x7f1a8c14e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c15d220 0x7f1a8c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c133f90 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c1340f0 con 0x7f1a8c15d220 2023-10-26T05:51:57.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c14e940 con 0x7f1a8c13c220 2023-10-26T05:51:57.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 0x7f1a8c133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 0x7f1a8c133a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57796/0 (socket says 172.21.15.71:57796) 2023-10-26T05:51:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 -- 172.21.15.71:0/1778554803 learned_addr learned my addr 172.21.15.71:0/1778554803 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8a7fc640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a8c13c220 0x7f1a8c14e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8b7fe640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c15d220 0x7f1a8c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 -- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a8c13c220 msgr2=0x7f1a8c14e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a8c13c220 0x7f1a8c14e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 -- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c15d220 msgr2=0x7f1a8c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c15d220 0x7f1a8c158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:57.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 -- 172.21.15.71:0/1778554803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c0094a0 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a8affd640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 0x7f1a8c133a50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009790 tx=0x7f1a7c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:57.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0044f0 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a7c005120 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c134340 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c005510 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.852+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c138040 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c066b10 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1a7c0b4040 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a737fe640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a78066470 0x7f1a78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:57.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a7c11dd00 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f1a7806dfa0 con 0x7f1a8c0cbf30 2023-10-26T05:51:57.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a8a7fc640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a78066470 0x7f1a78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:57.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.856+0000 7f1a8a7fc640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a78066470 0x7f1a78068930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c134f20 tx=0x7f1a80009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:57.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:57.860+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1a7c0b2020 con 0x7f1a8c0cbf30 2023-10-26T05:51:58.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.000+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a8c06b0e0 con 0x7f1a8c0cbf30 2023-10-26T05:51:58.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.000+0000 7f1a737fe640 1 -- 172.21.15.71:0/1778554803 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a7c0f1030 con 0x7f1a8c0cbf30 2023-10-26T05:51:58.007 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:51:58.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a78066470 msgr2=0x7f1a78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a78066470 0x7f1a78068930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c134f20 tx=0x7f1a80009040 comp rx=0 tx=0).stop 2023-10-26T05:51:58.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 msgr2=0x7f1a8c133a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 0x7f1a8c133a50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009790 tx=0x7f1a7c004040 comp rx=0 tx=0).stop 2023-10-26T05:51:58.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 shutdown_connections 2023-10-26T05:51:58.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a8c15d220 0x7f1a8c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a8c13c220 0x7f1a8c14e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a78066470 0x7f1a78068930 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 --2- 172.21.15.71:0/1778554803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a8c0cbf30 0x7f1a8c133a50 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 >> 172.21.15.71:0/1778554803 conn(0x7f1a8c0aec60 msgr2=0x7f1a8c15a820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 shutdown_connections 2023-10-26T05:51:58.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.004+0000 7f1a91781640 1 -- 172.21.15.71:0/1778554803 wait complete. 2023-10-26T05:51:58.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:51:58.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 -- 172.21.15.71:0/1504995649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 msgr2=0x7f9f34003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 --2- 172.21.15.71:0/1504995649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f34003da0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0016b0 tx=0x7f9f3c01c750 comp rx=0 tx=0).stop 2023-10-26T05:51:58.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 -- 172.21.15.71:0/1504995649 shutdown_connections 2023-10-26T05:51:58.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 --2- 172.21.15.71:0/1504995649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f34003da0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 -- 172.21.15.71:0/1504995649 >> 172.21.15.71:0/1504995649 conn(0x7f9f4c0cdd80 msgr2=0x7f9f4c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 -- 172.21.15.71:0/1504995649 shutdown_connections 2023-10-26T05:51:58.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.128+0000 7f9f52e22640 1 -- 172.21.15.71:0/1504995649 wait complete. 2023-10-26T05:51:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 Processor -- start 2023-10-26T05:51:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- start start 2023-10-26T05:51:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f4c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4c1f5250 0x7f9f4c0c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f4c158640 0x7f9f4c0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c06a650 con 0x7f9f34003980 2023-10-26T05:51:58.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0c6550 con 0x7f9f4c1f5250 2023-10-26T05:51:58.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0c5d50 con 0x7f9f4c158640 2023-10-26T05:51:58.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f4c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f4c0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57818/0 (socket says 172.21.15.71:57818) 2023-10-26T05:51:58.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 -- 172.21.15.71:0/4290569616 learned_addr learned my addr 172.21.15.71:0/4290569616 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:58.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 -- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f4c158640 msgr2=0x7f9f4c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f4c158640 0x7f9f4c0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 -- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4c1f5250 msgr2=0x7f9f4c0c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4c1f5250 0x7f9f4c0c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c0c7d10 con 0x7f9f34003980 2023-10-26T05:51:58.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f50bbe640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f4c0b14e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005040 tx=0x7f9f3c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:58.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0a9040 con 0x7f9f34003980 2023-10-26T05:51:58.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f3c006150 con 0x7f9f34003980 2023-10-26T05:51:58.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0a5550 con 0x7f9f34003980 2023-10-26T05:51:58.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c0c5f90 con 0x7f9f34003980 2023-10-26T05:51:58.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c0c95c0 con 0x7f9f34003980 2023-10-26T05:51:58.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.132+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c066b10 con 0x7f9f34003980 2023-10-26T05:51:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.136+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9f3c0b5020 con 0x7f9f34003980 2023-10-26T05:51:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.136+0000 7f9f497fa640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f38066460 0x7f9f38068920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.136+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f3c0aa070 con 0x7f9f34003980 2023-10-26T05:51:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.136+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=90}) v3 -- 0x7f9f3806df90 con 0x7f9f34003980 2023-10-26T05:51:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.136+0000 7f9f4bfff640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f38066460 0x7f9f38068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.136+0000 7f9f4bfff640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f38066460 0x7f9f38068920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0b52f0 tx=0x7f9f40001550 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:58.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.140+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9f3c0bd020 con 0x7f9f34003980 2023-10-26T05:51:58.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.180+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 7 ==== osd_map(90..90 src has 1..90) v4 ==== 296+0+0 (secure 0 0 0) 0x7f9f3c0ef730 con 0x7f9f34003980 2023-10-26T05:51:58.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.180+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f9f3806e840 con 0x7f9f34003980 2023-10-26T05:51:58.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.292+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f4c0c9420 con 0x7f9f34003980 2023-10-26T05:51:58.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.292+0000 7f9f497fa640 1 -- 172.21.15.71:0/4290569616 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f4c0c9420 con 0x7f9f34003980 2023-10-26T05:51:58.299 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:51:58.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f38066460 msgr2=0x7f9f38068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f38066460 0x7f9f38068920 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0b52f0 tx=0x7f9f40001550 comp rx=0 tx=0).stop 2023-10-26T05:51:58.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 msgr2=0x7f9f4c0b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f4c0b14e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c005040 tx=0x7f9f3c003c80 comp rx=0 tx=0).stop 2023-10-26T05:51:58.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 shutdown_connections 2023-10-26T05:51:58.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f4c158640 0x7f9f4c0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f4c1f5250 0x7f9f4c0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f38066460 0x7f9f38068920 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 --2- 172.21.15.71:0/4290569616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f34003980 0x7f9f4c0b14e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 >> 172.21.15.71:0/4290569616 conn(0x7f9f4c0cdd80 msgr2=0x7f9f4c15e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 shutdown_connections 2023-10-26T05:51:58.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.296+0000 7f9f52e22640 1 -- 172.21.15.71:0/4290569616 wait complete. 2023-10-26T05:51:58.317 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:58.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1301288566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 msgr2=0x7f897c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 --2- 172.21.15.71:0/1301288566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c070690 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f897400acd0 tx=0x7f897401ded0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1301288566 shutdown_connections 2023-10-26T05:51:58.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 --2- 172.21.15.71:0/1301288566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c070690 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 --2- 172.21.15.71:0/1301288566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f897c0ac740 0x7f897c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1301288566 >> 172.21.15.71:0/1301288566 conn(0x7f897c069770 msgr2=0x7f897c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1301288566 shutdown_connections 2023-10-26T05:51:58.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1301288566 wait complete. 2023-10-26T05:51:58.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 Processor -- start 2023-10-26T05:51:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- start start 2023-10-26T05:51:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f897c0ac740 0x7f897c135d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c13d420 0x7f897c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b1910 con 0x7f897c15b9f0 2023-10-26T05:51:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b0e80 con 0x7f897c13d420 2023-10-26T05:51:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b1070 con 0x7f897c0ac740 2023-10-26T05:51:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c14db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57838/0 (socket says 172.21.15.71:57838) 2023-10-26T05:51:58.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 -- 172.21.15.71:0/1390518924 learned_addr learned my addr 172.21.15.71:0/1390518924 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8980d2b640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c13d420 0x7f897c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f898152c640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f897c0ac740 0x7f897c135d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 -- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f897c0ac740 msgr2=0x7f897c135d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f897c0ac740 0x7f897c135d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 -- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c13d420 msgr2=0x7f897c0b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c13d420 0x7f897c0b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 -- 172.21.15.71:0/1390518924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897400a930 con 0x7f897c15b9f0 2023-10-26T05:51:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8980d2b640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c13d420 0x7f897c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:51:58.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8981d2d640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c14db50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f89740999c0 tx=0x7f89740af730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974003e50 con 0x7f897c15b9f0 2023-10-26T05:51:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f89740afa10 con 0x7f897c15b9f0 2023-10-26T05:51:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c0b1350 con 0x7f897c15b9f0 2023-10-26T05:51:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740afd80 con 0x7f897c15b9f0 2023-10-26T05:51:58.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.428+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c14f0f0 con 0x7f897c15b9f0 2023-10-26T05:51:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c066b10 con 0x7f897c15b9f0 2023-10-26T05:51:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89740b5040 con 0x7f897c15b9f0 2023-10-26T05:51:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f8971ffb640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f89600664a0 0x7f8960068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f897411f4a0 con 0x7f897c15b9f0 2023-10-26T05:51:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7f896006dfd0 con 0x7f897c15b9f0 2023-10-26T05:51:58.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f898152c640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f89600664a0 0x7f8960068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.432+0000 7f898152c640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f89600664a0 0x7f8960068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8978001380 tx=0x7f8978009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:58.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.436+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89740b3020 con 0x7f897c15b9f0 2023-10-26T05:51:58.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f897c14ef30 con 0x7f897c15b9f0 2023-10-26T05:51:58.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8971ffb640 1 -- 172.21.15.71:0/1390518924 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f89740ec920 con 0x7f897c15b9f0 2023-10-26T05:51:58.617 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:58.618 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:58.618 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:58.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f89600664a0 msgr2=0x7f8960068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f89600664a0 0x7f8960068960 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8978001380 tx=0x7f8978009040 comp rx=0 tx=0).stop 2023-10-26T05:51:58.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 msgr2=0x7f897c14db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c14db50 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f89740999c0 tx=0x7f89740af730 comp rx=0 tx=0).stop 2023-10-26T05:51:58.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 shutdown_connections 2023-10-26T05:51:58.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f89600664a0 0x7f8960068960 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f897c15b9f0 0x7f897c14db50 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f897c13d420 0x7f897c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 --2- 172.21.15.71:0/1390518924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f897c0ac740 0x7f897c135d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 >> 172.21.15.71:0/1390518924 conn(0x7f897c069770 msgr2=0x7f897c13d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 shutdown_connections 2023-10-26T05:51:58.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.612+0000 7f8983790640 1 -- 172.21.15.71:0/1390518924 wait complete. 2023-10-26T05:51:58.635 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:58.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 -- 172.21.15.71:0/1474534943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84815b5c0 msgr2=0x7ff8480ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 --2- 172.21.15.71:0/1474534943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84815b5c0 0x7ff8480ccec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff838009840 tx=0x7ff8480cfdc0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 -- 172.21.15.71:0/1474534943 shutdown_connections 2023-10-26T05:51:58.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 --2- 172.21.15.71:0/1474534943 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff848069770 0x7ff84806a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 --2- 172.21.15.71:0/1474534943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff84815b5c0 0x7ff8480ccec0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 -- 172.21.15.71:0/1474534943 >> 172.21.15.71:0/1474534943 conn(0x7ff8480d12d0 msgr2=0x7ff8480d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 -- 172.21.15.71:0/1474534943 shutdown_connections 2023-10-26T05:51:58.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.740+0000 7ff84d918640 1 -- 172.21.15.71:0/1474534943 wait complete. 2023-10-26T05:51:58.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 Processor -- start 2023-10-26T05:51:58.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- start start 2023-10-26T05:51:58.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 0x7ff848158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff848069bb0 0x7ff84815c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff84815b5c0 0x7ff84815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84815d920 con 0x7ff848069770 2023-10-26T05:51:58.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84815dd40 con 0x7ff848069bb0 2023-10-26T05:51:58.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff848158940 con 0x7ff84815b5c0 2023-10-26T05:51:58.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 0x7ff848158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 0x7ff848158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57850/0 (socket says 172.21.15.71:57850) 2023-10-26T05:51:58.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 -- 172.21.15.71:0/294842493 learned_addr learned my addr 172.21.15.71:0/294842493 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:58.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff8467fc640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff848069bb0 0x7ff84815c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff8477fe640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff84815b5c0 0x7ff84815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 -- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff84815b5c0 msgr2=0x7ff84815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff84815b5c0 0x7ff84815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 -- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff848069bb0 msgr2=0x7ff84815c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff848069bb0 0x7ff84815c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 -- 172.21.15.71:0/294842493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380094a0 con 0x7ff848069770 2023-10-26T05:51:58.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff846ffd640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 0x7ff848158400 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff83801d0a0 tx=0x7ff8380a64d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:58.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8380be070 con 0x7ff848069770 2023-10-26T05:51:58.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff838004040 con 0x7ff848069770 2023-10-26T05:51:58.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84815cca0 con 0x7ff848069770 2023-10-26T05:51:58.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8380043f0 con 0x7ff848069770 2023-10-26T05:51:58.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84815d260 con 0x7ff848069770 2023-10-26T05:51:58.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff848066b10 con 0x7ff848069770 2023-10-26T05:51:58.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff838003600 con 0x7ff848069770 2023-10-26T05:51:58.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff82ffff640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff834066470 0x7ff834068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:58.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.744+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8380aa070 con 0x7ff848069770 2023-10-26T05:51:58.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.748+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7ff83406dfa0 con 0x7ff848069770 2023-10-26T05:51:58.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.748+0000 7ff8467fc640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff834066470 0x7ff834068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:58.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.748+0000 7ff8467fc640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff834066470 0x7ff834068930 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8480cb070 tx=0x7ff83c009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:58.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.748+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff8380ec460 con 0x7ff848069770 2023-10-26T05:51:58.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.904+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff84815db90 con 0x7ff848069770 2023-10-26T05:51:58.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.904+0000 7ff82ffff640 1 -- 172.21.15.71:0/294842493 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16513 (secure 0 0 0) 0x7ff8380ade80 con 0x7ff848069770 2023-10-26T05:51:58.912 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:58.913 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":90,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:58.176503+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff834066470 msgr2=0x7ff834068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff834066470 0x7ff834068930 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8480cb070 tx=0x7ff83c009260 comp rx=0 tx=0).stop 2023-10-26T05:51:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 msgr2=0x7ff848158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 0x7ff848158400 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7ff83801d0a0 tx=0x7ff8380a64d0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 shutdown_connections 2023-10-26T05:51:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff84815b5c0 0x7ff84815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff848069bb0 0x7ff84815c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff834066470 0x7ff834068930 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 --2- 172.21.15.71:0/294842493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff848069770 0x7ff848158400 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:58.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 >> 172.21.15.71:0/294842493 conn(0x7ff8480d12d0 msgr2=0x7ff8480aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:58.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 shutdown_connections 2023-10-26T05:51:58.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:58.908+0000 7ff84d918640 1 -- 172.21.15.71:0/294842493 wait complete. 2023-10-26T05:51:58.930 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:51:58.931 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:59.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- 172.21.15.71:0/4145172556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 msgr2=0x7fcdc006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/4145172556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc006b110 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009bc0 tx=0x7fcdb001cb10 comp rx=0 tx=0).stop 2023-10-26T05:51:59.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- 172.21.15.71:0/4145172556 shutdown_connections 2023-10-26T05:51:59.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/4145172556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc006b110 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- 172.21.15.71:0/4145172556 >> 172.21.15.71:0/4145172556 conn(0x7fcdc00b00e0 msgr2=0x7fcdc00b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:59.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- 172.21.15.71:0/4145172556 shutdown_connections 2023-10-26T05:51:59.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- 172.21.15.71:0/4145172556 wait complete. 2023-10-26T05:51:59.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 Processor -- start 2023-10-26T05:51:59.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- start start 2023-10-26T05:51:59.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc00cc810 0x7fcdc0137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc0141350 0x7fcdc0151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc0156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0153730 con 0x7fcdc015c3b0 2023-10-26T05:51:59.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0152500 con 0x7fcdc00cc810 2023-10-26T05:51:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdc54b0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc01392c0 con 0x7fcdc0141350 2023-10-26T05:51:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc0156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc0156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57868/0 (socket says 172.21.15.71:57868) 2023-10-26T05:51:59.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 -- 172.21.15.71:0/1414243829 learned_addr learned my addr 172.21.15.71:0/1414243829 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 -- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc0141350 msgr2=0x7fcdc0151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc0141350 0x7fcdc0151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 -- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc00cc810 msgr2=0x7fcdc0137660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc00cc810 0x7fcdc0137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.048+0000 7fcdbf7fe640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0009820 con 0x7fcdc015c3b0 2023-10-26T05:51:59.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdbf7fe640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc0156ff0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0019720 tx=0x7fcdb00a5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0004d10 con 0x7fcdc015c3b0 2023-10-26T05:51:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdb0003fe0 con 0x7fcdc015c3b0 2023-10-26T05:51:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0004370 con 0x7fcdc015c3b0 2023-10-26T05:51:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc0157dc0 con 0x7fcdc015c3b0 2023-10-26T05:51:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc0138fe0 con 0x7fcdc015c3b0 2023-10-26T05:51:59.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0139450 con 0x7fcdc015c3b0 2023-10-26T05:51:59.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcdb00b4040 con 0x7fcdc015c3b0 2023-10-26T05:51:59.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcda0066470 0x7fcda0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdb011deb0 con 0x7fcdc015c3b0 2023-10-26T05:51:59.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=91}) v3 -- 0x7fcda006dfa0 con 0x7fcdc015c3b0 2023-10-26T05:51:59.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdbeffd640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcda0066470 0x7fcda0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.052+0000 7fcdbeffd640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcda0066470 0x7fcda0068930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcdc01f5a10 tx=0x7fcdb80079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:59.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.056+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcdb00b2060 con 0x7fcdc015c3b0 2023-10-26T05:51:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.192+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 7 ==== osd_map(91..91 src has 1..91) v4 ==== 1476+0+0 (secure 0 0 0) 0x7fcdb00ef7d0 con 0x7fcdc015c3b0 2023-10-26T05:51:59.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.192+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=92}) v3 -- 0x7fcda006f230 con 0x7fcdc015c3b0 2023-10-26T05:51:59.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc013f630 con 0x7fcdc015c3b0 2023-10-26T05:51:59.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdb77fe640 1 -- 172.21.15.71:0/1414243829 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7fcdc013f630 con 0x7fcdc015c3b0 2023-10-26T05:51:59.243 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:59.243 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:59.243 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:59.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcda0066470 msgr2=0x7fcda0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcda0066470 0x7fcda0068930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcdc01f5a10 tx=0x7fcdb80079c0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 msgr2=0x7fcdc0156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc0156ff0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0019720 tx=0x7fcdb00a5b60 comp rx=0 tx=0).stop 2023-10-26T05:51:59.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 shutdown_connections 2023-10-26T05:51:59.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcda0066470 0x7fcda0068930 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc015c3b0 0x7fcdc0156ff0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc0141350 0x7fcdc0151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 --2- 172.21.15.71:0/1414243829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdc00cc810 0x7fcdc0137660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.236+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 >> 172.21.15.71:0/1414243829 conn(0x7fcdc00b00e0 msgr2=0x7fcdc00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:59.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.240+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 shutdown_connections 2023-10-26T05:51:59.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.240+0000 7fcdc54b0640 1 -- 172.21.15.71:0/1414243829 wait complete. 2023-10-26T05:51:59.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:51:59.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/2743190866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000b2a60 msgr2=0x7f5c000caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 --2- 172.21.15.71:0/2743190866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000b2a60 0x7f5c000caad0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009840 tx=0x7f5bf401c800 comp rx=0 tx=0).stop 2023-10-26T05:51:59.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/2743190866 shutdown_connections 2023-10-26T05:51:59.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 --2- 172.21.15.71:0/2743190866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c000cb010 0x7f5c00159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 --2- 172.21.15.71:0/2743190866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000b2a60 0x7f5c000caad0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/2743190866 >> 172.21.15.71:0/2743190866 conn(0x7f5c00069770 msgr2=0x7f5c00069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:59.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/2743190866 shutdown_connections 2023-10-26T05:51:59.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/2743190866 wait complete. 2023-10-26T05:51:59.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 Processor -- start 2023-10-26T05:51:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- start start 2023-10-26T05:51:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c000b2a60 0x7f5c00153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 0x7f5c00150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c0013c2f0 0x7f5c0015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00133830 con 0x7f5c000cb010 2023-10-26T05:51:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00153700 con 0x7f5c000b2a60 2023-10-26T05:51:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00132f00 con 0x7f5c0013c2f0 2023-10-26T05:51:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 0x7f5c00150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 0x7f5c00150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57872/0 (socket says 172.21.15.71:57872) 2023-10-26T05:51:59.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 -- 172.21.15.71:0/1267764821 learned_addr learned my addr 172.21.15.71:0/1267764821 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:59.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 -- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c0013c2f0 msgr2=0x7f5c0015ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c0013c2f0 0x7f5c0015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 -- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c000b2a60 msgr2=0x7f5c00153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c000b2a60 0x7f5c00153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 -- 172.21.15.71:0/1267764821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf40094a0 con 0x7f5c000cb010 2023-10-26T05:51:59.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfffff640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 0x7f5c00150070 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40b0040 tx=0x7f5bf40a6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:59.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40bf070 con 0x7f5c000cb010 2023-10-26T05:51:59.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5bf4004230 con 0x7f5c000cb010 2023-10-26T05:51:59.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c001331e0 con 0x7f5c000cb010 2023-10-26T05:51:59.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4003600 con 0x7f5c000cb010 2023-10-26T05:51:59.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c00133cc0 con 0x7f5c000cb010 2023-10-26T05:51:59.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.364+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c00066b10 con 0x7f5c000cb010 2023-10-26T05:51:59.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.368+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5bf4003760 con 0x7f5c000cb010 2023-10-26T05:51:59.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.368+0000 7f5bfd7fa640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5bec066470 0x7f5bec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.368+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bf40aa070 con 0x7f5c000cb010 2023-10-26T05:51:59.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.368+0000 7f5c04c25640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5bec066470 0x7f5bec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.368+0000 7f5c04c25640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5bec066470 0x7f5bec068930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0007fd0 tx=0x7f5bf0007940 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:59.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.372+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5bf40ec410 con 0x7f5c000cb010 2023-10-26T05:51:59.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.560+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c00133040 con 0x7f5c000cb010 2023-10-26T05:51:59.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.560+0000 7f5bfd7fa640 1 -- 172.21.15.71:0/1267764821 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f5c00133040 con 0x7f5c000cb010 2023-10-26T05:51:59.568 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:59.568 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:51:17.658710+0000","modified":"2023-10-26T05:51:19.700419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6657},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6657":{"gid":6657,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6833/2132957941","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":2132957941},{"type":"v1","addr":"172.21.15.71:6833","nonce":2132957941}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-26T05:51:59.569 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2023-10-26T05:51:59.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5bec066470 msgr2=0x7f5bec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5bec066470 0x7f5bec068930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0007fd0 tx=0x7f5bf0007940 comp rx=0 tx=0).stop 2023-10-26T05:51:59.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 msgr2=0x7f5c00150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 0x7f5c00150070 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f5bf40b0040 tx=0x7f5bf40a6790 comp rx=0 tx=0).stop 2023-10-26T05:51:59.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 shutdown_connections 2023-10-26T05:51:59.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5c0013c2f0 0x7f5c0015ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5bec066470 0x7f5bec068930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c000cb010 0x7f5c00150070 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 --2- 172.21.15.71:0/1267764821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c000b2a60 0x7f5c00153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 >> 172.21.15.71:0/1267764821 conn(0x7f5c00069770 msgr2=0x7f5c000accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:59.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 shutdown_connections 2023-10-26T05:51:59.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.564+0000 7f5c06e89640 1 -- 172.21.15.71:0/1267764821 wait complete. 2023-10-26T05:51:59.587 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:51:59.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- 172.21.15.71:0/3692926515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413d800 msgr2=0x7f04e415b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/3692926515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413d800 0x7f04e415b4b0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f04e406db30 tx=0x7f04d801c7e0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- 172.21.15.71:0/3692926515 shutdown_connections 2023-10-26T05:51:59.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/3692926515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413d800 0x7f04e415b4b0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/3692926515 >> v2:172.21.15.86:3300/0 conn(0x7f04e413d420 0x7f04e40ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- 172.21.15.71:0/3692926515 >> 172.21.15.71:0/3692926515 conn(0x7f04e4069770 msgr2=0x7f04e4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:59.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- 172.21.15.71:0/3692926515 shutdown_connections 2023-10-26T05:51:59.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- 172.21.15.71:0/3692926515 wait complete. 2023-10-26T05:51:59.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 Processor -- start 2023-10-26T05:51:59.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- start start 2023-10-26T05:51:59.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04e413d420 0x7f04e413bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04e413d800 0x7f04e4156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 0x7f04e41570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e40a64d0 con 0x7f04e413dc50 2023-10-26T05:51:59.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e40a6a90 con 0x7f04e413d800 2023-10-26T05:51:59.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e413c400 con 0x7f04e413d420 2023-10-26T05:51:59.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 0x7f04e41570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 0x7f04e41570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57900/0 (socket says 172.21.15.71:57900) 2023-10-26T05:51:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 -- 172.21.15.71:0/1975510918 learned_addr learned my addr 172.21.15.71:0/1975510918 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:51:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04e98c9640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04e413d420 0x7f04e413bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04e90c8640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04e413d800 0x7f04e4156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 -- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04e413d420 msgr2=0x7f04e413bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04e413d420 0x7f04e413bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 -- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04e413d800 msgr2=0x7f04e4156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04e413d800 0x7f04e4156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 -- 172.21.15.71:0/1975510918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d80094a0 con 0x7f04e413dc50 2023-10-26T05:51:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ea0ca640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 0x7f04e41570c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c3c0 tx=0x7f04e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0011790 con 0x7f04e413dc50 2023-10-26T05:51:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f04e00119e0 con 0x7f04e413dc50 2023-10-26T05:51:59.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.704+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e40a7c10 con 0x7f04e413dc50 2023-10-26T05:51:59.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0011e30 con 0x7f04e413dc50 2023-10-26T05:51:59.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e40a6720 con 0x7f04e413dc50 2023-10-26T05:51:59.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4066b10 con 0x7f04e413dc50 2023-10-26T05:51:59.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f04e001c460 con 0x7f04e413dc50 2023-10-26T05:51:59.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04d67fc640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f04c8066470 0x7f04c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:51:59.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04e0083ed0 con 0x7f04e413dc50 2023-10-26T05:51:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04e98c9640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f04c8066470 0x7f04c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:51:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.708+0000 7f04e98c9640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f04c8066470 0x7f04c8068930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005c50 tx=0x7f04dc005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:51:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.712+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f04e0051350 con 0x7f04e413dc50 2023-10-26T05:51:59.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.860+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04e40a8130 con 0x7f04e413dc50 2023-10-26T05:51:59.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04d67fc640 1 -- 172.21.15.71:0/1975510918 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16484 (secure 0 0 0) 0x7f04e0054970 con 0x7f04e413dc50 2023-10-26T05:51:59.871 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:51:59.871 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":91,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:51:59.184517+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:51:15.888674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-26T05:51:16.655596+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:51:59.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f04c8066470 msgr2=0x7f04c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f04c8066470 0x7f04c8068930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005c50 tx=0x7f04dc005be0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 msgr2=0x7f04e41570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:51:59.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 0x7f04e41570c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c3c0 tx=0x7f04e000c880 comp rx=0 tx=0).stop 2023-10-26T05:51:59.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 shutdown_connections 2023-10-26T05:51:59.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f04c8066470 0x7f04c8068930 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e413dc50 0x7f04e41570c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f04e413d800 0x7f04e4156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 --2- 172.21.15.71:0/1975510918 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f04e413d420 0x7f04e413bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:51:59.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 >> 172.21.15.71:0/1975510918 conn(0x7f04e4069770 msgr2=0x7f04e415a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:51:59.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 shutdown_connections 2023-10-26T05:51:59.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:51:59.864+0000 7f04ebb2d640 1 -- 172.21.15.71:0/1975510918 wait complete. 2023-10-26T05:51:59.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:52:00.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- 172.21.15.71:0/1917068101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c003980 msgr2=0x7f133c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 --2- 172.21.15.71:0/1917068101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c003980 0x7f133c003da0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1344009b60 tx=0x7f134401d250 comp rx=0 tx=0).stop 2023-10-26T05:52:00.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- 172.21.15.71:0/1917068101 shutdown_connections 2023-10-26T05:52:00.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 --2- 172.21.15.71:0/1917068101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f133c003980 0x7f133c003da0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- 172.21.15.71:0/1917068101 >> 172.21.15.71:0/1917068101 conn(0x7f135406b8c0 msgr2=0x7f135406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:00.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- 172.21.15.71:0/1917068101 shutdown_connections 2023-10-26T05:52:00.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- 172.21.15.71:0/1917068101 wait complete. 2023-10-26T05:52:00.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 Processor -- start 2023-10-26T05:52:00.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- start start 2023-10-26T05:52:00.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133c003980 0x7f13540a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 0x7f1354152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13540acbd0 0x7f13540ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13540a6a10 con 0x7f1354158660 2023-10-26T05:52:00.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13540ad5a0 con 0x7f13540acbd0 2023-10-26T05:52:00.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1358faf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13540708c0 con 0x7f133c003980 2023-10-26T05:52:00.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1351d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 0x7f1354152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1351d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 0x7f1354152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57918/0 (socket says 172.21.15.71:57918) 2023-10-26T05:52:00.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.012+0000 7f1351d9b640 1 -- 172.21.15.71:0/1175642062 learned_addr learned my addr 172.21.15.71:0/1175642062 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:00.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f135259c640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133c003980 0x7f13540a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1352d9d640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13540acbd0 0x7f13540ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1351d9b640 1 -- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133c003980 msgr2=0x7f13540a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1351d9b640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133c003980 0x7f13540a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1351d9b640 1 -- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13540acbd0 msgr2=0x7f13540ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1351d9b640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13540acbd0 0x7f13540ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1351d9b640 1 -- 172.21.15.71:0/1175642062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1354070ba0 con 0x7f1354158660 2023-10-26T05:52:00.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1351d9b640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 0x7f1354152f40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1348004d90 tx=0x7f1348001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:00.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13480107d0 con 0x7f1354158660 2023-10-26T05:52:00.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1348010a20 con 0x7f1354158660 2023-10-26T05:52:00.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1354155c10 con 0x7f1354158660 2023-10-26T05:52:00.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348010e30 con 0x7f1354158660 2023-10-26T05:52:00.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1354155900 con 0x7f1354158660 2023-10-26T05:52:00.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1354066b10 con 0x7f1354158660 2023-10-26T05:52:00.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f134801b460 con 0x7f1354158660 2023-10-26T05:52:00.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1342ffd640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1334066470 0x7f1334068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13480847b0 con 0x7f1354158660 2023-10-26T05:52:00.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.016+0000 7f135259c640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1334066470 0x7f1334068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.020+0000 7f135259c640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1334066470 0x7f1334068930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f13440ae000 tx=0x7f1344019ff0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:00.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.020+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1348051ca0 con 0x7f1354158660 2023-10-26T05:52:00.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.212+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f135406c7d0 con 0x7f1354158660 2023-10-26T05:52:00.218 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:52:00.212+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:52:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.216+0000 7f1342ffd640 1 -- 172.21.15.71:0/1175642062 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f13480552c0 con 0x7f1354158660 2023-10-26T05:52:00.222 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:52:00.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1334066470 msgr2=0x7f1334068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1334066470 0x7f1334068930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f13440ae000 tx=0x7f1344019ff0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 msgr2=0x7f1354152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 0x7f1354152f40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1348004d90 tx=0x7f1348001640 comp rx=0 tx=0).stop 2023-10-26T05:52:00.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 shutdown_connections 2023-10-26T05:52:00.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13540acbd0 0x7f13540ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1334066470 0x7f1334068930 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1354158660 0x7f1354152f40 secure :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1348004d90 tx=0x7f1348001640 comp rx=0 tx=0).stop 2023-10-26T05:52:00.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 --2- 172.21.15.71:0/1175642062 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f133c003980 0x7f13540a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 >> 172.21.15.71:0/1175642062 conn(0x7f135406b8c0 msgr2=0x7f135413efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:00.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 shutdown_connections 2023-10-26T05:52:00.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.224+0000 7f1358faf640 1 -- 172.21.15.71:0/1175642062 wait complete. 2023-10-26T05:52:00.247 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:52:00.275 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:52:00.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/3448627109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d4071270 msgr2=0x7fa4d415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/3448627109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d4071270 0x7fa4d415bb60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009840 tx=0x7fa4c401c800 comp rx=0 tx=0).stop 2023-10-26T05:52:00.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/3448627109 shutdown_connections 2023-10-26T05:52:00.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/3448627109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d415c0a0 0x7fa4d406a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/3448627109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d4071270 0x7fa4d415bb60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/3448627109 >> 172.21.15.71:0/3448627109 conn(0x7fa4d40aec60 msgr2=0x7fa4d40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:00.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/3448627109 shutdown_connections 2023-10-26T05:52:00.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/3448627109 wait complete. 2023-10-26T05:52:00.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 Processor -- start 2023-10-26T05:52:00.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- start start 2023-10-26T05:52:00.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d4071270 0x7fa4d413daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d414e7b0 0x7fa4d413fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 0x7fa4d41400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d40ca970 con 0x7fa4d415c0a0 2023-10-26T05:52:00.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d4140760 con 0x7fa4d414e7b0 2023-10-26T05:52:00.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d40ca040 con 0x7fa4d4071270 2023-10-26T05:52:00.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 0x7fa4d41400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 0x7fa4d41400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57954/0 (socket says 172.21.15.71:57954) 2023-10-26T05:52:00.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 -- 172.21.15.71:0/1153700636 learned_addr learned my addr 172.21.15.71:0/1153700636 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:00.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4d94a5640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d414e7b0 0x7fa4d413fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4d9ca6640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d4071270 0x7fa4d413daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 -- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d4071270 msgr2=0x7fa4d413daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d4071270 0x7fa4d413daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 -- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d414e7b0 msgr2=0x7fa4d413fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:00.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d414e7b0 0x7fa4d413fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:00.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 -- 172.21.15.71:0/1153700636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40094a0 con 0x7fa4d415c0a0 2023-10-26T05:52:00.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4da4a7640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 0x7fa4d41400e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009fd0 tx=0x7fa4cc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:00.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00ce20 con 0x7fa4d415c0a0 2023-10-26T05:52:00.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa4cc004b30 con 0x7fa4d415c0a0 2023-10-26T05:52:00.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d40ca320 con 0x7fa4d415c0a0 2023-10-26T05:52:00.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc005020 con 0x7fa4d415c0a0 2023-10-26T05:52:00.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.364+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d4143c50 con 0x7fa4d415c0a0 2023-10-26T05:52:00.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.368+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d4066b10 con 0x7fa4d415c0a0 2023-10-26T05:52:00.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.368+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa4cc005180 con 0x7fa4d415c0a0 2023-10-26T05:52:00.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.368+0000 7fa4ca7fc640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4b8066470 0x7fa4b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:00.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.368+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa4cc014070 con 0x7fa4d415c0a0 2023-10-26T05:52:00.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.368+0000 7fa4d9ca6640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4b8066470 0x7fa4b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:00.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.368+0000 7fa4d9ca6640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4b8066470 0x7fa4b8068930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00046c0 tx=0x7fa4d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:00.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.372+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa4cc052000 con 0x7fa4d415c0a0 2023-10-26T05:52:00.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:00.556+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa4d40721e0 con 0x7fa4d415c0a0 2023-10-26T05:52:01.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4ca7fc640 1 -- 172.21.15.71:0/1153700636 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4cc055620 con 0x7fa4d415c0a0 2023-10-26T05:52:01.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4b8066470 msgr2=0x7fa4b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:01.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4b8066470 0x7fa4b8068930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa4d00046c0 tx=0x7fa4d0009040 comp rx=0 tx=0).stop 2023-10-26T05:52:01.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 msgr2=0x7fa4d41400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:01.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 0x7fa4d41400e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009fd0 tx=0x7fa4cc00ef10 comp rx=0 tx=0).stop 2023-10-26T05:52:01.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 shutdown_connections 2023-10-26T05:52:01.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4b8066470 0x7fa4b8068930 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4d415c0a0 0x7fa4d41400e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4d414e7b0 0x7fa4d413fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 --2- 172.21.15.71:0/1153700636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa4d4071270 0x7fa4d413daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 >> 172.21.15.71:0/1153700636 conn(0x7fa4d40aec60 msgr2=0x7fa4d4159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:01.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 shutdown_connections 2023-10-26T05:52:01.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.220+0000 7fa4dbf0a640 1 -- 172.21.15.71:0/1153700636 wait complete. 2023-10-26T05:52:01.248 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:52:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2067485073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe660005190 msgr2=0x7fe66000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:01.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 --2- 172.21.15.71:0/2067485073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe660005190 0x7fe66000b990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009e30 tx=0x7fe66c01d190 comp rx=0 tx=0).stop 2023-10-26T05:52:01.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2067485073 shutdown_connections 2023-10-26T05:52:01.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 --2- 172.21.15.71:0/2067485073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe660005190 0x7fe66000b990 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2067485073 >> 172.21.15.71:0/2067485073 conn(0x7fe6740aec60 msgr2=0x7fe6740af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:01.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2067485073 shutdown_connections 2023-10-26T05:52:01.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2067485073 wait complete. 2023-10-26T05:52:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 Processor -- start 2023-10-26T05:52:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- start start 2023-10-26T05:52:01.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe660005190 0x7fe674156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 0x7fe6740c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6740c73c0 0x7fe67413c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6740c3550 con 0x7fe6740c5b70 2023-10-26T05:52:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6740c2fc0 con 0x7fe6740c73c0 2023-10-26T05:52:01.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6740c2720 con 0x7fe660005190 2023-10-26T05:52:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 0x7fe6740c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 0x7fe6740c1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57972/0 (socket says 172.21.15.71:57972) 2023-10-26T05:52:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 -- 172.21.15.71:0/2494374357 learned_addr learned my addr 172.21.15.71:0/2494374357 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67a6e1640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6740c73c0 0x7fe67413c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe679ee0640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe660005190 0x7fe674156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:01.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 -- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe660005190 msgr2=0x7fe674156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe660005190 0x7fe674156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 -- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6740c73c0 msgr2=0x7fe67413c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6740c73c0 0x7fe67413c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 -- 172.21.15.71:0/2494374357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6740c2a00 con 0x7fe6740c5b70 2023-10-26T05:52:01.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe6796df640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 0x7fe6740c1ed0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe66400c730 tx=0x7fe66400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:01.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664005270 con 0x7fe6740c5b70 2023-10-26T05:52:01.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe66400a640 con 0x7fe6740c5b70 2023-10-26T05:52:01.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6740c2ce0 con 0x7fe6740c5b70 2023-10-26T05:52:01.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640054c0 con 0x7fe6740c5b70 2023-10-26T05:52:01.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.420+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6740c7850 con 0x7fe6740c5b70 2023-10-26T05:52:01.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.424+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe674066b10 con 0x7fe6740c5b70 2023-10-26T05:52:01.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.424+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe66400ac40 con 0x7fe6740c5b70 2023-10-26T05:52:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.424+0000 7fe66a7fc640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6580664a0 0x7fe658068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.424+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe664083900 con 0x7fe6740c5b70 2023-10-26T05:52:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.424+0000 7fe679ee0640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6580664a0 0x7fe658068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.424+0000 7fe679ee0640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6580664a0 0x7fe658068960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe6741f2c30 tx=0x7fe670009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:01.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.428+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe664050d20 con 0x7fe6740c5b70 2023-10-26T05:52:01.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:01.584+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6740c9770 con 0x7fe6740c5b70 2023-10-26T05:52:02.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.228+0000 7fe66a7fc640 1 -- 172.21.15.71:0/2494374357 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe6740c9770 con 0x7fe6740c5b70 2023-10-26T05:52:02.235 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:52:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.232+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6580664a0 msgr2=0x7fe658068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.232+0000 7fe67c144640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6580664a0 0x7fe658068960 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe6741f2c30 tx=0x7fe670009040 comp rx=0 tx=0).stop 2023-10-26T05:52:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.232+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 msgr2=0x7fe6740c1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:02.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.232+0000 7fe67c144640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 0x7fe6740c1ed0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe66400c730 tx=0x7fe66400cbf0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 shutdown_connections 2023-10-26T05:52:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6740c73c0 0x7fe67413c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6580664a0 0x7fe658068960 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6740c5b70 0x7fe6740c1ed0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 --2- 172.21.15.71:0/2494374357 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe660005190 0x7fe674156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 >> 172.21.15.71:0/2494374357 conn(0x7fe6740aec60 msgr2=0x7fe6740b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:02.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 shutdown_connections 2023-10-26T05:52:02.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.236+0000 7fe67c144640 1 -- 172.21.15.71:0/2494374357 wait complete. 2023-10-26T05:52:02.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:52:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 -- 172.21.15.71:0/3480808921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 msgr2=0x7f37d8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:02.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 --2- 172.21.15.71:0/3480808921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37d8004c50 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f37ec06fc50 tx=0x7f37dc01f6b0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 -- 172.21.15.71:0/3480808921 shutdown_connections 2023-10-26T05:52:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 --2- 172.21.15.71:0/3480808921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37d8004c50 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 -- 172.21.15.71:0/3480808921 >> 172.21.15.71:0/3480808921 conn(0x7f37ec0aec60 msgr2=0x7f37ec0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 -- 172.21.15.71:0/3480808921 shutdown_connections 2023-10-26T05:52:02.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.384+0000 7f37f3763640 1 -- 172.21.15.71:0/3480808921 wait complete. 2023-10-26T05:52:02.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 Processor -- start 2023-10-26T05:52:02.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- start start 2023-10-26T05:52:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37ec0abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec156e10 0x7f37ec0a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec0aa3c0 0x7f37ec0a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0aa840 con 0x7f37d80047d0 2023-10-26T05:52:02.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0cb320 con 0x7f37ec156e10 2023-10-26T05:52:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0adbe0 con 0x7f37ec0aa3c0 2023-10-26T05:52:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37ec0abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37ec0abe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59554/0 (socket says 172.21.15.71:59554) 2023-10-26T05:52:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 -- 172.21.15.71:0/2066552948 learned_addr learned my addr 172.21.15.71:0/2066552948 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f1d00640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec0aa3c0 0x7f37ec0a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:02.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f0cfe640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec156e10 0x7f37ec0a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 -- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec0aa3c0 msgr2=0x7f37ec0a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec0aa3c0 0x7f37ec0a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 -- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec156e10 msgr2=0x7f37ec0a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec156e10 0x7f37ec0a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:02.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 -- 172.21.15.71:0/2066552948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0ae110 con 0x7f37d80047d0 2023-10-26T05:52:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f1d00640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec0aa3c0 0x7f37ec0a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f14ff640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37ec0abe70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e6a0 tx=0x7f37e800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e800ca50 con 0x7f37d80047d0 2023-10-26T05:52:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f37e800cca0 con 0x7f37d80047d0 2023-10-26T05:52:02.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0ade30 con 0x7f37d80047d0 2023-10-26T05:52:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80116d0 con 0x7f37d80047d0 2023-10-26T05:52:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec1566d0 con 0x7f37d80047d0 2023-10-26T05:52:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.388+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec066b10 con 0x7f37d80047d0 2023-10-26T05:52:02.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.392+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f37e8011830 con 0x7f37d80047d0 2023-10-26T05:52:02.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.392+0000 7f37e1ffb640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f37d00664a0 0x7f37d0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.392+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f37e80847e0 con 0x7f37d80047d0 2023-10-26T05:52:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.392+0000 7f37f0cfe640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f37d00664a0 0x7f37d0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.392+0000 7f37f0cfe640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f37d00664a0 0x7f37d0068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0b0000 tx=0x7f37dc0b5040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:02.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.396+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f37e8051dc0 con 0x7f37d80047d0 2023-10-26T05:52:02.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:02.540+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37ec0721e0 con 0x7f37d80047d0 2023-10-26T05:52:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.232+0000 7f37e1ffb640 1 -- 172.21.15.71:0/2066552948 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7f37e80553e0 con 0x7f37d80047d0 2023-10-26T05:52:03.238 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:52:03.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f37d00664a0 msgr2=0x7f37d0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f37d00664a0 0x7f37d0068960 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0b0000 tx=0x7f37dc0b5040 comp rx=0 tx=0).stop 2023-10-26T05:52:03.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 msgr2=0x7f37ec0abe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37ec0abe70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e6a0 tx=0x7f37e800eb60 comp rx=0 tx=0).stop 2023-10-26T05:52:03.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 shutdown_connections 2023-10-26T05:52:03.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37ec0aa3c0 0x7f37ec0a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37ec156e10 0x7f37ec0a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f37d00664a0 0x7f37d0068960 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 --2- 172.21.15.71:0/2066552948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d80047d0 0x7f37ec0abe70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 >> 172.21.15.71:0/2066552948 conn(0x7f37ec0aec60 msgr2=0x7f37ec13d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:03.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 shutdown_connections 2023-10-26T05:52:03.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.236+0000 7f37f3763640 1 -- 172.21.15.71:0/2066552948 wait complete. 2023-10-26T05:52:03.275 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:52:03.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- 172.21.15.71:0/2746878472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec0ca7f0 msgr2=0x7f2fec159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/2746878472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec0ca7f0 0x7f2fec159700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009bc0 tx=0x7f2fdc01cab0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- 172.21.15.71:0/2746878472 shutdown_connections 2023-10-26T05:52:03.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/2746878472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec0ca7f0 0x7f2fec159700 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- 172.21.15.71:0/2746878472 >> 172.21.15.71:0/2746878472 conn(0x7f2fec06b8c0 msgr2=0x7f2fec06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:03.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- 172.21.15.71:0/2746878472 shutdown_connections 2023-10-26T05:52:03.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- 172.21.15.71:0/2746878472 wait complete. 2023-10-26T05:52:03.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 Processor -- start 2023-10-26T05:52:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- start start 2023-10-26T05:52:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 0x7f2fec136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fec0ca7f0 0x7f2fec152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fec0cabd0 0x7f2fec1354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec153160 con 0x7f2fec069d30 2023-10-26T05:52:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec135cf0 con 0x7f2fec0cabd0 2023-10-26T05:52:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2ff24d9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec135fd0 con 0x7f2fec0ca7f0 2023-10-26T05:52:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.392+0000 7f2febfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 0x7f2fec136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 0x7f2fec136d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59570/0 (socket says 172.21.15.71:59570) 2023-10-26T05:52:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 -- 172.21.15.71:0/1667468233 learned_addr learned my addr 172.21.15.71:0/1667468233 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2ff0a76640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fec0cabd0 0x7f2fec1354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 -- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fec0ca7f0 msgr2=0x7f2fec152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2feb7fe640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fec0ca7f0 0x7f2fec152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fec0ca7f0 0x7f2fec152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 -- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fec0cabd0 msgr2=0x7f2fec1354a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fec0cabd0 0x7f2fec1354a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 -- 172.21.15.71:0/1667468233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009820 con 0x7f2fec069d30 2023-10-26T05:52:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2ff0a76640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fec0cabd0 0x7f2fec1354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2febfff640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 0x7f2fec136d50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000f90 tx=0x7f2fdc0a5a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:03.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc005240 con 0x7f2fec069d30 2023-10-26T05:52:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2fdc005490 con 0x7f2fec069d30 2023-10-26T05:52:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc005840 con 0x7f2fec069d30 2023-10-26T05:52:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec1362b0 con 0x7f2fec069d30 2023-10-26T05:52:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec1359e0 con 0x7f2fec069d30 2023-10-26T05:52:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec066b10 con 0x7f2fec069d30 2023-10-26T05:52:03.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2fdc0b7440 con 0x7f2fec069d30 2023-10-26T05:52:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2fe8ff9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2fd8066470 0x7f2fd8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2fdc11e670 con 0x7f2fec069d30 2023-10-26T05:52:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2feb7fe640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2fd8066470 0x7f2fd8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.396+0000 7f2feb7fe640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2fd8066470 0x7f2fd8068930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2fec15a4c0 tx=0x7f2fe0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.400+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2fdc0ebdf0 con 0x7f2fec069d30 2023-10-26T05:52:03.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.548+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f2fec06c7d0 con 0x7f2fec069d30 2023-10-26T05:52:03.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.556+0000 7f2fe8ff9640 1 -- 172.21.15.71:0/1667468233 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v26) v1 ==== 71+0+0 (secure 0 0 0) 0x7f2fdc0b2080 con 0x7f2fec069d30 2023-10-26T05:52:03.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2fd8066470 msgr2=0x7f2fd8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2fd8066470 0x7f2fd8068930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2fec15a4c0 tx=0x7f2fe0009040 comp rx=0 tx=0).stop 2023-10-26T05:52:03.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 msgr2=0x7f2fec136d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 0x7f2fec136d50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000f90 tx=0x7f2fdc0a5a80 comp rx=0 tx=0).stop 2023-10-26T05:52:03.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 shutdown_connections 2023-10-26T05:52:03.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fec0cabd0 0x7f2fec1354a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2fec0ca7f0 0x7f2fec152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2fd8066470 0x7f2fd8068930 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 --2- 172.21.15.71:0/1667468233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2fec069d30 0x7f2fec136d50 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 >> 172.21.15.71:0/1667468233 conn(0x7f2fec06b8c0 msgr2=0x7f2fec141950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:03.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 shutdown_connections 2023-10-26T05:52:03.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.560+0000 7f2ff24d9640 1 -- 172.21.15.71:0/1667468233 wait complete. 2023-10-26T05:52:03.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-10-26T05:52:03.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 -- 172.21.15.71:0/194016976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc15c2d0 msgr2=0x7f6fbc0cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 --2- 172.21.15.71:0/194016976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc15c2d0 0x7f6fbc0cc700 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc071af0 tx=0x7f6fb401cb30 comp rx=0 tx=0).stop 2023-10-26T05:52:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 -- 172.21.15.71:0/194016976 shutdown_connections 2023-10-26T05:52:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 --2- 172.21.15.71:0/194016976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc15c2d0 0x7f6fbc0cc700 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 --2- 172.21.15.71:0/194016976 >> v2:172.21.15.86:3300/0 conn(0x7f6fbc141520 0x7f6fbc06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 -- 172.21.15.71:0/194016976 >> 172.21.15.71:0/194016976 conn(0x7f6fbc0b00e0 msgr2=0x7f6fbc0b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 -- 172.21.15.71:0/194016976 shutdown_connections 2023-10-26T05:52:03.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 -- 172.21.15.71:0/194016976 wait complete. 2023-10-26T05:52:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 Processor -- start 2023-10-26T05:52:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.696+0000 7f6fc4778640 1 -- start start 2023-10-26T05:52:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 0x7f6fbc151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fbc141520 0x7f6fbc1f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fbc15c2d0 0x7f6fbc06e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc1f8590 con 0x7f6fbc0b2390 2023-10-26T05:52:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc13eae0 con 0x7f6fbc15c2d0 2023-10-26T05:52:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc1f87d0 con 0x7f6fbc141520 2023-10-26T05:52:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 0x7f6fbc151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 0x7f6fbc151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59592/0 (socket says 172.21.15.71:59592) 2023-10-26T05:52:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 -- 172.21.15.71:0/3985389950 learned_addr learned my addr 172.21.15.71:0/3985389950 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 -- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fbc141520 msgr2=0x7f6fbc1f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fbc141520 0x7f6fbc1f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 -- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fbc15c2d0 msgr2=0x7f6fbc06e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fbc15c2d0 0x7f6fbc06e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 -- 172.21.15.71:0/3985389950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009820 con 0x7f6fbc0b2390 2023-10-26T05:52:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc2514640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 0x7f6fbc151b10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800d950 tx=0x7f6fb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8009c30 con 0x7f6fbc0b2390 2023-10-26T05:52:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6fb8009e80 con 0x7f6fbc0b2390 2023-10-26T05:52:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc1f8ab0 con 0x7f6fbc0b2390 2023-10-26T05:52:03.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb80106c0 con 0x7f6fbc0b2390 2023-10-26T05:52:03.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc13ed30 con 0x7f6fbc0b2390 2023-10-26T05:52:03.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc066b10 con 0x7f6fbc0b2390 2023-10-26T05:52:03.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6fb8010820 con 0x7f6fbc0b2390 2023-10-26T05:52:03.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fb2ffd640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa0066470 0x7f6fa0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:03.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6fb8084000 con 0x7f6fbc0b2390 2023-10-26T05:52:03.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.700+0000 7f6fc1d13640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa0066470 0x7f6fa0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:03.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.704+0000 7f6fc1d13640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa0066470 0x7f6fa0068930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fac007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:03.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.704+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6fb8051780 con 0x7f6fbc0b2390 2023-10-26T05:52:03.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:03.920+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f6fbc066d00 con 0x7f6fbc0b2390 2023-10-26T05:52:04.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.264+0000 7f6fb2ffd640 1 -- 172.21.15.71:0/3985389950 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v255) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6fb80200c0 con 0x7f6fbc0b2390 2023-10-26T05:52:04.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa0066470 msgr2=0x7f6fa0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa0066470 0x7f6fa0068930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fac007920 comp rx=0 tx=0).stop 2023-10-26T05:52:04.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 msgr2=0x7f6fbc151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 0x7f6fbc151b10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800d950 tx=0x7f6fb800de10 comp rx=0 tx=0).stop 2023-10-26T05:52:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 shutdown_connections 2023-10-26T05:52:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fbc15c2d0 0x7f6fbc06e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fbc141520 0x7f6fbc1f8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa0066470 0x7f6fa0068930 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 --2- 172.21.15.71:0/3985389950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6fbc0b2390 0x7f6fbc151b10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 >> 172.21.15.71:0/3985389950 conn(0x7f6fbc0b00e0 msgr2=0x7f6fbc0aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:04.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 shutdown_connections 2023-10-26T05:52:04.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.268+0000 7f6fc4778640 1 -- 172.21.15.71:0/3985389950 wait complete. 2023-10-26T05:52:04.295 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-26T05:52:04.295 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-10-26T05:52:04.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- 172.21.15.71:0/1296161311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 msgr2=0x7f94dc15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- 172.21.15.71:0/1296161311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc15aff0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f94dc15bfd0 tx=0x7f94cc01c810 comp rx=0 tx=0).stop 2023-10-26T05:52:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- 172.21.15.71:0/1296161311 shutdown_connections 2023-10-26T05:52:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- 172.21.15.71:0/1296161311 >> v2:172.21.15.86:3300/0 conn(0x7f94dc15b530 0x7f94dc157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- 172.21.15.71:0/1296161311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc15aff0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- 172.21.15.71:0/1296161311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc14e300 0x7f94dc0cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- 172.21.15.71:0/1296161311 >> 172.21.15.71:0/1296161311 conn(0x7f94dc069770 msgr2=0x7f94dc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- 172.21.15.71:0/1296161311 shutdown_connections 2023-10-26T05:52:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- 172.21.15.71:0/1296161311 wait complete. 2023-10-26T05:52:04.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 Processor -- start 2023-10-26T05:52:04.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- start start 2023-10-26T05:52:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94dc14e300 0x7f94dc142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc15b530 0x7f94dc159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc149250 con 0x7f94dc14e6e0 2023-10-26T05:52:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1494a0 con 0x7f94dc14e300 2023-10-26T05:52:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc149780 con 0x7f94dc15b530 2023-10-26T05:52:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59616/0 (socket says 172.21.15.71:59616) 2023-10-26T05:52:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 -- 172.21.15.71:0/1462032283 learned_addr learned my addr 172.21.15.71:0/1462032283 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94db7fe640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc15b530 0x7f94dc159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94daffd640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94dc14e300 0x7f94dc142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 -- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc15b530 msgr2=0x7f94dc159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc15b530 0x7f94dc159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 -- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94dc14e300 msgr2=0x7f94dc142c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94dc14e300 0x7f94dc142c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 -- 172.21.15.71:0/1462032283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc009df0 con 0x7f94dc14e6e0 2023-10-26T05:52:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94db7fe640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc15b530 0x7f94dc159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94da7fc640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc143c90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f94cc01aec0 tx=0x7f94cc0a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004380 con 0x7f94dc14e6e0 2023-10-26T05:52:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f94cc005030 con 0x7f94dc14e6e0 2023-10-26T05:52:04.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.420+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc149a60 con 0x7f94dc14e6e0 2023-10-26T05:52:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc005440 con 0x7f94dc14e6e0 2023-10-26T05:52:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc145140 con 0x7f94dc14e6e0 2023-10-26T05:52:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc066b10 con 0x7f94dc14e6e0 2023-10-26T05:52:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f94cc0b4040 con 0x7f94dc14e6e0 2023-10-26T05:52:04.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94c37fe640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f94c8066470 0x7f94c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f94cc11d780 con 0x7f94dc14e6e0 2023-10-26T05:52:04.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94daffd640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f94c8066470 0x7f94c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:04.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.424+0000 7f94daffd640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f94c8066470 0x7f94c8068930 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005fd0 tx=0x7f94d0005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:04.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.428+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f94cc0eaf10 con 0x7f94dc14e6e0 2023-10-26T05:52:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.580+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94dc0721e0 con 0x7f94dc14e6e0 2023-10-26T05:52:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94c37fe640 1 -- 172.21.15.71:0/1462032283 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+12903 (secure 0 0 0) 0x7f94cc0bc090 con 0x7f94dc14e6e0 2023-10-26T05:52:04.589 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:04.589 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:03.232344+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6833/2132957941":"2023-10-27T05:52:00.216331+0000","172.21.15.71:6832/2132957941":"2023-10-27T05:52:00.216331+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:04.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f94c8066470 msgr2=0x7f94c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f94c8066470 0x7f94c8068930 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f94d0005fd0 tx=0x7f94d0005e60 comp rx=0 tx=0).stop 2023-10-26T05:52:04.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 msgr2=0x7f94dc143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc143c90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f94cc01aec0 tx=0x7f94cc0a5780 comp rx=0 tx=0).stop 2023-10-26T05:52:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 shutdown_connections 2023-10-26T05:52:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f94dc15b530 0x7f94dc159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f94c8066470 0x7f94c8068930 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f94dc14e6e0 0x7f94dc143c90 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 --2- 172.21.15.71:0/1462032283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94dc14e300 0x7f94dc142c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 >> 172.21.15.71:0/1462032283 conn(0x7f94dc069770 msgr2=0x7f94dc15a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 shutdown_connections 2023-10-26T05:52:04.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.584+0000 7f94e1567640 1 -- 172.21.15.71:0/1462032283 wait complete. 2023-10-26T05:52:04.608 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:52:04.609 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:52:04.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- 172.21.15.71:0/1768329058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 msgr2=0x7f99f40cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1768329058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f40cb390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009c80 tx=0x7f99ec01cd50 comp rx=0 tx=0).stop 2023-10-26T05:52:04.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- 172.21.15.71:0/1768329058 shutdown_connections 2023-10-26T05:52:04.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1768329058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f40cb390 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1768329058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f4141980 0x7f99f40b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- 172.21.15.71:0/1768329058 >> 172.21.15.71:0/1768329058 conn(0x7f99f40aec60 msgr2=0x7f99f40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:04.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- 172.21.15.71:0/1768329058 shutdown_connections 2023-10-26T05:52:04.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- 172.21.15.71:0/1768329058 wait complete. 2023-10-26T05:52:04.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 Processor -- start 2023-10-26T05:52:04.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- start start 2023-10-26T05:52:04.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f413f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f40b2390 0x7f99f413e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4141980 0x7f99f41f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f4157f70 con 0x7f99f4070680 2023-10-26T05:52:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f413e880 con 0x7f99f4141980 2023-10-26T05:52:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99fb1da640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f41f3dd0 con 0x7f99f40b2390 2023-10-26T05:52:04.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99f8f76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f413f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99f8f76640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f413f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59636/0 (socket says 172.21.15.71:59636) 2023-10-26T05:52:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.732+0000 7f99f8f76640 1 -- 172.21.15.71:0/1005069257 learned_addr learned my addr 172.21.15.71:0/1005069257 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f8f76640 1 -- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f40b2390 msgr2=0x7f99f413e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f8f76640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f40b2390 0x7f99f413e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f8f76640 1 -- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4141980 msgr2=0x7f99f41f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f8f76640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4141980 0x7f99f41f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f8f76640 1 -- 172.21.15.71:0/1005069257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0098e0 con 0x7f99f4070680 2023-10-26T05:52:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f8f76640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f413f8c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f99e400c9f0 tx=0x7f99e400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4007bb0 con 0x7f99f4070680 2023-10-26T05:52:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99e4007e00 con 0x7f99f4070680 2023-10-26T05:52:04.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f413ea70 con 0x7f99f4070680 2023-10-26T05:52:04.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e400f6b0 con 0x7f99f4070680 2023-10-26T05:52:04.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f4161500 con 0x7f99f4070680 2023-10-26T05:52:04.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f4066b10 con 0x7f99f4070680 2023-10-26T05:52:04.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f99e400f810 con 0x7f99f4070680 2023-10-26T05:52:04.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f17fa640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99e0066470 0x7f99e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:04.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f99e4084440 con 0x7f99f4070680 2023-10-26T05:52:04.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f3fff640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99e0066470 0x7f99e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:04.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.736+0000 7f99f3fff640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99e0066470 0x7f99e0068930 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f99e8000f70 tx=0x7f99e80079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:04.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.740+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f99e4051bc0 con 0x7f99f4070680 2023-10-26T05:52:04.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99f40721e0 con 0x7f99f4070680 2023-10-26T05:52:04.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99f17fa640 1 -- 172.21.15.71:0/1005069257 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99e40551e0 con 0x7f99f4070680 2023-10-26T05:52:04.898 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:52:04.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99e0066470 msgr2=0x7f99e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99e0066470 0x7f99e0068930 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f99e8000f70 tx=0x7f99e80079c0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 msgr2=0x7f99f413f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:04.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f413f8c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f99e400c9f0 tx=0x7f99e400ceb0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 shutdown_connections 2023-10-26T05:52:04.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99f4141980 0x7f99f41f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99f40b2390 0x7f99f413e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99e0066470 0x7f99e0068930 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 --2- 172.21.15.71:0/1005069257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99f4070680 0x7f99f413f8c0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 >> 172.21.15.71:0/1005069257 conn(0x7f99f40aec60 msgr2=0x7f99f415a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 shutdown_connections 2023-10-26T05:52:04.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:04.892+0000 7f99fb1da640 1 -- 172.21.15.71:0/1005069257 wait complete. 2023-10-26T05:52:04.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-26T05:52:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/1147692129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8680ac740 msgr2=0x7fc86815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 --2- 172.21.15.71:0/1147692129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8680ac740 0x7fc86815b4b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc8680caa50 tx=0x7fc85801c860 comp rx=0 tx=0).stop 2023-10-26T05:52:05.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/1147692129 shutdown_connections 2023-10-26T05:52:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 --2- 172.21.15.71:0/1147692129 >> v2:172.21.15.86:3300/0 conn(0x7fc86815b9f0 0x7fc868070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 --2- 172.21.15.71:0/1147692129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8680ac740 0x7fc86815b4b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/1147692129 >> 172.21.15.71:0/1147692129 conn(0x7fc868069770 msgr2=0x7fc868069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/1147692129 shutdown_connections 2023-10-26T05:52:05.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/1147692129 wait complete. 2023-10-26T05:52:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 Processor -- start 2023-10-26T05:52:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- start start 2023-10-26T05:52:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8680ac740 0x7fc8680b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc86813d420 0x7fc86814dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 0x7fc868138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680b1ba0 con 0x7fc86815b9f0 2023-10-26T05:52:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86814e190 con 0x7fc8680ac740 2023-10-26T05:52:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680b0f20 con 0x7fc86813d420 2023-10-26T05:52:05.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 0x7fc868138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 0x7fc868138d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59652/0 (socket says 172.21.15.71:59652) 2023-10-26T05:52:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 -- 172.21.15.71:0/2390795140 learned_addr learned my addr 172.21.15.71:0/2390795140 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86caa1640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8680ac740 0x7fc8680b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc85ffff640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc86813d420 0x7fc86814dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 -- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc86813d420 msgr2=0x7fc86814dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc86813d420 0x7fc86814dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 -- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8680ac740 msgr2=0x7fc8680b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8680ac740 0x7fc8680b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 -- 172.21.15.71:0/2390795140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858009560 con 0x7fc86815b9f0 2023-10-26T05:52:05.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86caa1640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8680ac740 0x7fc8680b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:05.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc85ffff640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc86813d420 0x7fc86814dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:05.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86d2a2640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 0x7fc868138d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc86000ca60 tx=0x7fc86000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:05.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860007d90 con 0x7fc86815b9f0 2023-10-26T05:52:05.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc860004b30 con 0x7fc86815b9f0 2023-10-26T05:52:05.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8680b1200 con 0x7fc86815b9f0 2023-10-26T05:52:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860005020 con 0x7fc86815b9f0 2023-10-26T05:52:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.036+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8680b17c0 con 0x7fc86815b9f0 2023-10-26T05:52:05.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.040+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868066b10 con 0x7fc86815b9f0 2023-10-26T05:52:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.040+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc860005180 con 0x7fc86815b9f0 2023-10-26T05:52:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.040+0000 7fc85d7fa640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc854066470 0x7fc854068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.040+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc860084330 con 0x7fc86815b9f0 2023-10-26T05:52:05.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.040+0000 7fc86caa1640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc854066470 0x7fc854068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.040+0000 7fc86caa1640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc854066470 0x7fc854068930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc86814e7f0 tx=0x7fc864009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:05.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.044+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc860051b40 con 0x7fc86815b9f0 2023-10-26T05:52:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.252+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7fc8680b1ce0 con 0x7fc86815b9f0 2023-10-26T05:52:05.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.264+0000 7fc85d7fa640 1 -- 172.21.15.71:0/2390795140 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v256) v1 ==== 140+0+0 (secure 0 0 0) 0x7fc8680b1ce0 con 0x7fc86815b9f0 2023-10-26T05:52:05.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc854066470 msgr2=0x7fc854068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc854066470 0x7fc854068930 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc86814e7f0 tx=0x7fc864009040 comp rx=0 tx=0).stop 2023-10-26T05:52:05.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 msgr2=0x7fc868138d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 0x7fc868138d90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc86000ca60 tx=0x7fc86000cf20 comp rx=0 tx=0).stop 2023-10-26T05:52:05.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 shutdown_connections 2023-10-26T05:52:05.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc854066470 0x7fc854068930 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc86815b9f0 0x7fc868138d90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc86813d420 0x7fc86814dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 --2- 172.21.15.71:0/2390795140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8680ac740 0x7fc8680b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 >> 172.21.15.71:0/2390795140 conn(0x7fc868069770 msgr2=0x7fc86813c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:05.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 shutdown_connections 2023-10-26T05:52:05.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.268+0000 7fc86ed05640 1 -- 172.21.15.71:0/2390795140 wait complete. 2023-10-26T05:52:05.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-10-26T05:52:05.424 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_objectstore":"memstore"} 2023-10-26T05:52:05.424 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-10-26T05:52:05.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:52:05.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 -- 172.21.15.71:0/1728361949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 msgr2=0x7f32ec0aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/1728361949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0aead0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0d0200 tx=0x7f32e401cc70 comp rx=0 tx=0).stop 2023-10-26T05:52:05.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 -- 172.21.15.71:0/1728361949 shutdown_connections 2023-10-26T05:52:05.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/1728361949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0aead0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/1728361949 >> v2:172.21.15.86:3300/0 conn(0x7f32ec0b0420 0x7f32ec0ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 -- 172.21.15.71:0/1728361949 >> 172.21.15.71:0/1728361949 conn(0x7f32ec0b0c80 msgr2=0x7f32ec0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:05.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 -- 172.21.15.71:0/1728361949 shutdown_connections 2023-10-26T05:52:05.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 -- 172.21.15.71:0/1728361949 wait complete. 2023-10-26T05:52:05.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 Processor -- start 2023-10-26T05:52:05.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.548+0000 7f32f3bcd640 1 -- start start 2023-10-26T05:52:05.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32ec0b0420 0x7f32ec163b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ec159210 0x7f32ec0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec164060 con 0x7f32ec0cd240 2023-10-26T05:52:05.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec163230 con 0x7f32ec159210 2023-10-26T05:52:05.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec162380 con 0x7f32ec0b0420 2023-10-26T05:52:05.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59664/0 (socket says 172.21.15.71:59664) 2023-10-26T05:52:05.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 -- 172.21.15.71:0/3790965937 learned_addr learned my addr 172.21.15.71:0/3790965937 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:05.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1969640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32ec0b0420 0x7f32ec163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f216a640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ec159210 0x7f32ec0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 -- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32ec0b0420 msgr2=0x7f32ec163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32ec0b0420 0x7f32ec163b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 -- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ec159210 msgr2=0x7f32ec0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ec159210 0x7f32ec0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 -- 172.21.15.71:0/3790965937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40098e0 con 0x7f32ec0cd240 2023-10-26T05:52:05.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1168640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0c1370 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d980 tx=0x7f32dc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:05.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc009c30 con 0x7f32ec0cd240 2023-10-26T05:52:05.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f32dc005020 con 0x7f32ec0cd240 2023-10-26T05:52:05.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec162620 con 0x7f32ec0cd240 2023-10-26T05:52:05.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc005430 con 0x7f32ec0cd240 2023-10-26T05:52:05.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec162be0 con 0x7f32ec0cd240 2023-10-26T05:52:05.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec066b10 con 0x7f32ec0cd240 2023-10-26T05:52:05.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f32dc0047b0 con 0x7f32ec0cd240 2023-10-26T05:52:05.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32e27fc640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f32d0066470 0x7f32d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f32dc0841b0 con 0x7f32ec0cd240 2023-10-26T05:52:05.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.552+0000 7f32f1969640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f32d0066470 0x7f32d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.556+0000 7f32f1969640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f32d0066470 0x7f32d0068930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32e800b810 tx=0x7f32e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:05.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.556+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32dc051a10 con 0x7f32ec0cd240 2023-10-26T05:52:05.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.708+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f32ec1633c0 con 0x7f32ec0cd240 2023-10-26T05:52:05.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32e27fc640 1 -- 172.21.15.71:0/3790965937 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v26) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f32dc017080 con 0x7f32ec0cd240 2023-10-26T05:52:05.717 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:52:05.717 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:52:05.717 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:52:05.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:52:05.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:52:05.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:52:05.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:52:05.718 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:52:05.719 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:52:05.719 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:52:05.719 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:52:05.719 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:52:05.719 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:52:05.720 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:52:05.720 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:52:05.720 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:52:05.720 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:52:05.720 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:52:05.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f32d0066470 msgr2=0x7f32d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f32d0066470 0x7f32d0068930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32e800b810 tx=0x7f32e8008040 comp rx=0 tx=0).stop 2023-10-26T05:52:05.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 msgr2=0x7f32ec0c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0c1370 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d980 tx=0x7f32dc00de40 comp rx=0 tx=0).stop 2023-10-26T05:52:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 shutdown_connections 2023-10-26T05:52:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ec159210 0x7f32ec0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f32d0066470 0x7f32d0068930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32ec0cd240 0x7f32ec0c1370 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 --2- 172.21.15.71:0/3790965937 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32ec0b0420 0x7f32ec163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 >> 172.21.15.71:0/3790965937 conn(0x7f32ec0b0c80 msgr2=0x7f32ec0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:05.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 shutdown_connections 2023-10-26T05:52:05.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.712+0000 7f32f3bcd640 1 -- 172.21.15.71:0/3790965937 wait complete. 2023-10-26T05:52:05.735 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:52:05.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 -- 172.21.15.71:0/2757143059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 msgr2=0x7fc94013e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 --2- 172.21.15.71:0/2757143059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc94013e850 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc930009840 tx=0x7fc93001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 -- 172.21.15.71:0/2757143059 shutdown_connections 2023-10-26T05:52:05.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 --2- 172.21.15.71:0/2757143059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc94013e850 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 --2- 172.21.15.71:0/2757143059 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94013c120 0x7fc9400cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 -- 172.21.15.71:0/2757143059 >> 172.21.15.71:0/2757143059 conn(0x7fc9400aec60 msgr2=0x7fc9400af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:05.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 -- 172.21.15.71:0/2757143059 shutdown_connections 2023-10-26T05:52:05.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 -- 172.21.15.71:0/2757143059 wait complete. 2023-10-26T05:52:05.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 Processor -- start 2023-10-26T05:52:05.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.856+0000 7fc948729640 1 -- start start 2023-10-26T05:52:05.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc9400b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94013c120 0x7fc9401f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94015a3f0 0x7fc940070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f8a90 con 0x7fc9400cbe00 2023-10-26T05:52:05.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f9010 con 0x7fc94015a3f0 2023-10-26T05:52:05.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9401f99e0 con 0x7fc94013c120 2023-10-26T05:52:05.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc9400b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc9400b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59680/0 (socket says 172.21.15.71:59680) 2023-10-26T05:52:05.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 -- 172.21.15.71:0/786474325 learned_addr learned my addr 172.21.15.71:0/786474325 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:05.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc945cc4640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94013c120 0x7fc9401f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 -- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94013c120 msgr2=0x7fc9401f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94013c120 0x7fc9401f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc946cc6640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94015a3f0 0x7fc940070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 -- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94015a3f0 msgr2=0x7fc940070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:05.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94015a3f0 0x7fc940070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:05.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 -- 172.21.15.71:0/786474325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300094a0 con 0x7fc9400cbe00 2023-10-26T05:52:05.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc9464c5640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc9400b25c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00d950 tx=0x7fc93c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:05.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c009c30 con 0x7fc9400cbe00 2023-10-26T05:52:05.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc93c009e80 con 0x7fc9400cbe00 2023-10-26T05:52:05.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9400710e0 con 0x7fc9400cbe00 2023-10-26T05:52:05.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0106c0 con 0x7fc9400cbe00 2023-10-26T05:52:05.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9401f8ca0 con 0x7fc9400cbe00 2023-10-26T05:52:05.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940066b10 con 0x7fc9400cbe00 2023-10-26T05:52:05.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc93c010820 con 0x7fc9400cbe00 2023-10-26T05:52:05.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc936ffd640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc924066470 0x7fc924068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:05.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc93c084030 con 0x7fc9400cbe00 2023-10-26T05:52:05.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.860+0000 7fc945cc4640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc924066470 0x7fc924068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:05.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.864+0000 7fc945cc4640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc924066470 0x7fc924068930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc9300af000 tx=0x7fc93001b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:05.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:05.864+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc93c0552d0 con 0x7fc9400cbe00 2023-10-26T05:52:06.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.020+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc9400721e0 con 0x7fc9400cbe00 2023-10-26T05:52:06.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.028+0000 7fc936ffd640 1 -- 172.21.15.71:0/786474325 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v27)=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc93c0200c0 con 0x7fc9400cbe00 2023-10-26T05:52:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc924066470 msgr2=0x7fc924068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc924066470 0x7fc924068930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc9300af000 tx=0x7fc93001b010 comp rx=0 tx=0).stop 2023-10-26T05:52:06.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 msgr2=0x7fc9400b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc9400b25c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00d950 tx=0x7fc93c00de10 comp rx=0 tx=0).stop 2023-10-26T05:52:06.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 shutdown_connections 2023-10-26T05:52:06.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc94015a3f0 0x7fc940070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc94013c120 0x7fc9401f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc924066470 0x7fc924068930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 --2- 172.21.15.71:0/786474325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc9400cbe00 0x7fc9400b25c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 >> 172.21.15.71:0/786474325 conn(0x7fc9400aec60 msgr2=0x7fc9401385f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 shutdown_connections 2023-10-26T05:52:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.032+0000 7fc948729640 1 -- 172.21.15.71:0/786474325 wait complete. 2023-10-26T05:52:06.055 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:52:06.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:52:06.110 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:52:06.160 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:52:06.160 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:06.161 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:06.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:52:06.166 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:52:06.167 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:06.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:52:06.173 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:06.173 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:06.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:52:06.215 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:52:06.216 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:06.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:52:06.224 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:52:06.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:06.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/1979042709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd0140cc0 msgr2=0x7f5dd015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 --2- 172.21.15.71:0/1979042709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015c760 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc801c810 comp rx=0 tx=0).stop 2023-10-26T05:52:06.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/1979042709 shutdown_connections 2023-10-26T05:52:06.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 --2- 172.21.15.71:0/1979042709 >> v2:172.21.15.86:3300/0 conn(0x7f5dd0070fe0 0x7f5dd0071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 --2- 172.21.15.71:0/1979042709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015c760 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/1979042709 >> 172.21.15.71:0/1979042709 conn(0x7f5dd0069770 msgr2=0x7f5dd0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/1979042709 shutdown_connections 2023-10-26T05:52:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/1979042709 wait complete. 2023-10-26T05:52:06.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 Processor -- start 2023-10-26T05:52:06.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- start start 2023-10-26T05:52:06.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd0070fe0 0x7f5dd01366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 0x7f5dd0156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0157c30 con 0x7f5dd00b0040 2023-10-26T05:52:06.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd013def0 con 0x7f5dd0140cc0 2023-10-26T05:52:06.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0157670 con 0x7f5dd0070fe0 2023-10-26T05:52:06.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 0x7f5dd0156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 0x7f5dd0156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59692/0 (socket says 172.21.15.71:59692) 2023-10-26T05:52:06.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 -- 172.21.15.71:0/2658680224 learned_addr learned my addr 172.21.15.71:0/2658680224 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:06.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd0070fe0 0x7f5dd01366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dcf7fe640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 -- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd0070fe0 msgr2=0x7f5dd01366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd0070fe0 0x7f5dd01366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 -- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd0140cc0 msgr2=0x7f5dd015a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 -- 172.21.15.71:0/2658680224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8009560 con 0x7f5dd00b0040 2023-10-26T05:52:06.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dcf7fe640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:06.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dce7fc640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 0x7f5dd0156cd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c970 tx=0x7f5db800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:06.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8007bb0 con 0x7f5dd00b0040 2023-10-26T05:52:06.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5db8007e20 con 0x7f5dd00b0040 2023-10-26T05:52:06.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd0157950 con 0x7f5dd00b0040 2023-10-26T05:52:06.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8005280 con 0x7f5dd00b0040 2023-10-26T05:52:06.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.380+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd013e470 con 0x7f5dd00b0040 2023-10-26T05:52:06.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.384+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd0066b10 con 0x7f5dd00b0040 2023-10-26T05:52:06.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.384+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5db80053e0 con 0x7f5dd00b0040 2023-10-26T05:52:06.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.384+0000 7f5dc77fe640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.384+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5db8084490 con 0x7f5dd00b0040 2023-10-26T05:52:06.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.384+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.384+0000 7f5dceffd640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80af000 tx=0x7f5dc801b050 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:06.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.388+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5db8051c00 con 0x7f5dd00b0040 2023-10-26T05:52:06.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dd013e690 con 0x7f5dd00b0040 2023-10-26T05:52:06.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dc77fe640 1 -- 172.21.15.71:0/2658680224 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f5db801d020 con 0x7f5dd00b0040 2023-10-26T05:52:06.574 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:06.575 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1125781144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1125781144},{"type":"v1","addr":"172.21.15.86:6835","nonce":1125781144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5542,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/824918596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":824918596},{"type":"v1","addr":"172.21.15.71:6835","nonce":824918596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5554,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3793497269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3793497269},{"type":"v1","addr":"172.21.15.86:6837","nonce":3793497269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/297266206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":297266206},{"type":"v1","addr":"172.21.15.71:6833","nonce":297266206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-10-26T05:52:06.575 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 34 2023-10-26T05:52:06.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 msgr2=0x7f5db0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80af000 tx=0x7f5dc801b050 comp rx=0 tx=0).stop 2023-10-26T05:52:06.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 msgr2=0x7f5dd0156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 0x7f5dd0156cd0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5db800c970 tx=0x7f5db800ce30 comp rx=0 tx=0).stop 2023-10-26T05:52:06.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 shutdown_connections 2023-10-26T05:52:06.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd0140cc0 0x7f5dd015a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5db0066470 0x7f5db0068930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5dd00b0040 0x7f5dd0156cd0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 --2- 172.21.15.71:0/2658680224 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5dd0070fe0 0x7f5dd01366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 >> 172.21.15.71:0/2658680224 conn(0x7f5dd0069770 msgr2=0x7f5dd006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:06.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 shutdown_connections 2023-10-26T05:52:06.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.568+0000 7f5dd5365640 1 -- 172.21.15.71:0/2658680224 wait complete. 2023-10-26T05:52:06.592 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:52:06.592 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:52:06.592 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:52:06.593 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:52:06.593 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:52:06.593 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:52:06.594 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:52:06.594 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:52:06.594 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:52:06.594 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:52:06.595 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:52:06.595 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:52:06.595 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:52:06.595 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:52:06.596 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:52:06.596 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:52:06.603 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:52:06.604 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:52:06.605 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:52:06.605 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:52:06.609 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:52:06.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:52:06.614 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:52:06.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:52:06.649 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:52:06.650 DEBUG:teuthology.parallel:result is None 2023-10-26T05:52:06.651 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:52:06.651 DEBUG:teuthology.parallel:result is None 2023-10-26T05:52:06.663 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:52:06.664 DEBUG:teuthology.parallel:result is None 2023-10-26T05:52:06.665 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:52:06.665 DEBUG:teuthology.parallel:result is None 2023-10-26T05:52:06.666 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:52:06.711 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:52:06.715 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:52:06.734 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:52:06.736 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:52:06.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- 172.21.15.71:0/3360887604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 msgr2=0x7fe9740b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 --2- 172.21.15.71:0/3360887604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9740b1300 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe968009900 tx=0x7fe96801c7f0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- 172.21.15.71:0/3360887604 shutdown_connections 2023-10-26T05:52:06.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 --2- 172.21.15.71:0/3360887604 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe974069d30 0x7fe97406a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 --2- 172.21.15.71:0/3360887604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9740b1300 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- 172.21.15.71:0/3360887604 >> 172.21.15.71:0/3360887604 conn(0x7fe97406b8c0 msgr2=0x7fe97406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:06.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- 172.21.15.71:0/3360887604 shutdown_connections 2023-10-26T05:52:06.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- 172.21.15.71:0/3360887604 wait complete. 2023-10-26T05:52:06.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 Processor -- start 2023-10-26T05:52:06.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- start start 2023-10-26T05:52:06.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe974069d30 0x7fe9741521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9740ca7f0 0x7fe974070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9741581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe974137970 con 0x7fe974159700 2023-10-26T05:52:06.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9741f83b0 con 0x7fe9740ca7f0 2023-10-26T05:52:06.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97c598640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe974138fe0 con 0x7fe974069d30 2023-10-26T05:52:06.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9741581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9741581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59734/0 (socket says 172.21.15.71:59734) 2023-10-26T05:52:06.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 -- 172.21.15.71:0/1700867374 learned_addr learned my addr 172.21.15.71:0/1700867374 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:06.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97a334640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe974069d30 0x7fe9741521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe979b33640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9740ca7f0 0x7fe974070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 -- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe974069d30 msgr2=0x7fe9741521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe974069d30 0x7fe9741521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 -- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9740ca7f0 msgr2=0x7fe974070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:06.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9740ca7f0 0x7fe974070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:06.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 -- 172.21.15.71:0/1700867374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe968009560 con 0x7fe974159700 2023-10-26T05:52:06.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe97ab35640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9741581d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00db20 tx=0x7fe96c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:06.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.772+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00bea0 con 0x7fe974159700 2023-10-26T05:52:06.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe96c004b30 con 0x7fe974159700 2023-10-26T05:52:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe974139280 con 0x7fe974159700 2023-10-26T05:52:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c005020 con 0x7fe974159700 2023-10-26T05:52:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe974139840 con 0x7fe974159700 2023-10-26T05:52:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe974066b10 con 0x7fe974159700 2023-10-26T05:52:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe96c005180 con 0x7fe974159700 2023-10-26T05:52:06.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe972ffd640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe960066470 0x7fe960068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:06.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe96c083f00 con 0x7fe974159700 2023-10-26T05:52:06.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe97a334640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe960066470 0x7fe960068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.776+0000 7fe97a334640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe960066470 0x7fe960068930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe964007fd0 tx=0x7fe964009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:06.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.780+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe96c0517c0 con 0x7fe974159700 2023-10-26T05:52:06.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:06.944+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe97406c7d0 con 0x7fe974159700 2023-10-26T05:52:08.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.164+0000 7fe972ffd640 1 -- 172.21.15.71:0/1700867374 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe96c017080 con 0x7fe974159700 2023-10-26T05:52:08.173 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:52:08.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe960066470 msgr2=0x7fe960068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe960066470 0x7fe960068930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe964007fd0 tx=0x7fe964009040 comp rx=0 tx=0).stop 2023-10-26T05:52:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 msgr2=0x7fe9741581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9741581d0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00db20 tx=0x7fe96c00b260 comp rx=0 tx=0).stop 2023-10-26T05:52:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 shutdown_connections 2023-10-26T05:52:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe960066470 0x7fe960068930 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe974159700 0x7fe9741581d0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9740ca7f0 0x7fe974070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 --2- 172.21.15.71:0/1700867374 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe974069d30 0x7fe9741521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 >> 172.21.15.71:0/1700867374 conn(0x7fe97406b8c0 msgr2=0x7fe97413e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 shutdown_connections 2023-10-26T05:52:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.168+0000 7fe97c598640 1 -- 172.21.15.71:0/1700867374 wait complete. 2023-10-26T05:52:08.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:52:08.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/3700706412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f415b9f0 msgr2=0x7f29f413d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- 172.21.15.71:0/3700706412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f415b9f0 0x7f29f413d420 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f29f415bfb0 tx=0x7f29ec01cd40 comp rx=0 tx=0).stop 2023-10-26T05:52:08.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/3700706412 shutdown_connections 2023-10-26T05:52:08.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- 172.21.15.71:0/3700706412 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29f406fb80 0x7f29f415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- 172.21.15.71:0/3700706412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f415b9f0 0x7f29f413d420 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- 172.21.15.71:0/3700706412 >> v2:172.21.15.86:3300/0 conn(0x7f29f40ac740 0x7f29f406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/3700706412 >> 172.21.15.71:0/3700706412 conn(0x7f29f4069770 msgr2=0x7f29f4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:08.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/3700706412 shutdown_connections 2023-10-26T05:52:08.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/3700706412 wait complete. 2023-10-26T05:52:08.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 Processor -- start 2023-10-26T05:52:08.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- start start 2023-10-26T05:52:08.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f406fb80 0x7f29f4138890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 0x7f29f40add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29f415b9f0 0x7f29f40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f41524f0 con 0x7f29f40ac740 2023-10-26T05:52:08.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f413b1d0 con 0x7f29f406fb80 2023-10-26T05:52:08.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f413b4b0 con 0x7f29f415b9f0 2023-10-26T05:52:08.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 0x7f29f40add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 0x7f29f40add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59756/0 (socket says 172.21.15.71:59756) 2023-10-26T05:52:08.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 -- 172.21.15.71:0/2139836921 learned_addr learned my addr 172.21.15.71:0/2139836921 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:08.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f9d8c640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f406fb80 0x7f29f4138890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 -- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29f415b9f0 msgr2=0x7f29f40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fa58d640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29f415b9f0 0x7f29f40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29f415b9f0 0x7f29f40b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 -- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f406fb80 msgr2=0x7f29f4138890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f406fb80 0x7f29f4138890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 -- 172.21.15.71:0/2139836921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0098e0 con 0x7f29f40ac740 2023-10-26T05:52:08.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f9d8c640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f406fb80 0x7f29f4138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:08.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29f958b640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 0x7f29f40add30 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f29e400b550 tx=0x7f29e400ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:08.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4010870 con 0x7f29f40ac740 2023-10-26T05:52:08.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f29e4010ae0 con 0x7f29f40ac740 2023-10-26T05:52:08.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f413b790 con 0x7f29f40ac740 2023-10-26T05:52:08.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400f3c0 con 0x7f29f40ac740 2023-10-26T05:52:08.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.368+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f4138dd0 con 0x7f29f40ac740 2023-10-26T05:52:08.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.372+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f4066b10 con 0x7f29f40ac740 2023-10-26T05:52:08.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.372+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f29e400f520 con 0x7f29f40ac740 2023-10-26T05:52:08.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.372+0000 7f29ea7fc640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29d8066470 0x7f29d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.372+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f29e4083e80 con 0x7f29f40ac740 2023-10-26T05:52:08.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.372+0000 7f29f9d8c640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29d8066470 0x7f29d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.372+0000 7f29f9d8c640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29d8066470 0x7f29d8068930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f29f40b2970 tx=0x7f29f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:08.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.376+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f29e4051630 con 0x7f29f40ac740 2023-10-26T05:52:08.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f29f40721e0 con 0x7f29f40ac740 2023-10-26T05:52:08.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29ea7fc640 1 -- 172.21.15.71:0/2139836921 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f29e401e090 con 0x7f29f40ac740 2023-10-26T05:52:08.583 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:08.583 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:08.583 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:52:08.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.583 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:52:08.584 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:52:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:52:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:52:08.585 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.586 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.586 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.586 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:52:08.586 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:52:08.587 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.587 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.587 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:52:08.587 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:52:08.587 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.588 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.588 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.588 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:52:08.588 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:52:08.588 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:52:08.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:52:08.589 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:52:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:52:08.590 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.591 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.591 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:52:08.591 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:52:08.591 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.591 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.592 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:52:08.592 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:52:08.592 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.592 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.593 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.593 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.593 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.594 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:52:08.594 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:52:08.594 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.595 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.595 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.595 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.595 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.596 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.596 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:52:08.596 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:52:08.596 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.596 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.597 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.597 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.597 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.597 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:52:08.598 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:52:08.598 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.598 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.598 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.598 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.599 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.599 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.599 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:52:08.599 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:52:08.600 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.600 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.600 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.600 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.600 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.601 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.601 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:52:08.601 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:52:08.601 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.601 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.602 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.602 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.602 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.602 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.603 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:52:08.603 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:52:08.603 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.603 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.603 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.605 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.605 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:52:08.605 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:52:08.605 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.605 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:52:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:52:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:52:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:52:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.607 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:52:08.608 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:52:08.608 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.608 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.608 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.608 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:52:08.609 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:52:08.609 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.609 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.609 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:52:08.609 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:52:08.610 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:52:08.610 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.610 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.610 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:52:08.611 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:52:08.611 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.611 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.611 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:52:08.611 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:52:08.612 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:52:08.612 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.612 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.612 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:52:08.612 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:52:08.613 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.613 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:52:08.613 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:52:08.613 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:52:08.613 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:52:08.614 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.614 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.616 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:52:08.616 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:52:08.616 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.616 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:52:08.616 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:52:08.617 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:52:08.617 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.617 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.617 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.618 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:52:08.618 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:52:08.618 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.618 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:52:08.618 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:52:08.619 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:08.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:08.620 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:52:08.620 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:52:08.620 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:52:08.620 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:52:08.621 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:52:08.621 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:52:08.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:08.621 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:08.623 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:08.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29d8066470 msgr2=0x7f29d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29d8066470 0x7f29d8068930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f29f40b2970 tx=0x7f29f0008040 comp rx=0 tx=0).stop 2023-10-26T05:52:08.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 msgr2=0x7f29f40add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 0x7f29f40add30 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f29e400b550 tx=0x7f29e400ba10 comp rx=0 tx=0).stop 2023-10-26T05:52:08.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 shutdown_connections 2023-10-26T05:52:08.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29f415b9f0 0x7f29f40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f29d8066470 0x7f29d8068930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f40ac740 0x7f29f40add30 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 --2- 172.21.15.71:0/2139836921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29f406fb80 0x7f29f4138890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 >> 172.21.15.71:0/2139836921 conn(0x7f29f4069770 msgr2=0x7f29f40cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:08.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 shutdown_connections 2023-10-26T05:52:08.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.576+0000 7f29fbff0640 1 -- 172.21.15.71:0/2139836921 wait complete. 2023-10-26T05:52:08.630 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:08.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 -- 172.21.15.71:0/920947139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 msgr2=0x7eff240b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 --2- 172.21.15.71:0/920947139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff240b17d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7eff14009900 tx=0x7eff1401c850 comp rx=0 tx=0).stop 2023-10-26T05:52:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 -- 172.21.15.71:0/920947139 shutdown_connections 2023-10-26T05:52:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 --2- 172.21.15.71:0/920947139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff240b17d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 --2- 172.21.15.71:0/920947139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff24070fe0 0x7eff24071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 -- 172.21.15.71:0/920947139 >> 172.21.15.71:0/920947139 conn(0x7eff24069770 msgr2=0x7eff24069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 -- 172.21.15.71:0/920947139 shutdown_connections 2023-10-26T05:52:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.744+0000 7eff29e07640 1 -- 172.21.15.71:0/920947139 wait complete. 2023-10-26T05:52:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 Processor -- start 2023-10-26T05:52:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- start start 2023-10-26T05:52:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff24070fe0 0x7eff2412f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff2413c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff24140cc0 0x7eff2413ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2412fc40 con 0x7eff240b1360 2023-10-26T05:52:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2415d200 con 0x7eff24070fe0 2023-10-26T05:52:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2415d690 con 0x7eff24140cc0 2023-10-26T05:52:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff2413c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff2413c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59774/0 (socket says 172.21.15.71:59774) 2023-10-26T05:52:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 -- 172.21.15.71:0/2480466277 learned_addr learned my addr 172.21.15.71:0/2480466277 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff237fe640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff24070fe0 0x7eff2412f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 -- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff24140cc0 msgr2=0x7eff2413ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff24140cc0 0x7eff2413ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 -- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff24070fe0 msgr2=0x7eff2412f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff24070fe0 0x7eff2412f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 -- 172.21.15.71:0/2480466277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14009560 con 0x7eff240b1360 2023-10-26T05:52:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff237fe640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff24070fe0 0x7eff2412f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff22ffd640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff2413c4e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d980 tx=0x7eff0c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c009c30 con 0x7eff240b1360 2023-10-26T05:52:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7eff0c005020 con 0x7eff240b1360 2023-10-26T05:52:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2415c5f0 con 0x7eff240b1360 2023-10-26T05:52:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c005430 con 0x7eff240b1360 2023-10-26T05:52:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2415cbb0 con 0x7eff240b1360 2023-10-26T05:52:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff24066b10 con 0x7eff240b1360 2023-10-26T05:52:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.748+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7eff0c0047b0 con 0x7eff240b1360 2023-10-26T05:52:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.752+0000 7eff1bfff640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff04066470 0x7eff04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.752+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7eff0c083da0 con 0x7eff240b1360 2023-10-26T05:52:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.752+0000 7eff237fe640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff04066470 0x7eff04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.752+0000 7eff237fe640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff04066470 0x7eff04068930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7eff1c007e90 tx=0x7eff1c0073a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.756+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7eff0c051560 con 0x7eff240b1360 2023-10-26T05:52:08.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.908+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff240721e0 con 0x7eff240b1360 2023-10-26T05:52:08.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff1bfff640 1 -- 172.21.15.71:0/2480466277 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12774 (secure 0 0 0) 0x7eff0c017020 con 0x7eff240b1360 2023-10-26T05:52:08.918 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:08.918 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":99,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:08.166877+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff04066470 msgr2=0x7eff04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff04066470 0x7eff04068930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7eff1c007e90 tx=0x7eff1c0073a0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 msgr2=0x7eff2413c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff2413c4e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d980 tx=0x7eff0c00de40 comp rx=0 tx=0).stop 2023-10-26T05:52:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 shutdown_connections 2023-10-26T05:52:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7eff24140cc0 0x7eff2413ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff04066470 0x7eff04068930 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff240b1360 0x7eff2413c4e0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 --2- 172.21.15.71:0/2480466277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff24070fe0 0x7eff2412f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 >> 172.21.15.71:0/2480466277 conn(0x7eff24069770 msgr2=0x7eff2406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 shutdown_connections 2023-10-26T05:52:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:08.912+0000 7eff29e07640 1 -- 172.21.15.71:0/2480466277 wait complete. 2023-10-26T05:52:08.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:52:08.938 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:52:09.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/4232036063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 msgr2=0x7f8be4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/4232036063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8be4003da0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4001060 tx=0x7f8bf401cfb0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c02082640 1 -- 172.21.15.71:0/4232036063 reap_dead start 2023-10-26T05:52:09.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/4232036063 shutdown_connections 2023-10-26T05:52:09.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/4232036063 >> 172.21.15.71:0/4232036063 conn(0x7f8bfc0aec60 msgr2=0x7f8bfc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:09.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/4232036063 shutdown_connections 2023-10-26T05:52:09.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/4232036063 wait complete. 2023-10-26T05:52:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 Processor -- start 2023-10-26T05:52:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- start start 2023-10-26T05:52:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8bfc1362a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bfc1357e0 0x7f8bfc1351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bfc1f02d0 0x7f8bfc13a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1505b0 con 0x7f8be4003980 2023-10-26T05:52:09.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc1371b0 con 0x7f8bfc1357e0 2023-10-26T05:52:09.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bfc136910 con 0x7f8bfc1f02d0 2023-10-26T05:52:09.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8bfc1362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8bfc1362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59784/0 (socket says 172.21.15.71:59784) 2023-10-26T05:52:09.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 -- 172.21.15.71:0/3559022451 learned_addr learned my addr 172.21.15.71:0/3559022451 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:09.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c02082640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bfc1f02d0 0x7f8bfc13a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01080640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bfc1357e0 0x7f8bfc1351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 -- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bfc1f02d0 msgr2=0x7f8bfc13a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bfc1f02d0 0x7f8bfc13a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 -- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bfc1357e0 msgr2=0x7f8bfc1351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bfc1357e0 0x7f8bfc1351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 -- 172.21.15.71:0/3559022451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc136bf0 con 0x7f8be4003980 2023-10-26T05:52:09.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c02082640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bfc1f02d0 0x7f8bfc13a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:09.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c01881640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8bfc1362a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4000c10 tx=0x7f8bf4000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:09.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40b4070 con 0x7f8be4003980 2023-10-26T05:52:09.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8bf4001680 con 0x7f8be4003980 2023-10-26T05:52:09.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc136ed0 con 0x7f8be4003980 2023-10-26T05:52:09.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4006330 con 0x7f8be4003980 2023-10-26T05:52:09.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.060+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc1f0750 con 0x7f8be4003980 2023-10-26T05:52:09.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.064+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bfc066b10 con 0x7f8be4003980 2023-10-26T05:52:09.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.064+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8bf4006490 con 0x7f8be4003980 2023-10-26T05:52:09.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.064+0000 7f8bfa7fc640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8be8066470 0x7f8be8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.064+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8bf411d390 con 0x7f8be4003980 2023-10-26T05:52:09.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.064+0000 7f8c01080640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8be8066470 0x7f8be8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.064+0000 7f8c01080640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8be8066470 0x7f8be8068930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009470 tx=0x7f8bf00093c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:09.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.068+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8bf40eab60 con 0x7f8be4003980 2023-10-26T05:52:09.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.216+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bfc1f0a30 con 0x7f8be4003980 2023-10-26T05:52:09.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.216+0000 7f8bfa7fc640 1 -- 172.21.15.71:0/3559022451 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bf40ee180 con 0x7f8be4003980 2023-10-26T05:52:09.224 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:52:09.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8be8066470 msgr2=0x7f8be8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8be8066470 0x7f8be8068930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009470 tx=0x7f8bf00093c0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 msgr2=0x7f8bfc1362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8bfc1362a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4000c10 tx=0x7f8bf4000fb0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 shutdown_connections 2023-10-26T05:52:09.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bfc1f02d0 0x7f8bfc13a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bfc1357e0 0x7f8bfc1351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8be8066470 0x7f8be8068930 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 --2- 172.21.15.71:0/3559022451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8be4003980 0x7f8bfc1362a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 >> 172.21.15.71:0/3559022451 conn(0x7f8bfc0aec60 msgr2=0x7f8bfc152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:09.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 shutdown_connections 2023-10-26T05:52:09.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.220+0000 7f8c03ae5640 1 -- 172.21.15.71:0/3559022451 wait complete. 2023-10-26T05:52:09.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:52:09.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3922210413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1000cede0 msgr2=0x7fe10015ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 --2- 172.21.15.71:0/3922210413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1000cede0 0x7fe10015ce50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe100071290 tx=0x7fe0f401c7e0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3922210413 shutdown_connections 2023-10-26T05:52:09.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 --2- 172.21.15.71:0/3922210413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1000cede0 0x7fe10015ce50 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 --2- 172.21.15.71:0/3922210413 >> v2:172.21.15.86:3300/0 conn(0x7fe10015c720 0x7fe1000b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3922210413 >> 172.21.15.71:0/3922210413 conn(0x7fe1000b0f20 msgr2=0x7fe1000b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:09.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3922210413 shutdown_connections 2023-10-26T05:52:09.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3922210413 wait complete. 2023-10-26T05:52:09.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 Processor -- start 2023-10-26T05:52:09.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- start start 2023-10-26T05:52:09.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1000cede0 0x7fe1000cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 0x7fe1000c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe10015d390 0x7fe100158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001522e0 con 0x7fe10015c720 2023-10-26T05:52:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100145190 con 0x7fe10015d390 2023-10-26T05:52:09.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe100152880 con 0x7fe1000cede0 2023-10-26T05:52:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 0x7fe1000c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 0x7fe1000c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59816/0 (socket says 172.21.15.71:59816) 2023-10-26T05:52:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 -- 172.21.15.71:0/3836987984 learned_addr learned my addr 172.21.15.71:0/3836987984 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0feffd640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1000cede0 0x7fe1000cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0ff7fe640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe10015d390 0x7fe100158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 -- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1000cede0 msgr2=0x7fe1000cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1000cede0 0x7fe1000cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 -- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe10015d390 msgr2=0x7fe100158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe10015d390 0x7fe100158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 -- 172.21.15.71:0/3836987984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009560 con 0x7fe10015c720 2023-10-26T05:52:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0fe7fc640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 0x7fe1000c98b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009850 tx=0x7fe0f40a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4005220 con 0x7fe10015c720 2023-10-26T05:52:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe0f4005380 con 0x7fe10015c720 2023-10-26T05:52:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40046d0 con 0x7fe10015c720 2023-10-26T05:52:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1000cc730 con 0x7fe10015c720 2023-10-26T05:52:09.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.348+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe100144eb0 con 0x7fe10015c720 2023-10-26T05:52:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.352+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe100066d00 con 0x7fe10015c720 2023-10-26T05:52:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.352+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe0f40b4040 con 0x7fe10015c720 2023-10-26T05:52:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.352+0000 7fe0e77fe640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0ec066470 0x7fe0ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.352+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe0f411de60 con 0x7fe10015c720 2023-10-26T05:52:09.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.352+0000 7fe0feffd640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0ec066470 0x7fe0ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.352+0000 7fe0feffd640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0ec066470 0x7fe0ec068930 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000a830 tx=0x7fe0f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:09.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.356+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe0f40eb5a0 con 0x7fe10015c720 2023-10-26T05:52:09.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe10015cbe0 con 0x7fe10015c720 2023-10-26T05:52:09.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe0e77fe640 1 -- 172.21.15.71:0/3836987984 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe0f40b2020 con 0x7fe10015c720 2023-10-26T05:52:09.518 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:09.518 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:52:09.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:09.519 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:52:09.519 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:52:09.519 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:09.519 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:52:09.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0ec066470 msgr2=0x7fe0ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0ec066470 0x7fe0ec068930 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000a830 tx=0x7fe0f0008040 comp rx=0 tx=0).stop 2023-10-26T05:52:09.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 msgr2=0x7fe1000c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 0x7fe1000c98b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009850 tx=0x7fe0f40a5720 comp rx=0 tx=0).stop 2023-10-26T05:52:09.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 shutdown_connections 2023-10-26T05:52:09.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe10015d390 0x7fe100158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe0ec066470 0x7fe0ec068930 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe10015c720 0x7fe1000c98b0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 --2- 172.21.15.71:0/3836987984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe1000cede0 0x7fe1000cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 >> 172.21.15.71:0/3836987984 conn(0x7fe1000b0f20 msgr2=0x7fe1000b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:09.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 shutdown_connections 2023-10-26T05:52:09.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.512+0000 7fe10574e640 1 -- 172.21.15.71:0/3836987984 wait complete. 2023-10-26T05:52:09.537 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:52:09.537 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:52:09.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/2100352461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b413c120 msgr2=0x7fd9b40cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/2100352461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b413c120 0x7fd9b40cb8c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009900 tx=0x7fd9a401c850 comp rx=0 tx=0).stop 2023-10-26T05:52:09.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/2100352461 shutdown_connections 2023-10-26T05:52:09.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/2100352461 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b40cbe00 0x7fd9b413e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/2100352461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b413c120 0x7fd9b40cb8c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/2100352461 >> 172.21.15.71:0/2100352461 conn(0x7fd9b40aec60 msgr2=0x7fd9b40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:09.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/2100352461 shutdown_connections 2023-10-26T05:52:09.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/2100352461 wait complete. 2023-10-26T05:52:09.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 Processor -- start 2023-10-26T05:52:09.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- start start 2023-10-26T05:52:09.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b40cbe00 0x7fd9b4148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b413c120 0x7fd9b4148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 0x7fd9b4149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4137a60 con 0x7fd9b415a3f0 2023-10-26T05:52:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b413d2d0 con 0x7fd9b40cbe00 2023-10-26T05:52:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b8ddb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b413d5b0 con 0x7fd9b413c120 2023-10-26T05:52:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 0x7fd9b4149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 0x7fd9b4149950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59834/0 (socket says 172.21.15.71:59834) 2023-10-26T05:52:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.648+0000 7fd9b2d9d640 1 -- 172.21.15.71:0/1086251601 learned_addr learned my addr 172.21.15.71:0/1086251601 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b1d9b640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b413c120 0x7fd9b4148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b259c640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b40cbe00 0x7fd9b4148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b2d9d640 1 -- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b413c120 msgr2=0x7fd9b4148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b2d9d640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b413c120 0x7fd9b4148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b2d9d640 1 -- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b40cbe00 msgr2=0x7fd9b4148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b2d9d640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b40cbe00 0x7fd9b4148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b2d9d640 1 -- 172.21.15.71:0/1086251601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4009560 con 0x7fd9b415a3f0 2023-10-26T05:52:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b2d9d640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 0x7fd9b4149950 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b9e0 tx=0x7fd9ac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac003b80 con 0x7fd9b415a3f0 2023-10-26T05:52:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd9ac003dd0 con 0x7fd9b415a3f0 2023-10-26T05:52:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b41373a0 con 0x7fd9b415a3f0 2023-10-26T05:52:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00cb80 con 0x7fd9b415a3f0 2023-10-26T05:52:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b4149380 con 0x7fd9b415a3f0 2023-10-26T05:52:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b4066b10 con 0x7fd9b415a3f0 2023-10-26T05:52:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd9ac00cce0 con 0x7fd9b415a3f0 2023-10-26T05:52:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9a2ffd640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd994066470 0x7fd994068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd9ac083a60 con 0x7fd9b415a3f0 2023-10-26T05:52:09.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b259c640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd994066470 0x7fd994068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:09.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.652+0000 7fd9b259c640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd994066470 0x7fd994068930 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40af000 tx=0x7fd9a401b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:09.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.656+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd9ac0512c0 con 0x7fd9b415a3f0 2023-10-26T05:52:09.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:09.796+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9b41495a0 con 0x7fd9b415a3f0 2023-10-26T05:52:10.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.316+0000 7fd9a2ffd640 1 -- 172.21.15.71:0/1086251601 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9b41495a0 con 0x7fd9b415a3f0 2023-10-26T05:52:10.323 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:52:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd994066470 msgr2=0x7fd994068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd994066470 0x7fd994068930 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40af000 tx=0x7fd9a401b090 comp rx=0 tx=0).stop 2023-10-26T05:52:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 msgr2=0x7fd9b4149950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 0x7fd9b4149950 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b9e0 tx=0x7fd9ac00bea0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 shutdown_connections 2023-10-26T05:52:10.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd994066470 0x7fd994068930 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9b415a3f0 0x7fd9b4149950 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd9b413c120 0x7fd9b4148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 --2- 172.21.15.71:0/1086251601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b40cbe00 0x7fd9b4148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 >> 172.21.15.71:0/1086251601 conn(0x7fd9b40aec60 msgr2=0x7fd9b415b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:10.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 shutdown_connections 2023-10-26T05:52:10.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.320+0000 7fd9b8ddb640 1 -- 172.21.15.71:0/1086251601 wait complete. 2023-10-26T05:52:10.348 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:52:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fe7e1640 1 -- 172.21.15.71:0/2307602588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e0003f30 msgr2=0x7f98e00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/2307602588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98e0003f30 0x7f98e00047d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00c260 tx=0x7f98ec01f740 comp rx=0 tx=0).stop 2023-10-26T05:52:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fcd7e640 1 -- 172.21.15.71:0/2307602588 reap_dead start 2023-10-26T05:52:10.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fe7e1640 1 -- 172.21.15.71:0/2307602588 shutdown_connections 2023-10-26T05:52:10.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fe7e1640 1 -- 172.21.15.71:0/2307602588 >> 172.21.15.71:0/2307602588 conn(0x7f98f80aec60 msgr2=0x7f98f80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:10.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fe7e1640 1 -- 172.21.15.71:0/2307602588 shutdown_connections 2023-10-26T05:52:10.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.476+0000 7f98fe7e1640 1 -- 172.21.15.71:0/2307602588 wait complete. 2023-10-26T05:52:10.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 Processor -- start 2023-10-26T05:52:10.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- start start 2023-10-26T05:52:10.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0003f30 0x7f98f81612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:10.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8157740 0x7f98f8157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:10.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 0x7f98f815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:10.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8158100 con 0x7f98f815aba0 2023-10-26T05:52:10.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f815b930 con 0x7f98f8157740 2023-10-26T05:52:10.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f815bc10 con 0x7f98e0003f30 2023-10-26T05:52:10.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 0x7f98f815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:10.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 0x7f98f815b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59852/0 (socket says 172.21.15.71:59852) 2023-10-26T05:52:10.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 -- 172.21.15.71:0/3730214641 learned_addr learned my addr 172.21.15.71:0/3730214641 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:10.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98f7fff640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0003f30 0x7f98f81612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:10.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98f77fe640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8157740 0x7f98f8157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:10.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 -- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0003f30 msgr2=0x7f98f81612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:10.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0003f30 0x7f98f81612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 -- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8157740 msgr2=0x7f98f8157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:10.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8157740 0x7f98f8157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:10.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 -- 172.21.15.71:0/3730214641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f815bef0 con 0x7f98f815aba0 2023-10-26T05:52:10.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fcd7e640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 0x7f98f815b020 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b0e70 tx=0x7f98f000fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:10.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000dce0 con 0x7f98f815aba0 2023-10-26T05:52:10.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98f0011440 con 0x7f98f815aba0 2023-10-26T05:52:10.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f81395a0 con 0x7f98f815aba0 2023-10-26T05:52:10.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0011850 con 0x7f98f815aba0 2023-10-26T05:52:10.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f815b560 con 0x7f98f815aba0 2023-10-26T05:52:10.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f8066b10 con 0x7f98f815aba0 2023-10-26T05:52:10.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.480+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f98f00119b0 con 0x7f98f815aba0 2023-10-26T05:52:10.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.484+0000 7f98f4ff9640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e40664a0 0x7f98e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:10.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.484+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f98f0084820 con 0x7f98f815aba0 2023-10-26T05:52:10.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.484+0000 7f98f7fff640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e40664a0 0x7f98e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:10.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.484+0000 7f98f7fff640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e40664a0 0x7f98e4068960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001170 tx=0x7f98e8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:10.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.488+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f98f0051f20 con 0x7f98f815aba0 2023-10-26T05:52:10.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:10.628+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f98f80721e0 con 0x7f98f815aba0 2023-10-26T05:52:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.332+0000 7f98f4ff9640 1 -- 172.21.15.71:0/3730214641 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f98f0055540 con 0x7f98f815aba0 2023-10-26T05:52:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:52:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e40664a0 msgr2=0x7f98e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e40664a0 0x7f98e4068960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001170 tx=0x7f98e8009040 comp rx=0 tx=0).stop 2023-10-26T05:52:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 msgr2=0x7f98f815b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 0x7f98f815b020 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b0e70 tx=0x7f98f000fe20 comp rx=0 tx=0).stop 2023-10-26T05:52:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 shutdown_connections 2023-10-26T05:52:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f98e40664a0 0x7f98e4068960 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98f815aba0 0x7f98f815b020 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98f8157740 0x7f98f8157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 --2- 172.21.15.71:0/3730214641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98e0003f30 0x7f98f81612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 >> 172.21.15.71:0/3730214641 conn(0x7f98f80aec60 msgr2=0x7f98f80cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 shutdown_connections 2023-10-26T05:52:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.336+0000 7f98fe7e1640 1 -- 172.21.15.71:0/3730214641 wait complete. 2023-10-26T05:52:11.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:52:11.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/3476926536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cdc003fb0 msgr2=0x7f2cdc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:11.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 --2- 172.21.15.71:0/3476926536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cdc003fb0 0x7f2cdc0047d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f2ce800c260 tx=0x7f2ce801f760 comp rx=0 tx=0).stop 2023-10-26T05:52:11.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf37fe640 1 -- 172.21.15.71:0/3476926536 reap_dead start 2023-10-26T05:52:11.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/3476926536 shutdown_connections 2023-10-26T05:52:11.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/3476926536 >> 172.21.15.71:0/3476926536 conn(0x7f2cf40aec60 msgr2=0x7f2cf40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:11.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/3476926536 shutdown_connections 2023-10-26T05:52:11.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/3476926536 wait complete. 2023-10-26T05:52:11.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 Processor -- start 2023-10-26T05:52:11.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- start start 2023-10-26T05:52:11.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cdc003fb0 0x7f2cf4137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:11.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 0x7f2cf415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:11.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf41384d0 0x7f2cf4138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:11.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4138e90 con 0x7f2cf4157a90 2023-10-26T05:52:11.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40c6ef0 con 0x7f2cf41384d0 2023-10-26T05:52:11.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf4138290 con 0x7f2cdc003fb0 2023-10-26T05:52:11.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 0x7f2cf415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 0x7f2cf415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59864/0 (socket says 172.21.15.71:59864) 2023-10-26T05:52:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 -- 172.21.15.71:0/2563029336 learned_addr learned my addr 172.21.15.71:0/2563029336 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf2ffd640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cdc003fb0 0x7f2cf4137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf37fe640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf41384d0 0x7f2cf4138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 -- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cdc003fb0 msgr2=0x7f2cf4137d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cdc003fb0 0x7f2cf4137d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 -- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf41384d0 msgr2=0x7f2cf4138950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf41384d0 0x7f2cf4138950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 -- 172.21.15.71:0/2563029336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40c62e0 con 0x7f2cf4157a90 2023-10-26T05:52:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf37fe640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf41384d0 0x7f2cf4138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf27fc640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 0x7f2cf415b050 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8001940 tx=0x7f2ce8006670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8001740 con 0x7f2cf4157a90 2023-10-26T05:52:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ce8003600 con 0x7f2cf4157a90 2023-10-26T05:52:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf40c65c0 con 0x7f2cf4157a90 2023-10-26T05:52:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8003a30 con 0x7f2cf4157a90 2023-10-26T05:52:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.492+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf40c6b80 con 0x7f2cf4157a90 2023-10-26T05:52:11.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.496+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf4066b10 con 0x7f2cf4157a90 2023-10-26T05:52:11.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.496+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2ce8003b90 con 0x7f2cf4157a90 2023-10-26T05:52:11.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.496+0000 7f2cdbfff640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ce0066470 0x7f2ce0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:11.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.496+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2ce8121ff0 con 0x7f2cf4157a90 2023-10-26T05:52:11.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.496+0000 7f2cf2ffd640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ce0066470 0x7f2ce0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:11.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.496+0000 7f2cf2ffd640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ce0066470 0x7f2ce0068930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4005bd0 tx=0x7f2ce4005a80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:11.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.500+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2ce80ef4d0 con 0x7f2cf4157a90 2023-10-26T05:52:11.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:11.688+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2cf40721e0 con 0x7f2cf4157a90 2023-10-26T05:52:12.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.344+0000 7f2cdbfff640 1 -- 172.21.15.71:0/2563029336 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2ce80f2af0 con 0x7f2cf4157a90 2023-10-26T05:52:12.349 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 12 and data pool 13 2023-10-26T05:52:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ce0066470 msgr2=0x7f2ce0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ce0066470 0x7f2ce0068930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4005bd0 tx=0x7f2ce4005a80 comp rx=0 tx=0).stop 2023-10-26T05:52:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 msgr2=0x7f2cf415b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 0x7f2cf415b050 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8001940 tx=0x7f2ce8006670 comp rx=0 tx=0).stop 2023-10-26T05:52:12.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 shutdown_connections 2023-10-26T05:52:12.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf41384d0 0x7f2cf4138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2ce0066470 0x7f2ce0068930 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf4157a90 0x7f2cf415b050 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 --2- 172.21.15.71:0/2563029336 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cdc003fb0 0x7f2cf4137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 >> 172.21.15.71:0/2563029336 conn(0x7f2cf40aec60 msgr2=0x7f2cf4158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:12.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 shutdown_connections 2023-10-26T05:52:12.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.348+0000 7f2cf9809640 1 -- 172.21.15.71:0/2563029336 wait complete. 2023-10-26T05:52:12.381 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:12.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- 172.21.15.71:0/2435538763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28415b5b0 msgr2=0x7fd2840b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2435538763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28415b5b0 0x7fd2840b4720 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd284072da0 tx=0x7fd27c01cd30 comp rx=0 tx=0).stop 2023-10-26T05:52:12.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- 172.21.15.71:0/2435538763 shutdown_connections 2023-10-26T05:52:12.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2435538763 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2840cbc40 0x7fd28406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2435538763 >> v2:172.21.15.86:3300/0 conn(0x7fd28406b0a0 0x7fd2840d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2435538763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28415b5b0 0x7fd2840b4720 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- 172.21.15.71:0/2435538763 >> 172.21.15.71:0/2435538763 conn(0x7fd2840ce070 msgr2=0x7fd2840ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:12.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- 172.21.15.71:0/2435538763 shutdown_connections 2023-10-26T05:52:12.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- 172.21.15.71:0/2435538763 wait complete. 2023-10-26T05:52:12.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 Processor -- start 2023-10-26T05:52:12.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- start start 2023-10-26T05:52:12.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 0x7fd28415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd28415b5b0 0x7fd2841467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd284155020 0x7fd2841554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28415ae50 con 0x7fd28406b0a0 2023-10-26T05:52:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2841559f0 con 0x7fd284155020 2023-10-26T05:52:12.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd28b08a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2841581e0 con 0x7fd28415b5b0 2023-10-26T05:52:12.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 0x7fd28415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 0x7fd28415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59970/0 (socket says 172.21.15.71:59970) 2023-10-26T05:52:12.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 -- 172.21.15.71:0/2532000154 learned_addr learned my addr 172.21.15.71:0/2532000154 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:12.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd289627640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd284155020 0x7fd2841554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd283fff640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd28415b5b0 0x7fd2841467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 -- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd28415b5b0 msgr2=0x7fd2841467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd28415b5b0 0x7fd2841467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 -- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd284155020 msgr2=0x7fd2841554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd284155020 0x7fd2841554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 -- 172.21.15.71:0/2532000154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c0098e0 con 0x7fd28406b0a0 2023-10-26T05:52:12.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.500+0000 7fd288e26640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 0x7fd28415a910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000f90 tx=0x7fd27c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:12.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c004c50 con 0x7fd28406b0a0 2023-10-26T05:52:12.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd27c004000 con 0x7fd28406b0a0 2023-10-26T05:52:12.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284157bd0 con 0x7fd28406b0a0 2023-10-26T05:52:12.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c004ea0 con 0x7fd28406b0a0 2023-10-26T05:52:12.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd284157e70 con 0x7fd28406b0a0 2023-10-26T05:52:12.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd284066d00 con 0x7fd28406b0a0 2023-10-26T05:52:12.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd27c0b4040 con 0x7fd28406b0a0 2023-10-26T05:52:12.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd2817fa640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd270066470 0x7fd270068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd27c11e2c0 con 0x7fd28406b0a0 2023-10-26T05:52:12.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd283fff640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd270066470 0x7fd270068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.504+0000 7fd283fff640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd270066470 0x7fd270068930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd2780058d0 tx=0x7fd278005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:12.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.508+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd27c0eb750 con 0x7fd28406b0a0 2023-10-26T05:52:12.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.664+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd28406a940 con 0x7fd28406b0a0 2023-10-26T05:52:12.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.664+0000 7fd2817fa640 1 -- 172.21.15.71:0/2532000154 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16500 (secure 0 0 0) 0x7fd27c0eed70 con 0x7fd28406b0a0 2023-10-26T05:52:12.708 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:12.708 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":102,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:11.691622+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:12.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd270066470 msgr2=0x7fd270068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd270066470 0x7fd270068930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd2780058d0 tx=0x7fd278005860 comp rx=0 tx=0).stop 2023-10-26T05:52:12.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 msgr2=0x7fd28415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 0x7fd28415a910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000f90 tx=0x7fd27c009360 comp rx=0 tx=0).stop 2023-10-26T05:52:12.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 shutdown_connections 2023-10-26T05:52:12.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd284155020 0x7fd2841554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd28415b5b0 0x7fd2841467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd270066470 0x7fd270068930 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 --2- 172.21.15.71:0/2532000154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd28406b0a0 0x7fd28415a910 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 >> 172.21.15.71:0/2532000154 conn(0x7fd2840ce070 msgr2=0x7fd28415dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:12.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 shutdown_connections 2023-10-26T05:52:12.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.668+0000 7fd28b08a640 1 -- 172.21.15.71:0/2532000154 wait complete. 2023-10-26T05:52:12.713 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:12.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3884480237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 msgr2=0x7f3d9c166350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3884480237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c166350 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009c80 tx=0x7f3d9401ccb0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3884480237 shutdown_connections 2023-10-26T05:52:12.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3884480237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c166350 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3884480237 >> 172.21.15.71:0/3884480237 conn(0x7f3d9c069770 msgr2=0x7f3d9c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:12.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3884480237 shutdown_connections 2023-10-26T05:52:12.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3884480237 wait complete. 2023-10-26T05:52:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 Processor -- start 2023-10-26T05:52:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- start start 2023-10-26T05:52:12.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d9c0b0750 0x7f3d9c133980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9c13e430 0x7f3d9c132250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c133560 con 0x7f3d9c15c460 2023-10-26T05:52:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0ae9f0 con 0x7f3d9c13e430 2023-10-26T05:52:12.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c1328e0 con 0x7f3d9c0b0750 2023-10-26T05:52:12.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c0ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60002/0 (socket says 172.21.15.71:60002) 2023-10-26T05:52:12.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 -- 172.21.15.71:0/3804928855 learned_addr learned my addr 172.21.15.71:0/3804928855 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:12.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1286640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d9c0b0750 0x7f3d9c133980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da0a85640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9c13e430 0x7f3d9c132250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 -- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d9c0b0750 msgr2=0x7f3d9c133980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d9c0b0750 0x7f3d9c133980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 -- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9c13e430 msgr2=0x7f3d9c132250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:12.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9c13e430 0x7f3d9c132250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:12.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 -- 172.21.15.71:0/3804928855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940098e0 con 0x7f3d9c15c460 2023-10-26T05:52:12.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da0a85640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9c13e430 0x7f3d9c132250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:12.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da1a87640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c0ae4b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3d9401a8e0 tx=0x7f3d940093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:12.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94004410 con 0x7f3d9c15c460 2023-10-26T05:52:12.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d94004660 con 0x7f3d9c15c460 2023-10-26T05:52:12.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c132bc0 con 0x7f3d9c15c460 2023-10-26T05:52:12.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940a5b00 con 0x7f3d9c15c460 2023-10-26T05:52:12.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.828+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c133180 con 0x7f3d9c15c460 2023-10-26T05:52:12.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.832+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c066b10 con 0x7f3d9c15c460 2023-10-26T05:52:12.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.832+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3d940b4040 con 0x7f3d9c15c460 2023-10-26T05:52:12.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.832+0000 7f3d91ffb640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d80066470 0x7f3d80068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:12.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.832+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d9411ed30 con 0x7f3d9c15c460 2023-10-26T05:52:12.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.832+0000 7f3da1286640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d80066470 0x7f3d80068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:12.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.832+0000 7f3da1286640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d80066470 0x7f3d80068930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d980016c0 tx=0x7f3d98009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:12.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.836+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3d940ec1b0 con 0x7f3d9c15c460 2023-10-26T05:52:12.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.988+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d9c066b10 con 0x7f3d9c15c460 2023-10-26T05:52:12.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3d91ffb640 1 -- 172.21.15.71:0/3804928855 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16500 (secure 0 0 0) 0x7f3d940b2020 con 0x7f3d9c15c460 2023-10-26T05:52:12.998 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:12.998 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":102,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:11.691622+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d80066470 msgr2=0x7f3d80068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d80066470 0x7f3d80068930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3d980016c0 tx=0x7f3d98009040 comp rx=0 tx=0).stop 2023-10-26T05:52:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 msgr2=0x7f3d9c0ae4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c0ae4b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3d9401a8e0 tx=0x7f3d940093d0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 shutdown_connections 2023-10-26T05:52:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d80066470 0x7f3d80068930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d9c15c460 0x7f3d9c0ae4b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d9c13e430 0x7f3d9c132250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 --2- 172.21.15.71:0/3804928855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d9c0b0750 0x7f3d9c133980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 >> 172.21.15.71:0/3804928855 conn(0x7f3d9c069770 msgr2=0x7f3d9c069f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 shutdown_connections 2023-10-26T05:52:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:12.992+0000 7f3da34ea640 1 -- 172.21.15.71:0/3804928855 wait complete. 2023-10-26T05:52:13.016 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:52:13.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/3701460999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90380b1230 msgr2=0x7f90380b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 --2- 172.21.15.71:0/3701460999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90380b1230 0x7f90380b1c60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9024009840 tx=0x7f902401eff0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/3701460999 shutdown_connections 2023-10-26T05:52:13.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 --2- 172.21.15.71:0/3701460999 >> v2:172.21.15.86:3300/0 conn(0x7f90380b21a0 0x7f90380ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 --2- 172.21.15.71:0/3701460999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90380b1230 0x7f90380b1c60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/3701460999 >> 172.21.15.71:0/3701460999 conn(0x7f90380aec60 msgr2=0x7f90380af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:13.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/3701460999 shutdown_connections 2023-10-26T05:52:13.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/3701460999 wait complete. 2023-10-26T05:52:13.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 Processor -- start 2023-10-26T05:52:13.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- start start 2023-10-26T05:52:13.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 0x7f903813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90380b21a0 0x7f90381e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9038141a40 0x7f903813f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90381580d0 con 0x7f90380b1230 2023-10-26T05:52:13.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90381491b0 con 0x7f90380b21a0 2023-10-26T05:52:13.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903813b2e0 con 0x7f9038141a40 2023-10-26T05:52:13.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 0x7f903813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 0x7f903813ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60004/0 (socket says 172.21.15.71:60004) 2023-10-26T05:52:13.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 -- 172.21.15.71:0/971752051 learned_addr learned my addr 172.21.15.71:0/971752051 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:13.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903d6ae640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9038141a40 0x7f903813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f9037fff640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90380b21a0 0x7f90381e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 -- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9038141a40 msgr2=0x7f903813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9038141a40 0x7f903813f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 -- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90380b21a0 msgr2=0x7f90381e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90380b21a0 0x7f90381e4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 -- 172.21.15.71:0/971752051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9030009820 con 0x7f90380b1230 2023-10-26T05:52:13.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903cead640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 0x7f903813ada0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f902c00c3a0 tx=0x7f902c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:13.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c004230 con 0x7f90380b1230 2023-10-26T05:52:13.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90240094a0 con 0x7f90380b1230 2023-10-26T05:52:13.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f902c0044a0 con 0x7f90380b1230 2023-10-26T05:52:13.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c005280 con 0x7f90380b1230 2023-10-26T05:52:13.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.128+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90381386c0 con 0x7f90380b1230 2023-10-26T05:52:13.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.132+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9038066b10 con 0x7f90380b1230 2023-10-26T05:52:13.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.132+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f902c0053e0 con 0x7f90380b1230 2023-10-26T05:52:13.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.132+0000 7f90357fa640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f901c066470 0x7f901c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.132+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f902c083340 con 0x7f90380b1230 2023-10-26T05:52:13.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.132+0000 7f9037fff640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f901c066470 0x7f901c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.132+0000 7f9037fff640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f901c066470 0x7f901c068930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f90240b2000 tx=0x7f902401d830 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:13.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.136+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f902c050740 con 0x7f90380b1230 2023-10-26T05:52:13.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.308+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f90380c9770 con 0x7f90380b1230 2023-10-26T05:52:13.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f90357fa640 1 -- 172.21.15.71:0/971752051 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7f90380c9770 con 0x7f90380b1230 2023-10-26T05:52:13.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f901c066470 msgr2=0x7f901c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f901c066470 0x7f901c068930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f90240b2000 tx=0x7f902401d830 comp rx=0 tx=0).stop 2023-10-26T05:52:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 msgr2=0x7f903813ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 0x7f903813ada0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f902c00c3a0 tx=0x7f902c00c860 comp rx=0 tx=0).stop 2023-10-26T05:52:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 shutdown_connections 2023-10-26T05:52:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9038141a40 0x7f903813f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90380b21a0 0x7f90381e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f901c066470 0x7f901c068930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 --2- 172.21.15.71:0/971752051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90380b1230 0x7f903813ada0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 >> 172.21.15.71:0/971752051 conn(0x7f90380aec60 msgr2=0x7f9038140c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:13.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 shutdown_connections 2023-10-26T05:52:13.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.356+0000 7f903f111640 1 -- 172.21.15.71:0/971752051 wait complete. 2023-10-26T05:52:13.380 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:52:13.380 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:52:13.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.496+0000 7f78aa6ea640 1 -- 172.21.15.71:0/3345305206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 msgr2=0x7f78a413e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.496+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/3345305206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a413e850 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f7898009900 tx=0x7f789801c850 comp rx=0 tx=0).stop 2023-10-26T05:52:13.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/3345305206 shutdown_connections 2023-10-26T05:52:13.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/3345305206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a413e850 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/3345305206 >> 172.21.15.71:0/3345305206 conn(0x7f78a40aec60 msgr2=0x7f78a40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:13.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/3345305206 shutdown_connections 2023-10-26T05:52:13.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/3345305206 wait complete. 2023-10-26T05:52:13.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 Processor -- start 2023-10-26T05:52:13.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- start start 2023-10-26T05:52:13.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78a413c120 0x7f78a4135950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a413ed90 0x7f78a4152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a4164300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a41f4730 con 0x7f78a415a3f0 2023-10-26T05:52:13.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4135e90 con 0x7f78a413ed90 2023-10-26T05:52:13.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4152c40 con 0x7f78a413c120 2023-10-26T05:52:13.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a4164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a4164300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60022/0 (socket says 172.21.15.71:60022) 2023-10-26T05:52:13.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 -- 172.21.15.71:0/1521907623 learned_addr learned my addr 172.21.15.71:0/1521907623 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:13.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a37fe640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a413ed90 0x7f78a4152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a3fff640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78a413c120 0x7f78a4135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 -- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78a413c120 msgr2=0x7f78a4135950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78a413c120 0x7f78a4135950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 -- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a413ed90 msgr2=0x7f78a4152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:13.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a413ed90 0x7f78a4152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:13.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 -- 172.21.15.71:0/1521907623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898009560 con 0x7f78a415a3f0 2023-10-26T05:52:13.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a8c87640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a4164300 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c780 tx=0x7f789c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:13.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c003d90 con 0x7f78a415a3f0 2023-10-26T05:52:13.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f789c00ce90 con 0x7f78a415a3f0 2023-10-26T05:52:13.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a4152f20 con 0x7f78a415a3f0 2023-10-26T05:52:13.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c0052d0 con 0x7f78a415a3f0 2023-10-26T05:52:13.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a4153200 con 0x7f78a415a3f0 2023-10-26T05:52:13.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.500+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a4066b10 con 0x7f78a415a3f0 2023-10-26T05:52:13.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.504+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f789c005430 con 0x7f78a415a3f0 2023-10-26T05:52:13.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.504+0000 7f78a0ff9640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7890066470 0x7f7890068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:13.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.504+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f789c0839a0 con 0x7f78a415a3f0 2023-10-26T05:52:13.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.504+0000 7f78a3fff640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7890066470 0x7f7890068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:13.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.504+0000 7f78a3fff640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7890066470 0x7f7890068930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f78a406d6a0 tx=0x7f7894009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:13.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.508+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f789c050e20 con 0x7f78a415a3f0 2023-10-26T05:52:13.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:13.696+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f78a406aeb0 con 0x7f78a415a3f0 2023-10-26T05:52:14.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.364+0000 7f78a0ff9640 1 -- 172.21.15.71:0/1521907623 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f789c017070 con 0x7f78a415a3f0 2023-10-26T05:52:14.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7890066470 msgr2=0x7f7890068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7890066470 0x7f7890068930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f78a406d6a0 tx=0x7f7894009040 comp rx=0 tx=0).stop 2023-10-26T05:52:14.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 msgr2=0x7f78a4164300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a4164300 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f789c00c780 tx=0x7f789c00cc40 comp rx=0 tx=0).stop 2023-10-26T05:52:14.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 shutdown_connections 2023-10-26T05:52:14.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7890066470 0x7f7890068930 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f78a415a3f0 0x7f78a4164300 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78a413ed90 0x7f78a4152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 --2- 172.21.15.71:0/1521907623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78a413c120 0x7f78a4135950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 >> 172.21.15.71:0/1521907623 conn(0x7f78a40aec60 msgr2=0x7f78a40af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:14.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 shutdown_connections 2023-10-26T05:52:14.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.368+0000 7f78aa6ea640 1 -- 172.21.15.71:0/1521907623 wait complete. 2023-10-26T05:52:14.396 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:52:14.397 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:14.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 -- 172.21.15.71:0/3777354838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa900b1360 msgr2=0x7faa900b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 --2- 172.21.15.71:0/3777354838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa900b1360 0x7faa900b17d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7faa88009bc0 tx=0x7faa8801cb50 comp rx=0 tx=0).stop 2023-10-26T05:52:14.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 -- 172.21.15.71:0/3777354838 shutdown_connections 2023-10-26T05:52:14.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 --2- 172.21.15.71:0/3777354838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa900b1360 0x7faa900b17d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 --2- 172.21.15.71:0/3777354838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa90070fe0 0x7faa90071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 -- 172.21.15.71:0/3777354838 >> 172.21.15.71:0/3777354838 conn(0x7faa90069770 msgr2=0x7faa90069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:14.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 -- 172.21.15.71:0/3777354838 shutdown_connections 2023-10-26T05:52:14.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.516+0000 7faa97614640 1 -- 172.21.15.71:0/3777354838 wait complete. 2023-10-26T05:52:14.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 Processor -- start 2023-10-26T05:52:14.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- start start 2023-10-26T05:52:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa90070fe0 0x7faa90157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa900b1360 0x7faa901587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 0x7faa900b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900ad040 con 0x7faa90140cc0 2023-10-26T05:52:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900ad320 con 0x7faa90070fe0 2023-10-26T05:52:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa900ad600 con 0x7faa900b1360 2023-10-26T05:52:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 0x7faa900b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 0x7faa900b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60044/0 (socket says 172.21.15.71:60044) 2023-10-26T05:52:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 -- 172.21.15.71:0/311689157 learned_addr learned my addr 172.21.15.71:0/311689157 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa94baf640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa900b1360 0x7faa901587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa953b0640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa90070fe0 0x7faa90157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 -- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa900b1360 msgr2=0x7faa901587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa900b1360 0x7faa901587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 -- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa90070fe0 msgr2=0x7faa90157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa90070fe0 0x7faa90157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 -- 172.21.15.71:0/311689157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88009820 con 0x7faa90140cc0 2023-10-26T05:52:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa953b0640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa90070fe0 0x7faa90157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa95bb1640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 0x7faa900b0310 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7faa88009b10 tx=0x7faa880a5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88004d10 con 0x7faa90140cc0 2023-10-26T05:52:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa88004e70 con 0x7faa90140cc0 2023-10-26T05:52:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa900ad8e0 con 0x7faa90140cc0 2023-10-26T05:52:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880041e0 con 0x7faa90140cc0 2023-10-26T05:52:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa901580c0 con 0x7faa90140cc0 2023-10-26T05:52:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa90066b10 con 0x7faa90140cc0 2023-10-26T05:52:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.520+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faa880b4040 con 0x7faa90140cc0 2023-10-26T05:52:14.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.524+0000 7faa85ffb640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa74066470 0x7faa74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.524+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa8811e900 con 0x7faa90140cc0 2023-10-26T05:52:14.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.524+0000 7faa953b0640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa74066470 0x7faa74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:14.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.524+0000 7faa953b0640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa74066470 0x7faa74068930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000f70 tx=0x7faa8c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:14.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.528+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faa880ebe10 con 0x7faa90140cc0 2023-10-26T05:52:14.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.700+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa900721e0 con 0x7faa90140cc0 2023-10-26T05:52:14.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.700+0000 7faa85ffb640 1 -- 172.21.15.71:0/311689157 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7faa880b2020 con 0x7faa90140cc0 2023-10-26T05:52:14.707 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:14.707 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:14.708 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:14.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa74066470 msgr2=0x7faa74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa74066470 0x7faa74068930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa8c000f70 tx=0x7faa8c009040 comp rx=0 tx=0).stop 2023-10-26T05:52:14.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 msgr2=0x7faa900b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 0x7faa900b0310 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7faa88009b10 tx=0x7faa880a5b80 comp rx=0 tx=0).stop 2023-10-26T05:52:14.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 shutdown_connections 2023-10-26T05:52:14.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa74066470 0x7faa74068930 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa90140cc0 0x7faa900b0310 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa900b1360 0x7faa901587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 --2- 172.21.15.71:0/311689157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa90070fe0 0x7faa90157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 >> 172.21.15.71:0/311689157 conn(0x7faa90069770 msgr2=0x7faa9013be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 shutdown_connections 2023-10-26T05:52:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.704+0000 7faa97614640 1 -- 172.21.15.71:0/311689157 wait complete. 2023-10-26T05:52:14.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:14.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- 172.21.15.71:0/3271587181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 msgr2=0x7f57a8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/3271587181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8070690 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009bc0 tx=0x7f57a001cb80 comp rx=0 tx=0).stop 2023-10-26T05:52:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- 172.21.15.71:0/3271587181 shutdown_connections 2023-10-26T05:52:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/3271587181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8070690 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/3271587181 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57a813d420 0x7f57a80ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- 172.21.15.71:0/3271587181 >> 172.21.15.71:0/3271587181 conn(0x7f57a8069770 msgr2=0x7f57a8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- 172.21.15.71:0/3271587181 shutdown_connections 2023-10-26T05:52:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- 172.21.15.71:0/3271587181 wait complete. 2023-10-26T05:52:14.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 Processor -- start 2023-10-26T05:52:14.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- start start 2023-10-26T05:52:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57a813d420 0x7f57a814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57a813d800 0x7f57a8159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80ac9e0 con 0x7f57a815bbc0 2023-10-26T05:52:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a814b3e0 con 0x7f57a813d420 2023-10-26T05:52:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8136050 con 0x7f57a813d800 2023-10-26T05:52:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60062/0 (socket says 172.21.15.71:60062) 2023-10-26T05:52:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 -- 172.21.15.71:0/4254658142 learned_addr learned my addr 172.21.15.71:0/4254658142 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 -- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57a813d800 msgr2=0x7f57a8159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57a813d800 0x7f57a8159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 -- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57a813d420 msgr2=0x7f57a814def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57a813d420 0x7f57a814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 -- 172.21.15.71:0/4254658142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a0009820 con 0x7f57a815bbc0 2023-10-26T05:52:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57a759e640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8159a50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f57a001b230 tx=0x7f57a00a5aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0003980 con 0x7f57a815bbc0 2023-10-26T05:52:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8136330 con 0x7f57a815bbc0 2023-10-26T05:52:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f57a00a6e90 con 0x7f57a815bbc0 2023-10-26T05:52:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.840+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a0003e30 con 0x7f57a815bbc0 2023-10-26T05:52:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a8135c20 con 0x7f57a815bbc0 2023-10-26T05:52:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8066b10 con 0x7f57a815bbc0 2023-10-26T05:52:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57a00b4040 con 0x7f57a815bbc0 2023-10-26T05:52:14.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f579ffff640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5788066470 0x7f5788068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:14.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57a011e440 con 0x7f57a815bbc0 2023-10-26T05:52:14.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f57a6d9d640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5788066470 0x7f5788068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:14.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.844+0000 7f57a6d9d640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5788066470 0x7f5788068930 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f579800a9b0 tx=0x7f5798005c80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:14.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.848+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f57a00eb9f0 con 0x7f57a815bbc0 2023-10-26T05:52:14.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.992+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a8135e10 con 0x7f57a815bbc0 2023-10-26T05:52:15.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.992+0000 7f579ffff640 1 -- 172.21.15.71:0/4254658142 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16487 (secure 0 0 0) 0x7f57a8135e10 con 0x7f57a815bbc0 2023-10-26T05:52:15.000 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:15.000 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":103,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:13.139734+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:15.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5788066470 msgr2=0x7f5788068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5788066470 0x7f5788068930 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f579800a9b0 tx=0x7f5798005c80 comp rx=0 tx=0).stop 2023-10-26T05:52:15.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 msgr2=0x7f57a8159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8159a50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f57a001b230 tx=0x7f57a00a5aa0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 shutdown_connections 2023-10-26T05:52:15.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5788066470 0x7f5788068930 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57a815bbc0 0x7f57a8159a50 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57a813d800 0x7f57a8159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 --2- 172.21.15.71:0/4254658142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57a813d420 0x7f57a814def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 >> 172.21.15.71:0/4254658142 conn(0x7f57a8069770 msgr2=0x7f57a815a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 shutdown_connections 2023-10-26T05:52:15.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:14.996+0000 7f57ad0e5640 1 -- 172.21.15.71:0/4254658142 wait complete. 2023-10-26T05:52:15.018 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:52:15.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:52:15.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/22023153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908003f00 msgr2=0x7f49080047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 --2- 172.21.15.71:0/22023153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908003f00 0x7f49080047d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f490c00c290 tx=0x7f490c01f760 comp rx=0 tx=0).stop 2023-10-26T05:52:15.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49220a8640 1 -- 172.21.15.71:0/22023153 reap_dead start 2023-10-26T05:52:15.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/22023153 shutdown_connections 2023-10-26T05:52:15.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/22023153 >> 172.21.15.71:0/22023153 conn(0x7f491c069770 msgr2=0x7f491c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:15.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/22023153 shutdown_connections 2023-10-26T05:52:15.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/22023153 wait complete. 2023-10-26T05:52:15.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 Processor -- start 2023-10-26T05:52:15.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- start start 2023-10-26T05:52:15.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4908003f00 0x7f491c1df4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 0x7f491c1ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f491c1ef980 0x7f491c1dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1f41a0 con 0x7f491c1e1010 2023-10-26T05:52:15.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1f32d0 con 0x7f4908003f00 2023-10-26T05:52:15.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491c1dfa10 con 0x7f491c1ef980 2023-10-26T05:52:15.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 0x7f491c1ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 0x7f491c1ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60070/0 (socket says 172.21.15.71:60070) 2023-10-26T05:52:15.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 -- 172.21.15.71:0/1804506677 learned_addr learned my addr 172.21.15.71:0/1804506677 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:15.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49218a7640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4908003f00 0x7f491c1df4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49220a8640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f491c1ef980 0x7f491c1dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 -- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f491c1ef980 msgr2=0x7f491c1dd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f491c1ef980 0x7f491c1dd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 -- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4908003f00 msgr2=0x7f491c1df4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4908003f00 0x7f491c1df4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 -- 172.21.15.71:0/1804506677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c141800 con 0x7f491c1e1010 2023-10-26T05:52:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49220a8640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f491c1ef980 0x7f491c1dd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:15.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49210a6640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 0x7f491c1ef440 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f490c01dfe0 tx=0x7f490c006650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:15.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c001640 con 0x7f491c1e1010 2023-10-26T05:52:15.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f490c003600 con 0x7f491c1e1010 2023-10-26T05:52:15.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c1f3520 con 0x7f491c1e1010 2023-10-26T05:52:15.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0039d0 con 0x7f491c1e1010 2023-10-26T05:52:15.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.136+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c1f3ae0 con 0x7f491c1e1010 2023-10-26T05:52:15.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.140+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c066b10 con 0x7f491c1e1010 2023-10-26T05:52:15.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.140+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f490c003b30 con 0x7f491c1e1010 2023-10-26T05:52:15.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.140+0000 7f49127fc640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4900066470 0x7f4900068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.140+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f490c122a10 con 0x7f491c1e1010 2023-10-26T05:52:15.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.140+0000 7f49218a7640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4900066470 0x7f4900068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.140+0000 7f49218a7640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4900066470 0x7f4900068930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4918005fd0 tx=0x7f4918005f40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:15.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.144+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f490c0efeb0 con 0x7f491c1e1010 2023-10-26T05:52:15.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.296+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f491c0cba10 con 0x7f4900066470 2023-10-26T05:52:15.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.316+0000 7f49127fc640 1 -- 172.21.15.71:0/1804506677 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272119 (secure 0 0 0) 0x7f4918016070 con 0x7f4900066470 2023-10-26T05:52:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:15.326 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:52:15.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4900066470 msgr2=0x7f4900068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4900066470 0x7f4900068930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4918005fd0 tx=0x7f4918005f40 comp rx=0 tx=0).stop 2023-10-26T05:52:15.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 msgr2=0x7f491c1ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 0x7f491c1ef440 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f490c01dfe0 tx=0x7f490c006650 comp rx=0 tx=0).stop 2023-10-26T05:52:15.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 shutdown_connections 2023-10-26T05:52:15.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f491c1ef980 0x7f491c1dd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4900066470 0x7f4900068930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f491c1e1010 0x7f491c1ef440 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 --2- 172.21.15.71:0/1804506677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4908003f00 0x7f491c1df4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 >> 172.21.15.71:0/1804506677 conn(0x7f491c069770 msgr2=0x7f491c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:15.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 shutdown_connections 2023-10-26T05:52:15.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.320+0000 7f4923b0b640 1 -- 172.21.15.71:0/1804506677 wait complete. 2023-10-26T05:52:15.344 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2023-10-26T05:52:13.509338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557168,"statfs":{"total":1600000000,"available":1594540840,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001138"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:11.320614+0000","last_change":"2023-10-26T05:52:11.320614+0000","last_active":"2023-10-26T05:52:11.320614+0000","last_peered":"2023-10-26T05:52:11.320614+0000","last_clean":"2023-10-26T05:52:11.320614+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:11.320614+0000","last_undegraded":"2023-10-26T05:52:11.320614+0000","last_fullsized":"2023-10-26T05:52:11.320614+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-26T05:52:10.308164+0000","last_change":"2023-10-26T05:52:10.308164+0000","last_active":"2023-10-26T05:52:10.308164+0000","last_peered":"2023-10-26T05:52:10.308164+0000","last_clean":"2023-10-26T05:52:10.308164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:52:10.308164+0000","last_undegraded":"2023-10-26T05:52:10.308164+0000","last_fullsized":"2023-10-26T05:52:10.308164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-26T05:52:07.249229+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:52:07.249229+0000","last_peered":"2023-10-26T05:52:07.249229+0000","last_clean":"2023-10-26T05:52:07.249229+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:52:07.249229+0000","last_undegraded":"2023-10-26T05:52:07.249229+0000","last_fullsized":"2023-10-26T05:52:07.249229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:25.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1098,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194213,"statfs":{"total":200000000,"available":198874829,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.86199999999999999},"min":{"1min":0.58799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.502,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.286,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.218,"5min":1.391,"15min":1.391},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.73499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.3340000000000001,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1639999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.91800000000000004,"15min":0.93600000000000005},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1859999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.04,"5min":0.97599999999999998,"15min":0.97399999999999998},"min":{"1min":0.69199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.242,"5min":1.375,"15min":1.375},"last":0.89700000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.63200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2909999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.95299999999999996,"15min":0.93999999999999995},"min":{"1min":0.748,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2190000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0669999999999999,"15min":1.125},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2370000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.046,"15min":1.089},"min":{"1min":0.69899999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.361,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.92000000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.077,"15min":1.111},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.413,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.091},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0509999999999999,"15min":1.0489999999999999},"min":{"1min":0.79700000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3280000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.05}]}]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194790,"statfs":{"total":200000000,"available":199465197,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.52600000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1100000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.88700000000000001,"15min":0.86599999999999999},"min":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.248,"5min":1.3,"15min":1.3},"last":0.754}]},{"osd":1,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.76900000000000002},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0840000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88,"15min":0.83599999999999997},"min":{"1min":0.58199999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3069999999999999,"5min":1.405,"15min":1.405},"last":0.623}]},{"osd":2,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85499999999999998,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85799999999999998,"15min":0.83199999999999996},"min":{"1min":0.626,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.202,"5min":1.23,"15min":1.23},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.93700000000000006,"15min":0.88500000000000001},"min":{"1min":0.65500000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.262,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0109999999999999,"15min":0.95699999999999996},"min":{"1min":0.70699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.3169999999999999,"5min":1.399,"15min":1.399},"last":0.89500000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98099999999999998,"15min":0.97299999999999998},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.294,"5min":1.365,"15min":1.365},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.04,"15min":1.05},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3480000000000001,"5min":1.554,"15min":1.554},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.96699999999999997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.105},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.161},{"interface":"front","average":{"1min":1.042,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.339,"5min":1.464,"15min":1.464},"last":1.123}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194790,"statfs":{"total":200000000,"available":199465197,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97099999999999997,"15min":0.95499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.93200000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.90000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2130000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.85799999999999998,"15min":0.85699999999999998},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.79600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.89300000000000002,"15min":0.92700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2450000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92300000000000004,"15min":0.93300000000000005},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2709999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.272}]},{"osd":3,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92000000000000004,"15min":0.92400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.409,"5min":1.548,"15min":1.548},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98199999999999998,"15min":1.006},"min":{"1min":0.73399999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4139999999999999,"5min":1.587,"15min":1.587},"last":1.151}]},{"osd":4,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.96899999999999997,"15min":1.01},"min":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.075,"15min":1.1060000000000001},"min":{"1min":0.67300000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.262,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.97499999999999998,"15min":1.032},"min":{"1min":0.65500000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2290000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.008,"15min":1.034},"min":{"1min":0.63100000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.4279999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.056,"15min":1.077},"min":{"1min":0.66800000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.534,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.071,"5min":1.1279999999999999,"15min":1.1699999999999999},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.212}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194790,"statfs":{"total":200000000,"available":199465197,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85399999999999998,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.86499999999999999,"15min":0.877},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.502,"15min":1.502},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1819999999999999,"5min":1.375,"15min":1.375},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.91600000000000004,"15min":0.88900000000000001},"min":{"1min":0.755,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.282,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.93400000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.501,"15min":0.501},"max":{"1min":1.1910000000000001,"5min":1.29,"15min":1.29},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.85999999999999999},"min":{"1min":0.57599999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.226,"5min":1.373,"15min":1.373},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.66500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96499999999999997,"15min":0.93999999999999995},"min":{"1min":0.73199999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3300000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.97099999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":0.999},"min":{"1min":0.64200000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2470000000000001,"5min":1.673,"15min":1.673},"last":1.0740000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0069999999999999,"15min":0.998},"min":{"1min":0.69499999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.383,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.054},{"interface":"front","average":{"1min":0.999,"5min":1.032,"15min":1.0369999999999999},"min":{"1min":0.72899999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4690000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.034},"min":{"1min":0.83599999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4330000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.119,"15min":1.115},"min":{"1min":0.95199999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.403,"5min":1.728,"15min":1.728},"last":1.028}]}]},{"osd":3,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194790,"statfs":{"total":200000000,"available":199465197,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.95099999999999996,"5min":1.155,"15min":1.155},"last":0.624},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.073,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.0680000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64200000000000002,"15min":0.65600000000000003},"min":{"1min":0.54300000000000004,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.038,"5min":1.085,"15min":1.085},"last":0.53100000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.57999999999999996,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.278,"5min":1.367,"15min":1.367},"last":1.3460000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.236,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0189999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.85899999999999999,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.92900000000000005,"15min":0.92700000000000005},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.27,"5min":1.3,"15min":1.3},"last":1.4810000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":0.96599999999999997},"min":{"1min":0.71899999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.248,"5min":1.478,"15min":1.478},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.90000000000000002},"min":{"1min":0.64300000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3100000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.244}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.96899999999999997,"15min":1.004},"min":{"1min":0.66200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":1.155},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0589999999999999,"15min":1.03},"min":{"1min":0.71399999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2330000000000001,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.94099999999999995,"15min":0.91500000000000004},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.383,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.395},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.038,"15min":1.046},"min":{"1min":0.68899999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.4019999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":2.004}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194790,"statfs":{"total":200000000,"available":199465197,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.42399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.64200000000000002,"15min":0.58199999999999996},"min":{"1min":0.50800000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.012,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.54900000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2090000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.58399999999999996,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0740000000000001,"5min":1.294,"15min":1.294},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.77700000000000002,"15min":0.72299999999999998},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.113,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.56799999999999995,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.86499999999999999,"15min":0.81200000000000006},"min":{"1min":0.78200000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.224,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2310000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94799999999999995,"15min":0.89700000000000002},"min":{"1min":0.77700000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.353,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.345},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.92400000000000004,"15min":0.88800000000000001},"min":{"1min":0.77600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4370000000000001,"5min":1.653,"15min":1.653},"last":0.97799999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.95399999999999996,"15min":0.90400000000000003},"min":{"1min":0.88500000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.526,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.05,"15min":0.95699999999999996},"min":{"1min":0.72899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.492,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":1.385}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.91300000000000003,"15min":0.88700000000000001},"min":{"1min":0.84899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.21,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.159,"15min":1.0129999999999999},"min":{"1min":0.91800000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":15.352,"15min":15.352},"last":1.2929999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194790,"statfs":{"total":200000000,"available":199465197,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.311,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0209999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.72999999999999998,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0860000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.165,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0700000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.56899999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1379999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96899999999999997,"15min":0.94499999999999995},"min":{"1min":0.75900000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.286,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.73899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.262,"5min":1.492,"15min":1.492},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.208,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.208,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.296},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.72599999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.23,"5min":1.837,"15min":1.837},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.01,"15min":1.028},"min":{"1min":0.71299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.248,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.095,"15min":1.093},"min":{"1min":0.64900000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.141}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1098,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194213,"statfs":{"total":200000000,"available":198874829,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0189999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.63}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0329999999999999,"5min":1.173,"15min":1.173},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.177},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.97599999999999998,"5min":1.409,"15min":1.409},"last":0.379},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.67500000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.72999999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.877,"15min":0.84399999999999997},"min":{"1min":0.67900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.88900000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.24,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.79400000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.92100000000000004},"min":{"1min":0.627,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.28,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.22,"5min":1.27,"15min":1.27},"last":0.70799999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92000000000000004,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.02,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.764,"15min":1.764},"last":0.84599999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98799999999999999,"15min":0.99299999999999999},"min":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5780000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.288,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.82899999999999996}]}]}],"pool_statfs":[]}} 2023-10-26T05:52:15.350 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:52:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/1050799205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 msgr2=0x7f8b2c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 --2- 172.21.15.71:0/1050799205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b2c004c50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800bb10 tx=0x7f8b3801cdc0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b42d9d640 1 -- 172.21.15.71:0/1050799205 reap_dead start 2023-10-26T05:52:15.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/1050799205 shutdown_connections 2023-10-26T05:52:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/1050799205 >> 172.21.15.71:0/1050799205 conn(0x7f8b44069770 msgr2=0x7f8b44069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/1050799205 shutdown_connections 2023-10-26T05:52:15.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/1050799205 wait complete. 2023-10-26T05:52:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 Processor -- start 2023-10-26T05:52:15.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- start start 2023-10-26T05:52:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b441f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b44139190 0x7f8b441ef2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b441f1f70 0x7f8b4413aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441f2f10 con 0x7f8b2c0047d0 2023-10-26T05:52:15.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441f6440 con 0x7f8b44139190 2023-10-26T05:52:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b441f2950 con 0x7f8b441f1f70 2023-10-26T05:52:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b441f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b441f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60082/0 (socket says 172.21.15.71:60082) 2023-10-26T05:52:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 -- 172.21.15.71:0/3317113250 learned_addr learned my addr 172.21.15.71:0/3317113250 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:15.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b42d9d640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b441f1f70 0x7f8b4413aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b41d9b640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b44139190 0x7f8b441ef2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 -- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b441f1f70 msgr2=0x7f8b4413aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b441f1f70 0x7f8b4413aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 -- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b44139190 msgr2=0x7f8b441ef2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b44139190 0x7f8b441ef2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 -- 172.21.15.71:0/3317113250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b441f2c30 con 0x7f8b2c0047d0 2023-10-26T05:52:15.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b41d9b640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b44139190 0x7f8b441ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:15.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b4259c640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b441f5f00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8b34011a80 tx=0x7f8b34011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:15.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3400fe10 con 0x7f8b2c0047d0 2023-10-26T05:52:15.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8b34012450 con 0x7f8b2c0047d0 2023-10-26T05:52:15.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b441f5700 con 0x7f8b2c0047d0 2023-10-26T05:52:15.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34012840 con 0x7f8b2c0047d0 2023-10-26T05:52:15.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.504+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b441ef810 con 0x7f8b2c0047d0 2023-10-26T05:52:15.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.508+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b44066b10 con 0x7f8b2c0047d0 2023-10-26T05:52:15.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.508+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8b340129a0 con 0x7f8b2c0047d0 2023-10-26T05:52:15.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.508+0000 7f8b32ffd640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8b24066470 0x7f8b24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.508+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b34084de0 con 0x7f8b2c0047d0 2023-10-26T05:52:15.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.508+0000 7f8b41d9b640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8b24066470 0x7f8b24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.508+0000 7f8b41d9b640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8b24066470 0x7f8b24068930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8b380b0000 tx=0x7f8b38019b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:15.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.512+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8b34052290 con 0x7f8b2c0047d0 2023-10-26T05:52:15.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:52:15.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:52:15.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.664+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b44070b90 con 0x7f8b24066470 2023-10-26T05:52:15.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.684+0000 7f8b32ffd640 1 -- 172.21.15.71:0/3317113250 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283685 (secure 0 0 0) 0x7f8b380a5c70 con 0x7f8b24066470 2023-10-26T05:52:15.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:15.696 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:52:15.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8b24066470 msgr2=0x7f8b24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8b24066470 0x7f8b24068930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8b380b0000 tx=0x7f8b38019b60 comp rx=0 tx=0).stop 2023-10-26T05:52:15.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 msgr2=0x7f8b441f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b441f5f00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8b34011a80 tx=0x7f8b34011f40 comp rx=0 tx=0).stop 2023-10-26T05:52:15.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 shutdown_connections 2023-10-26T05:52:15.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8b441f1f70 0x7f8b4413aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b44139190 0x7f8b441ef2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8b24066470 0x7f8b24068930 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 --2- 172.21.15.71:0/3317113250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b2c0047d0 0x7f8b441f5f00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 >> 172.21.15.71:0/3317113250 conn(0x7f8b44069770 msgr2=0x7f8b4406a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:15.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 shutdown_connections 2023-10-26T05:52:15.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.688+0000 7f8b48810640 1 -- 172.21.15.71:0/3317113250 wait complete. 2023-10-26T05:52:15.713 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-10-26T05:52:15.509856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5526,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556973,"statfs":{"total":1600000000,"available":1594340984,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.185048"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442115+0000","last_change":"2023-10-26T05:52:12.173905+0000","last_active":"2023-10-26T05:52:13.442115+0000","last_peered":"2023-10-26T05:52:13.442115+0000","last_clean":"2023-10-26T05:52:13.442115+0000","last_became_active":"2023-10-26T05:52:12.172224+0000","last_became_peered":"2023-10-26T05:52:12.172224+0000","last_unstale":"2023-10-26T05:52:13.442115+0000","last_undegraded":"2023-10-26T05:52:13.442115+0000","last_fullsized":"2023-10-26T05:52:13.442115+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:54.318113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.437801+0000","last_change":"2023-10-26T05:52:12.153462+0000","last_active":"2023-10-26T05:52:13.437801+0000","last_peered":"2023-10-26T05:52:13.437801+0000","last_clean":"2023-10-26T05:52:13.437801+0000","last_became_active":"2023-10-26T05:52:12.150588+0000","last_became_peered":"2023-10-26T05:52:12.150588+0000","last_unstale":"2023-10-26T05:52:13.437801+0000","last_undegraded":"2023-10-26T05:52:13.437801+0000","last_fullsized":"2023-10-26T05:52:13.437801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:54.873609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441431+0000","last_change":"2023-10-26T05:52:12.172316+0000","last_active":"2023-10-26T05:52:13.441431+0000","last_peered":"2023-10-26T05:52:13.441431+0000","last_clean":"2023-10-26T05:52:13.441431+0000","last_became_active":"2023-10-26T05:52:12.171702+0000","last_became_peered":"2023-10-26T05:52:12.171702+0000","last_unstale":"2023-10-26T05:52:13.441431+0000","last_undegraded":"2023-10-26T05:52:13.441431+0000","last_fullsized":"2023-10-26T05:52:13.441431+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:43:18.761521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.849344+0000","last_change":"2023-10-26T05:52:12.178421+0000","last_active":"2023-10-26T05:52:13.849344+0000","last_peered":"2023-10-26T05:52:13.849344+0000","last_clean":"2023-10-26T05:52:13.849344+0000","last_became_active":"2023-10-26T05:52:12.175894+0000","last_became_peered":"2023-10-26T05:52:12.175894+0000","last_unstale":"2023-10-26T05:52:13.849344+0000","last_undegraded":"2023-10-26T05:52:13.849344+0000","last_fullsized":"2023-10-26T05:52:13.849344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:49:42.191261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.150101+0000","last_change":"2023-10-26T05:52:12.182121+0000","last_active":"2023-10-26T05:52:13.150101+0000","last_peered":"2023-10-26T05:52:13.150101+0000","last_clean":"2023-10-26T05:52:13.150101+0000","last_became_active":"2023-10-26T05:52:12.181442+0000","last_became_peered":"2023-10-26T05:52:12.181442+0000","last_unstale":"2023-10-26T05:52:13.150101+0000","last_undegraded":"2023-10-26T05:52:13.150101+0000","last_fullsized":"2023-10-26T05:52:13.150101+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:38.828021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148331+0000","last_change":"2023-10-26T05:52:12.166823+0000","last_active":"2023-10-26T05:52:13.148331+0000","last_peered":"2023-10-26T05:52:13.148331+0000","last_clean":"2023-10-26T05:52:13.148331+0000","last_became_active":"2023-10-26T05:52:12.165400+0000","last_became_peered":"2023-10-26T05:52:12.165400+0000","last_unstale":"2023-10-26T05:52:13.148331+0000","last_undegraded":"2023-10-26T05:52:13.148331+0000","last_fullsized":"2023-10-26T05:52:13.148331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:29:22.461324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.849927+0000","last_change":"2023-10-26T05:52:12.166975+0000","last_active":"2023-10-26T05:52:13.849927+0000","last_peered":"2023-10-26T05:52:13.849927+0000","last_clean":"2023-10-26T05:52:13.849927+0000","last_became_active":"2023-10-26T05:52:12.166191+0000","last_became_peered":"2023-10-26T05:52:12.166191+0000","last_unstale":"2023-10-26T05:52:13.849927+0000","last_undegraded":"2023-10-26T05:52:13.849927+0000","last_fullsized":"2023-10-26T05:52:13.849927+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:39.622427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148168+0000","last_change":"2023-10-26T05:52:12.152246+0000","last_active":"2023-10-26T05:52:13.148168+0000","last_peered":"2023-10-26T05:52:13.148168+0000","last_clean":"2023-10-26T05:52:13.148168+0000","last_became_active":"2023-10-26T05:52:12.151624+0000","last_became_peered":"2023-10-26T05:52:12.151624+0000","last_unstale":"2023-10-26T05:52:13.148168+0000","last_undegraded":"2023-10-26T05:52:13.148168+0000","last_fullsized":"2023-10-26T05:52:13.148168+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:15:07.071933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148201+0000","last_change":"2023-10-26T05:52:12.149902+0000","last_active":"2023-10-26T05:52:13.148201+0000","last_peered":"2023-10-26T05:52:13.148201+0000","last_clean":"2023-10-26T05:52:13.148201+0000","last_became_active":"2023-10-26T05:52:12.149273+0000","last_became_peered":"2023-10-26T05:52:12.149273+0000","last_unstale":"2023-10-26T05:52:13.148201+0000","last_undegraded":"2023-10-26T05:52:13.148201+0000","last_fullsized":"2023-10-26T05:52:13.148201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:47:30.269419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442627+0000","last_change":"2023-10-26T05:52:12.159892+0000","last_active":"2023-10-26T05:52:13.442627+0000","last_peered":"2023-10-26T05:52:13.442627+0000","last_clean":"2023-10-26T05:52:13.442627+0000","last_became_active":"2023-10-26T05:52:12.159306+0000","last_became_peered":"2023-10-26T05:52:12.159306+0000","last_unstale":"2023-10-26T05:52:13.442627+0000","last_undegraded":"2023-10-26T05:52:13.442627+0000","last_fullsized":"2023-10-26T05:52:13.442627+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:38.828021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438141+0000","last_change":"2023-10-26T05:52:12.179509+0000","last_active":"2023-10-26T05:52:13.438141+0000","last_peered":"2023-10-26T05:52:13.438141+0000","last_clean":"2023-10-26T05:52:13.438141+0000","last_became_active":"2023-10-26T05:52:12.178728+0000","last_became_peered":"2023-10-26T05:52:12.178728+0000","last_unstale":"2023-10-26T05:52:13.438141+0000","last_undegraded":"2023-10-26T05:52:13.438141+0000","last_fullsized":"2023-10-26T05:52:13.438141+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:59.850424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443322+0000","last_change":"2023-10-26T05:52:12.177169+0000","last_active":"2023-10-26T05:52:13.443322+0000","last_peered":"2023-10-26T05:52:13.443322+0000","last_clean":"2023-10-26T05:52:13.443322+0000","last_became_active":"2023-10-26T05:52:12.176489+0000","last_became_peered":"2023-10-26T05:52:12.176489+0000","last_unstale":"2023-10-26T05:52:13.443322+0000","last_undegraded":"2023-10-26T05:52:13.443322+0000","last_fullsized":"2023-10-26T05:52:13.443322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:44.521062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441722+0000","last_change":"2023-10-26T05:52:12.165574+0000","last_active":"2023-10-26T05:52:13.441722+0000","last_peered":"2023-10-26T05:52:13.441722+0000","last_clean":"2023-10-26T05:52:13.441722+0000","last_became_active":"2023-10-26T05:52:12.164931+0000","last_became_peered":"2023-10-26T05:52:12.164931+0000","last_unstale":"2023-10-26T05:52:13.441722+0000","last_undegraded":"2023-10-26T05:52:13.441722+0000","last_fullsized":"2023-10-26T05:52:13.441722+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:27.047224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.849543+0000","last_change":"2023-10-26T05:52:12.177648+0000","last_active":"2023-10-26T05:52:13.849543+0000","last_peered":"2023-10-26T05:52:13.849543+0000","last_clean":"2023-10-26T05:52:13.849543+0000","last_became_active":"2023-10-26T05:52:12.174957+0000","last_became_peered":"2023-10-26T05:52:12.174957+0000","last_unstale":"2023-10-26T05:52:13.849543+0000","last_undegraded":"2023-10-26T05:52:13.849543+0000","last_fullsized":"2023-10-26T05:52:13.849543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:32.295793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.171063+0000","last_change":"2023-10-26T05:52:12.165527+0000","last_active":"2023-10-26T05:52:12.171063+0000","last_peered":"2023-10-26T05:52:12.171063+0000","last_clean":"2023-10-26T05:52:12.171063+0000","last_became_active":"2023-10-26T05:52:12.164771+0000","last_became_peered":"2023-10-26T05:52:12.164771+0000","last_unstale":"2023-10-26T05:52:12.171063+0000","last_undegraded":"2023-10-26T05:52:12.171063+0000","last_fullsized":"2023-10-26T05:52:12.171063+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:59.965331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.151411+0000","last_change":"2023-10-26T05:52:12.174694+0000","last_active":"2023-10-26T05:52:13.151411+0000","last_peered":"2023-10-26T05:52:13.151411+0000","last_clean":"2023-10-26T05:52:13.151411+0000","last_became_active":"2023-10-26T05:52:12.172812+0000","last_became_peered":"2023-10-26T05:52:12.172812+0000","last_unstale":"2023-10-26T05:52:13.151411+0000","last_undegraded":"2023-10-26T05:52:13.151411+0000","last_fullsized":"2023-10-26T05:52:13.151411+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:04.882269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149622+0000","last_change":"2023-10-26T05:52:12.177953+0000","last_active":"2023-10-26T05:52:13.149622+0000","last_peered":"2023-10-26T05:52:13.149622+0000","last_clean":"2023-10-26T05:52:13.149622+0000","last_became_active":"2023-10-26T05:52:12.174395+0000","last_became_peered":"2023-10-26T05:52:12.174395+0000","last_unstale":"2023-10-26T05:52:13.149622+0000","last_undegraded":"2023-10-26T05:52:13.149622+0000","last_fullsized":"2023-10-26T05:52:13.149622+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:54.318113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.850025+0000","last_change":"2023-10-26T05:52:12.179097+0000","last_active":"2023-10-26T05:52:13.850025+0000","last_peered":"2023-10-26T05:52:13.850025+0000","last_clean":"2023-10-26T05:52:13.850025+0000","last_became_active":"2023-10-26T05:52:12.177926+0000","last_became_peered":"2023-10-26T05:52:12.177926+0000","last_unstale":"2023-10-26T05:52:13.850025+0000","last_undegraded":"2023-10-26T05:52:13.850025+0000","last_fullsized":"2023-10-26T05:52:13.850025+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:45:03.364070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143125+0000","last_change":"2023-10-26T05:52:12.161765+0000","last_active":"2023-10-26T05:52:13.143125+0000","last_peered":"2023-10-26T05:52:13.143125+0000","last_clean":"2023-10-26T05:52:13.143125+0000","last_became_active":"2023-10-26T05:52:12.161070+0000","last_became_peered":"2023-10-26T05:52:12.161070+0000","last_unstale":"2023-10-26T05:52:13.143125+0000","last_undegraded":"2023-10-26T05:52:13.143125+0000","last_fullsized":"2023-10-26T05:52:13.143125+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:45.996029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.850295+0000","last_change":"2023-10-26T05:52:12.165132+0000","last_active":"2023-10-26T05:52:13.850295+0000","last_peered":"2023-10-26T05:52:13.850295+0000","last_clean":"2023-10-26T05:52:13.850295+0000","last_became_active":"2023-10-26T05:52:12.164373+0000","last_became_peered":"2023-10-26T05:52:12.164373+0000","last_unstale":"2023-10-26T05:52:13.850295+0000","last_undegraded":"2023-10-26T05:52:13.850295+0000","last_fullsized":"2023-10-26T05:52:13.850295+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:25:02.365335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.850275+0000","last_change":"2023-10-26T05:52:12.169449+0000","last_active":"2023-10-26T05:52:13.850275+0000","last_peered":"2023-10-26T05:52:13.850275+0000","last_clean":"2023-10-26T05:52:13.850275+0000","last_became_active":"2023-10-26T05:52:12.168719+0000","last_became_peered":"2023-10-26T05:52:12.168719+0000","last_unstale":"2023-10-26T05:52:13.850275+0000","last_undegraded":"2023-10-26T05:52:13.850275+0000","last_fullsized":"2023-10-26T05:52:13.850275+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:32.667200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.437945+0000","last_change":"2023-10-26T05:52:12.152267+0000","last_active":"2023-10-26T05:52:13.437945+0000","last_peered":"2023-10-26T05:52:13.437945+0000","last_clean":"2023-10-26T05:52:13.437945+0000","last_became_active":"2023-10-26T05:52:12.149540+0000","last_became_peered":"2023-10-26T05:52:12.149540+0000","last_unstale":"2023-10-26T05:52:13.437945+0000","last_undegraded":"2023-10-26T05:52:13.437945+0000","last_fullsized":"2023-10-26T05:52:13.437945+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:02:26.626331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441607+0000","last_change":"2023-10-26T05:52:12.158379+0000","last_active":"2023-10-26T05:52:13.441607+0000","last_peered":"2023-10-26T05:52:13.441607+0000","last_clean":"2023-10-26T05:52:13.441607+0000","last_became_active":"2023-10-26T05:52:12.157254+0000","last_became_peered":"2023-10-26T05:52:12.157254+0000","last_unstale":"2023-10-26T05:52:13.441607+0000","last_undegraded":"2023-10-26T05:52:13.441607+0000","last_fullsized":"2023-10-26T05:52:13.441607+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:21:49.827053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.849801+0000","last_change":"2023-10-26T05:52:12.178841+0000","last_active":"2023-10-26T05:52:13.849801+0000","last_peered":"2023-10-26T05:52:13.849801+0000","last_clean":"2023-10-26T05:52:13.849801+0000","last_became_active":"2023-10-26T05:52:12.176931+0000","last_became_peered":"2023-10-26T05:52:12.176931+0000","last_unstale":"2023-10-26T05:52:13.849801+0000","last_undegraded":"2023-10-26T05:52:13.849801+0000","last_fullsized":"2023-10-26T05:52:13.849801+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:35:17.868146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.437891+0000","last_change":"2023-10-26T05:52:12.152451+0000","last_active":"2023-10-26T05:52:13.437891+0000","last_peered":"2023-10-26T05:52:13.437891+0000","last_clean":"2023-10-26T05:52:13.437891+0000","last_became_active":"2023-10-26T05:52:12.151949+0000","last_became_peered":"2023-10-26T05:52:12.151949+0000","last_unstale":"2023-10-26T05:52:13.437891+0000","last_undegraded":"2023-10-26T05:52:13.437891+0000","last_fullsized":"2023-10-26T05:52:13.437891+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:43.341925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443009+0000","last_change":"2023-10-26T05:52:12.183566+0000","last_active":"2023-10-26T05:52:13.443009+0000","last_peered":"2023-10-26T05:52:13.443009+0000","last_clean":"2023-10-26T05:52:13.443009+0000","last_became_active":"2023-10-26T05:52:12.182899+0000","last_became_peered":"2023-10-26T05:52:12.182899+0000","last_unstale":"2023-10-26T05:52:13.443009+0000","last_undegraded":"2023-10-26T05:52:13.443009+0000","last_fullsized":"2023-10-26T05:52:13.443009+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:46:11.106041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.150470+0000","last_change":"2023-10-26T05:52:12.179781+0000","last_active":"2023-10-26T05:52:13.150470+0000","last_peered":"2023-10-26T05:52:13.150470+0000","last_clean":"2023-10-26T05:52:13.150470+0000","last_became_active":"2023-10-26T05:52:12.178918+0000","last_became_peered":"2023-10-26T05:52:12.178918+0000","last_unstale":"2023-10-26T05:52:13.150470+0000","last_undegraded":"2023-10-26T05:52:13.150470+0000","last_fullsized":"2023-10-26T05:52:13.150470+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:31:52.802583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441926+0000","last_change":"2023-10-26T05:52:12.167058+0000","last_active":"2023-10-26T05:52:13.441926+0000","last_peered":"2023-10-26T05:52:13.441926+0000","last_clean":"2023-10-26T05:52:13.441926+0000","last_became_active":"2023-10-26T05:52:12.164956+0000","last_became_peered":"2023-10-26T05:52:12.164956+0000","last_unstale":"2023-10-26T05:52:13.441926+0000","last_undegraded":"2023-10-26T05:52:13.441926+0000","last_fullsized":"2023-10-26T05:52:13.441926+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:55.993051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438047+0000","last_change":"2023-10-26T05:52:12.158595+0000","last_active":"2023-10-26T05:52:13.438047+0000","last_peered":"2023-10-26T05:52:13.438047+0000","last_clean":"2023-10-26T05:52:13.438047+0000","last_became_active":"2023-10-26T05:52:12.157952+0000","last_became_peered":"2023-10-26T05:52:12.157952+0000","last_unstale":"2023-10-26T05:52:13.438047+0000","last_undegraded":"2023-10-26T05:52:13.438047+0000","last_fullsized":"2023-10-26T05:52:13.438047+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:21.983715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438073+0000","last_change":"2023-10-26T05:52:12.161432+0000","last_active":"2023-10-26T05:52:13.438073+0000","last_peered":"2023-10-26T05:52:13.438073+0000","last_clean":"2023-10-26T05:52:13.438073+0000","last_became_active":"2023-10-26T05:52:12.160821+0000","last_became_peered":"2023-10-26T05:52:12.160821+0000","last_unstale":"2023-10-26T05:52:13.438073+0000","last_undegraded":"2023-10-26T05:52:13.438073+0000","last_fullsized":"2023-10-26T05:52:13.438073+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:26.709985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.151855+0000","last_change":"2023-10-26T05:52:12.176022+0000","last_active":"2023-10-26T05:52:13.151855+0000","last_peered":"2023-10-26T05:52:13.151855+0000","last_clean":"2023-10-26T05:52:13.151855+0000","last_became_active":"2023-10-26T05:52:12.173580+0000","last_became_peered":"2023-10-26T05:52:12.173580+0000","last_unstale":"2023-10-26T05:52:13.151855+0000","last_undegraded":"2023-10-26T05:52:13.151855+0000","last_fullsized":"2023-10-26T05:52:13.151855+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:44.521062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443488+0000","last_change":"2023-10-26T05:52:12.157311+0000","last_active":"2023-10-26T05:52:13.443488+0000","last_peered":"2023-10-26T05:52:13.443488+0000","last_clean":"2023-10-26T05:52:13.443488+0000","last_became_active":"2023-10-26T05:52:12.156683+0000","last_became_peered":"2023-10-26T05:52:12.156683+0000","last_unstale":"2023-10-26T05:52:13.443488+0000","last_undegraded":"2023-10-26T05:52:13.443488+0000","last_fullsized":"2023-10-26T05:52:13.443488+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:42.497148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148121+0000","last_change":"2023-10-26T05:52:12.168483+0000","last_active":"2023-10-26T05:52:13.148121+0000","last_peered":"2023-10-26T05:52:13.148121+0000","last_clean":"2023-10-26T05:52:13.148121+0000","last_became_active":"2023-10-26T05:52:12.167866+0000","last_became_peered":"2023-10-26T05:52:12.167866+0000","last_unstale":"2023-10-26T05:52:13.148121+0000","last_undegraded":"2023-10-26T05:52:13.148121+0000","last_fullsized":"2023-10-26T05:52:13.148121+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:21:49.827053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442510+0000","last_change":"2023-10-26T05:52:11.367540+0000","last_active":"2023-10-26T05:52:13.442510+0000","last_peered":"2023-10-26T05:52:13.442510+0000","last_clean":"2023-10-26T05:52:13.442510+0000","last_became_active":"2023-10-26T05:52:11.362766+0000","last_became_peered":"2023-10-26T05:52:11.362766+0000","last_unstale":"2023-10-26T05:52:13.442510+0000","last_undegraded":"2023-10-26T05:52:13.442510+0000","last_fullsized":"2023-10-26T05:52:13.442510+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:32:03.317710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442262+0000","last_change":"2023-10-26T05:52:12.160070+0000","last_active":"2023-10-26T05:52:13.442262+0000","last_peered":"2023-10-26T05:52:13.442262+0000","last_clean":"2023-10-26T05:52:13.442262+0000","last_became_active":"2023-10-26T05:52:12.159441+0000","last_became_peered":"2023-10-26T05:52:12.159441+0000","last_unstale":"2023-10-26T05:52:13.442262+0000","last_undegraded":"2023-10-26T05:52:13.442262+0000","last_fullsized":"2023-10-26T05:52:13.442262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:51.755179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149351+0000","last_change":"2023-10-26T05:52:11.361843+0000","last_active":"2023-10-26T05:52:13.149351+0000","last_peered":"2023-10-26T05:52:13.149351+0000","last_clean":"2023-10-26T05:52:13.149351+0000","last_became_active":"2023-10-26T05:52:11.361220+0000","last_became_peered":"2023-10-26T05:52:11.361220+0000","last_unstale":"2023-10-26T05:52:13.149351+0000","last_undegraded":"2023-10-26T05:52:13.149351+0000","last_fullsized":"2023-10-26T05:52:13.149351+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:47:29.256970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442299+0000","last_change":"2023-10-26T05:52:12.156313+0000","last_active":"2023-10-26T05:52:13.442299+0000","last_peered":"2023-10-26T05:52:13.442299+0000","last_clean":"2023-10-26T05:52:13.442299+0000","last_became_active":"2023-10-26T05:52:12.153426+0000","last_became_peered":"2023-10-26T05:52:12.153426+0000","last_unstale":"2023-10-26T05:52:13.442299+0000","last_undegraded":"2023-10-26T05:52:13.442299+0000","last_fullsized":"2023-10-26T05:52:13.442299+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:21:37.965746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143385+0000","last_change":"2023-10-26T05:52:11.362777+0000","last_active":"2023-10-26T05:52:13.143385+0000","last_peered":"2023-10-26T05:52:13.143385+0000","last_clean":"2023-10-26T05:52:13.143385+0000","last_became_active":"2023-10-26T05:52:11.356934+0000","last_became_peered":"2023-10-26T05:52:11.356934+0000","last_unstale":"2023-10-26T05:52:13.143385+0000","last_undegraded":"2023-10-26T05:52:13.143385+0000","last_fullsized":"2023-10-26T05:52:13.143385+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:14:37.815571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442337+0000","last_change":"2023-10-26T05:52:12.173691+0000","last_active":"2023-10-26T05:52:13.442337+0000","last_peered":"2023-10-26T05:52:13.442337+0000","last_clean":"2023-10-26T05:52:13.442337+0000","last_became_active":"2023-10-26T05:52:12.169247+0000","last_became_peered":"2023-10-26T05:52:12.169247+0000","last_unstale":"2023-10-26T05:52:13.442337+0000","last_undegraded":"2023-10-26T05:52:13.442337+0000","last_fullsized":"2023-10-26T05:52:13.442337+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:45.996029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143335+0000","last_change":"2023-10-26T05:52:11.359077+0000","last_active":"2023-10-26T05:52:13.143335+0000","last_peered":"2023-10-26T05:52:13.143335+0000","last_clean":"2023-10-26T05:52:13.143335+0000","last_became_active":"2023-10-26T05:52:11.358356+0000","last_became_peered":"2023-10-26T05:52:11.358356+0000","last_unstale":"2023-10-26T05:52:13.143335+0000","last_undegraded":"2023-10-26T05:52:13.143335+0000","last_fullsized":"2023-10-26T05:52:13.143335+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:38:59.198202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.437969+0000","last_change":"2023-10-26T05:52:12.171975+0000","last_active":"2023-10-26T05:52:13.437969+0000","last_peered":"2023-10-26T05:52:13.437969+0000","last_clean":"2023-10-26T05:52:13.437969+0000","last_became_active":"2023-10-26T05:52:12.170929+0000","last_became_peered":"2023-10-26T05:52:12.170929+0000","last_unstale":"2023-10-26T05:52:13.437969+0000","last_undegraded":"2023-10-26T05:52:13.437969+0000","last_fullsized":"2023-10-26T05:52:13.437969+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:43.579399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.150166+0000","last_change":"2023-10-26T05:52:11.372451+0000","last_active":"2023-10-26T05:52:13.150166+0000","last_peered":"2023-10-26T05:52:13.150166+0000","last_clean":"2023-10-26T05:52:13.150166+0000","last_became_active":"2023-10-26T05:52:11.369918+0000","last_became_peered":"2023-10-26T05:52:11.369918+0000","last_unstale":"2023-10-26T05:52:13.150166+0000","last_undegraded":"2023-10-26T05:52:13.150166+0000","last_fullsized":"2023-10-26T05:52:13.150166+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:13:13.623654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149937+0000","last_change":"2023-10-26T05:52:12.177152+0000","last_active":"2023-10-26T05:52:13.149937+0000","last_peered":"2023-10-26T05:52:13.149937+0000","last_clean":"2023-10-26T05:52:13.149937+0000","last_became_active":"2023-10-26T05:52:12.166835+0000","last_became_peered":"2023-10-26T05:52:12.166835+0000","last_unstale":"2023-10-26T05:52:13.149937+0000","last_undegraded":"2023-10-26T05:52:13.149937+0000","last_fullsized":"2023-10-26T05:52:13.149937+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:21:37.965746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149745+0000","last_change":"2023-10-26T05:52:11.361698+0000","last_active":"2023-10-26T05:52:13.149745+0000","last_peered":"2023-10-26T05:52:13.149745+0000","last_clean":"2023-10-26T05:52:13.149745+0000","last_became_active":"2023-10-26T05:52:11.361131+0000","last_became_peered":"2023-10-26T05:52:11.361131+0000","last_unstale":"2023-10-26T05:52:13.149745+0000","last_undegraded":"2023-10-26T05:52:13.149745+0000","last_fullsized":"2023-10-26T05:52:13.149745+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:25.325418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441705+0000","last_change":"2023-10-26T05:52:12.168310+0000","last_active":"2023-10-26T05:52:13.441705+0000","last_peered":"2023-10-26T05:52:13.441705+0000","last_clean":"2023-10-26T05:52:13.441705+0000","last_became_active":"2023-10-26T05:52:12.166920+0000","last_became_peered":"2023-10-26T05:52:12.166920+0000","last_unstale":"2023-10-26T05:52:13.441705+0000","last_undegraded":"2023-10-26T05:52:13.441705+0000","last_fullsized":"2023-10-26T05:52:13.441705+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:04.213991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143275+0000","last_change":"2023-10-26T05:52:11.363038+0000","last_active":"2023-10-26T05:52:13.143275+0000","last_peered":"2023-10-26T05:52:13.143275+0000","last_clean":"2023-10-26T05:52:13.143275+0000","last_became_active":"2023-10-26T05:52:11.362812+0000","last_became_peered":"2023-10-26T05:52:11.362812+0000","last_unstale":"2023-10-26T05:52:13.143275+0000","last_undegraded":"2023-10-26T05:52:13.143275+0000","last_fullsized":"2023-10-26T05:52:13.143275+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:43:17.749072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.171118+0000","last_change":"2023-10-26T05:52:12.168799+0000","last_active":"2023-10-26T05:52:12.171118+0000","last_peered":"2023-10-26T05:52:12.171118+0000","last_clean":"2023-10-26T05:52:12.171118+0000","last_became_active":"2023-10-26T05:52:12.168139+0000","last_became_peered":"2023-10-26T05:52:12.168139+0000","last_unstale":"2023-10-26T05:52:12.171118+0000","last_undegraded":"2023-10-26T05:52:12.171118+0000","last_fullsized":"2023-10-26T05:52:12.171118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:04.957048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438140+0000","last_change":"2023-10-26T05:52:11.354210+0000","last_active":"2023-10-26T05:52:13.438140+0000","last_peered":"2023-10-26T05:52:13.438140+0000","last_clean":"2023-10-26T05:52:13.438140+0000","last_became_active":"2023-10-26T05:52:11.353474+0000","last_became_peered":"2023-10-26T05:52:11.353474+0000","last_unstale":"2023-10-26T05:52:13.438140+0000","last_undegraded":"2023-10-26T05:52:13.438140+0000","last_fullsized":"2023-10-26T05:52:13.438140+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:42.122748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.174135+0000","last_change":"2023-10-26T05:52:12.173959+0000","last_active":"2023-10-26T05:52:12.174135+0000","last_peered":"2023-10-26T05:52:12.174135+0000","last_clean":"2023-10-26T05:52:12.174135+0000","last_became_active":"2023-10-26T05:52:12.173190+0000","last_became_peered":"2023-10-26T05:52:12.173190+0000","last_unstale":"2023-10-26T05:52:12.174135+0000","last_undegraded":"2023-10-26T05:52:12.174135+0000","last_fullsized":"2023-10-26T05:52:12.174135+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:53.474625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442492+0000","last_change":"2023-10-26T05:52:11.356833+0000","last_active":"2023-10-26T05:52:13.442492+0000","last_peered":"2023-10-26T05:52:13.442492+0000","last_clean":"2023-10-26T05:52:13.442492+0000","last_became_active":"2023-10-26T05:52:11.355851+0000","last_became_peered":"2023-10-26T05:52:11.355851+0000","last_unstale":"2023-10-26T05:52:13.442492+0000","last_undegraded":"2023-10-26T05:52:13.442492+0000","last_fullsized":"2023-10-26T05:52:13.442492+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:44:46.895931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148656+0000","last_change":"2023-10-26T05:52:12.167189+0000","last_active":"2023-10-26T05:52:13.148656+0000","last_peered":"2023-10-26T05:52:13.148656+0000","last_clean":"2023-10-26T05:52:13.148656+0000","last_became_active":"2023-10-26T05:52:12.166185+0000","last_became_peered":"2023-10-26T05:52:12.166185+0000","last_unstale":"2023-10-26T05:52:13.148656+0000","last_undegraded":"2023-10-26T05:52:13.148656+0000","last_fullsized":"2023-10-26T05:52:13.148656+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:24.570381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443594+0000","last_change":"2023-10-26T05:52:11.373536+0000","last_active":"2023-10-26T05:52:13.443594+0000","last_peered":"2023-10-26T05:52:13.443594+0000","last_clean":"2023-10-26T05:52:13.443594+0000","last_became_active":"2023-10-26T05:52:11.372909+0000","last_became_peered":"2023-10-26T05:52:11.372909+0000","last_unstale":"2023-10-26T05:52:13.443594+0000","last_undegraded":"2023-10-26T05:52:13.443594+0000","last_fullsized":"2023-10-26T05:52:13.443594+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:38:18.593868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.142947+0000","last_change":"2023-10-26T05:52:12.149538+0000","last_active":"2023-10-26T05:52:13.142947+0000","last_peered":"2023-10-26T05:52:13.142947+0000","last_clean":"2023-10-26T05:52:13.142947+0000","last_became_active":"2023-10-26T05:52:12.148900+0000","last_became_peered":"2023-10-26T05:52:12.148900+0000","last_unstale":"2023-10-26T05:52:13.142947+0000","last_undegraded":"2023-10-26T05:52:13.142947+0000","last_fullsized":"2023-10-26T05:52:13.142947+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:04.957048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443097+0000","last_change":"2023-10-26T05:52:11.369224+0000","last_active":"2023-10-26T05:52:13.443097+0000","last_peered":"2023-10-26T05:52:13.443097+0000","last_clean":"2023-10-26T05:52:13.443097+0000","last_became_active":"2023-10-26T05:52:11.368195+0000","last_became_peered":"2023-10-26T05:52:11.368195+0000","last_unstale":"2023-10-26T05:52:13.443097+0000","last_undegraded":"2023-10-26T05:52:13.443097+0000","last_fullsized":"2023-10-26T05:52:13.443097+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:13:13.623654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149435+0000","last_change":"2023-10-26T05:52:12.174106+0000","last_active":"2023-10-26T05:52:13.149435+0000","last_peered":"2023-10-26T05:52:13.149435+0000","last_clean":"2023-10-26T05:52:13.149435+0000","last_became_active":"2023-10-26T05:52:12.173329+0000","last_became_peered":"2023-10-26T05:52:12.173329+0000","last_unstale":"2023-10-26T05:52:13.149435+0000","last_undegraded":"2023-10-26T05:52:13.149435+0000","last_fullsized":"2023-10-26T05:52:13.149435+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:55.993051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148427+0000","last_change":"2023-10-26T05:52:11.359649+0000","last_active":"2023-10-26T05:52:13.148427+0000","last_peered":"2023-10-26T05:52:13.148427+0000","last_clean":"2023-10-26T05:52:13.148427+0000","last_became_active":"2023-10-26T05:52:11.358343+0000","last_became_peered":"2023-10-26T05:52:11.358343+0000","last_unstale":"2023-10-26T05:52:13.148427+0000","last_undegraded":"2023-10-26T05:52:13.148427+0000","last_fullsized":"2023-10-26T05:52:13.148427+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:29:46.572537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143246+0000","last_change":"2023-10-26T05:52:12.170904+0000","last_active":"2023-10-26T05:52:13.143246+0000","last_peered":"2023-10-26T05:52:13.143246+0000","last_clean":"2023-10-26T05:52:13.143246+0000","last_became_active":"2023-10-26T05:52:12.166556+0000","last_became_peered":"2023-10-26T05:52:12.166556+0000","last_unstale":"2023-10-26T05:52:13.143246+0000","last_undegraded":"2023-10-26T05:52:13.143246+0000","last_fullsized":"2023-10-26T05:52:13.143246+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:41.544834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438209+0000","last_change":"2023-10-26T05:52:11.352394+0000","last_active":"2023-10-26T05:52:13.438209+0000","last_peered":"2023-10-26T05:52:13.438209+0000","last_clean":"2023-10-26T05:52:13.438209+0000","last_became_active":"2023-10-26T05:52:11.350296+0000","last_became_peered":"2023-10-26T05:52:11.350296+0000","last_unstale":"2023-10-26T05:52:13.438209+0000","last_undegraded":"2023-10-26T05:52:13.438209+0000","last_fullsized":"2023-10-26T05:52:13.438209+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:07:58.955346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442025+0000","last_change":"2023-10-26T05:52:12.169850+0000","last_active":"2023-10-26T05:52:13.442025+0000","last_peered":"2023-10-26T05:52:13.442025+0000","last_clean":"2023-10-26T05:52:13.442025+0000","last_became_active":"2023-10-26T05:52:12.167720+0000","last_became_peered":"2023-10-26T05:52:12.167720+0000","last_unstale":"2023-10-26T05:52:13.442025+0000","last_undegraded":"2023-10-26T05:52:13.442025+0000","last_fullsized":"2023-10-26T05:52:13.442025+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:43:04.957048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.156651+0000","last_change":"2023-10-26T05:52:11.354537+0000","last_active":"2023-10-26T05:52:12.156651+0000","last_peered":"2023-10-26T05:52:12.156651+0000","last_clean":"2023-10-26T05:52:12.156651+0000","last_became_active":"2023-10-26T05:52:11.353068+0000","last_became_peered":"2023-10-26T05:52:11.353068+0000","last_unstale":"2023-10-26T05:52:12.156651+0000","last_undegraded":"2023-10-26T05:52:12.156651+0000","last_fullsized":"2023-10-26T05:52:12.156651+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:40.532385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.437806+0000","last_change":"2023-10-26T05:52:12.171536+0000","last_active":"2023-10-26T05:52:13.437806+0000","last_peered":"2023-10-26T05:52:13.437806+0000","last_clean":"2023-10-26T05:52:13.437806+0000","last_became_active":"2023-10-26T05:52:12.169819+0000","last_became_peered":"2023-10-26T05:52:12.169819+0000","last_unstale":"2023-10-26T05:52:13.437806+0000","last_undegraded":"2023-10-26T05:52:13.437806+0000","last_fullsized":"2023-10-26T05:52:13.437806+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:57:38.612793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148661+0000","last_change":"2023-10-26T05:52:11.373456+0000","last_active":"2023-10-26T05:52:13.148661+0000","last_peered":"2023-10-26T05:52:13.148661+0000","last_clean":"2023-10-26T05:52:13.148661+0000","last_became_active":"2023-10-26T05:52:11.372810+0000","last_became_peered":"2023-10-26T05:52:11.372810+0000","last_unstale":"2023-10-26T05:52:13.148661+0000","last_undegraded":"2023-10-26T05:52:13.148661+0000","last_fullsized":"2023-10-26T05:52:13.148661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:15.054581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438069+0000","last_change":"2023-10-26T05:52:12.154070+0000","last_active":"2023-10-26T05:52:13.438069+0000","last_peered":"2023-10-26T05:52:13.438069+0000","last_clean":"2023-10-26T05:52:13.438069+0000","last_became_active":"2023-10-26T05:52:12.152912+0000","last_became_peered":"2023-10-26T05:52:12.152912+0000","last_unstale":"2023-10-26T05:52:13.438069+0000","last_undegraded":"2023-10-26T05:52:13.438069+0000","last_fullsized":"2023-10-26T05:52:13.438069+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:40:43.161984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143129+0000","last_change":"2023-10-26T05:52:11.358966+0000","last_active":"2023-10-26T05:52:13.143129+0000","last_peered":"2023-10-26T05:52:13.143129+0000","last_clean":"2023-10-26T05:52:13.143129+0000","last_became_active":"2023-10-26T05:52:11.358695+0000","last_became_peered":"2023-10-26T05:52:11.358695+0000","last_unstale":"2023-10-26T05:52:13.143129+0000","last_undegraded":"2023-10-26T05:52:13.143129+0000","last_fullsized":"2023-10-26T05:52:13.143129+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:03.869820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442634+0000","last_change":"2023-10-26T05:52:12.175054+0000","last_active":"2023-10-26T05:52:13.442634+0000","last_peered":"2023-10-26T05:52:13.442634+0000","last_clean":"2023-10-26T05:52:13.442634+0000","last_became_active":"2023-10-26T05:52:12.170554+0000","last_became_peered":"2023-10-26T05:52:12.170554+0000","last_unstale":"2023-10-26T05:52:13.442634+0000","last_undegraded":"2023-10-26T05:52:13.442634+0000","last_fullsized":"2023-10-26T05:52:13.442634+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:35:41.544834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441916+0000","last_change":"2023-10-26T05:52:11.368882+0000","last_active":"2023-10-26T05:52:13.441916+0000","last_peered":"2023-10-26T05:52:13.441916+0000","last_clean":"2023-10-26T05:52:13.441916+0000","last_became_active":"2023-10-26T05:52:11.366803+0000","last_became_peered":"2023-10-26T05:52:11.366803+0000","last_unstale":"2023-10-26T05:52:13.441916+0000","last_undegraded":"2023-10-26T05:52:13.441916+0000","last_fullsized":"2023-10-26T05:52:13.441916+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:25.325418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441678+0000","last_change":"2023-10-26T05:52:12.171486+0000","last_active":"2023-10-26T05:52:13.441678+0000","last_peered":"2023-10-26T05:52:13.441678+0000","last_clean":"2023-10-26T05:52:13.441678+0000","last_became_active":"2023-10-26T05:52:12.170878+0000","last_became_peered":"2023-10-26T05:52:12.170878+0000","last_unstale":"2023-10-26T05:52:13.441678+0000","last_undegraded":"2023-10-26T05:52:13.441678+0000","last_fullsized":"2023-10-26T05:52:13.441678+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:22.522337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143517+0000","last_change":"2023-10-26T05:52:11.357702+0000","last_active":"2023-10-26T05:52:13.143517+0000","last_peered":"2023-10-26T05:52:13.143517+0000","last_clean":"2023-10-26T05:52:13.143517+0000","last_became_active":"2023-10-26T05:52:11.356028+0000","last_became_peered":"2023-10-26T05:52:11.356028+0000","last_unstale":"2023-10-26T05:52:13.143517+0000","last_undegraded":"2023-10-26T05:52:13.143517+0000","last_fullsized":"2023-10-26T05:52:13.143517+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:41.484699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.152153+0000","last_change":"2023-10-26T05:52:12.182720+0000","last_active":"2023-10-26T05:52:13.152153+0000","last_peered":"2023-10-26T05:52:13.152153+0000","last_clean":"2023-10-26T05:52:13.152153+0000","last_became_active":"2023-10-26T05:52:12.181820+0000","last_became_peered":"2023-10-26T05:52:12.181820+0000","last_unstale":"2023-10-26T05:52:13.152153+0000","last_undegraded":"2023-10-26T05:52:13.152153+0000","last_fullsized":"2023-10-26T05:52:13.152153+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:42.497148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.142995+0000","last_change":"2023-10-26T05:52:11.360567+0000","last_active":"2023-10-26T05:52:13.142995+0000","last_peered":"2023-10-26T05:52:13.142995+0000","last_clean":"2023-10-26T05:52:13.142995+0000","last_became_active":"2023-10-26T05:52:11.359735+0000","last_became_peered":"2023-10-26T05:52:11.359735+0000","last_unstale":"2023-10-26T05:52:13.142995+0000","last_undegraded":"2023-10-26T05:52:13.142995+0000","last_fullsized":"2023-10-26T05:52:13.142995+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:21:36.953297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.849974+0000","last_change":"2023-10-26T05:52:12.170320+0000","last_active":"2023-10-26T05:52:13.849974+0000","last_peered":"2023-10-26T05:52:13.849974+0000","last_clean":"2023-10-26T05:52:13.849974+0000","last_became_active":"2023-10-26T05:52:12.169627+0000","last_became_peered":"2023-10-26T05:52:12.169627+0000","last_unstale":"2023-10-26T05:52:13.849974+0000","last_undegraded":"2023-10-26T05:52:13.849974+0000","last_fullsized":"2023-10-26T05:52:13.849974+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:01.513630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.364475+0000","last_change":"2023-10-26T05:52:11.358312+0000","last_active":"2023-10-26T05:52:12.364475+0000","last_peered":"2023-10-26T05:52:12.364475+0000","last_clean":"2023-10-26T05:52:12.364475+0000","last_became_active":"2023-10-26T05:52:11.357991+0000","last_became_peered":"2023-10-26T05:52:11.357991+0000","last_unstale":"2023-10-26T05:52:12.364475+0000","last_undegraded":"2023-10-26T05:52:12.364475+0000","last_fullsized":"2023-10-26T05:52:12.364475+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:24:44.983580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148523+0000","last_change":"2023-10-26T05:52:12.160967+0000","last_active":"2023-10-26T05:52:13.148523+0000","last_peered":"2023-10-26T05:52:13.148523+0000","last_clean":"2023-10-26T05:52:13.148523+0000","last_became_active":"2023-10-26T05:52:12.159380+0000","last_became_peered":"2023-10-26T05:52:12.159380+0000","last_unstale":"2023-10-26T05:52:13.148523+0000","last_undegraded":"2023-10-26T05:52:13.148523+0000","last_fullsized":"2023-10-26T05:52:13.148523+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:26.337867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.850093+0000","last_change":"2023-10-26T05:52:11.357411+0000","last_active":"2023-10-26T05:52:13.850093+0000","last_peered":"2023-10-26T05:52:13.850093+0000","last_clean":"2023-10-26T05:52:13.850093+0000","last_became_active":"2023-10-26T05:52:11.355996+0000","last_became_peered":"2023-10-26T05:52:11.355996+0000","last_unstale":"2023-10-26T05:52:13.850093+0000","last_undegraded":"2023-10-26T05:52:13.850093+0000","last_fullsized":"2023-10-26T05:52:13.850093+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:03:40.592471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.850515+0000","last_change":"2023-10-26T05:52:12.163567+0000","last_active":"2023-10-26T05:52:13.850515+0000","last_peered":"2023-10-26T05:52:13.850515+0000","last_clean":"2023-10-26T05:52:13.850515+0000","last_became_active":"2023-10-26T05:52:12.162910+0000","last_became_peered":"2023-10-26T05:52:12.162910+0000","last_unstale":"2023-10-26T05:52:13.850515+0000","last_undegraded":"2023-10-26T05:52:13.850515+0000","last_fullsized":"2023-10-26T05:52:13.850515+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:06:51.339481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.850693+0000","last_change":"2023-10-26T05:52:11.359942+0000","last_active":"2023-10-26T05:52:13.850693+0000","last_peered":"2023-10-26T05:52:13.850693+0000","last_clean":"2023-10-26T05:52:13.850693+0000","last_became_active":"2023-10-26T05:52:11.359673+0000","last_became_peered":"2023-10-26T05:52:11.359673+0000","last_unstale":"2023-10-26T05:52:13.850693+0000","last_undegraded":"2023-10-26T05:52:13.850693+0000","last_fullsized":"2023-10-26T05:52:13.850693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:40:15.274790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148282+0000","last_change":"2023-10-26T05:52:12.152835+0000","last_active":"2023-10-26T05:52:13.148282+0000","last_peered":"2023-10-26T05:52:13.148282+0000","last_clean":"2023-10-26T05:52:13.148282+0000","last_became_active":"2023-10-26T05:52:12.152208+0000","last_became_peered":"2023-10-26T05:52:12.152208+0000","last_unstale":"2023-10-26T05:52:13.148282+0000","last_undegraded":"2023-10-26T05:52:13.148282+0000","last_fullsized":"2023-10-26T05:52:13.148282+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:44:47.908380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.156992+0000","last_change":"2023-10-26T05:52:11.357088+0000","last_active":"2023-10-26T05:52:12.156992+0000","last_peered":"2023-10-26T05:52:12.156992+0000","last_clean":"2023-10-26T05:52:12.156992+0000","last_became_active":"2023-10-26T05:52:11.356681+0000","last_became_peered":"2023-10-26T05:52:11.356681+0000","last_unstale":"2023-10-26T05:52:12.156992+0000","last_undegraded":"2023-10-26T05:52:12.156992+0000","last_fullsized":"2023-10-26T05:52:12.156992+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:46:10.093592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149620+0000","last_change":"2023-10-26T05:52:12.169635+0000","last_active":"2023-10-26T05:52:13.149620+0000","last_peered":"2023-10-26T05:52:13.149620+0000","last_clean":"2023-10-26T05:52:13.149620+0000","last_became_active":"2023-10-26T05:52:12.168719+0000","last_became_peered":"2023-10-26T05:52:12.168719+0000","last_unstale":"2023-10-26T05:52:13.149620+0000","last_undegraded":"2023-10-26T05:52:13.149620+0000","last_fullsized":"2023-10-26T05:52:13.149620+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:51.755179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438343+0000","last_change":"2023-10-26T05:52:11.354068+0000","last_active":"2023-10-26T05:52:13.438343+0000","last_peered":"2023-10-26T05:52:13.438343+0000","last_clean":"2023-10-26T05:52:13.438343+0000","last_became_active":"2023-10-26T05:52:11.353468+0000","last_became_peered":"2023-10-26T05:52:11.353468+0000","last_unstale":"2023-10-26T05:52:13.438343+0000","last_undegraded":"2023-10-26T05:52:13.438343+0000","last_fullsized":"2023-10-26T05:52:13.438343+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:38:00.815498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442307+0000","last_change":"2023-10-26T05:52:12.173135+0000","last_active":"2023-10-26T05:52:13.442307+0000","last_peered":"2023-10-26T05:52:13.442307+0000","last_clean":"2023-10-26T05:52:13.442307+0000","last_became_active":"2023-10-26T05:52:12.172539+0000","last_became_peered":"2023-10-26T05:52:12.172539+0000","last_unstale":"2023-10-26T05:52:13.442307+0000","last_undegraded":"2023-10-26T05:52:13.442307+0000","last_fullsized":"2023-10-26T05:52:13.442307+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:59.965331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438038+0000","last_change":"2023-10-26T05:52:11.360937+0000","last_active":"2023-10-26T05:52:13.438038+0000","last_peered":"2023-10-26T05:52:13.438038+0000","last_clean":"2023-10-26T05:52:13.438038+0000","last_became_active":"2023-10-26T05:52:11.359219+0000","last_became_peered":"2023-10-26T05:52:11.359219+0000","last_unstale":"2023-10-26T05:52:13.438038+0000","last_undegraded":"2023-10-26T05:52:13.438038+0000","last_fullsized":"2023-10-26T05:52:13.438038+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:22:55.251992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442968+0000","last_change":"2023-10-26T05:52:12.164622+0000","last_active":"2023-10-26T05:52:13.442968+0000","last_peered":"2023-10-26T05:52:13.442968+0000","last_clean":"2023-10-26T05:52:13.442968+0000","last_became_active":"2023-10-26T05:52:12.163944+0000","last_became_peered":"2023-10-26T05:52:12.163944+0000","last_unstale":"2023-10-26T05:52:13.442968+0000","last_undegraded":"2023-10-26T05:52:13.442968+0000","last_fullsized":"2023-10-26T05:52:13.442968+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:04.882269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442143+0000","last_change":"2023-10-26T05:52:11.367947+0000","last_active":"2023-10-26T05:52:13.442143+0000","last_peered":"2023-10-26T05:52:13.442143+0000","last_clean":"2023-10-26T05:52:13.442143+0000","last_became_active":"2023-10-26T05:52:11.365222+0000","last_became_peered":"2023-10-26T05:52:11.365222+0000","last_unstale":"2023-10-26T05:52:13.442143+0000","last_undegraded":"2023-10-26T05:52:13.442143+0000","last_fullsized":"2023-10-26T05:52:13.442143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:12:30.466887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149083+0000","last_change":"2023-10-26T05:52:12.166222+0000","last_active":"2023-10-26T05:52:13.149083+0000","last_peered":"2023-10-26T05:52:13.149083+0000","last_clean":"2023-10-26T05:52:13.149083+0000","last_became_active":"2023-10-26T05:52:12.165592+0000","last_became_peered":"2023-10-26T05:52:12.165592+0000","last_unstale":"2023-10-26T05:52:13.149083+0000","last_undegraded":"2023-10-26T05:52:13.149083+0000","last_fullsized":"2023-10-26T05:52:13.149083+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:22.522337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443177+0000","last_change":"2023-10-26T05:52:11.358484+0000","last_active":"2023-10-26T05:52:13.443177+0000","last_peered":"2023-10-26T05:52:13.443177+0000","last_clean":"2023-10-26T05:52:13.443177+0000","last_became_active":"2023-10-26T05:52:11.357858+0000","last_became_peered":"2023-10-26T05:52:11.357858+0000","last_unstale":"2023-10-26T05:52:13.443177+0000","last_undegraded":"2023-10-26T05:52:13.443177+0000","last_fullsized":"2023-10-26T05:52:13.443177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:56:15.054581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.148460+0000","last_change":"2023-10-26T05:52:12.178666+0000","last_active":"2023-10-26T05:52:13.148460+0000","last_peered":"2023-10-26T05:52:13.148460+0000","last_clean":"2023-10-26T05:52:13.148460+0000","last_became_active":"2023-10-26T05:52:12.178088+0000","last_became_peered":"2023-10-26T05:52:12.178088+0000","last_unstale":"2023-10-26T05:52:13.148460+0000","last_undegraded":"2023-10-26T05:52:13.148460+0000","last_fullsized":"2023-10-26T05:52:13.148460+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:02:18.799229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143413+0000","last_change":"2023-10-26T05:52:11.363241+0000","last_active":"2023-10-26T05:52:13.143413+0000","last_peered":"2023-10-26T05:52:13.143413+0000","last_clean":"2023-10-26T05:52:13.143413+0000","last_became_active":"2023-10-26T05:52:11.358779+0000","last_became_peered":"2023-10-26T05:52:11.358779+0000","last_unstale":"2023-10-26T05:52:13.143413+0000","last_undegraded":"2023-10-26T05:52:13.143413+0000","last_fullsized":"2023-10-26T05:52:13.143413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:03.201542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.143053+0000","last_change":"2023-10-26T05:52:12.171565+0000","last_active":"2023-10-26T05:52:13.143053+0000","last_peered":"2023-10-26T05:52:13.143053+0000","last_clean":"2023-10-26T05:52:13.143053+0000","last_became_active":"2023-10-26T05:52:12.169127+0000","last_became_peered":"2023-10-26T05:52:12.169127+0000","last_unstale":"2023-10-26T05:52:13.143053+0000","last_undegraded":"2023-10-26T05:52:13.143053+0000","last_fullsized":"2023-10-26T05:52:13.143053+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:08:59.965331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.150662+0000","last_change":"2023-10-26T05:52:11.372884+0000","last_active":"2023-10-26T05:52:13.150662+0000","last_peered":"2023-10-26T05:52:13.150662+0000","last_clean":"2023-10-26T05:52:13.150662+0000","last_became_active":"2023-10-26T05:52:11.370819+0000","last_became_peered":"2023-10-26T05:52:11.370819+0000","last_unstale":"2023-10-26T05:52:13.150662+0000","last_undegraded":"2023-10-26T05:52:13.150662+0000","last_fullsized":"2023-10-26T05:52:13.150662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:45:04.052648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.441539+0000","last_change":"2023-10-26T05:52:12.182371+0000","last_active":"2023-10-26T05:52:13.441539+0000","last_peered":"2023-10-26T05:52:13.441539+0000","last_clean":"2023-10-26T05:52:13.441539+0000","last_became_active":"2023-10-26T05:52:12.181661+0000","last_became_peered":"2023-10-26T05:52:12.181661+0000","last_unstale":"2023-10-26T05:52:13.441539+0000","last_undegraded":"2023-10-26T05:52:13.441539+0000","last_fullsized":"2023-10-26T05:52:13.441539+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:00.210652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443313+0000","last_change":"2023-10-26T05:52:11.368799+0000","last_active":"2023-10-26T05:52:13.443313+0000","last_peered":"2023-10-26T05:52:13.443313+0000","last_clean":"2023-10-26T05:52:13.443313+0000","last_became_active":"2023-10-26T05:52:11.366933+0000","last_became_peered":"2023-10-26T05:52:11.366933+0000","last_unstale":"2023-10-26T05:52:13.443313+0000","last_undegraded":"2023-10-26T05:52:13.443313+0000","last_fullsized":"2023-10-26T05:52:13.443313+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:45:04.052648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.442869+0000","last_change":"2023-10-26T05:52:12.179772+0000","last_active":"2023-10-26T05:52:13.442869+0000","last_peered":"2023-10-26T05:52:13.442869+0000","last_clean":"2023-10-26T05:52:13.442869+0000","last_became_active":"2023-10-26T05:52:12.178107+0000","last_became_peered":"2023-10-26T05:52:12.178107+0000","last_unstale":"2023-10-26T05:52:13.442869+0000","last_undegraded":"2023-10-26T05:52:13.442869+0000","last_fullsized":"2023-10-26T05:52:13.442869+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:31:52.802583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.149585+0000","last_change":"2023-10-26T05:52:11.363951+0000","last_active":"2023-10-26T05:52:13.149585+0000","last_peered":"2023-10-26T05:52:13.149585+0000","last_clean":"2023-10-26T05:52:13.149585+0000","last_became_active":"2023-10-26T05:52:11.363326+0000","last_became_peered":"2023-10-26T05:52:11.363326+0000","last_unstale":"2023-10-26T05:52:13.149585+0000","last_undegraded":"2023-10-26T05:52:13.149585+0000","last_fullsized":"2023-10-26T05:52:13.149585+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:29:21.448875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.438262+0000","last_change":"2023-10-26T05:52:12.153795+0000","last_active":"2023-10-26T05:52:13.438262+0000","last_peered":"2023-10-26T05:52:13.438262+0000","last_clean":"2023-10-26T05:52:13.438262+0000","last_became_active":"2023-10-26T05:52:12.151626+0000","last_became_peered":"2023-10-26T05:52:12.151626+0000","last_unstale":"2023-10-26T05:52:13.438262+0000","last_undegraded":"2023-10-26T05:52:13.438262+0000","last_fullsized":"2023-10-26T05:52:13.438262+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:11.320614+0000","last_clean_scrub_stamp":"2023-10-26T05:52:11.320614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:34:49.750101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-26T05:52:13.443139+0000","last_change":"2023-10-26T05:52:11.348667+0000","last_active":"2023-10-26T05:52:13.443139+0000","last_peered":"2023-10-26T05:52:13.443139+0000","last_clean":"2023-10-26T05:52:13.443139+0000","last_became_active":"2023-10-26T05:52:11.348012+0000","last_became_peered":"2023-10-26T05:52:11.348012+0000","last_unstale":"2023-10-26T05:52:13.443139+0000","last_undegraded":"2023-10-26T05:52:13.443139+0000","last_fullsized":"2023-10-26T05:52:13.443139+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:52:10.308164+0000","last_clean_scrub_stamp":"2023-10-26T05:52:10.308164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:41.484699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":244,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-10-26T05:52:12.162327+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:52:12.162327+0000","last_peered":"2023-10-26T05:52:12.162327+0000","last_clean":"2023-10-26T05:52:12.162327+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:52:12.162327+0000","last_undegraded":"2023-10-26T05:52:12.162327+0000","last_fullsized":"2023-10-26T05:52:12.162327+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:22:25.321192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1123,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194189,"statfs":{"total":200000000,"available":198849592,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84599999999999997,"15min":0.86199999999999999},"min":{"1min":0.58799999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.502,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.286,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.0680000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0109999999999999,"15min":1.0489999999999999},"min":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.218,"5min":1.391,"15min":1.391},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.95699999999999996,"15min":0.96899999999999997},"min":{"1min":0.73499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.3340000000000001,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.117}]},{"osd":2,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82799999999999996,"15min":0.81399999999999995},"min":{"1min":0.55700000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1639999999999999,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.91800000000000004,"15min":0.93600000000000005},"min":{"1min":0.66700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1859999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.96399999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.04,"5min":0.97599999999999998,"15min":0.97399999999999998},"min":{"1min":0.69199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.242,"5min":1.375,"15min":1.375},"last":1.1419999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91000000000000003,"15min":0.91800000000000004},"min":{"1min":0.63200000000000001,"5min":0.433,"15min":0.433},"max":{"1min":1.2909999999999999,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.95299999999999996,"15min":0.93999999999999995},"min":{"1min":0.748,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.2190000000000001,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0669999999999999,"15min":1.125},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2370000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.014,"5min":1.046,"15min":1.089},"min":{"1min":0.69899999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.361,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:51:12 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.077,"15min":1.111},"min":{"1min":0.66200000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.413,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0509999999999999,"15min":1.0489999999999999},"min":{"1min":0.79700000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3280000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.095}]}]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199439336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.82399999999999995,"15min":0.80800000000000005},"min":{"1min":0.52600000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1100000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.88700000000000001,"15min":0.86599999999999999},"min":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.248,"5min":1.3,"15min":1.3},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.81299999999999994,"15min":0.76900000000000002},"min":{"1min":0.56699999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0840000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88,"15min":0.83599999999999997},"min":{"1min":0.58199999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3069999999999999,"5min":1.405,"15min":1.405},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85499999999999998,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.85799999999999998,"15min":0.83199999999999996},"min":{"1min":0.626,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.202,"5min":1.23,"15min":1.23},"last":1.036}]},{"osd":3,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.93700000000000006,"15min":0.88500000000000001},"min":{"1min":0.65500000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.262,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0109999999999999,"15min":0.95699999999999996},"min":{"1min":0.70699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.3169999999999999,"5min":1.399,"15min":1.399},"last":1.1499999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95099999999999996,"15min":0.96099999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.998},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.98099999999999998,"15min":0.97299999999999998},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.294,"5min":1.365,"15min":1.365},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.04,"15min":1.05},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3480000000000001,"5min":1.554,"15min":1.554},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.96699999999999997,"15min":0.997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Oct 26 05:51:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.105},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.042,"5min":1.1040000000000001,"15min":1.1180000000000001},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.339,"5min":1.464,"15min":1.464},"last":1.1819999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199438396,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.55200000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97099999999999997,"15min":0.95499999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.876,"15min":0.90000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.2130000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.04},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.85799999999999998,"15min":0.85699999999999998},"min":{"1min":0.62,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.80800000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.89300000000000002,"15min":0.92700000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2450000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.139},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92300000000000004,"15min":0.93300000000000005},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2709999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92000000000000004,"15min":0.92400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.409,"5min":1.548,"15min":1.548},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.98199999999999998,"15min":1.006},"min":{"1min":0.73399999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4139999999999999,"5min":1.587,"15min":1.587},"last":1.077}]},{"osd":4,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.96899999999999997,"15min":1.01},"min":{"1min":0.60599999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.075,"15min":1.1060000000000001},"min":{"1min":0.67300000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.262,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.97499999999999998,"15min":1.032},"min":{"1min":0.65500000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2290000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.62},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.008,"15min":1.034},"min":{"1min":0.63100000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.056,"15min":1.077},"min":{"1min":0.66800000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.534,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.171},{"interface":"front","average":{"1min":1.071,"5min":1.1279999999999999,"15min":1.1699999999999999},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.996}]}]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199440030,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85399999999999998,"15min":0.84399999999999997},"min":{"1min":0.58299999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.194,"5min":1.347,"15min":1.347},"last":1.2130000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.86499999999999999,"15min":0.877},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1359999999999999,"5min":1.502,"15min":1.502},"last":1.083}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.83599999999999997,"15min":0.80800000000000005},"min":{"1min":0.54500000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1819999999999999,"5min":1.375,"15min":1.375},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.91600000000000004,"15min":0.88900000000000001},"min":{"1min":0.755,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.282,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.163}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.84499999999999997,"15min":0.80900000000000005},"min":{"1min":0.63,"5min":0.501,"15min":0.501},"max":{"1min":1.1910000000000001,"5min":1.29,"15min":1.29},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.85999999999999999},"min":{"1min":0.57599999999999996,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.226,"5min":1.373,"15min":1.373},"last":0.995}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.92600000000000005,"15min":0.94199999999999995},"min":{"1min":0.66500000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.107},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96499999999999997,"15min":0.93999999999999995},"min":{"1min":0.73199999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.3300000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.1359999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89500000000000002,"15min":0.88300000000000001},"min":{"1min":0.60999999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.998,"15min":0.999},"min":{"1min":0.64200000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.2470000000000001,"5min":1.673,"15min":1.673},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0069999999999999,"15min":0.998},"min":{"1min":0.69499999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.383,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.999,"5min":1.032,"15min":1.0369999999999999},"min":{"1min":0.72899999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4690000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.044,"15min":1.034},"min":{"1min":0.83599999999999997,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.4330000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.05},{"interface":"front","average":{"1min":1.157,"5min":1.119,"15min":1.115},"min":{"1min":0.95199999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.403,"5min":1.728,"15min":1.728},"last":0.74199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":542,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194770,"statfs":{"total":200000000,"available":199444718,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.33600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.95099999999999996,"5min":1.155,"15min":1.155},"last":0.505},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.43099999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.073,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.64200000000000002,"15min":0.65600000000000003},"min":{"1min":0.54300000000000004,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.629,"15min":0.59399999999999997},"min":{"1min":0.32200000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.038,"5min":1.085,"15min":1.085},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76400000000000001,"15min":0.75},"min":{"1min":0.57999999999999996,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.278,"5min":1.367,"15min":1.367},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.39300000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.236,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.85899999999999999,"15min":0.82299999999999995},"min":{"1min":0.65900000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.92900000000000005,"15min":0.92700000000000005},"min":{"1min":0.629,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.27,"5min":1.3,"15min":1.3},"last":0.72199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.96399999999999997,"15min":0.96599999999999997},"min":{"1min":0.71899999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.248,"5min":1.478,"15min":1.478},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.92200000000000004,"15min":0.90000000000000002},"min":{"1min":0.64300000000000002,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3100000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.80500000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.96899999999999997,"15min":1.004},"min":{"1min":0.66200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0589999999999999,"15min":1.03},"min":{"1min":0.71399999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2330000000000001,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.94099999999999995,"15min":0.91500000000000004},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.383,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.038,"15min":1.046},"min":{"1min":0.68899999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.4019999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.42399999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.64200000000000002,"15min":0.58199999999999996},"min":{"1min":0.50800000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.012,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.65400000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.54900000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2090000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.63100000000000001,"15min":0.57999999999999996},"min":{"1min":0.58399999999999996,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.0740000000000001,"5min":1.294,"15min":1.294},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.77700000000000002,"15min":0.72299999999999998},"min":{"1min":0.66400000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.113,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.628,"15min":0.57899999999999996},"min":{"1min":0.56799999999999995,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.86499999999999999,"15min":0.81200000000000006},"min":{"1min":0.78200000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.224,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88900000000000001,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2310000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":1.02}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94799999999999995,"15min":0.89700000000000002},"min":{"1min":0.77700000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.353,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.92400000000000004,"15min":0.88800000000000001},"min":{"1min":0.77600000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4370000000000001,"5min":1.653,"15min":1.653},"last":1.2110000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.95399999999999996,"15min":0.90400000000000003},"min":{"1min":0.88500000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.526,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.05,"15min":0.95699999999999996},"min":{"1min":0.72899999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.492,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.91300000000000003,"15min":0.88700000000000001},"min":{"1min":0.84899999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.21,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.159,"15min":1.0129999999999999},"min":{"1min":0.91800000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.571,"5min":15.352,"15min":15.352},"last":1.1739999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194765,"statfs":{"total":200000000,"available":199439836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40699999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.97699999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.63700000000000001,"15min":0.626},"min":{"1min":0.311,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.39100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0209999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.72999999999999998,"15min":0.71499999999999997},"min":{"1min":0.434,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0860000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.35299999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.165,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.496},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.69599999999999995,"15min":0.68100000000000005},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0700000000000001,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.873,"15min":0.86399999999999999},"min":{"1min":0.56899999999999995,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1379999999999999,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.96899999999999997,"15min":0.94499999999999995},"min":{"1min":0.75900000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.286,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.73899999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.262,"5min":1.492,"15min":1.492},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.93000000000000005,"15min":0.93400000000000005},"min":{"1min":0.61199999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.208,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.747}]},{"osd":6,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.026,"15min":1.04},"min":{"1min":0.61899999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.208,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.0129999999999999,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.72599999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.23,"5min":1.837,"15min":1.837},"last":0.876}]},{"osd":7,"last update":"Thu Oct 26 05:51:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.01,"15min":1.028},"min":{"1min":0.71299999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.248,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.095,"15min":1.093},"min":{"1min":0.64900000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.258,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":0.89700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1123,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194189,"statfs":{"total":200000000,"available":198849628,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.55500000000000005,"15min":0.53300000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.93200000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58899999999999997},"min":{"1min":0.317,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.0189999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.622}]},{"osd":2,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.68600000000000005,"15min":0.64600000000000002},"min":{"1min":0.34799999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.0329999999999999,"5min":1.173,"15min":1.173},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.622,"15min":0.59599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.177},"last":0.57199999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.627,"15min":0.61299999999999999},"min":{"1min":0.33300000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.97599999999999998,"5min":1.409,"15min":1.409},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70099999999999996,"15min":0.67500000000000004},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.2090000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.85099999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.877,"15min":0.84399999999999997},"min":{"1min":0.67900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.90700000000000003,"15min":0.88900000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.24,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.1160000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.92900000000000005,"15min":0.92100000000000004},"min":{"1min":0.627,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.28,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.91000000000000003,"15min":0.89800000000000002},"min":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.22,"5min":1.27,"15min":1.27},"last":1.139}]},{"osd":6,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92000000000000004,"15min":0.90000000000000002},"min":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.012},{"interface":"front","average":{"1min":1.02,"5min":0.98299999999999998,"15min":0.95299999999999996},"min":{"1min":0.60499999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.274,"5min":1.764,"15min":1.764},"last":1.0600000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:51:14 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.98799999999999999,"15min":0.99299999999999999},"min":{"1min":0.62,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5780000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.747,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.288,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.093}]}]}],"pool_statfs":[]}} 2023-10-26T05:52:15.719 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:52:15.719 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:52:15.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 -- 172.21.15.71:0/2354810063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0850 msgr2=0x7f90a815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 --2- 172.21.15.71:0/2354810063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0850 0x7f90a815b3a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f909c009840 tx=0x7f909c01c6b0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 -- 172.21.15.71:0/2354810063 shutdown_connections 2023-10-26T05:52:15.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 --2- 172.21.15.71:0/2354810063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0850 0x7f90a815b3a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 --2- 172.21.15.71:0/2354810063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90a80d0470 0x7f90a806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 -- 172.21.15.71:0/2354810063 >> 172.21.15.71:0/2354810063 conn(0x7f90a80cdd80 msgr2=0x7f90a80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:15.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 -- 172.21.15.71:0/2354810063 shutdown_connections 2023-10-26T05:52:15.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 -- 172.21.15.71:0/2354810063 wait complete. 2023-10-26T05:52:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.884+0000 7f90aed82640 1 Processor -- start 2023-10-26T05:52:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- start start 2023-10-26T05:52:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90a80cd880 0x7f90a8154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 0x7f90a815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a80d0850 0x7f90a81532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a815d0d0 con 0x7f90a80d0470 2023-10-26T05:52:15.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a815c4c0 con 0x7f90a80d0850 2023-10-26T05:52:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a815c7a0 con 0x7f90a80cd880 2023-10-26T05:52:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 0x7f90a815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 0x7f90a815a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60086/0 (socket says 172.21.15.71:60086) 2023-10-26T05:52:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 -- 172.21.15.71:0/202734333 learned_addr learned my addr 172.21.15.71:0/202734333 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:15.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90acb1e640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90a80cd880 0x7f90a8154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 -- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90a80cd880 msgr2=0x7f90a8154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90a80cd880 0x7f90a8154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 -- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a80d0850 msgr2=0x7f90a81532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:15.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a80d0850 0x7f90a81532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:15.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 -- 172.21.15.71:0/202734333 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0094a0 con 0x7f90a80d0470 2023-10-26T05:52:15.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a7fff640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 0x7f90a815a010 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f909c009790 tx=0x7f909c0a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:15.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0042f0 con 0x7f90a80d0470 2023-10-26T05:52:15.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f909c005070 con 0x7f90a80d0470 2023-10-26T05:52:15.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c005460 con 0x7f90a80d0470 2023-10-26T05:52:15.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a815ca80 con 0x7f90a80d0470 2023-10-26T05:52:15.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a815d560 con 0x7f90a80d0470 2023-10-26T05:52:15.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a8066b10 con 0x7f90a80d0470 2023-10-26T05:52:15.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f909c0b4040 con 0x7f90a80d0470 2023-10-26T05:52:15.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a57fa640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9094066470 0x7f9094068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:15.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.888+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f909c11d990 con 0x7f90a80d0470 2023-10-26T05:52:15.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.892+0000 7f90acb1e640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9094066470 0x7f9094068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:15.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.892+0000 7f90acb1e640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9094066470 0x7f9094068930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f90980058d0 tx=0x7f9098005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:15.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:15.896+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f909c0eae10 con 0x7f90a80d0470 2023-10-26T05:52:16.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.104+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f90a806ade0 con 0x7f90a80d0470 2023-10-26T05:52:16.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.108+0000 7f90a57fa640 1 -- 172.21.15.71:0/202734333 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) v1 ==== 190+0+147 (secure 0 0 0) 0x7f909c0b2020 con 0x7f90a80d0470 2023-10-26T05:52:16.116 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:52:16.116 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:52:16.117 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:52:16.117 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:52:16.117 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:52:16.117 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:52:16.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9094066470 msgr2=0x7f9094068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9094066470 0x7f9094068930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f90980058d0 tx=0x7f9098005860 comp rx=0 tx=0).stop 2023-10-26T05:52:16.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 msgr2=0x7f90a815a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 0x7f90a815a010 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f909c009790 tx=0x7f909c0a5660 comp rx=0 tx=0).stop 2023-10-26T05:52:16.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 shutdown_connections 2023-10-26T05:52:16.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90a80d0850 0x7f90a81532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9094066470 0x7f9094068930 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90a80d0470 0x7f90a815a010 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 --2- 172.21.15.71:0/202734333 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f90a80cd880 0x7f90a8154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 >> 172.21.15.71:0/202734333 conn(0x7f90a80cdd80 msgr2=0x7f90a80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:16.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 shutdown_connections 2023-10-26T05:52:16.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.112+0000 7f90aed82640 1 -- 172.21.15.71:0/202734333 wait complete. 2023-10-26T05:52:16.135 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:16.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/2114972417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fd0005190 msgr2=0x7f9fd000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/2114972417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fd0005190 0x7f9fd000b990 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c5c0 tx=0x7f9fdc01d340 comp rx=0 tx=0).stop 2023-10-26T05:52:16.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/2114972417 shutdown_connections 2023-10-26T05:52:16.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/2114972417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fd0005190 0x7f9fd000b990 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/2114972417 >> 172.21.15.71:0/2114972417 conn(0x7f9fe4069770 msgr2=0x7f9fe4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:16.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/2114972417 shutdown_connections 2023-10-26T05:52:16.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/2114972417 wait complete. 2023-10-26T05:52:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 Processor -- start 2023-10-26T05:52:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- start start 2023-10-26T05:52:16.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd0005190 0x7f9fe4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe41352f0 0x7f9fe413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 0x7f9fe41589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40c2190 con 0x7f9fe4136900 2023-10-26T05:52:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4157fc0 con 0x7f9fd0005190 2023-10-26T05:52:16.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe9d3a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4135860 con 0x7f9fe41352f0 2023-10-26T05:52:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 0x7f9fe41589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 0x7f9fe41589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60112/0 (socket says 172.21.15.71:60112) 2023-10-26T05:52:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.240+0000 7f9fe3fff640 1 -- 172.21.15.71:0/530027615 learned_addr learned my addr 172.21.15.71:0/530027615 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe37fe640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd0005190 0x7f9fe4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fdaffd640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe41352f0 0x7f9fe413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:16.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe3fff640 1 -- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe41352f0 msgr2=0x7f9fe413fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe3fff640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe41352f0 0x7f9fe413fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe3fff640 1 -- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd0005190 msgr2=0x7f9fe4157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe3fff640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd0005190 0x7f9fe4157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe3fff640 1 -- 172.21.15.71:0/530027615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4135b40 con 0x7f9fe4136900 2023-10-26T05:52:16.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe3fff640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 0x7f9fe41589e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc01dbe0 tx=0x7f9fdc0013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:16.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0b4070 con 0x7f9fe4136900 2023-10-26T05:52:16.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9fdc004b10 con 0x7f9fe4136900 2023-10-26T05:52:16.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe4135e20 con 0x7f9fe4136900 2023-10-26T05:52:16.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc001a00 con 0x7f9fe4136900 2023-10-26T05:52:16.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe41363e0 con 0x7f9fe4136900 2023-10-26T05:52:16.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4066b10 con 0x7f9fe4136900 2023-10-26T05:52:16.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9fdc0ab030 con 0x7f9fe4136900 2023-10-26T05:52:16.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe0ff9640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc40664a0 0x7f9fc4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:16.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fdc11d6b0 con 0x7f9fe4136900 2023-10-26T05:52:16.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe37fe640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc40664a0 0x7f9fc4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:16.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.244+0000 7f9fe37fe640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc40664a0 0x7f9fc4068960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400db30 tx=0x7f9fd400d360 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:16.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.248+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9fdc0eab60 con 0x7f9fe4136900 2023-10-26T05:52:16.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fe40721e0 con 0x7f9fe4136900 2023-10-26T05:52:16.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe0ff9640 1 -- 172.21.15.71:0/530027615 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f9fdc0ee180 con 0x7f9fe4136900 2023-10-26T05:52:16.430 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:16.431 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:16.431 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:16.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc40664a0 msgr2=0x7f9fc4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc40664a0 0x7f9fc4068960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400db30 tx=0x7f9fd400d360 comp rx=0 tx=0).stop 2023-10-26T05:52:16.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 msgr2=0x7f9fe41589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 0x7f9fe41589e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc01dbe0 tx=0x7f9fdc0013a0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 shutdown_connections 2023-10-26T05:52:16.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc40664a0 0x7f9fc4068960 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4136900 0x7f9fe41589e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe41352f0 0x7f9fe413fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 --2- 172.21.15.71:0/530027615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd0005190 0x7f9fe4157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 >> 172.21.15.71:0/530027615 conn(0x7f9fe4069770 msgr2=0x7f9fe40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:16.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 shutdown_connections 2023-10-26T05:52:16.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.424+0000 7f9fe9d3a640 1 -- 172.21.15.71:0/530027615 wait complete. 2023-10-26T05:52:16.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:52:11.692352+0000', 'modified': '2023-10-26T05:52:13.702898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7074}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7074': {'gid': 7074, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/2440514167', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 2440514167}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2440514167}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:52:16.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:52:16.852 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T05:52:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 -- 172.21.15.71:0/2379551438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0b1ae0 msgr2=0x7f2bcc0b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/2379551438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0b1ae0 0x7f2bcc0b1f70 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc070860 tx=0x7f2bc401c830 comp rx=0 tx=0).stop 2023-10-26T05:52:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 -- 172.21.15.71:0/2379551438 shutdown_connections 2023-10-26T05:52:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/2379551438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0b1ae0 0x7f2bcc0b1f70 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/2379551438 >> v2:172.21.15.86:3300/0 conn(0x7f2bcc157da0 0x7f2bcc158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 -- 172.21.15.71:0/2379551438 >> 172.21.15.71:0/2379551438 conn(0x7f2bcc06b8c0 msgr2=0x7f2bcc06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 -- 172.21.15.71:0/2379551438 shutdown_connections 2023-10-26T05:52:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.980+0000 7f2bd36f5640 1 -- 172.21.15.71:0/2379551438 wait complete. 2023-10-26T05:52:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 Processor -- start 2023-10-26T05:52:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 -- start start 2023-10-26T05:52:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2bcc0b1ae0 0x7f2bcc0aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 0x7f2bcc0ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc157da0 0x7f2bcc149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc149ca0 con 0x7f2bcc0cc910 2023-10-26T05:52:17.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc149f80 con 0x7f2bcc157da0 2023-10-26T05:52:17.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc14a260 con 0x7f2bcc0b1ae0 2023-10-26T05:52:17.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 0x7f2bcc0ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 0x7f2bcc0ab940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60130/0 (socket says 172.21.15.71:60130) 2023-10-26T05:52:17.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 -- 172.21.15.71:0/3030464762 learned_addr learned my addr 172.21.15.71:0/3030464762 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:17.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd1491640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2bcc0b1ae0 0x7f2bcc0aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd1c92640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc157da0 0x7f2bcc149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 -- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2bcc0b1ae0 msgr2=0x7f2bcc0aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2bcc0b1ae0 0x7f2bcc0aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 -- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc157da0 msgr2=0x7f2bcc149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc157da0 0x7f2bcc149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 -- 172.21.15.71:0/3030464762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4009560 con 0x7f2bcc0cc910 2023-10-26T05:52:17.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 0x7f2bcc0ab940 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00c9f0 tx=0x7f2bbc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bc1ffb640 1 -- 172.21.15.71:0/3030464762 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc007bb0 con 0x7f2bcc0cc910 2023-10-26T05:52:17.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bc1ffb640 1 -- 172.21.15.71:0/3030464762 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2bbc007e00 con 0x7f2bcc0cc910 2023-10-26T05:52:17.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bcc158fe0 con 0x7f2bcc0cc910 2023-10-26T05:52:17.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bc1ffb640 1 -- 172.21.15.71:0/3030464762 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00f6b0 con 0x7f2bcc0cc910 2023-10-26T05:52:17.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc149890 con 0x7f2bcc0cc910 2023-10-26T05:52:17.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.984+0000 7f2bc1ffb640 1 -- 172.21.15.71:0/3030464762 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2bbc00f810 con 0x7f2bcc0cc910 2023-10-26T05:52:17.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bc1ffb640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2bb0066470 0x7f2bb0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bc1ffb640 1 -- 172.21.15.71:0/3030464762 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bbc0846d0 con 0x7f2bcc0cc910 2023-10-26T05:52:17.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2bb0066470 0x7f2bb0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 Processor -- start 2023-10-26T05:52:17.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- start start 2023-10-26T05:52:17.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2bb0066470 0x7f2bb0068930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005530 tx=0x7f2bc8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f452454da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f45240aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f45240b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4521ffcc0 con 0x55f4524549c0 2023-10-26T05:52:17.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f452262920 con 0x55f4524552e0 2023-10-26T05:52:17.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f45225fe50 con 0x55f45240af80 2023-10-26T05:52:17.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f452454da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f452454da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60132/0 (socket says 172.21.15.71:60132) 2023-10-26T05:52:17.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 -- 172.21.15.71:0/457158233 learned_addr learned my addr 172.21.15.71:0/457158233 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:17.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f45240aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 -- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 msgr2=0x55f45240b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f45240b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 -- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 msgr2=0x55f45240aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f45240aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 -- 172.21.15.71:0/457158233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f451f54890 con 0x55f4524549c0 2023-10-26T05:52:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd1491640 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f452454da0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800eaf0 tx=0x7f2bc8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cb8480c9a5b7fa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bb6ffd640 1 -- 172.21.15.71:0/457158233 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8015c80 con 0x55f4524549c0 2023-10-26T05:52:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bb6ffd640 1 -- 172.21.15.71:0/457158233 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2bc8020400 con 0x55f4524549c0 2023-10-26T05:52:17.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bb6ffd640 1 -- 172.21.15.71:0/457158233 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80207d0 con 0x55f4524549c0 2023-10-26T05:52:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- 172.21.15.71:0/457158233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 msgr2=0x55f452454da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f452454da0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800eaf0 tx=0x7f2bc8011000 comp rx=0 tx=0).stop 2023-10-26T05:52:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- 172.21.15.71:0/457158233 shutdown_connections 2023-10-26T05:52:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f45240b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f45240aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 --2- 172.21.15.71:0/457158233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f452454da0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- 172.21.15.71:0/457158233 >> 172.21.15.71:0/457158233 conn(0x55f4524022f0 msgr2=0x55f452409320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- 172.21.15.71:0/457158233 shutdown_connections 2023-10-26T05:52:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- 172.21.15.71:0/457158233 wait complete. 2023-10-26T05:52:17.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 -- start start 2023-10-26T05:52:17.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.988+0000 7f2bd683f000 1 Processor -- start 2023-10-26T05:52:17.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452415b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f4524160d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f452416610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45225b740 con 0x55f4524549c0 2023-10-26T05:52:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45221ad10 con 0x55f4524552e0 2023-10-26T05:52:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f45221a9b0 con 0x55f45240af80 2023-10-26T05:52:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 10 client.0 _ll_register_callbacks cb 0x55f4523f66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:52:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f4524160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f4524160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60138/0 (socket says 172.21.15.71:60138) 2023-10-26T05:52:17.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 -- 172.21.15.71:0/2924019248 learned_addr learned my addr 172.21.15.71:0/2924019248 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452415b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd1c92640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f452416610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 -- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 msgr2=0x55f452415b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452415b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 -- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 msgr2=0x55f452416610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f452416610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 -- 172.21.15.71:0/2924019248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc800f040 con 0x55f4524549c0 2023-10-26T05:52:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f4524160d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc05d5e0 tx=0x7f2bbc0558d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe5570ef3db3d13 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:52:17.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 1 -- 172.21.15.71:0/2924019248 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc054d00 con 0x55f4524549c0 2023-10-26T05:52:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f45220f370 con 0x55f4524549c0 2023-10-26T05:52:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 1 -- 172.21.15.71:0/2924019248 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bbc054e60 con 0x55f4524549c0 2023-10-26T05:52:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 1 -- 172.21.15.71:0/2924019248 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bbc05d680 con 0x55f4524549c0 2023-10-26T05:52:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 10 client.7221 fetch_fsmap learned FSMap version 38 2023-10-26T05:52:17.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55f45221d460 con 0x55f4524549c0 2023-10-26T05:52:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 1 -- 172.21.15.71:0/2924019248 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55f45220f370 con 0x55f4524549c0 2023-10-26T05:52:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 1 -- 172.21.15.71:0/2924019248 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bbc016020 con 0x55f4524549c0 2023-10-26T05:52:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 10 client.7221 fetch_fsmap finished waiting for FSMap version 38 2023-10-26T05:52:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 10 client.7221 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:52:17.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 20 client.7221 populate_metadata read hostname 'smithi071' 2023-10-26T05:52:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f452463a50 0x55f452465e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 4 client.7221 mds_command: new command op to 7074 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:52:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 --> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4521f52d0 con 0x55f452463a50 2023-10-26T05:52:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f452463a50 0x55f452465e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f452463a50 0x55f452465e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acdd0d3b15adc28f server_cookie=17af935c31294830 in_seq=0 out_seq=0 2023-10-26T05:52:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 10 client.7221 ms_handle_connect on v2:172.21.15.71:6834/2440514167 2023-10-26T05:52:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 1 -- 172.21.15.71:0/2924019248 <== mds.0 v2:172.21.15.71:6834/2440514167 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4521f52d0 con 0x55f452463a50 2023-10-26T05:52:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 10 client.7221 handle_command_reply: tid=0 2023-10-26T05:52:17.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 client.7221 shutdown 2023-10-26T05:52:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 msgr2=0x55f4524160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f4524160d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc05d5e0 tx=0x7f2bbc0558d0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 shutdown_connections 2023-10-26T05:52:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524552e0 0x55f452416610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f452463a50 0x55f452465e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524549c0 0x55f4524160d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2babfff640 0 client.7221 ms_handle_reset on v2:172.21.15.71:6834/2440514167 2023-10-26T05:52:17.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2924019248 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452415b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 >> 172.21.15.71:0/2924019248 conn(0x55f4524022f0 msgr2=0x55f452409a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 shutdown_connections 2023-10-26T05:52:17.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 1 -- 172.21.15.71:0/2924019248 wait complete. 2023-10-26T05:52:17.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:16.992+0000 7f2bd683f000 20 client.7221 trim_cache size 0 max 16384 2023-10-26T05:52:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 Processor -- start 2023-10-26T05:52:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- start start 2023-10-26T05:52:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 0x55f4524aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f4524aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f4524552e0 0x55f4524af490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f451f15940 con 0x55f45240af80 2023-10-26T05:52:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4522096c0 con 0x55f4524549c0 2023-10-26T05:52:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f451f551b0 con 0x55f4524552e0 2023-10-26T05:52:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 0x55f4524aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 0x55f4524aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60142/0 (socket says 172.21.15.71:60142) 2023-10-26T05:52:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 -- 172.21.15.71:0/3555028044 learned_addr learned my addr 172.21.15.71:0/3555028044 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1c92640 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f4524552e0 0x55f4524af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 -- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f4524552e0 msgr2=0x55f4524af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f4524552e0 0x55f4524af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 -- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 msgr2=0x55f4524aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f4524aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f4524aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 -- 172.21.15.71:0/3555028044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f452203760 con 0x55f45240af80 2023-10-26T05:52:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd1491640 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 0x55f4524aea10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005200 tx=0x7f2bc8003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34d9bea6b3b8aee server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bb77fe640 1 -- 172.21.15.71:0/3555028044 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80204e0 con 0x55f45240af80 2023-10-26T05:52:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bb77fe640 1 -- 172.21.15.71:0/3555028044 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2bc8019050 con 0x55f45240af80 2023-10-26T05:52:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bb77fe640 1 -- 172.21.15.71:0/3555028044 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc802fcd0 con 0x55f45240af80 2023-10-26T05:52:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- 172.21.15.71:0/3555028044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 msgr2=0x55f4524aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 0x55f4524aea10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005200 tx=0x7f2bc8003af0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- 172.21.15.71:0/3555028044 shutdown_connections 2023-10-26T05:52:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f4524552e0 0x55f4524af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f4524aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 --2- 172.21.15.71:0/3555028044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f45240af80 0x55f4524aea10 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.008+0000 7f2bd683f000 1 -- 172.21.15.71:0/3555028044 >> 172.21.15.71:0/3555028044 conn(0x55f4524022f0 msgr2=0x55f4524666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/3555028044 shutdown_connections 2023-10-26T05:52:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/3555028044 wait complete. 2023-10-26T05:52:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- start start 2023-10-26T05:52:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 Processor -- start 2023-10-26T05:52:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452453130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f45246f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 0x55f45246f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f452224720 con 0x55f4524552e0 2023-10-26T05:52:17.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f452207cc0 con 0x55f4524549c0 2023-10-26T05:52:17.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4521f90c0 con 0x55f45240af80 2023-10-26T05:52:17.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 10 client.0 _ll_register_callbacks cb 0x55f452259ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:52:17.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 0x55f45246f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 0x55f45246f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60144/0 (socket says 172.21.15.71:60144) 2023-10-26T05:52:17.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 -- 172.21.15.71:0/2287857849 learned_addr learned my addr 172.21.15.71:0/2287857849 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:17.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452453130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd0c90640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f45246f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 -- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 msgr2=0x55f452453130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452453130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 -- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 msgr2=0x55f45246f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:52:17.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f45246f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 -- 172.21.15.71:0/2287857849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f451f54890 con 0x55f4524552e0 2023-10-26T05:52:17.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452453130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:17.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1c92640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 0x55f45246f740 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2bc401aee0 tx=0x7f2bc40a5900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c991bd59d9de72 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:52:17.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 1 -- 172.21.15.71:0/2287857849 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004240 con 0x55f4524552e0 2023-10-26T05:52:17.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 1 -- 172.21.15.71:0/2287857849 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bc40043a0 con 0x55f4524552e0 2023-10-26T05:52:17.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f45240e8f0 con 0x55f4524552e0 2023-10-26T05:52:17.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 1 -- 172.21.15.71:0/2287857849 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc4005040 con 0x55f4524552e0 2023-10-26T05:52:17.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 10 client.7227 fetch_fsmap learned FSMap version 38 2023-10-26T05:52:17.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x55f451f54cc0 con 0x55f4524552e0 2023-10-26T05:52:17.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 1 -- 172.21.15.71:0/2287857849 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x55f45240e8f0 con 0x55f4524552e0 2023-10-26T05:52:17.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 1 -- 172.21.15.71:0/2287857849 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2bc40bf060 con 0x55f4524552e0 2023-10-26T05:52:17.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 10 client.7227 fetch_fsmap finished waiting for FSMap version 38 2023-10-26T05:52:17.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 10 client.7227 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:52:17.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 20 client.7227 populate_metadata read hostname 'smithi071' 2023-10-26T05:52:17.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f4524b27a0 0x55f4524b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 4 client.7227 mds_command: new command op to 7074 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:52:17.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 --> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f4521f36f0 con 0x55f4524b27a0 2023-10-26T05:52:17.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f4524b27a0 0x55f4524b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 10 client.7227 ms_handle_connect on v2:172.21.15.71:6834/2440514167 2023-10-26T05:52:17.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd1491640 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f4524b27a0 0x55f4524b2b80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=582f8de9d36131f server_cookie=5437115905ccebb5 in_seq=0 out_seq=0 2023-10-26T05:52:17.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 1 -- 172.21.15.71:0/2287857849 <== mds.0 v2:172.21.15.71:6834/2440514167 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55f4521f36f0 con 0x55f4524b27a0 2023-10-26T05:52:17.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 10 client.7227 handle_command_reply: tid=0 2023-10-26T05:52:17.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 client.7227 shutdown 2023-10-26T05:52:17.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 msgr2=0x55f45246f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 0x55f45246f740 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2bc401aee0 tx=0x7f2bc40a5900 comp rx=0 tx=0).stop 2023-10-26T05:52:17.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 shutdown_connections 2023-10-26T05:52:17.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:6834/2440514167,v1:172.21.15.71:6835/2440514167] conn(0x55f4524b27a0 0x55f4524b2b80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f4524552e0 0x55f45246f740 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f4524549c0 0x55f45246f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 --2- 172.21.15.71:0/2287857849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55f45240af80 0x55f452453130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2babfff640 0 client.7227 ms_handle_reset on v2:172.21.15.71:6834/2440514167 2023-10-26T05:52:17.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 >> 172.21.15.71:0/2287857849 conn(0x55f4524022f0 msgr2=0x55f452466dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 shutdown_connections 2023-10-26T05:52:17.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 1 -- 172.21.15.71:0/2287857849 wait complete. 2023-10-26T05:52:17.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.012+0000 7f2bd683f000 20 client.7227 trim_cache size 0 max 16384 2023-10-26T05:52:17.056 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:52:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7074, 2023-10-26T05:52:17.057 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:52:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:52:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:52:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 4.6699390760000004, 2023-10-26T05:52:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 38, 2023-10-26T05:52:17.058 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 103, 2023-10-26T05:52:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 103, 2023-10-26T05:52:17.059 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 10.303417122999999 2023-10-26T05:52:17.059 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:17.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2bb0066470 msgr2=0x7f2bb0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2bb0066470 0x7f2bb0068930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8005530 tx=0x7f2bc8009040 comp rx=0 tx=0).stop 2023-10-26T05:52:17.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 msgr2=0x7f2bcc0ab940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 0x7f2bcc0ab940 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00c9f0 tx=0x7f2bbc00ceb0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 shutdown_connections 2023-10-26T05:52:17.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bcc157da0 0x7f2bcc149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2bb0066470 0x7f2bb0068930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bcc0cc910 0x7f2bcc0ab940 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 --2- 172.21.15.71:0/3030464762 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2bcc0b1ae0 0x7f2bcc0aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 >> 172.21.15.71:0/3030464762 conn(0x7f2bcc06b8c0 msgr2=0x7f2bcc06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 shutdown_connections 2023-10-26T05:52:17.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.016+0000 7f2bd36f5640 1 -- 172.21.15.71:0/3030464762 wait complete. 2023-10-26T05:52:17.063 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:52:17.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 -- 172.21.15.71:0/2107208870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4140cc0 msgr2=0x7f21b415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 --2- 172.21.15.71:0/2107208870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4140cc0 0x7f21b415c760 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009870 tx=0x7f21a401c790 comp rx=0 tx=0).stop 2023-10-26T05:52:17.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 -- 172.21.15.71:0/2107208870 shutdown_connections 2023-10-26T05:52:17.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 --2- 172.21.15.71:0/2107208870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4140cc0 0x7f21b415c760 secure :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009870 tx=0x7f21a401c790 comp rx=0 tx=0).stop 2023-10-26T05:52:17.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 -- 172.21.15.71:0/2107208870 >> 172.21.15.71:0/2107208870 conn(0x7f21b4069770 msgr2=0x7f21b4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 -- 172.21.15.71:0/2107208870 shutdown_connections 2023-10-26T05:52:17.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.168+0000 7f21b91e0640 1 -- 172.21.15.71:0/2107208870 wait complete. 2023-10-26T05:52:17.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 Processor -- start 2023-10-26T05:52:17.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- start start 2023-10-26T05:52:17.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4070fe0 0x7f21b4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21b40b2370 0x7f21b41392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 0x7f21b4135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41639b0 con 0x7f21b4161440 2023-10-26T05:52:17.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4163e40 con 0x7f21b4070fe0 2023-10-26T05:52:17.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4162b00 con 0x7f21b40b2370 2023-10-26T05:52:17.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 0x7f21b4135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 0x7f21b4135270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60160/0 (socket says 172.21.15.71:60160) 2023-10-26T05:52:17.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 -- 172.21.15.71:0/3419929275 learned_addr learned my addr 172.21.15.71:0/3419929275 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:17.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b259c640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21b40b2370 0x7f21b41392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b2d9d640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4070fe0 0x7f21b4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 -- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21b40b2370 msgr2=0x7f21b41392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21b40b2370 0x7f21b41392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 -- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4070fe0 msgr2=0x7f21b4156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4070fe0 0x7f21b4156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 -- 172.21.15.71:0/3419929275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40094a0 con 0x7f21b4161440 2023-10-26T05:52:17.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b2d9d640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4070fe0 0x7f21b4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:17.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b359e640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 0x7f21b4135270 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00cb40 tx=0x7f21ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00aa50 con 0x7f21b4161440 2023-10-26T05:52:17.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f21ac00aca0 con 0x7f21b4161440 2023-10-26T05:52:17.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b4162da0 con 0x7f21b4161440 2023-10-26T05:52:17.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac012680 con 0x7f21b4161440 2023-10-26T05:52:17.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b4163360 con 0x7f21b4161440 2023-10-26T05:52:17.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b4066b10 con 0x7f21b4161440 2023-10-26T05:52:17.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.172+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f21ac01b400 con 0x7f21b4161440 2023-10-26T05:52:17.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.176+0000 7f21ab7fe640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21940664a0 0x7f2194068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:17.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.176+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21ac084670 con 0x7f21b4161440 2023-10-26T05:52:17.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.176+0000 7f21b2d9d640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21940664a0 0x7f2194068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:17.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.176+0000 7f21b2d9d640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21940664a0 0x7f2194068960 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f21a40af000 tx=0x7f21a401afd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:17.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.180+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f21ac051a00 con 0x7f21b4161440 2023-10-26T05:52:17.296 INFO:tasks.ceph.mds.c.smithi071.stderr:2023-10-26T05:52:17.288+0000 7f268f737640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:52:17.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.368+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21b40721e0 con 0x7f21b4161440 2023-10-26T05:52:17.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.368+0000 7f21ab7fe640 1 -- 172.21.15.71:0/3419929275 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f21ac055020 con 0x7f21b4161440 2023-10-26T05:52:17.376 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:52:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21940664a0 msgr2=0x7f2194068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21940664a0 0x7f2194068960 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f21a40af000 tx=0x7f21a401afd0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 msgr2=0x7f21b4135270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 0x7f21b4135270 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00cb40 tx=0x7f21ac00cc30 comp rx=0 tx=0).stop 2023-10-26T05:52:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 shutdown_connections 2023-10-26T05:52:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21940664a0 0x7f2194068960 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21b4161440 0x7f21b4135270 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f21b40b2370 0x7f21b41392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 --2- 172.21.15.71:0/3419929275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b4070fe0 0x7f21b4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 >> 172.21.15.71:0/3419929275 conn(0x7f21b4069770 msgr2=0x7f21b40b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 shutdown_connections 2023-10-26T05:52:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:17.372+0000 7f21b91e0640 1 -- 172.21.15.71:0/3419929275 wait complete. 2023-10-26T05:52:17.395 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:52:17.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:52:17.396 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:52:17.396 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:52:17.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:17.398 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:52:17.398 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:52:17.399 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:52:17.399 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:52:17.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.400 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:52:17.406 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:52:17.407 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:52:17.408 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:52:17.408 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.408 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:52:17.408 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.408 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:52:17.409 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:17.409 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:52:17.409 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:52:17.409 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:17.409 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:52:17.410 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:52:17.410 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:17.410 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:52:17.410 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:17.410 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:52:17.411 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42239sec preferred_lft 42239sec 2023-10-26T05:52:17.411 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:52:17.411 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.411 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:52:17.411 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:17.412 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:52:17.412 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.412 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:52:17.412 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:17.412 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:52:17.413 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.413 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:52:17.413 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.413 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:52:17.413 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:17.414 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:52:17.414 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.414 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:52:17.414 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:17.414 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:52:17.415 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:17.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.415 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:52:17.458 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:52:17.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.460 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:52:17.506 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:52:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:52:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:52:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:52:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:52:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:52:17.508 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:52:17.508 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:52:17.508 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:52:17.508 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:52:17.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.509 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:52:17.509 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:52:17.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:52:17.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:52:17.511 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:52:17.511 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:52:17.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.577 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:52:17.624 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:52:17.625 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:52:17.625 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:52:17.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.626 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:52:17.678 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:52:17.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:52:17.679 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:17.725 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:52:17.726 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:52:17.726 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:17.773 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:52:17.822 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:52:17.822 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:52:17.823 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:52:17.823 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:52:17.823 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:52:17.823 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:52:17.823 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:52:17.824 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:52:17.824 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:52:17.824 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:52:17.824 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:52:17.824 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:52:17.825 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:52:17.825 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:52:17.825 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:52:17.826 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:52:17.826 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:52:17.826 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:52:17.827 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:52:17.827 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:52:17.827 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:52:17.827 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:52:17.827 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:52:17.828 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:52:17.828 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:52:17.828 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:52:17.828 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:52:17.829 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:52:17.829 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:52:17.829 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:17.829 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:17.830 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:17.830 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:17.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.831 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:52:17.877 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:52:17.878 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:52:17.922 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:52:17.929 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:52:17.930 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:52:17.930 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:52:17.930 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:52:17.931 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:52:17.931 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:52:17.931 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:52:17.931 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:52:17.931 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:52:17.932 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:52:17.932 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:52:17.932 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:52:17.932 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:52:17.932 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:52:17.933 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:52:17.933 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:52:17.933 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:52:17.933 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:52:17.933 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:52:17.934 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:52:17.934 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:52:17.935 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:52:17.935 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:52:17.935 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:52:17.935 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:52:17.936 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:52:17.937 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:52:17.937 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:52:17.937 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:52:17.938 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:17.938 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:17.938 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:17.939 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:17.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:17.941 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:52:18.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:52:18.000+0000 7fa8b9823580 -1 init, newargv = 0x55ca4999f430 newargc=15 2023-10-26T05:52:18.005 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[45599]: starting ceph client 2023-10-26T05:52:18.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[45599]: starting fuse 2023-10-26T05:52:19.096 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:52:19.105 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:52:19.105 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:52:19.106 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:52:19.106 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:52:19.107 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:52:19.107 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:52:19.107 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:52:19.107 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:52:19.108 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:52:19.108 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:52:19.108 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:52:19.109 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:52:19.109 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:52:19.109 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:52:19.110 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:52:19.110 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:52:19.110 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:52:19.111 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:52:19.111 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:52:19.111 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:52:19.111 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:52:19.111 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:52:19.111 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:52:19.112 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:52:19.112 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:52:19.112 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:52:19.112 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:52:19.112 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:52:19.113 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:52:19.113 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:19.113 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:52:19.113 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:19.114 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:52:19.114 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:19.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:19.115 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:52:19.161 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:52:19.162 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:52:19.162 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:52:19.163 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:52:19.163 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:52:19.164 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:52:19.164 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:52:19.165 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:52:19.165 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:52:19.165 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:52:19.166 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:52:19.166 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:52:19.167 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:52:19.167 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:52:19.168 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:52:19.168 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:52:19.169 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:52:19.169 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:52:19.169 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:52:19.170 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:52:19.170 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:52:19.171 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:52:19.171 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:52:19.171 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:52:19.172 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:52:19.173 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:52:19.173 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:52:19.174 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:52:19.175 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:52:19.287 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.45599.asok 2023-10-26T05:52:19.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:19.298 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45599.asok 2023-10-26T05:52:19.299 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:19.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.45599.asok status 2023-10-26T05:52:19.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:19.497 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:52:19.497 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:52:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:52:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:52:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:52:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:52:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "45599", 2023-10-26T05:52:19.499 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:52:19.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:19.499 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:52:19.499 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:52:19.499 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7242, 2023-10-26T05:52:19.499 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:52:19.500 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:52:19.500 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:52:19.500 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 7242 2023-10-26T05:52:19.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:19.500 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:52:19.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:19.501 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:52:19.501 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3031474529 2023-10-26T05:52:19.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:19.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:19.501 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:52:19.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:19.502 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:52:19.502 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3031474529 2023-10-26T05:52:19.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:19.502 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.7242 192.168.0.1:0/3031474529", 2023-10-26T05:52:19.503 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/3031474529", 2023-10-26T05:52:19.503 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:52:19.503 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 38, 2023-10-26T05:52:19.503 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 103, 2023-10-26T05:52:19.503 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:52:19.503 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:52:19.504 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:52:19.504 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:19.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:19.518 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:19.518 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:52:19.519 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:19.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:19.562 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:52:19.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:19.569 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:19.569 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:19.570 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:19.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:19.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:19.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:52:19.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 -- 172.21.15.71:0/276960586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 msgr2=0x7f83040b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 --2- 172.21.15.71:0/276960586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f83040b1350 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f830406c3c0 tx=0x7f82f401c7e0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 -- 172.21.15.71:0/276960586 shutdown_connections 2023-10-26T05:52:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 --2- 172.21.15.71:0/276960586 >> v2:172.21.15.86:3300/0 conn(0x7f83040c9220 0x7f830406f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 --2- 172.21.15.71:0/276960586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f83040b1350 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 -- 172.21.15.71:0/276960586 >> 172.21.15.71:0/276960586 conn(0x7f830406b780 msgr2=0x7f830406bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:19.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 -- 172.21.15.71:0/276960586 shutdown_connections 2023-10-26T05:52:19.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.784+0000 7f83089a1640 1 -- 172.21.15.71:0/276960586 wait complete. 2023-10-26T05:52:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 Processor -- start 2023-10-26T05:52:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- start start 2023-10-26T05:52:19.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f8304155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f830406ab60 0x7f83041368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:19.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83040c9220 0x7f830415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304154480 con 0x7f830406a780 2023-10-26T05:52:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830415b9f0 con 0x7f83040c9220 2023-10-26T05:52:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83041546b0 con 0x7f830406ab60 2023-10-26T05:52:19.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f8304155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:19.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f8304155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60180/0 (socket says 172.21.15.71:60180) 2023-10-26T05:52:19.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 -- 172.21.15.71:0/3742008029 learned_addr learned my addr 172.21.15.71:0/3742008029 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:19.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 -- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f830406ab60 msgr2=0x7f83041368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:19.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f830406ab60 0x7f83041368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 -- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83040c9220 msgr2=0x7f830415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:19.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83040c9220 0x7f830415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 -- 172.21.15.71:0/3742008029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40094a0 con 0x7f830406a780 2023-10-26T05:52:19.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f830259c640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f8304155470 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f82f4000c10 tx=0x7f82f40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:19.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4005030 con 0x7f830406a780 2023-10-26T05:52:19.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82f4005280 con 0x7f830406a780 2023-10-26T05:52:19.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8304154990 con 0x7f830406a780 2023-10-26T05:52:19.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4005610 con 0x7f830406a780 2023-10-26T05:52:19.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8304154f50 con 0x7f830406a780 2023-10-26T05:52:19.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.788+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8304066b10 con 0x7f830406a780 2023-10-26T05:52:19.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.792+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82f40b4040 con 0x7f830406a780 2023-10-26T05:52:19.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.792+0000 7f82f2ffd640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82e40664a0 0x7f82e4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:19.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.792+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82f411dd40 con 0x7f830406a780 2023-10-26T05:52:19.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.792+0000 7f8301d9b640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82e40664a0 0x7f82e4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:19.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.792+0000 7f8301d9b640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82e40664a0 0x7f82e4068960 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f830415af30 tx=0x7f82f8006c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:19.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.796+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82f40eb130 con 0x7f830406a780 2023-10-26T05:52:19.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83040721e0 con 0x7f830406a780 2023-10-26T05:52:19.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f82f2ffd640 1 -- 172.21.15.71:0/3742008029 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f82f40f2070 con 0x7f830406a780 2023-10-26T05:52:19.995 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:52:19.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82e40664a0 msgr2=0x7f82e4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82e40664a0 0x7f82e4068960 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f830415af30 tx=0x7f82f8006c00 comp rx=0 tx=0).stop 2023-10-26T05:52:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 msgr2=0x7f8304155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f8304155470 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f82f4000c10 tx=0x7f82f40040c0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 shutdown_connections 2023-10-26T05:52:19.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83040c9220 0x7f830415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f830406ab60 0x7f83041368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82e40664a0 0x7f82e4068960 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 --2- 172.21.15.71:0/3742008029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f830406a780 0x7f8304155470 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:19.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.988+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 >> 172.21.15.71:0/3742008029 conn(0x7f830406b780 msgr2=0x7f830406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:19.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.992+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 shutdown_connections 2023-10-26T05:52:19.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:19.992+0000 7f83089a1640 1 -- 172.21.15.71:0/3742008029 wait complete. 2023-10-26T05:52:20.014 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:52:20.015 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:52:20.015 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:52:20.015 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:52:20.016 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:20.016 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:52:20.016 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:52:20.017 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:52:20.017 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:52:20.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.018 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:52:20.026 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:52:20.026 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:52:20.026 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:52:20.026 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:52:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.027 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:52:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:52:20.028 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:52:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:52:20.028 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:52:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:20.028 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:52:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:52:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42231sec preferred_lft 42231sec 2023-10-26T05:52:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:52:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:52:20.030 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:20.030 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:52:20.030 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.031 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:52:20.031 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:52:20.031 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:52:20.031 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.031 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:52:20.032 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.032 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:52:20.032 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:20.032 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:52:20.033 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.033 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:52:20.033 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:20.034 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:52:20.034 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:52:20.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.035 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:52:20.082 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:52:20.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.084 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:52:20.134 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:52:20.135 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:52:20.135 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:52:20.135 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:52:20.135 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:52:20.136 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:52:20.136 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:52:20.136 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:52:20.136 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:52:20.136 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:52:20.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.137 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:52:20.138 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:52:20.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:52:20.138 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:52:20.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:52:20.139 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:52:20.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.211 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:52:20.259 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:52:20.260 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:52:20.260 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:52:20.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:52:20.315 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:52:20.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:52:20.316 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:20.366 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:52:20.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:52:20.367 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:20.419 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:52:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:52:20.468 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:52:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:52:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:52:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:52:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:52:20.469 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:52:20.470 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:52:20.470 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:52:20.470 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:52:20.470 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:52:20.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:52:20.471 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:52:20.471 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:52:20.471 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:52:20.471 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:52:20.472 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:52:20.472 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:52:20.472 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:52:20.472 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:52:20.472 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:52:20.473 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:52:20.473 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:52:20.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:52:20.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:52:20.473 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:52:20.474 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:52:20.474 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:52:20.474 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:52:20.474 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:20.475 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:20.475 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:20.475 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:20.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.478 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:52:20.524 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:52:20.524 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:52:20.568 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:52:20.574 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:52:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:52:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:52:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:52:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:52:20.575 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:52:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:52:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:52:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:52:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:52:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:52:20.576 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:52:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:52:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:52:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:52:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:52:20.577 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:52:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:52:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:52:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:52:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:52:20.578 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:52:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:52:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:52:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:52:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:52:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:52:20.579 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:52:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:52:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:20.580 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:20.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:20.582 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:52:20.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:52:20.648+0000 7f90ac4f6580 -1 init, newargv = 0x556903730390 newargc=15 2023-10-26T05:52:20.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34740]: starting ceph client 2023-10-26T05:52:20.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34740]: starting fuse 2023-10-26T05:52:21.732 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:52:21.740 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:52:21.740 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:52:21.740 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:52:21.740 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:52:21.740 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:52:21.741 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:52:21.741 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:52:21.741 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:52:21.742 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:52:21.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:52:21.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:52:21.743 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:52:21.743 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:52:21.743 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:52:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:52:21.744 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:52:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:52:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:52:21.745 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:52:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:52:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:52:21.746 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:52:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:52:21.747 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:52:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:52:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:52:21.748 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:52:21.749 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:52:21.749 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:52:21.749 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:21.750 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:52:21.750 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:21.750 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:52:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:21.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:21.753 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:52:21.799 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:52:21.800 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:52:21.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:52:21.801 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:52:21.801 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:52:21.801 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:52:21.802 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:52:21.802 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:52:21.803 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:52:21.803 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:52:21.803 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:52:21.804 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:52:21.804 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:52:21.805 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:52:21.805 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:52:21.805 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:52:21.806 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:52:21.806 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:52:21.807 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:52:21.807 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:52:21.807 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:52:21.808 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:52:21.808 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:52:21.809 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:52:21.809 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:52:21.809 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:52:21.810 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:52:21.811 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:52:21.811 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:52:21.922 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.34740.asok 2023-10-26T05:52:21.926 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:21.933 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34740.asok 2023-10-26T05:52:21.934 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:21.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.34740.asok status 2023-10-26T05:52:22.022 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:22.123 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:52:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:52:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:52:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:52:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:52:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:52:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:52:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34740", 2023-10-26T05:52:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:52:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:52:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:52:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:52:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5465, 2023-10-26T05:52:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:52:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:52:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:52:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 5465 2023-10-26T05:52:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:52:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:52:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:52:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:52:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2798878792 2023-10-26T05:52:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:52:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:52:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:52:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:52:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:52:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2798878792 2023-10-26T05:52:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:52:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.5465 192.168.0.1:0/2798878792", 2023-10-26T05:52:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2798878792", 2023-10-26T05:52:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:52:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 38, 2023-10-26T05:52:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 103, 2023-10-26T05:52:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:52:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:52:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:52:22.131 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:52:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:22.143 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:22.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:52:22.144 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:22.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:22.189 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:52:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:22.196 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:22.197 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:22.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:22.289 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:22.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:22.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3931026673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a40ac740 msgr2=0x7ff2a415b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:22.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3931026673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a40ac740 0x7ff2a415b4b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff294009900 tx=0x7ff29401c8c0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3931026673 shutdown_connections 2023-10-26T05:52:22.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3931026673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2a415b9f0 0x7ff2a4070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3931026673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a40ac740 0x7ff2a415b4b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3931026673 >> 172.21.15.71:0/3931026673 conn(0x7ff2a4069770 msgr2=0x7ff2a4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:22.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3931026673 shutdown_connections 2023-10-26T05:52:22.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3931026673 wait complete. 2023-10-26T05:52:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 Processor -- start 2023-10-26T05:52:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- start start 2023-10-26T05:52:22.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 0x7ff2a40a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2a413d420 0x7ff2a4161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:22.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a415b9f0 0x7ff2a413b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:22.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40a8c20 con 0x7ff2a40ac740 2023-10-26T05:52:22.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a413c880 con 0x7ff2a415b9f0 2023-10-26T05:52:22.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4162510 con 0x7ff2a413d420 2023-10-26T05:52:22.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 0x7ff2a40a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:22.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 0x7ff2a40a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43696/0 (socket says 172.21.15.71:43696) 2023-10-26T05:52:22.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 -- 172.21.15.71:0/3693133651 learned_addr learned my addr 172.21.15.71:0/3693133651 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:22.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2a9977640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2a413d420 0x7ff2a4161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:22.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa979640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a415b9f0 0x7ff2a413b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:22.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 -- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2a413d420 msgr2=0x7ff2a4161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:22.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2a413d420 0x7ff2a4161fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 -- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a415b9f0 msgr2=0x7ff2a413b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:22.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a415b9f0 0x7ff2a413b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 -- 172.21.15.71:0/3693133651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff294009560 con 0x7ff2a40ac740 2023-10-26T05:52:22.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2aa178640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 0x7ff2a40a86e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c560 tx=0x7ff2a000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:22.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005220 con 0x7ff2a40ac740 2023-10-26T05:52:22.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a40a8e30 con 0x7ff2a40ac740 2023-10-26T05:52:22.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40a93f0 con 0x7ff2a40ac740 2023-10-26T05:52:22.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff2a0005380 con 0x7ff2a40ac740 2023-10-26T05:52:22.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.428+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0007270 con 0x7ff2a40ac740 2023-10-26T05:52:22.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.432+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28c005260 con 0x7ff2a40ac740 2023-10-26T05:52:22.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.432+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff2a00073d0 con 0x7ff2a40ac740 2023-10-26T05:52:22.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.432+0000 7ff29affd640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff288066470 0x7ff288068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:22.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.432+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2a00839b0 con 0x7ff2a40ac740 2023-10-26T05:52:22.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.432+0000 7ff2a9977640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff288066470 0x7ff288068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:22.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.432+0000 7ff2a9977640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff288066470 0x7ff288068930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff2940af000 tx=0x7ff29401b100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:22.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.436+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff2a0050d80 con 0x7ff2a40ac740 2023-10-26T05:52:22.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.620+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff28c0050d0 con 0x7ff2a40ac740 2023-10-26T05:52:22.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.620+0000 7ff29affd640 1 -- 172.21.15.71:0/3693133651 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7ff2a00543a0 con 0x7ff2a40ac740 2023-10-26T05:52:22.627 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:22.627 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:22.628 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:22.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.620+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff288066470 msgr2=0x7ff288068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:22.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.620+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff288066470 0x7ff288068930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff2940af000 tx=0x7ff29401b100 comp rx=0 tx=0).stop 2023-10-26T05:52:22.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.620+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 msgr2=0x7ff2a40a86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:22.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.620+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 0x7ff2a40a86e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000c560 tx=0x7ff2a000ca20 comp rx=0 tx=0).stop 2023-10-26T05:52:22.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 shutdown_connections 2023-10-26T05:52:22.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2a415b9f0 0x7ff2a413b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff2a413d420 0x7ff2a4161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff288066470 0x7ff288068930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 --2- 172.21.15.71:0/3693133651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2a40ac740 0x7ff2a40a86e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:22.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 >> 172.21.15.71:0/3693133651 conn(0x7ff2a4069770 msgr2=0x7ff2a40abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:22.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 shutdown_connections 2023-10-26T05:52:22.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:22.624+0000 7ff2ac3dc640 1 -- 172.21.15.71:0/3693133651 wait complete. 2023-10-26T05:52:22.644 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:52:22.777 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":7074,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.413126902,"mdsmap_epoch":38,"osdmap_epoch":103,"osdmap_epoch_barrier":103,"uptime":16.046608373000002} 2023-10-26T05:52:22.778 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 7074, "mdsmap_epoch": 38, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "rank_uptime": 10.413126902, "state": "up:active", "uptime": 16.046608373, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:52:22.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:52:22.906 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:52:22.907 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:52:22.907 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-10-26T05:52:22.908 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:22.908 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:22.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:22.915 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-10-26T05:52:25.815 INFO:teuthology.orchestra.run.smithi071.stderr:265+0 records in 2023-10-26T05:52:25.815 INFO:teuthology.orchestra.run.smithi071.stderr:265+0 records out 2023-10-26T05:52:25.815 INFO:teuthology.orchestra.run.smithi071.stderr:277872640 bytes (278 MB, 265 MiB) copied, 2.84635 s, 97.6 MB/s 2023-10-26T05:52:25.816 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:25.817 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:25.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:25.823 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-10-26T05:52:27.956 INFO:teuthology.orchestra.run.smithi071.stderr:292+0 records in 2023-10-26T05:52:27.956 INFO:teuthology.orchestra.run.smithi071.stderr:292+0 records out 2023-10-26T05:52:27.956 INFO:teuthology.orchestra.run.smithi071.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.08057 s, 147 MB/s 2023-10-26T05:52:27.957 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T05:52:27.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:28.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- 172.21.15.71:0/1923903563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0470 msgr2=0x7ff15806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/1923903563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0470 0x7ff15806aa70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7ff15808a3b0 tx=0x7ff14001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- 172.21.15.71:0/1923903563 shutdown_connections 2023-10-26T05:52:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/1923903563 >> v2:172.21.15.86:3300/0 conn(0x7ff1580cd880 0x7ff15815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/1923903563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0470 0x7ff15806aa70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- 172.21.15.71:0/1923903563 >> 172.21.15.71:0/1923903563 conn(0x7ff1580cdd80 msgr2=0x7ff1580ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- 172.21.15.71:0/1923903563 shutdown_connections 2023-10-26T05:52:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- 172.21.15.71:0/1923903563 wait complete. 2023-10-26T05:52:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 Processor -- start 2023-10-26T05:52:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- start start 2023-10-26T05:52:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580cd880 0x7ff15814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1580d0470 0x7ff158154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 0x7ff158151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15815d010 con 0x7ff1580d0850 2023-10-26T05:52:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff158155310 con 0x7ff1580cd880 2023-10-26T05:52:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff15ddd1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15814f100 con 0x7ff1580d0470 2023-10-26T05:52:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 0x7ff158151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 0x7ff158151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43712/0 (socket says 172.21.15.71:43712) 2023-10-26T05:52:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 -- 172.21.15.71:0/2703053941 learned_addr learned my addr 172.21.15.71:0/2703053941 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff1577fe640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580cd880 0x7ff15814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:28.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff156ffd640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1580d0470 0x7ff158154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:28.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 -- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1580d0470 msgr2=0x7ff158154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:28.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1580d0470 0x7ff158154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 -- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580cd880 msgr2=0x7ff15814ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:28.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580cd880 0x7ff15814ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 -- 172.21.15.71:0/2703053941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140009560 con 0x7ff1580d0850 2023-10-26T05:52:28.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff157fff640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 0x7ff158151330 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b970 tx=0x7ff15000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:28.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.080+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000cb40 con 0x7ff1580d0850 2023-10-26T05:52:28.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15814f3e0 con 0x7ff1580d0850 2023-10-26T05:52:28.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff15000cdb0 con 0x7ff1580d0850 2023-10-26T05:52:28.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15815cca0 con 0x7ff1580d0850 2023-10-26T05:52:28.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1500146a0 con 0x7ff1580d0850 2023-10-26T05:52:28.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff158066b10 con 0x7ff1580d0850 2023-10-26T05:52:28.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff15001d3b0 con 0x7ff1580d0850 2023-10-26T05:52:28.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff14ffff640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:28.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff150084010 con 0x7ff1580d0850 2023-10-26T05:52:28.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff1577fe640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:28.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.084+0000 7ff1577fe640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1400af000 tx=0x7ff14001afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:28.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.088+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1500514c0 con 0x7ff1580d0850 2023-10-26T05:52:28.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.248+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff15815c820 con 0x7ff1580d0850 2023-10-26T05:52:28.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.248+0000 7ff14ffff640 1 -- 172.21.15.71:0/2703053941 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) v1 ==== 81+0+33922 (secure 0 0 0) 0x7ff15815c820 con 0x7ff1580d0850 2023-10-26T05:52:28.257 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:28.257 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:28.258 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 103, 2023-10-26T05:52:28.258 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:28.258 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:28.258 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:13.139734+0000", 2023-10-26T05:52:28.258 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:28.259 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:28.259 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:28.259 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:28.259 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:28.259 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:28.261 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:28.261 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:28.261 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:28.261 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:28.262 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:28.262 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:28.262 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:28.262 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:28.262 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:28.263 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:28.263 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:28.263 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:28.263 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:28.264 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:28.264 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:28.264 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:28.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.264 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:28.265 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:28.265 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:28.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:28.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:28.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:28.267 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:28.267 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:28.267 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:28.267 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:28.268 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:28.268 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:28.268 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:28.268 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:28.269 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:28.269 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:28.269 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:28.269 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:28.269 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:28.270 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:28.270 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:28.270 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:28.270 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:28.270 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:28.271 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:28.271 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:28.271 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:28.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.271 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "91", 2023-10-26T05:52:28.272 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:28.272 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:28.272 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:28.272 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:28.272 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:28.272 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:28.273 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:28.273 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:28.273 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:28.273 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:28.273 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:28.274 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:28.274 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:28.274 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:28.274 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:28.275 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:28.275 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:28.275 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:28.275 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:28.275 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:28.275 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:28.276 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:28.276 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:28.276 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:28.276 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:28.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:28.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.277 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:28.277 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:28.277 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:28.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:28.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:28.278 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:28.278 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:28.278 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:28.278 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:28.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.280 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:28.280 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:28.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.280 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:28.280 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:28.281 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:28.281 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:28.281 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:28.281 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:28.281 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:28.282 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:28.282 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:28.282 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.283 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:28.283 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:28.283 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:28.283 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:28.283 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:28.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:28.284 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:28.284 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:28.284 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:28.284 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:28.284 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:28.285 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:28.285 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:28.285 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:28.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:28.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:28.285 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:28.286 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:28.286 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:28.286 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:28.286 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:28.286 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:28.287 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:28.287 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:28.287 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:28.287 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:28.287 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:28.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "102", 2023-10-26T05:52:28.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:28.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:28.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:28.289 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:28.289 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:28.289 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:28.289 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:28.289 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:28.290 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:28.290 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:28.290 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:28.290 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:28.290 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:28.290 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:28.291 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:28.291 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:28.291 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:28.291 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:28.291 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:28.292 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:28.292 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:28.292 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:28.292 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:28.292 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:28.292 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:28.293 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:28.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.293 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:28.293 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:28.293 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:28.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:28.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:28.294 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:28.294 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:28.294 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:28.294 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:28.295 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:28.295 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:28.295 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:28.295 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:28.295 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:28.296 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:28.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.296 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:28.296 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:28.296 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:28.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.297 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:28.297 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:28.297 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:28.297 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:28.298 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:28.298 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:28.298 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:28.298 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:28.298 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:28.299 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.299 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:28.299 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:28.299 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:28.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:28.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:28.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:28.300 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:28.300 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:28.302 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:28.302 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:28.302 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:28.302 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:28.302 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:28.303 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:28.303 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:28.303 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:28.303 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:28.303 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:28.304 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:28.304 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:28.304 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:28.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.304 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "103", 2023-10-26T05:52:28.304 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:28.305 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:28.305 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:28.305 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:28.305 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:28.305 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:28.306 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:28.306 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:28.306 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:28.306 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:28.306 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:28.307 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:28.307 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:28.307 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:28.307 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:28.307 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:28.307 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:28.308 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:28.308 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:28.308 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:28.308 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:28.308 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:28.309 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:28.309 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:28.309 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:28.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:28.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.310 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:28.310 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:28.310 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:28.310 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:28.310 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:28.311 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:28.311 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:28.311 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:28.311 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:28.311 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:28.311 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:28.312 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:28.312 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:28.312 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:28.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.312 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:28.313 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:28.313 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:28.313 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.313 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:28.313 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:28.314 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:28.314 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:28.314 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:28.314 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:28.314 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:28.315 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:28.315 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:28.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.315 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:28.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:28.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:28.316 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.316 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.316 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.317 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.317 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.317 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.317 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.317 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.318 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.318 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.318 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.318 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.318 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.319 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:28.319 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.319 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.320 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:28.320 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.320 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.320 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.321 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.321 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.321 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:28.322 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.322 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:28.323 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.323 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.323 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.324 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:28.324 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:28.325 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.325 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.326 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.326 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.327 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:28.327 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.328 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:28.328 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:28.328 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.328 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.329 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.330 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.330 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.330 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:28.331 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.331 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.331 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.331 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.331 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.332 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.332 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.332 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.332 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.332 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.333 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.333 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.333 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.334 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:28.334 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.335 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:28.336 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.337 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.341 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.342 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:28.343 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.344 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:28.345 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.346 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.347 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:28.348 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.353 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:28.354 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.355 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:28.358 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.360 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.361 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.370 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.371 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:28.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.373 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.374 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.375 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.376 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.389 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.390 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.392 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.393 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.399 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:28.400 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:28.401 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.402 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.403 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.404 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:28.409 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.410 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:28.416 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.417 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:28.418 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:28.419 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.420 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.421 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:28.422 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.424 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:28.426 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:28.427 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.427 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:28.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:28.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:28.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:28.428 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:28.429 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:28.429 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:28.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.430 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.430 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:28.430 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:28.430 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:28.431 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:28.431 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:28.431 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:28.431 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:28.431 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:28.432 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.433 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:28.434 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.435 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:28.436 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:28.437 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.438 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:28.439 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.440 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.442 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.444 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:28.445 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:28.446 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:28.446 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:28.446 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:28.446 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:28.446 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:28.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.447 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:28.447 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:28.447 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:28.447 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:28.447 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:28.448 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:28.448 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:28.448 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:28.448 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:28.448 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:28.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:28.450 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:28.451 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:28.451 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:28.451 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:28.451 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:28.451 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:28.452 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:28.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:28.452 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:28.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 msgr2=0x7ff138068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:28.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff1400af000 tx=0x7ff14001afe0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 msgr2=0x7ff158151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:28.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 0x7ff158151330 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b970 tx=0x7ff15000be30 comp rx=0 tx=0).stop 2023-10-26T05:52:28.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 shutdown_connections 2023-10-26T05:52:28.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1580d0850 0x7ff158151330 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1580d0470 0x7ff158154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 --2- 172.21.15.71:0/2703053941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1580cd880 0x7ff15814ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:28.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 >> 172.21.15.71:0/2703053941 conn(0x7ff1580cdd80 msgr2=0x7ff1580b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:28.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 shutdown_connections 2023-10-26T05:52:28.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:28.252+0000 7ff15ddd1640 1 -- 172.21.15.71:0/2703053941 wait complete. 2023-10-26T05:52:28.456 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:52:28.457 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:52:28.420+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2023-10-26T05:52:33.457 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:33.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/1883609447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 msgr2=0x7f6f9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:33.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/1883609447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6f9c003dd0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40016b0 tx=0x7f6fa401c7a0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb359e640 1 -- 172.21.15.71:0/1883609447 reap_dead start 2023-10-26T05:52:33.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/1883609447 shutdown_connections 2023-10-26T05:52:33.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/1883609447 >> 172.21.15.71:0/1883609447 conn(0x7f6fb4069770 msgr2=0x7f6fb4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:33.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/1883609447 shutdown_connections 2023-10-26T05:52:33.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/1883609447 wait complete. 2023-10-26T05:52:33.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 Processor -- start 2023-10-26T05:52:33.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- start start 2023-10-26T05:52:33.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6fb4153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:33.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb4152fb0 0x7f6fb4136990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:33.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fb40adba0 0x7f6fb40ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:33.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40ae560 con 0x7f6f9c0039b0 2023-10-26T05:52:33.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb413d170 con 0x7f6fb4152fb0 2023-10-26T05:52:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4156470 con 0x7f6fb40adba0 2023-10-26T05:52:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6fb4153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6fb4153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46970/0 (socket says 172.21.15.71:46970) 2023-10-26T05:52:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 -- 172.21.15.71:0/2865683886 learned_addr learned my addr 172.21.15.71:0/2865683886 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb259c640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb4152fb0 0x7f6fb4136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb359e640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fb40adba0 0x7f6fb40ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:33.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 -- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fb40adba0 msgr2=0x7f6fb40ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:33.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fb40adba0 0x7f6fb40ae020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 -- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb4152fb0 msgr2=0x7f6fb4136990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:33.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb4152fb0 0x7f6fb4136990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 -- 172.21.15.71:0/2865683886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4137430 con 0x7f6f9c0039b0 2023-10-26T05:52:33.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb2d9d640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6fb4153400 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40016b0 tx=0x7f6fa4006a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:33.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4003a40 con 0x7f6f9c0039b0 2023-10-26T05:52:33.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6fa4003ba0 con 0x7f6f9c0039b0 2023-10-26T05:52:33.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b5430 con 0x7f6f9c0039b0 2023-10-26T05:52:33.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4137710 con 0x7f6f9c0039b0 2023-10-26T05:52:33.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.588+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb4136ed0 con 0x7f6f9c0039b0 2023-10-26T05:52:33.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4066b10 con 0x7f6f9c0039b0 2023-10-26T05:52:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6fa40a8410 con 0x7f6f9c0039b0 2023-10-26T05:52:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6f9b7fe640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa00664a0 0x7f6fa0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fa411e170 con 0x7f6f9c0039b0 2023-10-26T05:52:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=107}) v3 -- 0x7f6fa006dfd0 con 0x7f6f9c0039b0 2023-10-26T05:52:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6fb259c640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa00664a0 0x7f6fa0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.592+0000 7f6fb259c640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa00664a0 0x7f6fa0068960 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6fb413f7b0 tx=0x7f6fa8001630 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.596+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6fa40ae060 con 0x7f6f9c0039b0 2023-10-26T05:52:33.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.744+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6fb40721e0 con 0x7f6f9c0039b0 2023-10-26T05:52:33.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6f9b7fe640 1 -- 172.21.15.71:0/2865683886 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+34090 (secure 0 0 0) 0x7f6fa40eb5f0 con 0x7f6f9c0039b0 2023-10-26T05:52:33.755 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:33.756 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:33.756 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 106, 2023-10-26T05:52:33.756 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:33.756 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:33.756 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:31.489420+0000", 2023-10-26T05:52:33.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:33.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:33.757 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:33.758 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:33.758 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:33.758 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:33.758 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:33.758 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:33.759 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:33.759 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:33.759 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:33.759 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:33.759 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:33.759 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:33.760 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:33.760 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:33.760 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:33.760 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:33.760 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:33.761 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:33.761 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:33.761 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:33.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.761 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:33.762 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:33.762 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:33.762 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:33.762 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:33.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:33.763 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:33.763 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:33.763 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:33.763 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:33.763 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:33.764 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:33.764 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:33.764 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:33.764 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:33.764 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:33.764 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:33.765 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:33.765 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:33.765 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:33.765 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:33.765 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:33.766 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:33.766 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:33.766 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:33.766 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:33.766 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:33.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:33.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:33.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:33.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:33.768 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:33.768 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:33.768 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:33.768 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:33.768 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:33.769 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:33.770 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:33.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:33.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:33.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:33.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:33.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:33.772 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:33.772 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:33.772 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:33.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.772 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:33.773 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:33.773 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:33.773 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:33.773 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:33.773 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:33.774 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:33.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:33.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:33.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.775 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:33.775 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:33.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.776 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:33.776 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:33.776 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:33.776 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:33.777 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:33.777 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:33.777 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:33.777 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:33.777 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:33.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.778 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:33.778 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:33.778 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:33.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:33.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:33.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:33.779 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:33.779 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:33.781 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:33.781 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:33.781 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:33.781 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:33.782 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:33.782 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:33.782 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:33.782 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:33.783 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:33.783 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:33.783 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:33.783 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:33.783 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:33.783 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:33.784 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:33.784 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:33.784 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:33.784 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:33.785 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:33.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.785 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:33.785 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:33.785 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:33.786 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:33.786 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:33.786 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:33.786 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:33.786 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:33.787 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:33.787 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:33.787 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:33.787 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:33.787 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:33.787 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:33.788 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:33.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:33.790 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:33.791 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:33.791 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:33.791 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:33.791 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:33.791 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:33.792 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:33.792 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:33.792 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:33.792 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:33.792 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:33.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:33.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:33.793 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:33.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.793 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:33.794 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:33.794 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:33.794 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.794 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:33.795 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:33.795 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:33.795 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:33.795 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:33.795 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:33.795 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:33.796 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:33.796 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:33.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.797 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:33.797 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:33.797 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:33.797 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:33.797 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:33.798 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:33.798 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:33.798 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:33.798 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:33.798 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:33.799 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:33.799 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:33.799 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:33.799 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:33.799 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:33.800 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:33.800 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:33.800 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:33.800 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:33.801 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:33.801 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:33.801 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:33.801 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:33.802 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:33.802 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:33.802 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:33.802 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:33.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.803 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:33.803 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:33.803 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:33.803 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:33.803 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:33.804 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:33.804 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:33.804 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:33.804 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:33.805 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:33.805 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:33.805 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:33.805 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:33.805 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:33.805 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:33.806 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:33.806 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:33.806 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:33.806 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:33.806 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:33.807 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:33.807 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:33.807 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:33.807 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:33.807 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:33.808 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:33.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:33.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.808 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:33.808 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:33.809 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:33.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:33.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:33.809 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:33.809 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:33.810 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:33.810 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:33.810 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:33.810 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:33.810 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:33.811 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:33.811 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:33.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.811 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:33.811 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:33.812 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:33.812 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.812 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:33.812 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:33.812 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:33.813 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:33.813 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:33.813 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:33.813 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:33.813 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:33.814 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:33.814 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.814 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.814 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:33.814 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:33.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.815 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:33.815 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.815 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.815 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.816 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.816 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.816 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.816 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.816 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.816 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.817 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.817 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.817 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.817 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.817 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.818 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:33.818 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.819 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.819 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:33.819 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.820 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.820 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.820 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.821 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:33.821 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.821 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.822 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:33.822 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.822 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:33.823 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.824 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.824 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:33.824 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.825 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.825 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.825 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.825 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.826 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:33.826 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.827 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:33.827 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:33.827 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.827 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.828 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:33.828 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:33.828 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:33.828 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:33.829 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.829 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.829 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:33.829 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.830 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:33.830 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.830 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.830 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.831 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.831 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.831 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.831 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.831 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.832 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.832 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.832 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.832 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.832 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.833 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:33.833 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.834 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:33.834 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.835 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.835 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.835 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.836 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:33.836 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.837 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:33.837 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.837 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.838 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.838 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.838 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:33.839 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.839 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:33.839 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.840 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.840 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.840 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.841 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:33.841 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.842 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:33.842 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:33.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.843 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.843 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:33.843 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:33.844 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:33.844 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:33.844 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.844 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.844 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:33.845 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.845 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:33.846 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.846 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.846 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.846 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.846 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.846 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.847 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.847 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.847 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.847 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.847 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.848 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.848 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.848 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:33.849 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.850 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.850 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.850 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.851 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.851 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:33.851 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.852 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:33.852 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.852 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.853 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.853 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.853 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.854 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:33.854 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.855 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.855 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:33.855 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.855 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.855 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.855 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.856 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.856 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.856 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.856 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:33.857 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.857 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:33.858 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:33.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.858 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:33.858 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:33.859 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:33.859 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:33.859 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.859 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.859 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:33.860 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.860 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.860 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:33.861 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.861 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.861 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.861 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.861 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.861 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.862 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.862 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.862 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.862 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.862 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.863 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:33.864 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.864 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:33.865 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.865 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.865 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.865 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.866 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:33.866 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.867 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:33.867 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.867 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.868 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.868 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.868 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.869 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:33.869 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.869 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.870 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:33.870 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.870 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.870 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.871 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:33.871 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.872 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:33.872 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:33.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.873 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.873 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:33.873 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:33.874 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:33.874 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:33.874 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.874 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.874 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.875 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:33.875 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.876 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.876 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.876 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.876 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.876 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.876 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.877 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.877 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.877 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.877 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.877 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.878 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.878 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:33.878 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.879 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.879 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.879 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:33.879 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.880 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.880 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.880 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.880 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.881 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.881 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:33.881 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.882 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:33.882 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.882 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.885 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.886 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.886 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.886 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.887 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:33.887 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.888 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:33.888 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:33.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.888 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.889 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:33.889 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:33.889 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:33.889 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:33.889 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.890 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.890 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:33.890 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.890 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.891 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:33.891 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.891 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.891 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.891 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.892 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.892 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.892 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.892 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.892 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.893 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.893 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.893 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.893 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.896 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.896 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.896 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:33.897 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.897 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:33.898 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.898 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.899 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.899 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.899 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:33.899 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.900 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:33.900 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.900 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.901 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.901 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:33.902 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.902 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:33.903 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:33.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.903 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.904 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:33.904 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:33.904 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:33.904 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:33.904 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.905 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.905 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.905 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.906 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:33.906 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.906 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.906 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.906 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.907 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.907 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.907 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.907 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.907 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.907 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.908 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.908 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.908 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.909 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:33.909 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.910 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:33.910 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.910 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.911 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.911 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.911 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.911 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:33.911 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.912 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:33.912 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.913 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.913 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.913 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.913 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.914 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:33.914 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:33.915 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.915 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.915 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.916 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.916 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:33.917 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.918 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:33.918 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:33.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.918 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.919 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:33.919 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:33.919 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:33.919 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:33.919 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.920 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.920 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:33.920 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.920 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.921 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:33.921 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:33.921 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:33.921 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:33.922 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:33.922 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:33.922 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:33.922 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:33.922 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:33.923 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:33.923 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:33.923 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:33.923 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:33.923 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.924 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.924 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:33.924 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.925 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.925 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:33.925 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.925 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.926 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.926 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:33.926 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.926 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.927 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:33.927 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:33.928 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.928 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.929 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:33.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:33.930 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.931 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:33.931 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.931 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.931 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.931 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:33.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:33.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:33.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:33.932 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:33.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:33.933 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:33.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.934 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:33.934 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:33.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:33.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:33.935 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:33.935 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:52:33.935 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:33.935 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:33.936 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:33.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.936 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:33.936 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:33.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.937 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:33.937 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.937 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.937 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.937 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.938 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.938 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:33.938 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.939 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:33.939 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.939 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.939 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.939 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.940 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.940 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:33.940 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.940 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:33.941 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.941 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.941 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.941 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.941 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.942 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:33.942 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.942 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:33.943 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.943 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.943 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.943 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.943 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.944 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:33.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.944 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:33.945 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.945 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.945 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.945 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.945 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.945 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:33.946 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.946 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:33.946 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.947 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.947 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.947 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.947 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.947 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:33.948 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:33.948 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.949 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.949 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.949 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:33.949 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:33.950 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:33.950 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:33.950 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:33.951 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:33.951 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:33.951 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:33.951 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:33.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:33.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.952 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:33.952 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:33.952 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:33.952 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:33.953 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:33.953 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:33.953 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:33.953 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:33.953 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:33.953 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:33.954 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:33.954 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:33.954 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:33.954 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:33.954 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:33.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:33.955 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:33.955 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:33.955 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:33.956 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:33.956 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:33.956 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:33.956 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:33.956 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:33.957 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:33.957 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:33.957 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:33.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:33.957 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:33.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa00664a0 msgr2=0x7f6fa0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:33.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa00664a0 0x7f6fa0068960 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6fb413f7b0 tx=0x7f6fa8001630 comp rx=0 tx=0).stop 2023-10-26T05:52:33.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 msgr2=0x7f6fb4153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:33.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6fb4153400 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40016b0 tx=0x7f6fa4006a00 comp rx=0 tx=0).stop 2023-10-26T05:52:33.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 shutdown_connections 2023-10-26T05:52:33.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6fb40adba0 0x7f6fb40ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fb4152fb0 0x7f6fb4136990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6fa00664a0 0x7f6fa0068960 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 --2- 172.21.15.71:0/2865683886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f9c0039b0 0x7f6fb4153400 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:33.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 >> 172.21.15.71:0/2865683886 conn(0x7f6fb4069770 msgr2=0x7f6fb40caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:33.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.748+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 shutdown_connections 2023-10-26T05:52:33.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:33.752+0000 7f6fb91c7640 1 -- 172.21.15.71:0/2865683886 wait complete. 2023-10-26T05:52:33.962 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T05:52:33.962 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:33.963 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:33.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:33.969 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-10-26T05:52:34.100 INFO:teuthology.orchestra.run.smithi071.stderr:dd: error writing 'large_file_b': No space left on device 2023-10-26T05:52:34.101 INFO:teuthology.orchestra.run.smithi071.stderr:dd: closing output file 'large_file_b': No space left on device 2023-10-26T05:52:34.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:52:34.102 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1005064217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f50003980 msgr2=0x7f7f50003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1005064217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f50003980 0x7f7f50003da0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000bbb0 tx=0x7f7f6001cd70 comp rx=0 tx=0).stop 2023-10-26T05:52:34.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1005064217 shutdown_connections 2023-10-26T05:52:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1005064217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f50003980 0x7f7f50003da0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1005064217 >> 172.21.15.71:0/1005064217 conn(0x7f7f640cdd80 msgr2=0x7f7f640ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1005064217 shutdown_connections 2023-10-26T05:52:34.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1005064217 wait complete. 2023-10-26T05:52:34.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 Processor -- start 2023-10-26T05:52:34.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- start start 2023-10-26T05:52:34.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f50003980 0x7f7f640c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64154370 0x7f7f640c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 0x7f7f6415f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6406a650 con 0x7f7f641f8a00 2023-10-26T05:52:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640cab60 con 0x7f7f64154370 2023-10-26T05:52:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f641547f0 con 0x7f7f50003980 2023-10-26T05:52:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 0x7f7f6415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 0x7f7f6415f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46986/0 (socket says 172.21.15.71:46986) 2023-10-26T05:52:34.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 -- 172.21.15.71:0/1530159190 learned_addr learned my addr 172.21.15.71:0/1530159190 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:34.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69499640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f50003980 0x7f7f640c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f68c98640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64154370 0x7f7f640c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 -- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f50003980 msgr2=0x7f7f640c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f50003980 0x7f7f640c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 -- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64154370 msgr2=0x7f7f640c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64154370 0x7f7f640c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 -- 172.21.15.71:0/1530159190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f641f8290 con 0x7f7f641f8a00 2023-10-26T05:52:34.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f68c98640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64154370 0x7f7f640c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:34.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f69c9a640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 0x7f7f6415f280 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00d7c0 tx=0x7f7f5c00dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:34.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c003d90 con 0x7f7f641f8a00 2023-10-26T05:52:34.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f5c00de70 con 0x7f7f641f8a00 2023-10-26T05:52:34.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f640c7620 con 0x7f7f641f8a00 2023-10-26T05:52:34.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0052b0 con 0x7f7f641f8a00 2023-10-26T05:52:34.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.224+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f640c7be0 con 0x7f7f641f8a00 2023-10-26T05:52:34.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f640c7da0 con 0x7f7f641f8a00 2023-10-26T05:52:34.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7f5c005410 con 0x7f7f641f8a00 2023-10-26T05:52:34.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f55ffb640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f48066470 0x7f7f48068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f5c083120 con 0x7f7f641f8a00 2023-10-26T05:52:34.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=107}) v3 -- 0x7f7f4806dfa0 con 0x7f7f641f8a00 2023-10-26T05:52:34.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f69499640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f48066470 0x7f7f48068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.240 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.228+0000 7f7f69499640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f48066470 0x7f7f48068930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7f600b0000 tx=0x7f7f600a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:34.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.232+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7f5c050500 con 0x7f7f641f8a00 2023-10-26T05:52:34.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.376+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7f64066b10 con 0x7f7f641f8a00 2023-10-26T05:52:34.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f55ffb640 1 -- 172.21.15.71:0/1530159190 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+34090 (secure 0 0 0) 0x7f7f5c053b20 con 0x7f7f641f8a00 2023-10-26T05:52:34.386 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:34.386 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:34.386 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 106, 2023-10-26T05:52:34.387 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:34.387 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:34.387 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:31.489420+0000", 2023-10-26T05:52:34.387 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:34.387 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:34.387 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:34.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:34.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:34.388 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:34.388 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:34.388 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:34.389 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:34.389 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:34.389 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:34.389 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:34.389 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:34.390 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:34.390 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:34.390 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:34.390 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:34.390 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:34.391 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:34.391 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:34.391 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:34.391 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:34.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:34.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:34.392 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:34.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:34.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:34.393 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:34.393 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:34.393 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:34.393 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:34.393 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:34.393 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:34.394 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:34.394 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:34.394 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:34.394 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:34.394 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:34.397 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:34.397 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:34.397 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:34.397 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:34.398 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:34.398 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:34.398 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:34.398 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:34.398 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:34.399 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:34.399 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:34.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.399 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:34.399 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:34.399 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:34.400 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:34.400 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:34.400 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:34.400 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:34.400 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:34.401 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:34.401 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:34.401 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:34.401 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:34.401 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:34.402 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:34.402 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:34.402 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:34.402 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:34.402 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:34.402 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:34.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:34.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:34.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:34.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:34.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:34.404 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:34.404 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:34.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:34.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.404 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:34.405 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:34.405 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:34.405 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:34.405 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:34.405 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:34.405 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:34.406 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:34.406 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:34.406 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:34.406 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:34.406 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:34.407 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:34.407 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:34.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.407 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:34.407 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:34.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.408 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:34.408 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:34.408 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:34.408 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:34.408 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:34.409 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:34.409 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:34.409 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:34.409 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:34.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.410 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:34.410 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:34.410 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:34.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:34.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:34.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:34.411 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:34.411 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:34.412 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:34.413 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:34.413 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:34.413 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:34.413 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:34.414 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:34.414 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:34.414 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:34.414 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:34.414 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:34.415 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:34.415 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:34.415 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:34.415 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:34.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:34.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:34.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:34.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:34.417 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:34.417 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:34.417 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:34.417 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:34.417 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:34.417 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:34.418 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:34.418 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:34.418 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:34.418 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:34.418 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:34.419 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:34.419 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:34.419 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:34.419 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:34.419 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:34.419 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:34.420 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:34.420 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:34.420 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:34.420 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:34.420 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:34.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:34.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.421 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:34.421 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:34.421 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:34.422 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:34.422 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:34.422 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:34.422 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:34.422 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:34.422 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:34.423 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:34.423 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:34.423 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:34.423 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:34.423 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:34.424 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:34.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.424 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:34.424 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:34.424 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:34.425 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.425 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:34.425 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:34.425 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:34.425 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:34.426 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:34.426 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:34.426 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:34.426 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:34.426 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:34.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.427 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:34.427 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:34.428 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:34.429 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:34.429 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:34.429 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:34.429 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:34.429 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:34.430 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:34.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:34.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:34.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:34.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:34.431 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:34.431 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:34.431 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:34.431 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:34.431 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:34.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:34.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:34.432 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:34.432 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:34.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.432 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:34.433 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:34.433 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:34.433 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:34.433 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:34.433 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:34.434 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:34.434 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:34.434 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:34.434 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:34.434 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:34.435 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:34.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:34.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:34.435 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:34.435 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:34.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:34.436 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:34.436 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:34.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:34.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:34.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:34.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:34.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:34.437 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:34.437 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:34.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:34.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.438 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:34.438 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:34.438 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:34.438 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:34.439 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:34.439 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:34.439 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:34.439 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:34.439 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:34.439 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:34.440 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:34.440 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:34.440 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:34.440 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:34.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.441 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:34.441 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:34.441 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:34.441 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.442 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:34.442 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:34.442 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:34.442 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:34.442 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:34.443 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:34.443 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:34.443 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:34.443 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:34.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.444 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:34.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:34.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:34.444 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.445 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.445 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.445 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.445 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.445 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.445 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.446 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.446 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.446 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.446 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.446 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.447 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.447 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:34.447 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.448 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:34.448 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.448 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.449 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.449 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.449 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:34.450 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.451 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:34.451 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.451 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.452 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.452 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.452 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:34.453 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.453 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:34.454 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.454 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.454 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.454 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.454 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.455 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.455 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:34.455 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.456 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.456 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:34.456 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:34.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.456 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.457 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:34.457 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:34.457 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:34.457 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:34.458 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.458 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.458 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:34.458 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.458 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.459 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:34.459 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.459 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.459 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.460 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.460 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.460 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.460 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.460 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.461 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.461 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.461 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.461 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.461 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.462 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.462 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:34.462 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.463 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:34.463 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.463 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.463 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.464 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.464 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.464 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.465 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:34.465 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.465 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:34.466 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.466 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.466 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.467 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.467 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:34.467 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.469 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.471 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.472 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:34.472 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:34.472 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:34.472 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:34.472 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.473 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.473 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:34.473 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.473 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.474 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:34.474 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.474 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.474 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.474 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.475 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.475 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.475 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.475 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.475 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.476 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.476 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.476 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.476 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.476 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.477 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:34.477 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.478 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:34.478 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.478 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.479 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.479 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.479 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:34.479 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.480 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:34.480 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.481 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.481 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.481 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.482 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.482 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:34.482 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.483 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:34.483 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.483 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.484 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.484 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.484 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.484 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.484 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:34.485 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.485 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.485 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:34.486 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:34.486 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.486 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.486 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:34.487 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:34.487 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:34.487 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:34.487 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.487 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.487 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:34.488 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.488 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.488 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:34.489 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.489 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.489 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.489 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.489 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.490 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.490 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.490 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.490 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.490 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.491 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.491 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.491 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.491 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:34.492 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.492 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:34.493 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.493 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.493 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.494 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.494 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.494 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:34.494 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.495 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.495 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:34.495 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.496 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.496 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.496 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.497 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:34.497 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.498 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.498 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:34.498 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.498 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.498 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.499 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.499 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.499 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.499 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.500 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:34.500 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.500 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.500 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:34.501 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:34.501 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.501 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.501 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:34.502 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:34.502 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:34.502 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:34.502 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.503 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.503 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.503 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.503 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:34.504 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.504 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.504 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.504 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.504 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.505 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.505 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.505 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.505 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.506 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.506 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.506 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.506 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.507 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:34.507 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.508 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:34.508 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.508 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.508 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.509 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.509 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.509 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:34.509 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.510 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.510 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.510 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:34.511 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.511 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.511 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.511 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.511 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.512 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.512 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:34.512 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.513 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.513 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.513 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:34.513 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.513 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.514 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.514 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.514 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.514 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.515 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:34.515 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.516 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:34.516 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:34.516 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.516 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.517 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:34.517 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:34.517 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:34.517 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:34.517 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.518 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.518 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:34.518 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.518 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.519 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:34.519 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.519 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.519 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.520 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.520 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.520 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.520 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.521 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.521 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.521 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.521 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.521 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.522 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:34.522 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.523 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.523 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:34.523 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.523 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.523 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.524 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.524 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.524 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:34.525 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.525 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.525 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.525 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:34.526 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.526 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.526 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.526 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.527 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.527 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:34.527 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.530 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:34.530 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.531 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:34.531 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:34.531 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.531 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.532 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:34.532 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:34.532 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:34.532 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:34.533 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.533 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.533 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.533 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.534 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:34.534 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.534 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.534 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.535 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.535 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.535 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.535 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.535 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.536 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.536 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.536 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.536 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.536 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.537 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:34.537 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.538 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:34.538 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.538 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.538 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.539 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.539 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.539 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:34.540 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.540 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.540 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.540 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:34.541 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.541 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.541 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.541 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.542 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.542 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:34.542 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.543 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.543 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:34.543 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.544 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.544 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.544 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.545 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:34.545 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.546 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:34.546 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:34.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.546 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.547 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:34.547 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:34.547 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:34.547 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:34.547 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.547 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:52:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:34.584 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:34.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f48066470 msgr2=0x7f7f48068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f48066470 0x7f7f48068930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7f600b0000 tx=0x7f7f600a7040 comp rx=0 tx=0).stop 2023-10-26T05:52:34.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 msgr2=0x7f7f6415f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 0x7f7f6415f280 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00d7c0 tx=0x7f7f5c00dc80 comp rx=0 tx=0).stop 2023-10-26T05:52:34.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 shutdown_connections 2023-10-26T05:52:34.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f48066470 0x7f7f48068930 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f641f8a00 0x7f7f6415f280 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f64154370 0x7f7f640c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 --2- 172.21.15.71:0/1530159190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f50003980 0x7f7f640c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 >> 172.21.15.71:0/1530159190 conn(0x7f7f640cdd80 msgr2=0x7f7f6415e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:34.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 shutdown_connections 2023-10-26T05:52:34.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.380+0000 7f7f6b6fd640 1 -- 172.21.15.71:0/1530159190 wait complete. 2023-10-26T05:52:34.589 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:34.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 -- 172.21.15.71:0/2671140213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a415b9f0 msgr2=0x7ff4a4070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 --2- 172.21.15.71:0/2671140213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a415b9f0 0x7ff4a4070690 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009bc0 tx=0x7ff49c01cb80 comp rx=0 tx=0).stop 2023-10-26T05:52:34.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 -- 172.21.15.71:0/2671140213 shutdown_connections 2023-10-26T05:52:34.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 --2- 172.21.15.71:0/2671140213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a415b9f0 0x7ff4a4070690 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 --2- 172.21.15.71:0/2671140213 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4a40ac740 0x7ff4a415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 -- 172.21.15.71:0/2671140213 >> 172.21.15.71:0/2671140213 conn(0x7ff4a4069770 msgr2=0x7ff4a4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:34.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 -- 172.21.15.71:0/2671140213 shutdown_connections 2023-10-26T05:52:34.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.712+0000 7ff4ab890640 1 -- 172.21.15.71:0/2671140213 wait complete. 2023-10-26T05:52:34.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 Processor -- start 2023-10-26T05:52:34.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- start start 2023-10-26T05:52:34.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 0x7ff4a414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a413d420 0x7ff4a4138a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4a415b9f0 0x7ff4a41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40aeb90 con 0x7ff4a40ac740 2023-10-26T05:52:34.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40b0940 con 0x7ff4a413d420 2023-10-26T05:52:34.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41392a0 con 0x7ff4a415b9f0 2023-10-26T05:52:34.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 0x7ff4a414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 0x7ff4a414dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47008/0 (socket says 172.21.15.71:47008) 2023-10-26T05:52:34.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 -- 172.21.15.71:0/3781211985 learned_addr learned my addr 172.21.15.71:0/3781211985 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:34.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a9e2d640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4a415b9f0 0x7ff4a41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 -- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4a415b9f0 msgr2=0x7ff4a41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a8e2b640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a413d420 0x7ff4a4138a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4a415b9f0 0x7ff4a41524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 -- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a413d420 msgr2=0x7ff4a4138a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a413d420 0x7ff4a4138a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 -- 172.21.15.71:0/3781211985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c009820 con 0x7ff4a40ac740 2023-10-26T05:52:34.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4a962c640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 0x7ff4a414dc50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c700 tx=0x7ff4a000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:34.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0004230 con 0x7ff4a40ac740 2023-10-26T05:52:34.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff4a00044a0 con 0x7ff4a40ac740 2023-10-26T05:52:34.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a4139580 con 0x7ff4a40ac740 2023-10-26T05:52:34.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000a8a0 con 0x7ff4a40ac740 2023-10-26T05:52:34.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40b0b90 con 0x7ff4a40ac740 2023-10-26T05:52:34.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4066b10 con 0x7ff4a40ac740 2023-10-26T05:52:34.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.716+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4a000aa00 con 0x7ff4a40ac740 2023-10-26T05:52:34.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.720+0000 7ff495ffb640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff488066470 0x7ff488068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:34.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.720+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4a0084080 con 0x7ff4a40ac740 2023-10-26T05:52:34.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.720+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=107}) v3 -- 0x7ff48806dfa0 con 0x7ff4a40ac740 2023-10-26T05:52:34.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.720+0000 7ff4a8e2b640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff488066470 0x7ff488068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:34.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.720+0000 7ff4a8e2b640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff488066470 0x7ff488068930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff4980058d0 tx=0x7ff498005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:34.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.724+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff4a0051590 con 0x7ff4a40ac740 2023-10-26T05:52:34.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4a415be40 con 0x7ff4a40ac740 2023-10-26T05:52:34.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff495ffb640 1 -- 172.21.15.71:0/3781211985 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7ff4a415be40 con 0x7ff4a40ac740 2023-10-26T05:52:34.902 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:34.903 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:34.903 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:34.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff488066470 msgr2=0x7ff488068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff488066470 0x7ff488068930 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff4980058d0 tx=0x7ff498005860 comp rx=0 tx=0).stop 2023-10-26T05:52:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 msgr2=0x7ff4a414dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 0x7ff4a414dc50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000c700 tx=0x7ff4a000cbc0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 shutdown_connections 2023-10-26T05:52:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4a415b9f0 0x7ff4a41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a413d420 0x7ff4a4138a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff488066470 0x7ff488068930 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 --2- 172.21.15.71:0/3781211985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a40ac740 0x7ff4a414dc50 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 >> 172.21.15.71:0/3781211985 conn(0x7ff4a4069770 msgr2=0x7ff4a413d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 shutdown_connections 2023-10-26T05:52:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:34.896+0000 7ff4ab890640 1 -- 172.21.15.71:0/3781211985 wait complete. 2023-10-26T05:52:34.920 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:52:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":7074,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.684165303,"mdsmap_epoch":38,"osdmap_epoch":106,"osdmap_epoch_barrier":103,"uptime":28.317648930000001} 2023-10-26T05:52:35.049 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 7074, "mdsmap_epoch": 38, "osdmap_epoch": 106, "osdmap_epoch_barrier": 103, "rank_uptime": 22.684165303, "state": "up:active", "uptime": 28.31764893, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:52:35.049 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:52:35.050 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:35.050 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:35.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:35.057 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-26T05:52:35.112 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:52:35.104+0000 7fa89effd640 -1 client.7242 handle_client_reply no pending request on tid 14 safe is:1 2023-10-26T05:52:35.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:52:35.113 INFO:teuthology.orchestra.run.smithi071.stderr:dd: failed to open 'small_file_1': Permission denied 2023-10-26T05:52:35.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:35.114 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T05:52:35.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:35.171 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T05:52:35.251 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:35.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/3793789897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 msgr2=0x7f7bb815b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:35.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/3793789897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb815b9f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7bb806db30 tx=0x7f7bb001cd60 comp rx=0 tx=0).stop 2023-10-26T05:52:35.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/3793789897 shutdown_connections 2023-10-26T05:52:35.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/3793789897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bb813d800 0x7f7bb815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/3793789897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb815b9f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/3793789897 >> v2:172.21.15.86:3300/0 conn(0x7f7bb813d420 0x7f7bb80ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/3793789897 >> 172.21.15.71:0/3793789897 conn(0x7f7bb8069770 msgr2=0x7f7bb8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:35.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/3793789897 shutdown_connections 2023-10-26T05:52:35.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/3793789897 wait complete. 2023-10-26T05:52:35.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 Processor -- start 2023-10-26T05:52:35.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- start start 2023-10-26T05:52:35.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb813d420 0x7f7bb81346e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:35.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bb813d800 0x7f7bb8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:35.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb8157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:35.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80ae630 con 0x7f7bb813dc20 2023-10-26T05:52:35.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80ad780 con 0x7f7bb813d420 2023-10-26T05:52:35.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80ada20 con 0x7f7bb813d800 2023-10-26T05:52:35.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb8157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:35.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb8157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47030/0 (socket says 172.21.15.71:47030) 2023-10-26T05:52:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 -- 172.21.15.71:0/2568671674 learned_addr learned my addr 172.21.15.71:0/2568671674 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbdf93640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb813d420 0x7f7bb81346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbd792640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bb813d800 0x7f7bb8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 -- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bb813d800 msgr2=0x7f7bb8156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bb813d800 0x7f7bb8156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 -- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb813d420 msgr2=0x7f7bb81346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb813d420 0x7f7bb81346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 -- 172.21.15.71:0/2568671674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00098e0 con 0x7f7bb813dc20 2023-10-26T05:52:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbdf93640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb813d420 0x7f7bb81346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bbe794640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb8157460 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7bb001b410 tx=0x7f7bb0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0004410 con 0x7f7bb813dc20 2023-10-26T05:52:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7bb0004660 con 0x7f7bb813dc20 2023-10-26T05:52:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb80add00 con 0x7f7bb813dc20 2023-10-26T05:52:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00a5c60 con 0x7f7bb813dc20 2023-10-26T05:52:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.420+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb80ae2c0 con 0x7f7bb813dc20 2023-10-26T05:52:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb8066b10 con 0x7f7bb813dc20 2023-10-26T05:52:35.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7bb00b4040 con 0x7f7bb813dc20 2023-10-26T05:52:35.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7baa7fc640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b9c066470 0x7f7b9c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:35.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bb011ef10 con 0x7f7bb813dc20 2023-10-26T05:52:35.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=107}) v3 -- 0x7f7b9c06dfa0 con 0x7f7bb813dc20 2023-10-26T05:52:35.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7bbdf93640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b9c066470 0x7f7b9c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:35.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.424+0000 7f7bbdf93640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b9c066470 0x7f7b9c068930 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7bb81559d0 tx=0x7f7bb4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:35.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.428+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7bb00b2020 con 0x7f7bb813dc20 2023-10-26T05:52:35.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.568+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7bb80721e0 con 0x7f7bb813dc20 2023-10-26T05:52:35.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.576+0000 7f7baa7fc640 1 -- 172.21.15.71:0/2568671674 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v106) v1 ==== 81+0+34090 (secure 0 0 0) 0x7f7bb00ec420 con 0x7f7bb813dc20 2023-10-26T05:52:35.584 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:35.584 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:35.584 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 106, 2023-10-26T05:52:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:31.489420+0000", 2023-10-26T05:52:35.585 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:35.586 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:35.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:35.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:35.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:35.586 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:35.587 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:35.587 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:35.587 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:35.587 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:35.587 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:35.588 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:35.588 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:35.588 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:35.588 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:35.588 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:35.588 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:35.589 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:35.589 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:35.589 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:35.589 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:35.589 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:35.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:35.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:35.590 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:35.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:35.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:35.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:35.591 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:35.591 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:35.592 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:35.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:35.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:35.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:35.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:35.593 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:35.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:35.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:35.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:35.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:35.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:35.594 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:35.594 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:35.594 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:35.594 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:35.594 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:35.595 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:35.595 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:35.595 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:35.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.595 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:35.596 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:35.596 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:35.596 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:35.596 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:35.596 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:35.597 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:35.597 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:35.597 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:35.597 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:35.597 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:35.598 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:35.598 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:35.598 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:35.598 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:35.598 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:35.599 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:35.599 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:35.599 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:35.599 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:35.599 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:35.600 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:35.600 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:35.600 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:35.600 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:35.600 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:35.601 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:35.601 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.601 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:35.601 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:35.601 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:35.601 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:35.602 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:35.602 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:35.602 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:35.602 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:35.602 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:35.603 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:35.603 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:35.603 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:35.603 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:35.603 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:35.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.604 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:35.604 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:35.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.604 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:35.605 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:35.605 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:35.605 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:35.605 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:35.605 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:35.605 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:35.606 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:35.606 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:35.606 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.606 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.607 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:35.607 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:35.607 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:35.607 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:35.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:35.608 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:35.608 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:35.608 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:35.608 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:35.609 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:35.609 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:35.609 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:35.609 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:35.609 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:35.609 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:35.610 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:35.610 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:35.610 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:35.610 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:35.610 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:35.611 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:35.611 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:35.611 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:35.611 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:35.611 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:35.612 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:35.612 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:35.612 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.612 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:35.612 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:35.612 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:35.613 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:35.613 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:35.613 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:35.613 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:35.613 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:35.614 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:35.614 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:35.614 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:35.614 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:35.614 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:35.615 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:35.615 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:35.615 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:35.615 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:35.615 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:35.616 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:35.616 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:35.616 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:35.616 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:35.616 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:35.617 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:35.617 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:35.617 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:35.617 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:35.617 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.618 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:35.618 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:35.618 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:35.618 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:35.618 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:35.619 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:35.619 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:35.619 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:35.619 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:35.619 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:35.620 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:35.620 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:35.620 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:35.620 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:35.620 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:35.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.621 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:35.621 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:35.621 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:35.621 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.622 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:35.622 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:35.622 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:35.622 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:35.622 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:35.623 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:35.623 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:35.623 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:35.623 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:35.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.624 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:35.624 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:35.624 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:35.624 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:35.625 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:35.625 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:35.625 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:35.625 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:35.626 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:35.626 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:35.626 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:35.626 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:35.626 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:35.627 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:35.627 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:35.627 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:35.627 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:35.627 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:35.627 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:35.628 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:35.628 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:35.628 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:35.628 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:35.628 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:35.629 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:35.629 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:35.629 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:35.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.629 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:35.629 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:35.630 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:35.630 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:35.630 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:35.630 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:35.630 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:35.631 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:35.631 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:35.631 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:35.631 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:35.631 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:35.633 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:35.633 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:35.634 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:35.634 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:35.634 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:35.634 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:35.634 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:35.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:35.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:35.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:35.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:35.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:35.636 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:35.637 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:35.637 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:35.637 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:35.637 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:35.637 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:35.638 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:35.638 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:35.638 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:35.638 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:35.638 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:35.639 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:35.639 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:35.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.639 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:35.639 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:35.640 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:35.640 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.640 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:35.640 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:35.641 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:35.641 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:35.641 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:35.641 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:35.641 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:35.641 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:35.642 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:35.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.642 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:35.642 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:35.643 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.643 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:35.643 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.643 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.643 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.644 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.644 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.644 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.644 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.644 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.645 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.645 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.645 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.645 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.645 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.645 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.646 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.646 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:35.646 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.646 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.648 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.648 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.648 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.648 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.650 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.650 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.650 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.650 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.651 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:35.651 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.652 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.652 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:35.652 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.652 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.653 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.653 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.653 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.653 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.654 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:35.654 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.654 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.654 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.655 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:35.655 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:35.655 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.655 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.656 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:35.656 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:35.656 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:35.656 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:35.656 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.657 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.657 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:35.657 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.657 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.658 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.658 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:35.658 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.658 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.658 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.659 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.659 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.659 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.659 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.659 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.659 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.660 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.660 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.660 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.660 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.660 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.661 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:35.661 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.662 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:35.662 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.662 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.662 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.663 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.663 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.663 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.663 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.663 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:35.664 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.664 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:35.665 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.665 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.665 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.666 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.666 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.666 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:35.666 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.667 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:35.667 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.668 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.669 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.669 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:35.669 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.670 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:35.670 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:35.670 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.671 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.671 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:35.671 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:35.672 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:35.672 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:35.672 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.672 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.673 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:35.673 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.673 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.674 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:35.674 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.674 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.674 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.674 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.675 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.675 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.675 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.675 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.675 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.676 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.676 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.676 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.676 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.677 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:35.677 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.677 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:35.678 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.678 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.678 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.679 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:35.680 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.680 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.680 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:35.681 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.681 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.681 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.681 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.682 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.682 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.682 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:35.682 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.683 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.683 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.683 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:35.683 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.683 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.684 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.684 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:35.685 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.686 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:35.686 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:35.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.686 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.687 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:35.687 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:35.687 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:35.687 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:35.687 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.689 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:35.689 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.689 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.689 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.689 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.690 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.690 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.690 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.690 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.690 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.691 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.691 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.691 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.693 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:35.693 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.694 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.695 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:35.695 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.695 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.696 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:35.696 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.696 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.697 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:35.698 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.698 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.698 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:35.699 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.699 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.699 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.700 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:35.700 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.701 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.701 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.701 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:35.701 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:35.701 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.701 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.702 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:35.702 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:35.702 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:35.702 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:35.703 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.703 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.703 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.703 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:35.704 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.704 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.704 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.708 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:35.708 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.708 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.709 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.709 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.709 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:35.710 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.710 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:35.711 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.711 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.711 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.712 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.712 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.712 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:35.712 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.713 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.713 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:35.713 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.714 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.714 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:35.715 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:35.716 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:35.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.716 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.717 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:35.717 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:35.717 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:35.717 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:35.718 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.718 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.718 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:35.718 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.718 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.719 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:35.719 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.719 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.719 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.720 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.720 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.720 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.720 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.720 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.721 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.721 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.721 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.721 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.721 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.722 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.722 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:35.722 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.723 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:35.723 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.723 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.724 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.724 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.724 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.724 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.725 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:35.725 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.726 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:35.726 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.726 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.727 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.727 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.727 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:35.728 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.728 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.728 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.729 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.730 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.730 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:35.730 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.731 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.731 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:35.731 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:35.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.732 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.732 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:35.732 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:35.733 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:35.733 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:35.733 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.733 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.733 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.734 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.734 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:35.734 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.735 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.735 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.735 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.735 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.735 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.735 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.736 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.736 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.736 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.736 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.736 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:35.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:35.738 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.739 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.739 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.739 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.739 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.740 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.740 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:35.740 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.741 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:35.741 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.741 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.742 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.742 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.742 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.743 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:35.743 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.743 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:35.744 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.744 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.744 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.745 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:35.746 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.746 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.746 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:35.747 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:35.747 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.747 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.747 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:35.747 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:35.748 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:35.748 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:35.748 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.748 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.749 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:35.749 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.749 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.749 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:35.750 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:35.750 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:35.750 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:35.750 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:35.750 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:35.751 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:35.751 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:35.751 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:35.751 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:35.752 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:35.752 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:35.752 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:35.752 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.753 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:35.753 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.754 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.756 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.758 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:35.759 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:35.760 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:35.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:35.763 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:52:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:35.764 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.765 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.766 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:35.767 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.768 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.769 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:35.770 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.771 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.773 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.775 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.776 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.777 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:35.778 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:35.779 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:35.780 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:35.781 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:35.782 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:35.783 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:35.784 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:35.785 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:35.785 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:35.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b9c066470 msgr2=0x7f7b9c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:35.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b9c066470 0x7f7b9c068930 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7bb81559d0 tx=0x7f7bb4008040 comp rx=0 tx=0).stop 2023-10-26T05:52:35.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 msgr2=0x7f7bb8157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:35.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb8157460 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7bb001b410 tx=0x7f7bb0003f80 comp rx=0 tx=0).stop 2023-10-26T05:52:35.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 shutdown_connections 2023-10-26T05:52:35.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7b9c066470 0x7f7b9c068930 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bb813dc20 0x7f7bb8157460 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bb813d800 0x7f7bb8156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 --2- 172.21.15.71:0/2568671674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bb813d420 0x7f7bb81346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:35.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 >> 172.21.15.71:0/2568671674 conn(0x7f7bb8069770 msgr2=0x7f7bb815a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:35.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 shutdown_connections 2023-10-26T05:52:35.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:35.580+0000 7f7bc01f7640 1 -- 172.21.15.71:0/2568671674 wait complete. 2023-10-26T05:52:35.790 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:52:36.585 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:52:36.580+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-10-26T05:52:40.791 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:40.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2339527190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f437806a820 msgr2=0x7f43780ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:40.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2339527190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f437806a820 0x7f43780ceea0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4370009900 tx=0x7f437001c880 comp rx=0 tx=0).stop 2023-10-26T05:52:40.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2339527190 shutdown_connections 2023-10-26T05:52:40.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2339527190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f437806a820 0x7f43780ceea0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:40.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2339527190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437815b5c0 0x7f43780ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:40.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2339527190 >> 172.21.15.71:0/2339527190 conn(0x7f43780d12d0 msgr2=0x7f43780d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:40.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2339527190 shutdown_connections 2023-10-26T05:52:40.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2339527190 wait complete. 2023-10-26T05:52:40.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 Processor -- start 2023-10-26T05:52:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- start start 2023-10-26T05:52:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 0x7f437814f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437806a820 0x7f4378159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:40.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437815b5c0 0x7f43781547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437815a6c0 con 0x7f4378069770 2023-10-26T05:52:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437814f780 con 0x7f437806a820 2023-10-26T05:52:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437815aae0 con 0x7f437815b5c0 2023-10-26T05:52:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 0x7f437814f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 0x7f437814f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47044/0 (socket says 172.21.15.71:47044) 2023-10-26T05:52:40.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 -- 172.21.15.71:0/2704571807 learned_addr learned my addr 172.21.15.71:0/2704571807 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 -- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437815b5c0 msgr2=0x7f43781547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437815b5c0 0x7f43781547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 -- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437806a820 msgr2=0x7f4378159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437806a820 0x7f4378159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:40.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 -- 172.21.15.71:0/2704571807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370009560 con 0x7f4378069770 2023-10-26T05:52:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437659c640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 0x7f437814f240 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f436000d720 tx=0x7f436000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360009c30 con 0x7f4378069770 2023-10-26T05:52:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4360009e80 con 0x7f4378069770 2023-10-26T05:52:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4378159a40 con 0x7f4378069770 2023-10-26T05:52:40.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360005260 con 0x7f4378069770 2023-10-26T05:52:40.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.924+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437815a000 con 0x7f4378069770 2023-10-26T05:52:40.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4378066b10 con 0x7f4378069770 2023-10-26T05:52:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f43600053c0 con 0x7f4378069770 2023-10-26T05:52:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f436effd640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43580664a0 0x7f4358068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4360084000 con 0x7f4378069770 2023-10-26T05:52:40.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=110}) v3 -- 0x7f435806dfd0 con 0x7f4378069770 2023-10-26T05:52:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f4375d9b640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43580664a0 0x7f4358068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.928+0000 7f4375d9b640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43580664a0 0x7f4358068960 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4368007fd0 tx=0x7f4368007d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:40.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:40.932+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4360017060 con 0x7f4378069770 2023-10-26T05:52:41.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.088+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f437815a930 con 0x7f4378069770 2023-10-26T05:52:41.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.088+0000 7f436effd640 1 -- 172.21.15.71:0/2704571807 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v109) v1 ==== 81+0+34050 (secure 0 0 0) 0x7f4360020020 con 0x7f4378069770 2023-10-26T05:52:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:41.097 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:41.097 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 109, 2023-10-26T05:52:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:40.166563+0000", 2023-10-26T05:52:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:41.098 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:41.099 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:41.100 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:41.101 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:41.102 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.103 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:41.104 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:41.105 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:41.106 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:41.107 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:41.109 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:41.110 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:41.111 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:41.112 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:41.113 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:41.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:41.115 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:41.116 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:41.117 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:41.118 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:41.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:41.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:41.121 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:41.123 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:41.124 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:41.125 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:41.127 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:41.131 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:41.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.133 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:41.134 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:41.135 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:41.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:41.138 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:41.139 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:41.140 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 3, 2023-10-26T05:52:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full", 2023-10-26T05:52:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:41.141 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:41.142 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:41.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:41.144 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "104", 2023-10-26T05:52:41.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:41.146 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:41.147 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:41.148 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:41.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:41.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:41.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:41.152 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:41.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:41.155 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:41.157 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.158 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.159 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:41.160 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:41.163 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.167 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:41.168 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.169 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.171 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:41.172 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.173 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.173 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.173 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.173 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.174 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:41.174 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.174 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.174 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.174 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.175 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.175 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.175 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.175 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.175 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.176 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.176 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.176 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.176 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:41.177 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.178 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:41.178 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.178 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.179 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:41.180 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.180 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:41.181 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.181 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.181 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.182 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:41.182 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.183 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:41.183 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.184 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.184 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.184 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.187 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.189 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.190 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.194 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:41.195 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.196 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.198 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.199 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.200 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.201 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: "full", 2023-10-26T05:52:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.203 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.204 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.205 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.206 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.207 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.208 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:41.210 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.213 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.214 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.215 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.216 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.217 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:41.218 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.219 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.220 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:41.221 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:41.222 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.225 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.227 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.228 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:41.230 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:41.231 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:52:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.232 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.233 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.234 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.234 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.234 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.234 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.234 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.235 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.235 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.235 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.235 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.235 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:41.237 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.238 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.238 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.239 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:41.239 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:41.240 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.240 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.241 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:41.241 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:41.242 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.243 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.244 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.244 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:41.244 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.245 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.245 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:41.245 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:41.245 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.245 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.246 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:41.246 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:41.246 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:41.246 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:41.247 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.247 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.247 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.248 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:41.248 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.248 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.248 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.248 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.249 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.249 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.249 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.249 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.249 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.250 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.250 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.250 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.251 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:41.251 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:41.252 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.252 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.252 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.252 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:41.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.254 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:41.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.255 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.255 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.255 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.257 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:41.257 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:41.259 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.259 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:41.260 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:41.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.260 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.260 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:41.260 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:41.261 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:41.261 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:41.261 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.261 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.261 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.262 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.262 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:41.262 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:41.263 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:41.263 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:41.263 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:41.263 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:41.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:41.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:41.264 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:41.264 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:41.264 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:41.264 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:41.264 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:41.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:41.265 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:41.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.267 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.267 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:41.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:41.268 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.270 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:41.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:41.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:41.272 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.272 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.272 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.272 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:41.273 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:41.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:41.273 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:41.273 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:41.274 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:41.274 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:41.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.275 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.275 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:41.275 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:41.275 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:41.276 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:41.276 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:41.276 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull", 2023-10-26T05:52:41.276 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:41.276 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:41.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.278 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:41.278 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.278 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.278 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.278 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.278 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.279 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:41.279 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.279 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:41.280 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.280 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.280 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.280 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.280 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.281 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:41.281 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.281 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:41.281 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.282 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.282 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.282 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.282 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.282 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:41.283 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.283 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:41.283 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.284 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.284 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.284 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.284 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.284 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:41.284 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.285 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:41.285 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.285 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.286 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.286 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.286 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.286 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:41.286 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.287 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:41.287 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.287 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.287 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.288 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.288 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:41.288 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.289 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:41.289 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.289 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.289 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.290 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.290 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.290 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:41.290 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:41.291 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:41.291 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:41.291 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:41.291 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.292 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:41.293 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:41.293 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:41.293 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:41.293 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:41.293 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:41.294 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:41.294 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:41.294 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:41.294 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:41.294 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:41.295 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:41.295 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:41.295 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:41.295 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:41.295 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:41.296 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:41.296 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:41.296 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:41.296 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:41.296 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:41.297 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:41.297 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:41.297 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:41.297 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:41.297 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:41.297 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:41.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:41.298 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43580664a0 msgr2=0x7f4358068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43580664a0 0x7f4358068960 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4368007fd0 tx=0x7f4368007d40 comp rx=0 tx=0).stop 2023-10-26T05:52:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 msgr2=0x7f437814f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 0x7f437814f240 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f436000d720 tx=0x7f436000dbe0 comp rx=0 tx=0).stop 2023-10-26T05:52:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 shutdown_connections 2023-10-26T05:52:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437815b5c0 0x7f43781547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437806a820 0x7f4378159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43580664a0 0x7f4358068960 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 --2- 172.21.15.71:0/2704571807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4378069770 0x7f437814f240 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:41.301 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 >> 172.21.15.71:0/2704571807 conn(0x7f43780d12d0 msgr2=0x7f43780b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:41.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 shutdown_connections 2023-10-26T05:52:41.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:41.092+0000 7f437ca2a640 1 -- 172.21.15.71:0/2704571807 wait complete. 2023-10-26T05:52:41.302 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:52:42.145 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:52:42.140+0000 7fc579fce640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-10-26T05:52:45.580 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:52:45.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:52:46.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:46.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 -- 172.21.15.71:0/1599779889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc80c004a90 msgr2=0x7fc80c004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 --2- 172.21.15.71:0/1599779889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc80c004a90 0x7fc80c004eb0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c5c0 tx=0x7fc81001d340 comp rx=0 tx=0).stop 2023-10-26T05:52:46.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 -- 172.21.15.71:0/1599779889 shutdown_connections 2023-10-26T05:52:46.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 --2- 172.21.15.71:0/1599779889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc80c004a90 0x7fc80c004eb0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 -- 172.21.15.71:0/1599779889 >> 172.21.15.71:0/1599779889 conn(0x7fc820069770 msgr2=0x7fc820069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:46.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 -- 172.21.15.71:0/1599779889 shutdown_connections 2023-10-26T05:52:46.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 -- 172.21.15.71:0/1599779889 wait complete. 2023-10-26T05:52:46.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 Processor -- start 2023-10-26T05:52:46.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.436+0000 7fc82482b640 1 -- start start 2023-10-26T05:52:46.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc80c004a90 0x7fc820145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc820143840 0x7fc820144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 0x7fc820159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820145980 con 0x7fc820144740 2023-10-26T05:52:46.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820146350 con 0x7fc820143840 2023-10-26T05:52:46.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc820144d70 con 0x7fc80c004a90 2023-10-26T05:52:46.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 0x7fc820159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 0x7fc820159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41512/0 (socket says 172.21.15.71:41512) 2023-10-26T05:52:46.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 -- 172.21.15.71:0/4156127246 learned_addr learned my addr 172.21.15.71:0/4156127246 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:46.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81dd9b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc820143840 0x7fc820144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81e59c640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc80c004a90 0x7fc820145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 -- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc80c004a90 msgr2=0x7fc820145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc80c004a90 0x7fc820145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 -- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc820143840 msgr2=0x7fc820144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc820143840 0x7fc820144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 -- 172.21.15.71:0/4156127246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820145050 con 0x7fc820144740 2023-10-26T05:52:46.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81ed9d640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 0x7fc820159ef0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fc82006b600 tx=0x7fc818010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:46.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81801a070 con 0x7fc820144740 2023-10-26T05:52:46.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc818004160 con 0x7fc820144740 2023-10-26T05:52:46.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc820145330 con 0x7fc820144740 2023-10-26T05:52:46.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818005020 con 0x7fc820144740 2023-10-26T05:52:46.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82014a4a0 con 0x7fc820144740 2023-10-26T05:52:46.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc820066b10 con 0x7fc820144740 2023-10-26T05:52:46.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc818005180 con 0x7fc820144740 2023-10-26T05:52:46.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc816ffd640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8000664a0 0x7fc800068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc818083f60 con 0x7fc820144740 2023-10-26T05:52:46.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.440+0000 7fc81e59c640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8000664a0 0x7fc800068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.444+0000 7fc81e59c640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8000664a0 0x7fc800068960 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8100ac000 tx=0x7fc8100b1040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:46.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.444+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc818051350 con 0x7fc820144740 2023-10-26T05:52:46.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.604+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc8200721e0 con 0x7fc820144740 2023-10-26T05:52:46.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.604+0000 7fc816ffd640 1 -- 172.21.15.71:0/4156127246 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+33923 (secure 0 0 0) 0x7fc818054970 con 0x7fc820144740 2023-10-26T05:52:46.612 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:46.612 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:46.613 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 112, 2023-10-26T05:52:46.613 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:46.613 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:46.613 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:43.204060+0000", 2023-10-26T05:52:46.613 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:46.614 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:46.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:46.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:46.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:46.614 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:46.615 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:46.615 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:46.615 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:46.615 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:46.615 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:46.616 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:46.616 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:46.616 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:46.616 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:46.616 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:46.617 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:46.617 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:46.617 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:46.617 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:46.617 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:46.618 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:46.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.618 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:46.618 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:46.618 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:46.618 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:46.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:46.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:46.619 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:46.619 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:46.619 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:46.620 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:46.620 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:46.620 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:46.620 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:46.620 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:46.620 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:46.621 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:46.621 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:46.621 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:46.621 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:46.621 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:46.622 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:46.622 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:46.622 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:46.622 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:46.622 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:46.623 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:46.623 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:46.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.623 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:52:46.623 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:46.623 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:46.624 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:46.624 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:46.624 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:46.624 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:46.624 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:46.625 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:46.625 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:46.625 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:46.625 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:46.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:46.626 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:46.626 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:46.626 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:46.626 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:46.626 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:46.626 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:46.627 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:46.627 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:46.627 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:46.627 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:46.627 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:46.628 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:46.628 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:46.628 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:46.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.628 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:46.629 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:46.629 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:46.629 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:46.629 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:46.629 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:46.629 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:46.630 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:46.630 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:46.630 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:46.630 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:46.630 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:46.631 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:46.631 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:46.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.631 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:46.631 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:46.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.632 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:46.632 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:46.632 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:46.632 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:46.633 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:46.633 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:46.633 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:46.633 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:46.633 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:46.633 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.634 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.634 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:46.634 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:46.634 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:46.635 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:46.635 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:46.635 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:46.635 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:46.635 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:46.635 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:46.636 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:46.636 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:46.636 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:46.636 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:46.636 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:46.637 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:46.637 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:46.637 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:46.637 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:46.637 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:46.638 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:46.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:46.638 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:46.638 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:46.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:46.638 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:46.639 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:46.639 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:46.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.639 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:52:46.639 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:46.640 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:46.640 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:46.640 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:46.640 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:46.640 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:46.640 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:46.641 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:46.641 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:46.641 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:46.641 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:46.641 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:46.642 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:46.642 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:46.642 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:46.642 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:46.642 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:46.643 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:46.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:46.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:46.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:46.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:46.644 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:46.644 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:46.644 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:46.644 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:46.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.644 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:46.645 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:46.645 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:46.645 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:46.645 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:46.645 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:46.646 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:46.646 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:46.646 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:46.646 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:46.646 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:46.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.648 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:46.648 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:46.648 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:46.648 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.649 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:46.649 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:46.649 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:46.649 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:46.649 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:46.649 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:46.650 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:46.650 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:46.650 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:46.650 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.651 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:46.651 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:46.651 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:46.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:46.652 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:46.652 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:46.652 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:46.652 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:46.652 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:46.652 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:46.653 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:46.653 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:46.653 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:46.653 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:46.653 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:46.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:46.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:46.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:46.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:46.654 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:46.655 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:46.655 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:46.655 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:46.655 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:46.655 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:46.655 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:46.656 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:46.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.656 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:52:46.656 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:46.656 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:46.657 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:46.658 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:46.658 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:46.658 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:46.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:46.658 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:46.659 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:46.659 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:46.659 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:46.659 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:46.659 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:46.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:46.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:46.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:46.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:46.660 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:46.660 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:46.661 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:46.661 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:46.661 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.661 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:46.661 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:46.662 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:46.662 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:46.662 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:46.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:46.662 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:46.662 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:46.663 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:46.663 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:46.663 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:46.663 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:46.663 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:46.664 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:46.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.664 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:46.664 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:46.664 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:46.664 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.665 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:46.665 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:46.665 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:46.665 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:46.666 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:46.666 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:46.666 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:46.666 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:46.666 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:46.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.667 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.667 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:46.667 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:46.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.667 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:46.668 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.668 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.668 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.668 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.668 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.668 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.669 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.669 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.669 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.669 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.669 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.670 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.670 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.670 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:46.671 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.671 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.671 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.671 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:46.671 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.672 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.672 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.672 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.673 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:46.673 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.674 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.674 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:46.674 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.674 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.674 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.675 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.675 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.675 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.675 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:46.676 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.676 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:46.676 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.677 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.677 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.677 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.678 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:46.678 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:46.679 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:46.679 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.679 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.680 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:46.680 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:46.680 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:46.680 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:46.681 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.681 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.681 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.681 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.682 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:46.682 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.682 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.682 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.682 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.683 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.683 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.683 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.683 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.683 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.684 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.684 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.684 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:46.685 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.685 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:46.686 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.686 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.686 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.686 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.687 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.687 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.687 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.687 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:46.687 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.688 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.688 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:46.688 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.689 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.689 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.689 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.689 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.690 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.690 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:46.690 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:46.691 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.691 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.692 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:46.693 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.693 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:46.694 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:46.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.694 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.694 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:46.695 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:46.695 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:46.695 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:46.695 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.695 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.695 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.696 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.696 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:46.696 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.697 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.697 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.697 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.697 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.697 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.698 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.698 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.698 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.698 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.698 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.699 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:46.699 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.700 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:46.700 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.701 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.701 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.701 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.701 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.702 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:46.702 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.703 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:46.703 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.703 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.704 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.704 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.705 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:46.705 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.705 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.705 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:46.706 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.706 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.706 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.706 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.707 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.707 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:46.707 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.708 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.708 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:46.708 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:46.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.708 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.709 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:46.709 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:46.709 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:46.709 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:46.709 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.710 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.710 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.710 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.711 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.712 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.712 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.712 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.712 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.712 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.712 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.713 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.713 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.713 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.713 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:46.713 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:46.714 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.715 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:46.716 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.717 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:46.717 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.717 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.718 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.718 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.718 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.718 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:46.719 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.719 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:46.720 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.720 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.721 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.722 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.722 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.722 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.722 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:46.723 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.723 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:46.723 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:46.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.724 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.724 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:46.724 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:46.725 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:46.725 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:46.725 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.725 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.725 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.726 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.726 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:46.726 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.727 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.727 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.727 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.727 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.727 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.728 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.728 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.728 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.728 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.728 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.729 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:46.729 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.730 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:46.730 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.731 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.731 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.732 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:46.732 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.733 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.733 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:46.733 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.733 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.734 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:46.735 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.735 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:46.736 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.736 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:46.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:46.738 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:46.739 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.739 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.739 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:46.739 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:46.740 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:46.740 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:46.740 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.740 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.740 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.741 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:46.741 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.741 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.742 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.742 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.742 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.742 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.742 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.743 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.743 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.743 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.743 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.743 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.743 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:46.744 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:46.745 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.745 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.746 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.746 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.747 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:46.747 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.748 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:46.748 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.748 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.749 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.749 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.749 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:46.750 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.750 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:46.751 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.751 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.751 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.752 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.752 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:46.752 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.753 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.753 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:46.753 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:46.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.754 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.754 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.754 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:46.754 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:46.754 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:46.755 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:46.755 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.755 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.755 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.756 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.756 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:46.756 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.756 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.756 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.757 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.757 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.757 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.758 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.758 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.758 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.758 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.758 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.760 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.760 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:46.760 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.760 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.761 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.761 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.762 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:46.762 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.762 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.763 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:46.763 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.763 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.763 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.764 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.764 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.764 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.764 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:46.765 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.765 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.765 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.765 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:46.766 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.766 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.766 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.766 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.767 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.767 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:46.767 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.768 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.768 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.768 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:46.768 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:46.768 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.769 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:46.769 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:46.769 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:46.770 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:46.770 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.770 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.770 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.770 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.771 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.771 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:46.771 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:46.771 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:46.772 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:46.772 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:46.772 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:46.772 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:46.772 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:46.773 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:46.773 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:46.773 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:46.773 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:46.773 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:46.773 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.774 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:46.774 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:46.775 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.775 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.776 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.776 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:46.776 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.777 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.777 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:46.777 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.778 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:46.778 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.778 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.778 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.779 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:46.779 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.779 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.779 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:46.780 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:46.807 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8000664a0 msgr2=0x7fc800068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8000664a0 0x7fc800068960 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc8100ac000 tx=0x7fc8100b1040 comp rx=0 tx=0).stop 2023-10-26T05:52:46.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 msgr2=0x7fc820159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 0x7fc820159ef0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fc82006b600 tx=0x7fc818010b50 comp rx=0 tx=0).stop 2023-10-26T05:52:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 shutdown_connections 2023-10-26T05:52:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc8000664a0 0x7fc800068960 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc820144740 0x7fc820159ef0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc820143840 0x7fc820144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 --2- 172.21.15.71:0/4156127246 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc80c004a90 0x7fc820145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 >> 172.21.15.71:0/4156127246 conn(0x7fc820069770 msgr2=0x7fc82015c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:46.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 shutdown_connections 2023-10-26T05:52:46.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.608+0000 7fc82482b640 1 -- 172.21.15.71:0/4156127246 wait complete. 2023-10-26T05:52:46.812 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T05:52:46.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:52:46.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/3382719086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88e80047d0 msgr2=0x7f88e8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 --2- 172.21.15.71:0/3382719086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88e80047d0 0x7f88e8004c50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f88f400bb10 tx=0x7f88f401cd80 comp rx=0 tx=0).stop 2023-10-26T05:52:46.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88ff59e640 1 -- 172.21.15.71:0/3382719086 reap_dead start 2023-10-26T05:52:46.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/3382719086 shutdown_connections 2023-10-26T05:52:46.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/3382719086 >> 172.21.15.71:0/3382719086 conn(0x7f8900069770 msgr2=0x7f8900069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:46.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/3382719086 shutdown_connections 2023-10-26T05:52:46.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/3382719086 wait complete. 2023-10-26T05:52:46.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 Processor -- start 2023-10-26T05:52:46.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- start start 2023-10-26T05:52:46.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88e80047d0 0x7f8900157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 0x7f89001f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890013bdc0 0x7f890014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890015ba40 con 0x7f890015b5e0 2023-10-26T05:52:46.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900139a40 con 0x7f890013bdc0 2023-10-26T05:52:46.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89001f20b0 con 0x7f88e80047d0 2023-10-26T05:52:46.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 0x7f89001f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 0x7f89001f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41524/0 (socket says 172.21.15.71:41524) 2023-10-26T05:52:46.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 -- 172.21.15.71:0/2421288191 learned_addr learned my addr 172.21.15.71:0/2421288191 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:46.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88ff59e640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890013bdc0 0x7f890014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 -- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88e80047d0 msgr2=0x7f8900157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fed9d640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88e80047d0 0x7f8900157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88e80047d0 0x7f8900157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 -- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890013bdc0 msgr2=0x7f890014a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:46.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890013bdc0 0x7f890014a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:46.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 -- 172.21.15.71:0/2421288191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890015bc80 con 0x7f890015b5e0 2023-10-26T05:52:46.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88ff59e640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890013bdc0 0x7f890014a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:46.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88fe59c640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 0x7f89001f1b70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f88f400ba60 tx=0x7f88f4005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:46.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40be070 con 0x7f890015b5e0 2023-10-26T05:52:46.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88f4004370 con 0x7f890015b5e0 2023-10-26T05:52:46.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40052a0 con 0x7f890015b5e0 2023-10-26T05:52:46.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8900139c90 con 0x7f890015b5e0 2023-10-26T05:52:46.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.944+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890013a250 con 0x7f890015b5e0 2023-10-26T05:52:46.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.948+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8900066b10 con 0x7f890015b5e0 2023-10-26T05:52:46.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.948+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f88f4005400 con 0x7f890015b5e0 2023-10-26T05:52:46.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.948+0000 7f88e77fe640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88ec066470 0x7f88ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:46.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.948+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88f411de50 con 0x7f890015b5e0 2023-10-26T05:52:46.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.948+0000 7f88fed9d640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88ec066470 0x7f88ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:46.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.948+0000 7f88fed9d640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88ec066470 0x7f88ec068930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f88f8003a30 tx=0x7f88f8001740 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:46.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:46.952+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88f40eb240 con 0x7f890015b5e0 2023-10-26T05:52:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.108+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f890013a3e0 con 0x7f890015b5e0 2023-10-26T05:52:47.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.108+0000 7f88e77fe640 1 -- 172.21.15.71:0/2421288191 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v112) v1 ==== 81+0+33923 (secure 0 0 0) 0x7f88f40ab070 con 0x7f890015b5e0 2023-10-26T05:52:47.116 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:47.117 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:52:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 112, 2023-10-26T05:52:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:52:47.117 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:52:47.118 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:52:43.204060+0000", 2023-10-26T05:52:47.118 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:52:47.118 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:52:47.118 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:52:47.118 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:52:47.119 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:52:47.119 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:52:47.119 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:52:47.119 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:52:47.119 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:52:47.120 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:47.120 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:52:47.120 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:52:47.120 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:52:47.120 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:52:47.121 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:52:47.121 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 13, 2023-10-26T05:52:47.121 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:52:47.121 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:52:47.121 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:52:47.122 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:52:47.122 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:52:47.122 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:52:47.122 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.122 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:52:47.123 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:52:47.123 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:52:47.123 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:47.123 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:47.123 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:47.124 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:47.124 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:47.124 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:47.124 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:47.124 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:47.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:47.125 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:47.125 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:47.125 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:47.125 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:52:47.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:52:47.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:52:47.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:52:47.126 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:52:47.126 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:47.127 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:47.127 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:47.127 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:47.127 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:47.127 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:47.127 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:47.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:52:47.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:47.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:47.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:47.129 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:47.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:47.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:47.129 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:47.129 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:47.130 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:47.130 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:47.130 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:47.130 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:47.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:47.131 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:47.131 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:47.131 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:47.131 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:47.132 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:47.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:47.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:47.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:47.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:47.133 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:47.133 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:47.133 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:47.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:47.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.134 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:47.134 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:47.134 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:47.134 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:47.134 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:47.135 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:47.135 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:47.135 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:47.135 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:47.135 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:47.136 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:47.136 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:47.136 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:52:47.136 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:52:47.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.137 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:47.137 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:52:47.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.137 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:47.137 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:52:47.138 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:52:47.138 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:52:47.138 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:47.138 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:47.138 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:47.139 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:47.139 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:47.139 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.139 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 12, 2023-10-26T05:52:47.140 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:52:47.140 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:09.840186+0000", 2023-10-26T05:52:47.140 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:47.140 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:47.140 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:47.141 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:47.141 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:47.141 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:47.141 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:47.141 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:47.142 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:47.142 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:47.142 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:47.142 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:47.142 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:52:47.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:52:47.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:52:47.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:52:47.143 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:52:47.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:47.143 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:47.144 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:47.144 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:47.144 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:47.144 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:47.144 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:47.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:52:47.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:47.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:47.145 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:47.146 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:47.146 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:47.146 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:47.146 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:47.146 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:47.147 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:47.147 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:47.147 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:47.147 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:47.147 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:47.148 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:47.148 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:47.148 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:47.148 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:47.148 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:47.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:47.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:47.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:47.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:47.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:47.150 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:47.150 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:47.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:47.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.150 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:47.151 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:47.151 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:47.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:47.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:47.151 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:47.152 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:47.152 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:47.152 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:47.152 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:47.152 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:47.153 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:47.153 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:52:47.153 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:52:47.153 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:52:47.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.153 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:47.154 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:47.154 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:52:47.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.154 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:47.155 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:52:47.155 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:52:47.155 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:47.155 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:52:47.155 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:52:47.156 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:47.156 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:47.156 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:47.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.157 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.157 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 13, 2023-10-26T05:52:47.157 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:52:47.157 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:52:10.672466+0000", 2023-10-26T05:52:47.157 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:52:47.158 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:52:47.158 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:52:47.158 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:52:47.158 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:52:47.158 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:52:47.159 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:52:47.159 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:52:47.159 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:52:47.159 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:52:47.159 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:52:47.159 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:52:47.160 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:52:47.160 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:52:47.160 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:52:47.160 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:52:47.161 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:52:47.161 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:52:47.161 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:52:47.161 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:52:47.161 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:52:47.162 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:52:47.162 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:52:47.162 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:52:47.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.162 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:52:47.163 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:52:47.163 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:52:47.163 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:52:47.163 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:52:47.163 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:52:47.163 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:52:47.164 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:52:47.164 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:52:47.164 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:52:47.164 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:52:47.164 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:52:47.165 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:52:47.165 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:52:47.165 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:52:47.165 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:52:47.165 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:52:47.166 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:52:47.166 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:52:47.166 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:52:47.166 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:52:47.166 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:52:47.167 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:52:47.167 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:52:47.167 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:52:47.167 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:52:47.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:52:47.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.168 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:52:47.168 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:52:47.168 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:52:47.168 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:52:47.168 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:52:47.169 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:52:47.169 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:52:47.169 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:52:47.169 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:52:47.169 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:52:47.170 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:52:47.170 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:52:47.170 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:52:47.170 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:52:47.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.171 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:52:47.171 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:52:47.171 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:52:47.171 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.172 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:52:47.172 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:52:47.172 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:52:47.172 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:52:47.172 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:52:47.173 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:52:47.173 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:52:47.173 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:52:47.173 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:52:47.173 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.173 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.174 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:47.174 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:52:47.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.174 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:47.175 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.175 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.175 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.175 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.175 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.176 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.176 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.176 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.176 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.176 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:52:47.178 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.178 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.178 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.178 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:52:47.179 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.179 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.179 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.179 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.180 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:52:47.180 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.181 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.181 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:52:47.181 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.182 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.182 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.182 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.183 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:52:47.183 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.184 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.184 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:52:47.184 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.184 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.185 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.185 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.185 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.186 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:52:47.186 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.186 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.187 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:52:47.187 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:52:47.187 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.187 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.188 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:52:47.188 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:52:47.188 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:52:47.188 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:52:47.188 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.189 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.189 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.189 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.189 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:47.190 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.190 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.190 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.190 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.191 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.191 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.191 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.191 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.191 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.191 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.192 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.192 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.192 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.193 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:52:47.193 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.193 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:52:47.194 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.194 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.194 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:52:47.196 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.196 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.196 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:52:47.196 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.197 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.197 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.197 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.197 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.198 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.198 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.198 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:52:47.198 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.199 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:52:47.199 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.200 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.200 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.200 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.200 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.201 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:52:47.201 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.202 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.202 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:52:47.202 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:52:47.202 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.202 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.203 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:52:47.203 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:52:47.203 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:52:47.203 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:52:47.203 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.204 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.204 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.204 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.205 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:47.205 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.205 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.205 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.205 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.205 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.206 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.207 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:52:47.208 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.208 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:52:47.209 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.209 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.209 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.209 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.209 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.209 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.210 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:52:47.210 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.211 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.211 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:52:47.211 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.211 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.212 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.212 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.212 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.213 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:52:47.213 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.213 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.214 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:52:47.214 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.214 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.215 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.215 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.215 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.215 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:52:47.216 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.216 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.216 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:52:47.217 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:52:47.217 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.217 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.217 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:52:47.217 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:52:47.218 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:52:47.218 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:52:47.218 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.218 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.218 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.219 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.219 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:47.219 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.220 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.220 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.220 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.220 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.220 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.220 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.221 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.221 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.221 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.221 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.221 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:52:47.222 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:52:47.223 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.224 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.224 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:52:47.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:52:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.227 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.227 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.228 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:52:47.228 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.229 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:52:47.229 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.229 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.229 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.230 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.230 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.230 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.230 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:52:47.231 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.231 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.231 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:52:47.232 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:52:47.232 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.232 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.232 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:52:47.232 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:52:47.233 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:52:47.233 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:52:47.233 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.233 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.233 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.234 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.234 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:47.234 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.235 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.235 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.235 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.235 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.235 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.236 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.236 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.236 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.236 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.236 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.236 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:52:47.237 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.238 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:52:47.238 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.239 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.239 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.239 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:52:47.240 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:52:47.241 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.241 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.242 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.242 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.243 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:52:47.243 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.244 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:52:47.244 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.244 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.244 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.244 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.245 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.245 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.245 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.245 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:52:47.245 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.246 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.246 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.246 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:52:47.246 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:52:47.247 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.247 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:52:47.247 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:52:47.248 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:52:47.248 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:52:47.248 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.248 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.248 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.249 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.249 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:47.249 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.250 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.250 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.250 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.250 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.250 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.251 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.251 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.251 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.251 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.251 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.252 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:52:47.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:52:47.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.254 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.254 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:52:47.255 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:52:47.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.257 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.257 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:52:47.258 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.259 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:52:47.259 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.259 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.260 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.260 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.260 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.260 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.260 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:52:47.261 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.261 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:52:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:52:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.262 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:52:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:52:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:52:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:52:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.263 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.264 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.264 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.264 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:47.264 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.265 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.265 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.265 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.265 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.265 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.266 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.266 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.266 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.266 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.266 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.267 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.267 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:52:47.268 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:52:47.268 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.269 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.269 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:52:47.270 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:52:47.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.272 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.272 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.273 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:52:47.273 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.274 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:52:47.274 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.275 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.275 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.275 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.276 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:52:47.276 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.276 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:52:47.277 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:52:47.277 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.277 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.278 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:52:47.278 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:52:47.278 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:52:47.278 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:52:47.278 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.279 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.279 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.279 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:47.280 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:52:47.280 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:52:47.280 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:52:47.280 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:52:47.280 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:52:47.281 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:52:47.281 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:52:47.281 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:52:47.281 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 101, 2023-10-26T05:52:47.281 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:52:47.282 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:52:47.282 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:52:47.282 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.282 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.283 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:52:47.283 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.284 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:52:47.284 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.284 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.285 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:52:47.285 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.285 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:52:47.286 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.286 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:52:47.286 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.287 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.287 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:52:47.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.288 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:52:47.288 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.289 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:52:47.289 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.290 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.290 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:52:47.290 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:52:47.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.291 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:52:47.291 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:52:47.291 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:52:47.292 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:52:47.292 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:52:47.292 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.292 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.293 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:52:47.293 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:52:47.293 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:52:47.293 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:52:47.294 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:52:47.294 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:52:47.294 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:52:47.294 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:52:47.294 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.295 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:47.295 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:52:47.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.295 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:52:47.295 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.296 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.296 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.296 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.296 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.296 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:52:47.297 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.297 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:52:47.297 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.298 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.298 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.298 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.298 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.298 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:52:47.298 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.299 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:52:47.299 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.299 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.300 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.300 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.300 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.300 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:52:47.300 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.301 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:52:47.301 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.301 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.302 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.302 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.302 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.302 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:52:47.302 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.303 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:52:47.303 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.303 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.304 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.304 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.304 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.304 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:52:47.304 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.305 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:52:47.305 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.305 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.306 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.306 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.306 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:52:47.306 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.307 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:52:47.307 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.307 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.307 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.308 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.308 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.308 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:52:47.308 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:52:47.309 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:52:47.309 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:52:47.309 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:52:47.309 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:52:47.310 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:52:47.310 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:52:47.310 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:52:47.310 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:52:47.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.311 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:52:47.311 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:52:47.311 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:52:47.311 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:52:47.312 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:52:47.312 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:52:47.312 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:52:47.312 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:52:47.312 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:52:47.313 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:52:47.313 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:52:47.313 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:52:47.313 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:52:47.313 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:52:47.314 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:52:47.314 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:52:47.314 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:52:47.314 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:52:47.315 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:52:47.315 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:52:47.315 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:52:47.315 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:52:47.315 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:52:47.315 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:52:47.316 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:52:47.316 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:52:47.316 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:52:47.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:52:47.316 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:52:47.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88ec066470 msgr2=0x7f88ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88ec066470 0x7f88ec068930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f88f8003a30 tx=0x7f88f8001740 comp rx=0 tx=0).stop 2023-10-26T05:52:47.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 msgr2=0x7f89001f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 0x7f89001f1b70 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f88f400ba60 tx=0x7f88f4005900 comp rx=0 tx=0).stop 2023-10-26T05:52:47.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 shutdown_connections 2023-10-26T05:52:47.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f890013bdc0 0x7f890014a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88ec066470 0x7f88ec068930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f890015b5e0 0x7f89001f1b70 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 --2- 172.21.15.71:0/2421288191 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88e80047d0 0x7f8900157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 >> 172.21.15.71:0/2421288191 conn(0x7f8900069770 msgr2=0x7f89000b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:47.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 shutdown_connections 2023-10-26T05:52:47.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.112+0000 7f8905034640 1 -- 172.21.15.71:0/2421288191 wait complete. 2023-10-26T05:52:47.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- 172.21.15.71:0/788806712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec13c2f0 msgr2=0x7fc0ec0b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/788806712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec13c2f0 0x7fc0ec0b2350 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0cc8d0 tx=0x7fc0e801c5e0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- 172.21.15.71:0/788806712 shutdown_connections 2023-10-26T05:52:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/788806712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0ec0cb010 0x7fc0ec06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/788806712 >> v2:172.21.15.86:3300/0 conn(0x7fc0ec0b2890 0x7fc0ec0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/788806712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec13c2f0 0x7fc0ec0b2350 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- 172.21.15.71:0/788806712 >> 172.21.15.71:0/788806712 conn(0x7fc0ec069770 msgr2=0x7fc0ec069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:47.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- 172.21.15.71:0/788806712 shutdown_connections 2023-10-26T05:52:47.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- 172.21.15.71:0/788806712 wait complete. 2023-10-26T05:52:47.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 Processor -- start 2023-10-26T05:52:47.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- start start 2023-10-26T05:52:47.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ec0b2890 0x7fc0ec153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:47.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 0x7fc0ec0b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:47.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0ec13c2f0 0x7fc0ec155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:47.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0b20e0 con 0x7fc0ec0cb010 2023-10-26T05:52:47.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec1544b0 con 0x7fc0ec0b2890 2023-10-26T05:52:47.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec154790 con 0x7fc0ec13c2f0 2023-10-26T05:52:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 0x7fc0ec0b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 0x7fc0ec0b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41550/0 (socket says 172.21.15.71:41550) 2023-10-26T05:52:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 -- 172.21.15.71:0/2704012933 learned_addr learned my addr 172.21.15.71:0/2704012933 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1f8a640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ec0b2890 0x7fc0ec153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:47.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f278b640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0ec13c2f0 0x7fc0ec155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:47.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 -- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0ec13c2f0 msgr2=0x7fc0ec155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0ec13c2f0 0x7fc0ec155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 -- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ec0b2890 msgr2=0x7fc0ec153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ec0b2890 0x7fc0ec153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 -- 172.21.15.71:0/2704012933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e8009560 con 0x7fc0ec0cb010 2023-10-26T05:52:47.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f1789640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 0x7fc0ec0b1ba0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00c970 tx=0x7fc0dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:47.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc007c70 con 0x7fc0ec0cb010 2023-10-26T05:52:47.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc0dc00f450 con 0x7fc0ec0cb010 2023-10-26T05:52:47.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec154a70 con 0x7fc0ec0cb010 2023-10-26T05:52:47.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.448+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc00f860 con 0x7fc0ec0cb010 2023-10-26T05:52:47.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec155030 con 0x7fc0ec0cb010 2023-10-26T05:52:47.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec066b10 con 0x7fc0ec0cb010 2023-10-26T05:52:47.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc0dc018400 con 0x7fc0ec0cb010 2023-10-26T05:52:47.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0e27fc640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0d0066470 0x7fc0d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:47.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0dc0847b0 con 0x7fc0ec0cb010 2023-10-26T05:52:47.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0f1f8a640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0d0066470 0x7fc0d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:47.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.452+0000 7fc0f1f8a640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0d0066470 0x7fc0d0068930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80ae000 tx=0x7fc0e801ae20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:47.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.456+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc0dc051c30 con 0x7fc0ec0cb010 2023-10-26T05:52:47.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.640+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0ec070ca0 con 0x7fc0ec0cb010 2023-10-26T05:52:47.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0e27fc640 1 -- 172.21.15.71:0/2704012933 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7fc0dc055250 con 0x7fc0ec0cb010 2023-10-26T05:52:47.649 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:47.650 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:47.650 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:47.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0d0066470 msgr2=0x7fc0d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0d0066470 0x7fc0d0068930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80ae000 tx=0x7fc0e801ae20 comp rx=0 tx=0).stop 2023-10-26T05:52:47.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 msgr2=0x7fc0ec0b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:47.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 0x7fc0ec0b1ba0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00c970 tx=0x7fc0dc00ce30 comp rx=0 tx=0).stop 2023-10-26T05:52:47.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 shutdown_connections 2023-10-26T05:52:47.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0ec13c2f0 0x7fc0ec155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0d0066470 0x7fc0d0068930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0ec0cb010 0x7fc0ec0b1ba0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 --2- 172.21.15.71:0/2704012933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0ec0b2890 0x7fc0ec153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:47.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 >> 172.21.15.71:0/2704012933 conn(0x7fc0ec069770 msgr2=0x7fc0ec15bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:47.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 shutdown_connections 2023-10-26T05:52:47.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:47.644+0000 7fc0f41ee640 1 -- 172.21.15.71:0/2704012933 wait complete. 2023-10-26T05:52:47.669 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:52:47.793 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":7074,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":35.431167991000002,"mdsmap_epoch":38,"osdmap_epoch":112,"osdmap_epoch_barrier":106,"uptime":41.064660998000001} 2023-10-26T05:52:47.794 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 7074, "mdsmap_epoch": 38, "osdmap_epoch": 112, "osdmap_epoch_barrier": 106, "rank_uptime": 35.431167991, "state": "up:active", "uptime": 41.064660998, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:52:47.794 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:52:47.795 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:47.795 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:47.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:47.801 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T05:52:48.245 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records in 2023-10-26T05:52:48.246 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records out 2023-10-26T05:52:48.246 INFO:teuthology.orchestra.run.smithi071.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.387436 s, 135 MB/s 2023-10-26T05:52:48.247 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:52:48.248 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:52:48.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:52:48.368 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:52:48.368 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:48.369 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:48.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:48.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/2960104309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 msgr2=0x7f85100caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 --2- 172.21.15.71:0/2960104309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f85100caad0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8504009bc0 tx=0x7f850401cb80 comp rx=0 tx=0).stop 2023-10-26T05:52:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/2960104309 shutdown_connections 2023-10-26T05:52:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 --2- 172.21.15.71:0/2960104309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f85100caad0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 --2- 172.21.15.71:0/2960104309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851013c2f0 0x7f85100b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/2960104309 >> 172.21.15.71:0/2960104309 conn(0x7f8510069770 msgr2=0x7f8510069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:48.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/2960104309 shutdown_connections 2023-10-26T05:52:48.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/2960104309 wait complete. 2023-10-26T05:52:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 Processor -- start 2023-10-26T05:52:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- start start 2023-10-26T05:52:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f851013afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85100cb010 0x7f8510135a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851013c2f0 0x7f851014e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510138bd0 con 0x7f85100b2890 2023-10-26T05:52:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851013b500 con 0x7f85100cb010 2023-10-26T05:52:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510135f40 con 0x7f851013c2f0 2023-10-26T05:52:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f851013afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f851013afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41572/0 (socket says 172.21.15.71:41572) 2023-10-26T05:52:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 -- 172.21.15.71:0/3337998520 learned_addr learned my addr 172.21.15.71:0/3337998520 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851608e640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851013c2f0 0x7f851014e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851508c640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85100cb010 0x7f8510135a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 -- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851013c2f0 msgr2=0x7f851014e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851013c2f0 0x7f851014e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 -- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85100cb010 msgr2=0x7f8510135a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85100cb010 0x7f8510135a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 -- 172.21.15.71:0/3337998520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504009820 con 0x7f85100b2890 2023-10-26T05:52:48.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f851588d640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f851013afc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f850c00d980 tx=0x7f850c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:48.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c009c30 con 0x7f85100b2890 2023-10-26T05:52:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f850c009e80 con 0x7f85100b2890 2023-10-26T05:52:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851014e970 con 0x7f85100b2890 2023-10-26T05:52:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c0106a0 con 0x7f85100b2890 2023-10-26T05:52:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.540+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8510138860 con 0x7f85100b2890 2023-10-26T05:52:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.544+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8510066b10 con 0x7f85100b2890 2023-10-26T05:52:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.544+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f850c010800 con 0x7f85100b2890 2023-10-26T05:52:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.544+0000 7f85027fc640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84f40664a0 0x7f84f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.544+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f850c0843b0 con 0x7f85100b2890 2023-10-26T05:52:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.544+0000 7f851508c640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84f40664a0 0x7f84f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.544+0000 7f851508c640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84f40664a0 0x7f84f4068960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85040af000 tx=0x7f850401b3c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:48.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.548+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f850c051830 con 0x7f85100b2890 2023-10-26T05:52:48.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.692+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8510070ca0 con 0x7f85100b2890 2023-10-26T05:52:48.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.692+0000 7f85027fc640 1 -- 172.21.15.71:0/3337998520 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16488 (secure 0 0 0) 0x7f850c054e50 con 0x7f85100b2890 2023-10-26T05:52:48.700 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:48.700 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":112,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:43.204060+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84f40664a0 msgr2=0x7f84f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84f40664a0 0x7f84f4068960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85040af000 tx=0x7f850401b3c0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 msgr2=0x7f851013afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f851013afc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f850c00d980 tx=0x7f850c00de40 comp rx=0 tx=0).stop 2023-10-26T05:52:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 shutdown_connections 2023-10-26T05:52:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851013c2f0 0x7f851014e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85100cb010 0x7f8510135a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84f40664a0 0x7f84f4068960 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 --2- 172.21.15.71:0/3337998520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85100b2890 0x7f851013afc0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 >> 172.21.15.71:0/3337998520 conn(0x7f8510069770 msgr2=0x7f8510157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 shutdown_connections 2023-10-26T05:52:48.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.696+0000 7f8517af1640 1 -- 172.21.15.71:0/3337998520 wait complete. 2023-10-26T05:52:48.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:52:09.840186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:52:10.672466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:52:48.718 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:52:48.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 -- 172.21.15.71:0/997221952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c13c220 msgr2=0x7fc26c15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 --2- 172.21.15.71:0/997221952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c13c220 0x7fc26c15cce0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009840 tx=0x7fc25c01c740 comp rx=0 tx=0).stop 2023-10-26T05:52:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 -- 172.21.15.71:0/997221952 shutdown_connections 2023-10-26T05:52:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 --2- 172.21.15.71:0/997221952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc26c15d220 0x7fc26c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 --2- 172.21.15.71:0/997221952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c13c220 0x7fc26c15cce0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 -- 172.21.15.71:0/997221952 >> 172.21.15.71:0/997221952 conn(0x7fc26c0aec60 msgr2=0x7fc26c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:48.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 -- 172.21.15.71:0/997221952 shutdown_connections 2023-10-26T05:52:48.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.828+0000 7fc270bef640 1 -- 172.21.15.71:0/997221952 wait complete. 2023-10-26T05:52:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 Processor -- start 2023-10-26T05:52:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- start start 2023-10-26T05:52:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 0x7fc26c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c13c220 0x7fc26c134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc26c15d220 0x7fc26c152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c151550 con 0x7fc26c0cbd60 2023-10-26T05:52:48.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c1f71a0 con 0x7fc26c13c220 2023-10-26T05:52:48.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c1f63e0 con 0x7fc26c15d220 2023-10-26T05:52:48.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 0x7fc26c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 0x7fc26c15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41592/0 (socket says 172.21.15.71:41592) 2023-10-26T05:52:48.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 -- 172.21.15.71:0/3843941266 learned_addr learned my addr 172.21.15.71:0/3843941266 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:48.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc269d9b640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c13c220 0x7fc26c134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 -- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc26c15d220 msgr2=0x7fc26c152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc26c15d220 0x7fc26c152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 -- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c13c220 msgr2=0x7fc26c134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:48.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c13c220 0x7fc26c134110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:48.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 -- 172.21.15.71:0/3843941266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0094a0 con 0x7fc26c0cbd60 2023-10-26T05:52:48.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc26a59c640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 0x7fc26c15bfb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009790 tx=0x7fc25c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:48.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0044f0 con 0x7fc26c0cbd60 2023-10-26T05:52:48.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc25c005030 con 0x7fc26c0cbd60 2023-10-26T05:52:48.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c1f66c0 con 0x7fc26c0cbd60 2023-10-26T05:52:48.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c005420 con 0x7fc26c0cbd60 2023-10-26T05:52:48.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c1f6c80 con 0x7fc26c0cbd60 2023-10-26T05:52:48.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c066b10 con 0x7fc26c0cbd60 2023-10-26T05:52:48.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc25c0b4040 con 0x7fc26c0cbd60 2023-10-26T05:52:48.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc25affd640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc24c066470 0x7fc24c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:48.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.832+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc25c11e310 con 0x7fc26c0cbd60 2023-10-26T05:52:48.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.836+0000 7fc269d9b640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc24c066470 0x7fc24c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:48.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.836+0000 7fc269d9b640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc24c066470 0x7fc24c068930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc26c1f6210 tx=0x7fc260005c30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:48.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.836+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc25c0eb820 con 0x7fc26c0cbd60 2023-10-26T05:52:48.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.992+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc26c1f6e70 con 0x7fc26c0cbd60 2023-10-26T05:52:48.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.992+0000 7fc25affd640 1 -- 172.21.15.71:0/3843941266 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc25c0eee40 con 0x7fc26c0cbd60 2023-10-26T05:52:48.999 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:52:49.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc24c066470 msgr2=0x7fc24c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc24c066470 0x7fc24c068930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc26c1f6210 tx=0x7fc260005c30 comp rx=0 tx=0).stop 2023-10-26T05:52:49.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 msgr2=0x7fc26c15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 0x7fc26c15bfb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009790 tx=0x7fc25c004040 comp rx=0 tx=0).stop 2023-10-26T05:52:49.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 shutdown_connections 2023-10-26T05:52:49.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc26c15d220 0x7fc26c152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc26c13c220 0x7fc26c134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc24c066470 0x7fc24c068930 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 --2- 172.21.15.71:0/3843941266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc26c0cbd60 0x7fc26c15bfb0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 >> 172.21.15.71:0/3843941266 conn(0x7fc26c0aec60 msgr2=0x7fc26c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 shutdown_connections 2023-10-26T05:52:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:48.996+0000 7fc270bef640 1 -- 172.21.15.71:0/3843941266 wait complete. 2023-10-26T05:52:49.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:52:49.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 -- 172.21.15.71:0/793012844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 msgr2=0x7fdd900cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 --2- 172.21.15.71:0/793012844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 0x7fdd900cb820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009840 tx=0x7fdd8c01c7d0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 -- 172.21.15.71:0/793012844 shutdown_connections 2023-10-26T05:52:49.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 --2- 172.21.15.71:0/793012844 >> v2:172.21.15.86:3300/0 conn(0x7fdd900cbd60 0x7fdd9013e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 --2- 172.21.15.71:0/793012844 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 0x7fdd900cb820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 -- 172.21.15.71:0/793012844 >> 172.21.15.71:0/793012844 conn(0x7fdd900aec60 msgr2=0x7fdd900af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 -- 172.21.15.71:0/793012844 shutdown_connections 2023-10-26T05:52:49.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 -- 172.21.15.71:0/793012844 wait complete. 2023-10-26T05:52:49.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 Processor -- start 2023-10-26T05:52:49.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.140+0000 7fdd98669640 1 -- start start 2023-10-26T05:52:49.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 0x7fdd90158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 0x7fdd90138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9013ee00 0x7fdd90137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015a7d0 con 0x7fdd900cbd60 2023-10-26T05:52:49.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90138eb0 con 0x7fdd9013ee00 2023-10-26T05:52:49.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015aa70 con 0x7fdd9013c190 2023-10-26T05:52:49.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 0x7fdd90158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 0x7fdd90158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41594/0 (socket says 172.21.15.71:41594) 2023-10-26T05:52:49.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 -- 172.21.15.71:0/4152469068 learned_addr learned my addr 172.21.15.71:0/4152469068 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:49.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd95c04640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 0x7fdd90138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96c06640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9013ee00 0x7fdd90137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 -- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 msgr2=0x7fdd90138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 0x7fdd90138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 -- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9013ee00 msgr2=0x7fdd90137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9013ee00 0x7fdd90137600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 -- 172.21.15.71:0/4152469068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c0094a0 con 0x7fdd900cbd60 2023-10-26T05:52:49.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd96405640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 0x7fdd90158230 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009790 tx=0x7fdd8c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:49.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c004230 con 0x7fdd900cbd60 2023-10-26T05:52:49.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdd8c004480 con 0x7fdd900cbd60 2023-10-26T05:52:49.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9015ad50 con 0x7fdd900cbd60 2023-10-26T05:52:49.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c005280 con 0x7fdd900cbd60 2023-10-26T05:52:49.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd90139190 con 0x7fdd900cbd60 2023-10-26T05:52:49.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd90066b10 con 0x7fdd900cbd60 2023-10-26T05:52:49.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdd8c0b4040 con 0x7fdd900cbd60 2023-10-26T05:52:49.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd86ffd640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd74066470 0x7fdd74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd8c11db50 con 0x7fdd900cbd60 2023-10-26T05:52:49.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.144+0000 7fdd95c04640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd74066470 0x7fdd74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.148+0000 7fdd95c04640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd74066470 0x7fdd74068930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdd900acf90 tx=0x7fdd800091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:49.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.148+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdd8c0eafd0 con 0x7fdd900cbd60 2023-10-26T05:52:49.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.304+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd900721e0 con 0x7fdd900cbd60 2023-10-26T05:52:49.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.304+0000 7fdd86ffd640 1 -- 172.21.15.71:0/4152469068 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd8c0ab340 con 0x7fdd900cbd60 2023-10-26T05:52:49.311 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:52:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.304+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd74066470 msgr2=0x7fdd74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.304+0000 7fdd98669640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd74066470 0x7fdd74068930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdd900acf90 tx=0x7fdd800091e0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.304+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 msgr2=0x7fdd90158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.304+0000 7fdd98669640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 0x7fdd90158230 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009790 tx=0x7fdd8c0a56b0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 shutdown_connections 2023-10-26T05:52:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd9013ee00 0x7fdd90137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd9013c190 0x7fdd90138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdd74066470 0x7fdd74068930 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 --2- 172.21.15.71:0/4152469068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd900cbd60 0x7fdd90158230 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 >> 172.21.15.71:0/4152469068 conn(0x7fdd900aec60 msgr2=0x7fdd9013c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 shutdown_connections 2023-10-26T05:52:49.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.308+0000 7fdd98669640 1 -- 172.21.15.71:0/4152469068 wait complete. 2023-10-26T05:52:49.330 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:52:49.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3351619558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb040047d0 msgr2=0x7fdb04004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3351619558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb040047d0 0x7fdb04004c50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800bb10 tx=0x7fdb0801cd80 comp rx=0 tx=0).stop 2023-10-26T05:52:49.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1ca88640 1 -- 172.21.15.71:0/3351619558 reap_dead start 2023-10-26T05:52:49.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3351619558 shutdown_connections 2023-10-26T05:52:49.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3351619558 >> 172.21.15.71:0/3351619558 conn(0x7fdb180aec60 msgr2=0x7fdb180af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3351619558 shutdown_connections 2023-10-26T05:52:49.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3351619558 wait complete. 2023-10-26T05:52:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 Processor -- start 2023-10-26T05:52:49.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.448+0000 7fdb1e4eb640 1 -- start start 2023-10-26T05:52:49.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb040047d0 0x7fdb181587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb180c7650 0x7fdb181e4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 0x7fdb181e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18159bf0 con 0x7fdb181e4790 2023-10-26T05:52:49.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb181e5150 con 0x7fdb180c7650 2023-10-26T05:52:49.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18158fe0 con 0x7fdb040047d0 2023-10-26T05:52:49.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 0x7fdb181e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 0x7fdb181e4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41620/0 (socket says 172.21.15.71:41620) 2023-10-26T05:52:49.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 -- 172.21.15.71:0/3572736389 learned_addr learned my addr 172.21.15.71:0/3572736389 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb0ffff640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb180c7650 0x7fdb181e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb17fff640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb040047d0 0x7fdb181587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 -- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb040047d0 msgr2=0x7fdb181587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb040047d0 0x7fdb181587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 -- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb180c7650 msgr2=0x7fdb181e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb180c7650 0x7fdb181e4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 -- 172.21.15.71:0/3572736389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb181592c0 con 0x7fdb181e4790 2023-10-26T05:52:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb0ffff640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb180c7650 0x7fdb181e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb17fff640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb040047d0 0x7fdb181587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1ca88640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 0x7fdb181e4c10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ca250 tx=0x7fdb10009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:49.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10011710 con 0x7fdb181e4790 2023-10-26T05:52:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdb10011960 con 0x7fdb181e4790 2023-10-26T05:52:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb181595a0 con 0x7fdb181e4790 2023-10-26T05:52:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb10011d70 con 0x7fdb181e4790 2023-10-26T05:52:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb18158ce0 con 0x7fdb181e4790 2023-10-26T05:52:49.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb18066b10 con 0x7fdb181e4790 2023-10-26T05:52:49.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdb1000f440 con 0x7fdb181e4790 2023-10-26T05:52:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb157fa640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdaf8066470 0x7fdaf8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb10084490 con 0x7fdb181e4790 2023-10-26T05:52:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.452+0000 7fdb17fff640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdaf8066470 0x7fdaf8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.456+0000 7fdb17fff640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdaf8066470 0x7fdaf8068930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb00001830 tx=0x7fdb00001120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:49.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.456+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdb100519a0 con 0x7fdb181e4790 2023-10-26T05:52:49.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.608+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb180721e0 con 0x7fdb181e4790 2023-10-26T05:52:49.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb157fa640 1 -- 172.21.15.71:0/3572736389 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb10054fc0 con 0x7fdb181e4790 2023-10-26T05:52:49.618 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:52:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdaf8066470 msgr2=0x7fdaf8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdaf8066470 0x7fdaf8068930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb00001830 tx=0x7fdb00001120 comp rx=0 tx=0).stop 2023-10-26T05:52:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 msgr2=0x7fdb181e4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 0x7fdb181e4c10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fdb180ca250 tx=0x7fdb10009b00 comp rx=0 tx=0).stop 2023-10-26T05:52:49.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 shutdown_connections 2023-10-26T05:52:49.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdaf8066470 0x7fdaf8068930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb181e4790 0x7fdb181e4c10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb180c7650 0x7fdb181e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 --2- 172.21.15.71:0/3572736389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdb040047d0 0x7fdb181587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 >> 172.21.15.71:0/3572736389 conn(0x7fdb180aec60 msgr2=0x7fdb1813f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 shutdown_connections 2023-10-26T05:52:49.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.612+0000 7fdb1e4eb640 1 -- 172.21.15.71:0/3572736389 wait complete. 2023-10-26T05:52:49.636 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:52:49.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 -- 172.21.15.71:0/3896682738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3d8005190 msgr2=0x7ff3d800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3896682738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3d8005190 0x7ff3d800b990 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b30 tx=0x7ff3dc01d250 comp rx=0 tx=0).stop 2023-10-26T05:52:49.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 -- 172.21.15.71:0/3896682738 shutdown_connections 2023-10-26T05:52:49.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3896682738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3d8005190 0x7ff3d800b990 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 -- 172.21.15.71:0/3896682738 >> 172.21.15.71:0/3896682738 conn(0x7ff3ec06fd90 msgr2=0x7ff3ec0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 -- 172.21.15.71:0/3896682738 shutdown_connections 2023-10-26T05:52:49.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 -- 172.21.15.71:0/3896682738 wait complete. 2023-10-26T05:52:49.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 Processor -- start 2023-10-26T05:52:49.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.752+0000 7ff3f151b640 1 -- start start 2023-10-26T05:52:49.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3d8005190 0x7ff3ec14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ec0c6190 0x7ff3ec0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 0x7ff3ec1e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec14fc20 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c6b50 con 0x7ff3d8005190 2023-10-26T05:52:49.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec1e3450 con 0x7ff3ec0c6190 2023-10-26T05:52:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 0x7ff3ec1e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 0x7ff3ec1e1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41638/0 (socket says 172.21.15.71:41638) 2023-10-26T05:52:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 -- 172.21.15.71:0/3860415143 learned_addr learned my addr 172.21.15.71:0/3860415143 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eaffd640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3d8005190 0x7ff3ec14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3ea7fc640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ec0c6190 0x7ff3ec0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 -- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ec0c6190 msgr2=0x7ff3ec0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ec0c6190 0x7ff3ec0c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 -- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3d8005190 msgr2=0x7ff3ec14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3d8005190 0x7ff3ec14f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 -- 172.21.15.71:0/3860415143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec1e2840 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eb7fe640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 0x7ff3ec1e1f90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b30 tx=0x7ff3dc005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0046d0 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3dc004920 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec1e2b20 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc004cd0 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec1e30e0 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec06a7b0 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff3dc0b7470 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3e37fe640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3cc066470 0x7ff3cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3dc11f290 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.756+0000 7ff3eaffd640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3cc066470 0x7ff3cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:49.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.760+0000 7ff3eaffd640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3cc066470 0x7ff3cc068930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4006fd0 tx=0x7ff3e40068b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:49.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.760+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff3dc0ec710 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff3ec04db60 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3e37fe640 1 -- 172.21.15.71:0/3860415143 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff3dc0efd30 con 0x7ff3ec1e1b10 2023-10-26T05:52:49.931 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:52:49.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3cc066470 msgr2=0x7ff3cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3cc066470 0x7ff3cc068930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4006fd0 tx=0x7ff3e40068b0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 msgr2=0x7ff3ec1e1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:49.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 0x7ff3ec1e1f90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009b30 tx=0x7ff3dc005680 comp rx=0 tx=0).stop 2023-10-26T05:52:49.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 shutdown_connections 2023-10-26T05:52:49.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3cc066470 0x7ff3cc068930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3ec1e1b10 0x7ff3ec1e1f90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3ec0c6190 0x7ff3ec0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 --2- 172.21.15.71:0/3860415143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3d8005190 0x7ff3ec14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:49.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 >> 172.21.15.71:0/3860415143 conn(0x7ff3ec06fd90 msgr2=0x7ff3ec0af940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:49.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 shutdown_connections 2023-10-26T05:52:49.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49.924+0000 7ff3f151b640 1 -- 172.21.15.71:0/3860415143 wait complete. 2023-10-26T05:52:49.948 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:52:49.948 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:49.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:52:49.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:49.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[45599]: fuse finished with error 0 and tester_r 0 2023-10-26T05:52:50.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:50.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:52:50.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:50.012 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:52:50.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:50.045 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:52:50.046 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:50.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:50.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- 172.21.15.71:0/4132617242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b00aaff0 msgr2=0x7f11b00b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 --2- 172.21.15.71:0/4132617242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b00aaff0 0x7f11b00b1e00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009bc0 tx=0x7f11a801cb50 comp rx=0 tx=0).stop 2023-10-26T05:52:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- 172.21.15.71:0/4132617242 shutdown_connections 2023-10-26T05:52:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 --2- 172.21.15.71:0/4132617242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b00aaff0 0x7f11b00b1e00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 --2- 172.21.15.71:0/4132617242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11b015b2e0 0x7f11b006c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- 172.21.15.71:0/4132617242 >> 172.21.15.71:0/4132617242 conn(0x7f11b0070cf0 msgr2=0x7f11b006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:50.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- 172.21.15.71:0/4132617242 shutdown_connections 2023-10-26T05:52:50.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- 172.21.15.71:0/4132617242 wait complete. 2023-10-26T05:52:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 Processor -- start 2023-10-26T05:52:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- start start 2023-10-26T05:52:50.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b00aaff0 0x7f11b014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11b00ac2e0 0x7f11b014a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 0x7f11b0146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b014b6c0 con 0x7f11b015b2e0 2023-10-26T05:52:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b00ae910 con 0x7f11b00aaff0 2023-10-26T05:52:50.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b8412640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b014aab0 con 0x7f11b00ac2e0 2023-10-26T05:52:50.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 0x7f11b0146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 0x7f11b0146190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41658/0 (socket says 172.21.15.71:41658) 2023-10-26T05:52:50.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 -- 172.21.15.71:0/4099613141 learned_addr learned my addr 172.21.15.71:0/4099613141 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:50.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b59ad640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11b00ac2e0 0x7f11b014a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b61ae640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b00aaff0 0x7f11b014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 -- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11b00ac2e0 msgr2=0x7f11b014a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11b00ac2e0 0x7f11b014a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 -- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b00aaff0 msgr2=0x7f11b014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b00aaff0 0x7f11b014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 -- 172.21.15.71:0/4099613141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8009820 con 0x7f11b015b2e0 2023-10-26T05:52:50.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b61ae640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b00aaff0 0x7f11b014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:50.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11b69af640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 0x7f11b0146190 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f11a801a780 tx=0x7f11a80a5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:50.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.196+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8004200 con 0x7f11b015b2e0 2023-10-26T05:52:50.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11a8004450 con 0x7f11b015b2e0 2023-10-26T05:52:50.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b014ad90 con 0x7f11b015b2e0 2023-10-26T05:52:50.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a80047e0 con 0x7f11b015b2e0 2023-10-26T05:52:50.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b014b350 con 0x7f11b015b2e0 2023-10-26T05:52:50.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b0066d00 con 0x7f11b015b2e0 2023-10-26T05:52:50.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11a80b4040 con 0x7f11b015b2e0 2023-10-26T05:52:50.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11a6ffd640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1194066470 0x7f1194068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11a811ecc0 con 0x7f11b015b2e0 2023-10-26T05:52:50.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11b61ae640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1194066470 0x7f1194068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.200+0000 7f11b61ae640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1194066470 0x7f1194068930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0058d0 tx=0x7f11ac005ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:50.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.204+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11a80ec1d0 con 0x7f11b015b2e0 2023-10-26T05:52:50.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.348+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11b014b800 con 0x7f11b015b2e0 2023-10-26T05:52:50.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.348+0000 7f11a6ffd640 1 -- 172.21.15.71:0/4099613141 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16488 (secure 0 0 0) 0x7f11b014b800 con 0x7f11b015b2e0 2023-10-26T05:52:50.357 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:50.357 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":112,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:43.204060+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:50.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1194066470 msgr2=0x7f1194068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1194066470 0x7f1194068930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0058d0 tx=0x7f11ac005ca0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 msgr2=0x7f11b0146190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 0x7f11b0146190 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f11a801a780 tx=0x7f11a80a5b80 comp rx=0 tx=0).stop 2023-10-26T05:52:50.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 shutdown_connections 2023-10-26T05:52:50.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1194066470 0x7f1194068930 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11b015b2e0 0x7f11b0146190 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11b00ac2e0 0x7f11b014a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 --2- 172.21.15.71:0/4099613141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11b00aaff0 0x7f11b014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 >> 172.21.15.71:0/4099613141 conn(0x7f11b0070cf0 msgr2=0x7f11b00aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:50.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 shutdown_connections 2023-10-26T05:52:50.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.352+0000 7f11b8412640 1 -- 172.21.15.71:0/4099613141 wait complete. 2023-10-26T05:52:50.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:52:09.840186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:52:10.672466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:52:50.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:52:50.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/1061867520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc15a3f0 msgr2=0x7f13bc13e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 --2- 172.21.15.71:0/1061867520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc15a3f0 0x7f13bc13e850 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f13bc06b260 tx=0x7f13b001c870 comp rx=0 tx=0).stop 2023-10-26T05:52:50.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/1061867520 shutdown_connections 2023-10-26T05:52:50.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 --2- 172.21.15.71:0/1061867520 >> v2:172.21.15.86:3300/0 conn(0x7f13bc13ed90 0x7f13bc069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 --2- 172.21.15.71:0/1061867520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc15a3f0 0x7f13bc13e850 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/1061867520 >> 172.21.15.71:0/1061867520 conn(0x7f13bc0aec60 msgr2=0x7f13bc0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:50.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/1061867520 shutdown_connections 2023-10-26T05:52:50.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/1061867520 wait complete. 2023-10-26T05:52:50.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 Processor -- start 2023-10-26T05:52:50.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- start start 2023-10-26T05:52:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc13c120 0x7f13bc1343f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 0x7f13bc13c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13bc15a3f0 0x7f13bc13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc13e1f0 con 0x7f13bc13ed90 2023-10-26T05:52:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc135a20 con 0x7f13bc13c120 2023-10-26T05:52:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc135d00 con 0x7f13bc15a3f0 2023-10-26T05:52:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 0x7f13bc13c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 0x7f13bc13c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41678/0 (socket says 172.21.15.71:41678) 2023-10-26T05:52:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 -- 172.21.15.71:0/3817351883 learned_addr learned my addr 172.21.15.71:0/3817351883 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c0fe1640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc13c120 0x7f13bc1343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c17e2640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13bc15a3f0 0x7f13bc13cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 -- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13bc15a3f0 msgr2=0x7f13bc13cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13bc15a3f0 0x7f13bc13cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 -- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc13c120 msgr2=0x7f13bc1343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc13c120 0x7f13bc1343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 -- 172.21.15.71:0/3817351883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b0009560 con 0x7f13bc13ed90 2023-10-26T05:52:50.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13bbfff640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 0x7f13bc13c6c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f13b00b0040 tx=0x7f13b0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:50.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0005220 con 0x7f13bc13ed90 2023-10-26T05:52:50.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f13b0005470 con 0x7f13bc13ed90 2023-10-26T05:52:50.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13bc13e490 con 0x7f13bc13ed90 2023-10-26T05:52:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0004930 con 0x7f13bc13ed90 2023-10-26T05:52:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.500+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13bc13d410 con 0x7f13bc13ed90 2023-10-26T05:52:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.504+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13bc066b10 con 0x7f13bc13ed90 2023-10-26T05:52:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.504+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f13b00b4070 con 0x7f13bc13ed90 2023-10-26T05:52:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.504+0000 7f13b97fa640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13a8066470 0x7f13a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.504+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13b011dfe0 con 0x7f13bc13ed90 2023-10-26T05:52:50.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.504+0000 7f13c0fe1640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13a8066470 0x7f13a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.504+0000 7f13c0fe1640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13a8066470 0x7f13a8068930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13ac007fd0 tx=0x7f13ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:50.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.508+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f13b00eb560 con 0x7f13bc13ed90 2023-10-26T05:52:50.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.664+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13bc135f20 con 0x7f13bc13ed90 2023-10-26T05:52:50.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.664+0000 7f13b97fa640 1 -- 172.21.15.71:0/3817351883 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13b00eeb80 con 0x7f13bc13ed90 2023-10-26T05:52:50.672 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:52:50.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13a8066470 msgr2=0x7f13a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13a8066470 0x7f13a8068930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13ac007fd0 tx=0x7f13ac009040 comp rx=0 tx=0).stop 2023-10-26T05:52:50.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 msgr2=0x7f13bc13c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 0x7f13bc13c6c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f13b00b0040 tx=0x7f13b0004600 comp rx=0 tx=0).stop 2023-10-26T05:52:50.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 shutdown_connections 2023-10-26T05:52:50.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f13bc15a3f0 0x7f13bc13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13a8066470 0x7f13a8068930 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f13bc13ed90 0x7f13bc13c6c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 --2- 172.21.15.71:0/3817351883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc13c120 0x7f13bc1343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 >> 172.21.15.71:0/3817351883 conn(0x7f13bc0aec60 msgr2=0x7f13bc0abb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:50.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 shutdown_connections 2023-10-26T05:52:50.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.668+0000 7f13c3245640 1 -- 172.21.15.71:0/3817351883 wait complete. 2023-10-26T05:52:50.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:52:50.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 -- 172.21.15.71:0/152276226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85180cede0 msgr2=0x7f851815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 --2- 172.21.15.71:0/152276226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85180cede0 0x7f851815ce50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f850c009900 tx=0x7f850c01c7c0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 -- 172.21.15.71:0/152276226 shutdown_connections 2023-10-26T05:52:50.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 --2- 172.21.15.71:0/152276226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85180cede0 0x7f851815ce50 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 --2- 172.21.15.71:0/152276226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f851815c720 0x7f85180b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 -- 172.21.15.71:0/152276226 >> 172.21.15.71:0/152276226 conn(0x7f85180b0f20 msgr2=0x7f85180b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:50.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 -- 172.21.15.71:0/152276226 shutdown_connections 2023-10-26T05:52:50.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.812+0000 7f851ea01640 1 -- 172.21.15.71:0/152276226 wait complete. 2023-10-26T05:52:50.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 Processor -- start 2023-10-26T05:52:50.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- start start 2023-10-26T05:52:50.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85180cede0 0x7f8518142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851815c720 0x7f85181442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 0x7f851806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518143a30 con 0x7f851815d390 2023-10-26T05:52:50.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518141200 con 0x7f851815c720 2023-10-26T05:52:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85180c68c0 con 0x7f85180cede0 2023-10-26T05:52:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 0x7f851806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 0x7f851806e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41682/0 (socket says 172.21.15.71:41682) 2023-10-26T05:52:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 -- 172.21.15.71:0/2118502303 learned_addr learned my addr 172.21.15.71:0/2118502303 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:50.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f85177fe640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851815c720 0x7f85181442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f8517fff640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85180cede0 0x7f8518142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 -- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85180cede0 msgr2=0x7f8518142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85180cede0 0x7f8518142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 -- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851815c720 msgr2=0x7f85181442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851815c720 0x7f85181442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 -- 172.21.15.71:0/2118502303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009560 con 0x7f851815d390 2023-10-26T05:52:50.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851cf9e640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 0x7f851806e6f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f851000b990 tx=0x7f851000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:50.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851000cba0 con 0x7f851815d390 2023-10-26T05:52:50.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f851000cdf0 con 0x7f851815d390 2023-10-26T05:52:50.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85180c6ba0 con 0x7f851815d390 2023-10-26T05:52:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85100146a0 con 0x7f851815d390 2023-10-26T05:52:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8518141450 con 0x7f851815d390 2023-10-26T05:52:50.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.816+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85180c70c0 con 0x7f851815d390 2023-10-26T05:52:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.820+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f851001e440 con 0x7f851815d390 2023-10-26T05:52:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.820+0000 7f8514ff9640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85040664a0 0x7f8504068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:50.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.820+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85100844c0 con 0x7f851815d390 2023-10-26T05:52:50.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.820+0000 7f8517fff640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85040664a0 0x7f8504068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:50.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.820+0000 7f8517fff640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85040664a0 0x7f8504068960 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f850800a830 tx=0x7f8508008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:50.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.824+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8510051990 con 0x7f851815d390 2023-10-26T05:52:50.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.968+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8518141640 con 0x7f851815d390 2023-10-26T05:52:50.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f8514ff9640 1 -- 172.21.15.71:0/2118502303 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85100190a0 con 0x7f851815d390 2023-10-26T05:52:50.977 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:52:50.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85040664a0 msgr2=0x7f8504068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85040664a0 0x7f8504068960 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f850800a830 tx=0x7f8508008040 comp rx=0 tx=0).stop 2023-10-26T05:52:50.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 msgr2=0x7f851806e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:50.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 0x7f851806e6f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f851000b990 tx=0x7f851000be50 comp rx=0 tx=0).stop 2023-10-26T05:52:50.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 shutdown_connections 2023-10-26T05:52:50.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85040664a0 0x7f8504068960 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f851815d390 0x7f851806e6f0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851815c720 0x7f85181442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 --2- 172.21.15.71:0/2118502303 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85180cede0 0x7f8518142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:50.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 >> 172.21.15.71:0/2118502303 conn(0x7f85180b0f20 msgr2=0x7f85180655c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:50.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 shutdown_connections 2023-10-26T05:52:50.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:50.972+0000 7f851ea01640 1 -- 172.21.15.71:0/2118502303 wait complete. 2023-10-26T05:52:50.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:52:51.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1751585738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c13c220 msgr2=0x7fb79c15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1751585738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c13c220 0x7fb79c15cce0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009840 tx=0x7fb78c01c700 comp rx=0 tx=0).stop 2023-10-26T05:52:51.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1751585738 shutdown_connections 2023-10-26T05:52:51.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1751585738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c13c220 0x7fb79c15cce0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1751585738 >> 172.21.15.71:0/1751585738 conn(0x7fb79c0aec60 msgr2=0x7fb79c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:51.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1751585738 shutdown_connections 2023-10-26T05:52:51.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1751585738 wait complete. 2023-10-26T05:52:51.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 Processor -- start 2023-10-26T05:52:51.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- start start 2023-10-26T05:52:51.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 0x7fb79c149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c13c220 0x7fb79c150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c15d220 0x7fb79c137490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c1506b0 con 0x7fb79c0cbf30 2023-10-26T05:52:51.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c14a8d0 con 0x7fb79c15d220 2023-10-26T05:52:51.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c137ed0 con 0x7fb79c13c220 2023-10-26T05:52:51.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 0x7fb79c149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 0x7fb79c149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41706/0 (socket says 172.21.15.71:41706) 2023-10-26T05:52:51.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 -- 172.21.15.71:0/1856233388 learned_addr learned my addr 172.21.15.71:0/1856233388 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:51.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 -- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c13c220 msgr2=0x7fb79c150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c13c220 0x7fb79c150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 -- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c15d220 msgr2=0x7fb79c137490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c15d220 0x7fb79c137490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 -- 172.21.15.71:0/1856233388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c0094a0 con 0x7fb79c0cbf30 2023-10-26T05:52:51.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb79affd640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 0x7fb79c149690 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb78c01cbe0 tx=0x7fb78c0a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:51.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0053c0 con 0x7fb79c0cbf30 2023-10-26T05:52:51.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb78c005610 con 0x7fb79c0cbf30 2023-10-26T05:52:51.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c1379d0 con 0x7fb79c0cbf30 2023-10-26T05:52:51.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c004930 con 0x7fb79c0cbf30 2023-10-26T05:52:51.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.112+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c149fa0 con 0x7fb79c0cbf30 2023-10-26T05:52:51.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.116+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c066b10 con 0x7fb79c0cbf30 2023-10-26T05:52:51.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.116+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb78c004a90 con 0x7fb79c0cbf30 2023-10-26T05:52:51.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.116+0000 7fb783fff640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7880664a0 0x7fb788068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.116+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb78c11ef80 con 0x7fb79c0cbf30 2023-10-26T05:52:51.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.116+0000 7fb79a7fc640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7880664a0 0x7fb788068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.116+0000 7fb79a7fc640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7880664a0 0x7fb788068960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb790000f70 tx=0x7fb7900079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:51.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.120+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb78c0ec520 con 0x7fb79c0cbf30 2023-10-26T05:52:51.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.272+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb79c0721e0 con 0x7fb79c0cbf30 2023-10-26T05:52:51.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.272+0000 7fb783fff640 1 -- 172.21.15.71:0/1856233388 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb78c0bb020 con 0x7fb79c0cbf30 2023-10-26T05:52:51.280 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:52:51.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7880664a0 msgr2=0x7fb788068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7880664a0 0x7fb788068960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb790000f70 tx=0x7fb7900079c0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 msgr2=0x7fb79c149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 0x7fb79c149690 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fb78c01cbe0 tx=0x7fb78c0a5880 comp rx=0 tx=0).stop 2023-10-26T05:52:51.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 shutdown_connections 2023-10-26T05:52:51.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb79c15d220 0x7fb79c137490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb79c13c220 0x7fb79c150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7880664a0 0x7fb788068960 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 --2- 172.21.15.71:0/1856233388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb79c0cbf30 0x7fb79c149690 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 >> 172.21.15.71:0/1856233388 conn(0x7fb79c0aec60 msgr2=0x7fb79c15bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:51.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 shutdown_connections 2023-10-26T05:52:51.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.276+0000 7fb7a187a640 1 -- 172.21.15.71:0/1856233388 wait complete. 2023-10-26T05:52:51.297 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:52:51.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd53a223640 1 -- 172.21.15.71:0/1406461729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5200047d0 msgr2=0x7fd520004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd53a223640 1 --2- 172.21.15.71:0/1406461729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5200047d0 0x7fd520004c50 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd5340cc920 tx=0x7fd51c01cda0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd533fff640 1 -- 172.21.15.71:0/1406461729 reap_dead start 2023-10-26T05:52:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd53a223640 1 -- 172.21.15.71:0/1406461729 shutdown_connections 2023-10-26T05:52:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd53a223640 1 -- 172.21.15.71:0/1406461729 >> 172.21.15.71:0/1406461729 conn(0x7fd534069d70 msgr2=0x7fd5340b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:51.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd53a223640 1 -- 172.21.15.71:0/1406461729 shutdown_connections 2023-10-26T05:52:51.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.416+0000 7fd53a223640 1 -- 172.21.15.71:0/1406461729 wait complete. 2023-10-26T05:52:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 Processor -- start 2023-10-26T05:52:51.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- start start 2023-10-26T05:52:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200047d0 0x7fd5340c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 0x7fd5340c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5340b23e0 0x7fd5340c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340c87c0 con 0x7fd5340b1f60 2023-10-26T05:52:51.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340c7bd0 con 0x7fd5200047d0 2023-10-26T05:52:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340c7e20 con 0x7fd5340b23e0 2023-10-26T05:52:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 0x7fd5340c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 0x7fd5340c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41720/0 (socket says 172.21.15.71:41720) 2023-10-26T05:52:51.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 -- 172.21.15.71:0/778240719 learned_addr learned my addr 172.21.15.71:0/778240719 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:51.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd5337fe640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200047d0 0x7fd5340c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd533fff640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5340b23e0 0x7fd5340c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 -- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5340b23e0 msgr2=0x7fd5340c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5340b23e0 0x7fd5340c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 -- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200047d0 msgr2=0x7fd5340c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200047d0 0x7fd5340c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 -- 172.21.15.71:0/778240719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340c8100 con 0x7fd5340b1f60 2023-10-26T05:52:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd52affd640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 0x7fd5340c4880 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0b1040 tx=0x7fd51c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0be070 con 0x7fd5340b1f60 2023-10-26T05:52:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd51c0052f0 con 0x7fd5340b1f60 2023-10-26T05:52:51.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5340c83e0 con 0x7fd5340b1f60 2023-10-26T05:52:51.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c004020 con 0x7fd5340b1f60 2023-10-26T05:52:51.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd534140d80 con 0x7fd5340b1f60 2023-10-26T05:52:51.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5340c8a30 con 0x7fd5340b1f60 2023-10-26T05:52:51.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.420+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd51c004180 con 0x7fd5340b1f60 2023-10-26T05:52:51.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.424+0000 7fd530ff9640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd514066470 0x7fd514068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.424+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd51c11e7f0 con 0x7fd5340b1f60 2023-10-26T05:52:51.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.424+0000 7fd5337fe640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd514066470 0x7fd514068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.424+0000 7fd5337fe640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd514066470 0x7fd514068930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd5240018f0 tx=0x7fd5240014c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:51.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.428+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd51c0ebc80 con 0x7fd5340b1f60 2023-10-26T05:52:51.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.592+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd534066ae0 con 0x7fd5340b1f60 2023-10-26T05:52:51.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.592+0000 7fd530ff9640 1 -- 172.21.15.71:0/778240719 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd51c0ab080 con 0x7fd5340b1f60 2023-10-26T05:52:51.600 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:52:51.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd514066470 msgr2=0x7fd514068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd514066470 0x7fd514068930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd5240018f0 tx=0x7fd5240014c0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 msgr2=0x7fd5340c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 0x7fd5340c4880 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0b1040 tx=0x7fd51c005880 comp rx=0 tx=0).stop 2023-10-26T05:52:51.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 shutdown_connections 2023-10-26T05:52:51.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5340b23e0 0x7fd5340c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd514066470 0x7fd514068930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5340b1f60 0x7fd5340c4880 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 --2- 172.21.15.71:0/778240719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200047d0 0x7fd5340c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 >> 172.21.15.71:0/778240719 conn(0x7fd534069d70 msgr2=0x7fd53415c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:51.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 shutdown_connections 2023-10-26T05:52:51.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.596+0000 7fd53a223640 1 -- 172.21.15.71:0/778240719 wait complete. 2023-10-26T05:52:51.619 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:52:51.619 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:52:51.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:52:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:52:51.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34740]: fuse finished with error 0 and tester_r 0 2023-10-26T05:52:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:51.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:52:51.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:52:51.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:51.685 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:52:51.691 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:52:51.691 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:51.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:52:51.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:52:51.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:51.698 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:52:51.745 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:52:51.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:52:51.747 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:52:51.747 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:52:51.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:52:51.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:52:51.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:52:51.753 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:52:51.798 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:52:51.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:52:51.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:51.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- 172.21.15.71:0/3621461687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 msgr2=0x7f671c15ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 --2- 172.21.15.71:0/3621461687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c15ce50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f671c071290 tx=0x7f671001c7e0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- 172.21.15.71:0/3621461687 shutdown_connections 2023-10-26T05:52:51.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 --2- 172.21.15.71:0/3621461687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c15ce50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 --2- 172.21.15.71:0/3621461687 >> v2:172.21.15.86:3300/0 conn(0x7f671c15c720 0x7f671c0b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- 172.21.15.71:0/3621461687 >> 172.21.15.71:0/3621461687 conn(0x7f671c0b0f20 msgr2=0x7f671c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- 172.21.15.71:0/3621461687 shutdown_connections 2023-10-26T05:52:51.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- 172.21.15.71:0/3621461687 wait complete. 2023-10-26T05:52:51.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 Processor -- start 2023-10-26T05:52:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- start start 2023-10-26T05:52:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c15c720 0x7f671c152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f671c15d390 0x7f671c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0c6f00 con 0x7f671c0cede0 2023-10-26T05:52:51.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0c71e0 con 0x7f671c15c720 2023-10-26T05:52:51.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6720cbf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671c0c74c0 con 0x7f671c15d390 2023-10-26T05:52:51.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c1579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44614/0 (socket says 172.21.15.71:44614) 2023-10-26T05:52:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 -- 172.21.15.71:0/1670229027 learned_addr learned my addr 172.21.15.71:0/1670229027 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f6719d9b640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c15c720 0x7f671c152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671ad9d640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f671c15d390 0x7f671c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 -- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f671c15d390 msgr2=0x7f671c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f671c15d390 0x7f671c152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 -- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c15c720 msgr2=0x7f671c152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c15c720 0x7f671c152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671a59c640 1 -- 172.21.15.71:0/1670229027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009560 con 0x7f671c0cede0 2023-10-26T05:52:51.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.924+0000 7f671ad9d640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f671c15d390 0x7f671c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f671a59c640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c1579e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f670c00e9f0 tx=0x7f670c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c009710 con 0x7f671c0cede0 2023-10-26T05:52:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f670c00ce70 con 0x7f671c0cede0 2023-10-26T05:52:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c010690 con 0x7f671c0cede0 2023-10-26T05:52:51.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c0c77a0 con 0x7f671c0cede0 2023-10-26T05:52:51.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c06ec50 con 0x7f671c0cede0 2023-10-26T05:52:51.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c066d00 con 0x7f671c0cede0 2023-10-26T05:52:51.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f670c0107f0 con 0x7f671c0cede0 2023-10-26T05:52:51.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f670affd640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f66fc066470 0x7f66fc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:51.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f670c014070 con 0x7f671c0cede0 2023-10-26T05:52:51.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f6719d9b640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f66fc066470 0x7f66fc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:51.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.928+0000 7f6719d9b640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f66fc066470 0x7f66fc068930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67100af000 tx=0x7f671001b020 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:51.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:51.932+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f670c050090 con 0x7f671c0cede0 2023-10-26T05:52:52.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671c0b0880 con 0x7f671c0cede0 2023-10-26T05:52:52.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f670affd640 1 -- 172.21.15.71:0/1670229027 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f670c055a70 con 0x7f671c0cede0 2023-10-26T05:52:52.115 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:52.115 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:52.115 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:52.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f66fc066470 msgr2=0x7f66fc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f66fc066470 0x7f66fc068930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67100af000 tx=0x7f671001b020 comp rx=0 tx=0).stop 2023-10-26T05:52:52.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 msgr2=0x7f671c1579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c1579e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f670c00e9f0 tx=0x7f670c00eeb0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 shutdown_connections 2023-10-26T05:52:52.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f671c15d390 0x7f671c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f671c15c720 0x7f671c152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f66fc066470 0x7f66fc068930 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 --2- 172.21.15.71:0/1670229027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f671c0cede0 0x7f671c1579e0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 >> 172.21.15.71:0/1670229027 conn(0x7f671c0b0f20 msgr2=0x7f671c158500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:52.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 shutdown_connections 2023-10-26T05:52:52.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.108+0000 7f6720cbf640 1 -- 172.21.15.71:0/1670229027 wait complete. 2023-10-26T05:52:52.132 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 -- 172.21.15.71:0/2867041031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa99013c2f0 msgr2=0x7fa9900b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 --2- 172.21.15.71:0/2867041031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa99013c2f0 0x7fa9900b2350 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa980009900 tx=0x7fa98001c8c0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 -- 172.21.15.71:0/2867041031 shutdown_connections 2023-10-26T05:52:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 --2- 172.21.15.71:0/2867041031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9900b2890 0x7fa9900caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 --2- 172.21.15.71:0/2867041031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa99013c2f0 0x7fa9900b2350 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 -- 172.21.15.71:0/2867041031 >> 172.21.15.71:0/2867041031 conn(0x7fa990069770 msgr2=0x7fa990069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 -- 172.21.15.71:0/2867041031 shutdown_connections 2023-10-26T05:52:52.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.248+0000 7fa9958cd640 1 -- 172.21.15.71:0/2867041031 wait complete. 2023-10-26T05:52:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 Processor -- start 2023-10-26T05:52:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- start start 2023-10-26T05:52:52.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9900b2890 0x7fa99014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 0x7fa990132ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99013c2f0 0x7fa990157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015a980 con 0x7fa9900cb010 2023-10-26T05:52:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015ab90 con 0x7fa99013c2f0 2023-10-26T05:52:52.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99015ae70 con 0x7fa9900b2890 2023-10-26T05:52:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 0x7fa990132ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 0x7fa990132ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44632/0 (socket says 172.21.15.71:44632) 2023-10-26T05:52:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 -- 172.21.15.71:0/1472954731 learned_addr learned my addr 172.21.15.71:0/1472954731 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98effd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9900b2890 0x7fa99014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 -- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9900b2890 msgr2=0x7fa99014c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9900b2890 0x7fa99014c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 -- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99013c2f0 msgr2=0x7fa990157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99013c2f0 0x7fa990157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 -- 172.21.15.71:0/1472954731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa980009560 con 0x7fa9900cb010 2023-10-26T05:52:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa98e7fc640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 0x7fa990132ef0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c970 tx=0x7fa98400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984007bb0 con 0x7fa9900cb010 2023-10-26T05:52:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa984007e00 con 0x7fa9900cb010 2023-10-26T05:52:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99015b150 con 0x7fa9900cb010 2023-10-26T05:52:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984005280 con 0x7fa9900cb010 2023-10-26T05:52:52.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa990133430 con 0x7fa9900cb010 2023-10-26T05:52:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa990066b10 con 0x7fa9900cb010 2023-10-26T05:52:52.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa9840053e0 con 0x7fa9900cb010 2023-10-26T05:52:52.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.252+0000 7fa977fff640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa97c066470 0x7fa97c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.256+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9840844c0 con 0x7fa9900cb010 2023-10-26T05:52:52.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.256+0000 7fa98effd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa97c066470 0x7fa97c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.256+0000 7fa98effd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa97c066470 0x7fa97c068930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9800af000 tx=0x7fa98001b100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:52.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.260+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa984051940 con 0x7fa9900cb010 2023-10-26T05:52:52.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.412+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9900721e0 con 0x7fa9900cb010 2023-10-26T05:52:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.412+0000 7fa977fff640 1 -- 172.21.15.71:0/1472954731 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16488 (secure 0 0 0) 0x7fa98401d090 con 0x7fa9900cb010 2023-10-26T05:52:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:52.421 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":112,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:43.204060+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa97c066470 msgr2=0x7fa97c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa97c066470 0x7fa97c068930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa9800af000 tx=0x7fa98001b100 comp rx=0 tx=0).stop 2023-10-26T05:52:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 msgr2=0x7fa990132ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 0x7fa990132ef0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c970 tx=0x7fa98400ce30 comp rx=0 tx=0).stop 2023-10-26T05:52:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 shutdown_connections 2023-10-26T05:52:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa99013c2f0 0x7fa990157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa97c066470 0x7fa97c068930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9900cb010 0x7fa990132ef0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 --2- 172.21.15.71:0/1472954731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa9900b2890 0x7fa99014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 >> 172.21.15.71:0/1472954731 conn(0x7fa990069770 msgr2=0x7fa9900ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 shutdown_connections 2023-10-26T05:52:52.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.416+0000 7fa9958cd640 1 -- 172.21.15.71:0/1472954731 wait complete. 2023-10-26T05:52:52.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:52:09.840186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:52:10.672466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:52:52.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:52:52.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- 172.21.15.71:0/3177452021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe058004d10 msgr2=0x7fe058005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 --2- 172.21.15.71:0/3177452021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe058004d10 0x7fe058005190 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c590 tx=0x7fe06001faa0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06d7be640 1 -- 172.21.15.71:0/3177452021 reap_dead start 2023-10-26T05:52:52.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- 172.21.15.71:0/3177452021 shutdown_connections 2023-10-26T05:52:52.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- 172.21.15.71:0/3177452021 >> 172.21.15.71:0/3177452021 conn(0x7fe0680ce270 msgr2=0x7fe0680ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- 172.21.15.71:0/3177452021 shutdown_connections 2023-10-26T05:52:52.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- 172.21.15.71:0/3177452021 wait complete. 2023-10-26T05:52:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 Processor -- start 2023-10-26T05:52:52.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- start start 2023-10-26T05:52:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 0x7fe0681576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe068157bf0 0x7fe0680aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0680af510 0x7fe0681f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680af9a0 con 0x7fe06814fda0 2023-10-26T05:52:52.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068155f10 con 0x7fe068157bf0 2023-10-26T05:52:52.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06f221640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681561f0 con 0x7fe0680af510 2023-10-26T05:52:52.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06cfbd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 0x7fe0681576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06cfbd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 0x7fe0681576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44660/0 (socket says 172.21.15.71:44660) 2023-10-26T05:52:52.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06cfbd640 1 -- 172.21.15.71:0/3812440477 learned_addr learned my addr 172.21.15.71:0/3812440477 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:52.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe067fff640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe068157bf0 0x7fe0680aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06d7be640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0680af510 0x7fe0681f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.560+0000 7fe06cfbd640 1 -- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0680af510 msgr2=0x7fe0681f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06cfbd640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0680af510 0x7fe0681f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06cfbd640 1 -- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe068157bf0 msgr2=0x7fe0680aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06cfbd640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe068157bf0 0x7fe0680aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06cfbd640 1 -- 172.21.15.71:0/3812440477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0681564d0 con 0x7fe06814fda0 2023-10-26T05:52:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06d7be640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0680af510 0x7fe0681f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06cfbd640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 0x7fe0681576b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00b5d0 tx=0x7fe05c00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0041b0 con 0x7fe06814fda0 2023-10-26T05:52:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe05c010ae0 con 0x7fe06814fda0 2023-10-26T05:52:52.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0681567b0 con 0x7fe06814fda0 2023-10-26T05:52:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c01a470 con 0x7fe06814fda0 2023-10-26T05:52:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe068155b50 con 0x7fe06814fda0 2023-10-26T05:52:52.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe068066d00 con 0x7fe06814fda0 2023-10-26T05:52:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe05c017070 con 0x7fe06814fda0 2023-10-26T05:52:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe0657fa640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe04c066470 0x7fe04c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe05c084a60 con 0x7fe06814fda0 2023-10-26T05:52:52.576 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe067fff640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe04c066470 0x7fe04c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.564+0000 7fe067fff640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe04c066470 0x7fe04c068930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe0680ad910 tx=0x7fe054006bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:52.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.568+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe05c051f70 con 0x7fe06814fda0 2023-10-26T05:52:52.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0680caa60 con 0x7fe06814fda0 2023-10-26T05:52:52.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe0657fa640 1 -- 172.21.15.71:0/3812440477 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe05c055590 con 0x7fe06814fda0 2023-10-26T05:52:52.726 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:52:52.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe04c066470 msgr2=0x7fe04c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe04c066470 0x7fe04c068930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe0680ad910 tx=0x7fe054006bf0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 msgr2=0x7fe0681576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 0x7fe0681576b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe05c00b5d0 tx=0x7fe05c00ba90 comp rx=0 tx=0).stop 2023-10-26T05:52:52.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 shutdown_connections 2023-10-26T05:52:52.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe0680af510 0x7fe0681f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe068157bf0 0x7fe0680aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe04c066470 0x7fe04c068930 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 --2- 172.21.15.71:0/3812440477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe06814fda0 0x7fe0681576b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 >> 172.21.15.71:0/3812440477 conn(0x7fe0680ce270 msgr2=0x7fe0680aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:52.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 shutdown_connections 2023-10-26T05:52:52.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.720+0000 7fe06f221640 1 -- 172.21.15.71:0/3812440477 wait complete. 2023-10-26T05:52:52.745 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:52:52.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/90879850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88780aff60 msgr2=0x7f88780b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 --2- 172.21.15.71:0/90879850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88780aff60 0x7f88780b0360 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f887806da20 tx=0x7f887001c7e0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/90879850 shutdown_connections 2023-10-26T05:52:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 --2- 172.21.15.71:0/90879850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88780aff60 0x7f88780b0360 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 --2- 172.21.15.71:0/90879850 >> v2:172.21.15.86:3300/0 conn(0x7f8878071270 0x7f887814e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/90879850 >> 172.21.15.71:0/90879850 conn(0x7f88780aec60 msgr2=0x7f88780af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:52.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/90879850 shutdown_connections 2023-10-26T05:52:52.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/90879850 wait complete. 2023-10-26T05:52:52.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 Processor -- start 2023-10-26T05:52:52.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- start start 2023-10-26T05:52:52.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8878071270 0x7f88780ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88780aff60 0x7f8878144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 0x7f88781510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88780aea90 con 0x7f8878157db0 2023-10-26T05:52:52.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88781450d0 con 0x7f88780aff60 2023-10-26T05:52:52.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8878144ba0 con 0x7f8878071270 2023-10-26T05:52:52.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 0x7f88781510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 0x7f88781510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44678/0 (socket says 172.21.15.71:44678) 2023-10-26T05:52:52.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 -- 172.21.15.71:0/1780426394 learned_addr learned my addr 172.21.15.71:0/1780426394 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:52.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f88777fe640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8878071270 0x7f88780ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 -- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8878071270 msgr2=0x7f88780ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8878071270 0x7f88780ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8876ffd640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88780aff60 0x7f8878144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 -- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88780aff60 msgr2=0x7f8878144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:52.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88780aff60 0x7f8878144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:52.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 -- 172.21.15.71:0/1780426394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700094a0 con 0x7f8878157db0 2023-10-26T05:52:52.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f88777fe640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8878071270 0x7f88780ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:52.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f8877fff640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 0x7f88781510f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f88700983c0 tx=0x7f8870005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:52.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870004230 con 0x7f8878157db0 2023-10-26T05:52:52.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8870004480 con 0x7f8878157db0 2023-10-26T05:52:52.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88780ad0d0 con 0x7f8878157db0 2023-10-26T05:52:52.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88700048f0 con 0x7f8878157db0 2023-10-26T05:52:52.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.864+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88780ad690 con 0x7f8878157db0 2023-10-26T05:52:52.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.868+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8878066b10 con 0x7f8878157db0 2023-10-26T05:52:52.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.868+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8870004a50 con 0x7f8878157db0 2023-10-26T05:52:52.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.868+0000 7f886ffff640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8858066470 0x7f8858068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:52.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.868+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f887011f140 con 0x7f8878157db0 2023-10-26T05:52:52.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.868+0000 7f88777fe640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8858066470 0x7f8858068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:52.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.868+0000 7f88777fe640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8858066470 0x7f8858068930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8868005c50 tx=0x7f8868008310 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:52.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:52.872+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88700f0060 con 0x7f8878157db0 2023-10-26T05:52:53.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.012+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88780721e0 con 0x7f8878157db0 2023-10-26T05:52:53.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f886ffff640 1 -- 172.21.15.71:0/1780426394 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88700efbf0 con 0x7f8878157db0 2023-10-26T05:52:53.021 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:52:53.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8858066470 msgr2=0x7f8858068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8858066470 0x7f8858068930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8868005c50 tx=0x7f8868008310 comp rx=0 tx=0).stop 2023-10-26T05:52:53.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 msgr2=0x7f88781510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 0x7f88781510f0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f88700983c0 tx=0x7f8870005880 comp rx=0 tx=0).stop 2023-10-26T05:52:53.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 shutdown_connections 2023-10-26T05:52:53.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8858066470 0x7f8858068930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8878157db0 0x7f88781510f0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88780aff60 0x7f8878144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 --2- 172.21.15.71:0/1780426394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8878071270 0x7f88780ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 >> 172.21.15.71:0/1780426394 conn(0x7f88780aec60 msgr2=0x7f887815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 shutdown_connections 2023-10-26T05:52:53.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.016+0000 7f887dda5640 1 -- 172.21.15.71:0/1780426394 wait complete. 2023-10-26T05:52:53.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:52:53.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 -- 172.21.15.71:0/50831209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b1230 msgr2=0x7f27200b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 --2- 172.21.15.71:0/50831209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b1230 0x7f27200b1c60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f272006e940 tx=0x7f271001f090 comp rx=0 tx=0).stop 2023-10-26T05:52:53.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 -- 172.21.15.71:0/50831209 shutdown_connections 2023-10-26T05:52:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 --2- 172.21.15.71:0/50831209 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27200b21a0 0x7f27200aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 --2- 172.21.15.71:0/50831209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b1230 0x7f27200b1c60 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 --2- 172.21.15.71:0/50831209 >> v2:172.21.15.86:3300/0 conn(0x7f2720141a40 0x7f27200c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 -- 172.21.15.71:0/50831209 >> 172.21.15.71:0/50831209 conn(0x7f27200aec60 msgr2=0x7f27200af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 -- 172.21.15.71:0/50831209 shutdown_connections 2023-10-26T05:52:53.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 -- 172.21.15.71:0/50831209 wait complete. 2023-10-26T05:52:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 Processor -- start 2023-10-26T05:52:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.160+0000 7f2727a0a640 1 -- start start 2023-10-26T05:52:53.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27200b1230 0x7f27201e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 0x7f2720157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2720141a40 0x7f2720136340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27201380f0 con 0x7f27200b21a0 2023-10-26T05:52:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27201383d0 con 0x7f27200b1230 2023-10-26T05:52:53.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27201386b0 con 0x7f2720141a40 2023-10-26T05:52:53.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 0x7f2720157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 0x7f2720157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44694/0 (socket says 172.21.15.71:44694) 2023-10-26T05:52:53.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 -- 172.21.15.71:0/2740149338 learned_addr learned my addr 172.21.15.71:0/2740149338 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:53.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f27257a6640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27200b1230 0x7f27201e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2725fa7640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2720141a40 0x7f2720136340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 -- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2720141a40 msgr2=0x7f2720136340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2720141a40 0x7f2720136340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 -- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27200b1230 msgr2=0x7f27201e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27200b1230 0x7f27201e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 -- 172.21.15.71:0/2740149338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2710009560 con 0x7f27200b21a0 2023-10-26T05:52:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2724fa5640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 0x7f2720157840 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f271001d740 tx=0x7f2710005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100be070 con 0x7f27200b21a0 2023-10-26T05:52:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f27100051e0 con 0x7f27200b21a0 2023-10-26T05:52:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2720138990 con 0x7f27200b21a0 2023-10-26T05:52:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100055f0 con 0x7f27200b21a0 2023-10-26T05:52:53.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27201e45f0 con 0x7f27200b21a0 2023-10-26T05:52:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2720066b10 con 0x7f27200b21a0 2023-10-26T05:52:53.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27100b7040 con 0x7f27200b21a0 2023-10-26T05:52:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2715ffb640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2704066470 0x7f2704068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2710121810 con 0x7f27200b21a0 2023-10-26T05:52:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.164+0000 7f27257a6640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2704066470 0x7f2704068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.168+0000 7f27257a6640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2704066470 0x7f2704068930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f271c0058d0 tx=0x7f271c005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.168+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f27100eec80 con 0x7f27200b21a0 2023-10-26T05:52:53.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.324+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2720138eb0 con 0x7f27200b21a0 2023-10-26T05:52:53.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.324+0000 7f2715ffb640 1 -- 172.21.15.71:0/2740149338 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2720138eb0 con 0x7f27200b21a0 2023-10-26T05:52:53.333 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:52:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2704066470 msgr2=0x7f2704068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2704066470 0x7f2704068930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f271c0058d0 tx=0x7f271c005860 comp rx=0 tx=0).stop 2023-10-26T05:52:53.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 msgr2=0x7f2720157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 0x7f2720157840 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f271001d740 tx=0x7f2710005e90 comp rx=0 tx=0).stop 2023-10-26T05:52:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 shutdown_connections 2023-10-26T05:52:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2720141a40 0x7f2720136340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2704066470 0x7f2704068930 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27200b21a0 0x7f2720157840 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 --2- 172.21.15.71:0/2740149338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27200b1230 0x7f27201e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 >> 172.21.15.71:0/2740149338 conn(0x7f27200aec60 msgr2=0x7f2720140a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 shutdown_connections 2023-10-26T05:52:53.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.328+0000 7f2727a0a640 1 -- 172.21.15.71:0/2740149338 wait complete. 2023-10-26T05:52:53.352 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:53.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.464+0000 7f3209165640 1 -- 172.21.15.71:0/2585218817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32040cc5f0 msgr2=0x7f32040cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.464+0000 7f3209165640 1 --2- 172.21.15.71:0/2585218817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32040cc5f0 0x7f32040cca80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009bc0 tx=0x7f31fc01cac0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.464+0000 7f3209165640 1 -- 172.21.15.71:0/2585218817 shutdown_connections 2023-10-26T05:52:53.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.464+0000 7f3209165640 1 --2- 172.21.15.71:0/2585218817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32040cc5f0 0x7f32040cca80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.464+0000 7f3209165640 1 --2- 172.21.15.71:0/2585218817 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f32041588f0 0x7f3204158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.464+0000 7f3209165640 1 -- 172.21.15.71:0/2585218817 >> 172.21.15.71:0/2585218817 conn(0x7f32040ce270 msgr2=0x7f32040ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- 172.21.15.71:0/2585218817 shutdown_connections 2023-10-26T05:52:53.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- 172.21.15.71:0/2585218817 wait complete. 2023-10-26T05:52:53.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 Processor -- start 2023-10-26T05:52:53.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- start start 2023-10-26T05:52:53.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040cc5f0 0x7f3204155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 0x7f320414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320414b8d0 0x7f3204157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32041559b0 con 0x7f32041588f0 2023-10-26T05:52:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320414ca70 con 0x7f32040cc5f0 2023-10-26T05:52:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320414cd50 con 0x7f320414b8d0 2023-10-26T05:52:53.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 0x7f320414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 0x7f320414b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44702/0 (socket says 172.21.15.71:44702) 2023-10-26T05:52:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 -- 172.21.15.71:0/1353592686 learned_addr learned my addr 172.21.15.71:0/1353592686 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f320359e640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320414b8d0 0x7f3204157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 -- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320414b8d0 msgr2=0x7f3204157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3202d9d640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040cc5f0 0x7f3204155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320414b8d0 0x7f3204157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 -- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040cc5f0 msgr2=0x7f3204155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040cc5f0 0x7f3204155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 -- 172.21.15.71:0/1353592686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc009820 con 0x7f32041588f0 2023-10-26T05:52:53.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f320359e640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320414b8d0 0x7f3204157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:53.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fa59c640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 0x7f320414b0e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c500 tx=0x7f31ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:53.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec012830 con 0x7f32041588f0 2023-10-26T05:52:53.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f31ec012a80 con 0x7f32041588f0 2023-10-26T05:52:53.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320414d030 con 0x7f32041588f0 2023-10-26T05:52:53.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec012e90 con 0x7f32041588f0 2023-10-26T05:52:53.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320414b620 con 0x7f32041588f0 2023-10-26T05:52:53.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.468+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320414c8d0 con 0x7f32041588f0 2023-10-26T05:52:53.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.472+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f31ec01a3b0 con 0x7f32041588f0 2023-10-26T05:52:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.472+0000 7f31fbfff640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31e4066470 0x7f31e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.472+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31ec0840a0 con 0x7f32041588f0 2023-10-26T05:52:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.472+0000 7f3202d9d640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31e4066470 0x7f31e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.472+0000 7f3202d9d640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31e4066470 0x7f31e4068930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f31f40058d0 tx=0x7f31f4005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:53.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.476+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f320414c8d0 con 0x7f32041588f0 2023-10-26T05:52:53.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.644+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f320414c8d0 con 0x7f32041588f0 2023-10-26T05:52:53.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f31fbfff640 1 -- 172.21.15.71:0/1353592686 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f320414c8d0 con 0x7f32041588f0 2023-10-26T05:52:53.653 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:53.654 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:53.654 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:53.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31e4066470 msgr2=0x7f31e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31e4066470 0x7f31e4068930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f31f40058d0 tx=0x7f31f4005860 comp rx=0 tx=0).stop 2023-10-26T05:52:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 msgr2=0x7f320414b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 0x7f320414b0e0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f31ec00c500 tx=0x7f31ec00c9c0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 shutdown_connections 2023-10-26T05:52:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f320414b8d0 0x7f3204157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f31e4066470 0x7f31e4068930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32041588f0 0x7f320414b0e0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 --2- 172.21.15.71:0/1353592686 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32040cc5f0 0x7f3204155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 >> 172.21.15.71:0/1353592686 conn(0x7f32040ce270 msgr2=0x7f320414ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 shutdown_connections 2023-10-26T05:52:53.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.648+0000 7f3209165640 1 -- 172.21.15.71:0/1353592686 wait complete. 2023-10-26T05:52:53.671 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:53.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/1569132650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 msgr2=0x7f30e40b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 --2- 172.21.15.71:0/1569132650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e40b2350 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f30e40cc8d0 tx=0x7f30d401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/1569132650 shutdown_connections 2023-10-26T05:52:53.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 --2- 172.21.15.71:0/1569132650 >> v2:172.21.15.86:3300/0 conn(0x7f30e40b2890 0x7f30e40caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 --2- 172.21.15.71:0/1569132650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e40b2350 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/1569132650 >> 172.21.15.71:0/1569132650 conn(0x7f30e4069770 msgr2=0x7f30e4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/1569132650 shutdown_connections 2023-10-26T05:52:53.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/1569132650 wait complete. 2023-10-26T05:52:53.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 Processor -- start 2023-10-26T05:52:53.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- start start 2023-10-26T05:52:53.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e40b2890 0x7f30e4152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30e40cb010 0x7f30e413ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4151c10 con 0x7f30e413c2f0 2023-10-26T05:52:53.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4150fd0 con 0x7f30e40b2890 2023-10-26T05:52:53.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4135db0 con 0x7f30e40cb010 2023-10-26T05:52:53.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e413cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44722/0 (socket says 172.21.15.71:44722) 2023-10-26T05:52:53.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 -- 172.21.15.71:0/2129276853 learned_addr learned my addr 172.21.15.71:0/2129276853 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:53.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e259c640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e40b2890 0x7f30e4152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e1d9b640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30e40cb010 0x7f30e413ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 -- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30e40cb010 msgr2=0x7f30e413ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30e40cb010 0x7f30e413ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 -- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e40b2890 msgr2=0x7f30e4152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e40b2890 0x7f30e4152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 -- 172.21.15.71:0/2129276853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40094a0 con 0x7f30e413c2f0 2023-10-26T05:52:53.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e1d9b640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30e40cb010 0x7f30e413ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:53.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e2d9d640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e413cf50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00cb10 tx=0x7f30dc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:53.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00aa50 con 0x7f30e413c2f0 2023-10-26T05:52:53.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f30dc00acc0 con 0x7f30e413c2f0 2023-10-26T05:52:53.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e4151270 con 0x7f30e413c2f0 2023-10-26T05:52:53.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc012680 con 0x7f30e413c2f0 2023-10-26T05:52:53.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.776+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e4151830 con 0x7f30e413c2f0 2023-10-26T05:52:53.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.780+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4151e80 con 0x7f30e413c2f0 2023-10-26T05:52:53.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.780+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f30dc0127e0 con 0x7f30e413c2f0 2023-10-26T05:52:53.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.780+0000 7f30d2ffd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f30c4066470 0x7f30c4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:53.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.780+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30dc084580 con 0x7f30e413c2f0 2023-10-26T05:52:53.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.780+0000 7f30e259c640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f30c4066470 0x7f30c4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:53.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.780+0000 7f30e259c640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f30c4066470 0x7f30c4068930 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f30d40af000 tx=0x7f30d401b000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:53.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.784+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f30dc051b40 con 0x7f30e413c2f0 2023-10-26T05:52:53.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.932+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30e4066b10 con 0x7f30e413c2f0 2023-10-26T05:52:53.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30d2ffd640 1 -- 172.21.15.71:0/2129276853 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16488 (secure 0 0 0) 0x7f30dc055160 con 0x7f30e413c2f0 2023-10-26T05:52:53.941 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:53.941 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":112,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:43.204060+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:53.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f30c4066470 msgr2=0x7f30c4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f30c4066470 0x7f30c4068930 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f30d40af000 tx=0x7f30d401b000 comp rx=0 tx=0).stop 2023-10-26T05:52:53.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 msgr2=0x7f30e413cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:53.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e413cf50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00cb10 tx=0x7f30dc00cc00 comp rx=0 tx=0).stop 2023-10-26T05:52:53.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 shutdown_connections 2023-10-26T05:52:53.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f30c4066470 0x7f30c4068930 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30e413c2f0 0x7f30e413cf50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f30e40cb010 0x7f30e413ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 --2- 172.21.15.71:0/2129276853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30e40b2890 0x7f30e4152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:53.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 >> 172.21.15.71:0/2129276853 conn(0x7f30e4069770 msgr2=0x7f30e40ad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:53.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 shutdown_connections 2023-10-26T05:52:53.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:53.936+0000 7f30e8abd640 1 -- 172.21.15.71:0/2129276853 wait complete. 2023-10-26T05:52:53.958 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:52:53.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 -- 172.21.15.71:0/239543344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d015bbc0 msgr2=0x7f11d015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 --2- 172.21.15.71:0/239543344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d015bbc0 0x7f11d015c050 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009930 tx=0x7f11c801c8c0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 -- 172.21.15.71:0/239543344 shutdown_connections 2023-10-26T05:52:54.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 --2- 172.21.15.71:0/239543344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d015bbc0 0x7f11d015c050 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 --2- 172.21.15.71:0/239543344 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11d006b110 0x7f11d00cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 -- 172.21.15.71:0/239543344 >> 172.21.15.71:0/239543344 conn(0x7f11d00b00e0 msgr2=0x7f11d00b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 -- 172.21.15.71:0/239543344 shutdown_connections 2023-10-26T05:52:54.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.068+0000 7f11d662a640 1 -- 172.21.15.71:0/239543344 wait complete. 2023-10-26T05:52:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 Processor -- start 2023-10-26T05:52:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- start start 2023-10-26T05:52:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d006b110 0x7f11d01f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 0x7f11d0137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11d015bbc0 0x7f11d0138060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01f5290 con 0x7f11d0141520 2023-10-26T05:52:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01f4960 con 0x7f11d006b110 2023-10-26T05:52:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01f4c40 con 0x7f11d015bbc0 2023-10-26T05:52:54.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 0x7f11d0137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 0x7f11d0137b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44736/0 (socket says 172.21.15.71:44736) 2023-10-26T05:52:54.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 -- 172.21.15.71:0/1765978415 learned_addr learned my addr 172.21.15.71:0/1765978415 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:54.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 -- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11d015bbc0 msgr2=0x7f11d0138060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11d015bbc0 0x7f11d0138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 -- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d006b110 msgr2=0x7f11d01f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d006b110 0x7f11d01f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 -- 172.21.15.71:0/1765978415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c8009560 con 0x7f11d0141520 2023-10-26T05:52:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11cf7fe640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 0x7f11d0137b20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f11c400d950 tx=0x7f11c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4009c30 con 0x7f11d0141520 2023-10-26T05:52:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11c4009e80 con 0x7f11d0141520 2023-10-26T05:52:54.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d01f4f20 con 0x7f11d0141520 2023-10-26T05:52:54.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40106c0 con 0x7f11d0141520 2023-10-26T05:52:54.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d01f45f0 con 0x7f11d0141520 2023-10-26T05:52:54.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d0066b10 con 0x7f11d0141520 2023-10-26T05:52:54.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.072+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11c4010820 con 0x7f11d0141520 2023-10-26T05:52:54.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.076+0000 7f11ccff9640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11bc0664a0 0x7f11bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.076+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c40842b0 con 0x7f11d0141520 2023-10-26T05:52:54.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.076+0000 7f11cffff640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11bc0664a0 0x7f11bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.076+0000 7f11cffff640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11bc0664a0 0x7f11bc068960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f11c0001d30 tx=0x7f11c0007ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:54.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.080+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11c4051730 con 0x7f11d0141520 2023-10-26T05:52:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.248+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d0066d00 con 0x7f11d0141520 2023-10-26T05:52:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.248+0000 7f11ccff9640 1 -- 172.21.15.71:0/1765978415 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f11c4054d50 con 0x7f11d0141520 2023-10-26T05:52:54.255 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:54.256 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:54.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11bc0664a0 msgr2=0x7f11bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11bc0664a0 0x7f11bc068960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f11c0001d30 tx=0x7f11c0007ca0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 msgr2=0x7f11d0137b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 0x7f11d0137b20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f11c400d950 tx=0x7f11c400de10 comp rx=0 tx=0).stop 2023-10-26T05:52:54.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 shutdown_connections 2023-10-26T05:52:54.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11d015bbc0 0x7f11d0138060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f11bc0664a0 0x7f11bc068960 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11d0141520 0x7f11d0137b20 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 --2- 172.21.15.71:0/1765978415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11d006b110 0x7f11d01f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 >> 172.21.15.71:0/1765978415 conn(0x7f11d00b00e0 msgr2=0x7f11d006d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:54.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 shutdown_connections 2023-10-26T05:52:54.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.252+0000 7f11d662a640 1 -- 172.21.15.71:0/1765978415 wait complete. 2023-10-26T05:52:54.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:52:54.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- 172.21.15.71:0/434240059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f8003f30 msgr2=0x7f87f80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 --2- 172.21.15.71:0/434240059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f8003f30 0x7f87f80047d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009b30 tx=0x7f87fc01d220 comp rx=0 tx=0).stop 2023-10-26T05:52:54.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 -- 172.21.15.71:0/434240059 reap_dead start 2023-10-26T05:52:54.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- 172.21.15.71:0/434240059 shutdown_connections 2023-10-26T05:52:54.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- 172.21.15.71:0/434240059 >> 172.21.15.71:0/434240059 conn(0x7f880c0b0c80 msgr2=0x7f880c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:54.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- 172.21.15.71:0/434240059 shutdown_connections 2023-10-26T05:52:54.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- 172.21.15.71:0/434240059 wait complete. 2023-10-26T05:52:54.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 Processor -- start 2023-10-26T05:52:54.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- start start 2023-10-26T05:52:54.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f8003f30 0x7f880c145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c1593c0 0x7f880c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 0x7f880c161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c146810 con 0x7f880c161a50 2023-10-26T05:52:54.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c159d80 con 0x7f880c1593c0 2023-10-26T05:52:54.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8813bc9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c1603b0 con 0x7f87f8003f30 2023-10-26T05:52:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 0x7f880c161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 0x7f880c161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44762/0 (socket says 172.21.15.71:44762) 2023-10-26T05:52:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 -- 172.21.15.71:0/2484753058 learned_addr learned my addr 172.21.15.71:0/2484753058 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8811965640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f8003f30 0x7f880c145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8811164640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c1593c0 0x7f880c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 -- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f8003f30 msgr2=0x7f880c145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f8003f30 0x7f880c145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 -- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c1593c0 msgr2=0x7f880c159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c1593c0 0x7f880c159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 -- 172.21.15.71:0/2484753058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c1463f0 con 0x7f880c161a50 2023-10-26T05:52:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8811164640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c1593c0 0x7f880c159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:54.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.380+0000 7f8812166640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 0x7f880c161ed0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8804001340 tx=0x7f880400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:54.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880400b8a0 con 0x7f880c161a50 2023-10-26T05:52:54.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f880400ba20 con 0x7f880c161a50 2023-10-26T05:52:54.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c162730 con 0x7f880c161a50 2023-10-26T05:52:54.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880400be30 con 0x7f880c161a50 2023-10-26T05:52:54.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c162cf0 con 0x7f880c161a50 2023-10-26T05:52:54.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c066b10 con 0x7f880c161a50 2023-10-26T05:52:54.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f880401a3b0 con 0x7f880c161a50 2023-10-26T05:52:54.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f88027fc640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87f0066470 0x7f87f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8804083fa0 con 0x7f880c161a50 2023-10-26T05:52:54.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f8811965640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87f0066470 0x7f87f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.384+0000 7f8811965640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87f0066470 0x7f87f0068930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8808000f30 tx=0x7f8808009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:54.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.388+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88040b5090 con 0x7f880c161a50 2023-10-26T05:52:54.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.560+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f880c146950 con 0x7f880c161a50 2023-10-26T05:52:54.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.560+0000 7f88027fc640 1 -- 172.21.15.71:0/2484753058 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f880c146950 con 0x7f880c161a50 2023-10-26T05:52:54.569 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:54.569 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":38,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:52:11.692352+0000","modified":"2023-10-26T05:52:13.702898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7074":{"gid":7074,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2440514167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2440514167},{"type":"v1","addr":"172.21.15.71:6835","nonce":2440514167}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-26T05:52:54.569 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 38 2023-10-26T05:52:54.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87f0066470 msgr2=0x7f87f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87f0066470 0x7f87f0068930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8808000f30 tx=0x7f8808009040 comp rx=0 tx=0).stop 2023-10-26T05:52:54.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 msgr2=0x7f880c161ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 0x7f880c161ed0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8804001340 tx=0x7f880400df80 comp rx=0 tx=0).stop 2023-10-26T05:52:54.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 shutdown_connections 2023-10-26T05:52:54.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87f0066470 0x7f87f0068930 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f880c161a50 0x7f880c161ed0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c1593c0 0x7f880c159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 --2- 172.21.15.71:0/2484753058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f8003f30 0x7f880c145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 >> 172.21.15.71:0/2484753058 conn(0x7f880c0b0c80 msgr2=0x7f880c06dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:54.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 shutdown_connections 2023-10-26T05:52:54.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.564+0000 7f8813bc9640 1 -- 172.21.15.71:0/2484753058 wait complete. 2023-10-26T05:52:54.586 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:54.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2585593404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0cc050 msgr2=0x7fa2cc0cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2585593404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0cc050 0x7fa2cc0cd9f0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009c80 tx=0x7fa2c401cc80 comp rx=0 tx=0).stop 2023-10-26T05:52:54.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2585593404 shutdown_connections 2023-10-26T05:52:54.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2585593404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0cc050 0x7fa2cc0cd9f0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2585593404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc0af990 0x7fa2cc0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2585593404 >> 172.21.15.71:0/2585593404 conn(0x7fa2cc06f440 msgr2=0x7fa2cc06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:54.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2585593404 shutdown_connections 2023-10-26T05:52:54.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2585593404 wait complete. 2023-10-26T05:52:54.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 Processor -- start 2023-10-26T05:52:54.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- start start 2023-10-26T05:52:54.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 0x7fa2cc1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0cc050 0x7fa2cc139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc142b30 0x7fa2cc159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15c630 con 0x7fa2cc0af990 2023-10-26T05:52:54.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15bd90 con 0x7fa2cc0cc050 2023-10-26T05:52:54.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15c070 con 0x7fa2cc142b30 2023-10-26T05:52:54.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cbfff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc142b30 0x7fa2cc159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 0x7fa2cc1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 0x7fa2cc1565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44780/0 (socket says 172.21.15.71:44780) 2023-10-26T05:52:54.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cbfff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc142b30 0x7fa2cc159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.71:54992/0 (socket says 172.21.15.71:54992) 2023-10-26T05:52:54.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 -- 172.21.15.71:0/2302857153 learned_addr learned my addr 172.21.15.71:0/2302857153 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:54.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2c2ffd640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0cc050 0x7fa2cc139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 -- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc142b30 msgr2=0x7fa2cc159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc142b30 0x7fa2cc159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 -- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0cc050 msgr2=0x7fa2cc139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0cc050 0x7fa2cc139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 -- 172.21.15.71:0/2302857153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40098e0 con 0x7fa2cc0af990 2023-10-26T05:52:54.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2cb7fe640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 0x7fa2cc1565b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00b410 tx=0x7fa2bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:54.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc004020 con 0x7fa2cc0af990 2023-10-26T05:52:54.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa2bc004270 con 0x7fa2cc0af990 2023-10-26T05:52:54.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc15c350 con 0x7fa2cc0af990 2023-10-26T05:52:54.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc011630 con 0x7fa2cc0af990 2023-10-26T05:52:54.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.704+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc15b9b0 con 0x7fa2cc0af990 2023-10-26T05:52:54.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.708+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc066b10 con 0x7fa2cc0af990 2023-10-26T05:52:54.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.708+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa2bc003600 con 0x7fa2cc0af990 2023-10-26T05:52:54.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.708+0000 7fa2c8ff9640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa2ac066470 0x7fa2ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:54.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.708+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2bc083e30 con 0x7fa2cc0af990 2023-10-26T05:52:54.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.708+0000 7fa2c2ffd640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa2ac066470 0x7fa2ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:54.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.708+0000 7fa2c2ffd640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa2ac066470 0x7fa2ac068930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40058d0 tx=0x7fa2b4005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:54.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.712+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa2bc051230 con 0x7fa2cc0af990 2023-10-26T05:52:54.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2cc0721e0 con 0x7fa2cc0af990 2023-10-26T05:52:54.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2c8ff9640 1 -- 172.21.15.71:0/2302857153 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16488 (secure 0 0 0) 0x7fa2bc054850 con 0x7fa2cc0af990 2023-10-26T05:52:54.863 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:54.863 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":112,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:43.204060+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:52:09.840186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-26T05:52:10.672466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa2ac066470 msgr2=0x7fa2ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa2ac066470 0x7fa2ac068930 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40058d0 tx=0x7fa2b4005860 comp rx=0 tx=0).stop 2023-10-26T05:52:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 msgr2=0x7fa2cc1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 0x7fa2cc1565b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00b410 tx=0x7fa2bc00b8d0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 shutdown_connections 2023-10-26T05:52:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2cc142b30 0x7fa2cc159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2cc0cc050 0x7fa2cc139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa2ac066470 0x7fa2ac068930 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 --2- 172.21.15.71:0/2302857153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2cc0af990 0x7fa2cc1565b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:54.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 >> 172.21.15.71:0/2302857153 conn(0x7fa2cc06f440 msgr2=0x7fa2cc06f820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 shutdown_connections 2023-10-26T05:52:54.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.856+0000 7fa2d1d77640 1 -- 172.21.15.71:0/2302857153 wait complete. 2023-10-26T05:52:54.881 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:52:55.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- 172.21.15.71:0/2658705865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 msgr2=0x7fe2901586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 --2- 172.21.15.71:0/2658705865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe2901586c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fe2900ca890 tx=0x7fe27801c870 comp rx=0 tx=0).stop 2023-10-26T05:52:55.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- 172.21.15.71:0/2658705865 shutdown_connections 2023-10-26T05:52:55.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 --2- 172.21.15.71:0/2658705865 >> v2:172.21.15.86:3300/0 conn(0x7fe29006a000 0x7fe29006a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 --2- 172.21.15.71:0/2658705865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe2901586c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- 172.21.15.71:0/2658705865 >> 172.21.15.71:0/2658705865 conn(0x7fe29006b8c0 msgr2=0x7fe29006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:55.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- 172.21.15.71:0/2658705865 shutdown_connections 2023-10-26T05:52:55.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- 172.21.15.71:0/2658705865 wait complete. 2023-10-26T05:52:55.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 Processor -- start 2023-10-26T05:52:55.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- start start 2023-10-26T05:52:55.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe29006a000 0x7fe290152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe290156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2900cc730 0x7fe290156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290150910 con 0x7fe2900b15f0 2023-10-26T05:52:55.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290152630 con 0x7fe2900cc730 2023-10-26T05:52:55.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe295a4f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe290150070 con 0x7fe29006a000 2023-10-26T05:52:55.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe290156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe290156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44796/0 (socket says 172.21.15.71:44796) 2023-10-26T05:52:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 -- 172.21.15.71:0/987647830 learned_addr learned my addr 172.21.15.71:0/987647830 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28f7fe640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2900cc730 0x7fe290156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28effd640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe29006a000 0x7fe290152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 -- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe29006a000 msgr2=0x7fe290152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe29006a000 0x7fe290152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 -- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2900cc730 msgr2=0x7fe290156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2900cc730 0x7fe290156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28e7fc640 1 -- 172.21.15.71:0/987647830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe278009560 con 0x7fe2900b15f0 2023-10-26T05:52:55.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:54.996+0000 7fe28f7fe640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2900cc730 0x7fe290156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:55.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe28e7fc640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe290156a20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe27801cd50 tx=0x7fe278005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:55.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278005230 con 0x7fe2900b15f0 2023-10-26T05:52:55.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2780054a0 con 0x7fe2900b15f0 2023-10-26T05:52:55.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe290150350 con 0x7fe2900b15f0 2023-10-26T05:52:55.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278004930 con 0x7fe2900b15f0 2023-10-26T05:52:55.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29014fcb0 con 0x7fe2900b15f0 2023-10-26T05:52:55.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290066b10 con 0x7fe2900b15f0 2023-10-26T05:52:55.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe2780b4040 con 0x7fe2900b15f0 2023-10-26T05:52:55.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe287fff640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe270066470 0x7fe270068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe27811e020 con 0x7fe2900b15f0 2023-10-26T05:52:55.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe28effd640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe270066470 0x7fe270068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.000+0000 7fe28effd640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe270066470 0x7fe270068930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe2901f5bb0 tx=0x7fe288009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:55.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.004+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe2780eb530 con 0x7fe2900b15f0 2023-10-26T05:52:55.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.180+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe2901527c0 con 0x7fe2900b15f0 2023-10-26T05:52:55.187 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:52:55.180+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:52:55.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.184+0000 7fe287fff640 1 -- 172.21.15.71:0/987647830 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe2780bc090 con 0x7fe2900b15f0 2023-10-26T05:52:55.190 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:52:55.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe270066470 msgr2=0x7fe270068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe270066470 0x7fe270068930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe2901f5bb0 tx=0x7fe288009040 comp rx=0 tx=0).stop 2023-10-26T05:52:55.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 msgr2=0x7fe290156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe290156a20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe27801cd50 tx=0x7fe278005e90 comp rx=0 tx=0).stop 2023-10-26T05:52:55.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 shutdown_connections 2023-10-26T05:52:55.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2900cc730 0x7fe290156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe270066470 0x7fe270068930 secure :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe2901f5bb0 tx=0x7fe288009040 comp rx=0 tx=0).stop 2023-10-26T05:52:55.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2900b15f0 0x7fe290156a20 secure :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe27801cd50 tx=0x7fe278005e90 comp rx=0 tx=0).stop 2023-10-26T05:52:55.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 --2- 172.21.15.71:0/987647830 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe29006a000 0x7fe290152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 >> 172.21.15.71:0/987647830 conn(0x7fe29006b8c0 msgr2=0x7fe29006bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:55.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 shutdown_connections 2023-10-26T05:52:55.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.188+0000 7fe295a4f640 1 -- 172.21.15.71:0/987647830 wait complete. 2023-10-26T05:52:55.204 INFO:tasks.ceph.mds.c.smithi071.stderr: -9638> 2023-10-26T05:52:17.288+0000 7f268f737640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:52:55.213 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:52:55.275 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:52:55.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/3231556305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68a80047d0 msgr2=0x7f68a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 --2- 172.21.15.71:0/3231556305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68a80047d0 0x7f68a8004c50 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f68c008a3b0 tx=0x7f68b401cf50 comp rx=0 tx=0).stop 2023-10-26T05:52:55.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 -- 172.21.15.71:0/3231556305 reap_dead start 2023-10-26T05:52:55.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/3231556305 shutdown_connections 2023-10-26T05:52:55.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/3231556305 >> 172.21.15.71:0/3231556305 conn(0x7f68c00cdd80 msgr2=0x7f68c00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:55.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/3231556305 shutdown_connections 2023-10-26T05:52:55.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/3231556305 wait complete. 2023-10-26T05:52:55.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 Processor -- start 2023-10-26T05:52:55.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- start start 2023-10-26T05:52:55.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a80047d0 0x7f68c014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c00c6170 0x7f68c00c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 0x7f68c0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c006a650 con 0x7f68c0156740 2023-10-26T05:52:55.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0157100 con 0x7f68a80047d0 2023-10-26T05:52:55.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00c6c30 con 0x7f68c00c6170 2023-10-26T05:52:55.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 0x7f68c0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 0x7f68c0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44842/0 (socket says 172.21.15.71:44842) 2023-10-26T05:52:55.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 -- 172.21.15.71:0/2343318820 learned_addr learned my addr 172.21.15.71:0/2343318820 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:55.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68bf7fe640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c00c6170 0x7f68c00c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68bffff640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a80047d0 0x7f68c014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 -- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c00c6170 msgr2=0x7f68c00c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c00c6170 0x7f68c00c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 -- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a80047d0 msgr2=0x7f68c014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:55.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a80047d0 0x7f68c014dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:55.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 -- 172.21.15.71:0/2343318820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c00c6f10 con 0x7f68c0156740 2023-10-26T05:52:55.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c4f3f640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 0x7f68c0156bc0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f68c00d0c30 tx=0x7f68b800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:55.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b8004d30 con 0x7f68c0156740 2023-10-26T05:52:55.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f68b800dec0 con 0x7f68c0156740 2023-10-26T05:52:55.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c00c71f0 con 0x7f68c0156740 2023-10-26T05:52:55.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b800bc60 con 0x7f68c0156740 2023-10-26T05:52:55.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.320+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c00c77b0 con 0x7f68c0156740 2023-10-26T05:52:55.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.324+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c0066b10 con 0x7f68c0156740 2023-10-26T05:52:55.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.324+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68b8010460 con 0x7f68c0156740 2023-10-26T05:52:55.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.324+0000 7f68bcff9640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68ac066470 0x7f68ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:55.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.324+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f68b80838e0 con 0x7f68c0156740 2023-10-26T05:52:55.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.324+0000 7f68bffff640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68ac066470 0x7f68ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:55.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.324+0000 7f68bffff640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68ac066470 0x7f68ac068930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f68b0007fd0 tx=0x7f68b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:55.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.328+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f68b8054660 con 0x7f68c0156740 2023-10-26T05:52:55.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:55.500+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f68c00b5170 con 0x7f68c0156740 2023-10-26T05:52:56.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.288+0000 7f68bcff9640 1 -- 172.21.15.71:0/2343318820 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f68b80190a0 con 0x7f68c0156740 2023-10-26T05:52:56.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68ac066470 msgr2=0x7f68ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:56.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68ac066470 0x7f68ac068930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f68b0007fd0 tx=0x7f68b0009040 comp rx=0 tx=0).stop 2023-10-26T05:52:56.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 msgr2=0x7f68c0156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:56.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 0x7f68c0156bc0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f68c00d0c30 tx=0x7f68b800dd90 comp rx=0 tx=0).stop 2023-10-26T05:52:56.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 shutdown_connections 2023-10-26T05:52:56.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f68ac066470 0x7f68ac068930 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68c0156740 0x7f68c0156bc0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68c00c6170 0x7f68c00c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 --2- 172.21.15.71:0/2343318820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68a80047d0 0x7f68c014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 >> 172.21.15.71:0/2343318820 conn(0x7f68c00cdd80 msgr2=0x7f68c00b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 shutdown_connections 2023-10-26T05:52:56.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.292+0000 7f68c69a2640 1 -- 172.21.15.71:0/2343318820 wait complete. 2023-10-26T05:52:56.321 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:52:56.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- 172.21.15.71:0/128787356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 msgr2=0x7f158013df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 --2- 172.21.15.71:0/128787356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f158013df20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f1578009c80 tx=0x7f157801cc80 comp rx=0 tx=0).stop 2023-10-26T05:52:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- 172.21.15.71:0/128787356 shutdown_connections 2023-10-26T05:52:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 --2- 172.21.15.71:0/128787356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f158013df20 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 --2- 172.21.15.71:0/128787356 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15800ca590 0x7f158015c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- 172.21.15.71:0/128787356 >> 172.21.15.71:0/128787356 conn(0x7f1580070d10 msgr2=0x7f158006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- 172.21.15.71:0/128787356 shutdown_connections 2023-10-26T05:52:56.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- 172.21.15.71:0/128787356 wait complete. 2023-10-26T05:52:56.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 Processor -- start 2023-10-26T05:52:56.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- start start 2023-10-26T05:52:56.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15800ca590 0x7f15801627f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:56.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15800ca970 0x7f15800ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:56.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f1580161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:56.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1580155000 con 0x7f1580158da0 2023-10-26T05:52:56.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801360e0 con 0x7f15800ca590 2023-10-26T05:52:56.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f1585c98640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1580162d30 con 0x7f15800ca970 2023-10-26T05:52:56.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f157ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f1580161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:56.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f157ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f1580161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44864/0 (socket says 172.21.15.71:44864) 2023-10-26T05:52:56.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.440+0000 7f157ffff640 1 -- 172.21.15.71:0/1818918309 learned_addr learned my addr 172.21.15.71:0/1818918309 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157effd640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15800ca970 0x7f15800ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157f7fe640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15800ca590 0x7f15801627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157ffff640 1 -- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15800ca970 msgr2=0x7f15800ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157ffff640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15800ca970 0x7f15800ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157ffff640 1 -- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15800ca590 msgr2=0x7f15801627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:56.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157ffff640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15800ca590 0x7f15801627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:56.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157ffff640 1 -- 172.21.15.71:0/1818918309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780098e0 con 0x7f1580158da0 2023-10-26T05:52:56.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157f7fe640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15800ca590 0x7f15801627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:56.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157ffff640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f1580161150 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f157801d520 tx=0x7f15780093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:56.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578004410 con 0x7f1580158da0 2023-10-26T05:52:56.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1578004660 con 0x7f1580158da0 2023-10-26T05:52:56.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15801354d0 con 0x7f1580158da0 2023-10-26T05:52:56.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15780a5c70 con 0x7f1580158da0 2023-10-26T05:52:56.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1580135a90 con 0x7f1580158da0 2023-10-26T05:52:56.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1580066b10 con 0x7f1580158da0 2023-10-26T05:52:56.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f15780b4040 con 0x7f1580158da0 2023-10-26T05:52:56.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1567fff640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f156c066470 0x7f156c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:56.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f157811ef00 con 0x7f1580158da0 2023-10-26T05:52:56.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157f7fe640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f156c066470 0x7f156c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:56.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.444+0000 7f157f7fe640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f156c066470 0x7f156c068930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1570005fd0 tx=0x7f1570005d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:56.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.448+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f15780ec320 con 0x7f1580158da0 2023-10-26T05:52:56.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:56.592+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1580066b10 con 0x7f1580158da0 2023-10-26T05:52:57.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.312+0000 7f1567fff640 1 -- 172.21.15.71:0/1818918309 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f15780b2020 con 0x7f1580158da0 2023-10-26T05:52:57.318 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:52:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f156c066470 msgr2=0x7f156c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f156c066470 0x7f156c068930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1570005fd0 tx=0x7f1570005d40 comp rx=0 tx=0).stop 2023-10-26T05:52:57.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 msgr2=0x7f1580161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f1580161150 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f157801d520 tx=0x7f15780093d0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 shutdown_connections 2023-10-26T05:52:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f156c066470 0x7f156c068930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1580158da0 0x7f1580161150 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f15800ca970 0x7f15800ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 --2- 172.21.15.71:0/1818918309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15800ca590 0x7f15801627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 >> 172.21.15.71:0/1818918309 conn(0x7f1580070d10 msgr2=0x7f158006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:57.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 shutdown_connections 2023-10-26T05:52:57.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.320+0000 7f1585c98640 1 -- 172.21.15.71:0/1818918309 wait complete. 2023-10-26T05:52:57.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:52:57.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- 172.21.15.71:0/891148611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 msgr2=0x7fe9c4144590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:57.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/891148611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4144590 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009bc0 tx=0x7fe9bc01cb10 comp rx=0 tx=0).stop 2023-10-26T05:52:57.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- 172.21.15.71:0/891148611 shutdown_connections 2023-10-26T05:52:57.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/891148611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4144590 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- 172.21.15.71:0/891148611 >> 172.21.15.71:0/891148611 conn(0x7fe9c406f440 msgr2=0x7fe9c406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:57.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- 172.21.15.71:0/891148611 shutdown_connections 2023-10-26T05:52:57.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- 172.21.15.71:0/891148611 wait complete. 2023-10-26T05:52:57.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 Processor -- start 2023-10-26T05:52:57.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- start start 2023-10-26T05:52:57.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:57.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c40cf6f0 0x7fe9c406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:57.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9c41441b0 0x7fe9c406e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:57.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4141520 con 0x7fe9c406c520 2023-10-26T05:52:57.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4140bf0 con 0x7fe9c40cf6f0 2023-10-26T05:52:57.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9cbf81640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4140ed0 con 0x7fe9c41441b0 2023-10-26T05:52:57.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9c9d1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:57.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9c9d1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44888/0 (socket says 172.21.15.71:44888) 2023-10-26T05:52:57.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.472+0000 7fe9c9d1d640 1 -- 172.21.15.71:0/1106903583 learned_addr learned my addr 172.21.15.71:0/1106903583 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:57.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ca51e640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9c41441b0 0x7fe9c406e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:57.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c951c640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c40cf6f0 0x7fe9c406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:57.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c9d1d640 1 -- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9c41441b0 msgr2=0x7fe9c406e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:57.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c9d1d640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9c41441b0 0x7fe9c406e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c9d1d640 1 -- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c40cf6f0 msgr2=0x7fe9c406e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:57.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c9d1d640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c40cf6f0 0x7fe9c406e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:57.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c9d1d640 1 -- 172.21.15.71:0/1106903583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009820 con 0x7fe9c406c520 2023-10-26T05:52:57.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c9d1d640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4158160 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c390 tx=0x7fe9c000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:57.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0004230 con 0x7fe9c406c520 2023-10-26T05:52:57.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9c00044a0 con 0x7fe9c406c520 2023-10-26T05:52:57.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c41411b0 con 0x7fe9c406c520 2023-10-26T05:52:57.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0003840 con 0x7fe9c406c520 2023-10-26T05:52:57.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c4143240 con 0x7fe9c406c520 2023-10-26T05:52:57.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c4066b10 con 0x7fe9c406c520 2023-10-26T05:52:57.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9c00039a0 con 0x7fe9c406c520 2023-10-26T05:52:57.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ba7fc640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9a8066470 0x7fe9a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:57.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe9c0083500 con 0x7fe9c406c520 2023-10-26T05:52:57.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c951c640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9a8066470 0x7fe9a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:57.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.476+0000 7fe9c951c640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9a8066470 0x7fe9a8068930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9c415bfe0 tx=0x7fe9b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:57.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.480+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe9c0050ae0 con 0x7fe9c406c520 2023-10-26T05:52:57.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:57.636+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9c40721e0 con 0x7fe9c406c520 2023-10-26T05:52:58.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.324+0000 7fe9ba7fc640 1 -- 172.21.15.71:0/1106903583 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe9c0054100 con 0x7fe9c406c520 2023-10-26T05:52:58.330 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:52:58.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9a8066470 msgr2=0x7fe9a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9a8066470 0x7fe9a8068930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe9c415bfe0 tx=0x7fe9b4009040 comp rx=0 tx=0).stop 2023-10-26T05:52:58.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 msgr2=0x7fe9c4158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4158160 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c390 tx=0x7fe9c000c850 comp rx=0 tx=0).stop 2023-10-26T05:52:58.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 shutdown_connections 2023-10-26T05:52:58.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9c41441b0 0x7fe9c406e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9c40cf6f0 0x7fe9c406e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9a8066470 0x7fe9a8068930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 --2- 172.21.15.71:0/1106903583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9c406c520 0x7fe9c4158160 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 >> 172.21.15.71:0/1106903583 conn(0x7fe9c406f440 msgr2=0x7fe9c413c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:58.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 shutdown_connections 2023-10-26T05:52:58.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.332+0000 7fe9cbf81640 1 -- 172.21.15.71:0/1106903583 wait complete. 2023-10-26T05:52:58.364 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:52:58.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/1030846929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e40047d0 msgr2=0x7fd7e4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/1030846929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7e40047d0 0x7fd7e4004c50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fb0 tx=0x7fd7e001cfc0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7f77fe640 1 -- 172.21.15.71:0/1030846929 reap_dead start 2023-10-26T05:52:58.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/1030846929 shutdown_connections 2023-10-26T05:52:58.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/1030846929 >> 172.21.15.71:0/1030846929 conn(0x7fd7f80d12d0 msgr2=0x7fd7f80d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:58.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/1030846929 shutdown_connections 2023-10-26T05:52:58.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.476+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/1030846929 wait complete. 2023-10-26T05:52:58.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 Processor -- start 2023-10-26T05:52:58.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- start start 2023-10-26T05:52:58.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e40047d0 0x7fd7f8151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 0x7fd7f815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7f81f5110 0x7fd7f81f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81f1ad0 con 0x7fd7f80cb010 2023-10-26T05:52:58.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81f5ae0 con 0x7fd7e40047d0 2023-10-26T05:52:58.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f815dd50 con 0x7fd7f81f5110 2023-10-26T05:52:58.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 0x7fd7f815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 0x7fd7f815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44894/0 (socket says 172.21.15.71:44894) 2023-10-26T05:52:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 -- 172.21.15.71:0/879598346 learned_addr learned my addr 172.21.15.71:0/879598346 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f77fe640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7f81f5110 0x7fd7f81f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 -- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7f81f5110 msgr2=0x7fd7f81f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f6ffd640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e40047d0 0x7fd7f8151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7f81f5110 0x7fd7f81f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 -- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e40047d0 msgr2=0x7fd7f8151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e40047d0 0x7fd7f8151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 -- 172.21.15.71:0/879598346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f815e030 con 0x7fd7f80cb010 2023-10-26T05:52:58.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f77fe640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7f81f5110 0x7fd7f81f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:52:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7f67fc640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 0x7fd7f815d4c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0001940 tx=0x7fd7e00060c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00b1030 con 0x7fd7f80cb010 2023-10-26T05:52:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd7e00065a0 con 0x7fd7f80cb010 2023-10-26T05:52:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f815e310 con 0x7fd7f80cb010 2023-10-26T05:52:58.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e00b4040 con 0x7fd7f80cb010 2023-10-26T05:52:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f80c8ae0 con 0x7fd7f80cb010 2023-10-26T05:52:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8066b10 con 0x7fd7f80cb010 2023-10-26T05:52:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd7e0003820 con 0x7fd7f80cb010 2023-10-26T05:52:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7ef7fe640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd7d8066470 0x7fd7d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.480+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd7e011dde0 con 0x7fd7f80cb010 2023-10-26T05:52:58.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.484+0000 7fd7f6ffd640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd7d8066470 0x7fd7d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.484+0000 7fd7f6ffd640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd7d8066470 0x7fd7d8068930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00058d0 tx=0x7fd7f00094d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:58.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.488+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd7e00eb560 con 0x7fd7f80cb010 2023-10-26T05:52:58.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.644+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7fd7f815e4a0 con 0x7fd7f80cb010 2023-10-26T05:52:58.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7ef7fe640 1 -- 172.21.15.71:0/879598346 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v28) v1 ==== 71+0+0 (secure 0 0 0) 0x7fd7e00bf020 con 0x7fd7f80cb010 2023-10-26T05:52:58.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd7d8066470 msgr2=0x7fd7d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd7d8066470 0x7fd7d8068930 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00058d0 tx=0x7fd7f00094d0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 msgr2=0x7fd7f815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 0x7fd7f815d4c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0001940 tx=0x7fd7e00060c0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 shutdown_connections 2023-10-26T05:52:58.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd7f81f5110 0x7fd7f81f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd7d8066470 0x7fd7d8068930 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd7f80cb010 0x7fd7f815d4c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 --2- 172.21.15.71:0/879598346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7e40047d0 0x7fd7f8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 >> 172.21.15.71:0/879598346 conn(0x7fd7f80d12d0 msgr2=0x7fd7f80b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:58.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 shutdown_connections 2023-10-26T05:52:58.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.656+0000 7fd7fd3ee640 1 -- 172.21.15.71:0/879598346 wait complete. 2023-10-26T05:52:58.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-10-26T05:52:58.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 -- 172.21.15.71:0/3785060589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc0005190 msgr2=0x7facc000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 --2- 172.21.15.71:0/3785060589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc0005190 0x7facc000b990 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7facc8001da0 tx=0x7facc801cb20 comp rx=0 tx=0).stop 2023-10-26T05:52:58.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 -- 172.21.15.71:0/3785060589 shutdown_connections 2023-10-26T05:52:58.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 --2- 172.21.15.71:0/3785060589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc0005190 0x7facc000b990 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 -- 172.21.15.71:0/3785060589 >> 172.21.15.71:0/3785060589 conn(0x7facd00cdd80 msgr2=0x7facd00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:58.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 -- 172.21.15.71:0/3785060589 shutdown_connections 2023-10-26T05:52:58.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.804+0000 7facd6f6a640 1 -- 172.21.15.71:0/3785060589 wait complete. 2023-10-26T05:52:58.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 Processor -- start 2023-10-26T05:52:58.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- start start 2023-10-26T05:52:58.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc0005190 0x7facd01f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 0x7facd00c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facd015a190 0x7facd00c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd006a650 con 0x7facd0159d10 2023-10-26T05:52:58.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd00c6ea0 con 0x7facc0005190 2023-10-26T05:52:58.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facd01f2ac0 con 0x7facd015a190 2023-10-26T05:52:58.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 0x7facd00c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 0x7facd00c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44914/0 (socket says 172.21.15.71:44914) 2023-10-26T05:52:58.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 -- 172.21.15.71:0/1218500897 learned_addr learned my addr 172.21.15.71:0/1218500897 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:58.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd4d06640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc0005190 0x7facd01f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd5507640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facd015a190 0x7facd00c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 -- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facd015a190 msgr2=0x7facd00c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facd015a190 0x7facd00c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 -- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc0005190 msgr2=0x7facd01f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:58.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc0005190 0x7facd01f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:58.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 -- 172.21.15.71:0/1218500897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00c6220 con 0x7facd0159d10 2023-10-26T05:52:58.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd5507640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facd015a190 0x7facd00c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:58.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccffff640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 0x7facd00c83d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7facbc011850 tx=0x7facbc011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:58.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc004460 con 0x7facd0159d10 2023-10-26T05:52:58.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7facbc00fa30 con 0x7facd0159d10 2023-10-26T05:52:58.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd00c6500 con 0x7facd0159d10 2023-10-26T05:52:58.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00fdc0 con 0x7facd0159d10 2023-10-26T05:52:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd00c6ac0 con 0x7facd0159d10 2023-10-26T05:52:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd0066b10 con 0x7facd0159d10 2023-10-26T05:52:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7facbc018070 con 0x7facd0159d10 2023-10-26T05:52:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccd7fa640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7facb4066470 0x7facb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.808+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7facbc084ed0 con 0x7facd0159d10 2023-10-26T05:52:58.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.812+0000 7facd4d06640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7facb4066470 0x7facb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:58.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.812+0000 7facd4d06640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7facb4066470 0x7facb4068930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7facc4000fc0 tx=0x7facc4006730 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:58.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:58.812+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7facbc0526a0 con 0x7facd0159d10 2023-10-26T05:52:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.024+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7facd00721e0 con 0x7facd0159d10 2023-10-26T05:52:59.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7faccd7fa640 1 -- 172.21.15.71:0/1218500897 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v303) v1 ==== 137+0+0 (secure 0 0 0) 0x7facbc055cc0 con 0x7facd0159d10 2023-10-26T05:52:59.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7facb4066470 msgr2=0x7facb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7facb4066470 0x7facb4068930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7facc4000fc0 tx=0x7facc4006730 comp rx=0 tx=0).stop 2023-10-26T05:52:59.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 msgr2=0x7facd00c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 0x7facd00c83d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7facbc011850 tx=0x7facbc011d10 comp rx=0 tx=0).stop 2023-10-26T05:52:59.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 shutdown_connections 2023-10-26T05:52:59.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7facd015a190 0x7facd00c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7facb4066470 0x7facb4068930 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd0159d10 0x7facd00c83d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 --2- 172.21.15.71:0/1218500897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc0005190 0x7facd01f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 >> 172.21.15.71:0/1218500897 conn(0x7facd00cdd80 msgr2=0x7facd015e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:59.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 shutdown_connections 2023-10-26T05:52:59.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.328+0000 7facd6f6a640 1 -- 172.21.15.71:0/1218500897 wait complete. 2023-10-26T05:52:59.356 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-10-26T05:52:59.357 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-10-26T05:52:59.357 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T05:52:59.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:52:59.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- 172.21.15.71:0/1203149606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b5013c2f0 msgr2=0x7f9b500b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 --2- 172.21.15.71:0/1203149606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b5013c2f0 0x7f9b500b2350 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9b500ac6e0 tx=0x7f9b4401c650 comp rx=0 tx=0).stop 2023-10-26T05:52:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- 172.21.15.71:0/1203149606 shutdown_connections 2023-10-26T05:52:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 --2- 172.21.15.71:0/1203149606 >> v2:172.21.15.86:3300/0 conn(0x7f9b500cb1e0 0x7f9b50159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 --2- 172.21.15.71:0/1203149606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b5013c2f0 0x7f9b500b2350 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- 172.21.15.71:0/1203149606 >> 172.21.15.71:0/1203149606 conn(0x7f9b50069770 msgr2=0x7f9b50069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- 172.21.15.71:0/1203149606 shutdown_connections 2023-10-26T05:52:59.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- 172.21.15.71:0/1203149606 wait complete. 2023-10-26T05:52:59.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 Processor -- start 2023-10-26T05:52:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- start start 2023-10-26T05:52:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b500b2890 0x7f9b50137c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 0x7f9b5013c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b50152ac0 0x7f9b50152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500b1030 con 0x7f9b500cb1e0 2023-10-26T05:52:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50153490 con 0x7f9b500b2890 2023-10-26T05:52:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b56102640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b50138170 con 0x7f9b50152ac0 2023-10-26T05:52:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 0x7f9b5013c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 0x7f9b5013c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44940/0 (socket says 172.21.15.71:44940) 2023-10-26T05:52:59.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 -- 172.21.15.71:0/4267597031 learned_addr learned my addr 172.21.15.71:0/4267597031 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4ffff640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b50152ac0 0x7f9b50152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 -- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b50152ac0 msgr2=0x7f9b50152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b50152ac0 0x7f9b50152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 -- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b500b2890 msgr2=0x7f9b50137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4f7fe640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b500b2890 0x7f9b50137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b500b2890 0x7f9b50137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.480+0000 7f9b4effd640 1 -- 172.21.15.71:0/4267597031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440094a0 con 0x7f9b500cb1e0 2023-10-26T05:52:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b4effd640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 0x7f9b5013c820 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ec0 tx=0x7f9b4000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40008ab0 con 0x7f9b500cb1e0 2023-10-26T05:52:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b40008d00 con 0x7f9b500cb1e0 2023-10-26T05:52:59.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b500b03b0 con 0x7f9b500cb1e0 2023-10-26T05:52:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40009290 con 0x7f9b500cb1e0 2023-10-26T05:52:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b500b0970 con 0x7f9b500cb1e0 2023-10-26T05:52:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b50066b10 con 0x7f9b500cb1e0 2023-10-26T05:52:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b400093f0 con 0x7f9b500cb1e0 2023-10-26T05:52:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b37fff640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b3c066470 0x7f9b3c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9b40086610 con 0x7f9b500cb1e0 2023-10-26T05:52:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b4f7fe640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b3c066470 0x7f9b3c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.484+0000 7f9b4f7fe640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b3c066470 0x7f9b3c068930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9b440af000 tx=0x7f9b4401ae90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:59.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.488+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b40053e20 con 0x7f9b500cb1e0 2023-10-26T05:52:59.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.644+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b500b1170 con 0x7f9b500cb1e0 2023-10-26T05:52:59.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.644+0000 7f9b37fff640 1 -- 172.21.15.71:0/4267597031 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+12913 (secure 0 0 0) 0x7f9b500b1170 con 0x7f9b500cb1e0 2023-10-26T05:52:59.652 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:52:59.653 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":115,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:52:58.315826+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/2440514167":"2023-10-27T05:52:55.184388+0000","172.21.15.71:6834/2440514167":"2023-10-27T05:52:55.184388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:52:59.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b3c066470 msgr2=0x7f9b3c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b3c066470 0x7f9b3c068930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9b440af000 tx=0x7f9b4401ae90 comp rx=0 tx=0).stop 2023-10-26T05:52:59.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 msgr2=0x7f9b5013c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 0x7f9b5013c820 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009ec0 tx=0x7f9b4000c430 comp rx=0 tx=0).stop 2023-10-26T05:52:59.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 shutdown_connections 2023-10-26T05:52:59.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b50152ac0 0x7f9b50152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b3c066470 0x7f9b3c068930 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b500cb1e0 0x7f9b5013c820 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 --2- 172.21.15.71:0/4267597031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b500b2890 0x7f9b50137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 >> 172.21.15.71:0/4267597031 conn(0x7f9b50069770 msgr2=0x7f9b50069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:59.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 shutdown_connections 2023-10-26T05:52:59.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.648+0000 7f9b56102640 1 -- 172.21.15.71:0/4267597031 wait complete. 2023-10-26T05:52:59.672 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:52:59.672 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:52:59.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 -- 172.21.15.71:0/461034799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 msgr2=0x7f0ef40cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 --2- 172.21.15.71:0/461034799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40cbb10 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009900 tx=0x7f0ee401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 -- 172.21.15.71:0/461034799 shutdown_connections 2023-10-26T05:52:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 --2- 172.21.15.71:0/461034799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40cbb10 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 --2- 172.21.15.71:0/461034799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef4142b30 0x7f0ef40b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 -- 172.21.15.71:0/461034799 >> 172.21.15.71:0/461034799 conn(0x7f0ef406f440 msgr2=0x7f0ef406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 -- 172.21.15.71:0/461034799 shutdown_connections 2023-10-26T05:52:59.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 -- 172.21.15.71:0/461034799 wait complete. 2023-10-26T05:52:59.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 Processor -- start 2023-10-26T05:52:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.796+0000 7f0efacc7640 1 -- start start 2023-10-26T05:52:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef40cc050 0x7f0ef40aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef4142b30 0x7f0ef4151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4159240 con 0x7f0ef40af990 2023-10-26T05:52:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef406c4a0 con 0x7f0ef4142b30 2023-10-26T05:52:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4158c80 con 0x7f0ef40cc050 2023-10-26T05:52:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44956/0 (socket says 172.21.15.71:44956) 2023-10-26T05:52:59.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 -- 172.21.15.71:0/88196049 learned_addr learned my addr 172.21.15.71:0/88196049 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:52:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0eebfff640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef40cc050 0x7f0ef40aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef9264640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef4142b30 0x7f0ef4151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 -- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef40cc050 msgr2=0x7f0ef40aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef40cc050 0x7f0ef40aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 -- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef4142b30 msgr2=0x7f0ef4151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef4142b30 0x7f0ef4151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 -- 172.21.15.71:0/88196049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4009560 con 0x7f0ef40af990 2023-10-26T05:52:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef9264640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef4142b30 0x7f0ef4151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0eebfff640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef40cc050 0x7f0ef40aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:52:59.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ef8a63640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40ae760 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c9f0 tx=0x7f0ef000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0007bb0 con 0x7f0ef40af990 2023-10-26T05:52:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ef0007e00 con 0x7f0ef40af990 2023-10-26T05:52:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4158f60 con 0x7f0ef40af990 2023-10-26T05:52:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000f6b0 con 0x7f0ef40af990 2023-10-26T05:52:59.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef406ca20 con 0x7f0ef40af990 2023-10-26T05:52:59.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4066b10 con 0x7f0ef40af990 2023-10-26T05:52:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0ef000f810 con 0x7f0ef40af990 2023-10-26T05:52:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ee97fa640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0ee0066470 0x7f0ee0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:52:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0ef0084450 con 0x7f0ef40af990 2023-10-26T05:52:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.800+0000 7f0eebfff640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0ee0066470 0x7f0ee0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:52:59.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.804+0000 7f0eebfff640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0ee0066470 0x7f0ee0068930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40af000 tx=0x7f0ee401b000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:52:59.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.804+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0ef0051bd0 con 0x7f0ef40af990 2023-10-26T05:52:59.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ef4143070 con 0x7f0ef40af990 2023-10-26T05:52:59.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0ee97fa640 1 -- 172.21.15.71:0/88196049 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ef00551f0 con 0x7f0ef40af990 2023-10-26T05:52:59.970 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:52:59.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0ee0066470 msgr2=0x7f0ee0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0ee0066470 0x7f0ee0068930 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40af000 tx=0x7f0ee401b000 comp rx=0 tx=0).stop 2023-10-26T05:52:59.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 msgr2=0x7f0ef40ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:52:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40ae760 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000c9f0 tx=0x7f0ef000ceb0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 shutdown_connections 2023-10-26T05:52:59.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ef4142b30 0x7f0ef4151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ef40cc050 0x7f0ef40aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0ee0066470 0x7f0ee0068930 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 --2- 172.21.15.71:0/88196049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ef40af990 0x7f0ef40ae760 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:52:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 >> 172.21.15.71:0/88196049 conn(0x7f0ef406f440 msgr2=0x7f0ef41534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:52:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 shutdown_connections 2023-10-26T05:52:59.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:52:59.964+0000 7f0efacc7640 1 -- 172.21.15.71:0/88196049 wait complete. 2023-10-26T05:52:59.989 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-26T05:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.104+0000 7f353c5e8640 1 -- 172.21.15.71:0/1618040712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 msgr2=0x7f353415b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.104+0000 7f353c5e8640 1 --2- 172.21.15.71:0/1618040712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353415b8e0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f35340ccd20 tx=0x7f352c01cbe0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- 172.21.15.71:0/1618040712 shutdown_connections 2023-10-26T05:53:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 --2- 172.21.15.71:0/1618040712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353415b8e0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 --2- 172.21.15.71:0/1618040712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 0x7f353415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- 172.21.15.71:0/1618040712 >> 172.21.15.71:0/1618040712 conn(0x7f35340cdd80 msgr2=0x7f35340ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:00.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- 172.21.15.71:0/1618040712 shutdown_connections 2023-10-26T05:53:00.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- 172.21.15.71:0/1618040712 wait complete. 2023-10-26T05:53:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 Processor -- start 2023-10-26T05:53:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- start start 2023-10-26T05:53:00.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35340d0470 0x7f3534159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 0x7f353415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534151410 con 0x7f35340cd880 2023-10-26T05:53:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3534150b70 con 0x7f35340d0470 2023-10-26T05:53:00.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3534150e50 con 0x7f35340d0850 2023-10-26T05:53:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44974/0 (socket says 172.21.15.71:44974) 2023-10-26T05:53:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 -- 172.21.15.71:0/2060269505 learned_addr learned my addr 172.21.15.71:0/2060269505 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353ab85640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 0x7f353415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f3539b83640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35340d0470 0x7f3534159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 -- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 msgr2=0x7f353415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 0x7f353415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 -- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35340d0470 msgr2=0x7f3534159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35340d0470 0x7f3534159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 -- 172.21.15.71:0/2060269505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0098e0 con 0x7f35340cd880 2023-10-26T05:53:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353ab85640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 0x7f353415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f3539b83640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35340d0470 0x7f3534159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353a384640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353414f800 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f353000c450 tx=0x7f353000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:00.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530012830 con 0x7f35340cd880 2023-10-26T05:53:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3530012aa0 con 0x7f35340cd880 2023-10-26T05:53:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3534151130 con 0x7f35340cd880 2023-10-26T05:53:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3530011380 con 0x7f35340cd880 2023-10-26T05:53:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.108+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3534151820 con 0x7f35340cd880 2023-10-26T05:53:00.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.112+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3534066b10 con 0x7f35340cd880 2023-10-26T05:53:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.112+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f35300114e0 con 0x7f35340cd880 2023-10-26T05:53:00.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.112+0000 7f352affd640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3518066470 0x7f3518068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.112+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3530083bd0 con 0x7f35340cd880 2023-10-26T05:53:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.112+0000 7f3539b83640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3518066470 0x7f3518068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.112+0000 7f3539b83640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3518066470 0x7f3518068930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3524005fd0 tx=0x7f3524005dc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:00.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.116+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f35300513e0 con 0x7f35340cd880 2023-10-26T05:53:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.320+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f35340721e0 con 0x7f35340cd880 2023-10-26T05:53:00.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f352affd640 1 -- 172.21.15.71:0/2060269505 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v304) v1 ==== 131+0+0 (secure 0 0 0) 0x7f3530054a00 con 0x7f35340cd880 2023-10-26T05:53:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3518066470 msgr2=0x7f3518068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3518066470 0x7f3518068930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3524005fd0 tx=0x7f3524005dc0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 msgr2=0x7f353414f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353414f800 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f353000c450 tx=0x7f353000c910 comp rx=0 tx=0).stop 2023-10-26T05:53:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 shutdown_connections 2023-10-26T05:53:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35340d0850 0x7f353415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35340d0470 0x7f3534159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3518066470 0x7f3518068930 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 --2- 172.21.15.71:0/2060269505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35340cd880 0x7f353414f800 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 >> 172.21.15.71:0/2060269505 conn(0x7f35340cdd80 msgr2=0x7f35340b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 shutdown_connections 2023-10-26T05:53:00.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.340+0000 7f353c5e8640 1 -- 172.21.15.71:0/2060269505 wait complete. 2023-10-26T05:53:00.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:53:00.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/351183477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 msgr2=0x7f92fc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 --2- 172.21.15.71:0/351183477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f92fc003db0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f930000c260 tx=0x7f930001cfc0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930ed9d640 1 -- 172.21.15.71:0/351183477 reap_dead start 2023-10-26T05:53:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/351183477 shutdown_connections 2023-10-26T05:53:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/351183477 >> 172.21.15.71:0/351183477 conn(0x7f9310069770 msgr2=0x7f9310069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/351183477 shutdown_connections 2023-10-26T05:53:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/351183477 wait complete. 2023-10-26T05:53:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 Processor -- start 2023-10-26T05:53:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- start start 2023-10-26T05:53:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f931013b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931014fdb0 0x7f931015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9310160050 0x7f9310150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93101604d0 con 0x7f92fc0039b0 2023-10-26T05:53:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931015a830 con 0x7f931014fdb0 2023-10-26T05:53:00.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931014eff0 con 0x7f9310160050 2023-10-26T05:53:00.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f931013b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f931013b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44998/0 (socket says 172.21.15.71:44998) 2023-10-26T05:53:00.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 -- 172.21.15.71:0/2863288983 learned_addr learned my addr 172.21.15.71:0/2863288983 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9305d9b640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931014fdb0 0x7f931015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 -- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9310160050 msgr2=0x7f9310150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9310160050 0x7f9310150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 -- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931014fdb0 msgr2=0x7f931015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931014fdb0 0x7f931015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 -- 172.21.15.71:0/2863288983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931014f2d0 con 0x7f92fc0039b0 2023-10-26T05:53:00.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9305d9b640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931014fdb0 0x7f931015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:00.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f930e59c640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f931013b8c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f930001cdf0 tx=0x7f9300000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:00.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93000b3070 con 0x7f92fc0039b0 2023-10-26T05:53:00.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f93000063a0 con 0x7f92fc0039b0 2023-10-26T05:53:00.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931014f5b0 con 0x7f92fc0039b0 2023-10-26T05:53:00.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300003620 con 0x7f92fc0039b0 2023-10-26T05:53:00.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.468+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931013be00 con 0x7f92fc0039b0 2023-10-26T05:53:00.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.472+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9310066b10 con 0x7f92fc0039b0 2023-10-26T05:53:00.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.472+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f93000ab020 con 0x7f92fc0039b0 2023-10-26T05:53:00.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.472+0000 7f93077fe640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92f0066470 0x7f92f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.472+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f930011d310 con 0x7f92fc0039b0 2023-10-26T05:53:00.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.472+0000 7f9305d9b640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92f0066470 0x7f92f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.472+0000 7f9305d9b640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92f0066470 0x7f92f0068930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92f8006fd0 tx=0x7f92f8006b90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:00.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.476+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f93000eaa90 con 0x7f92fc0039b0 2023-10-26T05:53:00.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f93100721e0 con 0x7f92fc0039b0 2023-10-26T05:53:00.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f93077fe640 1 -- 172.21.15.71:0/2863288983 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v28) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f93000bc080 con 0x7f92fc0039b0 2023-10-26T05:53:00.626 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:53:00.627 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:53:00.627 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:53:00.627 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:53:00.627 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:53:00.627 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:53:00.628 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:53:00.628 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:53:00.628 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:53:00.628 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:53:00.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:53:00.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:53:00.629 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:53:00.629 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:53:00.630 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:53:00.630 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:53:00.630 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:53:00.630 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:53:00.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92f0066470 msgr2=0x7f92f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92f0066470 0x7f92f0068930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f92f8006fd0 tx=0x7f92f8006b90 comp rx=0 tx=0).stop 2023-10-26T05:53:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 msgr2=0x7f931013b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f931013b8c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f930001cdf0 tx=0x7f9300000f90 comp rx=0 tx=0).stop 2023-10-26T05:53:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 shutdown_connections 2023-10-26T05:53:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9310160050 0x7f9310150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f931014fdb0 0x7f931015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92f0066470 0x7f92f0068930 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 --2- 172.21.15.71:0/2863288983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92fc0039b0 0x7f931013b8c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 >> 172.21.15.71:0/2863288983 conn(0x7f9310069770 msgr2=0x7f93100c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 shutdown_connections 2023-10-26T05:53:00.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.620+0000 7f9314bea640 1 -- 172.21.15.71:0/2863288983 wait complete. 2023-10-26T05:53:00.644 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:53:00.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 -- 172.21.15.71:0/267176430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b80cede0 msgr2=0x7fc8b815ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/267176430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b80cede0 0x7fc8b815ce50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009900 tx=0x7fc8a001c800 comp rx=0 tx=0).stop 2023-10-26T05:53:00.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 -- 172.21.15.71:0/267176430 shutdown_connections 2023-10-26T05:53:00.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/267176430 >> v2:172.21.15.86:3300/0 conn(0x7fc8b815d390 0x7fc8b815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/267176430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b80cede0 0x7fc8b815ce50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 -- 172.21.15.71:0/267176430 >> 172.21.15.71:0/267176430 conn(0x7fc8b80b0f20 msgr2=0x7fc8b80b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:00.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 -- 172.21.15.71:0/267176430 shutdown_connections 2023-10-26T05:53:00.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 -- 172.21.15.71:0/267176430 wait complete. 2023-10-26T05:53:00.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 Processor -- start 2023-10-26T05:53:00.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.756+0000 7fc8bd48e640 1 -- start start 2023-10-26T05:53:00.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b80cede0 0x7fc8b813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b815c720 0x7fc8b80c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 0x7fc8b814f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80c7f50 con 0x7fc8b815d390 2023-10-26T05:53:00.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80c8230 con 0x7fc8b80cede0 2023-10-26T05:53:00.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b80c8510 con 0x7fc8b815c720 2023-10-26T05:53:00.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 0x7fc8b814f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 0x7fc8b814f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45000/0 (socket says 172.21.15.71:45000) 2023-10-26T05:53:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 -- 172.21.15.71:0/1921126482 learned_addr learned my addr 172.21.15.71:0/1921126482 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8ae7fc640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b815c720 0x7fc8b80c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b6ffd640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b80cede0 0x7fc8b813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 -- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b815c720 msgr2=0x7fc8b80c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b815c720 0x7fc8b80c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 -- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b80cede0 msgr2=0x7fc8b813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b80cede0 0x7fc8b813fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 -- 172.21.15.71:0/1921126482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0009560 con 0x7fc8b815d390 2023-10-26T05:53:00.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8ae7fc640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b815c720 0x7fc8b80c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:53:00.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b77fe640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 0x7fc8b814f3a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ea80 tx=0x7fc8b000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:00.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000cea0 con 0x7fc8b815d390 2023-10-26T05:53:00.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b80c87f0 con 0x7fc8b815d390 2023-10-26T05:53:00.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc8b0004b30 con 0x7fc8b815d390 2023-10-26T05:53:00.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0005020 con 0x7fc8b815d390 2023-10-26T05:53:00.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b8140230 con 0x7fc8b815d390 2023-10-26T05:53:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b8066d00 con 0x7fc8b815d390 2023-10-26T05:53:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc8b0005180 con 0x7fc8b815d390 2023-10-26T05:53:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8affff640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc898066470 0x7fc898068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc8b0014070 con 0x7fc8b815d390 2023-10-26T05:53:00.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.760+0000 7fc8b6ffd640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc898066470 0x7fc898068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:00.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.764+0000 7fc8b6ffd640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc898066470 0x7fc898068930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8001650 tx=0x7fc8a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:00.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.764+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc8b0052060 con 0x7fc8b815d390 2023-10-26T05:53:00.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.908+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc8b815cf30 con 0x7fc8b815d390 2023-10-26T05:53:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8affff640 1 -- 172.21.15.71:0/1921126482 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v29)=0 v29) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc8b0055680 con 0x7fc8b815d390 2023-10-26T05:53:00.925 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc898066470 msgr2=0x7fc898068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc898066470 0x7fc898068930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8001650 tx=0x7fc8a8009040 comp rx=0 tx=0).stop 2023-10-26T05:53:00.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 msgr2=0x7fc8b814f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:00.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 0x7fc8b814f3a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000ea80 tx=0x7fc8b000ef40 comp rx=0 tx=0).stop 2023-10-26T05:53:00.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 shutdown_connections 2023-10-26T05:53:00.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc898066470 0x7fc898068930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b815d390 0x7fc8b814f3a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8b815c720 0x7fc8b80c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 --2- 172.21.15.71:0/1921126482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b80cede0 0x7fc8b813fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:00.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 >> 172.21.15.71:0/1921126482 conn(0x7fc8b80b0f20 msgr2=0x7fc8b81404d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:00.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 shutdown_connections 2023-10-26T05:53:00.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:00.916+0000 7fc8bd48e640 1 -- 172.21.15.71:0/1921126482 wait complete. 2023-10-26T05:53:00.940 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:53:00.940 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:53:00.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:53:01.047 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:53:01.048 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:01.048 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:01.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:53:01.054 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:53:01.054 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:01.062 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:53:01.063 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:01.063 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:01.101 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:53:01.101 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:53:01.102 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:01.110 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:53:01.111 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:53:01.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:01.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- 172.21.15.71:0/1532394873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e00052b0 msgr2=0x7f87e000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 --2- 172.21.15.71:0/1532394873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e00052b0 0x7f87e000b990 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f87ec002240 tx=0x7f87ec01fa80 comp rx=0 tx=0).stop 2023-10-26T05:53:01.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- 172.21.15.71:0/1532394873 shutdown_connections 2023-10-26T05:53:01.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 --2- 172.21.15.71:0/1532394873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87e00052b0 0x7f87e000b990 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- 172.21.15.71:0/1532394873 >> 172.21.15.71:0/1532394873 conn(0x7f87f40cdd80 msgr2=0x7f87f40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:01.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- 172.21.15.71:0/1532394873 shutdown_connections 2023-10-26T05:53:01.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- 172.21.15.71:0/1532394873 wait complete. 2023-10-26T05:53:01.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 Processor -- start 2023-10-26T05:53:01.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- start start 2023-10-26T05:53:01.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e00052b0 0x7f87f414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 0x7f87f414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f414d330 0x7f87f414d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f406a650 con 0x7f87f40a7e70 2023-10-26T05:53:01.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f415b1b0 con 0x7f87e00052b0 2023-10-26T05:53:01.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87fa34d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f415e190 con 0x7f87f414d330 2023-10-26T05:53:01.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 0x7f87f414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87ebfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 0x7f87f414cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45008/0 (socket says 172.21.15.71:45008) 2023-10-26T05:53:01.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87ebfff640 1 -- 172.21.15.71:0/101463086 learned_addr learned my addr 172.21.15.71:0/101463086 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:01.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87f3fff640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e00052b0 0x7f87f414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.264+0000 7f87f88ea640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f414d330 0x7f87f414d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87ebfff640 1 -- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f414d330 msgr2=0x7f87f414d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87ebfff640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f414d330 0x7f87f414d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87ebfff640 1 -- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e00052b0 msgr2=0x7f87f414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87ebfff640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e00052b0 0x7f87f414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87ebfff640 1 -- 172.21.15.71:0/101463086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f415deb0 con 0x7f87f40a7e70 2023-10-26T05:53:01.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87ebfff640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 0x7f87f414cdf0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00fa00 tx=0x7f87dc00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:01.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00bc50 con 0x7f87f40a7e70 2023-10-26T05:53:01.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f87dc011440 con 0x7f87f40a7e70 2023-10-26T05:53:01.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f414af60 con 0x7f87f40a7e70 2023-10-26T05:53:01.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00bea0 con 0x7f87f40a7e70 2023-10-26T05:53:01.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f414b520 con 0x7f87f40a7e70 2023-10-26T05:53:01.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f4066b10 con 0x7f87f40a7e70 2023-10-26T05:53:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87dc011830 con 0x7f87f40a7e70 2023-10-26T05:53:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f17fa640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87d40664a0 0x7f87d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f87dc084780 con 0x7f87f40a7e70 2023-10-26T05:53:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f3fff640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87d40664a0 0x7f87d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.268+0000 7f87f3fff640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87d40664a0 0x7f87d4068960 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f87e4000f70 tx=0x7f87e4006ad0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.272+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f87dc051e70 con 0x7f87f40a7e70 2023-10-26T05:53:01.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.460+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87f414adc0 con 0x7f87f40a7e70 2023-10-26T05:53:01.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.460+0000 7f87f17fa640 1 -- 172.21.15.71:0/101463086 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f87f414adc0 con 0x7f87f40a7e70 2023-10-26T05:53:01.468 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:01.468 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5375,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/305622372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":305622372},{"type":"v1","addr":"172.21.15.86:6835","nonce":305622372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3052964706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3052964706},{"type":"v1","addr":"172.21.15.86:6837","nonce":3052964706}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7071,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1291887689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1291887689},{"type":"v1","addr":"172.21.15.71:6833","nonce":1291887689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7467,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3102632834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3102632834},{"type":"v1","addr":"172.21.15.71:6835","nonce":3102632834}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[]} 2023-10-26T05:53:01.469 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 40 2023-10-26T05:53:01.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87d40664a0 msgr2=0x7f87d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87d40664a0 0x7f87d4068960 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f87e4000f70 tx=0x7f87e4006ad0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 msgr2=0x7f87f414cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 0x7f87f414cdf0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00fa00 tx=0x7f87dc00fec0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 shutdown_connections 2023-10-26T05:53:01.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87f414d330 0x7f87f414d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f87d40664a0 0x7f87d4068960 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87f40a7e70 0x7f87f414cdf0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 --2- 172.21.15.71:0/101463086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87e00052b0 0x7f87f414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 >> 172.21.15.71:0/101463086 conn(0x7f87f40cdd80 msgr2=0x7f87f40b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:01.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 shutdown_connections 2023-10-26T05:53:01.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.464+0000 7f87fa34d640 1 -- 172.21.15.71:0/101463086 wait complete. 2023-10-26T05:53:01.486 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:53:01.486 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:53:01.486 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:53:01.486 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:01.487 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:53:01.487 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:53:01.487 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:53:01.488 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:01.488 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:53:01.488 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:53:01.488 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:53:01.489 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:01.489 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:53:01.489 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:53:01.489 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:53:01.490 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:01.496 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:53:01.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:53:01.498 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:53:01.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:53:01.503 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:53:01.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:53:01.506 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:53:01.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:53:01.542 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:53:01.542 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:01.543 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:53:01.543 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:01.550 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:53:01.551 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:01.552 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:53:01.553 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:01.553 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:53:01.600 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:53:01.605 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:53:01.617 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:53:01.620 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:53:01.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2335905527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 msgr2=0x7f4e88004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2335905527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e88004830 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4e840017c0 tx=0x7f4e8401f010 comp rx=0 tx=0).stop 2023-10-26T05:53:01.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9b59e640 1 -- 172.21.15.71:0/2335905527 reap_dead start 2023-10-26T05:53:01.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2335905527 shutdown_connections 2023-10-26T05:53:01.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2335905527 >> 172.21.15.71:0/2335905527 conn(0x7f4e9c0d1330 msgr2=0x7f4e9c0d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:01.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2335905527 shutdown_connections 2023-10-26T05:53:01.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2335905527 wait complete. 2023-10-26T05:53:01.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 Processor -- start 2023-10-26T05:53:01.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- start start 2023-10-26T05:53:01.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e9c149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e9c0c8400 0x7f4e9c15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e9c158090 0x7f4e9c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0c9510 con 0x7f4e88004360 2023-10-26T05:53:01.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c149ea0 con 0x7f4e9c158090 2023-10-26T05:53:01.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0c8890 con 0x7f4e9c0c8400 2023-10-26T05:53:01.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e9c149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e9c149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52768/0 (socket says 172.21.15.71:52768) 2023-10-26T05:53:01.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 -- 172.21.15.71:0/2317516939 learned_addr learned my addr 172.21.15.71:0/2317516939 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:01.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9a59c640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e9c0c8400 0x7f4e9c15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9b59e640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e9c158090 0x7f4e9c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 -- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e9c0c8400 msgr2=0x7f4e9c15a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e9c0c8400 0x7f4e9c15a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 -- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e9c158090 msgr2=0x7f4e9c15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:01.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e9c158090 0x7f4e9c15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:01.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 -- 172.21.15.71:0/2317516939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0c8b70 con 0x7f4e88004360 2023-10-26T05:53:01.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e9ad9d640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e9c149960 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4e940062c0 tx=0x7f4e94006780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:01.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400fcc0 con 0x7f4e88004360 2023-10-26T05:53:01.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e9400fe20 con 0x7f4e88004360 2023-10-26T05:53:01.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c0c8e50 con 0x7f4e88004360 2023-10-26T05:53:01.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94011690 con 0x7f4e88004360 2023-10-26T05:53:01.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.664+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c0c9930 con 0x7f4e88004360 2023-10-26T05:53:01.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.668+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c066b10 con 0x7f4e88004360 2023-10-26T05:53:01.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.668+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4e940117f0 con 0x7f4e88004360 2023-10-26T05:53:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.668+0000 7f4e937fe640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e7c066470 0x7f4e7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.668+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4e94084b00 con 0x7f4e88004360 2023-10-26T05:53:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.668+0000 7f4e9a59c640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e7c066470 0x7f4e7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.668+0000 7f4e9a59c640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e7c066470 0x7f4e7c068930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e84001790 tx=0x7f4e840b3040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.672+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4e94052330 con 0x7f4e88004360 2023-10-26T05:53:01.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-26T05:53:01.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:01.824+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4e9c0c9780 con 0x7f4e88004360 2023-10-26T05:53:03.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4e937fe640 1 -- 172.21.15.71:0/2317516939 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4e94020080 con 0x7f4e88004360 2023-10-26T05:53:03.194 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:53:03.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e7c066470 msgr2=0x7f4e7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e7c066470 0x7f4e7c068930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4e84001790 tx=0x7f4e840b3040 comp rx=0 tx=0).stop 2023-10-26T05:53:03.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 msgr2=0x7f4e9c149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e9c149960 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4e940062c0 tx=0x7f4e94006780 comp rx=0 tx=0).stop 2023-10-26T05:53:03.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 shutdown_connections 2023-10-26T05:53:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e9c158090 0x7f4e9c15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e9c0c8400 0x7f4e9c15a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e7c066470 0x7f4e7c068930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 --2- 172.21.15.71:0/2317516939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e88004360 0x7f4e9c149960 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 >> 172.21.15.71:0/2317516939 conn(0x7f4e9c0d1330 msgr2=0x7f4e9c15fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:03.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 shutdown_connections 2023-10-26T05:53:03.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.188+0000 7f4ea1126640 1 -- 172.21.15.71:0/2317516939 wait complete. 2023-10-26T05:53:03.216 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:53:03.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/1122125822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 msgr2=0x7f4808003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 --2- 172.21.15.71:0/1122125822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f4808003da0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f481000bb40 tx=0x7f481001cdc0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481ffff640 1 -- 172.21.15.71:0/1122125822 reap_dead start 2023-10-26T05:53:03.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/1122125822 shutdown_connections 2023-10-26T05:53:03.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/1122125822 >> 172.21.15.71:0/1122125822 conn(0x7f4820069770 msgr2=0x7f4820069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:03.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/1122125822 shutdown_connections 2023-10-26T05:53:03.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/1122125822 wait complete. 2023-10-26T05:53:03.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 Processor -- start 2023-10-26T05:53:03.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- start start 2023-10-26T05:53:03.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f48201377a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48201564f0 0x7f48201548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820154e20 0x7f482015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4820156060 con 0x7f4808003980 2023-10-26T05:53:03.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482013b3b0 con 0x7f4820154e20 2023-10-26T05:53:03.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4820155450 con 0x7f48201564f0 2023-10-26T05:53:03.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f48201377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f48201377a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52786/0 (socket says 172.21.15.71:52786) 2023-10-26T05:53:03.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 -- 172.21.15.71:0/4269390725 learned_addr learned my addr 172.21.15.71:0/4269390725 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:03.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481ffff640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820154e20 0x7f482015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481effd640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48201564f0 0x7f48201548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 -- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48201564f0 msgr2=0x7f48201548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48201564f0 0x7f48201548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 -- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820154e20 msgr2=0x7f482015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820154e20 0x7f482015b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 -- 172.21.15.71:0/4269390725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4820155730 con 0x7f4808003980 2023-10-26T05:53:03.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481effd640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48201564f0 0x7f48201548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481ffff640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820154e20 0x7f482015b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f481f7fe640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f48201377a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f48100b1040 tx=0x7f4810005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810004480 con 0x7f4808003980 2023-10-26T05:53:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4810005040 con 0x7f4808003980 2023-10-26T05:53:03.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4820155a10 con 0x7f4808003980 2023-10-26T05:53:03.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810005450 con 0x7f4808003980 2023-10-26T05:53:03.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.340+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4820137ce0 con 0x7f4808003980 2023-10-26T05:53:03.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.344+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820066b10 con 0x7f4808003980 2023-10-26T05:53:03.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.344+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48100046d0 con 0x7f4808003980 2023-10-26T05:53:03.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.344+0000 7f4807fff640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f480c066470 0x7f480c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.344+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f481011e120 con 0x7f4808003980 2023-10-26T05:53:03.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.344+0000 7f481effd640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f480c066470 0x7f480c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.344+0000 7f481effd640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f480c066470 0x7f480c068930 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f48200aea30 tx=0x7f48140094d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:03.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.348+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48100eb7d0 con 0x7f4808003980 2023-10-26T05:53:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.552+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4820155200 con 0x7f4808003980 2023-10-26T05:53:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.552+0000 7f4807fff640 1 -- 172.21.15.71:0/4269390725 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f48100ed700 con 0x7f4808003980 2023-10-26T05:53:03.560 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:03.560 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:03.560 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:53:03.561 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.561 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:53:03.561 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:53:03.561 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.561 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.562 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:03.562 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:03.562 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.562 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.563 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.563 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:53:03.563 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:53:03.563 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.564 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.564 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:03.564 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:03.564 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.564 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.565 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.565 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:53:03.565 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:53:03.565 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.566 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.566 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:03.566 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:03.566 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.566 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.567 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.567 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.567 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:53:03.567 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:53:03.567 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.567 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.568 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:03.568 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:03.568 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.568 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.569 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.569 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:53:03.569 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:53:03.569 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.569 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.570 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.570 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.570 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.571 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:53:03.571 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:53:03.571 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.571 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.571 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.572 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.573 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:53:03.573 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:53:03.573 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.573 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.573 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.574 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.574 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.574 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:53:03.575 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:53:03.575 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.575 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.575 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.575 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.576 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.576 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:53:03.576 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:53:03.576 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.577 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.577 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.577 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.577 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.578 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.578 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:53:03.578 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:53:03.578 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.578 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.578 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.579 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.579 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.579 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.579 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:53:03.580 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:53:03.580 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.580 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.580 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.581 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:53:03.582 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:53:03.582 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.582 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:03.582 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:03.583 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.583 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.583 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.583 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:53:03.584 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:53:03.584 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.584 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.584 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:53:03.584 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:53:03.585 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.585 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.585 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:53:03.586 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:53:03.586 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.586 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.586 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:53:03.586 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:53:03.587 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:53:03.587 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.587 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.588 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:53:03.588 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:53:03.588 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.588 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.588 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:53:03.589 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:53:03.589 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:53:03.589 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.589 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.590 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:53:03.590 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:53:03.590 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.590 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:03.591 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:53:03.591 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:53:03.591 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:53:03.591 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.592 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.592 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:53:03.592 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:53:03.592 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.592 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:53:03.593 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:03.593 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:03.593 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.593 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.593 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.594 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:53:03.594 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:53:03.594 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.594 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:53:03.595 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:53:03.595 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.595 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:03.595 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:03.596 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:53:03.596 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:53:03.596 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:03.596 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:53:03.596 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:53:03.597 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:03.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:03.597 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:03.597 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:03.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f480c066470 msgr2=0x7f480c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f480c066470 0x7f480c068930 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f48200aea30 tx=0x7f48140094d0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 msgr2=0x7f48201377a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f48201377a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f48100b1040 tx=0x7f4810005900 comp rx=0 tx=0).stop 2023-10-26T05:53:03.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 shutdown_connections 2023-10-26T05:53:03.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4820154e20 0x7f482015b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f48201564f0 0x7f48201548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f480c066470 0x7f480c068930 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 --2- 172.21.15.71:0/4269390725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4808003980 0x7f48201377a0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 >> 172.21.15.71:0/4269390725 conn(0x7f4820069770 msgr2=0x7f482006a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:03.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 shutdown_connections 2023-10-26T05:53:03.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.556+0000 7f4825ceb640 1 -- 172.21.15.71:0/4269390725 wait complete. 2023-10-26T05:53:03.602 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:03.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2317740958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7680cb010 msgr2=0x7fa768159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2317740958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7680cb010 0x7fa768159810 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa7680cc8d0 tx=0x7fa76001cd40 comp rx=0 tx=0).stop 2023-10-26T05:53:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2317740958 shutdown_connections 2023-10-26T05:53:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2317740958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7680cb010 0x7fa768159810 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2317740958 >> v2:172.21.15.86:3300/0 conn(0x7fa7680b2a60 0x7fa7680caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2317740958 >> 172.21.15.71:0/2317740958 conn(0x7fa768069770 msgr2=0x7fa768069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2317740958 shutdown_connections 2023-10-26T05:53:03.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2317740958 wait complete. 2023-10-26T05:53:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 Processor -- start 2023-10-26T05:53:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- start start 2023-10-26T05:53:03.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7680b2a60 0x7fa76814e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7680cb010 0x7fa76813d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 0x7fa768152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7681391a0 con 0x7fa76813c2f0 2023-10-26T05:53:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa768151920 con 0x7fa7680cb010 2023-10-26T05:53:03.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76fd8b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7681503c0 con 0x7fa7680b2a60 2023-10-26T05:53:03.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 0x7fa768152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 0x7fa768152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52800/0 (socket says 172.21.15.71:52800) 2023-10-26T05:53:03.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 -- 172.21.15.71:0/2942600037 learned_addr learned my addr 172.21.15.71:0/2942600037 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:03.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76d326640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7680cb010 0x7fa76813d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76db27640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7680b2a60 0x7fa76814e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 -- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7680b2a60 msgr2=0x7fa76814e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7680b2a60 0x7fa76814e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 -- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7680cb010 msgr2=0x7fa76813d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7680cb010 0x7fa76813d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76e328640 1 -- 172.21.15.71:0/2942600037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7600098e0 con 0x7fa76813c2f0 2023-10-26T05:53:03.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.720+0000 7fa76db27640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7680b2a60 0x7fa76814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:03.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa76e328640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 0x7fa768152270 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa760019950 tx=0x7fa7600a5a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:03.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760004e60 con 0x7fa76813c2f0 2023-10-26T05:53:03.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa760004000 con 0x7fa76813c2f0 2023-10-26T05:53:03.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76814e900 con 0x7fa76813c2f0 2023-10-26T05:53:03.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600043f0 con 0x7fa76813c2f0 2023-10-26T05:53:03.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa768150ff0 con 0x7fa76813c2f0 2023-10-26T05:53:03.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa768066b10 con 0x7fa76813c2f0 2023-10-26T05:53:03.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa7600b4040 con 0x7fa76813c2f0 2023-10-26T05:53:03.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa75e7fc640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa74c066470 0x7fa74c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:03.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa76011deb0 con 0x7fa76813c2f0 2023-10-26T05:53:03.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa76db27640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa74c066470 0x7fa74c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:03.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.724+0000 7fa76db27640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa74c066470 0x7fa74c068930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa764000f70 tx=0x7fa764009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:03.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.728+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa7600eb680 con 0x7fa76813c2f0 2023-10-26T05:53:03.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.872+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa768151a90 con 0x7fa76813c2f0 2023-10-26T05:53:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.872+0000 7fa75e7fc640 1 -- 172.21.15.71:0/2942600037 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+12784 (secure 0 0 0) 0x7fa7600b2020 con 0x7fa76813c2f0 2023-10-26T05:53:03.880 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:03.880 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":120,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:03.187755+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa74c066470 msgr2=0x7fa74c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa74c066470 0x7fa74c068930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa764000f70 tx=0x7fa764009040 comp rx=0 tx=0).stop 2023-10-26T05:53:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 msgr2=0x7fa768152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 0x7fa768152270 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa760019950 tx=0x7fa7600a5a10 comp rx=0 tx=0).stop 2023-10-26T05:53:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 shutdown_connections 2023-10-26T05:53:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa74c066470 0x7fa74c068930 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa76813c2f0 0x7fa768152270 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7680cb010 0x7fa76813d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 --2- 172.21.15.71:0/2942600037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7680b2a60 0x7fa76814e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 >> 172.21.15.71:0/2942600037 conn(0x7fa768069770 msgr2=0x7fa76815c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 shutdown_connections 2023-10-26T05:53:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:03.876+0000 7fa76fd8b640 1 -- 172.21.15.71:0/2942600037 wait complete. 2023-10-26T05:53:03.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:53:03.898 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:53:04.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/669875697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 msgr2=0x7f1d1006f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 --2- 172.21.15.71:0/669875697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 0x7f1d1006f540 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009840 tx=0x7f1d0001c740 comp rx=0 tx=0).stop 2023-10-26T05:53:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/669875697 shutdown_connections 2023-10-26T05:53:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 --2- 172.21.15.71:0/669875697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d100affd0 0x7f1d1006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 --2- 172.21.15.71:0/669875697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 0x7f1d1006f540 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/669875697 >> 172.21.15.71:0/669875697 conn(0x7f1d100aec60 msgr2=0x7f1d100af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:04.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/669875697 shutdown_connections 2023-10-26T05:53:04.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/669875697 wait complete. 2023-10-26T05:53:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 Processor -- start 2023-10-26T05:53:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- start start 2023-10-26T05:53:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 0x7f1d10150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d100affd0 0x7f1d1013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 0x7f1d100cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1013ed90 con 0x7f1d1006a540 2023-10-26T05:53:04.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10151a30 con 0x7f1d10157620 2023-10-26T05:53:04.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10150e20 con 0x7f1d100affd0 2023-10-26T05:53:04.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 0x7f1d10150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 0x7f1d10150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52802/0 (socket says 172.21.15.71:52802) 2023-10-26T05:53:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 -- 172.21.15.71:0/4225118454 learned_addr learned my addr 172.21.15.71:0/4225118454 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 -- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d100affd0 msgr2=0x7f1d1013e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d100affd0 0x7f1d1013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 -- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 msgr2=0x7f1d100cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 0x7f1d100cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 -- 172.21.15.71:0/4225118454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000094a0 con 0x7f1d1006a540 2023-10-26T05:53:04.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d0f7fe640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 0x7f1d10150790 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1d0001cc20 tx=0x7f1d000a5610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:04.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000bc070 con 0x7f1d1006a540 2023-10-26T05:53:04.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d00005040 con 0x7f1d1006a540 2023-10-26T05:53:04.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d10151100 con 0x7f1d1006a540 2023-10-26T05:53:04.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d000053f0 con 0x7f1d1006a540 2023-10-26T05:53:04.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.016+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d101516c0 con 0x7f1d1006a540 2023-10-26T05:53:04.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.020+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d10066b10 con 0x7f1d1006a540 2023-10-26T05:53:04.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.020+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1d000b4040 con 0x7f1d1006a540 2023-10-26T05:53:04.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.020+0000 7f1cf7fff640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cfc066470 0x7f1cfc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.020+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1d0011de80 con 0x7f1d1006a540 2023-10-26T05:53:04.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.020+0000 7f1d0effd640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cfc066470 0x7f1cfc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.020+0000 7f1d0effd640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cfc066470 0x7f1cfc068930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1d100cdcb0 tx=0x7f1d0400a4d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:04.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.024+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1d000eb6d0 con 0x7f1d1006a540 2023-10-26T05:53:04.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.176+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d100721e0 con 0x7f1d1006a540 2023-10-26T05:53:04.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.176+0000 7f1cf7fff640 1 -- 172.21.15.71:0/4225118454 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d000b2070 con 0x7f1d1006a540 2023-10-26T05:53:04.183 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:53:04.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.176+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cfc066470 msgr2=0x7f1cfc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.176+0000 7f1d1605f640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cfc066470 0x7f1cfc068930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1d100cdcb0 tx=0x7f1d0400a4d0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.176+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 msgr2=0x7f1d10150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 0x7f1d10150790 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1d0001cc20 tx=0x7f1d000a5610 comp rx=0 tx=0).stop 2023-10-26T05:53:04.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 shutdown_connections 2023-10-26T05:53:04.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d10157620 0x7f1d100cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d100affd0 0x7f1d1013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1cfc066470 0x7f1cfc068930 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 --2- 172.21.15.71:0/4225118454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d1006a540 0x7f1d10150790 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 >> 172.21.15.71:0/4225118454 conn(0x7f1d100aec60 msgr2=0x7f1d100abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:04.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 shutdown_connections 2023-10-26T05:53:04.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.180+0000 7f1d1605f640 1 -- 172.21.15.71:0/4225118454 wait complete. 2023-10-26T05:53:04.202 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:53:04.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 -- 172.21.15.71:0/578269078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740d0470 msgr2=0x7f117406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 --2- 172.21.15.71:0/578269078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740d0470 0x7f117406aa70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f11740c4dd0 tx=0x7f116401c720 comp rx=0 tx=0).stop 2023-10-26T05:53:04.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 -- 172.21.15.71:0/578269078 shutdown_connections 2023-10-26T05:53:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 --2- 172.21.15.71:0/578269078 >> v2:172.21.15.86:3300/0 conn(0x7f11740d0850 0x7f117415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 --2- 172.21.15.71:0/578269078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740d0470 0x7f117406aa70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 -- 172.21.15.71:0/578269078 >> 172.21.15.71:0/578269078 conn(0x7f11740cdd80 msgr2=0x7f11740ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 -- 172.21.15.71:0/578269078 shutdown_connections 2023-10-26T05:53:04.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.316+0000 7f117a218640 1 -- 172.21.15.71:0/578269078 wait complete. 2023-10-26T05:53:04.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 Processor -- start 2023-10-26T05:53:04.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- start start 2023-10-26T05:53:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 0x7f1174154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11740d0470 0x7f1174154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11740d0850 0x7f117415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117415b170 con 0x7f11740cd880 2023-10-26T05:53:04.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117415e1f0 con 0x7f11740d0850 2023-10-26T05:53:04.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117415b3c0 con 0x7f11740d0470 2023-10-26T05:53:04.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 0x7f1174154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 0x7f1174154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52818/0 (socket says 172.21.15.71:52818) 2023-10-26T05:53:04.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 -- 172.21.15.71:0/423412709 learned_addr learned my addr 172.21.15.71:0/423412709 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f1172ffd640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11740d0470 0x7f1174154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f1173fff640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11740d0850 0x7f117415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 -- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11740d0470 msgr2=0x7f1174154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11740d0470 0x7f1174154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 -- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11740d0850 msgr2=0x7f117415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11740d0850 0x7f117415c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 -- 172.21.15.71:0/423412709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640094a0 con 0x7f11740cd880 2023-10-26T05:53:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f11737fe640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 0x7f1174154320 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1164000c10 tx=0x7f11640a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164004230 con 0x7f11740cd880 2023-10-26T05:53:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1164004480 con 0x7f11740cd880 2023-10-26T05:53:04.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117415b6a0 con 0x7f11740cd880 2023-10-26T05:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164003840 con 0x7f11740cd880 2023-10-26T05:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117415bc60 con 0x7f11740cd880 2023-10-26T05:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1174066b10 con 0x7f11740cd880 2023-10-26T05:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11640b4040 con 0x7f11740cd880 2023-10-26T05:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f115bfff640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1160066470 0x7f1160068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f116411d810 con 0x7f11740cd880 2023-10-26T05:53:04.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.320+0000 7f1172ffd640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1160066470 0x7f1160068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.324+0000 7f1172ffd640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1160066470 0x7f1160068930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f117415d3c0 tx=0x7f11680091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:04.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.324+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11640eafe0 con 0x7f11740cd880 2023-10-26T05:53:04.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f11740721e0 con 0x7f11740cd880 2023-10-26T05:53:04.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f115bfff640 1 -- 172.21.15.71:0/423412709 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+65 (secure 0 0 0) 0x7f11640b2090 con 0x7f11740cd880 2023-10-26T05:53:04.479 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:04.479 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:53:04.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:04.479 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:53:04.480 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:53:04.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:04.480 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:53:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1160066470 msgr2=0x7f1160068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1160066470 0x7f1160068930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f117415d3c0 tx=0x7f11680091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 msgr2=0x7f1174154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 0x7f1174154320 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1164000c10 tx=0x7f11640a5710 comp rx=0 tx=0).stop 2023-10-26T05:53:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 shutdown_connections 2023-10-26T05:53:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11740d0850 0x7f117415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11740d0470 0x7f1174154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1160066470 0x7f1160068930 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 --2- 172.21.15.71:0/423412709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11740cd880 0x7f1174154320 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 >> 172.21.15.71:0/423412709 conn(0x7f11740cdd80 msgr2=0x7f11740b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:04.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 shutdown_connections 2023-10-26T05:53:04.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.472+0000 7f117a218640 1 -- 172.21.15.71:0/423412709 wait complete. 2023-10-26T05:53:04.495 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:53:04.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:53:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/4065255603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 msgr2=0x7f7ff00b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/4065255603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff00b13a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009870 tx=0x7f7fe801c6b0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/4065255603 shutdown_connections 2023-10-26T05:53:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/4065255603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff00b13a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/4065255603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ff015fc40 0x7f7ff015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/4065255603 >> 172.21.15.71:0/4065255603 conn(0x7f7ff0070cb0 msgr2=0x7f7ff006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:04.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/4065255603 shutdown_connections 2023-10-26T05:53:04.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.620+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/4065255603 wait complete. 2023-10-26T05:53:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 Processor -- start 2023-10-26T05:53:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- start start 2023-10-26T05:53:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 0x7f7ff0156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ff015fc40 0x7f7ff006c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff014e930 con 0x7f7ff0155bd0 2023-10-26T05:53:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0153450 con 0x7f7ff015f570 2023-10-26T05:53:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff0156b00 con 0x7f7ff015fc40 2023-10-26T05:53:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 0x7f7ff0156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 0x7f7ff0156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52832/0 (socket says 172.21.15.71:52832) 2023-10-26T05:53:04.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 -- 172.21.15.71:0/3002870503 learned_addr learned my addr 172.21.15.71:0/3002870503 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:04.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fedd9b640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7feed9d640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ff015fc40 0x7f7ff006c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 -- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ff015fc40 msgr2=0x7f7ff006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ff015fc40 0x7f7ff006c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 -- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 msgr2=0x7f7ff014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 -- 172.21.15.71:0/3002870503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80094a0 con 0x7f7ff0155bd0 2023-10-26T05:53:04.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fedd9b640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:53:04.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fee59c640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 0x7f7ff0156520 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cbc0 tx=0x7f7fe000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:04.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0003b80 con 0x7f7ff0155bd0 2023-10-26T05:53:04.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7fe0003dd0 con 0x7f7ff0155bd0 2023-10-26T05:53:04.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff0156de0 con 0x7f7ff0155bd0 2023-10-26T05:53:04.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000aa50 con 0x7f7ff0155bd0 2023-10-26T05:53:04.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff01573a0 con 0x7f7ff0155bd0 2023-10-26T05:53:04.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.624+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff01535c0 con 0x7f7ff0155bd0 2023-10-26T05:53:04.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.628+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7fe000abb0 con 0x7f7ff0155bd0 2023-10-26T05:53:04.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.628+0000 7f7fe6ffd640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fd0066470 0x7f7fd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:04.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.628+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7fe0084240 con 0x7f7ff0155bd0 2023-10-26T05:53:04.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.628+0000 7f7fedd9b640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fd0066470 0x7f7fd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:04.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.628+0000 7f7fedd9b640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fd0066470 0x7f7fd0068930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00aadd0 tx=0x7f7fd80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:04.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.632+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7fe0051a20 con 0x7f7ff0155bd0 2023-10-26T05:53:04.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:04.784+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7ff0066d00 con 0x7f7ff0155bd0 2023-10-26T05:53:05.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.368+0000 7f7fe6ffd640 1 -- 172.21.15.71:0/3002870503 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7fe000ae60 con 0x7f7ff0155bd0 2023-10-26T05:53:05.377 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:53:05.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fd0066470 msgr2=0x7f7fd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:05.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fd0066470 0x7f7fd0068930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7ff00aadd0 tx=0x7f7fd80091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 msgr2=0x7f7ff0156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:05.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 0x7f7ff0156520 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cbc0 tx=0x7f7fe000ccb0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 shutdown_connections 2023-10-26T05:53:05.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7ff015fc40 0x7f7ff006c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff015f570 0x7f7ff014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fd0066470 0x7f7fd0068930 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 --2- 172.21.15.71:0/3002870503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ff0155bd0 0x7f7ff0156520 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 >> 172.21.15.71:0/3002870503 conn(0x7f7ff0070cb0 msgr2=0x7f7ff015d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:05.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 shutdown_connections 2023-10-26T05:53:05.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.372+0000 7f7ff4c8f640 1 -- 172.21.15.71:0/3002870503 wait complete. 2023-10-26T05:53:05.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:53:05.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1919552827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e54003980 msgr2=0x7f1e54003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1919552827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e54003980 0x7f1e54003da0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1e58005fd0 tx=0x7f1e5801cd50 comp rx=0 tx=0).stop 2023-10-26T05:53:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6759e640 1 -- 172.21.15.71:0/1919552827 reap_dead start 2023-10-26T05:53:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1919552827 shutdown_connections 2023-10-26T05:53:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1919552827 >> 172.21.15.71:0/1919552827 conn(0x7f1e680aec60 msgr2=0x7f1e680af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1919552827 shutdown_connections 2023-10-26T05:53:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.528+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1919552827 wait complete. 2023-10-26T05:53:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 Processor -- start 2023-10-26T05:53:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- start start 2023-10-26T05:53:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e54003980 0x7f1e6815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e6813ace0 0x7f1e6806a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:05.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 0x7f1e68160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:05.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68160ac0 con 0x7f1e68160100 2023-10-26T05:53:05.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68138e30 con 0x7f1e54003980 2023-10-26T05:53:05.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e68138820 con 0x7f1e6813ace0 2023-10-26T05:53:05.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 0x7f1e68160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 0x7f1e68160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52848/0 (socket says 172.21.15.71:52848) 2023-10-26T05:53:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 -- 172.21.15.71:0/1631376024 learned_addr learned my addr 172.21.15.71:0/1631376024 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e66d9d640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e54003980 0x7f1e6815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:05.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e5ffff640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e6813ace0 0x7f1e6806a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 -- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e6813ace0 msgr2=0x7f1e6806a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e6813ace0 0x7f1e6806a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 -- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e54003980 msgr2=0x7f1e6815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e54003980 0x7f1e6815a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 -- 172.21.15.71:0/1631376024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6813b4b0 con 0x7f1e68160100 2023-10-26T05:53:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6759e640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 0x7f1e68160580 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1e68070140 tx=0x7f1e6000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60004b30 con 0x7f1e68160100 2023-10-26T05:53:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1e60004d80 con 0x7f1e68160100 2023-10-26T05:53:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e68138ac0 con 0x7f1e68160100 2023-10-26T05:53:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000bc80 con 0x7f1e68160100 2023-10-26T05:53:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6813b790 con 0x7f1e68160100 2023-10-26T05:53:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e68066b10 con 0x7f1e68160100 2023-10-26T05:53:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.532+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1e60010460 con 0x7f1e68160100 2023-10-26T05:53:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.536+0000 7f1e5f7fe640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1e48066470 0x7f1e48068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.536+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1e600836d0 con 0x7f1e68160100 2023-10-26T05:53:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.536+0000 7f1e66d9d640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1e48066470 0x7f1e48068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.536+0000 7f1e66d9d640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1e48066470 0x7f1e48068930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e580b0000 tx=0x7f1e580a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.540+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1e60050d30 con 0x7f1e68160100 2023-10-26T05:53:05.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:05.692+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1e68138fc0 con 0x7f1e68160100 2023-10-26T05:53:06.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.376+0000 7f1e5f7fe640 1 -- 172.21.15.71:0/1631376024 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1e60054350 con 0x7f1e68160100 2023-10-26T05:53:06.382 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:53:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1e48066470 msgr2=0x7f1e48068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1e48066470 0x7f1e48068930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e580b0000 tx=0x7f1e580a7040 comp rx=0 tx=0).stop 2023-10-26T05:53:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 msgr2=0x7f1e68160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 0x7f1e68160580 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1e68070140 tx=0x7f1e6000dd10 comp rx=0 tx=0).stop 2023-10-26T05:53:06.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 shutdown_connections 2023-10-26T05:53:06.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1e48066470 0x7f1e48068930 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e68160100 0x7f1e68160580 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1e6813ace0 0x7f1e6806a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 --2- 172.21.15.71:0/1631376024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e54003980 0x7f1e6815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 >> 172.21.15.71:0/1631376024 conn(0x7f1e680aec60 msgr2=0x7f1e680cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:06.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 shutdown_connections 2023-10-26T05:53:06.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.380+0000 7f1e6d1e2640 1 -- 172.21.15.71:0/1631376024 wait complete. 2023-10-26T05:53:06.418 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:53:06.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/2029259722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891414e7b0 msgr2=0x7f8914157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:06.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 --2- 172.21.15.71:0/2029259722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891414e7b0 0x7f8914157db0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f8904009900 tx=0x7f890401c880 comp rx=0 tx=0).stop 2023-10-26T05:53:06.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/2029259722 shutdown_connections 2023-10-26T05:53:06.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 --2- 172.21.15.71:0/2029259722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f891406a000 0x7f891406a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 --2- 172.21.15.71:0/2029259722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891414e7b0 0x7f8914157db0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/2029259722 >> 172.21.15.71:0/2029259722 conn(0x7f89140aec60 msgr2=0x7f89140af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:06.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/2029259722 shutdown_connections 2023-10-26T05:53:06.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/2029259722 wait complete. 2023-10-26T05:53:06.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 Processor -- start 2023-10-26T05:53:06.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- start start 2023-10-26T05:53:06.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 0x7f8914142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:06.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8914071270 0x7f8914141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:06.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f891414e7b0 0x7f891413dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:06.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891413ec30 con 0x7f891406a000 2023-10-26T05:53:06.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891413ef10 con 0x7f891414e7b0 2023-10-26T05:53:06.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891413f1f0 con 0x7f8914071270 2023-10-26T05:53:06.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 0x7f8914142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:06.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 0x7f8914142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52858/0 (socket says 172.21.15.71:52858) 2023-10-26T05:53:06.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 -- 172.21.15.71:0/1557212815 learned_addr learned my addr 172.21.15.71:0/1557212815 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:06.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 -- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8914071270 msgr2=0x7f8914141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:06.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f890affd640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8914071270 0x7f8914141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:06.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8914071270 0x7f8914141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8913fff640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f891414e7b0 0x7f891413dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:06.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 -- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f891414e7b0 msgr2=0x7f891413dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:06.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f891414e7b0 0x7f891413dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:06.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 -- 172.21.15.71:0/1557212815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904009560 con 0x7f891406a000 2023-10-26T05:53:06.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8913fff640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f891414e7b0 0x7f891413dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:06.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f89137fe640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 0x7f8914142e00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8904009850 tx=0x7f89040a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:06.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904003800 con 0x7f891406a000 2023-10-26T05:53:06.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8904003960 con 0x7f891406a000 2023-10-26T05:53:06.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904003d30 con 0x7f891406a000 2023-10-26T05:53:06.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891413f4d0 con 0x7f891406a000 2023-10-26T05:53:06.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.536+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8914143340 con 0x7f891406a000 2023-10-26T05:53:06.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.540+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8914066b10 con 0x7f891406a000 2023-10-26T05:53:06.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.540+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89040b4070 con 0x7f891406a000 2023-10-26T05:53:06.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.540+0000 7f8910ff9640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88f4066470 0x7f88f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:06.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.540+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f890411dfa0 con 0x7f891406a000 2023-10-26T05:53:06.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.540+0000 7f890affd640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88f4066470 0x7f88f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:06.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.540+0000 7f890affd640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88f4066470 0x7f88f4068930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f891415a4e0 tx=0x7f88fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:06.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.544+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89040eb620 con 0x7f891406a000 2023-10-26T05:53:06.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:06.732+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f891406a990 con 0x7f891406a000 2023-10-26T05:53:07.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.424+0000 7f8910ff9640 1 -- 172.21.15.71:0/1557212815 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f891406a990 con 0x7f891406a000 2023-10-26T05:53:07.433 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 14 and data pool 15 2023-10-26T05:53:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88f4066470 msgr2=0x7f88f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88f4066470 0x7f88f4068930 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f891415a4e0 tx=0x7f88fc009040 comp rx=0 tx=0).stop 2023-10-26T05:53:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 msgr2=0x7f8914142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 0x7f8914142e00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8904009850 tx=0x7f89040a5780 comp rx=0 tx=0).stop 2023-10-26T05:53:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 shutdown_connections 2023-10-26T05:53:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f891414e7b0 0x7f891413dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8914071270 0x7f8914141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88f4066470 0x7f88f4068930 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 --2- 172.21.15.71:0/1557212815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f891406a000 0x7f8914142e00 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 >> 172.21.15.71:0/1557212815 conn(0x7f89140aec60 msgr2=0x7f8914157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 shutdown_connections 2023-10-26T05:53:07.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.432+0000 7f8919fd1640 1 -- 172.21.15.71:0/1557212815 wait complete. 2023-10-26T05:53:07.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:07.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 -- 172.21.15.71:0/135282996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f880cb010 msgr2=0x7f3f8806f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/135282996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f880cb010 0x7f3f8806f640 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3f880afd40 tx=0x7f3f8001c5e0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 -- 172.21.15.71:0/135282996 shutdown_connections 2023-10-26T05:53:07.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/135282996 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f880cb010 0x7f3f8806f640 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/135282996 >> v2:172.21.15.86:3300/0 conn(0x7f3f880b2890 0x7f3f880caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/135282996 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f8813c2f0 0x7f3f880b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 -- 172.21.15.71:0/135282996 >> 172.21.15.71:0/135282996 conn(0x7f3f88069770 msgr2=0x7f3f88069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:07.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 -- 172.21.15.71:0/135282996 shutdown_connections 2023-10-26T05:53:07.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.572+0000 7f3f8eb93640 1 -- 172.21.15.71:0/135282996 wait complete. 2023-10-26T05:53:07.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 Processor -- start 2023-10-26T05:53:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- start start 2023-10-26T05:53:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f880b2890 0x7f3f8813f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f880cb010 0x7f3f881408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 0x7f3f8813a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88139c90 con 0x7f3f8813c2f0 2023-10-26T05:53:07.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8813afc0 con 0x7f3f880cb010 2023-10-26T05:53:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8813b2a0 con 0x7f3f880b2890 2023-10-26T05:53:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 0x7f3f8813a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 0x7f3f8813a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52870/0 (socket says 172.21.15.71:52870) 2023-10-26T05:53:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 -- 172.21.15.71:0/120802351 learned_addr learned my addr 172.21.15.71:0/120802351 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:07.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8c92f640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f880b2890 0x7f3f8813f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f87fff640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f880cb010 0x7f3f881408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 -- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f880b2890 msgr2=0x7f3f8813f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f880b2890 0x7f3f8813f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 -- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f880cb010 msgr2=0x7f3f881408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f880cb010 0x7f3f881408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 -- 172.21.15.71:0/120802351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009560 con 0x7f3f8813c2f0 2023-10-26T05:53:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8c92f640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f880b2890 0x7f3f8813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8d130640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 0x7f3f8813a410 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009850 tx=0x7f3f800aa520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800aaa60 con 0x7f3f8813c2f0 2023-10-26T05:53:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3f800b8070 con 0x7f3f8813c2f0 2023-10-26T05:53:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80003600 con 0x7f3f8813c2f0 2023-10-26T05:53:07.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8813f6d0 con 0x7f3f8813c2f0 2023-10-26T05:53:07.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8813a950 con 0x7f3f8813c2f0 2023-10-26T05:53:07.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.576+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f88066b10 con 0x7f3f8813c2f0 2023-10-26T05:53:07.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.580+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3f80003760 con 0x7f3f8813c2f0 2023-10-26T05:53:07.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.580+0000 7f3f857fa640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f68066470 0x7f3f68068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.580+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f800ba070 con 0x7f3f8813c2f0 2023-10-26T05:53:07.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.580+0000 7f3f8c92f640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f68066470 0x7f3f68068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.580+0000 7f3f8c92f640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f68066470 0x7f3f68068930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800a870 tx=0x7f3f78008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:07.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.584+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3f800ec550 con 0x7f3f8813c2f0 2023-10-26T05:53:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.740+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f880721e0 con 0x7f3f8813c2f0 2023-10-26T05:53:07.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.740+0000 7f3f857fa640 1 -- 172.21.15.71:0/120802351 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16449 (secure 0 0 0) 0x7f3f800c4020 con 0x7f3f8813c2f0 2023-10-26T05:53:07.748 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:07.749 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":123,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:06.735773+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:07.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f68066470 msgr2=0x7f3f68068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f68066470 0x7f3f68068930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800a870 tx=0x7f3f78008040 comp rx=0 tx=0).stop 2023-10-26T05:53:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 msgr2=0x7f3f8813a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 0x7f3f8813a410 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009850 tx=0x7f3f800aa520 comp rx=0 tx=0).stop 2023-10-26T05:53:07.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 shutdown_connections 2023-10-26T05:53:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3f68066470 0x7f3f68068930 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8813c2f0 0x7f3f8813a410 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f880cb010 0x7f3f881408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 --2- 172.21.15.71:0/120802351 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3f880b2890 0x7f3f8813f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 >> 172.21.15.71:0/120802351 conn(0x7f3f88069770 msgr2=0x7f3f880acb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:07.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 shutdown_connections 2023-10-26T05:53:07.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.744+0000 7f3f8eb93640 1 -- 172.21.15.71:0/120802351 wait complete. 2023-10-26T05:53:07.767 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 -- 172.21.15.71:0/2170197030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc140cc0 msgr2=0x7fbfdc15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/2170197030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc15c760 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009840 tx=0x7fbfcc01c7d0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 -- 172.21.15.71:0/2170197030 shutdown_connections 2023-10-26T05:53:07.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/2170197030 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfdc0b2370 0x7fbfdc071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/2170197030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc15c760 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 -- 172.21.15.71:0/2170197030 >> 172.21.15.71:0/2170197030 conn(0x7fbfdc069770 msgr2=0x7fbfdc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:07.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 -- 172.21.15.71:0/2170197030 shutdown_connections 2023-10-26T05:53:07.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.872+0000 7fbfe16d0640 1 -- 172.21.15.71:0/2170197030 wait complete. 2023-10-26T05:53:07.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 Processor -- start 2023-10-26T05:53:07.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- start start 2023-10-26T05:53:07.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 0x7fbfdc12e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfdc0b2370 0x7fbfdc157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc12f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc13e510 con 0x7fbfdc070fe0 2023-10-26T05:53:07.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc136710 con 0x7fbfdc140cc0 2023-10-26T05:53:07.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc1369f0 con 0x7fbfdc0b2370 2023-10-26T05:53:07.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 0x7fbfdc12e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 0x7fbfdc12e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52886/0 (socket says 172.21.15.71:52886) 2023-10-26T05:53:07.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 -- 172.21.15.71:0/1834924287 learned_addr learned my addr 172.21.15.71:0/1834924287 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:07.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 -- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfdc0b2370 msgr2=0x7fbfdc157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfda7fc640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfdc0b2370 0x7fbfdc157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdb7fe640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc12f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfdc0b2370 0x7fbfdc157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 -- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc140cc0 msgr2=0x7fbfdc12f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:07.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc12f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:07.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 -- 172.21.15.71:0/1834924287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc0094a0 con 0x7fbfdc070fe0 2023-10-26T05:53:07.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdb7fe640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc12f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:07.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfdaffd640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 0x7fbfdc12e940 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000c10 tx=0x7fbfcc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:07.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc005430 con 0x7fbfdc070fe0 2023-10-26T05:53:07.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfcc005680 con 0x7fbfdc070fe0 2023-10-26T05:53:07.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0048f0 con 0x7fbfdc070fe0 2023-10-26T05:53:07.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc136cd0 con 0x7fbfdc070fe0 2023-10-26T05:53:07.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc1577b0 con 0x7fbfdc070fe0 2023-10-26T05:53:07.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc066b10 con 0x7fbfdc070fe0 2023-10-26T05:53:07.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.876+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbfcc0b4040 con 0x7fbfdc070fe0 2023-10-26T05:53:07.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.880+0000 7fbfc37fe640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfc8066470 0x7fbfc8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:07.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.880+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfcc11e7b0 con 0x7fbfdc070fe0 2023-10-26T05:53:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.880+0000 7fbfda7fc640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfc8066470 0x7fbfc8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.880+0000 7fbfda7fc640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfc8066470 0x7fbfc8068930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc069e00 tx=0x7fbfd0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:07.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:07.884+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbfcc0ebcc0 con 0x7fbfdc070fe0 2023-10-26T05:53:08.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.028+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfdc136b30 con 0x7fbfdc070fe0 2023-10-26T05:53:08.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.028+0000 7fbfc37fe640 1 -- 172.21.15.71:0/1834924287 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16449 (secure 0 0 0) 0x7fbfdc136b30 con 0x7fbfdc070fe0 2023-10-26T05:53:08.036 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:08.036 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":123,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:06.735773+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:08.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfc8066470 msgr2=0x7fbfc8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfc8066470 0x7fbfc8068930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc069e00 tx=0x7fbfd0009040 comp rx=0 tx=0).stop 2023-10-26T05:53:08.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 msgr2=0x7fbfdc12e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 0x7fbfdc12e940 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc000c10 tx=0x7fbfcc005040 comp rx=0 tx=0).stop 2023-10-26T05:53:08.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 shutdown_connections 2023-10-26T05:53:08.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfdc140cc0 0x7fbfdc12f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfdc0b2370 0x7fbfdc157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfc8066470 0x7fbfc8068930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 --2- 172.21.15.71:0/1834924287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfdc070fe0 0x7fbfdc12e940 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 >> 172.21.15.71:0/1834924287 conn(0x7fbfdc069770 msgr2=0x7fbfdc137b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:08.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 shutdown_connections 2023-10-26T05:53:08.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.032+0000 7fbfe16d0640 1 -- 172.21.15.71:0/1834924287 wait complete. 2023-10-26T05:53:08.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:53:08.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 -- 172.21.15.71:0/3396725015 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694406a000 msgr2=0x7f694406a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 --2- 172.21.15.71:0/3396725015 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694406a000 0x7f694406a450 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f693c00acd0 tx=0x7f693c01ded0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 -- 172.21.15.71:0/3396725015 shutdown_connections 2023-10-26T05:53:08.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 --2- 172.21.15.71:0/3396725015 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694406a000 0x7f694406a450 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 --2- 172.21.15.71:0/3396725015 >> v2:172.21.15.86:3300/0 conn(0x7f694414e7b0 0x7f6944157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 -- 172.21.15.71:0/3396725015 >> 172.21.15.71:0/3396725015 conn(0x7f69440aec60 msgr2=0x7f69440af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:08.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 -- 172.21.15.71:0/3396725015 shutdown_connections 2023-10-26T05:53:08.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 -- 172.21.15.71:0/3396725015 wait complete. 2023-10-26T05:53:08.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 Processor -- start 2023-10-26T05:53:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.164+0000 7f694a66b640 1 -- start start 2023-10-26T05:53:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406a000 0x7f6944148800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 0x7f69440cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694414e7b0 0x7f69440cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69441492c0 con 0x7f6944071270 2023-10-26T05:53:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69441475c0 con 0x7f694406a000 2023-10-26T05:53:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69441478a0 con 0x7f694414e7b0 2023-10-26T05:53:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 0x7f69440cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 0x7f69440cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52890/0 (socket says 172.21.15.71:52890) 2023-10-26T05:53:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 -- 172.21.15.71:0/2362635974 learned_addr learned my addr 172.21.15.71:0/2362635974 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:08.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6943fff640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406a000 0x7f6944148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6948c08640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694414e7b0 0x7f69440cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 -- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694414e7b0 msgr2=0x7f69440cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694414e7b0 0x7f69440cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 -- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406a000 msgr2=0x7f6944148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406a000 0x7f6944148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 -- 172.21.15.71:0/2362635974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c00a930 con 0x7f6944071270 2023-10-26T05:53:08.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6948c08640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694414e7b0 0x7f69440cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:08.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6943fff640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406a000 0x7f6944148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:08.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f69437fe640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 0x7f69440cb850 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f693400c420 tx=0x7f693400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:08.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934004230 con 0x7f6944071270 2023-10-26T05:53:08.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f69340044a0 con 0x7f6944071270 2023-10-26T05:53:08.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6944147b80 con 0x7f6944071270 2023-10-26T05:53:08.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934005260 con 0x7f6944071270 2023-10-26T05:53:08.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6944148380 con 0x7f6944071270 2023-10-26T05:53:08.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6944066b10 con 0x7f6944071270 2023-10-26T05:53:08.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f69340053c0 con 0x7f6944071270 2023-10-26T05:53:08.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6940ff9640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6930066470 0x7f6930068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69340833c0 con 0x7f6944071270 2023-10-26T05:53:08.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6943fff640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6930066470 0x7f6930068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.168+0000 7f6943fff640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6930066470 0x7f6930068930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6938009ac0 tx=0x7f69380091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:08.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.172+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f69340507c0 con 0x7f6944071270 2023-10-26T05:53:08.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.360+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f69441480a0 con 0x7f6944071270 2023-10-26T05:53:08.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-26T05:53:08.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.436+0000 7f6940ff9640 1 -- 172.21.15.71:0/2362635974 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f69441480a0 con 0x7f6944071270 2023-10-26T05:53:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6930066470 msgr2=0x7f6930068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6930066470 0x7f6930068930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6938009ac0 tx=0x7f69380091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 msgr2=0x7f69440cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 0x7f69440cb850 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f693400c420 tx=0x7f693400c8e0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 shutdown_connections 2023-10-26T05:53:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f694414e7b0 0x7f69440cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6930066470 0x7f6930068930 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6944071270 0x7f69440cb850 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 --2- 172.21.15.71:0/2362635974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f694406a000 0x7f6944148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 >> 172.21.15.71:0/2362635974 conn(0x7f69440aec60 msgr2=0x7f6944157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:08.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 shutdown_connections 2023-10-26T05:53:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.440+0000 7f694a66b640 1 -- 172.21.15.71:0/2362635974 wait complete. 2023-10-26T05:53:08.466 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:53:08.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:53:08.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- 172.21.15.71:0/291907418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 msgr2=0x7f846400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 --2- 172.21.15.71:0/291907418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f846400b990 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f847400c590 tx=0x7f847401faa0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- 172.21.15.71:0/291907418 shutdown_connections 2023-10-26T05:53:08.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 --2- 172.21.15.71:0/291907418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f846400b990 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- 172.21.15.71:0/291907418 >> 172.21.15.71:0/291907418 conn(0x7f847c0cdd80 msgr2=0x7f847c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:08.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- 172.21.15.71:0/291907418 shutdown_connections 2023-10-26T05:53:08.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- 172.21.15.71:0/291907418 wait complete. 2023-10-26T05:53:08.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 Processor -- start 2023-10-26T05:53:08.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- start start 2023-10-26T05:53:08.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f847c1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f847c14b530 0x7f847c15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f847c14b9b0 0x7f847c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c06a650 con 0x7f8464005190 2023-10-26T05:53:08.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c14b2a0 con 0x7f847c14b9b0 2023-10-26T05:53:08.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f8481c09640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15c1c0 con 0x7f847c14b530 2023-10-26T05:53:08.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f847b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f847c1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f847b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f847c1f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52912/0 (socket says 172.21.15.71:52912) 2023-10-26T05:53:08.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.592+0000 7f847b7fe640 1 -- 172.21.15.71:0/3878658392 learned_addr learned my addr 172.21.15.71:0/3878658392 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:08.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847affd640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f847c14b530 0x7f847c15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847b7fe640 1 -- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f847c14b530 msgr2=0x7f847c15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847bfff640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f847c14b9b0 0x7f847c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847b7fe640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f847c14b530 0x7f847c15d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847b7fe640 1 -- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f847c14b9b0 msgr2=0x7f847c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:08.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847b7fe640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f847c14b9b0 0x7f847c15d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:08.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847b7fe640 1 -- 172.21.15.71:0/3878658392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c15cd40 con 0x7f8464005190 2023-10-26T05:53:08.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847b7fe640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f847c1f6130 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f846c004d50 tx=0x7f846c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:08.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c016030 con 0x7f8464005190 2023-10-26T05:53:08.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f846c0108c0 con 0x7f8464005190 2023-10-26T05:53:08.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c15c410 con 0x7f8464005190 2023-10-26T05:53:08.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c010cb0 con 0x7f8464005190 2023-10-26T05:53:08.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c15c9d0 con 0x7f8464005190 2023-10-26T05:53:08.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c066b10 con 0x7f8464005190 2023-10-26T05:53:08.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f846c019070 con 0x7f8464005190 2023-10-26T05:53:08.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8463fff640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84680664a0 0x7f8468068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:08.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f846c017070 con 0x7f8464005190 2023-10-26T05:53:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847affd640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84680664a0 0x7f8468068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.596+0000 7f847affd640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84680664a0 0x7f8468068960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84700057d0 tx=0x7f8470009580 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:08.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.600+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f846c051980 con 0x7f8464005190 2023-10-26T05:53:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:08.780+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f847c0721e0 con 0x7f8464005190 2023-10-26T05:53:09.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.448+0000 7f8463fff640 1 -- 172.21.15.71:0/3878658392 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f846c054fa0 con 0x7f8464005190 2023-10-26T05:53:09.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84680664a0 msgr2=0x7f8468068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84680664a0 0x7f8468068960 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f84700057d0 tx=0x7f8470009580 comp rx=0 tx=0).stop 2023-10-26T05:53:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 msgr2=0x7f847c1f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f847c1f6130 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f846c004d50 tx=0x7f846c001640 comp rx=0 tx=0).stop 2023-10-26T05:53:09.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 shutdown_connections 2023-10-26T05:53:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f847c14b9b0 0x7f847c15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f847c14b530 0x7f847c15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f84680664a0 0x7f8468068960 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 --2- 172.21.15.71:0/3878658392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8464005190 0x7f847c1f6130 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 >> 172.21.15.71:0/3878658392 conn(0x7f847c0cdd80 msgr2=0x7f847c0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 shutdown_connections 2023-10-26T05:53:09.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.452+0000 7f8481c09640 1 -- 172.21.15.71:0/3878658392 wait complete. 2023-10-26T05:53:09.478 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:53:09.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:09.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- 172.21.15.71:0/908912446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 msgr2=0x7f832c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- 172.21.15.71:0/908912446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c15b3a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f832c0c8360 tx=0x7f831c01c760 comp rx=0 tx=0).stop 2023-10-26T05:53:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- 172.21.15.71:0/908912446 shutdown_connections 2023-10-26T05:53:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- 172.21.15.71:0/908912446 >> v2:172.21.15.86:3300/0 conn(0x7f832c0cd880 0x7f832c15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- 172.21.15.71:0/908912446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c15b3a0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- 172.21.15.71:0/908912446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f832c0d0470 0x7f832c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- 172.21.15.71:0/908912446 >> 172.21.15.71:0/908912446 conn(0x7f832c0cdd80 msgr2=0x7f832c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- 172.21.15.71:0/908912446 shutdown_connections 2023-10-26T05:53:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- 172.21.15.71:0/908912446 wait complete. 2023-10-26T05:53:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 Processor -- start 2023-10-26T05:53:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- start start 2023-10-26T05:53:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f832c0cd880 0x7f832c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f832c0d0470 0x7f832c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0c7b10 con 0x7f832c0d0850 2023-10-26T05:53:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c1f55e0 con 0x7f832c0d0470 2023-10-26T05:53:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8334105640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c1f58c0 con 0x7f832c0cd880 2023-10-26T05:53:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c0caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52926/0 (socket says 172.21.15.71:52926) 2023-10-26T05:53:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 -- 172.21.15.71:0/1731855028 learned_addr learned my addr 172.21.15.71:0/1731855028 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f8331ea1640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f832c0cd880 0x7f832c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83316a0640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f832c0d0470 0x7f832c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 -- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f832c0cd880 msgr2=0x7f832c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f832c0cd880 0x7f832c0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 -- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f832c0d0470 msgr2=0x7f832c1f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f832c0d0470 0x7f832c1f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.600+0000 7f83326a2640 1 -- 172.21.15.71:0/1731855028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c009560 con 0x7f832c0d0850 2023-10-26T05:53:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83326a2640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c0caa00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f832400ca60 tx=0x7f832400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400adc0 con 0x7f832c0d0850 2023-10-26T05:53:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8324004c40 con 0x7f832c0d0850 2023-10-26T05:53:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c1f5ba0 con 0x7f832c0d0850 2023-10-26T05:53:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324005280 con 0x7f832c0d0850 2023-10-26T05:53:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c1f5260 con 0x7f832c0d0850 2023-10-26T05:53:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c066b10 con 0x7f832c0d0850 2023-10-26T05:53:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f83240053e0 con 0x7f832c0d0850 2023-10-26T05:53:09.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83227fc640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8310066470 0x7f8310068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8324084670 con 0x7f832c0d0850 2023-10-26T05:53:09.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f8331ea1640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8310066470 0x7f8310068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.604+0000 7f8331ea1640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8310066470 0x7f8310068930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f832800bc00 tx=0x7f832800b390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.608+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8324051ba0 con 0x7f832c0d0850 2023-10-26T05:53:09.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832c0721e0 con 0x7f832c0d0850 2023-10-26T05:53:09.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f83227fc640 1 -- 172.21.15.71:0/1731855028 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f832401d090 con 0x7f832c0d0850 2023-10-26T05:53:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:09.807 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:09.807 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:09.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8310066470 msgr2=0x7f8310068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8310066470 0x7f8310068930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f832800bc00 tx=0x7f832800b390 comp rx=0 tx=0).stop 2023-10-26T05:53:09.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 msgr2=0x7f832c0caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c0caa00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f832400ca60 tx=0x7f832400cf20 comp rx=0 tx=0).stop 2023-10-26T05:53:09.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 shutdown_connections 2023-10-26T05:53:09.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8310066470 0x7f8310068930 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f832c0d0850 0x7f832c0caa00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f832c0d0470 0x7f832c1f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 --2- 172.21.15.71:0/1731855028 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f832c0cd880 0x7f832c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 >> 172.21.15.71:0/1731855028 conn(0x7f832c0cdd80 msgr2=0x7f832c15b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:09.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 shutdown_connections 2023-10-26T05:53:09.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.800+0000 7f8334105640 1 -- 172.21.15.71:0/1731855028 wait complete. 2023-10-26T05:53:09.825 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:09.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 -- 172.21.15.71:0/2557318411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 msgr2=0x7f39f80caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 --2- 172.21.15.71:0/2557318411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f80caad0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f39f806f190 tx=0x7f39e001c5e0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 -- 172.21.15.71:0/2557318411 shutdown_connections 2023-10-26T05:53:09.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 --2- 172.21.15.71:0/2557318411 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39f80cb010 0x7f39f806f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 --2- 172.21.15.71:0/2557318411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f80caad0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 --2- 172.21.15.71:0/2557318411 >> v2:172.21.15.86:3300/0 conn(0x7f39f813c2f0 0x7f39f80b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 -- 172.21.15.71:0/2557318411 >> 172.21.15.71:0/2557318411 conn(0x7f39f8069770 msgr2=0x7f39f8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:09.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 -- 172.21.15.71:0/2557318411 shutdown_connections 2023-10-26T05:53:09.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.928+0000 7f39fcf14640 1 -- 172.21.15.71:0/2557318411 wait complete. 2023-10-26T05:53:09.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 Processor -- start 2023-10-26T05:53:09.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- start start 2023-10-26T05:53:09.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f8153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39f80cb010 0x7f39f8155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f813c2f0 0x7f39f813cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f813d0c0 con 0x7f39f80b2890 2023-10-26T05:53:09.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8135cd0 con 0x7f39f813c2f0 2023-10-26T05:53:09.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8154560 con 0x7f39f80cb010 2023-10-26T05:53:09.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f8153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f8153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52936/0 (socket says 172.21.15.71:52936) 2023-10-26T05:53:09.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 -- 172.21.15.71:0/3408672588 learned_addr learned my addr 172.21.15.71:0/3408672588 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:09.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f6d9d640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f813c2f0 0x7f39f813cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 -- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39f80cb010 msgr2=0x7f39f8155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f5d9b640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39f80cb010 0x7f39f8155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39f80cb010 0x7f39f8155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 -- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f813c2f0 msgr2=0x7f39f813cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:09.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f813c2f0 0x7f39f813cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:09.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 -- 172.21.15.71:0/3408672588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0009560 con 0x7f39f80b2890 2023-10-26T05:53:09.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f6d9d640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f813c2f0 0x7f39f813cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:09.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39f659c640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f8153ed0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c940 tx=0x7f39e800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:09.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8004220 con 0x7f39f80b2890 2023-10-26T05:53:09.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f39e8004470 con 0x7f39f80b2890 2023-10-26T05:53:09.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8154840 con 0x7f39f80b2890 2023-10-26T05:53:09.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8007bf0 con 0x7f39f80b2890 2023-10-26T05:53:09.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f8154e00 con 0x7f39f80b2890 2023-10-26T05:53:09.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8066b10 con 0x7f39f80b2890 2023-10-26T05:53:09.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.932+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f39e8007d50 con 0x7f39f80b2890 2023-10-26T05:53:09.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.936+0000 7f39eeffd640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39d8066470 0x7f39d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:09.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.936+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39e8084340 con 0x7f39f80b2890 2023-10-26T05:53:09.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.936+0000 7f39f5d9b640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39d8066470 0x7f39d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:09.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.936+0000 7f39f5d9b640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39d8066470 0x7f39d8068930 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f39e00ae000 tx=0x7f39e001ae20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:09.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:09.940+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f39e80517c0 con 0x7f39f80b2890 2023-10-26T05:53:10.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.084+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39f8135e10 con 0x7f39f80b2890 2023-10-26T05:53:10.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39eeffd640 1 -- 172.21.15.71:0/3408672588 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16436 (secure 0 0 0) 0x7f39e801d090 con 0x7f39f80b2890 2023-10-26T05:53:10.093 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:10.094 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":124,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:08.156231+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:10.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39d8066470 msgr2=0x7f39d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39d8066470 0x7f39d8068930 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f39e00ae000 tx=0x7f39e001ae20 comp rx=0 tx=0).stop 2023-10-26T05:53:10.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 msgr2=0x7f39f8153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f8153ed0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f39e800c940 tx=0x7f39e800ce00 comp rx=0 tx=0).stop 2023-10-26T05:53:10.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 shutdown_connections 2023-10-26T05:53:10.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39f813c2f0 0x7f39f813cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f39f80cb010 0x7f39f8155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39d8066470 0x7f39d8068930 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 --2- 172.21.15.71:0/3408672588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39f80b2890 0x7f39f8153ed0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 >> 172.21.15.71:0/3408672588 conn(0x7f39f8069770 msgr2=0x7f39f8152940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:10.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 shutdown_connections 2023-10-26T05:53:10.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.088+0000 7f39fcf14640 1 -- 172.21.15.71:0/3408672588 wait complete. 2023-10-26T05:53:10.111 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:53:10.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/155707404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc958003f30 msgr2=0x7fc9580047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 --2- 172.21.15.71:0/155707404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc958003f30 0x7fc9580047d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc954009b60 tx=0x7fc95401d220 comp rx=0 tx=0).stop 2023-10-26T05:53:10.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 -- 172.21.15.71:0/155707404 reap_dead start 2023-10-26T05:53:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/155707404 shutdown_connections 2023-10-26T05:53:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/155707404 >> 172.21.15.71:0/155707404 conn(0x7fc96c0cdd80 msgr2=0x7fc96c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/155707404 shutdown_connections 2023-10-26T05:53:10.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/155707404 wait complete. 2023-10-26T05:53:10.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 Processor -- start 2023-10-26T05:53:10.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- start start 2023-10-26T05:53:10.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc96c14c0c0 0x7fc96c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc96c0c7480 0x7fc96c153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 0x7fc96c1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c06a650 con 0x7fc96c154470 2023-10-26T05:53:10.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c14ff10 con 0x7fc96c14c0c0 2023-10-26T05:53:10.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c153aa0 con 0x7fc96c0c7480 2023-10-26T05:53:10.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 0x7fc96c1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 0x7fc96c1548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52964/0 (socket says 172.21.15.71:52964) 2023-10-26T05:53:10.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 -- 172.21.15.71:0/1409525726 learned_addr learned my addr 172.21.15.71:0/1409525726 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:10.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc96bfff640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc96c0c7480 0x7fc96c153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc970ab9640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc96c14c0c0 0x7fc96c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 -- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc96c0c7480 msgr2=0x7fc96c153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc96c0c7480 0x7fc96c153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 -- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc96c14c0c0 msgr2=0x7fc96c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc96c14c0c0 0x7fc96c0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 -- 172.21.15.71:0/1409525726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c0c51e0 con 0x7fc96c154470 2023-10-26T05:53:10.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9712ba640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 0x7fc96c1548f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0cb900 tx=0x7fc96400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:10.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400a9c0 con 0x7fc96c154470 2023-10-26T05:53:10.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc96400ac10 con 0x7fc96c154470 2023-10-26T05:53:10.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c0c54c0 con 0x7fc96c154470 2023-10-26T05:53:10.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400cec0 con 0x7fc96c154470 2023-10-26T05:53:10.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.232+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c0c5a80 con 0x7fc96c154470 2023-10-26T05:53:10.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.236+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c066b10 con 0x7fc96c154470 2023-10-26T05:53:10.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.236+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc964007070 con 0x7fc96c154470 2023-10-26T05:53:10.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.236+0000 7fc9697fa640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc94c066470 0x7fc94c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.236+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc964083ba0 con 0x7fc96c154470 2023-10-26T05:53:10.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.236+0000 7fc970ab9640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc94c066470 0x7fc94c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.236+0000 7fc970ab9640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc94c066470 0x7fc94c068930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006590 tx=0x7fc95c006b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:10.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.240+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc964051020 con 0x7fc96c154470 2023-10-26T05:53:10.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.376+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc96c0721e0 con 0x7fc94c066470 2023-10-26T05:53:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.400+0000 7fc9697fa640 1 -- 172.21.15.71:0/1409525726 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284120 (secure 0 0 0) 0x7fc96c0721e0 con 0x7fc94c066470 2023-10-26T05:53:10.408 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc94c066470 msgr2=0x7fc94c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc94c066470 0x7fc94c068930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc95c006590 tx=0x7fc95c006b60 comp rx=0 tx=0).stop 2023-10-26T05:53:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 msgr2=0x7fc96c1548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 0x7fc96c1548f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0cb900 tx=0x7fc96400cbb0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 shutdown_connections 2023-10-26T05:53:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc94c066470 0x7fc94c068930 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96c154470 0x7fc96c1548f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc96c0c7480 0x7fc96c153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 --2- 172.21.15.71:0/1409525726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc96c14c0c0 0x7fc96c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 >> 172.21.15.71:0/1409525726 conn(0x7fc96c0cdd80 msgr2=0x7fc96c06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 shutdown_connections 2023-10-26T05:53:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.404+0000 7fc972d1d640 1 -- 172.21.15.71:0/1409525726 wait complete. 2023-10-26T05:53:10.427 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-26T05:53:09.527451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6496,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556003,"statfs":{"total":1600000000,"available":1593347832,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.147833"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.159973+0000","last_change":"2023-10-26T05:53:07.173723+0000","last_active":"2023-10-26T05:53:08.159973+0000","last_peered":"2023-10-26T05:53:08.159973+0000","last_clean":"2023-10-26T05:53:08.159973+0000","last_became_active":"2023-10-26T05:53:07.173085+0000","last_became_peered":"2023-10-26T05:53:07.173085+0000","last_unstale":"2023-10-26T05:53:08.159973+0000","last_undegraded":"2023-10-26T05:53:08.159973+0000","last_fullsized":"2023-10-26T05:53:08.159973+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:06.158808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837592+0000","last_change":"2023-10-26T05:53:07.173998+0000","last_active":"2023-10-26T05:53:08.837592+0000","last_peered":"2023-10-26T05:53:08.837592+0000","last_clean":"2023-10-26T05:53:08.837592+0000","last_became_active":"2023-10-26T05:53:07.172957+0000","last_became_peered":"2023-10-26T05:53:07.172957+0000","last_unstale":"2023-10-26T05:53:08.837592+0000","last_undegraded":"2023-10-26T05:53:08.837592+0000","last_fullsized":"2023-10-26T05:53:08.837592+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:29.535115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160119+0000","last_change":"2023-10-26T05:53:07.173609+0000","last_active":"2023-10-26T05:53:08.160119+0000","last_peered":"2023-10-26T05:53:08.160119+0000","last_clean":"2023-10-26T05:53:08.160119+0000","last_became_active":"2023-10-26T05:53:07.172968+0000","last_became_peered":"2023-10-26T05:53:07.172968+0000","last_unstale":"2023-10-26T05:53:08.160119+0000","last_undegraded":"2023-10-26T05:53:08.160119+0000","last_fullsized":"2023-10-26T05:53:08.160119+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:51.614835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836827+0000","last_change":"2023-10-26T05:53:07.205986+0000","last_active":"2023-10-26T05:53:08.836827+0000","last_peered":"2023-10-26T05:53:08.836827+0000","last_clean":"2023-10-26T05:53:08.836827+0000","last_became_active":"2023-10-26T05:53:07.204960+0000","last_became_peered":"2023-10-26T05:53:07.204960+0000","last_unstale":"2023-10-26T05:53:08.836827+0000","last_undegraded":"2023-10-26T05:53:08.836827+0000","last_fullsized":"2023-10-26T05:53:08.836827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:30.390769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839474+0000","last_change":"2023-10-26T05:53:07.201863+0000","last_active":"2023-10-26T05:53:08.839474+0000","last_peered":"2023-10-26T05:53:08.839474+0000","last_clean":"2023-10-26T05:53:08.839474+0000","last_became_active":"2023-10-26T05:53:07.201339+0000","last_became_peered":"2023-10-26T05:53:07.201339+0000","last_unstale":"2023-10-26T05:53:08.839474+0000","last_undegraded":"2023-10-26T05:53:08.839474+0000","last_fullsized":"2023-10-26T05:53:08.839474+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:29.535115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558331+0000","last_change":"2023-10-26T05:53:07.195436+0000","last_active":"2023-10-26T05:53:08.558331+0000","last_peered":"2023-10-26T05:53:08.558331+0000","last_clean":"2023-10-26T05:53:08.558331+0000","last_became_active":"2023-10-26T05:53:07.192362+0000","last_became_peered":"2023-10-26T05:53:07.192362+0000","last_unstale":"2023-10-26T05:53:08.558331+0000","last_undegraded":"2023-10-26T05:53:08.558331+0000","last_fullsized":"2023-10-26T05:53:08.558331+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:23.170274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558078+0000","last_change":"2023-10-26T05:53:07.191937+0000","last_active":"2023-10-26T05:53:08.558078+0000","last_peered":"2023-10-26T05:53:08.558078+0000","last_clean":"2023-10-26T05:53:08.558078+0000","last_became_active":"2023-10-26T05:53:07.191239+0000","last_became_peered":"2023-10-26T05:53:07.191239+0000","last_unstale":"2023-10-26T05:53:08.558078+0000","last_undegraded":"2023-10-26T05:53:08.558078+0000","last_fullsized":"2023-10-26T05:53:08.558078+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:59.146622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838567+0000","last_change":"2023-10-26T05:53:07.203045+0000","last_active":"2023-10-26T05:53:08.838567+0000","last_peered":"2023-10-26T05:53:08.838567+0000","last_clean":"2023-10-26T05:53:08.838567+0000","last_became_active":"2023-10-26T05:53:07.199707+0000","last_became_peered":"2023-10-26T05:53:07.199707+0000","last_unstale":"2023-10-26T05:53:08.838567+0000","last_undegraded":"2023-10-26T05:53:08.838567+0000","last_fullsized":"2023-10-26T05:53:08.838567+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:40.082223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167675+0000","last_change":"2023-10-26T05:53:07.203835+0000","last_active":"2023-10-26T05:53:08.167675+0000","last_peered":"2023-10-26T05:53:08.167675+0000","last_clean":"2023-10-26T05:53:08.167675+0000","last_became_active":"2023-10-26T05:53:07.203155+0000","last_became_peered":"2023-10-26T05:53:07.203155+0000","last_unstale":"2023-10-26T05:53:08.167675+0000","last_undegraded":"2023-10-26T05:53:08.167675+0000","last_fullsized":"2023-10-26T05:53:08.167675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:58.341073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839663+0000","last_change":"2023-10-26T05:53:07.185763+0000","last_active":"2023-10-26T05:53:08.839663+0000","last_peered":"2023-10-26T05:53:08.839663+0000","last_clean":"2023-10-26T05:53:08.839663+0000","last_became_active":"2023-10-26T05:53:07.184195+0000","last_became_peered":"2023-10-26T05:53:07.184195+0000","last_unstale":"2023-10-26T05:53:08.839663+0000","last_undegraded":"2023-10-26T05:53:08.839663+0000","last_fullsized":"2023-10-26T05:53:08.839663+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:12.818270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558654+0000","last_change":"2023-10-26T05:53:07.196068+0000","last_active":"2023-10-26T05:53:08.558654+0000","last_peered":"2023-10-26T05:53:08.558654+0000","last_clean":"2023-10-26T05:53:08.558654+0000","last_became_active":"2023-10-26T05:53:07.193268+0000","last_became_peered":"2023-10-26T05:53:07.193268+0000","last_unstale":"2023-10-26T05:53:08.558654+0000","last_undegraded":"2023-10-26T05:53:08.558654+0000","last_fullsized":"2023-10-26T05:53:08.558654+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:43:11.218597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168102+0000","last_change":"2023-10-26T05:53:07.179314+0000","last_active":"2023-10-26T05:53:08.168102+0000","last_peered":"2023-10-26T05:53:08.168102+0000","last_clean":"2023-10-26T05:53:08.168102+0000","last_became_active":"2023-10-26T05:53:07.175279+0000","last_became_peered":"2023-10-26T05:53:07.175279+0000","last_unstale":"2023-10-26T05:53:08.168102+0000","last_undegraded":"2023-10-26T05:53:08.168102+0000","last_fullsized":"2023-10-26T05:53:08.168102+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:23.912541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836644+0000","last_change":"2023-10-26T05:53:07.198975+0000","last_active":"2023-10-26T05:53:08.836644+0000","last_peered":"2023-10-26T05:53:08.836644+0000","last_clean":"2023-10-26T05:53:08.836644+0000","last_became_active":"2023-10-26T05:53:07.197531+0000","last_became_peered":"2023-10-26T05:53:07.197531+0000","last_unstale":"2023-10-26T05:53:08.836644+0000","last_undegraded":"2023-10-26T05:53:08.836644+0000","last_fullsized":"2023-10-26T05:53:08.836644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:11.011808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836245+0000","last_change":"2023-10-26T05:53:07.196413+0000","last_active":"2023-10-26T05:53:08.836245+0000","last_peered":"2023-10-26T05:53:08.836245+0000","last_clean":"2023-10-26T05:53:08.836245+0000","last_became_active":"2023-10-26T05:53:07.194842+0000","last_became_peered":"2023-10-26T05:53:07.194842+0000","last_unstale":"2023-10-26T05:53:08.836245+0000","last_undegraded":"2023-10-26T05:53:08.836245+0000","last_fullsized":"2023-10-26T05:53:08.836245+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:46:00.117863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836827+0000","last_change":"2023-10-26T05:53:07.207843+0000","last_active":"2023-10-26T05:53:08.836827+0000","last_peered":"2023-10-26T05:53:08.836827+0000","last_clean":"2023-10-26T05:53:08.836827+0000","last_became_active":"2023-10-26T05:53:07.206860+0000","last_became_peered":"2023-10-26T05:53:07.206860+0000","last_unstale":"2023-10-26T05:53:08.836827+0000","last_undegraded":"2023-10-26T05:53:08.836827+0000","last_fullsized":"2023-10-26T05:53:08.836827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:57.157667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558990+0000","last_change":"2023-10-26T05:53:07.201129+0000","last_active":"2023-10-26T05:53:08.558990+0000","last_peered":"2023-10-26T05:53:08.558990+0000","last_clean":"2023-10-26T05:53:08.558990+0000","last_became_active":"2023-10-26T05:53:07.200444+0000","last_became_peered":"2023-10-26T05:53:07.200444+0000","last_unstale":"2023-10-26T05:53:08.558990+0000","last_undegraded":"2023-10-26T05:53:08.558990+0000","last_fullsized":"2023-10-26T05:53:08.558990+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:10.556781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837713+0000","last_change":"2023-10-26T05:53:07.198950+0000","last_active":"2023-10-26T05:53:08.837713+0000","last_peered":"2023-10-26T05:53:08.837713+0000","last_clean":"2023-10-26T05:53:08.837713+0000","last_became_active":"2023-10-26T05:53:07.198294+0000","last_became_peered":"2023-10-26T05:53:07.198294+0000","last_unstale":"2023-10-26T05:53:08.837713+0000","last_undegraded":"2023-10-26T05:53:08.837713+0000","last_fullsized":"2023-10-26T05:53:08.837713+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:34:28.391351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836969+0000","last_change":"2023-10-26T05:53:07.205623+0000","last_active":"2023-10-26T05:53:08.836969+0000","last_peered":"2023-10-26T05:53:08.836969+0000","last_clean":"2023-10-26T05:53:08.836969+0000","last_became_active":"2023-10-26T05:53:07.204904+0000","last_became_peered":"2023-10-26T05:53:07.204904+0000","last_unstale":"2023-10-26T05:53:08.836969+0000","last_undegraded":"2023-10-26T05:53:08.836969+0000","last_fullsized":"2023-10-26T05:53:08.836969+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:18:22.118581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836461+0000","last_change":"2023-10-26T05:53:07.194083+0000","last_active":"2023-10-26T05:53:08.836461+0000","last_peered":"2023-10-26T05:53:08.836461+0000","last_clean":"2023-10-26T05:53:08.836461+0000","last_became_active":"2023-10-26T05:53:07.193066+0000","last_became_peered":"2023-10-26T05:53:07.193066+0000","last_unstale":"2023-10-26T05:53:08.836461+0000","last_undegraded":"2023-10-26T05:53:08.836461+0000","last_fullsized":"2023-10-26T05:53:08.836461+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:17.575104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166847+0000","last_change":"2023-10-26T05:53:07.201831+0000","last_active":"2023-10-26T05:53:08.166847+0000","last_peered":"2023-10-26T05:53:08.166847+0000","last_clean":"2023-10-26T05:53:08.166847+0000","last_became_active":"2023-10-26T05:53:07.201162+0000","last_became_peered":"2023-10-26T05:53:07.201162+0000","last_unstale":"2023-10-26T05:53:08.166847+0000","last_undegraded":"2023-10-26T05:53:08.166847+0000","last_fullsized":"2023-10-26T05:53:08.166847+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:24:38.356926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838098+0000","last_change":"2023-10-26T05:53:07.191881+0000","last_active":"2023-10-26T05:53:08.838098+0000","last_peered":"2023-10-26T05:53:08.838098+0000","last_clean":"2023-10-26T05:53:08.838098+0000","last_became_active":"2023-10-26T05:53:07.191258+0000","last_became_peered":"2023-10-26T05:53:07.191258+0000","last_unstale":"2023-10-26T05:53:08.838098+0000","last_undegraded":"2023-10-26T05:53:08.838098+0000","last_fullsized":"2023-10-26T05:53:08.838098+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:22:33.018513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838065+0000","last_change":"2023-10-26T05:53:07.173555+0000","last_active":"2023-10-26T05:53:08.838065+0000","last_peered":"2023-10-26T05:53:08.838065+0000","last_clean":"2023-10-26T05:53:08.838065+0000","last_became_active":"2023-10-26T05:53:07.172123+0000","last_became_peered":"2023-10-26T05:53:07.172123+0000","last_unstale":"2023-10-26T05:53:08.838065+0000","last_undegraded":"2023-10-26T05:53:08.838065+0000","last_fullsized":"2023-10-26T05:53:08.838065+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:48.778915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838690+0000","last_change":"2023-10-26T05:53:07.173759+0000","last_active":"2023-10-26T05:53:08.838690+0000","last_peered":"2023-10-26T05:53:08.838690+0000","last_clean":"2023-10-26T05:53:08.838690+0000","last_became_active":"2023-10-26T05:53:07.172986+0000","last_became_peered":"2023-10-26T05:53:07.172986+0000","last_unstale":"2023-10-26T05:53:08.838690+0000","last_undegraded":"2023-10-26T05:53:08.838690+0000","last_fullsized":"2023-10-26T05:53:08.838690+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:03.987842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558244+0000","last_change":"2023-10-26T05:53:07.197307+0000","last_active":"2023-10-26T05:53:08.558244+0000","last_peered":"2023-10-26T05:53:08.558244+0000","last_clean":"2023-10-26T05:53:08.558244+0000","last_became_active":"2023-10-26T05:53:07.196757+0000","last_became_peered":"2023-10-26T05:53:07.196757+0000","last_unstale":"2023-10-26T05:53:08.558244+0000","last_undegraded":"2023-10-26T05:53:08.558244+0000","last_fullsized":"2023-10-26T05:53:08.558244+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:30.904756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839831+0000","last_change":"2023-10-26T05:53:07.185298+0000","last_active":"2023-10-26T05:53:08.839831+0000","last_peered":"2023-10-26T05:53:08.839831+0000","last_clean":"2023-10-26T05:53:08.839831+0000","last_became_active":"2023-10-26T05:53:07.183264+0000","last_became_peered":"2023-10-26T05:53:07.183264+0000","last_unstale":"2023-10-26T05:53:08.839831+0000","last_undegraded":"2023-10-26T05:53:08.839831+0000","last_fullsized":"2023-10-26T05:53:08.839831+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:48.778915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838015+0000","last_change":"2023-10-26T05:53:07.169715+0000","last_active":"2023-10-26T05:53:08.838015+0000","last_peered":"2023-10-26T05:53:08.838015+0000","last_clean":"2023-10-26T05:53:08.838015+0000","last_became_active":"2023-10-26T05:53:07.169040+0000","last_became_peered":"2023-10-26T05:53:07.169040+0000","last_unstale":"2023-10-26T05:53:08.838015+0000","last_undegraded":"2023-10-26T05:53:08.838015+0000","last_fullsized":"2023-10-26T05:53:08.838015+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:40.617492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558427+0000","last_change":"2023-10-26T05:53:07.196187+0000","last_active":"2023-10-26T05:53:08.558427+0000","last_peered":"2023-10-26T05:53:08.558427+0000","last_clean":"2023-10-26T05:53:08.558427+0000","last_became_active":"2023-10-26T05:53:07.195014+0000","last_became_peered":"2023-10-26T05:53:07.195014+0000","last_unstale":"2023-10-26T05:53:08.558427+0000","last_undegraded":"2023-10-26T05:53:08.558427+0000","last_fullsized":"2023-10-26T05:53:08.558427+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:23.912541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839600+0000","last_change":"2023-10-26T05:53:07.204644+0000","last_active":"2023-10-26T05:53:08.839600+0000","last_peered":"2023-10-26T05:53:08.839600+0000","last_clean":"2023-10-26T05:53:08.839600+0000","last_became_active":"2023-10-26T05:53:07.202570+0000","last_became_peered":"2023-10-26T05:53:07.202570+0000","last_unstale":"2023-10-26T05:53:08.839600+0000","last_undegraded":"2023-10-26T05:53:08.839600+0000","last_fullsized":"2023-10-26T05:53:08.839600+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:34:28.391351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836878+0000","last_change":"2023-10-26T05:53:07.167725+0000","last_active":"2023-10-26T05:53:08.836878+0000","last_peered":"2023-10-26T05:53:08.836878+0000","last_clean":"2023-10-26T05:53:08.836878+0000","last_became_active":"2023-10-26T05:53:07.167139+0000","last_became_peered":"2023-10-26T05:53:07.167139+0000","last_unstale":"2023-10-26T05:53:08.836878+0000","last_undegraded":"2023-10-26T05:53:08.836878+0000","last_fullsized":"2023-10-26T05:53:08.836878+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:50:13.925866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838857+0000","last_change":"2023-10-26T05:53:07.193828+0000","last_active":"2023-10-26T05:53:08.838857+0000","last_peered":"2023-10-26T05:53:08.838857+0000","last_clean":"2023-10-26T05:53:08.838857+0000","last_became_active":"2023-10-26T05:53:07.193108+0000","last_became_peered":"2023-10-26T05:53:07.193108+0000","last_unstale":"2023-10-26T05:53:08.838857+0000","last_undegraded":"2023-10-26T05:53:08.838857+0000","last_fullsized":"2023-10-26T05:53:08.838857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:53:39.573829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838920+0000","last_change":"2023-10-26T05:53:07.198952+0000","last_active":"2023-10-26T05:53:08.838920+0000","last_peered":"2023-10-26T05:53:08.838920+0000","last_clean":"2023-10-26T05:53:08.838920+0000","last_became_active":"2023-10-26T05:53:07.198268+0000","last_became_peered":"2023-10-26T05:53:07.198268+0000","last_unstale":"2023-10-26T05:53:08.838920+0000","last_undegraded":"2023-10-26T05:53:08.838920+0000","last_fullsized":"2023-10-26T05:53:08.838920+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:26.786989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160060+0000","last_change":"2023-10-26T05:53:07.172143+0000","last_active":"2023-10-26T05:53:08.160060+0000","last_peered":"2023-10-26T05:53:08.160060+0000","last_clean":"2023-10-26T05:53:08.160060+0000","last_became_active":"2023-10-26T05:53:07.171308+0000","last_became_peered":"2023-10-26T05:53:07.171308+0000","last_unstale":"2023-10-26T05:53:08.160060+0000","last_undegraded":"2023-10-26T05:53:08.160060+0000","last_fullsized":"2023-10-26T05:53:08.160060+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:35:48.527391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836665+0000","last_change":"2023-10-26T05:53:07.198186+0000","last_active":"2023-10-26T05:53:08.836665+0000","last_peered":"2023-10-26T05:53:08.836665+0000","last_clean":"2023-10-26T05:53:08.836665+0000","last_became_active":"2023-10-26T05:53:07.195793+0000","last_became_peered":"2023-10-26T05:53:07.195793+0000","last_unstale":"2023-10-26T05:53:08.836665+0000","last_undegraded":"2023-10-26T05:53:08.836665+0000","last_fullsized":"2023-10-26T05:53:08.836665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:47:51.045817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838938+0000","last_change":"2023-10-26T05:53:06.395372+0000","last_active":"2023-10-26T05:53:08.838938+0000","last_peered":"2023-10-26T05:53:08.838938+0000","last_clean":"2023-10-26T05:53:08.838938+0000","last_became_active":"2023-10-26T05:53:06.395110+0000","last_became_peered":"2023-10-26T05:53:06.395110+0000","last_unstale":"2023-10-26T05:53:08.838938+0000","last_undegraded":"2023-10-26T05:53:08.838938+0000","last_fullsized":"2023-10-26T05:53:08.838938+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:01.916306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837295+0000","last_change":"2023-10-26T05:53:07.199346+0000","last_active":"2023-10-26T05:53:08.837295+0000","last_peered":"2023-10-26T05:53:08.837295+0000","last_clean":"2023-10-26T05:53:08.837295+0000","last_became_active":"2023-10-26T05:53:07.198341+0000","last_became_peered":"2023-10-26T05:53:07.198341+0000","last_unstale":"2023-10-26T05:53:08.837295+0000","last_undegraded":"2023-10-26T05:53:08.837295+0000","last_fullsized":"2023-10-26T05:53:08.837295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:34:42.563897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558476+0000","last_change":"2023-10-26T05:53:06.405867+0000","last_active":"2023-10-26T05:53:08.558476+0000","last_peered":"2023-10-26T05:53:08.558476+0000","last_clean":"2023-10-26T05:53:08.558476+0000","last_became_active":"2023-10-26T05:53:06.405143+0000","last_became_peered":"2023-10-26T05:53:06.405143+0000","last_unstale":"2023-10-26T05:53:08.558476+0000","last_undegraded":"2023-10-26T05:53:08.558476+0000","last_fullsized":"2023-10-26T05:53:08.558476+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:25.545917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160238+0000","last_change":"2023-10-26T05:53:07.169334+0000","last_active":"2023-10-26T05:53:08.160238+0000","last_peered":"2023-10-26T05:53:08.160238+0000","last_clean":"2023-10-26T05:53:08.160238+0000","last_became_active":"2023-10-26T05:53:07.168891+0000","last_became_peered":"2023-10-26T05:53:07.168891+0000","last_unstale":"2023-10-26T05:53:08.160238+0000","last_undegraded":"2023-10-26T05:53:08.160238+0000","last_fullsized":"2023-10-26T05:53:08.160238+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:08.686134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837390+0000","last_change":"2023-10-26T05:53:06.397662+0000","last_active":"2023-10-26T05:53:08.837390+0000","last_peered":"2023-10-26T05:53:08.837390+0000","last_clean":"2023-10-26T05:53:08.837390+0000","last_became_active":"2023-10-26T05:53:06.396440+0000","last_became_peered":"2023-10-26T05:53:06.396440+0000","last_unstale":"2023-10-26T05:53:08.837390+0000","last_undegraded":"2023-10-26T05:53:08.837390+0000","last_fullsized":"2023-10-26T05:53:08.837390+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:46.769372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167185+0000","last_change":"2023-10-26T05:53:07.180308+0000","last_active":"2023-10-26T05:53:08.167185+0000","last_peered":"2023-10-26T05:53:08.167185+0000","last_clean":"2023-10-26T05:53:08.167185+0000","last_became_active":"2023-10-26T05:53:07.179690+0000","last_became_peered":"2023-10-26T05:53:07.179690+0000","last_unstale":"2023-10-26T05:53:08.167185+0000","last_undegraded":"2023-10-26T05:53:08.167185+0000","last_fullsized":"2023-10-26T05:53:08.167185+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:59.146622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838247+0000","last_change":"2023-10-26T05:53:06.394372+0000","last_active":"2023-10-26T05:53:08.838247+0000","last_peered":"2023-10-26T05:53:08.838247+0000","last_clean":"2023-10-26T05:53:08.838247+0000","last_became_active":"2023-10-26T05:53:06.394131+0000","last_became_peered":"2023-10-26T05:53:06.394131+0000","last_unstale":"2023-10-26T05:53:08.838247+0000","last_undegraded":"2023-10-26T05:53:08.838247+0000","last_fullsized":"2023-10-26T05:53:08.838247+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:29.900525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167237+0000","last_change":"2023-10-26T05:53:07.179744+0000","last_active":"2023-10-26T05:53:08.167237+0000","last_peered":"2023-10-26T05:53:08.167237+0000","last_clean":"2023-10-26T05:53:08.167237+0000","last_became_active":"2023-10-26T05:53:07.178781+0000","last_became_peered":"2023-10-26T05:53:07.178781+0000","last_unstale":"2023-10-26T05:53:08.167237+0000","last_undegraded":"2023-10-26T05:53:08.167237+0000","last_fullsized":"2023-10-26T05:53:08.167237+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:22.956125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160333+0000","last_change":"2023-10-26T05:53:06.393786+0000","last_active":"2023-10-26T05:53:08.160333+0000","last_peered":"2023-10-26T05:53:08.160333+0000","last_clean":"2023-10-26T05:53:08.160333+0000","last_became_active":"2023-10-26T05:53:06.393408+0000","last_became_peered":"2023-10-26T05:53:06.393408+0000","last_unstale":"2023-10-26T05:53:08.160333+0000","last_undegraded":"2023-10-26T05:53:08.160333+0000","last_fullsized":"2023-10-26T05:53:08.160333+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:54.013866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558734+0000","last_change":"2023-10-26T05:53:07.195719+0000","last_active":"2023-10-26T05:53:08.558734+0000","last_peered":"2023-10-26T05:53:08.558734+0000","last_clean":"2023-10-26T05:53:08.558734+0000","last_became_active":"2023-10-26T05:53:07.194224+0000","last_became_peered":"2023-10-26T05:53:07.194224+0000","last_unstale":"2023-10-26T05:53:08.558734+0000","last_undegraded":"2023-10-26T05:53:08.558734+0000","last_fullsized":"2023-10-26T05:53:08.558734+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:02.920537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839032+0000","last_change":"2023-10-26T05:53:06.397518+0000","last_active":"2023-10-26T05:53:08.839032+0000","last_peered":"2023-10-26T05:53:08.839032+0000","last_clean":"2023-10-26T05:53:08.839032+0000","last_became_active":"2023-10-26T05:53:06.396476+0000","last_became_peered":"2023-10-26T05:53:06.396476+0000","last_unstale":"2023-10-26T05:53:08.839032+0000","last_undegraded":"2023-10-26T05:53:08.839032+0000","last_fullsized":"2023-10-26T05:53:08.839032+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:09.552549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837048+0000","last_change":"2023-10-26T05:53:07.207464+0000","last_active":"2023-10-26T05:53:08.837048+0000","last_peered":"2023-10-26T05:53:08.837048+0000","last_clean":"2023-10-26T05:53:08.837048+0000","last_became_active":"2023-10-26T05:53:07.206031+0000","last_became_peered":"2023-10-26T05:53:07.206031+0000","last_unstale":"2023-10-26T05:53:08.837048+0000","last_undegraded":"2023-10-26T05:53:08.837048+0000","last_fullsized":"2023-10-26T05:53:08.837048+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:47.773603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840267+0000","last_change":"2023-10-26T05:53:06.403340+0000","last_active":"2023-10-26T05:53:08.840267+0000","last_peered":"2023-10-26T05:53:08.840267+0000","last_clean":"2023-10-26T05:53:08.840267+0000","last_became_active":"2023-10-26T05:53:06.402322+0000","last_became_peered":"2023-10-26T05:53:06.402322+0000","last_unstale":"2023-10-26T05:53:08.840267+0000","last_undegraded":"2023-10-26T05:53:08.840267+0000","last_fullsized":"2023-10-26T05:53:08.840267+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:22.166043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167982+0000","last_change":"2023-10-26T05:53:07.182507+0000","last_active":"2023-10-26T05:53:08.167982+0000","last_peered":"2023-10-26T05:53:08.167982+0000","last_clean":"2023-10-26T05:53:08.167982+0000","last_became_active":"2023-10-26T05:53:07.181585+0000","last_became_peered":"2023-10-26T05:53:07.181585+0000","last_unstale":"2023-10-26T05:53:08.167982+0000","last_undegraded":"2023-10-26T05:53:08.167982+0000","last_fullsized":"2023-10-26T05:53:08.167982+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:23.170274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160639+0000","last_change":"2023-10-26T05:53:06.391373+0000","last_active":"2023-10-26T05:53:08.160639+0000","last_peered":"2023-10-26T05:53:08.160639+0000","last_clean":"2023-10-26T05:53:08.160639+0000","last_became_active":"2023-10-26T05:53:06.391035+0000","last_became_peered":"2023-10-26T05:53:06.391035+0000","last_unstale":"2023-10-26T05:53:08.160639+0000","last_undegraded":"2023-10-26T05:53:08.160639+0000","last_fullsized":"2023-10-26T05:53:08.160639+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837861+0000","last_change":"2023-10-26T05:53:07.171228+0000","last_active":"2023-10-26T05:53:08.837861+0000","last_peered":"2023-10-26T05:53:08.837861+0000","last_clean":"2023-10-26T05:53:08.837861+0000","last_became_active":"2023-10-26T05:53:07.170598+0000","last_became_peered":"2023-10-26T05:53:07.170598+0000","last_unstale":"2023-10-26T05:53:08.837861+0000","last_undegraded":"2023-10-26T05:53:08.837861+0000","last_fullsized":"2023-10-26T05:53:08.837861+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:12.818270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160374+0000","last_change":"2023-10-26T05:53:06.398884+0000","last_active":"2023-10-26T05:53:08.160374+0000","last_peered":"2023-10-26T05:53:08.160374+0000","last_clean":"2023-10-26T05:53:08.160374+0000","last_became_active":"2023-10-26T05:53:06.398615+0000","last_became_peered":"2023-10-26T05:53:06.398615+0000","last_unstale":"2023-10-26T05:53:08.160374+0000","last_undegraded":"2023-10-26T05:53:08.160374+0000","last_fullsized":"2023-10-26T05:53:08.160374+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:58.262527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168144+0000","last_change":"2023-10-26T05:53:07.177301+0000","last_active":"2023-10-26T05:53:08.168144+0000","last_peered":"2023-10-26T05:53:08.168144+0000","last_clean":"2023-10-26T05:53:08.168144+0000","last_became_active":"2023-10-26T05:53:07.176658+0000","last_became_peered":"2023-10-26T05:53:07.176658+0000","last_unstale":"2023-10-26T05:53:08.168144+0000","last_undegraded":"2023-10-26T05:53:08.168144+0000","last_fullsized":"2023-10-26T05:53:08.168144+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:43:11.218597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837412+0000","last_change":"2023-10-26T05:53:06.388081+0000","last_active":"2023-10-26T05:53:08.837412+0000","last_peered":"2023-10-26T05:53:08.837412+0000","last_clean":"2023-10-26T05:53:08.837412+0000","last_became_active":"2023-10-26T05:53:06.387677+0000","last_became_peered":"2023-10-26T05:53:06.387677+0000","last_unstale":"2023-10-26T05:53:08.837412+0000","last_undegraded":"2023-10-26T05:53:08.837412+0000","last_fullsized":"2023-10-26T05:53:08.837412+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:48:32.633122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839392+0000","last_change":"2023-10-26T05:53:07.182868+0000","last_active":"2023-10-26T05:53:08.839392+0000","last_peered":"2023-10-26T05:53:08.839392+0000","last_clean":"2023-10-26T05:53:08.839392+0000","last_became_active":"2023-10-26T05:53:07.182231+0000","last_became_peered":"2023-10-26T05:53:07.182231+0000","last_unstale":"2023-10-26T05:53:08.839392+0000","last_undegraded":"2023-10-26T05:53:08.839392+0000","last_fullsized":"2023-10-26T05:53:08.839392+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:50:13.925866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558396+0000","last_change":"2023-10-26T05:53:06.411876+0000","last_active":"2023-10-26T05:53:08.558396+0000","last_peered":"2023-10-26T05:53:08.558396+0000","last_clean":"2023-10-26T05:53:08.558396+0000","last_became_active":"2023-10-26T05:53:06.411270+0000","last_became_peered":"2023-10-26T05:53:06.411270+0000","last_unstale":"2023-10-26T05:53:08.558396+0000","last_undegraded":"2023-10-26T05:53:08.558396+0000","last_fullsized":"2023-10-26T05:53:08.558396+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:10.007577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558263+0000","last_change":"2023-10-26T05:53:07.207930+0000","last_active":"2023-10-26T05:53:08.558263+0000","last_peered":"2023-10-26T05:53:08.558263+0000","last_clean":"2023-10-26T05:53:08.558263+0000","last_became_active":"2023-10-26T05:53:07.207183+0000","last_became_peered":"2023-10-26T05:53:07.207183+0000","last_unstale":"2023-10-26T05:53:08.558263+0000","last_undegraded":"2023-10-26T05:53:08.558263+0000","last_fullsized":"2023-10-26T05:53:08.558263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:58.341073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168431+0000","last_change":"2023-10-26T05:53:06.398397+0000","last_active":"2023-10-26T05:53:08.168431+0000","last_peered":"2023-10-26T05:53:08.168431+0000","last_clean":"2023-10-26T05:53:08.168431+0000","last_became_active":"2023-10-26T05:53:06.397755+0000","last_became_peered":"2023-10-26T05:53:06.397755+0000","last_unstale":"2023-10-26T05:53:08.168431+0000","last_undegraded":"2023-10-26T05:53:08.168431+0000","last_fullsized":"2023-10-26T05:53:08.168431+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:29.386538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839235+0000","last_change":"2023-10-26T05:53:07.197465+0000","last_active":"2023-10-26T05:53:08.839235+0000","last_peered":"2023-10-26T05:53:08.839235+0000","last_clean":"2023-10-26T05:53:08.839235+0000","last_became_active":"2023-10-26T05:53:07.196658+0000","last_became_peered":"2023-10-26T05:53:07.196658+0000","last_unstale":"2023-10-26T05:53:08.839235+0000","last_undegraded":"2023-10-26T05:53:08.839235+0000","last_fullsized":"2023-10-26T05:53:08.839235+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:24:38.356926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839394+0000","last_change":"2023-10-26T05:53:06.408807+0000","last_active":"2023-10-26T05:53:08.839394+0000","last_peered":"2023-10-26T05:53:08.839394+0000","last_clean":"2023-10-26T05:53:08.839394+0000","last_became_active":"2023-10-26T05:53:06.408574+0000","last_became_peered":"2023-10-26T05:53:06.408574+0000","last_unstale":"2023-10-26T05:53:08.839394+0000","last_undegraded":"2023-10-26T05:53:08.839394+0000","last_fullsized":"2023-10-26T05:53:08.839394+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:32:58.378694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160497+0000","last_change":"2023-10-26T05:53:07.192552+0000","last_active":"2023-10-26T05:53:08.160497+0000","last_peered":"2023-10-26T05:53:08.160497+0000","last_clean":"2023-10-26T05:53:08.160497+0000","last_became_active":"2023-10-26T05:53:07.191785+0000","last_became_peered":"2023-10-26T05:53:07.191785+0000","last_unstale":"2023-10-26T05:53:08.160497+0000","last_undegraded":"2023-10-26T05:53:08.160497+0000","last_fullsized":"2023-10-26T05:53:08.160497+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:41:15.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838445+0000","last_change":"2023-10-26T05:53:06.395064+0000","last_active":"2023-10-26T05:53:08.838445+0000","last_peered":"2023-10-26T05:53:08.838445+0000","last_clean":"2023-10-26T05:53:08.838445+0000","last_became_active":"2023-10-26T05:53:06.394740+0000","last_became_peered":"2023-10-26T05:53:06.394740+0000","last_unstale":"2023-10-26T05:53:08.838445+0000","last_undegraded":"2023-10-26T05:53:08.838445+0000","last_fullsized":"2023-10-26T05:53:08.838445+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:25.527872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167218+0000","last_change":"2023-10-26T05:53:07.176936+0000","last_active":"2023-10-26T05:53:08.167218+0000","last_peered":"2023-10-26T05:53:08.167218+0000","last_clean":"2023-10-26T05:53:08.167218+0000","last_became_active":"2023-10-26T05:53:07.176302+0000","last_became_peered":"2023-10-26T05:53:07.176302+0000","last_unstale":"2023-10-26T05:53:08.167218+0000","last_undegraded":"2023-10-26T05:53:08.167218+0000","last_fullsized":"2023-10-26T05:53:08.167218+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:30.904756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168477+0000","last_change":"2023-10-26T05:53:06.411521+0000","last_active":"2023-10-26T05:53:08.168477+0000","last_peered":"2023-10-26T05:53:08.168477+0000","last_clean":"2023-10-26T05:53:08.168477+0000","last_became_active":"2023-10-26T05:53:06.410661+0000","last_became_peered":"2023-10-26T05:53:06.410661+0000","last_unstale":"2023-10-26T05:53:08.168477+0000","last_undegraded":"2023-10-26T05:53:08.168477+0000","last_fullsized":"2023-10-26T05:53:08.168477+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:34:41.559666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838559+0000","last_change":"2023-10-26T05:53:07.201568+0000","last_active":"2023-10-26T05:53:08.838559+0000","last_peered":"2023-10-26T05:53:08.838559+0000","last_clean":"2023-10-26T05:53:08.838559+0000","last_became_active":"2023-10-26T05:53:07.198500+0000","last_became_peered":"2023-10-26T05:53:07.198500+0000","last_unstale":"2023-10-26T05:53:08.838559+0000","last_undegraded":"2023-10-26T05:53:08.838559+0000","last_fullsized":"2023-10-26T05:53:08.838559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:59.935036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160537+0000","last_change":"2023-10-26T05:53:06.402525+0000","last_active":"2023-10-26T05:53:08.160537+0000","last_peered":"2023-10-26T05:53:08.160537+0000","last_clean":"2023-10-26T05:53:08.160537+0000","last_became_active":"2023-10-26T05:53:06.402284+0000","last_became_peered":"2023-10-26T05:53:06.402284+0000","last_unstale":"2023-10-26T05:53:08.160537+0000","last_undegraded":"2023-10-26T05:53:08.160537+0000","last_fullsized":"2023-10-26T05:53:08.160537+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:25:40.044565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839724+0000","last_change":"2023-10-26T05:53:07.185247+0000","last_active":"2023-10-26T05:53:08.839724+0000","last_peered":"2023-10-26T05:53:08.839724+0000","last_clean":"2023-10-26T05:53:08.839724+0000","last_became_active":"2023-10-26T05:53:07.184670+0000","last_became_peered":"2023-10-26T05:53:07.184670+0000","last_unstale":"2023-10-26T05:53:08.839724+0000","last_undegraded":"2023-10-26T05:53:08.839724+0000","last_fullsized":"2023-10-26T05:53:08.839724+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:40.617492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167426+0000","last_change":"2023-10-26T05:53:06.412164+0000","last_active":"2023-10-26T05:53:08.167426+0000","last_peered":"2023-10-26T05:53:08.167426+0000","last_clean":"2023-10-26T05:53:08.167426+0000","last_became_active":"2023-10-26T05:53:06.411078+0000","last_became_peered":"2023-10-26T05:53:06.411078+0000","last_unstale":"2023-10-26T05:53:08.167426+0000","last_undegraded":"2023-10-26T05:53:08.167426+0000","last_fullsized":"2023-10-26T05:53:08.167426+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:46.769372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166509+0000","last_change":"2023-10-26T05:53:07.184658+0000","last_active":"2023-10-26T05:53:08.166509+0000","last_peered":"2023-10-26T05:53:08.166509+0000","last_clean":"2023-10-26T05:53:08.166509+0000","last_became_active":"2023-10-26T05:53:07.183963+0000","last_became_peered":"2023-10-26T05:53:07.183963+0000","last_unstale":"2023-10-26T05:53:08.166509+0000","last_undegraded":"2023-10-26T05:53:08.166509+0000","last_fullsized":"2023-10-26T05:53:08.166509+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:40:56.956703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839442+0000","last_change":"2023-10-26T05:53:06.401661+0000","last_active":"2023-10-26T05:53:08.839442+0000","last_peered":"2023-10-26T05:53:08.839442+0000","last_clean":"2023-10-26T05:53:08.839442+0000","last_became_active":"2023-10-26T05:53:06.400939+0000","last_became_peered":"2023-10-26T05:53:06.400939+0000","last_unstale":"2023-10-26T05:53:08.839442+0000","last_undegraded":"2023-10-26T05:53:08.839442+0000","last_fullsized":"2023-10-26T05:53:08.839442+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:11.736607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.559300+0000","last_change":"2023-10-26T05:53:07.198191+0000","last_active":"2023-10-26T05:53:08.559300+0000","last_peered":"2023-10-26T05:53:08.559300+0000","last_clean":"2023-10-26T05:53:08.559300+0000","last_became_active":"2023-10-26T05:53:07.197501+0000","last_became_peered":"2023-10-26T05:53:07.197501+0000","last_unstale":"2023-10-26T05:53:08.559300+0000","last_undegraded":"2023-10-26T05:53:08.559300+0000","last_fullsized":"2023-10-26T05:53:08.559300+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:12.740838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838290+0000","last_change":"2023-10-26T05:53:06.397106+0000","last_active":"2023-10-26T05:53:08.838290+0000","last_peered":"2023-10-26T05:53:08.838290+0000","last_clean":"2023-10-26T05:53:08.838290+0000","last_became_active":"2023-10-26T05:53:06.396761+0000","last_became_peered":"2023-10-26T05:53:06.396761+0000","last_unstale":"2023-10-26T05:53:08.838290+0000","last_undegraded":"2023-10-26T05:53:08.838290+0000","last_fullsized":"2023-10-26T05:53:08.838290+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:13.044276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836876+0000","last_change":"2023-10-26T05:53:07.207074+0000","last_active":"2023-10-26T05:53:08.836876+0000","last_peered":"2023-10-26T05:53:08.836876+0000","last_clean":"2023-10-26T05:53:08.836876+0000","last_became_active":"2023-10-26T05:53:07.203547+0000","last_became_peered":"2023-10-26T05:53:07.203547+0000","last_unstale":"2023-10-26T05:53:08.836876+0000","last_undegraded":"2023-10-26T05:53:08.836876+0000","last_fullsized":"2023-10-26T05:53:08.836876+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:49.370880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839102+0000","last_change":"2023-10-26T05:53:06.397826+0000","last_active":"2023-10-26T05:53:08.839102+0000","last_peered":"2023-10-26T05:53:08.839102+0000","last_clean":"2023-10-26T05:53:08.839102+0000","last_became_active":"2023-10-26T05:53:06.397212+0000","last_became_peered":"2023-10-26T05:53:06.397212+0000","last_unstale":"2023-10-26T05:53:08.839102+0000","last_undegraded":"2023-10-26T05:53:08.839102+0000","last_fullsized":"2023-10-26T05:53:08.839102+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:11.736607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836372+0000","last_change":"2023-10-26T05:53:07.204887+0000","last_active":"2023-10-26T05:53:08.836372+0000","last_peered":"2023-10-26T05:53:08.836372+0000","last_clean":"2023-10-26T05:53:08.836372+0000","last_became_active":"2023-10-26T05:53:07.204398+0000","last_became_peered":"2023-10-26T05:53:07.204398+0000","last_unstale":"2023-10-26T05:53:08.836372+0000","last_undegraded":"2023-10-26T05:53:08.836372+0000","last_fullsized":"2023-10-26T05:53:08.836372+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:22:44.879819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.165912+0000","last_change":"2023-10-26T05:53:06.408819+0000","last_active":"2023-10-26T05:53:08.165912+0000","last_peered":"2023-10-26T05:53:08.165912+0000","last_clean":"2023-10-26T05:53:08.165912+0000","last_became_active":"2023-10-26T05:53:06.408262+0000","last_became_peered":"2023-10-26T05:53:06.408262+0000","last_unstale":"2023-10-26T05:53:08.165912+0000","last_undegraded":"2023-10-26T05:53:08.165912+0000","last_fullsized":"2023-10-26T05:53:08.165912+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:55.244617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838307+0000","last_change":"2023-10-26T05:53:07.202024+0000","last_active":"2023-10-26T05:53:08.838307+0000","last_peered":"2023-10-26T05:53:08.838307+0000","last_clean":"2023-10-26T05:53:08.838307+0000","last_became_active":"2023-10-26T05:53:07.201337+0000","last_became_peered":"2023-10-26T05:53:07.201337+0000","last_unstale":"2023-10-26T05:53:08.838307+0000","last_undegraded":"2023-10-26T05:53:08.838307+0000","last_fullsized":"2023-10-26T05:53:08.838307+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:15:33.880787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166346+0000","last_change":"2023-10-26T05:53:06.399738+0000","last_active":"2023-10-26T05:53:08.166346+0000","last_peered":"2023-10-26T05:53:08.166346+0000","last_clean":"2023-10-26T05:53:08.166346+0000","last_became_active":"2023-10-26T05:53:06.398936+0000","last_became_peered":"2023-10-26T05:53:06.398936+0000","last_unstale":"2023-10-26T05:53:08.166346+0000","last_undegraded":"2023-10-26T05:53:08.166346+0000","last_fullsized":"2023-10-26T05:53:08.166346+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:22.752003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840012+0000","last_change":"2023-10-26T05:53:07.184936+0000","last_active":"2023-10-26T05:53:08.840012+0000","last_peered":"2023-10-26T05:53:08.840012+0000","last_clean":"2023-10-26T05:53:08.840012+0000","last_became_active":"2023-10-26T05:53:07.182415+0000","last_became_peered":"2023-10-26T05:53:07.182415+0000","last_unstale":"2023-10-26T05:53:08.840012+0000","last_undegraded":"2023-10-26T05:53:08.840012+0000","last_fullsized":"2023-10-26T05:53:08.840012+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:03.987842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838453+0000","last_change":"2023-10-26T05:53:06.397742+0000","last_active":"2023-10-26T05:53:08.838453+0000","last_peered":"2023-10-26T05:53:08.838453+0000","last_clean":"2023-10-26T05:53:08.838453+0000","last_became_active":"2023-10-26T05:53:06.397344+0000","last_became_peered":"2023-10-26T05:53:06.397344+0000","last_unstale":"2023-10-26T05:53:08.838453+0000","last_undegraded":"2023-10-26T05:53:08.838453+0000","last_fullsized":"2023-10-26T05:53:08.838453+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:57.336842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838804+0000","last_change":"2023-10-26T05:53:07.173377+0000","last_active":"2023-10-26T05:53:08.838804+0000","last_peered":"2023-10-26T05:53:08.838804+0000","last_clean":"2023-10-26T05:53:08.838804+0000","last_became_active":"2023-10-26T05:53:07.172149+0000","last_became_peered":"2023-10-26T05:53:07.172149+0000","last_unstale":"2023-10-26T05:53:08.838804+0000","last_undegraded":"2023-10-26T05:53:08.838804+0000","last_fullsized":"2023-10-26T05:53:08.838804+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:08.955683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166746+0000","last_change":"2023-10-26T05:53:06.410619+0000","last_active":"2023-10-26T05:53:08.166746+0000","last_peered":"2023-10-26T05:53:08.166746+0000","last_clean":"2023-10-26T05:53:08.166746+0000","last_became_active":"2023-10-26T05:53:06.409962+0000","last_became_peered":"2023-10-26T05:53:06.409962+0000","last_unstale":"2023-10-26T05:53:08.166746+0000","last_undegraded":"2023-10-26T05:53:08.166746+0000","last_fullsized":"2023-10-26T05:53:08.166746+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:50.775359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160221+0000","last_change":"2023-10-26T05:53:07.191280+0000","last_active":"2023-10-26T05:53:08.160221+0000","last_peered":"2023-10-26T05:53:08.160221+0000","last_clean":"2023-10-26T05:53:08.160221+0000","last_became_active":"2023-10-26T05:53:07.190487+0000","last_became_peered":"2023-10-26T05:53:07.190487+0000","last_unstale":"2023-10-26T05:53:08.160221+0000","last_undegraded":"2023-10-26T05:53:08.160221+0000","last_fullsized":"2023-10-26T05:53:08.160221+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:47:19.408575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839166+0000","last_change":"2023-10-26T05:53:06.397697+0000","last_active":"2023-10-26T05:53:08.839166+0000","last_peered":"2023-10-26T05:53:08.839166+0000","last_clean":"2023-10-26T05:53:08.839166+0000","last_became_active":"2023-10-26T05:53:06.396875+0000","last_became_peered":"2023-10-26T05:53:06.396875+0000","last_unstale":"2023-10-26T05:53:08.839166+0000","last_undegraded":"2023-10-26T05:53:08.839166+0000","last_fullsized":"2023-10-26T05:53:08.839166+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:26.420989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160582+0000","last_change":"2023-10-26T05:53:07.182109+0000","last_active":"2023-10-26T05:53:08.160582+0000","last_peered":"2023-10-26T05:53:08.160582+0000","last_clean":"2023-10-26T05:53:08.160582+0000","last_became_active":"2023-10-26T05:53:07.181628+0000","last_became_peered":"2023-10-26T05:53:07.181628+0000","last_unstale":"2023-10-26T05:53:08.160582+0000","last_undegraded":"2023-10-26T05:53:08.160582+0000","last_fullsized":"2023-10-26T05:53:08.160582+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:12.418100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837275+0000","last_change":"2023-10-26T05:53:06.394474+0000","last_active":"2023-10-26T05:53:08.837275+0000","last_peered":"2023-10-26T05:53:08.837275+0000","last_clean":"2023-10-26T05:53:08.837275+0000","last_became_active":"2023-10-26T05:53:06.394227+0000","last_became_peered":"2023-10-26T05:53:06.394227+0000","last_unstale":"2023-10-26T05:53:08.837275+0000","last_undegraded":"2023-10-26T05:53:08.837275+0000","last_fullsized":"2023-10-26T05:53:08.837275+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:25.782758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840057+0000","last_change":"2023-10-26T05:53:07.200294+0000","last_active":"2023-10-26T05:53:08.840057+0000","last_peered":"2023-10-26T05:53:08.840057+0000","last_clean":"2023-10-26T05:53:08.840057+0000","last_became_active":"2023-10-26T05:53:07.199453+0000","last_became_peered":"2023-10-26T05:53:07.199453+0000","last_unstale":"2023-10-26T05:53:08.840057+0000","last_undegraded":"2023-10-26T05:53:08.840057+0000","last_fullsized":"2023-10-26T05:53:08.840057+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:08.955683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840134+0000","last_change":"2023-10-26T05:53:06.406227+0000","last_active":"2023-10-26T05:53:08.840134+0000","last_peered":"2023-10-26T05:53:08.840134+0000","last_clean":"2023-10-26T05:53:08.840134+0000","last_became_active":"2023-10-26T05:53:06.405621+0000","last_became_peered":"2023-10-26T05:53:06.405621+0000","last_unstale":"2023-10-26T05:53:08.840134+0000","last_undegraded":"2023-10-26T05:53:08.840134+0000","last_fullsized":"2023-10-26T05:53:08.840134+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:13.044276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838886+0000","last_change":"2023-10-26T05:53:07.201173+0000","last_active":"2023-10-26T05:53:08.838886+0000","last_peered":"2023-10-26T05:53:08.838886+0000","last_clean":"2023-10-26T05:53:08.838886+0000","last_became_active":"2023-10-26T05:53:07.197659+0000","last_became_peered":"2023-10-26T05:53:07.197659+0000","last_unstale":"2023-10-26T05:53:08.838886+0000","last_undegraded":"2023-10-26T05:53:08.838886+0000","last_fullsized":"2023-10-26T05:53:08.838886+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:37.549915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838676+0000","last_change":"2023-10-26T05:53:06.386286+0000","last_active":"2023-10-26T05:53:08.838676+0000","last_peered":"2023-10-26T05:53:08.838676+0000","last_clean":"2023-10-26T05:53:08.838676+0000","last_became_active":"2023-10-26T05:53:06.385960+0000","last_became_peered":"2023-10-26T05:53:06.385960+0000","last_unstale":"2023-10-26T05:53:08.838676+0000","last_undegraded":"2023-10-26T05:53:08.838676+0000","last_fullsized":"2023-10-26T05:53:08.838676+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:22.908310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168548+0000","last_change":"2023-10-26T05:53:07.181149+0000","last_active":"2023-10-26T05:53:08.168548+0000","last_peered":"2023-10-26T05:53:08.168548+0000","last_clean":"2023-10-26T05:53:08.168548+0000","last_became_active":"2023-10-26T05:53:07.180492+0000","last_became_peered":"2023-10-26T05:53:07.180492+0000","last_unstale":"2023-10-26T05:53:08.168548+0000","last_undegraded":"2023-10-26T05:53:08.168548+0000","last_fullsized":"2023-10-26T05:53:08.168548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:02.920537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839849+0000","last_change":"2023-10-26T05:53:06.396204+0000","last_active":"2023-10-26T05:53:08.839849+0000","last_peered":"2023-10-26T05:53:08.839849+0000","last_clean":"2023-10-26T05:53:08.839849+0000","last_became_active":"2023-10-26T05:53:06.395499+0000","last_became_peered":"2023-10-26T05:53:06.395499+0000","last_unstale":"2023-10-26T05:53:08.839849+0000","last_undegraded":"2023-10-26T05:53:08.839849+0000","last_fullsized":"2023-10-26T05:53:08.839849+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:43:10.214366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837077+0000","last_change":"2023-10-26T05:53:07.193645+0000","last_active":"2023-10-26T05:53:08.837077+0000","last_peered":"2023-10-26T05:53:08.837077+0000","last_clean":"2023-10-26T05:53:08.837077+0000","last_became_active":"2023-10-26T05:53:07.192339+0000","last_became_peered":"2023-10-26T05:53:07.192339+0000","last_unstale":"2023-10-26T05:53:08.837077+0000","last_undegraded":"2023-10-26T05:53:08.837077+0000","last_fullsized":"2023-10-26T05:53:08.837077+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:14:46.807945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837697+0000","last_change":"2023-10-26T05:53:06.398766+0000","last_active":"2023-10-26T05:53:08.837697+0000","last_peered":"2023-10-26T05:53:08.837697+0000","last_clean":"2023-10-26T05:53:08.837697+0000","last_became_active":"2023-10-26T05:53:06.398518+0000","last_became_peered":"2023-10-26T05:53:06.398518+0000","last_unstale":"2023-10-26T05:53:08.837697+0000","last_undegraded":"2023-10-26T05:53:08.837697+0000","last_fullsized":"2023-10-26T05:53:08.837697+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:52.229175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839940+0000","last_change":"2023-10-26T05:53:07.204212+0000","last_active":"2023-10-26T05:53:08.839940+0000","last_peered":"2023-10-26T05:53:08.839940+0000","last_clean":"2023-10-26T05:53:08.839940+0000","last_became_active":"2023-10-26T05:53:07.201824+0000","last_became_peered":"2023-10-26T05:53:07.201824+0000","last_unstale":"2023-10-26T05:53:08.839940+0000","last_undegraded":"2023-10-26T05:53:08.839940+0000","last_fullsized":"2023-10-26T05:53:08.839940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:40.082223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837169+0000","last_change":"2023-10-26T05:53:06.399777+0000","last_active":"2023-10-26T05:53:08.837169+0000","last_peered":"2023-10-26T05:53:08.837169+0000","last_clean":"2023-10-26T05:53:08.837169+0000","last_became_active":"2023-10-26T05:53:06.399571+0000","last_became_peered":"2023-10-26T05:53:06.399571+0000","last_unstale":"2023-10-26T05:53:08.837169+0000","last_undegraded":"2023-10-26T05:53:08.837169+0000","last_fullsized":"2023-10-26T05:53:08.837169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:14:08.684639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838527+0000","last_change":"2023-10-26T05:53:07.201930+0000","last_active":"2023-10-26T05:53:08.838527+0000","last_peered":"2023-10-26T05:53:08.838527+0000","last_clean":"2023-10-26T05:53:08.838527+0000","last_became_active":"2023-10-26T05:53:07.201156+0000","last_became_peered":"2023-10-26T05:53:07.201156+0000","last_unstale":"2023-10-26T05:53:08.838527+0000","last_undegraded":"2023-10-26T05:53:08.838527+0000","last_fullsized":"2023-10-26T05:53:08.838527+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:32:47.855349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558693+0000","last_change":"2023-10-26T05:53:06.407201+0000","last_active":"2023-10-26T05:53:08.558693+0000","last_peered":"2023-10-26T05:53:08.558693+0000","last_clean":"2023-10-26T05:53:08.558693+0000","last_became_active":"2023-10-26T05:53:06.406550+0000","last_became_peered":"2023-10-26T05:53:06.406550+0000","last_unstale":"2023-10-26T05:53:08.558693+0000","last_undegraded":"2023-10-26T05:53:08.558693+0000","last_fullsized":"2023-10-26T05:53:08.558693+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:52.229175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-26T05:53:07.189501+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:07.189501+0000","last_peered":"2023-10-26T05:53:07.189501+0000","last_clean":"2023-10-26T05:53:07.189501+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:07.189501+0000","last_undegraded":"2023-10-26T05:53:07.189501+0000","last_fullsized":"2023-10-26T05:53:07.189501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1244,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194067,"statfs":{"total":200000000,"available":198725406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84099999999999997,"15min":0.86099999999999999},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3100000000000001,"5min":1.347,"15min":1.347},"last":0.502},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84299999999999997,"15min":0.83499999999999996},"min":{"1min":0.54700000000000004,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.171,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.97799999999999998,"15min":1.038},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.96499999999999997,"15min":0.97199999999999998},"min":{"1min":0.75800000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.141}]},{"osd":2,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.93999999999999995},"min":{"1min":0.67600000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3700000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.72399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1479999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.749},{"interface":"front","average":{"1min":1.018,"5min":0.98499999999999999,"15min":0.97699999999999998},"min":{"1min":0.65200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2410000000000001,"5min":1.375,"15min":1.375},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.91300000000000003,"15min":0.91900000000000004},"min":{"1min":0.65200000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95399999999999996,"15min":0.93999999999999995},"min":{"1min":0.78300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.159,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.109}]},{"osd":5,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0469999999999999,"15min":1.1180000000000001},"min":{"1min":0.74099999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3899999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0389999999999999,"15min":1.087},"min":{"1min":0.625,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.25}]},{"osd":6,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.0780000000000001,"15min":1.111},"min":{"1min":0.76600000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0529999999999999,"15min":1.05},"min":{"1min":0.77100000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.262,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.28}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.80700000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.103},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.871,"15min":0.86099999999999999},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.04,"5min":1.3,"15min":1.3},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82499999999999996,"15min":0.77300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1399999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89400000000000002,"15min":0.84099999999999997},"min":{"1min":0.52300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.323,"5min":1.405,"15min":1.405},"last":1.167}]},{"osd":2,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.81000000000000005},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.874,"15min":0.83699999999999997},"min":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95199999999999996,"15min":0.89000000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2490000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.024,"15min":0.96199999999999997},"min":{"1min":0.73499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.2490000000000001,"5min":1.399,"15min":1.399},"last":1.056}]},{"osd":4,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.94099999999999995,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.429,"15min":1.429},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.97399999999999998,"15min":0.97099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.258,"5min":1.365,"15min":1.365},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.022,"15min":1.044},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3149999999999999,"5min":1.554,"15min":1.554},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.99099999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.262,"5min":1.282,"15min":1.282},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1040000000000001,"15min":1.101},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2909999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0900000000000001,"15min":1.1140000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.204,"5min":1.464,"15min":1.464},"last":1.0800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194644,"statfs":{"total":200000000,"available":199315816,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.95899999999999996},"min":{"1min":0.60599999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3700000000000001,"5min":1.579,"15min":1.579},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2030000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.1160000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.89600000000000002},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2170000000000001,"5min":1.464,"15min":1.464},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.874,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1850000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.272,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.90700000000000003,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.198,"5min":1.548,"15min":1.548},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.98099999999999998,"15min":1.006},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95199999999999996,"15min":1.004},"min":{"1min":0.66900000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1240000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.878},{"interface":"front","average":{"1min":1.097,"5min":1.0700000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.95899999999999996,"15min":1.0269999999999999},"min":{"1min":0.62,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3640000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.012},{"interface":"front","average":{"1min":1.036,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.624,"5min":0.504,"15min":0.504},"max":{"1min":1.4279999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.081}]},{"osd":7,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0589999999999999,"15min":1.0780000000000001},"min":{"1min":0.64500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.341,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.121,"15min":1.1679999999999999},"min":{"1min":0.89600000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3500000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1910000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314150,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.871,"15min":0.84999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2569999999999999,"5min":1.347,"15min":1.347},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.86899999999999999,"15min":0.879},"min":{"1min":0.61199999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85099999999999998,"15min":0.81299999999999994},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.25,"5min":1.375,"15min":1.375},"last":1.012},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92500000000000004,"15min":0.89200000000000002},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.262,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85399999999999998,"15min":0.81200000000000006},"min":{"1min":0.52400000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.1399999999999999,"5min":1.29,"15min":1.29},"last":1.052},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.89100000000000001,"15min":0.86399999999999999},"min":{"1min":0.57199999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1970000000000001,"5min":1.373,"15min":1.373},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92300000000000004,"15min":0.94099999999999995},"min":{"1min":0.70099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.996,"5min":0.97799999999999998,"15min":0.94399999999999995},"min":{"1min":0.64500000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90400000000000003,"15min":0.88600000000000001},"min":{"1min":0.55900000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.306,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0049999999999999,"15min":1.0009999999999999},"min":{"1min":0.58299999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4179999999999999,"5min":1.673,"15min":1.673},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.02,"15min":1.002},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.369,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.044,"5min":1.0329999999999999,"15min":1.0369999999999999},"min":{"1min":0.74299999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.292,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.871,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.341,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.74199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3100000000000001,"5min":1.728,"15min":1.728},"last":0.79900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":663,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194648,"statfs":{"total":200000000,"available":199320552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0680000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.115}]},{"osd":1,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64200000000000002,"15min":0.65600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.2,"5min":1.349,"15min":1.349},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.59799999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.075,"5min":1.085,"15min":1.085},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.75},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3460000000000001,"5min":1.367,"15min":1.367},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.41599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1850000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.872,"15min":0.82699999999999996},"min":{"1min":0.61499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.64300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.413,"5min":1.478,"15min":1.478},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.93899999999999995,"15min":0.90600000000000003},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.0509999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94899999999999995,"15min":0.997},"min":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.054,"15min":1.028},"min":{"1min":0.629,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":1.141}]},{"osd":7,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.96799999999999997,"15min":0.92400000000000004},"min":{"1min":0.69299999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.171},{"interface":"front","average":{"1min":1.04,"5min":1.036,"15min":1.0449999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.016}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194644,"statfs":{"total":200000000,"available":199315852,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.93300000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67900000000000005,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0409999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.66600000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.226,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67300000000000004,"15min":0.59399999999999997},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1970000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.66400000000000003,"15min":0.59099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.94299999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89700000000000002,"15min":0.82299999999999995},"min":{"1min":0.628,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3160000000000001,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91000000000000003,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2310000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":1.075}]},{"osd":5,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.97799999999999998,"15min":0.90700000000000003},"min":{"1min":0.68000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3819999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.96099999999999997,"15min":0.90000000000000002},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3069999999999999,"5min":1.653,"15min":1.653},"last":1.21}]},{"osd":6,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.98899999999999999,"15min":0.91500000000000004},"min":{"1min":0.65000000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.49,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.002},{"interface":"front","average":{"1min":1.107,"5min":1.089,"15min":0.96999999999999997},"min":{"1min":0.59999999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.4319999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":1.169}]},{"osd":7,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.94899999999999995,"15min":0.89900000000000002},"min":{"1min":0.63700000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.202,"15min":1.0269999999999999},"min":{"1min":0.76100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.615,"5min":15.352,"15min":15.352},"last":1.0980000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194644,"statfs":{"total":200000000,"available":199315886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.29599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80900000000000005,"5min":1.157,"15min":1.157},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.71999999999999997},"min":{"1min":0.38,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6080000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.38100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.121,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70199999999999996,"15min":0.68300000000000005},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.81200000000000006}]},{"osd":4,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.88600000000000001,"15min":0.86799999999999999},"min":{"1min":0.64800000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.755},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.68500000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5069999999999999,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.0589999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96399999999999997},"min":{"1min":0.66700000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.381,"5min":1.492,"15min":1.492},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.71899999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.91900000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.022,"15min":1.038},"min":{"1min":0.71299999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4490000000000001,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.71299999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.468,"5min":1.837,"15min":1.837},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0149999999999999,"15min":1.0289999999999999},"min":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4179999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.137},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.095,"15min":1.093},"min":{"1min":0.77300000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8180000000000001,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.1739999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1245,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194067,"statfs":{"total":200000000,"available":198724936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64100000000000001,"15min":0.59899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.72399999999999998,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.36599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.96699999999999997,"5min":1.177,"15min":1.177},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95299999999999996,"5min":1.409,"15min":1.409},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72799999999999998,"15min":0.68400000000000005},"min":{"1min":0.32100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.1060000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88900000000000001,"15min":0.84799999999999998},"min":{"1min":0.69899999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1200000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.93000000000000005,"15min":0.89600000000000002},"min":{"1min":0.59799999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.399,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.169}]},{"osd":5,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3169999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.90300000000000002},"min":{"1min":0.70799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1379999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.93700000000000006,"15min":0.90600000000000003},"min":{"1min":0.624,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.004,"15min":0.95999999999999996},"min":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.486,"5min":1.764,"15min":1.764},"last":1.117}]},{"osd":7,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.996},"min":{"1min":0.69799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.669,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.095},{"interface":"front","average":{"1min":1.05,"5min":1.012,"15min":0.98299999999999998},"min":{"1min":0.76700000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3080000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.2070000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:10.434 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- 172.21.15.71:0/3766457630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40052b0 msgr2=0x7f75d400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 --2- 172.21.15.71:0/3766457630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40052b0 0x7f75d400b990 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f75dc006fb0 tx=0x7f75dc01f700 comp rx=0 tx=0).stop 2023-10-26T05:53:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- 172.21.15.71:0/3766457630 shutdown_connections 2023-10-26T05:53:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 --2- 172.21.15.71:0/3766457630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d40052b0 0x7f75d400b990 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- 172.21.15.71:0/3766457630 >> 172.21.15.71:0/3766457630 conn(0x7f75ec0cdd80 msgr2=0x7f75ec0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- 172.21.15.71:0/3766457630 shutdown_connections 2023-10-26T05:53:10.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- 172.21.15.71:0/3766457630 wait complete. 2023-10-26T05:53:10.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 Processor -- start 2023-10-26T05:53:10.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- start start 2023-10-26T05:53:10.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d40052b0 0x7f75ec154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 0x7f75ec157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75ec15d1c0 0x7f75ec15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec06a650 con 0x7f75ec15bd60 2023-10-26T05:53:10.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec15db80 con 0x7f75d40052b0 2023-10-26T05:53:10.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75f19a5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec154a80 con 0x7f75ec15d1c0 2023-10-26T05:53:10.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 0x7f75ec157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 0x7f75ec157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52986/0 (socket says 172.21.15.71:52986) 2023-10-26T05:53:10.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 -- 172.21.15.71:0/1109087944 learned_addr learned my addr 172.21.15.71:0/1109087944 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:10.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75eaffd640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d40052b0 0x7f75ec154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75eb7fe640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75ec15d1c0 0x7f75ec15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 -- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75ec15d1c0 msgr2=0x7f75ec15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75ec15d1c0 0x7f75ec15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 -- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d40052b0 msgr2=0x7f75ec154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d40052b0 0x7f75ec154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75ea7fc640 1 -- 172.21.15.71:0/1109087944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec158320 con 0x7f75ec15bd60 2023-10-26T05:53:10.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.588+0000 7f75eaffd640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d40052b0 0x7f75ec154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:53:10.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75ea7fc640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 0x7f75ec157de0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f75e0004d20 tx=0x7f75e0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:10.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0016030 con 0x7f75ec15bd60 2023-10-26T05:53:10.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f75e00108c0 con 0x7f75ec15bd60 2023-10-26T05:53:10.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec15cf10 con 0x7f75ec15bd60 2023-10-26T05:53:10.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0010cb0 con 0x7f75ec15bd60 2023-10-26T05:53:10.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec15c5e0 con 0x7f75ec15bd60 2023-10-26T05:53:10.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec066b10 con 0x7f75ec15bd60 2023-10-26T05:53:10.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f75e0019070 con 0x7f75ec15bd60 2023-10-26T05:53:10.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75d3fff640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75d8066470 0x7f75d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75e0017070 con 0x7f75ec15bd60 2023-10-26T05:53:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75eaffd640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75d8066470 0x7f75d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.592+0000 7f75eaffd640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75d8066470 0x7f75d8068930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009df0 tx=0x7f75e4009790 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:10.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.596+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f75e0051890 con 0x7f75ec15bd60 2023-10-26T05:53:10.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.744+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75ec071e70 con 0x7f75d8066470 2023-10-26T05:53:10.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75d3fff640 1 -- 172.21.15.71:0/1109087944 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284120 (secure 0 0 0) 0x7f75ec071e70 con 0x7f75d8066470 2023-10-26T05:53:10.775 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:10.777 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:10.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75d8066470 msgr2=0x7f75d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75d8066470 0x7f75d8068930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009df0 tx=0x7f75e4009790 comp rx=0 tx=0).stop 2023-10-26T05:53:10.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 msgr2=0x7f75ec157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 0x7f75ec157de0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f75e0004d20 tx=0x7f75e0001640 comp rx=0 tx=0).stop 2023-10-26T05:53:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 shutdown_connections 2023-10-26T05:53:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f75ec15d1c0 0x7f75ec15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f75d8066470 0x7f75d8068930 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75ec15bd60 0x7f75ec157de0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 --2- 172.21.15.71:0/1109087944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75d40052b0 0x7f75ec154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 >> 172.21.15.71:0/1109087944 conn(0x7f75ec0cdd80 msgr2=0x7f75ec15e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:10.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 shutdown_connections 2023-10-26T05:53:10.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.768+0000 7f75f19a5640 1 -- 172.21.15.71:0/1109087944 wait complete. 2023-10-26T05:53:10.792 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2023-10-26T05:53:09.527451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6496,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556003,"statfs":{"total":1600000000,"available":1593347832,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.147833"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.159973+0000","last_change":"2023-10-26T05:53:07.173723+0000","last_active":"2023-10-26T05:53:08.159973+0000","last_peered":"2023-10-26T05:53:08.159973+0000","last_clean":"2023-10-26T05:53:08.159973+0000","last_became_active":"2023-10-26T05:53:07.173085+0000","last_became_peered":"2023-10-26T05:53:07.173085+0000","last_unstale":"2023-10-26T05:53:08.159973+0000","last_undegraded":"2023-10-26T05:53:08.159973+0000","last_fullsized":"2023-10-26T05:53:08.159973+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:47:06.158808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837592+0000","last_change":"2023-10-26T05:53:07.173998+0000","last_active":"2023-10-26T05:53:08.837592+0000","last_peered":"2023-10-26T05:53:08.837592+0000","last_clean":"2023-10-26T05:53:08.837592+0000","last_became_active":"2023-10-26T05:53:07.172957+0000","last_became_peered":"2023-10-26T05:53:07.172957+0000","last_unstale":"2023-10-26T05:53:08.837592+0000","last_undegraded":"2023-10-26T05:53:08.837592+0000","last_fullsized":"2023-10-26T05:53:08.837592+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:29.535115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160119+0000","last_change":"2023-10-26T05:53:07.173609+0000","last_active":"2023-10-26T05:53:08.160119+0000","last_peered":"2023-10-26T05:53:08.160119+0000","last_clean":"2023-10-26T05:53:08.160119+0000","last_became_active":"2023-10-26T05:53:07.172968+0000","last_became_peered":"2023-10-26T05:53:07.172968+0000","last_unstale":"2023-10-26T05:53:08.160119+0000","last_undegraded":"2023-10-26T05:53:08.160119+0000","last_fullsized":"2023-10-26T05:53:08.160119+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:11:51.614835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836827+0000","last_change":"2023-10-26T05:53:07.205986+0000","last_active":"2023-10-26T05:53:08.836827+0000","last_peered":"2023-10-26T05:53:08.836827+0000","last_clean":"2023-10-26T05:53:08.836827+0000","last_became_active":"2023-10-26T05:53:07.204960+0000","last_became_peered":"2023-10-26T05:53:07.204960+0000","last_unstale":"2023-10-26T05:53:08.836827+0000","last_undegraded":"2023-10-26T05:53:08.836827+0000","last_fullsized":"2023-10-26T05:53:08.836827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:30.390769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839474+0000","last_change":"2023-10-26T05:53:07.201863+0000","last_active":"2023-10-26T05:53:08.839474+0000","last_peered":"2023-10-26T05:53:08.839474+0000","last_clean":"2023-10-26T05:53:08.839474+0000","last_became_active":"2023-10-26T05:53:07.201339+0000","last_became_peered":"2023-10-26T05:53:07.201339+0000","last_unstale":"2023-10-26T05:53:08.839474+0000","last_undegraded":"2023-10-26T05:53:08.839474+0000","last_fullsized":"2023-10-26T05:53:08.839474+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:45:29.535115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558331+0000","last_change":"2023-10-26T05:53:07.195436+0000","last_active":"2023-10-26T05:53:08.558331+0000","last_peered":"2023-10-26T05:53:08.558331+0000","last_clean":"2023-10-26T05:53:08.558331+0000","last_became_active":"2023-10-26T05:53:07.192362+0000","last_became_peered":"2023-10-26T05:53:07.192362+0000","last_unstale":"2023-10-26T05:53:08.558331+0000","last_undegraded":"2023-10-26T05:53:08.558331+0000","last_fullsized":"2023-10-26T05:53:08.558331+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:23.170274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558078+0000","last_change":"2023-10-26T05:53:07.191937+0000","last_active":"2023-10-26T05:53:08.558078+0000","last_peered":"2023-10-26T05:53:08.558078+0000","last_clean":"2023-10-26T05:53:08.558078+0000","last_became_active":"2023-10-26T05:53:07.191239+0000","last_became_peered":"2023-10-26T05:53:07.191239+0000","last_unstale":"2023-10-26T05:53:08.558078+0000","last_undegraded":"2023-10-26T05:53:08.558078+0000","last_fullsized":"2023-10-26T05:53:08.558078+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:59.146622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838567+0000","last_change":"2023-10-26T05:53:07.203045+0000","last_active":"2023-10-26T05:53:08.838567+0000","last_peered":"2023-10-26T05:53:08.838567+0000","last_clean":"2023-10-26T05:53:08.838567+0000","last_became_active":"2023-10-26T05:53:07.199707+0000","last_became_peered":"2023-10-26T05:53:07.199707+0000","last_unstale":"2023-10-26T05:53:08.838567+0000","last_undegraded":"2023-10-26T05:53:08.838567+0000","last_fullsized":"2023-10-26T05:53:08.838567+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:40.082223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167675+0000","last_change":"2023-10-26T05:53:07.203835+0000","last_active":"2023-10-26T05:53:08.167675+0000","last_peered":"2023-10-26T05:53:08.167675+0000","last_clean":"2023-10-26T05:53:08.167675+0000","last_became_active":"2023-10-26T05:53:07.203155+0000","last_became_peered":"2023-10-26T05:53:07.203155+0000","last_unstale":"2023-10-26T05:53:08.167675+0000","last_undegraded":"2023-10-26T05:53:08.167675+0000","last_fullsized":"2023-10-26T05:53:08.167675+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:58.341073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839663+0000","last_change":"2023-10-26T05:53:07.185763+0000","last_active":"2023-10-26T05:53:08.839663+0000","last_peered":"2023-10-26T05:53:08.839663+0000","last_clean":"2023-10-26T05:53:08.839663+0000","last_became_active":"2023-10-26T05:53:07.184195+0000","last_became_peered":"2023-10-26T05:53:07.184195+0000","last_unstale":"2023-10-26T05:53:08.839663+0000","last_undegraded":"2023-10-26T05:53:08.839663+0000","last_fullsized":"2023-10-26T05:53:08.839663+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:12.818270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558654+0000","last_change":"2023-10-26T05:53:07.196068+0000","last_active":"2023-10-26T05:53:08.558654+0000","last_peered":"2023-10-26T05:53:08.558654+0000","last_clean":"2023-10-26T05:53:08.558654+0000","last_became_active":"2023-10-26T05:53:07.193268+0000","last_became_peered":"2023-10-26T05:53:07.193268+0000","last_unstale":"2023-10-26T05:53:08.558654+0000","last_undegraded":"2023-10-26T05:53:08.558654+0000","last_fullsized":"2023-10-26T05:53:08.558654+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:43:11.218597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168102+0000","last_change":"2023-10-26T05:53:07.179314+0000","last_active":"2023-10-26T05:53:08.168102+0000","last_peered":"2023-10-26T05:53:08.168102+0000","last_clean":"2023-10-26T05:53:08.168102+0000","last_became_active":"2023-10-26T05:53:07.175279+0000","last_became_peered":"2023-10-26T05:53:07.175279+0000","last_unstale":"2023-10-26T05:53:08.168102+0000","last_undegraded":"2023-10-26T05:53:08.168102+0000","last_fullsized":"2023-10-26T05:53:08.168102+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:23.912541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836644+0000","last_change":"2023-10-26T05:53:07.198975+0000","last_active":"2023-10-26T05:53:08.836644+0000","last_peered":"2023-10-26T05:53:08.836644+0000","last_clean":"2023-10-26T05:53:08.836644+0000","last_became_active":"2023-10-26T05:53:07.197531+0000","last_became_peered":"2023-10-26T05:53:07.197531+0000","last_unstale":"2023-10-26T05:53:08.836644+0000","last_undegraded":"2023-10-26T05:53:08.836644+0000","last_fullsized":"2023-10-26T05:53:08.836644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:11.011808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836245+0000","last_change":"2023-10-26T05:53:07.196413+0000","last_active":"2023-10-26T05:53:08.836245+0000","last_peered":"2023-10-26T05:53:08.836245+0000","last_clean":"2023-10-26T05:53:08.836245+0000","last_became_active":"2023-10-26T05:53:07.194842+0000","last_became_peered":"2023-10-26T05:53:07.194842+0000","last_unstale":"2023-10-26T05:53:08.836245+0000","last_undegraded":"2023-10-26T05:53:08.836245+0000","last_fullsized":"2023-10-26T05:53:08.836245+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:46:00.117863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836827+0000","last_change":"2023-10-26T05:53:07.207843+0000","last_active":"2023-10-26T05:53:08.836827+0000","last_peered":"2023-10-26T05:53:08.836827+0000","last_clean":"2023-10-26T05:53:08.836827+0000","last_became_active":"2023-10-26T05:53:07.206860+0000","last_became_peered":"2023-10-26T05:53:07.206860+0000","last_unstale":"2023-10-26T05:53:08.836827+0000","last_undegraded":"2023-10-26T05:53:08.836827+0000","last_fullsized":"2023-10-26T05:53:08.836827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:45:57.157667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558990+0000","last_change":"2023-10-26T05:53:07.201129+0000","last_active":"2023-10-26T05:53:08.558990+0000","last_peered":"2023-10-26T05:53:08.558990+0000","last_clean":"2023-10-26T05:53:08.558990+0000","last_became_active":"2023-10-26T05:53:07.200444+0000","last_became_peered":"2023-10-26T05:53:07.200444+0000","last_unstale":"2023-10-26T05:53:08.558990+0000","last_undegraded":"2023-10-26T05:53:08.558990+0000","last_fullsized":"2023-10-26T05:53:08.558990+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:10.556781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837713+0000","last_change":"2023-10-26T05:53:07.198950+0000","last_active":"2023-10-26T05:53:08.837713+0000","last_peered":"2023-10-26T05:53:08.837713+0000","last_clean":"2023-10-26T05:53:08.837713+0000","last_became_active":"2023-10-26T05:53:07.198294+0000","last_became_peered":"2023-10-26T05:53:07.198294+0000","last_unstale":"2023-10-26T05:53:08.837713+0000","last_undegraded":"2023-10-26T05:53:08.837713+0000","last_fullsized":"2023-10-26T05:53:08.837713+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:34:28.391351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836969+0000","last_change":"2023-10-26T05:53:07.205623+0000","last_active":"2023-10-26T05:53:08.836969+0000","last_peered":"2023-10-26T05:53:08.836969+0000","last_clean":"2023-10-26T05:53:08.836969+0000","last_became_active":"2023-10-26T05:53:07.204904+0000","last_became_peered":"2023-10-26T05:53:07.204904+0000","last_unstale":"2023-10-26T05:53:08.836969+0000","last_undegraded":"2023-10-26T05:53:08.836969+0000","last_fullsized":"2023-10-26T05:53:08.836969+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:18:22.118581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836461+0000","last_change":"2023-10-26T05:53:07.194083+0000","last_active":"2023-10-26T05:53:08.836461+0000","last_peered":"2023-10-26T05:53:08.836461+0000","last_clean":"2023-10-26T05:53:08.836461+0000","last_became_active":"2023-10-26T05:53:07.193066+0000","last_became_peered":"2023-10-26T05:53:07.193066+0000","last_unstale":"2023-10-26T05:53:08.836461+0000","last_undegraded":"2023-10-26T05:53:08.836461+0000","last_fullsized":"2023-10-26T05:53:08.836461+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:31:17.575104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166847+0000","last_change":"2023-10-26T05:53:07.201831+0000","last_active":"2023-10-26T05:53:08.166847+0000","last_peered":"2023-10-26T05:53:08.166847+0000","last_clean":"2023-10-26T05:53:08.166847+0000","last_became_active":"2023-10-26T05:53:07.201162+0000","last_became_peered":"2023-10-26T05:53:07.201162+0000","last_unstale":"2023-10-26T05:53:08.166847+0000","last_undegraded":"2023-10-26T05:53:08.166847+0000","last_fullsized":"2023-10-26T05:53:08.166847+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:24:38.356926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838098+0000","last_change":"2023-10-26T05:53:07.191881+0000","last_active":"2023-10-26T05:53:08.838098+0000","last_peered":"2023-10-26T05:53:08.838098+0000","last_clean":"2023-10-26T05:53:08.838098+0000","last_became_active":"2023-10-26T05:53:07.191258+0000","last_became_peered":"2023-10-26T05:53:07.191258+0000","last_unstale":"2023-10-26T05:53:08.838098+0000","last_undegraded":"2023-10-26T05:53:08.838098+0000","last_fullsized":"2023-10-26T05:53:08.838098+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:22:33.018513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838065+0000","last_change":"2023-10-26T05:53:07.173555+0000","last_active":"2023-10-26T05:53:08.838065+0000","last_peered":"2023-10-26T05:53:08.838065+0000","last_clean":"2023-10-26T05:53:08.838065+0000","last_became_active":"2023-10-26T05:53:07.172123+0000","last_became_peered":"2023-10-26T05:53:07.172123+0000","last_unstale":"2023-10-26T05:53:08.838065+0000","last_undegraded":"2023-10-26T05:53:08.838065+0000","last_fullsized":"2023-10-26T05:53:08.838065+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:48.778915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838690+0000","last_change":"2023-10-26T05:53:07.173759+0000","last_active":"2023-10-26T05:53:08.838690+0000","last_peered":"2023-10-26T05:53:08.838690+0000","last_clean":"2023-10-26T05:53:08.838690+0000","last_became_active":"2023-10-26T05:53:07.172986+0000","last_became_peered":"2023-10-26T05:53:07.172986+0000","last_unstale":"2023-10-26T05:53:08.838690+0000","last_undegraded":"2023-10-26T05:53:08.838690+0000","last_fullsized":"2023-10-26T05:53:08.838690+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:03.987842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558244+0000","last_change":"2023-10-26T05:53:07.197307+0000","last_active":"2023-10-26T05:53:08.558244+0000","last_peered":"2023-10-26T05:53:08.558244+0000","last_clean":"2023-10-26T05:53:08.558244+0000","last_became_active":"2023-10-26T05:53:07.196757+0000","last_became_peered":"2023-10-26T05:53:07.196757+0000","last_unstale":"2023-10-26T05:53:08.558244+0000","last_undegraded":"2023-10-26T05:53:08.558244+0000","last_fullsized":"2023-10-26T05:53:08.558244+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:30.904756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839831+0000","last_change":"2023-10-26T05:53:07.185298+0000","last_active":"2023-10-26T05:53:08.839831+0000","last_peered":"2023-10-26T05:53:08.839831+0000","last_clean":"2023-10-26T05:53:08.839831+0000","last_became_active":"2023-10-26T05:53:07.183264+0000","last_became_peered":"2023-10-26T05:53:07.183264+0000","last_unstale":"2023-10-26T05:53:08.839831+0000","last_undegraded":"2023-10-26T05:53:08.839831+0000","last_fullsized":"2023-10-26T05:53:08.839831+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:19:48.778915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838015+0000","last_change":"2023-10-26T05:53:07.169715+0000","last_active":"2023-10-26T05:53:08.838015+0000","last_peered":"2023-10-26T05:53:08.838015+0000","last_clean":"2023-10-26T05:53:08.838015+0000","last_became_active":"2023-10-26T05:53:07.169040+0000","last_became_peered":"2023-10-26T05:53:07.169040+0000","last_unstale":"2023-10-26T05:53:08.838015+0000","last_undegraded":"2023-10-26T05:53:08.838015+0000","last_fullsized":"2023-10-26T05:53:08.838015+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:40.617492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558427+0000","last_change":"2023-10-26T05:53:07.196187+0000","last_active":"2023-10-26T05:53:08.558427+0000","last_peered":"2023-10-26T05:53:08.558427+0000","last_clean":"2023-10-26T05:53:08.558427+0000","last_became_active":"2023-10-26T05:53:07.195014+0000","last_became_peered":"2023-10-26T05:53:07.195014+0000","last_unstale":"2023-10-26T05:53:08.558427+0000","last_undegraded":"2023-10-26T05:53:08.558427+0000","last_fullsized":"2023-10-26T05:53:08.558427+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:23.912541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839600+0000","last_change":"2023-10-26T05:53:07.204644+0000","last_active":"2023-10-26T05:53:08.839600+0000","last_peered":"2023-10-26T05:53:08.839600+0000","last_clean":"2023-10-26T05:53:08.839600+0000","last_became_active":"2023-10-26T05:53:07.202570+0000","last_became_peered":"2023-10-26T05:53:07.202570+0000","last_unstale":"2023-10-26T05:53:08.839600+0000","last_undegraded":"2023-10-26T05:53:08.839600+0000","last_fullsized":"2023-10-26T05:53:08.839600+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:34:28.391351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836878+0000","last_change":"2023-10-26T05:53:07.167725+0000","last_active":"2023-10-26T05:53:08.836878+0000","last_peered":"2023-10-26T05:53:08.836878+0000","last_clean":"2023-10-26T05:53:08.836878+0000","last_became_active":"2023-10-26T05:53:07.167139+0000","last_became_peered":"2023-10-26T05:53:07.167139+0000","last_unstale":"2023-10-26T05:53:08.836878+0000","last_undegraded":"2023-10-26T05:53:08.836878+0000","last_fullsized":"2023-10-26T05:53:08.836878+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:50:13.925866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838857+0000","last_change":"2023-10-26T05:53:07.193828+0000","last_active":"2023-10-26T05:53:08.838857+0000","last_peered":"2023-10-26T05:53:08.838857+0000","last_clean":"2023-10-26T05:53:08.838857+0000","last_became_active":"2023-10-26T05:53:07.193108+0000","last_became_peered":"2023-10-26T05:53:07.193108+0000","last_unstale":"2023-10-26T05:53:08.838857+0000","last_undegraded":"2023-10-26T05:53:08.838857+0000","last_fullsized":"2023-10-26T05:53:08.838857+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:53:39.573829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838920+0000","last_change":"2023-10-26T05:53:07.198952+0000","last_active":"2023-10-26T05:53:08.838920+0000","last_peered":"2023-10-26T05:53:08.838920+0000","last_clean":"2023-10-26T05:53:08.838920+0000","last_became_active":"2023-10-26T05:53:07.198268+0000","last_became_peered":"2023-10-26T05:53:07.198268+0000","last_unstale":"2023-10-26T05:53:08.838920+0000","last_undegraded":"2023-10-26T05:53:08.838920+0000","last_fullsized":"2023-10-26T05:53:08.838920+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:26.786989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160060+0000","last_change":"2023-10-26T05:53:07.172143+0000","last_active":"2023-10-26T05:53:08.160060+0000","last_peered":"2023-10-26T05:53:08.160060+0000","last_clean":"2023-10-26T05:53:08.160060+0000","last_became_active":"2023-10-26T05:53:07.171308+0000","last_became_peered":"2023-10-26T05:53:07.171308+0000","last_unstale":"2023-10-26T05:53:08.160060+0000","last_undegraded":"2023-10-26T05:53:08.160060+0000","last_fullsized":"2023-10-26T05:53:08.160060+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:35:48.527391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836665+0000","last_change":"2023-10-26T05:53:07.198186+0000","last_active":"2023-10-26T05:53:08.836665+0000","last_peered":"2023-10-26T05:53:08.836665+0000","last_clean":"2023-10-26T05:53:08.836665+0000","last_became_active":"2023-10-26T05:53:07.195793+0000","last_became_peered":"2023-10-26T05:53:07.195793+0000","last_unstale":"2023-10-26T05:53:08.836665+0000","last_undegraded":"2023-10-26T05:53:08.836665+0000","last_fullsized":"2023-10-26T05:53:08.836665+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:47:51.045817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"123'4","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838938+0000","last_change":"2023-10-26T05:53:06.395372+0000","last_active":"2023-10-26T05:53:08.838938+0000","last_peered":"2023-10-26T05:53:08.838938+0000","last_clean":"2023-10-26T05:53:08.838938+0000","last_became_active":"2023-10-26T05:53:06.395110+0000","last_became_peered":"2023-10-26T05:53:06.395110+0000","last_unstale":"2023-10-26T05:53:08.838938+0000","last_undegraded":"2023-10-26T05:53:08.838938+0000","last_fullsized":"2023-10-26T05:53:08.838938+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:01.916306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837295+0000","last_change":"2023-10-26T05:53:07.199346+0000","last_active":"2023-10-26T05:53:08.837295+0000","last_peered":"2023-10-26T05:53:08.837295+0000","last_clean":"2023-10-26T05:53:08.837295+0000","last_became_active":"2023-10-26T05:53:07.198341+0000","last_became_peered":"2023-10-26T05:53:07.198341+0000","last_unstale":"2023-10-26T05:53:08.837295+0000","last_undegraded":"2023-10-26T05:53:08.837295+0000","last_fullsized":"2023-10-26T05:53:08.837295+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:34:42.563897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558476+0000","last_change":"2023-10-26T05:53:06.405867+0000","last_active":"2023-10-26T05:53:08.558476+0000","last_peered":"2023-10-26T05:53:08.558476+0000","last_clean":"2023-10-26T05:53:08.558476+0000","last_became_active":"2023-10-26T05:53:06.405143+0000","last_became_peered":"2023-10-26T05:53:06.405143+0000","last_unstale":"2023-10-26T05:53:08.558476+0000","last_undegraded":"2023-10-26T05:53:08.558476+0000","last_fullsized":"2023-10-26T05:53:08.558476+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:25.545917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160238+0000","last_change":"2023-10-26T05:53:07.169334+0000","last_active":"2023-10-26T05:53:08.160238+0000","last_peered":"2023-10-26T05:53:08.160238+0000","last_clean":"2023-10-26T05:53:08.160238+0000","last_became_active":"2023-10-26T05:53:07.168891+0000","last_became_peered":"2023-10-26T05:53:07.168891+0000","last_unstale":"2023-10-26T05:53:08.160238+0000","last_undegraded":"2023-10-26T05:53:08.160238+0000","last_fullsized":"2023-10-26T05:53:08.160238+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:08.686134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837390+0000","last_change":"2023-10-26T05:53:06.397662+0000","last_active":"2023-10-26T05:53:08.837390+0000","last_peered":"2023-10-26T05:53:08.837390+0000","last_clean":"2023-10-26T05:53:08.837390+0000","last_became_active":"2023-10-26T05:53:06.396440+0000","last_became_peered":"2023-10-26T05:53:06.396440+0000","last_unstale":"2023-10-26T05:53:08.837390+0000","last_undegraded":"2023-10-26T05:53:08.837390+0000","last_fullsized":"2023-10-26T05:53:08.837390+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:46.769372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167185+0000","last_change":"2023-10-26T05:53:07.180308+0000","last_active":"2023-10-26T05:53:08.167185+0000","last_peered":"2023-10-26T05:53:08.167185+0000","last_clean":"2023-10-26T05:53:08.167185+0000","last_became_active":"2023-10-26T05:53:07.179690+0000","last_became_peered":"2023-10-26T05:53:07.179690+0000","last_unstale":"2023-10-26T05:53:08.167185+0000","last_undegraded":"2023-10-26T05:53:08.167185+0000","last_fullsized":"2023-10-26T05:53:08.167185+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:59.146622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838247+0000","last_change":"2023-10-26T05:53:06.394372+0000","last_active":"2023-10-26T05:53:08.838247+0000","last_peered":"2023-10-26T05:53:08.838247+0000","last_clean":"2023-10-26T05:53:08.838247+0000","last_became_active":"2023-10-26T05:53:06.394131+0000","last_became_peered":"2023-10-26T05:53:06.394131+0000","last_unstale":"2023-10-26T05:53:08.838247+0000","last_undegraded":"2023-10-26T05:53:08.838247+0000","last_fullsized":"2023-10-26T05:53:08.838247+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:29.900525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167237+0000","last_change":"2023-10-26T05:53:07.179744+0000","last_active":"2023-10-26T05:53:08.167237+0000","last_peered":"2023-10-26T05:53:08.167237+0000","last_clean":"2023-10-26T05:53:08.167237+0000","last_became_active":"2023-10-26T05:53:07.178781+0000","last_became_peered":"2023-10-26T05:53:07.178781+0000","last_unstale":"2023-10-26T05:53:08.167237+0000","last_undegraded":"2023-10-26T05:53:08.167237+0000","last_fullsized":"2023-10-26T05:53:08.167237+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:22.956125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160333+0000","last_change":"2023-10-26T05:53:06.393786+0000","last_active":"2023-10-26T05:53:08.160333+0000","last_peered":"2023-10-26T05:53:08.160333+0000","last_clean":"2023-10-26T05:53:08.160333+0000","last_became_active":"2023-10-26T05:53:06.393408+0000","last_became_peered":"2023-10-26T05:53:06.393408+0000","last_unstale":"2023-10-26T05:53:08.160333+0000","last_undegraded":"2023-10-26T05:53:08.160333+0000","last_fullsized":"2023-10-26T05:53:08.160333+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:09:54.013866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558734+0000","last_change":"2023-10-26T05:53:07.195719+0000","last_active":"2023-10-26T05:53:08.558734+0000","last_peered":"2023-10-26T05:53:08.558734+0000","last_clean":"2023-10-26T05:53:08.558734+0000","last_became_active":"2023-10-26T05:53:07.194224+0000","last_became_peered":"2023-10-26T05:53:07.194224+0000","last_unstale":"2023-10-26T05:53:08.558734+0000","last_undegraded":"2023-10-26T05:53:08.558734+0000","last_fullsized":"2023-10-26T05:53:08.558734+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:02.920537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839032+0000","last_change":"2023-10-26T05:53:06.397518+0000","last_active":"2023-10-26T05:53:08.839032+0000","last_peered":"2023-10-26T05:53:08.839032+0000","last_clean":"2023-10-26T05:53:08.839032+0000","last_became_active":"2023-10-26T05:53:06.396476+0000","last_became_peered":"2023-10-26T05:53:06.396476+0000","last_unstale":"2023-10-26T05:53:08.839032+0000","last_undegraded":"2023-10-26T05:53:08.839032+0000","last_fullsized":"2023-10-26T05:53:08.839032+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:09.552549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837048+0000","last_change":"2023-10-26T05:53:07.207464+0000","last_active":"2023-10-26T05:53:08.837048+0000","last_peered":"2023-10-26T05:53:08.837048+0000","last_clean":"2023-10-26T05:53:08.837048+0000","last_became_active":"2023-10-26T05:53:07.206031+0000","last_became_peered":"2023-10-26T05:53:07.206031+0000","last_unstale":"2023-10-26T05:53:08.837048+0000","last_undegraded":"2023-10-26T05:53:08.837048+0000","last_fullsized":"2023-10-26T05:53:08.837048+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:47.773603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840267+0000","last_change":"2023-10-26T05:53:06.403340+0000","last_active":"2023-10-26T05:53:08.840267+0000","last_peered":"2023-10-26T05:53:08.840267+0000","last_clean":"2023-10-26T05:53:08.840267+0000","last_became_active":"2023-10-26T05:53:06.402322+0000","last_became_peered":"2023-10-26T05:53:06.402322+0000","last_unstale":"2023-10-26T05:53:08.840267+0000","last_undegraded":"2023-10-26T05:53:08.840267+0000","last_fullsized":"2023-10-26T05:53:08.840267+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:22.166043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167982+0000","last_change":"2023-10-26T05:53:07.182507+0000","last_active":"2023-10-26T05:53:08.167982+0000","last_peered":"2023-10-26T05:53:08.167982+0000","last_clean":"2023-10-26T05:53:08.167982+0000","last_became_active":"2023-10-26T05:53:07.181585+0000","last_became_peered":"2023-10-26T05:53:07.181585+0000","last_unstale":"2023-10-26T05:53:08.167982+0000","last_undegraded":"2023-10-26T05:53:08.167982+0000","last_fullsized":"2023-10-26T05:53:08.167982+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:30:23.170274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160639+0000","last_change":"2023-10-26T05:53:06.391373+0000","last_active":"2023-10-26T05:53:08.160639+0000","last_peered":"2023-10-26T05:53:08.160639+0000","last_clean":"2023-10-26T05:53:08.160639+0000","last_became_active":"2023-10-26T05:53:06.391035+0000","last_became_peered":"2023-10-26T05:53:06.391035+0000","last_unstale":"2023-10-26T05:53:08.160639+0000","last_undegraded":"2023-10-26T05:53:08.160639+0000","last_fullsized":"2023-10-26T05:53:08.160639+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:12.810056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837861+0000","last_change":"2023-10-26T05:53:07.171228+0000","last_active":"2023-10-26T05:53:08.837861+0000","last_peered":"2023-10-26T05:53:08.837861+0000","last_clean":"2023-10-26T05:53:08.837861+0000","last_became_active":"2023-10-26T05:53:07.170598+0000","last_became_peered":"2023-10-26T05:53:07.170598+0000","last_unstale":"2023-10-26T05:53:08.837861+0000","last_undegraded":"2023-10-26T05:53:08.837861+0000","last_fullsized":"2023-10-26T05:53:08.837861+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:39:12.818270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160374+0000","last_change":"2023-10-26T05:53:06.398884+0000","last_active":"2023-10-26T05:53:08.160374+0000","last_peered":"2023-10-26T05:53:08.160374+0000","last_clean":"2023-10-26T05:53:08.160374+0000","last_became_active":"2023-10-26T05:53:06.398615+0000","last_became_peered":"2023-10-26T05:53:06.398615+0000","last_unstale":"2023-10-26T05:53:08.160374+0000","last_undegraded":"2023-10-26T05:53:08.160374+0000","last_fullsized":"2023-10-26T05:53:08.160374+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:46:58.262527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168144+0000","last_change":"2023-10-26T05:53:07.177301+0000","last_active":"2023-10-26T05:53:08.168144+0000","last_peered":"2023-10-26T05:53:08.168144+0000","last_clean":"2023-10-26T05:53:08.168144+0000","last_became_active":"2023-10-26T05:53:07.176658+0000","last_became_peered":"2023-10-26T05:53:07.176658+0000","last_unstale":"2023-10-26T05:53:08.168144+0000","last_undegraded":"2023-10-26T05:53:08.168144+0000","last_fullsized":"2023-10-26T05:53:08.168144+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:43:11.218597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837412+0000","last_change":"2023-10-26T05:53:06.388081+0000","last_active":"2023-10-26T05:53:08.837412+0000","last_peered":"2023-10-26T05:53:08.837412+0000","last_clean":"2023-10-26T05:53:08.837412+0000","last_became_active":"2023-10-26T05:53:06.387677+0000","last_became_peered":"2023-10-26T05:53:06.387677+0000","last_unstale":"2023-10-26T05:53:08.837412+0000","last_undegraded":"2023-10-26T05:53:08.837412+0000","last_fullsized":"2023-10-26T05:53:08.837412+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:48:32.633122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839392+0000","last_change":"2023-10-26T05:53:07.182868+0000","last_active":"2023-10-26T05:53:08.839392+0000","last_peered":"2023-10-26T05:53:08.839392+0000","last_clean":"2023-10-26T05:53:08.839392+0000","last_became_active":"2023-10-26T05:53:07.182231+0000","last_became_peered":"2023-10-26T05:53:07.182231+0000","last_unstale":"2023-10-26T05:53:08.839392+0000","last_undegraded":"2023-10-26T05:53:08.839392+0000","last_fullsized":"2023-10-26T05:53:08.839392+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:50:13.925866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"123'3","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558396+0000","last_change":"2023-10-26T05:53:06.411876+0000","last_active":"2023-10-26T05:53:08.558396+0000","last_peered":"2023-10-26T05:53:08.558396+0000","last_clean":"2023-10-26T05:53:08.558396+0000","last_became_active":"2023-10-26T05:53:06.411270+0000","last_became_peered":"2023-10-26T05:53:06.411270+0000","last_unstale":"2023-10-26T05:53:08.558396+0000","last_undegraded":"2023-10-26T05:53:08.558396+0000","last_fullsized":"2023-10-26T05:53:08.558396+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:10.007577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558263+0000","last_change":"2023-10-26T05:53:07.207930+0000","last_active":"2023-10-26T05:53:08.558263+0000","last_peered":"2023-10-26T05:53:08.558263+0000","last_clean":"2023-10-26T05:53:08.558263+0000","last_became_active":"2023-10-26T05:53:07.207183+0000","last_became_peered":"2023-10-26T05:53:07.207183+0000","last_unstale":"2023-10-26T05:53:08.558263+0000","last_undegraded":"2023-10-26T05:53:08.558263+0000","last_fullsized":"2023-10-26T05:53:08.558263+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:58.341073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"123'6","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168431+0000","last_change":"2023-10-26T05:53:06.398397+0000","last_active":"2023-10-26T05:53:08.168431+0000","last_peered":"2023-10-26T05:53:08.168431+0000","last_clean":"2023-10-26T05:53:08.168431+0000","last_became_active":"2023-10-26T05:53:06.397755+0000","last_became_peered":"2023-10-26T05:53:06.397755+0000","last_unstale":"2023-10-26T05:53:08.168431+0000","last_undegraded":"2023-10-26T05:53:08.168431+0000","last_fullsized":"2023-10-26T05:53:08.168431+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:49:29.386538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839235+0000","last_change":"2023-10-26T05:53:07.197465+0000","last_active":"2023-10-26T05:53:08.839235+0000","last_peered":"2023-10-26T05:53:08.839235+0000","last_clean":"2023-10-26T05:53:08.839235+0000","last_became_active":"2023-10-26T05:53:07.196658+0000","last_became_peered":"2023-10-26T05:53:07.196658+0000","last_unstale":"2023-10-26T05:53:08.839235+0000","last_undegraded":"2023-10-26T05:53:08.839235+0000","last_fullsized":"2023-10-26T05:53:08.839235+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:24:38.356926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839394+0000","last_change":"2023-10-26T05:53:06.408807+0000","last_active":"2023-10-26T05:53:08.839394+0000","last_peered":"2023-10-26T05:53:08.839394+0000","last_clean":"2023-10-26T05:53:08.839394+0000","last_became_active":"2023-10-26T05:53:06.408574+0000","last_became_peered":"2023-10-26T05:53:06.408574+0000","last_unstale":"2023-10-26T05:53:08.839394+0000","last_undegraded":"2023-10-26T05:53:08.839394+0000","last_fullsized":"2023-10-26T05:53:08.839394+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:32:58.378694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160497+0000","last_change":"2023-10-26T05:53:07.192552+0000","last_active":"2023-10-26T05:53:08.160497+0000","last_peered":"2023-10-26T05:53:08.160497+0000","last_clean":"2023-10-26T05:53:08.160497+0000","last_became_active":"2023-10-26T05:53:07.191785+0000","last_became_peered":"2023-10-26T05:53:07.191785+0000","last_unstale":"2023-10-26T05:53:08.160497+0000","last_undegraded":"2023-10-26T05:53:08.160497+0000","last_fullsized":"2023-10-26T05:53:08.160497+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:41:15.345863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838445+0000","last_change":"2023-10-26T05:53:06.395064+0000","last_active":"2023-10-26T05:53:08.838445+0000","last_peered":"2023-10-26T05:53:08.838445+0000","last_clean":"2023-10-26T05:53:08.838445+0000","last_became_active":"2023-10-26T05:53:06.394740+0000","last_became_peered":"2023-10-26T05:53:06.394740+0000","last_unstale":"2023-10-26T05:53:08.838445+0000","last_undegraded":"2023-10-26T05:53:08.838445+0000","last_fullsized":"2023-10-26T05:53:08.838445+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:13:25.527872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167218+0000","last_change":"2023-10-26T05:53:07.176936+0000","last_active":"2023-10-26T05:53:08.167218+0000","last_peered":"2023-10-26T05:53:08.167218+0000","last_clean":"2023-10-26T05:53:08.167218+0000","last_became_active":"2023-10-26T05:53:07.176302+0000","last_became_peered":"2023-10-26T05:53:07.176302+0000","last_unstale":"2023-10-26T05:53:08.167218+0000","last_undegraded":"2023-10-26T05:53:08.167218+0000","last_fullsized":"2023-10-26T05:53:08.167218+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:30.904756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168477+0000","last_change":"2023-10-26T05:53:06.411521+0000","last_active":"2023-10-26T05:53:08.168477+0000","last_peered":"2023-10-26T05:53:08.168477+0000","last_clean":"2023-10-26T05:53:08.168477+0000","last_became_active":"2023-10-26T05:53:06.410661+0000","last_became_peered":"2023-10-26T05:53:06.410661+0000","last_unstale":"2023-10-26T05:53:08.168477+0000","last_undegraded":"2023-10-26T05:53:08.168477+0000","last_fullsized":"2023-10-26T05:53:08.168477+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:34:41.559666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838559+0000","last_change":"2023-10-26T05:53:07.201568+0000","last_active":"2023-10-26T05:53:08.838559+0000","last_peered":"2023-10-26T05:53:08.838559+0000","last_clean":"2023-10-26T05:53:08.838559+0000","last_became_active":"2023-10-26T05:53:07.198500+0000","last_became_peered":"2023-10-26T05:53:07.198500+0000","last_unstale":"2023-10-26T05:53:08.838559+0000","last_undegraded":"2023-10-26T05:53:08.838559+0000","last_fullsized":"2023-10-26T05:53:08.838559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:27:59.935036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"123'8","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160537+0000","last_change":"2023-10-26T05:53:06.402525+0000","last_active":"2023-10-26T05:53:08.160537+0000","last_peered":"2023-10-26T05:53:08.160537+0000","last_clean":"2023-10-26T05:53:08.160537+0000","last_became_active":"2023-10-26T05:53:06.402284+0000","last_became_peered":"2023-10-26T05:53:06.402284+0000","last_unstale":"2023-10-26T05:53:08.160537+0000","last_undegraded":"2023-10-26T05:53:08.160537+0000","last_fullsized":"2023-10-26T05:53:08.160537+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:25:40.044565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839724+0000","last_change":"2023-10-26T05:53:07.185247+0000","last_active":"2023-10-26T05:53:08.839724+0000","last_peered":"2023-10-26T05:53:08.839724+0000","last_clean":"2023-10-26T05:53:08.839724+0000","last_became_active":"2023-10-26T05:53:07.184670+0000","last_became_peered":"2023-10-26T05:53:07.184670+0000","last_unstale":"2023-10-26T05:53:08.839724+0000","last_undegraded":"2023-10-26T05:53:08.839724+0000","last_fullsized":"2023-10-26T05:53:08.839724+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:40.617492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.167426+0000","last_change":"2023-10-26T05:53:06.412164+0000","last_active":"2023-10-26T05:53:08.167426+0000","last_peered":"2023-10-26T05:53:08.167426+0000","last_clean":"2023-10-26T05:53:08.167426+0000","last_became_active":"2023-10-26T05:53:06.411078+0000","last_became_peered":"2023-10-26T05:53:06.411078+0000","last_unstale":"2023-10-26T05:53:08.167426+0000","last_undegraded":"2023-10-26T05:53:08.167426+0000","last_fullsized":"2023-10-26T05:53:08.167426+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:36:46.769372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166509+0000","last_change":"2023-10-26T05:53:07.184658+0000","last_active":"2023-10-26T05:53:08.166509+0000","last_peered":"2023-10-26T05:53:08.166509+0000","last_clean":"2023-10-26T05:53:08.166509+0000","last_became_active":"2023-10-26T05:53:07.183963+0000","last_became_peered":"2023-10-26T05:53:07.183963+0000","last_unstale":"2023-10-26T05:53:08.166509+0000","last_undegraded":"2023-10-26T05:53:08.166509+0000","last_fullsized":"2023-10-26T05:53:08.166509+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:40:56.956703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839442+0000","last_change":"2023-10-26T05:53:06.401661+0000","last_active":"2023-10-26T05:53:08.839442+0000","last_peered":"2023-10-26T05:53:08.839442+0000","last_clean":"2023-10-26T05:53:08.839442+0000","last_became_active":"2023-10-26T05:53:06.400939+0000","last_became_peered":"2023-10-26T05:53:06.400939+0000","last_unstale":"2023-10-26T05:53:08.839442+0000","last_undegraded":"2023-10-26T05:53:08.839442+0000","last_fullsized":"2023-10-26T05:53:08.839442+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:11.736607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.559300+0000","last_change":"2023-10-26T05:53:07.198191+0000","last_active":"2023-10-26T05:53:08.559300+0000","last_peered":"2023-10-26T05:53:08.559300+0000","last_clean":"2023-10-26T05:53:08.559300+0000","last_became_active":"2023-10-26T05:53:07.197501+0000","last_became_peered":"2023-10-26T05:53:07.197501+0000","last_unstale":"2023-10-26T05:53:08.559300+0000","last_undegraded":"2023-10-26T05:53:08.559300+0000","last_fullsized":"2023-10-26T05:53:08.559300+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:12.740838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838290+0000","last_change":"2023-10-26T05:53:06.397106+0000","last_active":"2023-10-26T05:53:08.838290+0000","last_peered":"2023-10-26T05:53:08.838290+0000","last_clean":"2023-10-26T05:53:08.838290+0000","last_became_active":"2023-10-26T05:53:06.396761+0000","last_became_peered":"2023-10-26T05:53:06.396761+0000","last_unstale":"2023-10-26T05:53:08.838290+0000","last_undegraded":"2023-10-26T05:53:08.838290+0000","last_fullsized":"2023-10-26T05:53:08.838290+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:13.044276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836876+0000","last_change":"2023-10-26T05:53:07.207074+0000","last_active":"2023-10-26T05:53:08.836876+0000","last_peered":"2023-10-26T05:53:08.836876+0000","last_clean":"2023-10-26T05:53:08.836876+0000","last_became_active":"2023-10-26T05:53:07.203547+0000","last_became_peered":"2023-10-26T05:53:07.203547+0000","last_unstale":"2023-10-26T05:53:08.836876+0000","last_undegraded":"2023-10-26T05:53:08.836876+0000","last_fullsized":"2023-10-26T05:53:08.836876+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:49.370880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839102+0000","last_change":"2023-10-26T05:53:06.397826+0000","last_active":"2023-10-26T05:53:08.839102+0000","last_peered":"2023-10-26T05:53:08.839102+0000","last_clean":"2023-10-26T05:53:08.839102+0000","last_became_active":"2023-10-26T05:53:06.397212+0000","last_became_peered":"2023-10-26T05:53:06.397212+0000","last_unstale":"2023-10-26T05:53:08.839102+0000","last_undegraded":"2023-10-26T05:53:08.839102+0000","last_fullsized":"2023-10-26T05:53:08.839102+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:06:11.736607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.836372+0000","last_change":"2023-10-26T05:53:07.204887+0000","last_active":"2023-10-26T05:53:08.836372+0000","last_peered":"2023-10-26T05:53:08.836372+0000","last_clean":"2023-10-26T05:53:08.836372+0000","last_became_active":"2023-10-26T05:53:07.204398+0000","last_became_peered":"2023-10-26T05:53:07.204398+0000","last_unstale":"2023-10-26T05:53:08.836372+0000","last_undegraded":"2023-10-26T05:53:08.836372+0000","last_fullsized":"2023-10-26T05:53:08.836372+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:22:44.879819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.165912+0000","last_change":"2023-10-26T05:53:06.408819+0000","last_active":"2023-10-26T05:53:08.165912+0000","last_peered":"2023-10-26T05:53:08.165912+0000","last_clean":"2023-10-26T05:53:08.165912+0000","last_became_active":"2023-10-26T05:53:06.408262+0000","last_became_peered":"2023-10-26T05:53:06.408262+0000","last_unstale":"2023-10-26T05:53:08.165912+0000","last_undegraded":"2023-10-26T05:53:08.165912+0000","last_fullsized":"2023-10-26T05:53:08.165912+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:17:55.244617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838307+0000","last_change":"2023-10-26T05:53:07.202024+0000","last_active":"2023-10-26T05:53:08.838307+0000","last_peered":"2023-10-26T05:53:08.838307+0000","last_clean":"2023-10-26T05:53:08.838307+0000","last_became_active":"2023-10-26T05:53:07.201337+0000","last_became_peered":"2023-10-26T05:53:07.201337+0000","last_unstale":"2023-10-26T05:53:08.838307+0000","last_undegraded":"2023-10-26T05:53:08.838307+0000","last_fullsized":"2023-10-26T05:53:08.838307+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:15:33.880787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166346+0000","last_change":"2023-10-26T05:53:06.399738+0000","last_active":"2023-10-26T05:53:08.166346+0000","last_peered":"2023-10-26T05:53:08.166346+0000","last_clean":"2023-10-26T05:53:08.166346+0000","last_became_active":"2023-10-26T05:53:06.398936+0000","last_became_peered":"2023-10-26T05:53:06.398936+0000","last_unstale":"2023-10-26T05:53:08.166346+0000","last_undegraded":"2023-10-26T05:53:08.166346+0000","last_fullsized":"2023-10-26T05:53:08.166346+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:22.752003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840012+0000","last_change":"2023-10-26T05:53:07.184936+0000","last_active":"2023-10-26T05:53:08.840012+0000","last_peered":"2023-10-26T05:53:08.840012+0000","last_clean":"2023-10-26T05:53:08.840012+0000","last_became_active":"2023-10-26T05:53:07.182415+0000","last_became_peered":"2023-10-26T05:53:07.182415+0000","last_unstale":"2023-10-26T05:53:08.840012+0000","last_undegraded":"2023-10-26T05:53:08.840012+0000","last_fullsized":"2023-10-26T05:53:08.840012+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:03.987842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838453+0000","last_change":"2023-10-26T05:53:06.397742+0000","last_active":"2023-10-26T05:53:08.838453+0000","last_peered":"2023-10-26T05:53:08.838453+0000","last_clean":"2023-10-26T05:53:08.838453+0000","last_became_active":"2023-10-26T05:53:06.397344+0000","last_became_peered":"2023-10-26T05:53:06.397344+0000","last_unstale":"2023-10-26T05:53:08.838453+0000","last_undegraded":"2023-10-26T05:53:08.838453+0000","last_fullsized":"2023-10-26T05:53:08.838453+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:36:57.336842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838804+0000","last_change":"2023-10-26T05:53:07.173377+0000","last_active":"2023-10-26T05:53:08.838804+0000","last_peered":"2023-10-26T05:53:08.838804+0000","last_clean":"2023-10-26T05:53:08.838804+0000","last_became_active":"2023-10-26T05:53:07.172149+0000","last_became_peered":"2023-10-26T05:53:07.172149+0000","last_unstale":"2023-10-26T05:53:08.838804+0000","last_undegraded":"2023-10-26T05:53:08.838804+0000","last_fullsized":"2023-10-26T05:53:08.838804+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:08.955683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"123'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.166746+0000","last_change":"2023-10-26T05:53:06.410619+0000","last_active":"2023-10-26T05:53:08.166746+0000","last_peered":"2023-10-26T05:53:08.166746+0000","last_clean":"2023-10-26T05:53:08.166746+0000","last_became_active":"2023-10-26T05:53:06.409962+0000","last_became_peered":"2023-10-26T05:53:06.409962+0000","last_unstale":"2023-10-26T05:53:08.166746+0000","last_undegraded":"2023-10-26T05:53:08.166746+0000","last_fullsized":"2023-10-26T05:53:08.166746+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:50.775359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160221+0000","last_change":"2023-10-26T05:53:07.191280+0000","last_active":"2023-10-26T05:53:08.160221+0000","last_peered":"2023-10-26T05:53:08.160221+0000","last_clean":"2023-10-26T05:53:08.160221+0000","last_became_active":"2023-10-26T05:53:07.190487+0000","last_became_peered":"2023-10-26T05:53:07.190487+0000","last_unstale":"2023-10-26T05:53:08.160221+0000","last_undegraded":"2023-10-26T05:53:08.160221+0000","last_fullsized":"2023-10-26T05:53:08.160221+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:47:19.408575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839166+0000","last_change":"2023-10-26T05:53:06.397697+0000","last_active":"2023-10-26T05:53:08.839166+0000","last_peered":"2023-10-26T05:53:08.839166+0000","last_clean":"2023-10-26T05:53:08.839166+0000","last_became_active":"2023-10-26T05:53:06.396875+0000","last_became_peered":"2023-10-26T05:53:06.396875+0000","last_unstale":"2023-10-26T05:53:08.839166+0000","last_undegraded":"2023-10-26T05:53:08.839166+0000","last_fullsized":"2023-10-26T05:53:08.839166+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:46:26.420989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.160582+0000","last_change":"2023-10-26T05:53:07.182109+0000","last_active":"2023-10-26T05:53:08.160582+0000","last_peered":"2023-10-26T05:53:08.160582+0000","last_clean":"2023-10-26T05:53:08.160582+0000","last_became_active":"2023-10-26T05:53:07.181628+0000","last_became_peered":"2023-10-26T05:53:07.181628+0000","last_unstale":"2023-10-26T05:53:08.160582+0000","last_undegraded":"2023-10-26T05:53:08.160582+0000","last_fullsized":"2023-10-26T05:53:08.160582+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:27:12.418100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837275+0000","last_change":"2023-10-26T05:53:06.394474+0000","last_active":"2023-10-26T05:53:08.837275+0000","last_peered":"2023-10-26T05:53:08.837275+0000","last_clean":"2023-10-26T05:53:08.837275+0000","last_became_active":"2023-10-26T05:53:06.394227+0000","last_became_peered":"2023-10-26T05:53:06.394227+0000","last_unstale":"2023-10-26T05:53:08.837275+0000","last_undegraded":"2023-10-26T05:53:08.837275+0000","last_fullsized":"2023-10-26T05:53:08.837275+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:25.782758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840057+0000","last_change":"2023-10-26T05:53:07.200294+0000","last_active":"2023-10-26T05:53:08.840057+0000","last_peered":"2023-10-26T05:53:08.840057+0000","last_clean":"2023-10-26T05:53:08.840057+0000","last_became_active":"2023-10-26T05:53:07.199453+0000","last_became_peered":"2023-10-26T05:53:07.199453+0000","last_unstale":"2023-10-26T05:53:08.840057+0000","last_undegraded":"2023-10-26T05:53:08.840057+0000","last_fullsized":"2023-10-26T05:53:08.840057+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:08.955683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.840134+0000","last_change":"2023-10-26T05:53:06.406227+0000","last_active":"2023-10-26T05:53:08.840134+0000","last_peered":"2023-10-26T05:53:08.840134+0000","last_clean":"2023-10-26T05:53:08.840134+0000","last_became_active":"2023-10-26T05:53:06.405621+0000","last_became_peered":"2023-10-26T05:53:06.405621+0000","last_unstale":"2023-10-26T05:53:08.840134+0000","last_undegraded":"2023-10-26T05:53:08.840134+0000","last_fullsized":"2023-10-26T05:53:08.840134+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:23:13.044276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838886+0000","last_change":"2023-10-26T05:53:07.201173+0000","last_active":"2023-10-26T05:53:08.838886+0000","last_peered":"2023-10-26T05:53:08.838886+0000","last_clean":"2023-10-26T05:53:08.838886+0000","last_became_active":"2023-10-26T05:53:07.197659+0000","last_became_peered":"2023-10-26T05:53:07.197659+0000","last_unstale":"2023-10-26T05:53:08.838886+0000","last_undegraded":"2023-10-26T05:53:08.838886+0000","last_fullsized":"2023-10-26T05:53:08.838886+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:06:37.549915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838676+0000","last_change":"2023-10-26T05:53:06.386286+0000","last_active":"2023-10-26T05:53:08.838676+0000","last_peered":"2023-10-26T05:53:08.838676+0000","last_clean":"2023-10-26T05:53:08.838676+0000","last_became_active":"2023-10-26T05:53:06.385960+0000","last_became_peered":"2023-10-26T05:53:06.385960+0000","last_unstale":"2023-10-26T05:53:08.838676+0000","last_undegraded":"2023-10-26T05:53:08.838676+0000","last_fullsized":"2023-10-26T05:53:08.838676+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:22.908310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.168548+0000","last_change":"2023-10-26T05:53:07.181149+0000","last_active":"2023-10-26T05:53:08.168548+0000","last_peered":"2023-10-26T05:53:08.168548+0000","last_clean":"2023-10-26T05:53:08.168548+0000","last_became_active":"2023-10-26T05:53:07.180492+0000","last_became_peered":"2023-10-26T05:53:07.180492+0000","last_unstale":"2023-10-26T05:53:08.168548+0000","last_undegraded":"2023-10-26T05:53:08.168548+0000","last_fullsized":"2023-10-26T05:53:08.168548+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:01:02.920537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839849+0000","last_change":"2023-10-26T05:53:06.396204+0000","last_active":"2023-10-26T05:53:08.839849+0000","last_peered":"2023-10-26T05:53:08.839849+0000","last_clean":"2023-10-26T05:53:08.839849+0000","last_became_active":"2023-10-26T05:53:06.395499+0000","last_became_peered":"2023-10-26T05:53:06.395499+0000","last_unstale":"2023-10-26T05:53:08.839849+0000","last_undegraded":"2023-10-26T05:53:08.839849+0000","last_fullsized":"2023-10-26T05:53:08.839849+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:43:10.214366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837077+0000","last_change":"2023-10-26T05:53:07.193645+0000","last_active":"2023-10-26T05:53:08.837077+0000","last_peered":"2023-10-26T05:53:08.837077+0000","last_clean":"2023-10-26T05:53:08.837077+0000","last_became_active":"2023-10-26T05:53:07.192339+0000","last_became_peered":"2023-10-26T05:53:07.192339+0000","last_unstale":"2023-10-26T05:53:08.837077+0000","last_undegraded":"2023-10-26T05:53:08.837077+0000","last_fullsized":"2023-10-26T05:53:08.837077+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:14:46.807945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837697+0000","last_change":"2023-10-26T05:53:06.398766+0000","last_active":"2023-10-26T05:53:08.837697+0000","last_peered":"2023-10-26T05:53:08.837697+0000","last_clean":"2023-10-26T05:53:08.837697+0000","last_became_active":"2023-10-26T05:53:06.398518+0000","last_became_peered":"2023-10-26T05:53:06.398518+0000","last_unstale":"2023-10-26T05:53:08.837697+0000","last_undegraded":"2023-10-26T05:53:08.837697+0000","last_fullsized":"2023-10-26T05:53:08.837697+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:52.229175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.839940+0000","last_change":"2023-10-26T05:53:07.204212+0000","last_active":"2023-10-26T05:53:08.839940+0000","last_peered":"2023-10-26T05:53:08.839940+0000","last_clean":"2023-10-26T05:53:08.839940+0000","last_became_active":"2023-10-26T05:53:07.201824+0000","last_became_peered":"2023-10-26T05:53:07.201824+0000","last_unstale":"2023-10-26T05:53:08.839940+0000","last_undegraded":"2023-10-26T05:53:08.839940+0000","last_fullsized":"2023-10-26T05:53:08.839940+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:40.082223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.837169+0000","last_change":"2023-10-26T05:53:06.399777+0000","last_active":"2023-10-26T05:53:08.837169+0000","last_peered":"2023-10-26T05:53:08.837169+0000","last_clean":"2023-10-26T05:53:08.837169+0000","last_became_active":"2023-10-26T05:53:06.399571+0000","last_became_peered":"2023-10-26T05:53:06.399571+0000","last_unstale":"2023-10-26T05:53:08.837169+0000","last_undegraded":"2023-10-26T05:53:08.837169+0000","last_fullsized":"2023-10-26T05:53:08.837169+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:14:08.684639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.838527+0000","last_change":"2023-10-26T05:53:07.201930+0000","last_active":"2023-10-26T05:53:08.838527+0000","last_peered":"2023-10-26T05:53:08.838527+0000","last_clean":"2023-10-26T05:53:08.838527+0000","last_became_active":"2023-10-26T05:53:07.201156+0000","last_became_peered":"2023-10-26T05:53:07.201156+0000","last_unstale":"2023-10-26T05:53:08.838527+0000","last_undegraded":"2023-10-26T05:53:08.838527+0000","last_fullsized":"2023-10-26T05:53:08.838527+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:06.373380+0000","last_clean_scrub_stamp":"2023-10-26T05:53:06.373380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:32:47.855349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"123'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-26T05:53:08.558693+0000","last_change":"2023-10-26T05:53:06.407201+0000","last_active":"2023-10-26T05:53:08.558693+0000","last_peered":"2023-10-26T05:53:08.558693+0000","last_clean":"2023-10-26T05:53:08.558693+0000","last_became_active":"2023-10-26T05:53:06.406550+0000","last_became_peered":"2023-10-26T05:53:06.406550+0000","last_unstale":"2023-10-26T05:53:08.558693+0000","last_undegraded":"2023-10-26T05:53:08.558693+0000","last_fullsized":"2023-10-26T05:53:08.558693+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:05.369149+0000","last_clean_scrub_stamp":"2023-10-26T05:53:05.369149+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:52.229175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-26T05:53:07.189501+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:07.189501+0000","last_peered":"2023-10-26T05:53:07.189501+0000","last_clean":"2023-10-26T05:53:07.189501+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:07.189501+0000","last_undegraded":"2023-10-26T05:53:07.189501+0000","last_fullsized":"2023-10-26T05:53:07.189501+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1244,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194067,"statfs":{"total":200000000,"available":198725406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84099999999999997,"15min":0.86099999999999999},"min":{"1min":0.499,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3100000000000001,"5min":1.347,"15min":1.347},"last":0.502},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84299999999999997,"15min":0.83499999999999996},"min":{"1min":0.54700000000000004,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.171,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.97799999999999998,"15min":1.038},"min":{"1min":0.56000000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.96499999999999997,"15min":0.97199999999999998},"min":{"1min":0.75800000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.141}]},{"osd":2,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.51300000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.92900000000000005,"15min":0.93999999999999995},"min":{"1min":0.67600000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3700000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.72399999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1479999999999999,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.749},{"interface":"front","average":{"1min":1.018,"5min":0.98499999999999999,"15min":0.97699999999999998},"min":{"1min":0.65200000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2410000000000001,"5min":1.375,"15min":1.375},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.91300000000000003,"15min":0.91900000000000004},"min":{"1min":0.65200000000000002,"5min":0.433,"15min":0.433},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.95399999999999996,"15min":0.93999999999999995},"min":{"1min":0.78300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.159,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.109}]},{"osd":5,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0469999999999999,"15min":1.1180000000000001},"min":{"1min":0.74099999999999999,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3899999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0389999999999999,"15min":1.087},"min":{"1min":0.625,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.25}]},{"osd":6,"last update":"Thu Oct 26 05:52:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.0780000000000001,"15min":1.111},"min":{"1min":0.76600000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0529999999999999,"15min":1.05},"min":{"1min":0.77100000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.262,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.28}]}]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194643,"statfs":{"total":200000000,"available":199315234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82199999999999995,"15min":0.80700000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.212},"last":1.103},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.871,"15min":0.86099999999999999},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.04,"5min":1.3,"15min":1.3},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82499999999999996,"15min":0.77300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1399999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.89400000000000002,"15min":0.84099999999999997},"min":{"1min":0.52300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.323,"5min":1.405,"15min":1.405},"last":1.167}]},{"osd":2,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.88400000000000001,"15min":0.81000000000000005},"min":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.874,"15min":0.83699999999999997},"min":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.95199999999999996,"15min":0.89000000000000001},"min":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2490000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.996,"5min":1.024,"15min":0.96199999999999997},"min":{"1min":0.73499999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.2490000000000001,"5min":1.399,"15min":1.399},"last":1.056}]},{"osd":4,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.94099999999999995,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.502,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.429,"15min":1.429},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.97399999999999998,"15min":0.97099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.258,"5min":1.365,"15min":1.365},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":1.022,"15min":1.044},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3149999999999999,"5min":1.554,"15min":1.554},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.94999999999999996,"15min":0.99099999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.262,"5min":1.282,"15min":1.282},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:52:22 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.1040000000000001,"15min":1.101},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2909999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0900000000000001,"15min":1.1140000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.204,"5min":1.464,"15min":1.464},"last":1.0800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194644,"statfs":{"total":200000000,"available":199315816,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.95099999999999996,"15min":0.95899999999999996},"min":{"1min":0.60599999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3700000000000001,"5min":1.579,"15min":1.579},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.96299999999999997,"15min":0.95299999999999996},"min":{"1min":0.65600000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.2030000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.1160000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.86199999999999999,"15min":0.89600000000000002},"min":{"1min":0.57199999999999995,"5min":0.441,"15min":0.441},"max":{"1min":1.141,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2170000000000001,"5min":1.464,"15min":1.464},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.874,"15min":0.92000000000000004},"min":{"1min":0.55100000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1850000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.272,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.90700000000000003,"15min":0.91900000000000004},"min":{"1min":0.60899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.198,"5min":1.548,"15min":1.548},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.98099999999999998,"15min":1.006},"min":{"1min":0.58099999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.226,"5min":1.587,"15min":1.587},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.95199999999999996,"15min":1.004},"min":{"1min":0.66900000000000004,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1240000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.878},{"interface":"front","average":{"1min":1.097,"5min":1.0700000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3560000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.95899999999999996,"15min":1.0269999999999999},"min":{"1min":0.62,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3640000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.012},{"interface":"front","average":{"1min":1.036,"5min":1.006,"15min":1.0329999999999999},"min":{"1min":0.624,"5min":0.504,"15min":0.504},"max":{"1min":1.4279999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.081}]},{"osd":7,"last update":"Thu Oct 26 05:52:16 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.0589999999999999,"15min":1.0780000000000001},"min":{"1min":0.64500000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.341,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.121,"15min":1.1679999999999999},"min":{"1min":0.89600000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3500000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1910000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":669,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194642,"statfs":{"total":200000000,"available":199314150,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.871,"15min":0.84999999999999998},"min":{"1min":0.54000000000000004,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.2569999999999999,"5min":1.347,"15min":1.347},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.86899999999999999,"15min":0.879},"min":{"1min":0.61199999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.91900000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85099999999999998,"15min":0.81299999999999994},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.25,"5min":1.375,"15min":1.375},"last":1.012},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92500000000000004,"15min":0.89200000000000002},"min":{"1min":0.61899999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.262,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.89700000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85399999999999998,"15min":0.81200000000000006},"min":{"1min":0.52400000000000002,"5min":0.501,"15min":0.501},"max":{"1min":1.1399999999999999,"5min":1.29,"15min":1.29},"last":1.052},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.89100000000000001,"15min":0.86399999999999999},"min":{"1min":0.57199999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1970000000000001,"5min":1.373,"15min":1.373},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.92300000000000004,"15min":0.94099999999999995},"min":{"1min":0.70099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.996,"5min":0.97799999999999998,"15min":0.94399999999999995},"min":{"1min":0.64500000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1990000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.96699999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.90400000000000003,"15min":0.88600000000000001},"min":{"1min":0.55900000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.306,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.0049999999999999,"15min":1.0009999999999999},"min":{"1min":0.58299999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4179999999999999,"5min":1.673,"15min":1.673},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.02,"15min":1.002},"min":{"1min":0.65600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.369,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.044,"5min":1.0329999999999999,"15min":1.0369999999999999},"min":{"1min":0.74299999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.292,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:52:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0569999999999999,"15min":1.0389999999999999},"min":{"1min":0.871,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.341,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.74199999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.3100000000000001,"5min":1.728,"15min":1.728},"last":0.79900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":663,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194648,"statfs":{"total":200000000,"available":199320552,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60299999999999998,"15min":0.57299999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61599999999999999,"15min":0.57899999999999996},"min":{"1min":0.33900000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0680000000000001,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.115}]},{"osd":1,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.64200000000000002,"15min":0.65600000000000003},"min":{"1min":0.30199999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.2,"5min":1.349,"15min":1.349},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64100000000000001,"15min":0.59799999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.075,"5min":1.085,"15min":1.085},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.75},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.3460000000000001,"5min":1.367,"15min":1.367},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.41599999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.1850000000000001,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.872,"15min":0.82699999999999996},"min":{"1min":0.61499999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0820000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.64300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.413,"5min":1.478,"15min":1.478},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.93899999999999995,"15min":0.90600000000000003},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.0509999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94899999999999995,"15min":0.997},"min":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2889999999999999,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.054,"15min":1.028},"min":{"1min":0.629,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":1.141}]},{"osd":7,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.96799999999999997,"15min":0.92400000000000004},"min":{"1min":0.69299999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":1.171},{"interface":"front","average":{"1min":1.04,"5min":1.036,"15min":1.0449999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.016}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194644,"statfs":{"total":200000000,"available":199315852,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57599999999999996,"15min":0.53200000000000003},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.93300000000000005,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.67900000000000005,"15min":0.59499999999999997},"min":{"1min":0.40600000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.0409999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.72099999999999997,"15min":0.66600000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.226,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67300000000000004,"15min":0.59399999999999997},"min":{"1min":0.377,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.72899999999999998},"min":{"1min":0.34699999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.1970000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.66400000000000003,"15min":0.59099999999999997},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.94299999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89700000000000002,"15min":0.82299999999999995},"min":{"1min":0.628,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3160000000000001,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.91000000000000003,"15min":0.83999999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2310000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":1.075}]},{"osd":5,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.97799999999999998,"15min":0.90700000000000003},"min":{"1min":0.68000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.3819999999999999,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.96099999999999997,"15min":0.90000000000000002},"min":{"1min":0.625,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.3069999999999999,"5min":1.653,"15min":1.653},"last":1.21}]},{"osd":6,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.98899999999999999,"15min":0.91500000000000004},"min":{"1min":0.65000000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.49,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.002},{"interface":"front","average":{"1min":1.107,"5min":1.089,"15min":0.96999999999999997},"min":{"1min":0.59999999999999998,"5min":0.5,"15min":0.5},"max":{"1min":1.4319999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":1.169}]},{"osd":7,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.94899999999999995,"15min":0.89900000000000002},"min":{"1min":0.63700000000000001,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.202,"15min":1.0269999999999999},"min":{"1min":0.76100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.615,"5min":15.352,"15min":15.352},"last":1.0980000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":668,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194644,"statfs":{"total":200000000,"available":199315886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.29599999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.625},"min":{"1min":0.44800000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80900000000000005,"5min":1.157,"15min":1.157},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66400000000000003,"15min":0.65000000000000002},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.74399999999999999,"15min":0.71999999999999997},"min":{"1min":0.38,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.6080000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76700000000000002},"min":{"1min":0.38100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.121,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70199999999999996,"15min":0.68300000000000005},"min":{"1min":0.36099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.81200000000000006}]},{"osd":4,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.88600000000000001,"15min":0.86799999999999999},"min":{"1min":0.64800000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.755},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.68500000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5069999999999999,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":1.0589999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96399999999999997},"min":{"1min":0.66700000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.381,"5min":1.492,"15min":1.492},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.94199999999999995,"15min":0.93799999999999994},"min":{"1min":0.71899999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.91900000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.022,"15min":1.038},"min":{"1min":0.71299999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4490000000000001,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.0489999999999999,"5min":0.97299999999999998,"15min":0.97199999999999998},"min":{"1min":0.71299999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.468,"5min":1.837,"15min":1.837},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:52:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0149999999999999,"15min":1.0289999999999999},"min":{"1min":0.65700000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.4179999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.137},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.095,"15min":1.093},"min":{"1min":0.77300000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8180000000000001,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.1739999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574922,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1245,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194067,"statfs":{"total":200000000,"available":198724936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.32200000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.91800000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64100000000000001,"15min":0.59899999999999998},"min":{"1min":0.28299999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.72399999999999998,"15min":0.65800000000000003},"min":{"1min":0.435,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.64800000000000002,"15min":0.60399999999999998},"min":{"1min":0.36599999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.96699999999999997,"5min":1.177,"15min":1.177},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64100000000000001,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.95299999999999996,"5min":1.409,"15min":1.409},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72799999999999998,"15min":0.68400000000000005},"min":{"1min":0.32100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":1.1060000000000001,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.88200000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88900000000000001,"15min":0.84799999999999998},"min":{"1min":0.69899999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1200000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.93000000000000005,"15min":0.89600000000000002},"min":{"1min":0.59799999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.399,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.169}]},{"osd":5,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.3169999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.90300000000000002},"min":{"1min":0.70799999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1379999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.93700000000000006,"15min":0.90600000000000003},"min":{"1min":0.624,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.004,"15min":0.95999999999999996},"min":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.486,"5min":1.764,"15min":1.764},"last":1.117}]},{"osd":7,"last update":"Thu Oct 26 05:52:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.996,"15min":0.996},"min":{"1min":0.69799999999999995,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.669,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.095},{"interface":"front","average":{"1min":1.05,"5min":1.012,"15min":0.98299999999999998},"min":{"1min":0.76700000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.3080000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.2070000000000001}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:10.798 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:53:10.798 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:53:10.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 -- 172.21.15.71:0/275280330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40d0850 msgr2=0x7f59d415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 --2- 172.21.15.71:0/275280330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40d0850 0x7f59d415b3a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009900 tx=0x7f59c401c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 -- 172.21.15.71:0/275280330 shutdown_connections 2023-10-26T05:53:10.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 --2- 172.21.15.71:0/275280330 >> v2:172.21.15.86:3300/0 conn(0x7f59d40cd880 0x7f59d415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 --2- 172.21.15.71:0/275280330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40d0850 0x7f59d415b3a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 -- 172.21.15.71:0/275280330 >> 172.21.15.71:0/275280330 conn(0x7f59d40cdd80 msgr2=0x7f59d40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:10.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 -- 172.21.15.71:0/275280330 shutdown_connections 2023-10-26T05:53:10.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.948+0000 7f59dc20e640 1 -- 172.21.15.71:0/275280330 wait complete. 2023-10-26T05:53:10.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 Processor -- start 2023-10-26T05:53:10.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- start start 2023-10-26T05:53:10.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40cd880 0x7f59d4151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40d0470 0x7f59d40b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 0x7f59d40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41540b0 con 0x7f59d40d0850 2023-10-26T05:53:10.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4153010 con 0x7f59d40d0470 2023-10-26T05:53:10.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41532f0 con 0x7f59d40cd880 2023-10-26T05:53:10.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 0x7f59d40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 0x7f59d40b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52998/0 (socket says 172.21.15.71:52998) 2023-10-26T05:53:10.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 -- 172.21.15.71:0/1333020164 learned_addr learned my addr 172.21.15.71:0/1333020164 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:10.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59d9faa640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40cd880 0x7f59d4151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59d97a9640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40d0470 0x7f59d40b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 -- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40cd880 msgr2=0x7f59d4151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40cd880 0x7f59d4151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 -- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40d0470 msgr2=0x7f59d40b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:10.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40d0470 0x7f59d40b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:10.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 -- 172.21.15.71:0/1333020164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c4009560 con 0x7f59d40d0850 2023-10-26T05:53:10.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59d97a9640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40d0470 0x7f59d40b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:10.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59da7ab640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 0x7f59d40b1f80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e8c0 tx=0x7f59cc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:10.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc009730 con 0x7f59d40d0850 2023-10-26T05:53:10.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59cc00ccc0 con 0x7f59d40d0850 2023-10-26T05:53:10.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d41535d0 con 0x7f59d40d0850 2023-10-26T05:53:10.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc005280 con 0x7f59d40d0850 2023-10-26T05:53:10.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d4153b90 con 0x7f59d40d0850 2023-10-26T05:53:10.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d4066b10 con 0x7f59d40d0850 2023-10-26T05:53:10.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f59cc0053e0 con 0x7f59d40d0850 2023-10-26T05:53:10.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59ca7fc640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59b8066470 0x7f59b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:10.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59cc014070 con 0x7f59d40d0850 2023-10-26T05:53:10.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.952+0000 7f59d9faa640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59b8066470 0x7f59b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:10.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.956+0000 7f59d9faa640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59b8066470 0x7f59b8068930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f59d4151e00 tx=0x7f59d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:10.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:10.956+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59cc0558e0 con 0x7f59d40d0850 2023-10-26T05:53:11.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.160+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f59d4153d80 con 0x7f59d40d0850 2023-10-26T05:53:11.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.164+0000 7f59ca7fc640 1 -- 172.21.15.71:0/1333020164 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) v1 ==== 190+0+147 (secure 0 0 0) 0x7f59cc055470 con 0x7f59d40d0850 2023-10-26T05:53:11.173 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:53:11.173 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:53:11.173 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:53:11.173 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:53:11.174 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:53:11.174 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:53:11.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59b8066470 msgr2=0x7f59b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59b8066470 0x7f59b8068930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f59d4151e00 tx=0x7f59d0009040 comp rx=0 tx=0).stop 2023-10-26T05:53:11.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 msgr2=0x7f59d40b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 0x7f59d40b1f80 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00e8c0 tx=0x7f59cc00ed80 comp rx=0 tx=0).stop 2023-10-26T05:53:11.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 shutdown_connections 2023-10-26T05:53:11.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59b8066470 0x7f59b8068930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59d40d0850 0x7f59d40b1f80 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d40d0470 0x7f59d40b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 --2- 172.21.15.71:0/1333020164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59d40cd880 0x7f59d4151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 >> 172.21.15.71:0/1333020164 conn(0x7f59d40cdd80 msgr2=0x7f59d4154af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:11.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 shutdown_connections 2023-10-26T05:53:11.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.168+0000 7f59dc20e640 1 -- 172.21.15.71:0/1333020164 wait complete. 2023-10-26T05:53:11.190 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:11.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/2391615688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc0b2370 msgr2=0x7ff7fc071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 --2- 172.21.15.71:0/2391615688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc0b2370 0x7ff7fc071470 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009c80 tx=0x7ff7f401ccd0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/2391615688 shutdown_connections 2023-10-26T05:53:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 --2- 172.21.15.71:0/2391615688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc0b2370 0x7ff7fc071470 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 --2- 172.21.15.71:0/2391615688 >> v2:172.21.15.86:3300/0 conn(0x7ff7fc140cc0 0x7ff7fc15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/2391615688 >> 172.21.15.71:0/2391615688 conn(0x7ff7fc069770 msgr2=0x7ff7fc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/2391615688 shutdown_connections 2023-10-26T05:53:11.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/2391615688 wait complete. 2023-10-26T05:53:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 Processor -- start 2023-10-26T05:53:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- start start 2023-10-26T05:53:11.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 0x7ff7fc13cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7fc0b2370 0x7ff7fc13e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7fc140cc0 0x7ff7fc1577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc13e800 con 0x7ff7fc070fe0 2023-10-26T05:53:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc13ea10 con 0x7ff7fc140cc0 2023-10-26T05:53:11.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc13ecf0 con 0x7ff7fc0b2370 2023-10-26T05:53:11.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 0x7ff7fc13cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:11.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 0x7ff7fc13cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53012/0 (socket says 172.21.15.71:53012) 2023-10-26T05:53:11.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 -- 172.21.15.71:0/400027924 learned_addr learned my addr 172.21.15.71:0/400027924 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:11.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7fb7fe640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7fc140cc0 0x7ff7fc1577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:11.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7fa7fc640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7fc0b2370 0x7ff7fc13e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:11.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 -- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7fc0b2370 msgr2=0x7ff7fc13e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7fc0b2370 0x7ff7fc13e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 -- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7fc140cc0 msgr2=0x7ff7fc1577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7fc140cc0 0x7ff7fc1577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 -- 172.21.15.71:0/400027924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f40098e0 con 0x7ff7fc070fe0 2023-10-26T05:53:11.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7faffd640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 0x7ff7fc13cba0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c9f0 tx=0x7ff7ec00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:11.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec007bb0 con 0x7ff7fc070fe0 2023-10-26T05:53:11.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff7ec007e00 con 0x7ff7fc070fe0 2023-10-26T05:53:11.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc13efd0 con 0x7ff7fc070fe0 2023-10-26T05:53:11.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00f6b0 con 0x7ff7fc070fe0 2023-10-26T05:53:11.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.304+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc13f590 con 0x7ff7fc070fe0 2023-10-26T05:53:11.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.308+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc066b10 con 0x7ff7fc070fe0 2023-10-26T05:53:11.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.308+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7ec00f810 con 0x7ff7fc070fe0 2023-10-26T05:53:11.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.308+0000 7ff7e3fff640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7e8066470 0x7ff7e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:11.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.308+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7ec084640 con 0x7ff7fc070fe0 2023-10-26T05:53:11.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.308+0000 7ff7fa7fc640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7e8066470 0x7ff7e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:11.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.308+0000 7ff7fa7fc640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7e8066470 0x7ff7e8068930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc134100 tx=0x7ff7f0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:11.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.312+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff7ec051b50 con 0x7ff7fc070fe0 2023-10-26T05:53:11.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.484+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7fc0721e0 con 0x7ff7fc070fe0 2023-10-26T05:53:11.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.492+0000 7ff7e3fff640 1 -- 172.21.15.71:0/400027924 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7ff7ec055170 con 0x7ff7fc070fe0 2023-10-26T05:53:11.501 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:11.501 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:11.501 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7e8066470 msgr2=0x7ff7e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7e8066470 0x7ff7e8068930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc134100 tx=0x7ff7f0009040 comp rx=0 tx=0).stop 2023-10-26T05:53:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 msgr2=0x7ff7fc13cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 0x7ff7fc13cba0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c9f0 tx=0x7ff7ec00ceb0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 shutdown_connections 2023-10-26T05:53:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7fc140cc0 0x7ff7fc1577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7fc0b2370 0x7ff7fc13e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7e8066470 0x7ff7e8068930 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 --2- 172.21.15.71:0/400027924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc070fe0 0x7ff7fc13cba0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:11.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 >> 172.21.15.71:0/400027924 conn(0x7ff7fc069770 msgr2=0x7ff7fc0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 shutdown_connections 2023-10-26T05:53:11.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:11.496+0000 7ff801842640 1 -- 172.21.15.71:0/400027924 wait complete. 2023-10-26T05:53:11.922 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:53:06.736075+0000', 'modified': '2023-10-26T05:53:08.786697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7545}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7545': {'gid': 7545, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/1526458745', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 1526458745}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1526458745}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:53:11.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:53:11.923 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T05:53:12.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 -- 172.21.15.71:0/1930672068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 msgr2=0x7f700c13d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 --2- 172.21.15.71:0/1930672068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c13d770 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f700c06b1f0 tx=0x7f6ffc01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 -- 172.21.15.71:0/1930672068 shutdown_connections 2023-10-26T05:53:12.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 --2- 172.21.15.71:0/1930672068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c13d770 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 --2- 172.21.15.71:0/1930672068 >> v2:172.21.15.86:3300/0 conn(0x7f700c13ae90 0x7f700c0cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 -- 172.21.15.71:0/1930672068 >> 172.21.15.71:0/1930672068 conn(0x7f700c06b8c0 msgr2=0x7f700c06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 -- 172.21.15.71:0/1930672068 shutdown_connections 2023-10-26T05:53:12.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.048+0000 7f70142a5640 1 -- 172.21.15.71:0/1930672068 wait complete. 2023-10-26T05:53:12.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 Processor -- start 2023-10-26T05:53:12.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 -- start start 2023-10-26T05:53:12.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c13ae90 0x7f700c06a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f700c13dcb0 0x7f700c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c14e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c150b90 con 0x7f700c159060 2023-10-26T05:53:12.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c14ff80 con 0x7f700c13ae90 2023-10-26T05:53:12.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c150260 con 0x7f700c13dcb0 2023-10-26T05:53:12.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c14e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c14e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55554/0 (socket says 172.21.15.71:55554) 2023-10-26T05:53:12.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 -- 172.21.15.71:0/3184461532 learned_addr learned my addr 172.21.15.71:0/3184461532 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:12.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012041640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c13ae90 0x7f700c06a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7011840640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f700c13dcb0 0x7f700c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 -- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f700c13dcb0 msgr2=0x7f700c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f700c13dcb0 0x7f700c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 -- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c13ae90 msgr2=0x7f700c06a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c13ae90 0x7f700c06a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 -- 172.21.15.71:0/3184461532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009560 con 0x7f700c159060 2023-10-26T05:53:12.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7011840640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f700c13dcb0 0x7f700c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:12.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7012842640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c14e310 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f700400c780 tx=0x7f700400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7002ffd640 1 -- 172.21.15.71:0/3184461532 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004004b70 con 0x7f700c159060 2023-10-26T05:53:12.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7002ffd640 1 -- 172.21.15.71:0/3184461532 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f700400ce90 con 0x7f700c159060 2023-10-26T05:53:12.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c150540 con 0x7f700c159060 2023-10-26T05:53:12.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7002ffd640 1 -- 172.21.15.71:0/3184461532 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400aab0 con 0x7f700c159060 2023-10-26T05:53:12.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c14fc70 con 0x7f700c159060 2023-10-26T05:53:12.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.052+0000 7f7002ffd640 1 -- 172.21.15.71:0/3184461532 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f700400ac10 con 0x7f700c159060 2023-10-26T05:53:12.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7002ffd640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ff00664a0 0x7f6ff0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7002ffd640 1 -- 172.21.15.71:0/3184461532 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7004084380 con 0x7f700c159060 2023-10-26T05:53:12.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7012041640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ff00664a0 0x7f6ff0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 Processor -- start 2023-10-26T05:53:12.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- start start 2023-10-26T05:53:12.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7012041640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ff00664a0 0x7f6ff0068960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7008006fd0 tx=0x7f7008008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36c05da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36bbba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36bbbf80 0x557b36bbc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b369b0cc0 con 0x557b36c062e0 2023-10-26T05:53:12.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36a13920 con 0x557b36bbbf80 2023-10-26T05:53:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b36a10e50 con 0x557b36c059c0 2023-10-26T05:53:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36bbba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36bbba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55570/0 (socket says 172.21.15.71:55570) 2023-10-26T05:53:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 -- 172.21.15.71:0/369097369 learned_addr learned my addr 172.21.15.71:0/369097369 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:12.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7012041640 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36c05da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7012842640 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36bbbf80 0x557b36bbc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 -- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 msgr2=0x557b36c05da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36c05da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 -- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36bbbf80 msgr2=0x557b36bbc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36bbbf80 0x557b36bbc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 -- 172.21.15.71:0/369097369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b36705890 con 0x557b36c062e0 2023-10-26T05:53:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f7011840640 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36bbba40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0b0040 tx=0x7f6ffc019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aab4153f82774c server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f6feffff640 1 -- 172.21.15.71:0/369097369 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0a5df0 con 0x557b36c062e0 2023-10-26T05:53:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f6feffff640 1 -- 172.21.15.71:0/369097369 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ffc004110 con 0x557b36c062e0 2023-10-26T05:53:12.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f6feffff640 1 -- 172.21.15.71:0/369097369 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc004500 con 0x557b36c062e0 2023-10-26T05:53:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- 172.21.15.71:0/369097369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 msgr2=0x557b36bbba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36bbba40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0b0040 tx=0x7f6ffc019540 comp rx=0 tx=0).stop 2023-10-26T05:53:12.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- 172.21.15.71:0/369097369 shutdown_connections 2023-10-26T05:53:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36bbbf80 0x557b36bbc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36bbba40 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 --2- 172.21.15.71:0/369097369 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36c05da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- 172.21.15.71:0/369097369 >> 172.21.15.71:0/369097369 conn(0x557b36bb32f0 msgr2=0x557b36bba320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- 172.21.15.71:0/369097369 shutdown_connections 2023-10-26T05:53:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.056+0000 7f70173ef000 1 -- 172.21.15.71:0/369097369 wait complete. 2023-10-26T05:53:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- start start 2023-10-26T05:53:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 Processor -- start 2023-10-26T05:53:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 0x557b36c05410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36bc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36bc18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36a0c740 con 0x557b36bbbf80 2023-10-26T05:53:12.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b369cbd10 con 0x557b36c062e0 2023-10-26T05:53:12.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b369cb9b0 con 0x557b36c059c0 2023-10-26T05:53:12.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 10 client.0 _ll_register_callbacks cb 0x557b36ba76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:53:12.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 0x557b36c05410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 0x557b36c05410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55572/0 (socket says 172.21.15.71:55572) 2023-10-26T05:53:12.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 -- 172.21.15.71:0/2771451472 learned_addr learned my addr 172.21.15.71:0/2771451472 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:12.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012842640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36bc18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7011840640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36bc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 -- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 msgr2=0x557b36bc1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36bc1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 -- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 msgr2=0x557b36bc18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36bc18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 -- 172.21.15.71:0/2771451472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c0cc210 con 0x557b36bbbf80 2023-10-26T05:53:12.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7011840640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36bc1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:12.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012842640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36bc18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:12.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7012041640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 0x557b36c05410 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f700800df00 tx=0x7f70080046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d680f43f7e46855f server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:53:12.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 1 -- 172.21.15.71:0/2771451472 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008014e20 con 0x557b36bbbf80 2023-10-26T05:53:12.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 1 -- 172.21.15.71:0/2771451472 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7008020470 con 0x557b36bbbf80 2023-10-26T05:53:12.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b369c0370 con 0x557b36bbbf80 2023-10-26T05:53:12.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 1 -- 172.21.15.71:0/2771451472 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7008020630 con 0x557b36bbbf80 2023-10-26T05:53:12.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 10 client.7680 fetch_fsmap learned FSMap version 44 2023-10-26T05:53:12.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x557b369ce460 con 0x557b36bbbf80 2023-10-26T05:53:12.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 1 -- 172.21.15.71:0/2771451472 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x557b369c0370 con 0x557b36bbbf80 2023-10-26T05:53:12.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 1 -- 172.21.15.71:0/2771451472 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7008020880 con 0x557b36bbbf80 2023-10-26T05:53:12.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 10 client.7680 fetch_fsmap finished waiting for FSMap version 44 2023-10-26T05:53:12.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 10 client.7680 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:53:12.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 20 client.7680 populate_metadata read hostname 'smithi071' 2023-10-26T05:53:12.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36bc8410 0x557b36c17a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 4 client.7680 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:53:12.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 --> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b369a62d0 con 0x557b36bc8410 2023-10-26T05:53:12.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7011840640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36bc8410 0x557b36c17a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 10 client.7680 ms_handle_connect on v2:172.21.15.71:6834/1526458745 2023-10-26T05:53:12.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f7011840640 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36bc8410 0x557b36c17a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d44c10ca4c0a73 server_cookie=7135e9f75051046a in_seq=0 out_seq=0 2023-10-26T05:53:12.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 1 -- 172.21.15.71:0/2771451472 <== mds.0 v2:172.21.15.71:6834/1526458745 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557b369a62d0 con 0x557b36bc8410 2023-10-26T05:53:12.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 10 client.7680 handle_command_reply: tid=0 2023-10-26T05:53:12.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 client.7680 shutdown 2023-10-26T05:53:12.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 msgr2=0x557b36c05410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 0x557b36c05410 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f700800df00 tx=0x7f70080046d0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 shutdown_connections 2023-10-26T05:53:12.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36bc8410 0x557b36c17a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36bc18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36c059c0 0x557b36bc1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 --2- 172.21.15.71:0/2771451472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36bbbf80 0x557b36c05410 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f6fed7fa640 0 client.7680 ms_handle_reset on v2:172.21.15.71:6834/1526458745 2023-10-26T05:53:12.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 >> 172.21.15.71:0/2771451472 conn(0x557b36bb32f0 msgr2=0x557b36bbaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 shutdown_connections 2023-10-26T05:53:12.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 1 -- 172.21.15.71:0/2771451472 wait complete. 2023-10-26T05:53:12.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.060+0000 7f70173ef000 20 client.7680 trim_cache size 0 max 16384 2023-10-26T05:53:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 Processor -- start 2023-10-26T05:53:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 -- start start 2023-10-26T05:53:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 0x557b36c03f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36c20e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b366c6940 con 0x557b36c059c0 2023-10-26T05:53:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b369d5720 con 0x557b36c062e0 2023-10-26T05:53:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b36c1a080 con 0x557b36bbbf80 2023-10-26T05:53:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 0x557b36c03f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 0x557b36c03f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55584/0 (socket says 172.21.15.71:55584) 2023-10-26T05:53:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 -- 172.21.15.71:0/2393111277 learned_addr learned my addr 172.21.15.71:0/2393111277 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 -- 172.21.15.71:0/2393111277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 msgr2=0x557b36c039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 -- 172.21.15.71:0/2393111277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 msgr2=0x557b36c20e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36c20e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 -- 172.21.15.71:0/2393111277 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b369b4760 con 0x557b36c059c0 2023-10-26T05:53:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7011840640 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 0x557b36c03f10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0a8020 tx=0x7f6ffc01c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70699be089817f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7000ff9640 1 -- 172.21.15.71:0/2393111277 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc005da0 con 0x557b36c059c0 2023-10-26T05:53:12.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7000ff9640 1 -- 172.21.15.71:0/2393111277 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6ffc004240 con 0x557b36c059c0 2023-10-26T05:53:12.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.076+0000 7f7000ff9640 1 -- 172.21.15.71:0/2393111277 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0a5340 con 0x557b36c059c0 2023-10-26T05:53:12.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/2393111277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 msgr2=0x557b36c03f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 0x557b36c03f10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0a8020 tx=0x7f6ffc01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/2393111277 shutdown_connections 2023-10-26T05:53:12.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c062e0 0x557b36c20e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c059c0 0x557b36c03f10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/2393111277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/2393111277 >> 172.21.15.71:0/2393111277 conn(0x557b36bb32f0 msgr2=0x557b36bc7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/2393111277 shutdown_connections 2023-10-26T05:53:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/2393111277 wait complete. 2023-10-26T05:53:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- start start 2023-10-26T05:53:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 Processor -- start 2023-10-26T05:53:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c059c0 0x557b36c68fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36c23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b369cb4d0 con 0x557b36c062e0 2023-10-26T05:53:12.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b36bbf8b0 con 0x557b36c059c0 2023-10-26T05:53:12.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b36a0f2a0 con 0x557b36bbbf80 2023-10-26T05:53:12.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 10 client.0 _ll_register_callbacks cb 0x557b36a0aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:53:12.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36c23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36c23ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55590/0 (socket says 172.21.15.71:55590) 2023-10-26T05:53:12.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 -- 172.21.15.71:0/4235955474 learned_addr learned my addr 172.21.15.71:0/4235955474 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:12.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012041640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 -- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 msgr2=0x557b36c68a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7011840640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c059c0 0x557b36c68fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c68a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 -- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c059c0 msgr2=0x557b36c68fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:12.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c059c0 0x557b36c68fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 -- 172.21.15.71:0/4235955474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b36705890 con 0x557b36c062e0 2023-10-26T05:53:12.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012842640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36c23ac0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7004003b30 tx=0x7f70040554e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd90cb7823fcee18 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:53:12.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 1 -- 172.21.15.71:0/4235955474 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004051b30 con 0x557b36c062e0 2023-10-26T05:53:12.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 1 -- 172.21.15.71:0/4235955474 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7004051c90 con 0x557b36c062e0 2023-10-26T05:53:12.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b369a46f0 con 0x557b36c062e0 2023-10-26T05:53:12.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 1 -- 172.21.15.71:0/4235955474 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7004059150 con 0x557b36c062e0 2023-10-26T05:53:12.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 10 client.7686 fetch_fsmap learned FSMap version 44 2023-10-26T05:53:12.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x557b36705cc0 con 0x557b36c062e0 2023-10-26T05:53:12.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 1 -- 172.21.15.71:0/4235955474 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x557b369a46f0 con 0x557b36c062e0 2023-10-26T05:53:12.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 1 -- 172.21.15.71:0/4235955474 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f700406c030 con 0x557b36c062e0 2023-10-26T05:53:12.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 10 client.7686 fetch_fsmap finished waiting for FSMap version 44 2023-10-26T05:53:12.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 10 client.7686 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:53:12.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 20 client.7686 populate_metadata read hostname 'smithi071' 2023-10-26T05:53:12.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36c6f930 0x557b36c71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 4 client.7686 mds_command: new command op to 7545 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:53:12.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 --> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557b36a05720 con 0x557b36c6f930 2023-10-26T05:53:12.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012041640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36c6f930 0x557b36c71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 10 client.7686 ms_handle_connect on v2:172.21.15.71:6834/1526458745 2023-10-26T05:53:12.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f7012041640 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36c6f930 0x557b36c71d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417d54ef90b26581 server_cookie=47283058a25c92ab in_seq=0 out_seq=0 2023-10-26T05:53:12.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 1 -- 172.21.15.71:0/4235955474 <== mds.0 v2:172.21.15.71:6834/1526458745 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557b36a05720 con 0x557b36c6f930 2023-10-26T05:53:12.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 10 client.7686 handle_command_reply: tid=0 2023-10-26T05:53:12.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 client.7686 shutdown 2023-10-26T05:53:12.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 msgr2=0x557b36c23ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36c23ac0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7004003b30 tx=0x7f70040554e0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 shutdown_connections 2023-10-26T05:53:12.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:6834/1526458745,v1:172.21.15.71:6835/1526458745] conn(0x557b36c6f930 0x557b36c71d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557b36c062e0 0x557b36c23ac0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b36c059c0 0x557b36c68fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 --2- 172.21.15.71:0/4235955474 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x557b36bbbf80 0x557b36c68a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 >> 172.21.15.71:0/4235955474 conn(0x557b36bb32f0 msgr2=0x557b36c029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f6fed7fa640 0 client.7686 ms_handle_reset on v2:172.21.15.71:6834/1526458745 2023-10-26T05:53:12.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 shutdown_connections 2023-10-26T05:53:12.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 1 -- 172.21.15.71:0/4235955474 wait complete. 2023-10-26T05:53:12.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.080+0000 7f70173ef000 20 client.7686 trim_cache size 0 max 16384 2023-10-26T05:53:12.113 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:12.113 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:53:12.113 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:53:12.114 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7545, 2023-10-26T05:53:12.114 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:53:12.114 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:53:12.114 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:53:12.114 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 4.6545324810000004, 2023-10-26T05:53:12.115 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 44, 2023-10-26T05:53:12.115 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 124, 2023-10-26T05:53:12.115 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 124, 2023-10-26T05:53:12.115 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 10.480829277 2023-10-26T05:53:12.115 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:12.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ff00664a0 msgr2=0x7f6ff0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ff00664a0 0x7f6ff0068960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7008006fd0 tx=0x7f7008008040 comp rx=0 tx=0).stop 2023-10-26T05:53:12.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 msgr2=0x7f700c14e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c14e310 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f700400c780 tx=0x7f700400cc40 comp rx=0 tx=0).stop 2023-10-26T05:53:12.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 shutdown_connections 2023-10-26T05:53:12.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6ff00664a0 0x7f6ff0068960 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f700c159060 0x7f700c14e310 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f700c13dcb0 0x7f700c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 --2- 172.21.15.71:0/3184461532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f700c13ae90 0x7f700c06a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 >> 172.21.15.71:0/3184461532 conn(0x7f700c06b8c0 msgr2=0x7f700c139660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 shutdown_connections 2023-10-26T05:53:12.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.084+0000 7f70142a5640 1 -- 172.21.15.71:0/3184461532 wait complete. 2023-10-26T05:53:12.119 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:53:12.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 -- 172.21.15.71:0/211630648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 msgr2=0x7f2e6806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 --2- 172.21.15.71:0/211630648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e6806aa70 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2e6808a140 tx=0x7f2e6401c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 -- 172.21.15.71:0/211630648 shutdown_connections 2023-10-26T05:53:12.241 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 --2- 172.21.15.71:0/211630648 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 0x7f2e6815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 --2- 172.21.15.71:0/211630648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e6806aa70 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 -- 172.21.15.71:0/211630648 >> 172.21.15.71:0/211630648 conn(0x7f2e680cdd80 msgr2=0x7f2e680ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 -- 172.21.15.71:0/211630648 shutdown_connections 2023-10-26T05:53:12.242 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.232+0000 7f2e7035b640 1 -- 172.21.15.71:0/211630648 wait complete. 2023-10-26T05:53:12.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 Processor -- start 2023-10-26T05:53:12.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- start start 2023-10-26T05:53:12.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 0x7f2e68150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e68156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e680d0850 0x7f2e681574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6815dd20 con 0x7f2e680d0470 2023-10-26T05:53:12.245 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68151ed0 con 0x7f2e680d0850 2023-10-26T05:53:12.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6815d0a0 con 0x7f2e680cd880 2023-10-26T05:53:12.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e68156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e68156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55606/0 (socket says 172.21.15.71:55606) 2023-10-26T05:53:12.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 -- 172.21.15.71:0/2459973619 learned_addr learned my addr 172.21.15.71:0/2459973619 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:12.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6e0f7640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 0x7f2e68150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 -- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 msgr2=0x7f2e68150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 0x7f2e68150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 -- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e680d0850 msgr2=0x7f2e681574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e680d0850 0x7f2e681574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 -- 172.21.15.71:0/2459973619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64009560 con 0x7f2e680d0470 2023-10-26T05:53:12.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6e0f7640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 0x7f2e68150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:12.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e6d8f6640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e68156f90 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b3e0 tx=0x7f2e5800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58011830 con 0x7f2e680d0470 2023-10-26T05:53:12.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e58011aa0 con 0x7f2e680d0470 2023-10-26T05:53:12.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6815d380 con 0x7f2e680d0470 2023-10-26T05:53:12.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58010380 con 0x7f2e680d0470 2023-10-26T05:53:12.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6815d940 con 0x7f2e680d0470 2023-10-26T05:53:12.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e68066b10 con 0x7f2e680d0470 2023-10-26T05:53:12.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.236+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2e580104e0 con 0x7f2e680d0470 2023-10-26T05:53:12.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.240+0000 7f2e5effd640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e4c066470 0x7f2e4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:12.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.240+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e580842a0 con 0x7f2e680d0470 2023-10-26T05:53:12.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.240+0000 7f2e6e0f7640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e4c066470 0x7f2e4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:12.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.240+0000 7f2e6e0f7640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e4c066470 0x7f2e4c068930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2e640af000 tx=0x7f2e6401afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:12.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.244+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2e58051720 con 0x7f2e680d0470 2023-10-26T05:53:12.374 INFO:tasks.ceph.mds.a.smithi071.stderr:2023-10-26T05:53:12.368+0000 7f22f6239640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:53:12.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.432+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e6815de60 con 0x7f2e680d0470 2023-10-26T05:53:12.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.432+0000 7f2e5effd640 1 -- 172.21.15.71:0/2459973619 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e6815de60 con 0x7f2e680d0470 2023-10-26T05:53:12.440 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:53:12.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e4c066470 msgr2=0x7f2e4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e4c066470 0x7f2e4c068930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2e640af000 tx=0x7f2e6401afe0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 msgr2=0x7f2e68156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e68156f90 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b3e0 tx=0x7f2e5800b8a0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 shutdown_connections 2023-10-26T05:53:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e680d0850 0x7f2e681574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e4c066470 0x7f2e4c068930 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e680d0470 0x7f2e68156f90 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 --2- 172.21.15.71:0/2459973619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e680cd880 0x7f2e68150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 >> 172.21.15.71:0/2459973619 conn(0x7f2e680cdd80 msgr2=0x7f2e6815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 shutdown_connections 2023-10-26T05:53:12.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:12.436+0000 7f2e7035b640 1 -- 172.21.15.71:0/2459973619 wait complete. 2023-10-26T05:53:12.459 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:53:12.459 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:53:12.459 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:53:12.460 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:53:12.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:12.461 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:53:12.461 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:53:12.461 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:53:12.462 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:53:12.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.462 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:53:12.468 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:53:12.468 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:53:12.468 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:53:12.468 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.469 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:53:12.469 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.469 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:12.469 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:12.469 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:53:12.470 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:12.470 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:12.470 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:53:12.470 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:12.470 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:12.471 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:53:12.471 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:12.471 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:53:12.471 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42184sec preferred_lft 42184sec 2023-10-26T05:53:12.471 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:53:12.471 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.472 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:53:12.472 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:12.472 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:53:12.472 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.472 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:53:12.473 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:12.473 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:53:12.473 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.473 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:53:12.473 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.473 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:12.474 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:12.474 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:53:12.474 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.474 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:12.474 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:12.475 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:53:12.475 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:12.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.476 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:53:12.520 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:53:12.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.522 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:53:12.571 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:53:12.571 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:53:12.571 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:53:12.572 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:53:12.572 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:53:12.572 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:53:12.572 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:12.572 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:12.573 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:12.573 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:53:12.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.574 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:53:12.574 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:53:12.574 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:53:12.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:53:12.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:53:12.575 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:53:12.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.646 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:53:12.694 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:53:12.694 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:53:12.694 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:53:12.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:53:12.748 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:53:12.749 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:12.749 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:12.797 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:53:12.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:12.798 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:12.845 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:12.894 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:53:12.895 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:53:12.895 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:53:12.895 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:53:12.895 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:53:12.896 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:53:12.896 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:53:12.896 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:53:12.896 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:53:12.896 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:53:12.897 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:53:12.897 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:53:12.897 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:12.897 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:53:12.897 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:53:12.898 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:53:12.898 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:53:12.898 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:53:12.898 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:12.898 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:53:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:53:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:53:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:53:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:53:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:53:12.900 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:53:12.900 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:53:12.900 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:53:12.900 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:12.900 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:12.901 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:12.901 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:12.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:12.902 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:53:12.949 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:53:12.950 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:53:12.994 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:13.001 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:53:13.001 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:53:13.002 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:53:13.002 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:53:13.002 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:53:13.002 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:53:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:53:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:53:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:53:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:53:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:53:13.003 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:53:13.004 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:13.004 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:53:13.004 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:53:13.004 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:53:13.004 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:53:13.004 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:53:13.005 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:13.005 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:53:13.005 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:13.005 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:53:13.005 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:53:13.006 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:53:13.006 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:53:13.006 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:53:13.006 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:53:13.006 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:53:13.007 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:53:13.007 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:13.007 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:13.007 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:13.007 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:13.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:13.009 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:53:13.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:53:13.072+0000 7fb297dd3580 -1 init, newargv = 0x5559888fcc30 newargc=15 2023-10-26T05:53:13.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[47484]: starting ceph client 2023-10-26T05:53:13.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[47484]: starting fuse 2023-10-26T05:53:14.155 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:14.165 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:53:14.165 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:53:14.165 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:53:14.165 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:53:14.165 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:53:14.166 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:53:14.166 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:53:14.166 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:53:14.166 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:53:14.166 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:53:14.167 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:53:14.167 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:53:14.167 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:14.167 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:53:14.167 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:53:14.167 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:53:14.168 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:53:14.168 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:53:14.168 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:14.168 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:53:14.168 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:14.169 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:53:14.169 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:53:14.169 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:53:14.169 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:53:14.169 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:53:14.170 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:53:14.170 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:53:14.170 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:53:14.170 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:14.170 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:14.171 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:14.171 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:14.171 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:14.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:14.172 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:53:14.218 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:53:14.218 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:53:14.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:14.219 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:53:14.220 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:53:14.220 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:14.220 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:53:14.221 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:14.221 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:53:14.221 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:53:14.222 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:53:14.222 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:53:14.223 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:14.223 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:53:14.223 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:53:14.224 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:53:14.224 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:53:14.224 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:14.225 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:53:14.225 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:14.226 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:14.226 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:14.226 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:53:14.227 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:53:14.227 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:53:14.227 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:14.228 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:14.228 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:53:14.229 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:14.339 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:14.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:14.351 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:14.352 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:14.352 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.47484.asok status 2023-10-26T05:53:14.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:14.540 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:14.541 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:53:14.541 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:14.541 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:14.541 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:53:14.541 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:53:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:53:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "47484", 2023-10-26T05:53:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:53:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:14.542 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:53:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7698, 2023-10-26T05:53:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:53:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:53:14.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:53:14.544 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 7698 2023-10-26T05:53:14.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:14.544 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:14.544 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:14.544 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:14.545 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:14.545 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:14.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:14.545 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:14.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:14.545 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:14.546 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:14.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:14.546 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.7698 192.168.0.1:0/1780164823", 2023-10-26T05:53:14.546 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1780164823", 2023-10-26T05:53:14.546 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:53:14.547 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 44, 2023-10-26T05:53:14.547 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 124, 2023-10-26T05:53:14.547 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:53:14.547 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:53:14.547 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:53:14.548 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:14.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:14.562 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:14.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:53:14.563 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:14.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:14.609 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:53:14.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:14.616 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:14.617 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:14.617 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:14.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:14.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:14.712 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:53:14.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f350032f640 1 -- 172.21.15.71:0/1046938739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4003980 msgr2=0x7f34e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:14.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f350032f640 1 --2- 172.21.15.71:0/1046938739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4003980 0x7f34e4003da0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009b60 tx=0x7f34f401d220 comp rx=0 tx=0).stop 2023-10-26T05:53:14.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f34fe8cc640 1 -- 172.21.15.71:0/1046938739 reap_dead start 2023-10-26T05:53:14.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f350032f640 1 -- 172.21.15.71:0/1046938739 shutdown_connections 2023-10-26T05:53:14.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f350032f640 1 -- 172.21.15.71:0/1046938739 >> 172.21.15.71:0/1046938739 conn(0x7f34f806f440 msgr2=0x7f34f806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:14.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f350032f640 1 -- 172.21.15.71:0/1046938739 shutdown_connections 2023-10-26T05:53:14.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.832+0000 7f350032f640 1 -- 172.21.15.71:0/1046938739 wait complete. 2023-10-26T05:53:14.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 Processor -- start 2023-10-26T05:53:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- start start 2023-10-26T05:53:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e4003980 0x7f34f813c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 0x7f34f80b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34f8140360 0x7f34f81407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:14.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80b16b0 con 0x7f34f80b0cf0 2023-10-26T05:53:14.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8142150 con 0x7f34e4003980 2023-10-26T05:53:14.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f81410b0 con 0x7f34f8140360 2023-10-26T05:53:14.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 0x7f34f80b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:14.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 0x7f34f80b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55618/0 (socket says 172.21.15.71:55618) 2023-10-26T05:53:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 -- 172.21.15.71:0/2005193171 learned_addr learned my addr 172.21.15.71:0/2005193171 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fe8cc640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34f8140360 0x7f34f81407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fe0cb640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e4003980 0x7f34f813c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 -- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34f8140360 msgr2=0x7f34f81407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34f8140360 0x7f34f81407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 -- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e4003980 msgr2=0x7f34f813c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e4003980 0x7f34f813c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 -- 172.21.15.71:0/2005193171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8141390 con 0x7f34f80b0cf0 2023-10-26T05:53:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fe0cb640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e4003980 0x7f34f813c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34fd8ca640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 0x7f34f80b1170 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f34e800e590 tx=0x7f34e800ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8010890 con 0x7f34f80b0cf0 2023-10-26T05:53:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34e8010b00 con 0x7f34f80b0cf0 2023-10-26T05:53:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f8141670 con 0x7f34f80b0cf0 2023-10-26T05:53:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8011460 con 0x7f34f80b0cf0 2023-10-26T05:53:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8141c30 con 0x7f34f80b0cf0 2023-10-26T05:53:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.836+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f8066b10 con 0x7f34f80b0cf0 2023-10-26T05:53:14.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.840+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34e80115c0 con 0x7f34f80b0cf0 2023-10-26T05:53:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.840+0000 7f34eeffd640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34dc066470 0x7f34dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.840+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34e8084a20 con 0x7f34f80b0cf0 2023-10-26T05:53:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.840+0000 7f34fe0cb640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34dc066470 0x7f34dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.840+0000 7f34fe0cb640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34dc066470 0x7f34dc068930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f34f40ae000 tx=0x7f34f4019fc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:14.840+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34e8051f80 con 0x7f34f80b0cf0 2023-10-26T05:53:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.036+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34f8141e20 con 0x7f34f80b0cf0 2023-10-26T05:53:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.036+0000 7f34eeffd640 1 -- 172.21.15.71:0/2005193171 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f34e801f070 con 0x7f34f80b0cf0 2023-10-26T05:53:15.044 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:53:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34dc066470 msgr2=0x7f34dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34dc066470 0x7f34dc068930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f34f40ae000 tx=0x7f34f4019fc0 comp rx=0 tx=0).stop 2023-10-26T05:53:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 msgr2=0x7f34f80b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 0x7f34f80b1170 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f34e800e590 tx=0x7f34e800ea50 comp rx=0 tx=0).stop 2023-10-26T05:53:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 shutdown_connections 2023-10-26T05:53:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f34f8140360 0x7f34f81407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34dc066470 0x7f34dc068930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34f80b0cf0 0x7f34f80b1170 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 --2- 172.21.15.71:0/2005193171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e4003980 0x7f34f813c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 >> 172.21.15.71:0/2005193171 conn(0x7f34f806f440 msgr2=0x7f34f80b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 shutdown_connections 2023-10-26T05:53:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:15.040+0000 7f350032f640 1 -- 172.21.15.71:0/2005193171 wait complete. 2023-10-26T05:53:15.063 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:53:15.064 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:53:15.064 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:53:15.065 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:53:15.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:15.065 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:53:15.066 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:53:15.066 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:53:15.066 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:53:15.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.067 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:53:15.075 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:53:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:53:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:53:15.075 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:53:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.076 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:15.076 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:53:15.077 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:15.077 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:53:15.077 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:15.078 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:53:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:15.078 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:53:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42176sec preferred_lft 42176sec 2023-10-26T05:53:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:53:15.079 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.079 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:53:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:53:15.080 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.080 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:53:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:53:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:53:15.081 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.082 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:53:15.082 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.083 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:53:15.083 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:15.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.084 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:53:15.129 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:53:15.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.131 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:53:15.181 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:53:15.181 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:53:15.181 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:53:15.181 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:53:15.182 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:53:15.182 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:53:15.182 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:15.182 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:15.183 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:15.183 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:53:15.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.184 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:53:15.184 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:53:15.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:53:15.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:53:15.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:53:15.186 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:53:15.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.259 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:53:15.306 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:53:15.306 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:53:15.307 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:53:15.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.307 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:53:15.361 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:53:15.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:15.362 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:15.410 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:53:15.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:15.411 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:15.458 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:15.507 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:53:15.507 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:53:15.507 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:53:15.508 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:53:15.508 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:53:15.508 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:53:15.508 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:53:15.508 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:53:15.509 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:53:15.509 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:53:15.509 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:53:15.509 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:53:15.509 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:15.509 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:53:15.510 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:53:15.510 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:53:15.510 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:53:15.510 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:53:15.510 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:15.510 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:53:15.511 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:15.511 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:53:15.511 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:53:15.511 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:53:15.511 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:53:15.512 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:53:15.512 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:53:15.512 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:53:15.512 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:53:15.512 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:15.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:15.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:15.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:15.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.514 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:53:15.562 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:53:15.563 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:53:15.607 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:15.615 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:53:15.616 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:53:15.616 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:53:15.616 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:53:15.616 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:53:15.616 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:53:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:53:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:53:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:53:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:53:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:53:15.617 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:53:15.618 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:15.618 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:53:15.618 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:53:15.618 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:53:15.618 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:53:15.619 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:53:15.619 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:15.619 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:53:15.619 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:15.619 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:53:15.619 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:53:15.620 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:53:15.620 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:53:15.620 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:53:15.620 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:53:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:53:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:53:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:15.621 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:15.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:15.623 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:53:15.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:53:15.688+0000 7fbbb0af1580 -1 init, newargv = 0x5601a4f113e0 newargc=15 2023-10-26T05:53:15.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34894]: starting ceph client 2023-10-26T05:53:15.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34894]: starting fuse 2023-10-26T05:53:15.746 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:53:15.749 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:53:16.775 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:16.783 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:53:16.783 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:53:16.783 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:53:16.784 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:53:16.784 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:53:16.784 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:53:16.784 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:53:16.784 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:53:16.784 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:53:16.785 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:53:16.785 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:53:16.785 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:53:16.785 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:16.785 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:53:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:53:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:53:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:53:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:53:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:16.786 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:53:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:53:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:53:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:53:16.787 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:53:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:53:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:53:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:53:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:53:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:16.788 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:16.789 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:16.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:16.790 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:53:16.838 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:53:16.839 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:53:16.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:16.839 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:53:16.840 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:53:16.840 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:53:16.841 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:53:16.841 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:16.841 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:53:16.842 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:53:16.842 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:53:16.842 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:53:16.843 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:16.843 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:53:16.843 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:53:16.844 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:53:16.844 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:53:16.845 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:16.845 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:53:16.845 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:16.846 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:16.846 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:16.846 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:53:16.847 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:53:16.847 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:53:16.847 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:16.848 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:16.848 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:53:16.849 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:53:16.963 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.34894.asok 2023-10-26T05:53:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:16.975 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34894.asok 2023-10-26T05:53:16.975 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:16.976 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.34894.asok status 2023-10-26T05:53:17.069 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:17.173 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:53:17.174 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:53:17.174 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:17.175 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:17.175 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:53:17.175 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:53:17.176 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:53:17.176 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "34894", 2023-10-26T05:53:17.176 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:53:17.176 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:17.177 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:53:17.177 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:17.177 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6070, 2023-10-26T05:53:17.177 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:53:17.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:53:17.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:53:17.178 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 6070 2023-10-26T05:53:17.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:17.178 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:53:17.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:53:17.179 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:17.179 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 916779021 2023-10-26T05:53:17.179 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:53:17.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:17.179 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:53:17.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:53:17.180 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:17.180 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 916779021 2023-10-26T05:53:17.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:17.180 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.6070 192.168.0.1:0/916779021", 2023-10-26T05:53:17.181 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/916779021", 2023-10-26T05:53:17.181 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:53:17.181 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 44, 2023-10-26T05:53:17.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 124, 2023-10-26T05:53:17.181 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:53:17.181 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:53:17.182 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:53:17.182 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:53:17.186 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:17.193 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:17.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:53:17.194 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:17.237 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:17.238 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:53:17.239 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:17.245 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:17.245 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:17.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:17.333 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:17.334 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:17.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:17.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- 172.21.15.71:0/4092456935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c06fb80 msgr2=0x7f3d3c15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 --2- 172.21.15.71:0/4092456935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c06fb80 0x7f3d3c15b4b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0ccf60 tx=0x7f3d3001c8c0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- 172.21.15.71:0/4092456935 shutdown_connections 2023-10-26T05:53:17.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 --2- 172.21.15.71:0/4092456935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c06fb80 0x7f3d3c15b4b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 --2- 172.21.15.71:0/4092456935 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d3c0ac740 0x7f3d3c06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- 172.21.15.71:0/4092456935 >> 172.21.15.71:0/4092456935 conn(0x7f3d3c069770 msgr2=0x7f3d3c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:17.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- 172.21.15.71:0/4092456935 shutdown_connections 2023-10-26T05:53:17.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- 172.21.15.71:0/4092456935 wait complete. 2023-10-26T05:53:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 Processor -- start 2023-10-26T05:53:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- start start 2023-10-26T05:53:17.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d3c06fb80 0x7f3d3c14d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c0ac740 0x7f3d3c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 0x7f3d3c13c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c136c60 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c14dcb0 con 0x7f3d3c0ac740 2023-10-26T05:53:17.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d408a0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3c13b440 con 0x7f3d3c06fb80 2023-10-26T05:53:17.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 0x7f3d3c13c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 0x7f3d3c13c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55632/0 (socket says 172.21.15.71:55632) 2023-10-26T05:53:17.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 -- 172.21.15.71:0/1397444701 learned_addr learned my addr 172.21.15.71:0/1397444701 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:17.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d39d9b640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c0ac740 0x7f3d3c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3a59c640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d3c06fb80 0x7f3d3c14d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 -- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d3c06fb80 msgr2=0x7f3d3c14d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d3c06fb80 0x7f3d3c14d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 -- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c0ac740 msgr2=0x7f3d3c0b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c0ac740 0x7f3d3c0b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.472+0000 7f3d3ad9d640 1 -- 172.21.15.71:0/1397444701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d30009560 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d3ad9d640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 0x7f3d3c13c5a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c780 tx=0x7f3d3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:17.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003db0 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d3400ce30 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3c13b720 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340052b0 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3c13bce0 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3c066b10 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3d34005410 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d2affd640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d1c0664a0 0x7f3d1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d34083da0 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d3a59c640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d1c0664a0 0x7f3d1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.476+0000 7f3d3a59c640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d1c0664a0 0x7f3d1c068960 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00ba40 tx=0x7f3d2c005cd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.480+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3d34051220 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d3c0721e0 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d2affd640 1 -- 172.21.15.71:0/1397444701 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f3d34054840 con 0x7f3d3c15b9f0 2023-10-26T05:53:17.674 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:17.675 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:17.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d1c0664a0 msgr2=0x7f3d1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d1c0664a0 0x7f3d1c068960 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00ba40 tx=0x7f3d2c005cd0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 msgr2=0x7f3d3c13c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 0x7f3d3c13c5a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400c780 tx=0x7f3d3400cc40 comp rx=0 tx=0).stop 2023-10-26T05:53:17.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 shutdown_connections 2023-10-26T05:53:17.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d1c0664a0 0x7f3d1c068960 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d3c15b9f0 0x7f3d3c13c5a0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c0ac740 0x7f3d3c0b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 --2- 172.21.15.71:0/1397444701 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d3c06fb80 0x7f3d3c14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 >> 172.21.15.71:0/1397444701 conn(0x7f3d3c069770 msgr2=0x7f3d3c158ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 shutdown_connections 2023-10-26T05:53:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.668+0000 7f3d408a0640 1 -- 172.21.15.71:0/1397444701 wait complete. 2023-10-26T05:53:17.694 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T05:53:17.828 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":7545,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.379996897,"mdsmap_epoch":44,"osdmap_epoch":124,"osdmap_epoch_barrier":124,"uptime":16.206296693999999} 2023-10-26T05:53:17.828 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 7545, "mdsmap_epoch": 44, "osdmap_epoch": 124, "osdmap_epoch_barrier": 124, "rank_uptime": 10.379996897, "state": "up:active", "uptime": 16.206296694, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:53:17.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T05:53:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 -- 172.21.15.71:0/1846132803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1a70 msgr2=0x7fcd6c1419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/1846132803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1a70 0x7fcd6c1419a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0b2310 tx=0x7fcd6801c860 comp rx=0 tx=0).stop 2023-10-26T05:53:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 -- 172.21.15.71:0/1846132803 shutdown_connections 2023-10-26T05:53:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/1846132803 >> v2:172.21.15.86:3300/0 conn(0x7fcd6c0b1e50 0x7fcd6c141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/1846132803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1a70 0x7fcd6c1419a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 -- 172.21.15.71:0/1846132803 >> 172.21.15.71:0/1846132803 conn(0x7fcd6c0ca9a0 msgr2=0x7fcd6c0cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 -- 172.21.15.71:0/1846132803 shutdown_connections 2023-10-26T05:53:17.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 -- 172.21.15.71:0/1846132803 wait complete. 2023-10-26T05:53:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 Processor -- start 2023-10-26T05:53:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.948+0000 7fcd73d4e640 1 -- start start 2023-10-26T05:53:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd6c0b1a70 0x7fcd6c0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 0x7fcd6c13c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd6c0cbe20 0x7fcd6c13c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0c7970 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0c7c50 con 0x7fcd6c0b1a70 2023-10-26T05:53:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c0c7f30 con 0x7fcd6c0cbe20 2023-10-26T05:53:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 0x7fcd6c13c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 0x7fcd6c13c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55654/0 (socket says 172.21.15.71:55654) 2023-10-26T05:53:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 -- 172.21.15.71:0/2440395164 learned_addr learned my addr 172.21.15.71:0/2440395164 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd71aea640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd6c0b1a70 0x7fcd6c0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd722eb640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd6c0cbe20 0x7fcd6c13c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 -- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd6c0cbe20 msgr2=0x7fcd6c13c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd6c0cbe20 0x7fcd6c13c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 -- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd6c0b1a70 msgr2=0x7fcd6c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd6c0b1a70 0x7fcd6c0c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 -- 172.21.15.71:0/2440395164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009560 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd722eb640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd6c0cbe20 0x7fcd6c13c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd712e9640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 0x7fcd6c13c1e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00d980 tx=0x7fcd5c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c009c30 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd5c005020 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c005430 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd6c0c8210 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd6c0c7640 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6c066d00 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.952+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd5c0047b0 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.956+0000 7fcd627fc640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd50066470 0x7fcd50068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.956+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd5c0840d0 con 0x7fcd6c0b1e50 2023-10-26T05:53:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.956+0000 7fcd71aea640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd50066470 0x7fcd50068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.956+0000 7fcd71aea640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd50066470 0x7fcd50068930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd680af000 tx=0x7fcd6801b0a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:17.960+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd5c051550 con 0x7fcd6c0b1e50 2023-10-26T05:53:18.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.100+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fcd6c15a200 con 0x7fcd6c0b1e50 2023-10-26T05:53:18.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.168+0000 7fcd627fc640 1 -- 172.21.15.71:0/2440395164 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcd5c0200c0 con 0x7fcd6c0b1e50 2023-10-26T05:53:18.177 INFO:teuthology.orchestra.run.smithi071.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T05:53:18.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd50066470 msgr2=0x7fcd50068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd50066470 0x7fcd50068930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcd680af000 tx=0x7fcd6801b0a0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 msgr2=0x7fcd6c13c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 0x7fcd6c13c1e0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00d980 tx=0x7fcd5c00de40 comp rx=0 tx=0).stop 2023-10-26T05:53:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 shutdown_connections 2023-10-26T05:53:18.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd6c0cbe20 0x7fcd6c13c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd50066470 0x7fcd50068930 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd6c0b1e50 0x7fcd6c13c1e0 secure :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00d980 tx=0x7fcd5c00de40 comp rx=0 tx=0).stop 2023-10-26T05:53:18.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 --2- 172.21.15.71:0/2440395164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd6c0b1a70 0x7fcd6c0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 >> 172.21.15.71:0/2440395164 conn(0x7fcd6c0ca9a0 msgr2=0x7fcd6c0cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:18.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 shutdown_connections 2023-10-26T05:53:18.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.172+0000 7fcd73d4e640 1 -- 172.21.15.71:0/2440395164 wait complete. 2023-10-26T05:53:18.198 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:53:18.198 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:18.198 DEBUG:teuthology.orchestra.run.smithi071:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T05:53:18.199 DEBUG:teuthology.orchestra.run.smithi071:> os.fchmod(fd, 0O666) 2023-10-26T05:53:18.199 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(fd) 2023-10-26T05:53:18.199 DEBUG:teuthology.orchestra.run.smithi071:> os.close(fd) 2023-10-26T05:53:18.200 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:18.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:18.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:18.277 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:53:18.277 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:53:18.278 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:18.278 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:53:18.278 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:18.279 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:53:18.279 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:53:18.279 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:53:18.280 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:53:18.280 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:18.280 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:53:18.281 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:53:18.281 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:53:18.281 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:53:18.282 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:18.282 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:53:18.283 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:18.283 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:18.283 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:18.283 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:53:18.284 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:53:18.284 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:53:18.284 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:18.285 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:18.285 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:53:18.285 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:18.394 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:18.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:18.406 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:18.406 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:18.407 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.47484.asok status 2023-10-26T05:53:18.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:18.593 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:53:18.594 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:18.594 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:18.594 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:53:18.594 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:53:18.594 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:53:18.595 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "47484", 2023-10-26T05:53:18.595 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:53:18.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:18.595 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 1, 2023-10-26T05:53:18.595 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:18.595 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7698, 2023-10-26T05:53:18.596 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:53:18.596 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:53:18.596 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:53:18.596 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 7698 2023-10-26T05:53:18.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:18.597 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:18.597 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:18.597 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:18.597 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:18.597 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:18.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:18.598 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:18.598 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:18.598 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:18.598 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:18.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:18.599 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.7698 192.168.0.1:0/1780164823", 2023-10-26T05:53:18.599 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1780164823", 2023-10-26T05:53:18.599 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 2, 2023-10-26T05:53:18.599 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 44, 2023-10-26T05:53:18.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 124, 2023-10-26T05:53:18.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 124, 2023-10-26T05:53:18.600 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:53:18.600 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:53:18.600 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:18.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:18.614 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-10-26T05:53:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 -- 172.21.15.71:0/2017054153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a40159700 msgr2=0x7f6a400b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/2017054153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a40159700 0x7f6a400b1300 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009900 tx=0x7f6a2801c7b0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 -- 172.21.15.71:0/2017054153 shutdown_connections 2023-10-26T05:53:18.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/2017054153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a40159700 0x7f6a400b1300 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 -- 172.21.15.71:0/2017054153 >> 172.21.15.71:0/2017054153 conn(0x7f6a4006b8c0 msgr2=0x7f6a4006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:18.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 -- 172.21.15.71:0/2017054153 shutdown_connections 2023-10-26T05:53:18.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.780+0000 7f6a44d9f640 1 -- 172.21.15.71:0/2017054153 wait complete. 2023-10-26T05:53:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 Processor -- start 2023-10-26T05:53:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 -- start start 2023-10-26T05:53:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a40069d30 0x7f6a4014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 0x7f6a40135a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:18.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a40159700 0x7f6a400ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40136e30 con 0x7f6a400ca7f0 2023-10-26T05:53:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40135f80 con 0x7f6a40159700 2023-10-26T05:53:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401361b0 con 0x7f6a40069d30 2023-10-26T05:53:18.794 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 0x7f6a40135a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 0x7f6a40135a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55662/0 (socket says 172.21.15.71:55662) 2023-10-26T05:53:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 -- 172.21.15.71:0/3545891084 learned_addr learned my addr 172.21.15.71:0/3545891084 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3ed9d640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a40159700 0x7f6a400ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3e59c640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a40069d30 0x7f6a4014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 -- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a40069d30 msgr2=0x7f6a4014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a40069d30 0x7f6a4014eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 -- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a40159700 msgr2=0x7f6a400ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a40159700 0x7f6a400ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 -- 172.21.15.71:0/3545891084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a28009560 con 0x7f6a400ca7f0 2023-10-26T05:53:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3ed9d640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a40159700 0x7f6a400ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:18.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3e59c640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a40069d30 0x7f6a4014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a3dd9b640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 0x7f6a40135a40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f6a280b0040 tx=0x7f6a280a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28003800 con 0x7f6a400ca7f0 2023-10-26T05:53:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a28003960 con 0x7f6a400ca7f0 2023-10-26T05:53:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28003d30 con 0x7f6a400ca7f0 2023-10-26T05:53:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a40136490 con 0x7f6a400ca7f0 2023-10-26T05:53:18.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.784+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a40136a50 con 0x7f6a400ca7f0 2023-10-26T05:53:18.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.788+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a40066b10 con 0x7f6a400ca7f0 2023-10-26T05:53:18.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.788+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6a280b4070 con 0x7f6a400ca7f0 2023-10-26T05:53:18.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.788+0000 7f6a36ffd640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a20066470 0x7f6a20068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:18.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.788+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a2811e850 con 0x7f6a400ca7f0 2023-10-26T05:53:18.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.788+0000 7f6a3e59c640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a20066470 0x7f6a20068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:18.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.788+0000 7f6a3e59c640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a20066470 0x7f6a20068930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000f70 tx=0x7f6a30009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:18.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.792+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6a280ebe20 con 0x7f6a400ca7f0 2023-10-26T05:53:18.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:18.944+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f6a40136f70 con 0x7f6a400ca7f0 2023-10-26T05:53:19.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.172+0000 7f6a36ffd640 1 -- 172.21.15.71:0/3545891084 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v126) v1 ==== 93+0+0 (secure 0 0 0) 0x7f6a40136f70 con 0x7f6a400ca7f0 2023-10-26T05:53:19.180 INFO:teuthology.orchestra.run.smithi071.stderr:pauserd,pausewr is set 2023-10-26T05:53:19.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a20066470 msgr2=0x7f6a20068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:19.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a20066470 0x7f6a20068930 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6a30000f70 tx=0x7f6a30009040 comp rx=0 tx=0).stop 2023-10-26T05:53:19.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 msgr2=0x7f6a40135a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:19.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 0x7f6a40135a40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f6a280b0040 tx=0x7f6a280a56d0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 shutdown_connections 2023-10-26T05:53:19.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a40159700 0x7f6a400ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a20066470 0x7f6a20068930 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a400ca7f0 0x7f6a40135a40 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 --2- 172.21.15.71:0/3545891084 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a40069d30 0x7f6a4014eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 >> 172.21.15.71:0/3545891084 conn(0x7f6a4006b8c0 msgr2=0x7f6a40159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:19.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 shutdown_connections 2023-10-26T05:53:19.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.176+0000 7f6a44d9f640 1 -- 172.21.15.71:0/3545891084 wait complete. 2023-10-26T05:53:19.201 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-10-26T05:53:19.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/1731782104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f8815bd30 msgr2=0x7f0f880cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:19.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 --2- 172.21.15.71:0/1731782104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f8815bd30 0x7f0f880cef00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009bc0 tx=0x7f0f8001cb50 comp rx=0 tx=0).stop 2023-10-26T05:53:19.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/1731782104 shutdown_connections 2023-10-26T05:53:19.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 --2- 172.21.15.71:0/1731782104 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f8815bd30 0x7f0f880cef00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 --2- 172.21.15.71:0/1731782104 >> v2:172.21.15.86:3300/0 conn(0x7f0f88069980 0x7f0f8815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/1731782104 >> 172.21.15.71:0/1731782104 conn(0x7f0f880d1330 msgr2=0x7f0f880d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/1731782104 shutdown_connections 2023-10-26T05:53:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/1731782104 wait complete. 2023-10-26T05:53:19.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 Processor -- start 2023-10-26T05:53:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- start start 2023-10-26T05:53:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f88069980 0x7f0f880c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f8815b410 0x7f0f8814e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 0x7f0f88151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8814eef0 con 0x7f0f8815bd30 2023-10-26T05:53:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88150920 con 0x7f0f8815b410 2023-10-26T05:53:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8814fd10 con 0x7f0f88069980 2023-10-26T05:53:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 0x7f0f88151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 0x7f0f88151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55666/0 (socket says 172.21.15.71:55666) 2023-10-26T05:53:19.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 -- 172.21.15.71:0/232914619 learned_addr learned my addr 172.21.15.71:0/232914619 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:19.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8d31e640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f88069980 0x7f0f880c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:19.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8cb1d640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f8815b410 0x7f0f8814e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:19.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 -- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f88069980 msgr2=0x7f0f880c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:19.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f88069980 0x7f0f880c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 -- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f8815b410 msgr2=0x7f0f8814e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f8815b410 0x7f0f8814e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80009820 con 0x7f0f8815bd30 2023-10-26T05:53:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8db1f640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 0x7f0f88151e20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009b10 tx=0x7f0f800a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:19.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80003b80 con 0x7f0f8815bd30 2023-10-26T05:53:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f800a5c10 con 0x7f0f8815bd30 2023-10-26T05:53:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8814fff0 con 0x7f0f8815bd30 2023-10-26T05:53:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80004030 con 0x7f0f8815bd30 2023-10-26T05:53:19.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.320+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f881505b0 con 0x7f0f8815bd30 2023-10-26T05:53:19.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f88066b10 con 0x7f0f8815bd30 2023-10-26T05:53:19.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f800b4040 con 0x7f0f8815bd30 2023-10-26T05:53:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f7dffb640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f6c066470 0x7f0f6c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f8011eb20 con 0x7f0f8815bd30 2023-10-26T05:53:19.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=127+}) v3 -- 0x7f0f6c06dfa0 con 0x7f0f8815bd30 2023-10-26T05:53:19.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f8d31e640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f6c066470 0x7f0f6c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:19.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.324+0000 7f0f8d31e640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f6c066470 0x7f0f6c068930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0f880c7e60 tx=0x7f0f84009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:19.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.328+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f800b2020 con 0x7f0f8815bd30 2023-10-26T05:53:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:19.480+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f0f88150a90 con 0x7f0f8815bd30 2023-10-26T05:53:20.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.188+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v127) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0f800ec0c0 con 0x7f0f8815bd30 2023-10-26T05:53:20.196 INFO:teuthology.orchestra.run.smithi071.stderr:pauserd,pausewr is unset 2023-10-26T05:53:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 <== mon.0 v2:172.21.15.71:3300/0 8 ==== osd_map(127..127 src has 1..127) v4 ==== 288+0+0 (secure 0 0 0) 0x7f0f800f0480 con 0x7f0f8815bd30 2023-10-26T05:53:20.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f7dffb640 1 -- 172.21.15.71:0/232914619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=128}) v3 -- 0x7f0f6c06e7c0 con 0x7f0f8815bd30 2023-10-26T05:53:20.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f6c066470 msgr2=0x7f0f6c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f6c066470 0x7f0f6c068930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0f880c7e60 tx=0x7f0f84009040 comp rx=0 tx=0).stop 2023-10-26T05:53:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 msgr2=0x7f0f88151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 0x7f0f88151e20 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009b10 tx=0x7f0f800a5a20 comp rx=0 tx=0).stop 2023-10-26T05:53:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 shutdown_connections 2023-10-26T05:53:20.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f6c066470 0x7f0f6c068930 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f8815bd30 0x7f0f88151e20 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f8815b410 0x7f0f8814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 --2- 172.21.15.71:0/232914619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f88069980 0x7f0f880c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.192+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 >> 172.21.15.71:0/232914619 conn(0x7f0f880d1330 msgr2=0x7f0f880b5920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.196+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 shutdown_connections 2023-10-26T05:53:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.196+0000 7f0f8f582640 1 -- 172.21.15.71:0/232914619 wait complete. 2023-10-26T05:53:20.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:20.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/312203119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 msgr2=0x7f8c6c15aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 --2- 172.21.15.71:0/312203119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c15aa20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c072f50 tx=0x7f8c6401caa0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/312203119 shutdown_connections 2023-10-26T05:53:20.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 --2- 172.21.15.71:0/312203119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c15aa20 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 --2- 172.21.15.71:0/312203119 >> v2:172.21.15.86:3300/0 conn(0x7f8c6c0cede0 0x7f8c6c15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/312203119 >> 172.21.15.71:0/312203119 conn(0x7f8c6c0b0f20 msgr2=0x7f8c6c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:20.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/312203119 shutdown_connections 2023-10-26T05:53:20.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/312203119 wait complete. 2023-10-26T05:53:20.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 Processor -- start 2023-10-26T05:53:20.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- start start 2023-10-26T05:53:20.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c6c0cede0 0x7f8c6c146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:20.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6c15c720 0x7f8c6c143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:20.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c14f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:20.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c14f5d0 con 0x7f8c6c15d390 2023-10-26T05:53:20.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c06b240 con 0x7f8c6c15c720 2023-10-26T05:53:20.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c1500a0 con 0x7f8c6c0cede0 2023-10-26T05:53:20.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c14f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:20.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c14f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55678/0 (socket says 172.21.15.71:55678) 2023-10-26T05:53:20.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 -- 172.21.15.71:0/1693628807 learned_addr learned my addr 172.21.15.71:0/1693628807 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:20.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6a7fc640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6c15c720 0x7f8c6c143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:20.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6affd640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c6c0cede0 0x7f8c6c146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:20.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 -- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c6c0cede0 msgr2=0x7f8c6c146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c6c0cede0 0x7f8c6c146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 -- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6c15c720 msgr2=0x7f8c6c143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6c15c720 0x7f8c6c143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 -- 172.21.15.71:0/1693628807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64009820 con 0x7f8c6c15d390 2023-10-26T05:53:20.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6affd640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c6c0cede0 0x7f8c6c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:20.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c6b7fe640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c14f090 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009b10 tx=0x7f8c64009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:20.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64004040 con 0x7f8c6c15d390 2023-10-26T05:53:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8c64004290 con 0x7f8c6c15d390 2023-10-26T05:53:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c14f800 con 0x7f8c6c15d390 2023-10-26T05:53:20.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64004680 con 0x7f8c6c15d390 2023-10-26T05:53:20.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.348+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c14fdc0 con 0x7f8c6c15d390 2023-10-26T05:53:20.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.352+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c06b3f0 con 0x7f8c6c15d390 2023-10-26T05:53:20.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.352+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8c640b7440 con 0x7f8c6c15d390 2023-10-26T05:53:20.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.352+0000 7f8c637fe640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8c4c066470 0x7f8c4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:20.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.352+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c6411efb0 con 0x7f8c6c15d390 2023-10-26T05:53:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.352+0000 7f8c6affd640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8c4c066470 0x7f8c4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:20.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.352+0000 7f8c6affd640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8c4c066470 0x7f8c4c068930 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c15d950 tx=0x7f8c5c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:20.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.356+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8c64150090 con 0x7f8c6c15d390 2023-10-26T05:53:20.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.512+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c6c066d00 con 0x7f8c6c15d390 2023-10-26T05:53:20.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.512+0000 7f8c637fe640 1 -- 172.21.15.71:0/1693628807 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f8c6c06b3f0 con 0x7f8c6c15d390 2023-10-26T05:53:20.521 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:20.522 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":127,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:20.189455+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8c4c066470 msgr2=0x7f8c4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8c4c066470 0x7f8c4c068930 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c15d950 tx=0x7f8c5c009040 comp rx=0 tx=0).stop 2023-10-26T05:53:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 msgr2=0x7f8c6c14f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c14f090 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009b10 tx=0x7f8c64009280 comp rx=0 tx=0).stop 2023-10-26T05:53:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 shutdown_connections 2023-10-26T05:53:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8c4c066470 0x7f8c4c068930 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c6c15d390 0x7f8c6c14f090 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c6c15c720 0x7f8c6c143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 --2- 172.21.15.71:0/1693628807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8c6c0cede0 0x7f8c6c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:20.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 >> 172.21.15.71:0/1693628807 conn(0x7f8c6c0b0f20 msgr2=0x7f8c6c0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:20.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 shutdown_connections 2023-10-26T05:53:20.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20.516+0000 7f8c71381640 1 -- 172.21.15.71:0/1693628807 wait complete. 2023-10-26T05:53:20.540 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:53:20.540 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:20.541 DEBUG:teuthology.orchestra.run.smithi071:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T05:53:20.541 DEBUG:teuthology.orchestra.run.smithi071:> os.fchmod(fd, 0O666) 2023-10-26T05:53:20.542 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(fd) 2023-10-26T05:53:20.542 DEBUG:teuthology.orchestra.run.smithi071:> os.close(fd) 2023-10-26T05:53:20.542 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:20.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:20.617 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:20.617 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:53:20.618 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:53:20.618 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:20.619 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:53:20.619 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:20.619 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:53:20.620 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:53:20.620 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:53:20.620 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:53:20.621 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:20.621 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:53:20.621 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:53:20.622 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:53:20.622 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:53:20.622 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:20.623 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:53:20.623 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:20.623 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:20.624 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:20.624 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:53:20.625 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:53:20.625 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:53:20.625 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:20.626 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:20.626 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:53:20.627 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:20.742 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:20.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:20.754 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:20.755 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:20.755 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.47484.asok status 2023-10-26T05:53:20.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:20.949 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:20.949 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:53:20.949 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:20.950 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:20.950 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:53:20.950 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:53:20.950 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:53:20.951 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "47484", 2023-10-26T05:53:20.951 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:53:20.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:20.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 1, 2023-10-26T05:53:20.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:20.952 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7698, 2023-10-26T05:53:20.952 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:53:20.952 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:53:20.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:53:20.952 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 7698 2023-10-26T05:53:20.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:20.953 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:20.953 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:20.954 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:20.954 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:20.955 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:20.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:20.955 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:20.955 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:20.955 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:20.956 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:20.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:20.956 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.7698 192.168.0.1:0/1780164823", 2023-10-26T05:53:20.956 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1780164823", 2023-10-26T05:53:20.956 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 2, 2023-10-26T05:53:20.957 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 44, 2023-10-26T05:53:20.957 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 124, 2023-10-26T05:53:20.957 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 124, 2023-10-26T05:53:20.957 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:53:20.957 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:53:20.958 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:20.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:20.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/3291437820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d815b9f0 msgr2=0x7f49d8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/3291437820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d815b9f0 0x7f49d8070690 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f49d8158ed0 tx=0x7f49d001cd20 comp rx=0 tx=0).stop 2023-10-26T05:53:21.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/3291437820 shutdown_connections 2023-10-26T05:53:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/3291437820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d815b9f0 0x7f49d8070690 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/3291437820 >> v2:172.21.15.86:3300/0 conn(0x7f49d80ac740 0x7f49d815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/3291437820 >> 172.21.15.71:0/3291437820 conn(0x7f49d8069770 msgr2=0x7f49d8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/3291437820 shutdown_connections 2023-10-26T05:53:21.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/3291437820 wait complete. 2023-10-26T05:53:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 Processor -- start 2023-10-26T05:53:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- start start 2023-10-26T05:53:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 0x7f49d813b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49d813d420 0x7f49d8139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:21.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d815b9f0 0x7f49d814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8137c00 con 0x7f49d80ac740 2023-10-26T05:53:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8154fc0 con 0x7f49d815b9f0 2023-10-26T05:53:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d81552a0 con 0x7f49d813d420 2023-10-26T05:53:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 0x7f49d813b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:21.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 0x7f49d813b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55702/0 (socket says 172.21.15.71:55702) 2023-10-26T05:53:21.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 -- 172.21.15.71:0/880034999 learned_addr learned my addr 172.21.15.71:0/880034999 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:21.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d77fe640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d815b9f0 0x7f49d814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:21.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d67fc640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49d813d420 0x7f49d8139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:21.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 -- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49d813d420 msgr2=0x7f49d8139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:21.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49d813d420 0x7f49d8139460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 -- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d815b9f0 msgr2=0x7f49d814dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d815b9f0 0x7f49d814dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 -- 172.21.15.71:0/880034999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d00098e0 con 0x7f49d80ac740 2023-10-26T05:53:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49d6ffd640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 0x7f49d813b0f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c450 tx=0x7f49c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8012830 con 0x7f49d80ac740 2023-10-26T05:53:21.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f49c8012aa0 con 0x7f49d80ac740 2023-10-26T05:53:21.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8155580 con 0x7f49d80ac740 2023-10-26T05:53:21.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8011380 con 0x7f49d80ac740 2023-10-26T05:53:21.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.084+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d8154be0 con 0x7f49d80ac740 2023-10-26T05:53:21.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.088+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8066b10 con 0x7f49d80ac740 2023-10-26T05:53:21.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.088+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49c80114e0 con 0x7f49d80ac740 2023-10-26T05:53:21.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.088+0000 7f49cf7fe640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49b8066470 0x7f49b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:21.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.088+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49c8083f70 con 0x7f49d80ac740 2023-10-26T05:53:21.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.088+0000 7f49d67fc640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49b8066470 0x7f49b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:21.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.088+0000 7f49d67fc640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49b8066470 0x7f49b8068930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f49d813d190 tx=0x7f49c0005cd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:21.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.092+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49c80b5090 con 0x7f49d80ac740 2023-10-26T05:53:21.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.280+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49d80cca40 con 0x7f49d80ac740 2023-10-26T05:53:21.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.280+0000 7f49cf7fe640 1 -- 172.21.15.71:0/880034999 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f49c80513f0 con 0x7f49d80ac740 2023-10-26T05:53:21.287 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:21.288 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:21.288 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:21.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.280+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49b8066470 msgr2=0x7f49b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:21.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.280+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49b8066470 0x7f49b8068930 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f49d813d190 tx=0x7f49c0005cd0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.280+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 msgr2=0x7f49d813b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.280+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 0x7f49d813b0f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c450 tx=0x7f49c800c910 comp rx=0 tx=0).stop 2023-10-26T05:53:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 shutdown_connections 2023-10-26T05:53:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49d815b9f0 0x7f49d814dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49d813d420 0x7f49d8139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49b8066470 0x7f49b8068930 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 --2- 172.21.15.71:0/880034999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49d80ac740 0x7f49d813b0f0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 >> 172.21.15.71:0/880034999 conn(0x7f49d8069770 msgr2=0x7f49d813d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 shutdown_connections 2023-10-26T05:53:21.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21.284+0000 7f49dd3ee640 1 -- 172.21.15.71:0/880034999 wait complete. 2023-10-26T05:53:21.305 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 127 2023-10-26T05:53:21.437 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-10-26T05:53:21.438 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:53:21.438 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:21.439 DEBUG:teuthology.orchestra.run.smithi071:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-10-26T05:53:21.439 DEBUG:teuthology.orchestra.run.smithi071:> os.fchmod(fd, 0O666) 2023-10-26T05:53:21.439 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(fd) 2023-10-26T05:53:21.440 DEBUG:teuthology.orchestra.run.smithi071:> os.close(fd) 2023-10-26T05:53:21.440 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:21.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:21.516 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:21.516 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:53:21.516 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:53:21.517 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:21.517 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:53:21.517 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.518 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:53:21.518 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:53:21.518 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:53:21.519 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:53:21.519 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.519 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:53:21.520 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:53:21.520 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:53:21.520 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:53:21.521 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.521 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:53:21.521 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:21.522 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:21.522 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:21.523 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:53:21.523 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:53:21.523 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:53:21.524 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:21.524 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.524 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:53:21.525 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:21.637 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:21.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:21.648 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:21.648 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:21.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.47484.asok status 2023-10-26T05:53:21.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:21.835 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:21.835 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:53:21.836 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:21.836 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:21.836 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:53:21.837 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:53:21.837 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:53:21.837 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "47484", 2023-10-26T05:53:21.837 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:53:21.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:21.838 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 2, 2023-10-26T05:53:21.838 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:21.839 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7698, 2023-10-26T05:53:21.839 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:53:21.839 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:53:21.840 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:53:21.840 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 7698 2023-10-26T05:53:21.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:21.841 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:21.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:21.842 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:21.842 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:21.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:21.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:21.843 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:21.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:21.843 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:21.843 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:21.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:21.844 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.7698 192.168.0.1:0/1780164823", 2023-10-26T05:53:21.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1780164823", 2023-10-26T05:53:21.844 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 3, 2023-10-26T05:53:21.844 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 44, 2023-10-26T05:53:21.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 127, 2023-10-26T05:53:21.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 127, 2023-10-26T05:53:21.845 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:53:21.845 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:53:21.845 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:21.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:21.856 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:21.856 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:53:21.856 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:53:21.857 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:21.857 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:53:21.857 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.858 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:53:21.858 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:53:21.859 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:53:21.859 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:53:21.860 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.861 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:53:21.861 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:53:21.862 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:53:21.862 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:53:21.863 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.863 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:53:21.864 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:21.864 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:21.865 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:21.865 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:53:21.866 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:53:21.866 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:53:21.867 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:21.867 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:21.867 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:53:21.868 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:21.969 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:21.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:21.980 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47484.asok 2023-10-26T05:53:21.980 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:21.981 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.47484.asok status 2023-10-26T05:53:22.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:22.169 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:53:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:22.170 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:53:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:53:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:53:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "47484", 2023-10-26T05:53:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:53:22.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 2, 2023-10-26T05:53:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 7698, 2023-10-26T05:53:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:53:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:53:22.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:53:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 7698 2023-10-26T05:53:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:22.173 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:22.174 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1780164823 2023-10-26T05:53:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.7698 192.168.0.1:0/1780164823", 2023-10-26T05:53:22.175 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1780164823", 2023-10-26T05:53:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 3, 2023-10-26T05:53:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 44, 2023-10-26T05:53:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 127, 2023-10-26T05:53:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 127, 2023-10-26T05:53:22.176 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:53:22.177 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:53:22.177 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:22.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:22.190 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:53:22.190 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:53:22.191 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:53:22.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:53:22.271 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:53:22.272 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:22.272 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:22.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:22.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/3589246446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d420 msgr2=0x7f793c0ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 --2- 172.21.15.71:0/3589246446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d420 0x7f793c0ac740 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f792c00a950 tx=0x7f792c01db50 comp rx=0 tx=0).stop 2023-10-26T05:53:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/3589246446 shutdown_connections 2023-10-26T05:53:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 --2- 172.21.15.71:0/3589246446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c13d800 0x7f793c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 --2- 172.21.15.71:0/3589246446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d420 0x7f793c0ac740 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/3589246446 >> 172.21.15.71:0/3589246446 conn(0x7f793c069770 msgr2=0x7f793c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:22.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/3589246446 shutdown_connections 2023-10-26T05:53:22.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/3589246446 wait complete. 2023-10-26T05:53:22.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 Processor -- start 2023-10-26T05:53:22.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- start start 2023-10-26T05:53:22.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c13d420 0x7f793c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 0x7f793c1eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c13dc20 0x7f793c1eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c14f310 con 0x7f793c13d800 2023-10-26T05:53:22.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c14f5f0 con 0x7f793c13dc20 2023-10-26T05:53:22.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c14f8d0 con 0x7f793c13d420 2023-10-26T05:53:22.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 0x7f793c1eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 0x7f793c1eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37140/0 (socket says 172.21.15.71:37140) 2023-10-26T05:53:22.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 -- 172.21.15.71:0/2148325395 learned_addr learned my addr 172.21.15.71:0/2148325395 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:22.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793b7fe640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c13dc20 0x7f793c1eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793affd640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c13d420 0x7f793c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 -- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c13d420 msgr2=0x7f793c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c13d420 0x7f793c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 -- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c13dc20 msgr2=0x7f793c1eafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c13dc20 0x7f793c1eafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 -- 172.21.15.71:0/2148325395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c00a5b0 con 0x7f793c13d800 2023-10-26T05:53:22.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f793a7fc640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 0x7f793c1eaa70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f793000d980 tx=0x7f793000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:22.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930009c30 con 0x7f793c13d800 2023-10-26T05:53:22.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7930005020 con 0x7f793c13d800 2023-10-26T05:53:22.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c14fbb0 con 0x7f793c13d800 2023-10-26T05:53:22.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930005430 con 0x7f793c13d800 2023-10-26T05:53:22.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c135610 con 0x7f793c13d800 2023-10-26T05:53:22.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.444+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c066b10 con 0x7f793c13d800 2023-10-26T05:53:22.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.448+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f79300047b0 con 0x7f793c13d800 2023-10-26T05:53:22.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.448+0000 7f79237fe640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7928066470 0x7f7928068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.448+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79300840d0 con 0x7f793c13d800 2023-10-26T05:53:22.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.448+0000 7f793affd640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7928066470 0x7f7928068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.448+0000 7f793affd640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7928066470 0x7f7928068930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f792c0b0000 tx=0x7f792c01c390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:22.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.452+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7930051550 con 0x7f793c13d800 2023-10-26T05:53:22.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f793c139900 con 0x7f793c13d800 2023-10-26T05:53:22.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f79237fe640 1 -- 172.21.15.71:0/2148325395 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f793c139900 con 0x7f793c13d800 2023-10-26T05:53:22.615 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:22.615 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":127,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:20.189455+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7928066470 msgr2=0x7f7928068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7928066470 0x7f7928068930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f792c0b0000 tx=0x7f792c01c390 comp rx=0 tx=0).stop 2023-10-26T05:53:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 msgr2=0x7f793c1eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 0x7f793c1eaa70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f793000d980 tx=0x7f793000de40 comp rx=0 tx=0).stop 2023-10-26T05:53:22.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 shutdown_connections 2023-10-26T05:53:22.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c13dc20 0x7f793c1eafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7928066470 0x7f7928068930 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c13d800 0x7f793c1eaa70 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 --2- 172.21.15.71:0/2148325395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c13d420 0x7f793c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.608+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 >> 172.21.15.71:0/2148325395 conn(0x7f793c069770 msgr2=0x7f793c069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:22.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.612+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 shutdown_connections 2023-10-26T05:53:22.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.612+0000 7f7941422640 1 -- 172.21.15.71:0/2148325395 wait complete. 2023-10-26T05:53:22.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:53:04.827214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:53:05.732853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:53:22.635 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:53:22.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 -- 172.21.15.71:0/3818313307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 msgr2=0x7fe2f406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/3818313307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f406f540 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe2f406da30 tx=0x7fe2dc01c5b0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 -- 172.21.15.71:0/3818313307 shutdown_connections 2023-10-26T05:53:22.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/3818313307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 0x7fe2f40b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/3818313307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f406f540 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/3818313307 >> v2:172.21.15.86:3300/0 conn(0x7fe2f415bb90 0x7fe2f40b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 -- 172.21.15.71:0/3818313307 >> 172.21.15.71:0/3818313307 conn(0x7fe2f40aec60 msgr2=0x7fe2f40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:22.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 -- 172.21.15.71:0/3818313307 shutdown_connections 2023-10-26T05:53:22.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.748+0000 7fe2f95fb640 1 -- 172.21.15.71:0/3818313307 wait complete. 2023-10-26T05:53:22.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 Processor -- start 2023-10-26T05:53:22.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- start start 2023-10-26T05:53:22.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f4133e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2f415bb90 0x7fe2f413bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 0x7fe2f41359d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f41343b0 con 0x7fe2f4070dc0 2023-10-26T05:53:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f413c7c0 con 0x7fe2f415bb90 2023-10-26T05:53:22.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f413caa0 con 0x7fe2f415bf70 2023-10-26T05:53:22.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f4133e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f4133e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37166/0 (socket says 172.21.15.71:37166) 2023-10-26T05:53:22.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 -- 172.21.15.71:0/1095332541 learned_addr learned my addr 172.21.15.71:0/1095332541 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:22.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2ebfff640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2f415bb90 0x7fe2f413bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f37fe640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 0x7fe2f41359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 -- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 msgr2=0x7fe2f41359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 0x7fe2f41359d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 -- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2f415bb90 msgr2=0x7fe2f413bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2f415bb90 0x7fe2f413bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 -- 172.21.15.71:0/1095332541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc009560 con 0x7fe2f4070dc0 2023-10-26T05:53:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f37fe640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 0x7fe2f41359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f2ffd640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f4133e70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400d950 tx=0x7fe2e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:22.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4009c30 con 0x7fe2f4070dc0 2023-10-26T05:53:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe2e4009e80 con 0x7fe2f4070dc0 2023-10-26T05:53:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f413cd80 con 0x7fe2f4070dc0 2023-10-26T05:53:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40106c0 con 0x7fe2f4070dc0 2023-10-26T05:53:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f413d340 con 0x7fe2f4070dc0 2023-10-26T05:53:22.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.752+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f4066b10 con 0x7fe2f4070dc0 2023-10-26T05:53:22.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.756+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe2e4010820 con 0x7fe2f4070dc0 2023-10-26T05:53:22.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.756+0000 7fe2eb7fe640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2d4066470 0x7fe2d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:22.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.756+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2e4084450 con 0x7fe2f4070dc0 2023-10-26T05:53:22.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.756+0000 7fe2ebfff640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2d4066470 0x7fe2d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:22.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.756+0000 7fe2ebfff640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2d4066470 0x7fe2d4068930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0ae000 tx=0x7fe2dc0b3040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:22.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.760+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe2e40552d0 con 0x7fe2f4070dc0 2023-10-26T05:53:22.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2f40721e0 con 0x7fe2f4070dc0 2023-10-26T05:53:22.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2eb7fe640 1 -- 172.21.15.71:0/1095332541 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2e40200a0 con 0x7fe2f4070dc0 2023-10-26T05:53:22.911 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:53:22.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2d4066470 msgr2=0x7fe2d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2d4066470 0x7fe2d4068930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0ae000 tx=0x7fe2dc0b3040 comp rx=0 tx=0).stop 2023-10-26T05:53:22.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 msgr2=0x7fe2f4133e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:22.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f4133e70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe2e400d950 tx=0x7fe2e400de10 comp rx=0 tx=0).stop 2023-10-26T05:53:22.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 shutdown_connections 2023-10-26T05:53:22.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2f415bf70 0x7fe2f41359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2f415bb90 0x7fe2f413bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2d4066470 0x7fe2d4068930 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 --2- 172.21.15.71:0/1095332541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2f4070dc0 0x7fe2f4133e70 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:22.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 >> 172.21.15.71:0/1095332541 conn(0x7fe2f40aec60 msgr2=0x7fe2f40cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:22.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 shutdown_connections 2023-10-26T05:53:22.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:22.904+0000 7fe2f95fb640 1 -- 172.21.15.71:0/1095332541 wait complete. 2023-10-26T05:53:22.927 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:53:23.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 -- 172.21.15.71:0/26276549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40d0470 msgr2=0x7fade406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 --2- 172.21.15.71:0/26276549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40d0470 0x7fade406aa70 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009c80 tx=0x7fadd401cc80 comp rx=0 tx=0).stop 2023-10-26T05:53:23.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 -- 172.21.15.71:0/26276549 shutdown_connections 2023-10-26T05:53:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 --2- 172.21.15.71:0/26276549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40d0470 0x7fade406aa70 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 -- 172.21.15.71:0/26276549 >> 172.21.15.71:0/26276549 conn(0x7fade40cdd80 msgr2=0x7fade40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 -- 172.21.15.71:0/26276549 shutdown_connections 2023-10-26T05:53:23.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.032+0000 7fade8a65640 1 -- 172.21.15.71:0/26276549 wait complete. 2023-10-26T05:53:23.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 Processor -- start 2023-10-26T05:53:23.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- start start 2023-10-26T05:53:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 0x7fade41522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade40d0470 0x7fade40c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fade40d0850 0x7fade414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade406a650 con 0x7fade40cd880 2023-10-26T05:53:23.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c7780 con 0x7fade40d0470 2023-10-26T05:53:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c6b00 con 0x7fade40d0850 2023-10-26T05:53:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 0x7fade41522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 0x7fade41522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37186/0 (socket says 172.21.15.71:37186) 2023-10-26T05:53:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 -- 172.21.15.71:0/3211115058 learned_addr learned my addr 172.21.15.71:0/3211115058 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:23.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 -- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fade40d0850 msgr2=0x7fade414f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade1d9b640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade40d0470 0x7fade40c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade2d9d640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fade40d0850 0x7fade414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fade40d0850 0x7fade414f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 -- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade40d0470 msgr2=0x7fade40c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade40d0470 0x7fade40c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 -- 172.21.15.71:0/3211115058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40098e0 con 0x7fade40cd880 2023-10-26T05:53:23.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade259c640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 0x7fade41522f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fadd401a8b0 tx=0x7fadd40a5be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:23.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40bb070 con 0x7fade40cd880 2023-10-26T05:53:23.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fadd40ad8b0 con 0x7fade40cd880 2023-10-26T05:53:23.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade40c6de0 con 0x7fade40cd880 2023-10-26T05:53:23.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40adc60 con 0x7fade40cd880 2023-10-26T05:53:23.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40c73a0 con 0x7fade40cd880 2023-10-26T05:53:23.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade4066b10 con 0x7fade40cd880 2023-10-26T05:53:23.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fadd40b4040 con 0x7fade40cd880 2023-10-26T05:53:23.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.036+0000 7fadd2ffd640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fadc4066470 0x7fadc4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.040+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd411e760 con 0x7fade40cd880 2023-10-26T05:53:23.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.040+0000 7fade1d9b640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fadc4066470 0x7fadc4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.040+0000 7fade1d9b640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fadc4066470 0x7fadc4068930 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fade4156c30 tx=0x7fadd8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:23.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.044+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fadd40ebbe0 con 0x7fade40cd880 2023-10-26T05:53:23.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.188+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fade40721e0 con 0x7fade40cd880 2023-10-26T05:53:23.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.188+0000 7fadd2ffd640 1 -- 172.21.15.71:0/3211115058 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fadd40b20a0 con 0x7fade40cd880 2023-10-26T05:53:23.196 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:53:23.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fadc4066470 msgr2=0x7fadc4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fadc4066470 0x7fadc4068930 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fade4156c30 tx=0x7fadd8009040 comp rx=0 tx=0).stop 2023-10-26T05:53:23.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 msgr2=0x7fade41522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 0x7fade41522f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fadd401a8b0 tx=0x7fadd40a5be0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 shutdown_connections 2023-10-26T05:53:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fade40d0850 0x7fade414f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fade40d0470 0x7fade40c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fadc4066470 0x7fadc4068930 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 --2- 172.21.15.71:0/3211115058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fade40cd880 0x7fade41522f0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 >> 172.21.15.71:0/3211115058 conn(0x7fade40cdd80 msgr2=0x7fade40ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 shutdown_connections 2023-10-26T05:53:23.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.192+0000 7fade8a65640 1 -- 172.21.15.71:0/3211115058 wait complete. 2023-10-26T05:53:23.214 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:53:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- 172.21.15.71:0/4110850241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8708150310 msgr2=0x7f870815a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 --2- 172.21.15.71:0/4110850241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8708150310 0x7f870815a220 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009c80 tx=0x7f86f801cd70 comp rx=0 tx=0).stop 2023-10-26T05:53:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- 172.21.15.71:0/4110850241 shutdown_connections 2023-10-26T05:53:23.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 --2- 172.21.15.71:0/4110850241 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f87080b3370 0x7f870806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 --2- 172.21.15.71:0/4110850241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8708150310 0x7f870815a220 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- 172.21.15.71:0/4110850241 >> 172.21.15.71:0/4110850241 conn(0x7f8708070d10 msgr2=0x7f870806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:23.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- 172.21.15.71:0/4110850241 shutdown_connections 2023-10-26T05:53:23.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- 172.21.15.71:0/4110850241 wait complete. 2023-10-26T05:53:23.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 Processor -- start 2023-10-26T05:53:23.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- start start 2023-10-26T05:53:23.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 0x7f8708158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87080c9b40 0x7f87080b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8708150310 0x7f87080b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080b2050 con 0x7f87080b3370 2023-10-26T05:53:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080b2ae0 con 0x7f87080c9b40 2023-10-26T05:53:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870f1fb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87080b2240 con 0x7f8708150310 2023-10-26T05:53:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 0x7f8708158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 0x7f8708158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37196/0 (socket says 172.21.15.71:37196) 2023-10-26T05:53:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 -- 172.21.15.71:0/3820526163 learned_addr learned my addr 172.21.15.71:0/3820526163 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f8707fff640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87080c9b40 0x7f87080b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 -- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8708150310 msgr2=0x7f87080b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8708150310 0x7f87080b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 -- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87080c9b40 msgr2=0x7f87080b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87080c9b40 0x7f87080b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f870cf97640 1 -- 172.21.15.71:0/3820526163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80098e0 con 0x7f87080b3370 2023-10-26T05:53:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.320+0000 7f8707fff640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87080c9b40 0x7f87080b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f870cf97640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 0x7f8708158180 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009bd0 tx=0x7f86f80ae6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f80ba070 con 0x7f87080b3370 2023-10-26T05:53:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f86f8004e10 con 0x7f87080b3370 2023-10-26T05:53:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87080b2520 con 0x7f87080b3370 2023-10-26T05:53:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f80aeb00 con 0x7f87080b3370 2023-10-26T05:53:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87080ad860 con 0x7f87080b3370 2023-10-26T05:53:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8708066b10 con 0x7f87080b3370 2023-10-26T05:53:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f86f80b4040 con 0x7f87080b3370 2023-10-26T05:53:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f87057fa640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86f4066470 0x7f86f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86f811ea10 con 0x7f87080b3370 2023-10-26T05:53:23.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f8707fff640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86f4066470 0x7f86f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.324+0000 7f8707fff640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86f4066470 0x7f86f4068930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8708144a70 tx=0x7f86fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:23.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.328+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f86f80ebf20 con 0x7f87080b3370 2023-10-26T05:53:23.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.468+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87080aa8e0 con 0x7f87080b3370 2023-10-26T05:53:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f87057fa640 1 -- 172.21.15.71:0/3820526163 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87080aa8e0 con 0x7f87080b3370 2023-10-26T05:53:23.477 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:53:23.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86f4066470 msgr2=0x7f86f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86f4066470 0x7f86f4068930 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8708144a70 tx=0x7f86fc009040 comp rx=0 tx=0).stop 2023-10-26T05:53:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 msgr2=0x7f8708158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 0x7f8708158180 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009bd0 tx=0x7f86f80ae6b0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 shutdown_connections 2023-10-26T05:53:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8708150310 0x7f87080b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87080c9b40 0x7f87080b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f86f4066470 0x7f86f4068930 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 --2- 172.21.15.71:0/3820526163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87080b3370 0x7f8708158180 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 >> 172.21.15.71:0/3820526163 conn(0x7f8708070d10 msgr2=0x7f8708145c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:23.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 shutdown_connections 2023-10-26T05:53:23.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.472+0000 7f870f1fb640 1 -- 172.21.15.71:0/3820526163 wait complete. 2023-10-26T05:53:23.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:53:23.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- 172.21.15.71:0/2823088192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 msgr2=0x7f82f013d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 --2- 172.21.15.71:0/2823088192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f013d770 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f82f006b1f0 tx=0x7f82e001c720 comp rx=0 tx=0).stop 2023-10-26T05:53:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- 172.21.15.71:0/2823088192 shutdown_connections 2023-10-26T05:53:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 --2- 172.21.15.71:0/2823088192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f013d770 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 --2- 172.21.15.71:0/2823088192 >> v2:172.21.15.86:3300/0 conn(0x7f82f013ae90 0x7f82f00cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- 172.21.15.71:0/2823088192 >> 172.21.15.71:0/2823088192 conn(0x7f82f006b8c0 msgr2=0x7f82f006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- 172.21.15.71:0/2823088192 shutdown_connections 2023-10-26T05:53:23.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- 172.21.15.71:0/2823088192 wait complete. 2023-10-26T05:53:23.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 Processor -- start 2023-10-26T05:53:23.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- start start 2023-10-26T05:53:23.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82f013ae90 0x7f82f006a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f013dcb0 0x7f82f006a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f0154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f014ea40 con 0x7f82f0159060 2023-10-26T05:53:23.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f013d360 con 0x7f82f013dcb0 2023-10-26T05:53:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f7c55640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f013c790 con 0x7f82f013ae90 2023-10-26T05:53:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f0154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f0154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37216/0 (socket says 172.21.15.71:37216) 2023-10-26T05:53:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 -- 172.21.15.71:0/4062366081 learned_addr learned my addr 172.21.15.71:0/4062366081 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f51f0640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f013dcb0 0x7f82f006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f59f1640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82f013ae90 0x7f82f006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 -- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82f013ae90 msgr2=0x7f82f006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82f013ae90 0x7f82f006a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 -- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f013dcb0 msgr2=0x7f82f006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f013dcb0 0x7f82f006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 -- 172.21.15.71:0/4062366081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00094a0 con 0x7f82f0159060 2023-10-26T05:53:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f51f0640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f013dcb0 0x7f82f006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.616+0000 7f82f61f2640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f0154a20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f82e800cb40 tx=0x7f82e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800aa50 con 0x7f82f0159060 2023-10-26T05:53:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f82e800aca0 con 0x7f82f0159060 2023-10-26T05:53:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f01575f0 con 0x7f82f0159060 2023-10-26T05:53:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8012680 con 0x7f82f0159060 2023-10-26T05:53:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f013cd10 con 0x7f82f0159060 2023-10-26T05:53:23.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f0066b10 con 0x7f82f0159060 2023-10-26T05:53:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f82e8025480 con 0x7f82f0159060 2023-10-26T05:53:23.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82e67fc640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82d4066470 0x7f82d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82e80849e0 con 0x7f82f0159060 2023-10-26T05:53:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82f59f1640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82d4066470 0x7f82d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.620+0000 7f82f59f1640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82d4066470 0x7f82d4068930 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0058d0 tx=0x7f82ec005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:23.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.624+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f82e8051ed0 con 0x7f82f0159060 2023-10-26T05:53:23.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.788+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82f013d4d0 con 0x7f82f0159060 2023-10-26T05:53:23.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.788+0000 7f82e67fc640 1 -- 172.21.15.71:0/4062366081 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f82e80554f0 con 0x7f82f0159060 2023-10-26T05:53:23.796 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:53:23.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82d4066470 msgr2=0x7f82d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82d4066470 0x7f82d4068930 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0058d0 tx=0x7f82ec005860 comp rx=0 tx=0).stop 2023-10-26T05:53:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 msgr2=0x7f82f0154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f0154a20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f82e800cb40 tx=0x7f82e800cc30 comp rx=0 tx=0).stop 2023-10-26T05:53:23.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 shutdown_connections 2023-10-26T05:53:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f82d4066470 0x7f82d4068930 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f82f0159060 0x7f82f0154a20 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f82f013dcb0 0x7f82f006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 --2- 172.21.15.71:0/4062366081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f82f013ae90 0x7f82f006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:23.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 >> 172.21.15.71:0/4062366081 conn(0x7f82f006b8c0 msgr2=0x7f82f00af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:23.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 shutdown_connections 2023-10-26T05:53:23.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23.792+0000 7f82f7c55640 1 -- 172.21.15.71:0/4062366081 wait complete. 2023-10-26T05:53:23.815 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:53:23.815 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:23.815 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:23.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:23.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[47484]: fuse finished with error 0 and tester_r 0 2023-10-26T05:53:23.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:23.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:53:23.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:23.885 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:53:23.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:53:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:23.907 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:53:23.907 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:23.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:24.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 -- 172.21.15.71:0/2197021642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 msgr2=0x7f7f3815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 --2- 172.21.15.71:0/2197021642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815b3a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009900 tx=0x7f7f2c01c770 comp rx=0 tx=0).stop 2023-10-26T05:53:24.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 -- 172.21.15.71:0/2197021642 shutdown_connections 2023-10-26T05:53:24.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 --2- 172.21.15.71:0/2197021642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815b3a0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 --2- 172.21.15.71:0/2197021642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f380d0470 0x7f7f3806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 -- 172.21.15.71:0/2197021642 >> 172.21.15.71:0/2197021642 conn(0x7f7f380cdd80 msgr2=0x7f7f380ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 -- 172.21.15.71:0/2197021642 shutdown_connections 2023-10-26T05:53:24.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.048+0000 7f7f3f009640 1 -- 172.21.15.71:0/2197021642 wait complete. 2023-10-26T05:53:24.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 Processor -- start 2023-10-26T05:53:24.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- start start 2023-10-26T05:53:24.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f380cd880 0x7f7f38154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f380d0470 0x7f7f3815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3815e460 con 0x7f7f380d0850 2023-10-26T05:53:24.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3815ca20 con 0x7f7f380cd880 2023-10-26T05:53:24.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3815c0f0 con 0x7f7f380d0470 2023-10-26T05:53:24.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37224/0 (socket says 172.21.15.71:37224) 2023-10-26T05:53:24.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 -- 172.21.15.71:0/775697725 learned_addr learned my addr 172.21.15.71:0/775697725 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:24.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f37fff640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f380d0470 0x7f7f3815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3cda5640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f380cd880 0x7f7f38154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 -- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f380d0470 msgr2=0x7f7f3815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f380d0470 0x7f7f3815d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 -- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f380cd880 msgr2=0x7f7f38154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f380cd880 0x7f7f38154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 -- 172.21.15.71:0/775697725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2c009560 con 0x7f7f380d0850 2023-10-26T05:53:24.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3cda5640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f380cd880 0x7f7f38154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:24.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3d5a6640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815db60 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c770 tx=0x7f7f3000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30003b80 con 0x7f7f380d0850 2023-10-26T05:53:24.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7f30003ce0 con 0x7f7f380d0850 2023-10-26T05:53:24.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3815c3d0 con 0x7f7f380d0850 2023-10-26T05:53:24.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3000aab0 con 0x7f7f380d0850 2023-10-26T05:53:24.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3815e0a0 con 0x7f7f380d0850 2023-10-26T05:53:24.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f38066b10 con 0x7f7f380d0850 2023-10-26T05:53:24.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.052+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7f3000ac10 con 0x7f7f380d0850 2023-10-26T05:53:24.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.056+0000 7f7f357fa640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f24066470 0x7f7f24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.056+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f30083e70 con 0x7f7f380d0850 2023-10-26T05:53:24.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.056+0000 7f7f3cda5640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f24066470 0x7f7f24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.056+0000 7f7f3cda5640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f24066470 0x7f7f24068930 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7f38069d10 tx=0x7f7f28008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.060+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7f300512f0 con 0x7f7f380d0850 2023-10-26T05:53:24.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.212+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f3815cb60 con 0x7f7f380d0850 2023-10-26T05:53:24.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.216+0000 7f7f357fa640 1 -- 172.21.15.71:0/775697725 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f7f3815cb60 con 0x7f7f380d0850 2023-10-26T05:53:24.222 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:24.223 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":127,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:20.189455+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:24.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f24066470 msgr2=0x7f7f24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f24066470 0x7f7f24068930 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7f38069d10 tx=0x7f7f28008040 comp rx=0 tx=0).stop 2023-10-26T05:53:24.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 msgr2=0x7f7f3815db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815db60 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000c770 tx=0x7f7f3000cc30 comp rx=0 tx=0).stop 2023-10-26T05:53:24.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 shutdown_connections 2023-10-26T05:53:24.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7f24066470 0x7f7f24068930 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f380d0850 0x7f7f3815db60 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7f380d0470 0x7f7f3815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 --2- 172.21.15.71:0/775697725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f380cd880 0x7f7f38154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 >> 172.21.15.71:0/775697725 conn(0x7f7f380cdd80 msgr2=0x7f7f380cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 shutdown_connections 2023-10-26T05:53:24.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.220+0000 7f7f3f009640 1 -- 172.21.15.71:0/775697725 wait complete. 2023-10-26T05:53:24.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:53:04.827214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:53:05.732853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:53:24.245 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:53:24.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 -- 172.21.15.71:0/1936058339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13ed90 msgr2=0x7f3a0c069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 --2- 172.21.15.71:0/1936058339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13ed90 0x7f3a0c069c00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c06e990 tx=0x7f3a0401cb30 comp rx=0 tx=0).stop 2023-10-26T05:53:24.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 -- 172.21.15.71:0/1936058339 shutdown_connections 2023-10-26T05:53:24.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 --2- 172.21.15.71:0/1936058339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13ed90 0x7f3a0c069c00 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 --2- 172.21.15.71:0/1936058339 >> v2:172.21.15.86:3300/0 conn(0x7f3a0c15a3f0 0x7f3a0c13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 -- 172.21.15.71:0/1936058339 >> 172.21.15.71:0/1936058339 conn(0x7f3a0c0aec60 msgr2=0x7f3a0c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 -- 172.21.15.71:0/1936058339 shutdown_connections 2023-10-26T05:53:24.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.352+0000 7f3a13920640 1 -- 172.21.15.71:0/1936058339 wait complete. 2023-10-26T05:53:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 Processor -- start 2023-10-26T05:53:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 -- start start 2023-10-26T05:53:24.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 0x7f3a0c1361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a0c13ed90 0x7f3a0c137070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a0c15a3f0 0x7f3a0c14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1343b0 con 0x7f3a0c13c120 2023-10-26T05:53:24.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1375b0 con 0x7f3a0c15a3f0 2023-10-26T05:53:24.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c14ae40 con 0x7f3a0c13ed90 2023-10-26T05:53:24.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 0x7f3a0c1361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 0x7f3a0c1361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37250/0 (socket says 172.21.15.71:37250) 2023-10-26T05:53:24.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 -- 172.21.15.71:0/3210152911 learned_addr learned my addr 172.21.15.71:0/3210152911 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:24.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a11ebd640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a0c15a3f0 0x7f3a0c14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 -- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a0c13ed90 msgr2=0x7f3a0c137070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a0c13ed90 0x7f3a0c137070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 -- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a0c15a3f0 msgr2=0x7f3a0c14d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a0c15a3f0 0x7f3a0c14d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 -- 172.21.15.71:0/3210152911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04009820 con 0x7f3a0c13c120 2023-10-26T05:53:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a116bc640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 0x7f3a0c1361a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800c500 tx=0x7f3a0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08012830 con 0x7f3a0c13c120 2023-10-26T05:53:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3a08012a80 con 0x7f3a0c13c120 2023-10-26T05:53:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08012e90 con 0x7f3a0c13c120 2023-10-26T05:53:24.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c14a5a0 con 0x7f3a0c13c120 2023-10-26T05:53:24.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c14ab60 con 0x7f3a0c13c120 2023-10-26T05:53:24.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.356+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3a0801b400 con 0x7f3a0c13c120 2023-10-26T05:53:24.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.360+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c066b10 con 0x7f3a0c13c120 2023-10-26T05:53:24.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.360+0000 7f3a01ffb640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39f0066470 0x7f39f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.360+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a080843c0 con 0x7f3a0c13c120 2023-10-26T05:53:24.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.360+0000 7f3a10ebb640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39f0066470 0x7f39f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.360+0000 7f3a10ebb640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39f0066470 0x7f39f0068930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c15a840 tx=0x7f39fc005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.364+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3a080553a0 con 0x7f3a0c13c120 2023-10-26T05:53:24.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.504+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a0c0721e0 con 0x7f3a0c13c120 2023-10-26T05:53:24.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a01ffb640 1 -- 172.21.15.71:0/3210152911 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a08054f30 con 0x7f3a0c13c120 2023-10-26T05:53:24.514 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:53:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39f0066470 msgr2=0x7f39f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39f0066470 0x7f39f0068930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c15a840 tx=0x7f39fc005be0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 msgr2=0x7f3a0c1361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 0x7f3a0c1361a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800c500 tx=0x7f3a0800c9c0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 shutdown_connections 2023-10-26T05:53:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a0c15a3f0 0x7f3a0c14d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3a0c13ed90 0x7f3a0c137070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f39f0066470 0x7f39f0068930 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 --2- 172.21.15.71:0/3210152911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3a0c13c120 0x7f3a0c1361a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 >> 172.21.15.71:0/3210152911 conn(0x7f3a0c0aec60 msgr2=0x7f3a0c0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 shutdown_connections 2023-10-26T05:53:24.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.508+0000 7f3a13920640 1 -- 172.21.15.71:0/3210152911 wait complete. 2023-10-26T05:53:24.531 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:53:24.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 -- 172.21.15.71:0/3738212042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 msgr2=0x7feba415ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 --2- 172.21.15.71:0/3738212042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba415ccc0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7feb94009900 tx=0x7feb9401c850 comp rx=0 tx=0).stop 2023-10-26T05:53:24.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 -- 172.21.15.71:0/3738212042 shutdown_connections 2023-10-26T05:53:24.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 --2- 172.21.15.71:0/3738212042 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feba415d200 0x7feba40cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 --2- 172.21.15.71:0/3738212042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba415ccc0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 -- 172.21.15.71:0/3738212042 >> 172.21.15.71:0/3738212042 conn(0x7feba40aec60 msgr2=0x7feba40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 -- 172.21.15.71:0/3738212042 shutdown_connections 2023-10-26T05:53:24.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 -- 172.21.15.71:0/3738212042 wait complete. 2023-10-26T05:53:24.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 Processor -- start 2023-10-26T05:53:24.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.640+0000 7febaafd0640 1 -- start start 2023-10-26T05:53:24.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feba40cbcd0 0x7feba4135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba41379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba415d200 0x7feba4137f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba40abc90 con 0x7feba413c220 2023-10-26T05:53:24.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba4135d90 con 0x7feba415d200 2023-10-26T05:53:24.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba4070810 con 0x7feba40cbcd0 2023-10-26T05:53:24.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba41379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba41379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37262/0 (socket says 172.21.15.71:37262) 2023-10-26T05:53:24.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 -- 172.21.15.71:0/1922279516 learned_addr learned my addr 172.21.15.71:0/1922279516 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 -- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feba40cbcd0 msgr2=0x7feba4135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feba40cbcd0 0x7feba4135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 -- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba415d200 msgr2=0x7feba4137f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba415d200 0x7feba4137f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 -- 172.21.15.71:0/1922279516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb94009560 con 0x7feba413c220 2023-10-26T05:53:24.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba3fff640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba41379f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c9f0 tx=0x7feb9800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb98007bb0 con 0x7feba413c220 2023-10-26T05:53:24.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feb98007e00 con 0x7feba413c220 2023-10-26T05:53:24.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9800f6b0 con 0x7feba413c220 2023-10-26T05:53:24.656 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba40ab6d0 con 0x7feba413c220 2023-10-26T05:53:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba4070b60 con 0x7feba413c220 2023-10-26T05:53:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba4066b10 con 0x7feba413c220 2023-10-26T05:53:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb9800f810 con 0x7feba413c220 2023-10-26T05:53:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba17fa640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb90066470 0x7feb90068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb980845d0 con 0x7feba413c220 2023-10-26T05:53:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.644+0000 7feba8d6c640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb90066470 0x7feb90068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.648+0000 7feba8d6c640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb90066470 0x7feb90068930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb940af000 tx=0x7feb940195f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.648+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feb98051ae0 con 0x7feba413c220 2023-10-26T05:53:24.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.800+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feba40709c0 con 0x7feba413c220 2023-10-26T05:53:24.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.800+0000 7feba17fa640 1 -- 172.21.15.71:0/1922279516 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7feba40709c0 con 0x7feba413c220 2023-10-26T05:53:24.809 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:53:24.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb90066470 msgr2=0x7feb90068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb90066470 0x7feb90068930 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7feb940af000 tx=0x7feb940195f0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 msgr2=0x7feba41379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba41379f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c9f0 tx=0x7feb9800ceb0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 shutdown_connections 2023-10-26T05:53:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feba415d200 0x7feba4137f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7feb90066470 0x7feb90068930 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feba413c220 0x7feba41379f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 --2- 172.21.15.71:0/1922279516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feba40cbcd0 0x7feba4135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 >> 172.21.15.71:0/1922279516 conn(0x7feba40aec60 msgr2=0x7feba40aca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 shutdown_connections 2023-10-26T05:53:24.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.804+0000 7febaafd0640 1 -- 172.21.15.71:0/1922279516 wait complete. 2023-10-26T05:53:24.830 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:53:24.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/1456082635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0850 msgr2=0x7f6e4c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 --2- 172.21.15.71:0/1456082635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0850 0x7f6e4c15b3a0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009840 tx=0x7f6e4001c740 comp rx=0 tx=0).stop 2023-10-26T05:53:24.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/1456082635 shutdown_connections 2023-10-26T05:53:24.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 --2- 172.21.15.71:0/1456082635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0850 0x7f6e4c15b3a0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/1456082635 >> 172.21.15.71:0/1456082635 conn(0x7f6e4c0cdd80 msgr2=0x7f6e4c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:24.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/1456082635 shutdown_connections 2023-10-26T05:53:24.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/1456082635 wait complete. 2023-10-26T05:53:24.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 Processor -- start 2023-10-26T05:53:24.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- start start 2023-10-26T05:53:24.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0cd880 0x7f6e4c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 0x7f6e4c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e4c0d0850 0x7f6e4c0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c06a650 con 0x7f6e4c0d0470 2023-10-26T05:53:24.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c8ce0 con 0x7f6e4c0cd880 2023-10-26T05:53:24.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c8fc0 con 0x7f6e4c0d0850 2023-10-26T05:53:24.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 0x7f6e4c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 0x7f6e4c0c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37288/0 (socket says 172.21.15.71:37288) 2023-10-26T05:53:24.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 -- 172.21.15.71:0/633404605 learned_addr learned my addr 172.21.15.71:0/633404605 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:24.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4bfff640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0cd880 0x7f6e4c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e50995640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e4c0d0850 0x7f6e4c0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 -- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e4c0d0850 msgr2=0x7f6e4c0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e4c0d0850 0x7f6e4c0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 -- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0cd880 msgr2=0x7f6e4c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:24.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0cd880 0x7f6e4c15aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:24.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 -- 172.21.15.71:0/633404605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400094a0 con 0x7f6e4c0d0470 2023-10-26T05:53:24.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e50995640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e4c0d0850 0x7f6e4c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:24.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e4b7fe640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 0x7f6e4c0c6b20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6e4001cc20 tx=0x7f6e400a5760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40005240 con 0x7f6e4c0d0470 2023-10-26T05:53:24.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e40005490 con 0x7f6e4c0d0470 2023-10-26T05:53:24.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0c92a0 con 0x7f6e4c0d0470 2023-10-26T05:53:24.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e400046d0 con 0x7f6e4c0d0470 2023-10-26T05:53:24.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.952+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c0c9860 con 0x7f6e4c0d0470 2023-10-26T05:53:24.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.956+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c066b10 con 0x7f6e4c0d0470 2023-10-26T05:53:24.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.956+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e40004830 con 0x7f6e4c0d0470 2023-10-26T05:53:24.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.956+0000 7f6e48ff9640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e38066470 0x7f6e38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:24.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.956+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e4011ec40 con 0x7f6e4c0d0470 2023-10-26T05:53:24.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.956+0000 7f6e4bfff640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e38066470 0x7f6e38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:24.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.956+0000 7f6e4bfff640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e38066470 0x7f6e38068930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0011f0 tx=0x7f6e3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:24.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:24.960+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e400ec020 con 0x7f6e4c0d0470 2023-10-26T05:53:25.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e4c0721e0 con 0x7f6e4c0d0470 2023-10-26T05:53:25.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e48ff9640 1 -- 172.21.15.71:0/633404605 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e400bb0a0 con 0x7f6e4c0d0470 2023-10-26T05:53:25.115 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:53:25.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e38066470 msgr2=0x7f6e38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e38066470 0x7f6e38068930 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0011f0 tx=0x7f6e3c009040 comp rx=0 tx=0).stop 2023-10-26T05:53:25.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 msgr2=0x7f6e4c0c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 0x7f6e4c0c6b20 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6e4001cc20 tx=0x7f6e400a5760 comp rx=0 tx=0).stop 2023-10-26T05:53:25.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 shutdown_connections 2023-10-26T05:53:25.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e4c0d0850 0x7f6e4c0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e38066470 0x7f6e38068930 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e4c0d0470 0x7f6e4c0c6b20 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 --2- 172.21.15.71:0/633404605 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e4c0cd880 0x7f6e4c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 >> 172.21.15.71:0/633404605 conn(0x7f6e4c0cdd80 msgr2=0x7f6e4c0b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:25.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 shutdown_connections 2023-10-26T05:53:25.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.108+0000 7f6e523f8640 1 -- 172.21.15.71:0/633404605 wait complete. 2023-10-26T05:53:25.134 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:53:25.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/1113764985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 msgr2=0x7f45840058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 --2- 172.21.15.71:0/1113764985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45840058e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f45800016b0 tx=0x7f458001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f4596d9d640 1 -- 172.21.15.71:0/1113764985 reap_dead start 2023-10-26T05:53:25.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/1113764985 shutdown_connections 2023-10-26T05:53:25.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/1113764985 >> 172.21.15.71:0/1113764985 conn(0x7f459806b8c0 msgr2=0x7f459806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:25.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/1113764985 shutdown_connections 2023-10-26T05:53:25.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/1113764985 wait complete. 2023-10-26T05:53:25.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 Processor -- start 2023-10-26T05:53:25.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- start start 2023-10-26T05:53:25.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45981e5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459815a7f0 0x7f4598141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f459815ac50 0x7f4598141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45981534a0 con 0x7f4584005040 2023-10-26T05:53:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459814e1d0 con 0x7f459815a7f0 2023-10-26T05:53:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459814fc20 con 0x7f459815ac50 2023-10-26T05:53:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45981e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45981e5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37306/0 (socket says 172.21.15.71:37306) 2023-10-26T05:53:25.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 -- 172.21.15.71:0/3403473887 learned_addr learned my addr 172.21.15.71:0/3403473887 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f4595d9b640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459815a7f0 0x7f4598141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f4596d9d640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f459815ac50 0x7f4598141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 -- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f459815ac50 msgr2=0x7f4598141980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f459815ac50 0x7f4598141980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 -- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459815a7f0 msgr2=0x7f4598141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459815a7f0 0x7f4598141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 -- 172.21.15.71:0/3403473887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459814ff00 con 0x7f4584005040 2023-10-26T05:53:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459659c640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45981e5850 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f458800fc90 tx=0x7f458800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800dee0 con 0x7f4584005040 2023-10-26T05:53:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4588004e20 con 0x7f4584005040 2023-10-26T05:53:25.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588005280 con 0x7f4584005040 2023-10-26T05:53:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45981501e0 con 0x7f4584005040 2023-10-26T05:53:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.256+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f459814d5e0 con 0x7f4584005040 2023-10-26T05:53:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.260+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4598066b10 con 0x7f4584005040 2023-10-26T05:53:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.260+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45880053e0 con 0x7f4584005040 2023-10-26T05:53:25.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.260+0000 7f458effd640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45780664a0 0x7f4578068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.260+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4588084df0 con 0x7f4584005040 2023-10-26T05:53:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.260+0000 7f4595d9b640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45780664a0 0x7f4578068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.260+0000 7f4595d9b640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45780664a0 0x7f4578068960 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4580001680 tx=0x7f4580019540 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:25.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.264+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4588052280 con 0x7f4584005040 2023-10-26T05:53:25.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.416+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4598150700 con 0x7f4584005040 2023-10-26T05:53:25.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.416+0000 7f458effd640 1 -- 172.21.15.71:0/3403473887 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4598150700 con 0x7f4584005040 2023-10-26T05:53:25.425 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:53:25.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45780664a0 msgr2=0x7f4578068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45780664a0 0x7f4578068960 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4580001680 tx=0x7f4580019540 comp rx=0 tx=0).stop 2023-10-26T05:53:25.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 msgr2=0x7f45981e5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45981e5850 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f458800fc90 tx=0x7f458800d6b0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 shutdown_connections 2023-10-26T05:53:25.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f459815ac50 0x7f4598141980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f459815a7f0 0x7f4598141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f45780664a0 0x7f4578068960 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 --2- 172.21.15.71:0/3403473887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4584005040 0x7f45981e5850 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 >> 172.21.15.71:0/3403473887 conn(0x7f459806b8c0 msgr2=0x7f459815b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:25.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 shutdown_connections 2023-10-26T05:53:25.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.420+0000 7f459cc08640 1 -- 172.21.15.71:0/3403473887 wait complete. 2023-10-26T05:53:25.442 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:53:25.442 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:25.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:25.498 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:25.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[34894]: fuse finished with error 0 and tester_r 0 2023-10-26T05:53:25.499 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:25.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:53:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:25.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:53:25.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:25.508 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:53:25.514 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:25.515 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:25.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:53:25.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:53:25.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:25.521 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:53:25.565 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:53:25.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:53:25.567 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:53:25.567 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:25.572 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:53:25.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:53:25.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:25.573 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:53:25.618 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:53:25.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:53:25.619 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:25.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1841069584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c40047d0 msgr2=0x7f07c4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1841069584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c40047d0 0x7f07c4004c50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f07d0006fb0 tx=0x7f07d001f690 comp rx=0 tx=0).stop 2023-10-26T05:53:25.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1841069584 shutdown_connections 2023-10-26T05:53:25.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1841069584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c40047d0 0x7f07c4004c50 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1841069584 >> 172.21.15.71:0/1841069584 conn(0x7f07dc069770 msgr2=0x7f07dc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:25.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1841069584 shutdown_connections 2023-10-26T05:53:25.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1841069584 wait complete. 2023-10-26T05:53:25.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 Processor -- start 2023-10-26T05:53:25.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 -- start start 2023-10-26T05:53:25.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c40047d0 0x7f07dc1343f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07dc1366c0 0x7f07dc157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.744+0000 7f07e0e8a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 0x7f07dc06a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07e0e8a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc135900 con 0x7f07dc15b6f0 2023-10-26T05:53:25.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07e0e8a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc135be0 con 0x7f07c40047d0 2023-10-26T05:53:25.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07e0e8a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc135ec0 con 0x7f07dc1366c0 2023-10-26T05:53:25.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 0x7f07dc06a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 0x7f07dc06a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37332/0 (socket says 172.21.15.71:37332) 2023-10-26T05:53:25.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 -- 172.21.15.71:0/1911555195 learned_addr learned my addr 172.21.15.71:0/1911555195 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:25.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07d9d9b640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07dc1366c0 0x7f07dc157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07da59c640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c40047d0 0x7f07dc1343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 -- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07dc1366c0 msgr2=0x7f07dc157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07dc1366c0 0x7f07dc157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 -- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c40047d0 msgr2=0x7f07dc1343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c40047d0 0x7f07dc1343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 -- 172.21.15.71:0/1911555195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc1361a0 con 0x7f07dc15b6f0 2023-10-26T05:53:25.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07dad9d640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 0x7f07dc06a0f0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f07dc072f10 tx=0x7f07d400fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:25.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4004d30 con 0x7f07dc15b6f0 2023-10-26T05:53:25.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f07d400daa0 con 0x7f07dc15b6f0 2023-10-26T05:53:25.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d400de90 con 0x7f07dc15b6f0 2023-10-26T05:53:25.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc15bb70 con 0x7f07dc15b6f0 2023-10-26T05:53:25.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc06a800 con 0x7f07dc15b6f0 2023-10-26T05:53:25.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc066b10 con 0x7f07dc15b6f0 2023-10-26T05:53:25.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07d4011440 con 0x7f07dc15b6f0 2023-10-26T05:53:25.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07caffd640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07bc066470 0x7f07bc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:25.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07d4083c80 con 0x7f07dc15b6f0 2023-10-26T05:53:25.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.748+0000 7f07da59c640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07bc066470 0x7f07bc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:25.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.752+0000 7f07da59c640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07bc066470 0x7f07bc068930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f07cc006fd0 tx=0x7f07cc006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:25.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.752+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f07d4051100 con 0x7f07dc15b6f0 2023-10-26T05:53:25.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.924+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07dc135a40 con 0x7f07dc15b6f0 2023-10-26T05:53:25.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07caffd640 1 -- 172.21.15.71:0/1911555195 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f07dc135a40 con 0x7f07dc15b6f0 2023-10-26T05:53:25.934 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:25.934 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:25.934 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:25.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07bc066470 msgr2=0x7f07bc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07bc066470 0x7f07bc068930 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f07cc006fd0 tx=0x7f07cc006af0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 msgr2=0x7f07dc06a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:25.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 0x7f07dc06a0f0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f07dc072f10 tx=0x7f07d400fd90 comp rx=0 tx=0).stop 2023-10-26T05:53:25.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 shutdown_connections 2023-10-26T05:53:25.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f07bc066470 0x7f07bc068930 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07dc15b6f0 0x7f07dc06a0f0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07dc1366c0 0x7f07dc157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 --2- 172.21.15.71:0/1911555195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07c40047d0 0x7f07dc1343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:25.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 >> 172.21.15.71:0/1911555195 conn(0x7f07dc069770 msgr2=0x7f07dc13dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:25.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 shutdown_connections 2023-10-26T05:53:25.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:25.928+0000 7f07e0e8a640 1 -- 172.21.15.71:0/1911555195 wait complete. 2023-10-26T05:53:25.951 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:26.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 -- 172.21.15.71:0/752234833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59481588f0 msgr2=0x7f5948158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 --2- 172.21.15.71:0/752234833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59481588f0 0x7f5948158cd0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f59480afef0 tx=0x7f593801c520 comp rx=0 tx=0).stop 2023-10-26T05:53:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 -- 172.21.15.71:0/752234833 shutdown_connections 2023-10-26T05:53:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 --2- 172.21.15.71:0/752234833 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5948158500 0x7f59480b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 --2- 172.21.15.71:0/752234833 >> v2:172.21.15.86:3300/0 conn(0x7f59480b21b0 0x7f594815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 --2- 172.21.15.71:0/752234833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59481588f0 0x7f5948158cd0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 -- 172.21.15.71:0/752234833 >> 172.21.15.71:0/752234833 conn(0x7f59480ce270 msgr2=0x7f59480ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 -- 172.21.15.71:0/752234833 shutdown_connections 2023-10-26T05:53:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 -- 172.21.15.71:0/752234833 wait complete. 2023-10-26T05:53:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.052+0000 7f594f9ca640 1 Processor -- start 2023-10-26T05:53:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- start start 2023-10-26T05:53:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 0x7f594814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59481588f0 0x7f59481569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948156f20 0x7f59481526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59481573b0 con 0x7f59480b21b0 2023-10-26T05:53:26.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594814b370 con 0x7f5948156f20 2023-10-26T05:53:26.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594814bc70 con 0x7f59481588f0 2023-10-26T05:53:26.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 0x7f594814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 0x7f594814ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37356/0 (socket says 172.21.15.71:37356) 2023-10-26T05:53:26.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 -- 172.21.15.71:0/1716352240 learned_addr learned my addr 172.21.15.71:0/1716352240 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:26.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 -- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59481588f0 msgr2=0x7f59481569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59481588f0 0x7f59481569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 -- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948156f20 msgr2=0x7f59481526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948156f20 0x7f59481526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 -- 172.21.15.71:0/1716352240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5938009560 con 0x7f59480b21b0 2023-10-26T05:53:26.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594d766640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 0x7f594814ae30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f593801abd0 tx=0x7f59380a85f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59380b4040 con 0x7f59480b21b0 2023-10-26T05:53:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f59380042c0 con 0x7f59480b21b0 2023-10-26T05:53:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594814bf50 con 0x7f59480b21b0 2023-10-26T05:53:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938005020 con 0x7f59480b21b0 2023-10-26T05:53:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594814c510 con 0x7f59480b21b0 2023-10-26T05:53:26.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5948066d00 con 0x7f59480b21b0 2023-10-26T05:53:26.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f59380a6040 con 0x7f59480b21b0 2023-10-26T05:53:26.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f5945ffb640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59340664a0 0x7f5934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f593811e640 con 0x7f59480b21b0 2023-10-26T05:53:26.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.056+0000 7f594cf65640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59340664a0 0x7f5934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.060+0000 7f594cf65640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59340664a0 0x7f5934068960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5948157670 tx=0x7f593c005c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.060+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f59380ebad0 con 0x7f59480b21b0 2023-10-26T05:53:26.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.204+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59480ce900 con 0x7f59480b21b0 2023-10-26T05:53:26.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.208+0000 7f5945ffb640 1 -- 172.21.15.71:0/1716352240 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f59380ef0f0 con 0x7f59480b21b0 2023-10-26T05:53:26.214 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:26.214 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":127,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:20.189455+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:26.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59340664a0 msgr2=0x7f5934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59340664a0 0x7f5934068960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5948157670 tx=0x7f593c005c00 comp rx=0 tx=0).stop 2023-10-26T05:53:26.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 msgr2=0x7f594814ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 0x7f594814ae30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f593801abd0 tx=0x7f59380a85f0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 shutdown_connections 2023-10-26T05:53:26.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948156f20 0x7f59481526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f59481588f0 0x7f59481569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f59340664a0 0x7f5934068960 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 --2- 172.21.15.71:0/1716352240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f59480b21b0 0x7f594814ae30 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 >> 172.21.15.71:0/1716352240 conn(0x7f59480ce270 msgr2=0x7f59480b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 shutdown_connections 2023-10-26T05:53:26.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.212+0000 7f594f9ca640 1 -- 172.21.15.71:0/1716352240 wait complete. 2023-10-26T05:53:26.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:53:04.827214+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:53:05.732853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:53:26.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:53:26.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/1604662691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d81592a0 msgr2=0x7fc4d806ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/1604662691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d81592a0 0x7fc4d806ee10 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009bc0 tx=0x7fc4c801cb10 comp rx=0 tx=0).stop 2023-10-26T05:53:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/1604662691 shutdown_connections 2023-10-26T05:53:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/1604662691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d81592a0 0x7fc4d806ee10 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.349 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/1604662691 >> 172.21.15.71:0/1604662691 conn(0x7fc4d806d320 msgr2=0x7fc4d806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/1604662691 shutdown_connections 2023-10-26T05:53:26.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.340+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/1604662691 wait complete. 2023-10-26T05:53:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 Processor -- start 2023-10-26T05:53:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- start start 2023-10-26T05:53:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4d80cbb50 0x7fc4d80c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 0x7fc4d8150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d81592a0 0x7fc4d814b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80c6750 con 0x7fc4d80ce940 2023-10-26T05:53:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d8150d60 con 0x7fc4d81592a0 2023-10-26T05:53:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80c6940 con 0x7fc4d80cbb50 2023-10-26T05:53:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 0x7fc4d8150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 0x7fc4d8150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37386/0 (socket says 172.21.15.71:37386) 2023-10-26T05:53:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 -- 172.21.15.71:0/196174404 learned_addr learned my addr 172.21.15.71:0/196174404 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d77fe640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d81592a0 0x7fc4d814b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 -- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4d80cbb50 msgr2=0x7fc4d80c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4d80cbb50 0x7fc4d80c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 -- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d81592a0 msgr2=0x7fc4d814b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d81592a0 0x7fc4d814b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 -- 172.21.15.71:0/196174404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8009820 con 0x7fc4d80ce940 2023-10-26T05:53:26.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4d67fc640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 0x7fc4d8150820 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c490 tx=0x7fc4c0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0003600 con 0x7fc4d80ce940 2023-10-26T05:53:26.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4c0003780 con 0x7fc4d80ce940 2023-10-26T05:53:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d80c6c20 con 0x7fc4d80ce940 2023-10-26T05:53:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0003bd0 con 0x7fc4d80ce940 2023-10-26T05:53:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d80c71e0 con 0x7fc4d80ce940 2023-10-26T05:53:26.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d8066b10 con 0x7fc4d80ce940 2023-10-26T05:53:26.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4c0003d30 con 0x7fc4d80ce940 2023-10-26T05:53:26.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4cf7fe640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4b80664a0 0x7fc4b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.344+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4c0083570 con 0x7fc4d80ce940 2023-10-26T05:53:26.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.348+0000 7fc4d6ffd640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4b80664a0 0x7fc4b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.348+0000 7fc4d6ffd640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4b80664a0 0x7fc4b8068960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80af000 tx=0x7fc4c80198b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.348+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc4c0050a80 con 0x7fc4d80ce940 2023-10-26T05:53:26.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4d8066d00 con 0x7fc4d80ce940 2023-10-26T05:53:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4cf7fe640 1 -- 172.21.15.71:0/196174404 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4c0017020 con 0x7fc4d80ce940 2023-10-26T05:53:26.502 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:53:26.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4b80664a0 msgr2=0x7fc4b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4b80664a0 0x7fc4b8068960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80af000 tx=0x7fc4c80198b0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 msgr2=0x7fc4d8150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 0x7fc4d8150820 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000c490 tx=0x7fc4c0001e80 comp rx=0 tx=0).stop 2023-10-26T05:53:26.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 shutdown_connections 2023-10-26T05:53:26.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4d81592a0 0x7fc4d814b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4b80664a0 0x7fc4b8068960 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d80ce940 0x7fc4d8150820 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 --2- 172.21.15.71:0/196174404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4d80cbb50 0x7fc4d80c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 >> 172.21.15.71:0/196174404 conn(0x7fc4d806d320 msgr2=0x7fc4d80b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 shutdown_connections 2023-10-26T05:53:26.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.496+0000 7fc4dd4f7640 1 -- 172.21.15.71:0/196174404 wait complete. 2023-10-26T05:53:26.519 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:53:26.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/1869923653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa480004e30 msgr2=0x7fa48000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 --2- 172.21.15.71:0/1869923653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa480004e30 0x7fa48000b990 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00c5c0 tx=0x7fa48c01d340 comp rx=0 tx=0).stop 2023-10-26T05:53:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa498f78640 1 -- 172.21.15.71:0/1869923653 reap_dead start 2023-10-26T05:53:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/1869923653 shutdown_connections 2023-10-26T05:53:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/1869923653 >> 172.21.15.71:0/1869923653 conn(0x7fa4940c97f0 msgr2=0x7fa4940c9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/1869923653 shutdown_connections 2023-10-26T05:53:26.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/1869923653 wait complete. 2023-10-26T05:53:26.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 Processor -- start 2023-10-26T05:53:26.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- start start 2023-10-26T05:53:26.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa480004e30 0x7fa49414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 0x7fa494156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494152370 0x7fa494152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941568b0 con 0x7fa494155ef0 2023-10-26T05:53:26.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49414edb0 con 0x7fa494152370 2023-10-26T05:53:26.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa494152d40 con 0x7fa480004e30 2023-10-26T05:53:26.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 0x7fa494156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 0x7fa494156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37406/0 (socket says 172.21.15.71:37406) 2023-10-26T05:53:26.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 -- 172.21.15.71:0/34233587 learned_addr learned my addr 172.21.15.71:0/34233587 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:26.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa498f78640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494152370 0x7fa494152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa493fff640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa480004e30 0x7fa49414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 -- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa480004e30 msgr2=0x7fa49414e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa480004e30 0x7fa49414e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 -- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494152370 msgr2=0x7fa494152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494152370 0x7fa494152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 -- 172.21.15.71:0/34233587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494152ff0 con 0x7fa494155ef0 2023-10-26T05:53:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa493fff640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa480004e30 0x7fa49414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa498f78640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494152370 0x7fa494152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa48b7fe640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 0x7fa494156370 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00fa00 tx=0x7fa47c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004460 con 0x7fa494155ef0 2023-10-26T05:53:26.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa47c00b8a0 con 0x7fa494155ef0 2023-10-26T05:53:26.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4941532d0 con 0x7fa494155ef0 2023-10-26T05:53:26.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00bc90 con 0x7fa494155ef0 2023-10-26T05:53:26.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.632+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa494153890 con 0x7fa494155ef0 2023-10-26T05:53:26.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.636+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa494066b10 con 0x7fa494155ef0 2023-10-26T05:53:26.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.636+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa47c003600 con 0x7fa494155ef0 2023-10-26T05:53:26.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.636+0000 7fa4917fa640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa474066470 0x7fa474068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.636+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa47c084bb0 con 0x7fa494155ef0 2023-10-26T05:53:26.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.636+0000 7fa493fff640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa474066470 0x7fa474068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.636+0000 7fa493fff640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa474066470 0x7fa474068930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa484006fd0 tx=0x7fa484006a30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.640+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa47c052030 con 0x7fa494155ef0 2023-10-26T05:53:26.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.788+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa49404da80 con 0x7fa494155ef0 2023-10-26T05:53:26.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.788+0000 7fa4917fa640 1 -- 172.21.15.71:0/34233587 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa47c055650 con 0x7fa494155ef0 2023-10-26T05:53:26.797 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:53:26.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa474066470 msgr2=0x7fa474068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa474066470 0x7fa474068930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa484006fd0 tx=0x7fa484006a30 comp rx=0 tx=0).stop 2023-10-26T05:53:26.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 msgr2=0x7fa494156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 0x7fa494156370 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00fa00 tx=0x7fa47c00fec0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 shutdown_connections 2023-10-26T05:53:26.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa494152370 0x7fa494152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa474066470 0x7fa474068930 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa494155ef0 0x7fa494156370 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 --2- 172.21.15.71:0/34233587 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa480004e30 0x7fa49414e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 >> 172.21.15.71:0/34233587 conn(0x7fa4940c97f0 msgr2=0x7fa49406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 shutdown_connections 2023-10-26T05:53:26.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.792+0000 7fa499779640 1 -- 172.21.15.71:0/34233587 wait complete. 2023-10-26T05:53:26.815 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:53:26.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/1019258813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f099415d3f0 msgr2=0x7f09940cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 --2- 172.21.15.71:0/1019258813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f099415d3f0 0x7f09940cb820 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0984009840 tx=0x7f098401c700 comp rx=0 tx=0).stop 2023-10-26T05:53:26.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/1019258813 shutdown_connections 2023-10-26T05:53:26.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 --2- 172.21.15.71:0/1019258813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f099415d3f0 0x7f09940cb820 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/1019258813 >> 172.21.15.71:0/1019258813 conn(0x7f09940aec60 msgr2=0x7f09940af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:26.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/1019258813 shutdown_connections 2023-10-26T05:53:26.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/1019258813 wait complete. 2023-10-26T05:53:26.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 Processor -- start 2023-10-26T05:53:26.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- start start 2023-10-26T05:53:26.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 0x7f099413b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099413c220 0x7f09941327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f099415d3f0 0x7f099414e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09941357f0 con 0x7f09940cbd60 2023-10-26T05:53:26.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0994132cf0 con 0x7f099413c220 2023-10-26T05:53:26.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099413a9c0 con 0x7f099415d3f0 2023-10-26T05:53:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 0x7f099413b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 0x7f099413b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37426/0 (socket says 172.21.15.71:37426) 2023-10-26T05:53:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 -- 172.21.15.71:0/2236361125 learned_addr learned my addr 172.21.15.71:0/2236361125 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 -- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f099415d3f0 msgr2=0x7f099414e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f099415d3f0 0x7f099414e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 -- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099413c220 msgr2=0x7f09941327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099413c220 0x7f09941327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 -- 172.21.15.71:0/2236361125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840094a0 con 0x7f09940cbd60 2023-10-26T05:53:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f0999d65640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 0x7f099413b4c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f099000c480 tx=0x7f099000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990012830 con 0x7f09940cbd60 2023-10-26T05:53:26.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0990012a80 con 0x7f09940cbd60 2023-10-26T05:53:26.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990011380 con 0x7f09940cbd60 2023-10-26T05:53:26.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099414ddf0 con 0x7f09940cbd60 2023-10-26T05:53:26.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.924+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099414e3b0 con 0x7f09940cbd60 2023-10-26T05:53:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.928+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0994066b10 con 0x7f09940cbd60 2023-10-26T05:53:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.928+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f09900114e0 con 0x7f09940cbd60 2023-10-26T05:53:26.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.928+0000 7f098a7fc640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0978066470 0x7f0978068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.928+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09900848f0 con 0x7f09940cbd60 2023-10-26T05:53:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.928+0000 7f0999564640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0978066470 0x7f0978068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:26.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.928+0000 7f0999564640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0978066470 0x7f0978068930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f09840af000 tx=0x7f09840194a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:26.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:26.932+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0990051f90 con 0x7f09940cbd60 2023-10-26T05:53:27.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.072+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f099414dc20 con 0x7f09940cbd60 2023-10-26T05:53:27.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.072+0000 7f098a7fc640 1 -- 172.21.15.71:0/2236361125 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09900160a0 con 0x7f09940cbd60 2023-10-26T05:53:27.080 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:53:27.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0978066470 msgr2=0x7f0978068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0978066470 0x7f0978068930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f09840af000 tx=0x7f09840194a0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 msgr2=0x7f099413b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 0x7f099413b4c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f099000c480 tx=0x7f099000c940 comp rx=0 tx=0).stop 2023-10-26T05:53:27.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 shutdown_connections 2023-10-26T05:53:27.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f099415d3f0 0x7f099414e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f099413c220 0x7f09941327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0978066470 0x7f0978068930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 --2- 172.21.15.71:0/2236361125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09940cbd60 0x7f099413b4c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 >> 172.21.15.71:0/2236361125 conn(0x7f09940aec60 msgr2=0x7f09940b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:27.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 shutdown_connections 2023-10-26T05:53:27.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.076+0000 7f099bfc9640 1 -- 172.21.15.71:0/2236361125 wait complete. 2023-10-26T05:53:27.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:27.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.200+0000 7f2e5b611640 1 -- 172.21.15.71:0/967645898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54070fe0 msgr2=0x7f2e54071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.200+0000 7f2e5b611640 1 --2- 172.21.15.71:0/967645898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54070fe0 0x7f2e54071470 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2e44009900 tx=0x7f2e4401c810 comp rx=0 tx=0).stop 2023-10-26T05:53:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/967645898 shutdown_connections 2023-10-26T05:53:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 --2- 172.21.15.71:0/967645898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54070fe0 0x7f2e54071470 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 --2- 172.21.15.71:0/967645898 >> v2:172.21.15.86:3300/0 conn(0x7f2e54140cc0 0x7f2e5415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/967645898 >> 172.21.15.71:0/967645898 conn(0x7f2e54069770 msgr2=0x7f2e54069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/967645898 shutdown_connections 2023-10-26T05:53:27.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/967645898 wait complete. 2023-10-26T05:53:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 Processor -- start 2023-10-26T05:53:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- start start 2023-10-26T05:53:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e54070fe0 0x7f2e5413f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e540b0040 0x7f2e54136930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 0x7f2e541351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e541597c0 con 0x7f2e54140cc0 2023-10-26T05:53:27.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54158bb0 con 0x7f2e540b0040 2023-10-26T05:53:27.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e54158e90 con 0x7f2e54070fe0 2023-10-26T05:53:27.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 0x7f2e541351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 0x7f2e541351a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37446/0 (socket says 172.21.15.71:37446) 2023-10-26T05:53:27.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 -- 172.21.15.71:0/2041221322 learned_addr learned my addr 172.21.15.71:0/2041221322 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:27.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e58bac640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e540b0040 0x7f2e54136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e593ad640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e54070fe0 0x7f2e5413f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 -- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e54070fe0 msgr2=0x7f2e5413f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e54070fe0 0x7f2e5413f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 -- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e540b0040 msgr2=0x7f2e54136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e540b0040 0x7f2e54136930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 -- 172.21.15.71:0/2041221322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44009560 con 0x7f2e54140cc0 2023-10-26T05:53:27.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e59bae640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 0x7f2e541351a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c780 tx=0x7f2e4c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:27.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004b70 con 0x7f2e54140cc0 2023-10-26T05:53:27.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e4c00ce90 con 0x7f2e54140cc0 2023-10-26T05:53:27.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e54159170 con 0x7f2e54140cc0 2023-10-26T05:53:27.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00aab0 con 0x7f2e54140cc0 2023-10-26T05:53:27.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e541587d0 con 0x7f2e54140cc0 2023-10-26T05:53:27.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.204+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e541f2fc0 con 0x7f2e54140cc0 2023-10-26T05:53:27.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.208+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2e4c00ac10 con 0x7f2e54140cc0 2023-10-26T05:53:27.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.208+0000 7f2e49ffb640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e38066470 0x7f2e38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.208+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e4c084240 con 0x7f2e54140cc0 2023-10-26T05:53:27.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.208+0000 7f2e593ad640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e38066470 0x7f2e38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.208+0000 7f2e593ad640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e38066470 0x7f2e38068930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b9a0 tx=0x7f2e50005c30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:27.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.212+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2e4c0516c0 con 0x7f2e54140cc0 2023-10-26T05:53:27.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.384+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e54066b10 con 0x7f2e54140cc0 2023-10-26T05:53:27.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.384+0000 7f2e49ffb640 1 -- 172.21.15.71:0/2041221322 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f2e4c054ce0 con 0x7f2e54140cc0 2023-10-26T05:53:27.392 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:27.392 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e38066470 msgr2=0x7f2e38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e38066470 0x7f2e38068930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000b9a0 tx=0x7f2e50005c30 comp rx=0 tx=0).stop 2023-10-26T05:53:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 msgr2=0x7f2e541351a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 0x7f2e541351a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c780 tx=0x7f2e4c00cc40 comp rx=0 tx=0).stop 2023-10-26T05:53:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 shutdown_connections 2023-10-26T05:53:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e38066470 0x7f2e38068930 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e54140cc0 0x7f2e541351a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e540b0040 0x7f2e54136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 --2- 172.21.15.71:0/2041221322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e54070fe0 0x7f2e5413f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 >> 172.21.15.71:0/2041221322 conn(0x7f2e54069770 msgr2=0x7f2e540707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 shutdown_connections 2023-10-26T05:53:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.388+0000 7f2e5b611640 1 -- 172.21.15.71:0/2041221322 wait complete. 2023-10-26T05:53:27.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:27.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.512+0000 7fa56a97f640 1 -- 172.21.15.71:0/145023254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640b2b70 msgr2=0x7fa56406d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.512+0000 7fa56a97f640 1 --2- 172.21.15.71:0/145023254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640b2b70 0x7fa56406d250 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009c80 tx=0x7fa55c01cd40 comp rx=0 tx=0).stop 2023-10-26T05:53:27.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/145023254 shutdown_connections 2023-10-26T05:53:27.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 --2- 172.21.15.71:0/145023254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640b2b70 0x7fa56406d250 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 --2- 172.21.15.71:0/145023254 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640cd310 0x7fa56406cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/145023254 >> 172.21.15.71:0/145023254 conn(0x7fa564070650 msgr2=0x7fa564070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:27.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/145023254 shutdown_connections 2023-10-26T05:53:27.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/145023254 wait complete. 2023-10-26T05:53:27.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 Processor -- start 2023-10-26T05:53:27.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- start start 2023-10-26T05:53:27.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640b2b70 0x7fa56414af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 0x7fa56414b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5641583d0 0x7fa5640c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5641490b0 con 0x7fa5640cd310 2023-10-26T05:53:27.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa564149390 con 0x7fa5641583d0 2023-10-26T05:53:27.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa564149670 con 0x7fa5640b2b70 2023-10-26T05:53:27.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 0x7fa56414b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 0x7fa56414b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37470/0 (socket says 172.21.15.71:37470) 2023-10-26T05:53:27.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 -- 172.21.15.71:0/1286768301 learned_addr learned my addr 172.21.15.71:0/1286768301 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:27.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa568f1c640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5641583d0 0x7fa5640c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa563fff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640b2b70 0x7fa56414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 -- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640b2b70 msgr2=0x7fa56414af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640b2b70 0x7fa56414af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 -- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5641583d0 msgr2=0x7fa5640c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5641583d0 0x7fa5640c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 -- 172.21.15.71:0/1286768301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0098e0 con 0x7fa5640cd310 2023-10-26T05:53:27.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa568f1c640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5641583d0 0x7fa5640c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:27.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa563fff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640b2b70 0x7fa56414af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:27.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa55bfff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 0x7fa56414b740 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c970 tx=0x7fa54c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:27.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c007bb0 con 0x7fa5640cd310 2023-10-26T05:53:27.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa54c007e20 con 0x7fa5640cd310 2023-10-26T05:53:27.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa564149950 con 0x7fa5640cd310 2023-10-26T05:53:27.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c005280 con 0x7fa5640cd310 2023-10-26T05:53:27.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56414bc80 con 0x7fa5640cd310 2023-10-26T05:53:27.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.516+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5640adc50 con 0x7fa5640cd310 2023-10-26T05:53:27.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.520+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa54c0053e0 con 0x7fa5640cd310 2023-10-26T05:53:27.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.520+0000 7fa5617fa640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5480664a0 0x7fa548068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.520+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa54c084720 con 0x7fa5640cd310 2023-10-26T05:53:27.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.520+0000 7fa563fff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5480664a0 0x7fa548068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.520+0000 7fa563fff640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5480664a0 0x7fa548068960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa55400a850 tx=0x7fa554008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:27.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.524+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa5640adc50 con 0x7fa5640cd310 2023-10-26T05:53:27.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.684+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5640adc50 con 0x7fa5640cd310 2023-10-26T05:53:27.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa5617fa640 1 -- 172.21.15.71:0/1286768301 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16443 (secure 0 0 0) 0x7fa5640adc50 con 0x7fa5640cd310 2023-10-26T05:53:27.694 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:27.695 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":127,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:20.189455+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5480664a0 msgr2=0x7fa548068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5480664a0 0x7fa548068960 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa55400a850 tx=0x7fa554008040 comp rx=0 tx=0).stop 2023-10-26T05:53:27.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 msgr2=0x7fa56414b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 0x7fa56414b740 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c970 tx=0x7fa54c00ce30 comp rx=0 tx=0).stop 2023-10-26T05:53:27.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 shutdown_connections 2023-10-26T05:53:27.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5641583d0 0x7fa5640c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5480664a0 0x7fa548068960 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5640cd310 0x7fa56414b740 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 --2- 172.21.15.71:0/1286768301 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5640b2b70 0x7fa56414af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 >> 172.21.15.71:0/1286768301 conn(0x7fa564070650 msgr2=0x7fa5640b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:27.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 shutdown_connections 2023-10-26T05:53:27.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.688+0000 7fa56a97f640 1 -- 172.21.15.71:0/1286768301 wait complete. 2023-10-26T05:53:27.714 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:53:27.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:27.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.816+0000 7f60ee6c1640 1 -- 172.21.15.71:0/1392782955 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80cb1e0 msgr2=0x7f60e8159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.816+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/1392782955 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80cb1e0 0x7f60e8159810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009bc0 tx=0x7f60e001cb80 comp rx=0 tx=0).stop 2023-10-26T05:53:27.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/1392782955 shutdown_connections 2023-10-26T05:53:27.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/1392782955 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80cb1e0 0x7f60e8159810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/1392782955 >> v2:172.21.15.86:3300/0 conn(0x7f60e813c2f0 0x7f60e80b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/1392782955 >> 172.21.15.71:0/1392782955 conn(0x7f60e8069770 msgr2=0x7f60e8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:27.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/1392782955 shutdown_connections 2023-10-26T05:53:27.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/1392782955 wait complete. 2023-10-26T05:53:27.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 Processor -- start 2023-10-26T05:53:27.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- start start 2023-10-26T05:53:27.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80b2890 0x7f60e813a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 0x7f60e8157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e813c2f0 0x7f60e81373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e806a1f0 con 0x7f60e80cb1e0 2023-10-26T05:53:27.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e80b05a0 con 0x7f60e813c2f0 2023-10-26T05:53:27.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e813b700 con 0x7f60e80b2890 2023-10-26T05:53:27.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 0x7f60e8157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 0x7f60e8157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37476/0 (socket says 172.21.15.71:37476) 2023-10-26T05:53:27.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 -- 172.21.15.71:0/319689290 learned_addr learned my addr 172.21.15.71:0/319689290 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:27.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e7fff640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80b2890 0x7f60e813a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ecc5e640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e813c2f0 0x7f60e81373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 -- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80b2890 msgr2=0x7f60e813a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80b2890 0x7f60e813a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 -- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e813c2f0 msgr2=0x7f60e81373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:27.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e813c2f0 0x7f60e81373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:27.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 -- 172.21.15.71:0/319689290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009820 con 0x7f60e80cb1e0 2023-10-26T05:53:27.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ecc5e640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e813c2f0 0x7f60e81373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:27.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e77fe640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 0x7f60e8157690 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f60d000c450 tx=0x7f60d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:27.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004230 con 0x7f60e80cb1e0 2023-10-26T05:53:27.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e80b0270 con 0x7f60e80cb1e0 2023-10-26T05:53:27.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f60d00044a0 con 0x7f60e80cb1e0 2023-10-26T05:53:27.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0005260 con 0x7f60e80cb1e0 2023-10-26T05:53:27.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e813b420 con 0x7f60e80cb1e0 2023-10-26T05:53:27.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.820+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e8066b10 con 0x7f60e80cb1e0 2023-10-26T05:53:27.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.824+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f60d00053c0 con 0x7f60e80cb1e0 2023-10-26T05:53:27.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.824+0000 7f60e4ff9640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60c8066470 0x7f60c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:27.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.824+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60d0083bc0 con 0x7f60e80cb1e0 2023-10-26T05:53:27.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.824+0000 7f60e7fff640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60c8066470 0x7f60c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:27.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.824+0000 7f60e7fff640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60c8066470 0x7f60c8068930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009ad0 tx=0x7f60d80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:27.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:27.828+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f60d0051050 con 0x7f60e80cb1e0 2023-10-26T05:53:28.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.000+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e813aed0 con 0x7f60e80cb1e0 2023-10-26T05:53:28.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.000+0000 7f60e4ff9640 1 -- 172.21.15.71:0/319689290 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f60d0054670 con 0x7f60e80cb1e0 2023-10-26T05:53:28.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:28.008 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:28.008 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:28.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60c8066470 msgr2=0x7f60c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60c8066470 0x7f60c8068930 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009ad0 tx=0x7f60d80091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 msgr2=0x7f60e8157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 0x7f60e8157690 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f60d000c450 tx=0x7f60d000c910 comp rx=0 tx=0).stop 2023-10-26T05:53:28.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 shutdown_connections 2023-10-26T05:53:28.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60e813c2f0 0x7f60e81373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60c8066470 0x7f60c8068930 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60e80cb1e0 0x7f60e8157690 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 --2- 172.21.15.71:0/319689290 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60e80b2890 0x7f60e813a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 >> 172.21.15.71:0/319689290 conn(0x7f60e8069770 msgr2=0x7f60e815a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 shutdown_connections 2023-10-26T05:53:28.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.004+0000 7f60ee6c1640 1 -- 172.21.15.71:0/319689290 wait complete. 2023-10-26T05:53:28.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:28.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/2250016818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec003f30 msgr2=0x7f34ec0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 --2- 172.21.15.71:0/2250016818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34ec003f30 0x7f34ec0047d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f34f80016b0 tx=0x7f34f801c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350359e640 1 -- 172.21.15.71:0/2250016818 reap_dead start 2023-10-26T05:53:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/2250016818 shutdown_connections 2023-10-26T05:53:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/2250016818 >> 172.21.15.71:0/2250016818 conn(0x7f35040b0e00 msgr2=0x7f35040b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/2250016818 shutdown_connections 2023-10-26T05:53:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/2250016818 wait complete. 2023-10-26T05:53:28.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 Processor -- start 2023-10-26T05:53:28.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- start start 2023-10-26T05:53:28.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35041ef070 0x7f3504155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 0x7f35040c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35041553c0 0x7f35040c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3504150fb0 con 0x7f3504155fb0 2023-10-26T05:53:28.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35040a66f0 con 0x7f35041ef070 2023-10-26T05:53:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35041564c0 con 0x7f35041553c0 2023-10-26T05:53:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 0x7f35040c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 0x7f35040c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37504/0 (socket says 172.21.15.71:37504) 2023-10-26T05:53:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 -- 172.21.15.71:0/949869541 learned_addr learned my addr 172.21.15.71:0/949869541 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3502d9d640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35041ef070 0x7f3504155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350359e640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35041553c0 0x7f35040c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 -- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35041553c0 msgr2=0x7f35040c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35041553c0 0x7f35040c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 -- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35041ef070 msgr2=0x7f3504155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35041ef070 0x7f3504155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 -- 172.21.15.71:0/949869541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504154600 con 0x7f3504155fb0 2023-10-26T05:53:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f350259c640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 0x7f35040c4d90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f34f80016b0 tx=0x7f34f8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80a9030 con 0x7f3504155fb0 2023-10-26T05:53:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f34f8003e10 con 0x7f3504155fb0 2023-10-26T05:53:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35041548e0 con 0x7f3504155fb0 2023-10-26T05:53:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80a55a0 con 0x7f3504155fb0 2023-10-26T05:53:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3504154ea0 con 0x7f3504155fb0 2023-10-26T05:53:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.132+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3504066d00 con 0x7f3504155fb0 2023-10-26T05:53:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.136+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34f80b6020 con 0x7f3504155fb0 2023-10-26T05:53:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.136+0000 7f34eb7fe640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34f00664a0 0x7f34f0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.136+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34f80aa070 con 0x7f3504155fb0 2023-10-26T05:53:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.136+0000 7f3502d9d640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34f00664a0 0x7f34f0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.136+0000 7f3502d9d640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34f00664a0 0x7f34f0068960 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009ab0 tx=0x7f34f40091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.140+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f34f80eb400 con 0x7f3504155fb0 2023-10-26T05:53:28.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.312+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35040b03b0 con 0x7f3504155fb0 2023-10-26T05:53:28.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f34eb7fe640 1 -- 172.21.15.71:0/949869541 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f34f80be070 con 0x7f3504155fb0 2023-10-26T05:53:28.322 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:28.322 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":44,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:06.736075+0000","modified":"2023-10-26T05:53:08.786697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7545},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7545":{"gid":7545,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1526458745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1526458745},{"type":"v1","addr":"172.21.15.71:6835","nonce":1526458745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-26T05:53:28.323 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 44 2023-10-26T05:53:28.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34f00664a0 msgr2=0x7f34f0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34f00664a0 0x7f34f0068960 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009ab0 tx=0x7f34f40091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 msgr2=0x7f35040c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 0x7f35040c4d90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f34f80016b0 tx=0x7f34f8003c20 comp rx=0 tx=0).stop 2023-10-26T05:53:28.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 shutdown_connections 2023-10-26T05:53:28.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f35041553c0 0x7f35040c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f34f00664a0 0x7f34f0068960 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3504155fb0 0x7f35040c4d90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 --2- 172.21.15.71:0/949869541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35041ef070 0x7f3504155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 >> 172.21.15.71:0/949869541 conn(0x7f35040b0e00 msgr2=0x7f35040b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 shutdown_connections 2023-10-26T05:53:28.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.316+0000 7f3509223640 1 -- 172.21.15.71:0/949869541 wait complete. 2023-10-26T05:53:28.339 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:28.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3938357118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f370813c2f0 msgr2=0x7f37080b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- 172.21.15.71:0/3938357118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f370813c2f0 0x7f37080b2350 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f37080b32d0 tx=0x7f370401c5e0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3938357118 shutdown_connections 2023-10-26T05:53:28.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- 172.21.15.71:0/3938357118 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37080b2890 0x7f37080caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- 172.21.15.71:0/3938357118 >> v2:172.21.15.86:3300/0 conn(0x7f37080cb010 0x7f370806f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- 172.21.15.71:0/3938357118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f370813c2f0 0x7f37080b2350 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3938357118 >> 172.21.15.71:0/3938357118 conn(0x7f3708069770 msgr2=0x7f3708069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3938357118 shutdown_connections 2023-10-26T05:53:28.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3938357118 wait complete. 2023-10-26T05:53:28.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 Processor -- start 2023-10-26T05:53:28.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- start start 2023-10-26T05:53:28.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37080b2890 0x7f370815b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 0x7f370815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f370813c2f0 0x7f3708132540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708151090 con 0x7f37080cb010 2023-10-26T05:53:28.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370815c2c0 con 0x7f37080b2890 2023-10-26T05:53:28.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3708132240 con 0x7f370813c2f0 2023-10-26T05:53:28.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 0x7f370815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 0x7f370815bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37532/0 (socket says 172.21.15.71:37532) 2023-10-26T05:53:28.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 -- 172.21.15.71:0/3567944195 learned_addr learned my addr 172.21.15.71:0/3567944195 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:28.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370e393640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37080b2890 0x7f370815b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370eb94640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f370813c2f0 0x7f3708132540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 -- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f370813c2f0 msgr2=0x7f3708132540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f370813c2f0 0x7f3708132540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 -- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37080b2890 msgr2=0x7f370815b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37080b2890 0x7f370815b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 -- 172.21.15.71:0/3567944195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3704009560 con 0x7f37080cb010 2023-10-26T05:53:28.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370e393640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37080b2890 0x7f370815b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:28.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370eb94640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f370813c2f0 0x7f3708132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:28.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f370db92640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 0x7f370815bd80 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d750 tx=0x7f36f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:28.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8009c30 con 0x7f37080cb010 2023-10-26T05:53:28.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f36f8009e80 con 0x7f37080cb010 2023-10-26T05:53:28.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370814fa00 con 0x7f37080cb010 2023-10-26T05:53:28.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8005280 con 0x7f37080cb010 2023-10-26T05:53:28.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.460+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370815c780 con 0x7f37080cb010 2023-10-26T05:53:28.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.464+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3708066b10 con 0x7f37080cb010 2023-10-26T05:53:28.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.464+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f36f80053e0 con 0x7f37080cb010 2023-10-26T05:53:28.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.464+0000 7f36feffd640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f36ec0664a0 0x7f36ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.464+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36f8083f10 con 0x7f37080cb010 2023-10-26T05:53:28.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.464+0000 7f370e393640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f36ec0664a0 0x7f36ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.464+0000 7f370e393640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f36ec0664a0 0x7f36ec068960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37040ae000 tx=0x7f370401ae20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:28.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.468+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f36f8051420 con 0x7f37080cb010 2023-10-26T05:53:28.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.624+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37081511d0 con 0x7f37080cb010 2023-10-26T05:53:28.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.624+0000 7f36feffd640 1 -- 172.21.15.71:0/3567944195 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16443 (secure 0 0 0) 0x7f37081511d0 con 0x7f37080cb010 2023-10-26T05:53:28.632 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:28.633 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":127,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:20.189455+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:04.827214+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:05.732853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:28.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f36ec0664a0 msgr2=0x7f36ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f36ec0664a0 0x7f36ec068960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37040ae000 tx=0x7f370401ae20 comp rx=0 tx=0).stop 2023-10-26T05:53:28.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 msgr2=0x7f370815bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 0x7f370815bd80 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f36f800d750 tx=0x7f36f800dc10 comp rx=0 tx=0).stop 2023-10-26T05:53:28.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 shutdown_connections 2023-10-26T05:53:28.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f370813c2f0 0x7f3708132540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f36ec0664a0 0x7f36ec068960 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37080cb010 0x7f370815bd80 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 --2- 172.21.15.71:0/3567944195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37080b2890 0x7f370815b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 >> 172.21.15.71:0/3567944195 conn(0x7f3708069770 msgr2=0x7f3708137fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 shutdown_connections 2023-10-26T05:53:28.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.628+0000 7f37105f7640 1 -- 172.21.15.71:0/3567944195 wait complete. 2023-10-26T05:53:28.652 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:53:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3481807435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb294005190 msgr2=0x7fb29400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3481807435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb294005190 0x7fb29400b990 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009b30 tx=0x7fb2a401d250 comp rx=0 tx=0).stop 2023-10-26T05:53:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3481807435 shutdown_connections 2023-10-26T05:53:28.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3481807435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb294005190 0x7fb29400b990 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3481807435 >> 172.21.15.71:0/3481807435 conn(0x7fb2a806b8c0 msgr2=0x7fb2a806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3481807435 shutdown_connections 2023-10-26T05:53:28.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3481807435 wait complete. 2023-10-26T05:53:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 Processor -- start 2023-10-26T05:53:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- start start 2023-10-26T05:53:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb294005190 0x7fb2a8071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a813cd00 0x7fb2a813aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 0x7fb2a8158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8158ba0 con 0x7fb2a81581b0 2023-10-26T05:53:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a813b5d0 con 0x7fb2a813cd00 2023-10-26T05:53:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a813d180 con 0x7fb294005190 2023-10-26T05:53:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 0x7fb2a8158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 0x7fb2a8158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37556/0 (socket says 172.21.15.71:37556) 2023-10-26T05:53:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 -- 172.21.15.71:0/3553123494 learned_addr learned my addr 172.21.15.71:0/3553123494 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2adeda640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb294005190 0x7fb2a8071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ad6d9640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a813cd00 0x7fb2a813aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 -- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb294005190 msgr2=0x7fb2a8071210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb294005190 0x7fb2a8071210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 -- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a813cd00 msgr2=0x7fb2a813aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a813cd00 0x7fb2a813aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 -- 172.21.15.71:0/3553123494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a81f5340 con 0x7fb2a81581b0 2023-10-26T05:53:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2ae6db640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 0x7fb2a8158660 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009b30 tx=0x7fb2a4005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40bb070 con 0x7fb2a81581b0 2023-10-26T05:53:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2a40046d0 con 0x7fb2a81581b0 2023-10-26T05:53:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a81f5620 con 0x7fb2a81581b0 2023-10-26T05:53:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4004880 con 0x7fb2a81581b0 2023-10-26T05:53:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.772+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a81f5be0 con 0x7fb2a81581b0 2023-10-26T05:53:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.776+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a8066b10 con 0x7fb2a81581b0 2023-10-26T05:53:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.776+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2a40049e0 con 0x7fb2a81581b0 2023-10-26T05:53:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.776+0000 7fb29a7fc640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb28c066470 0x7fb28c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.776+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2a411e3d0 con 0x7fb2a81581b0 2023-10-26T05:53:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.776+0000 7fb2adeda640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb28c066470 0x7fb28c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.776+0000 7fb2adeda640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb28c066470 0x7fb28c068930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb2a81536b0 tx=0x7fb2a000c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.780+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2a40eb8e0 con 0x7fb2a81581b0 2023-10-26T05:53:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.960+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb2a81f5dd0 con 0x7fb2a81581b0 2023-10-26T05:53:28.966 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:53:28.960+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:53:28.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.960+0000 7fb29a7fc640 1 -- 172.21.15.71:0/3553123494 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb2a40eef00 con 0x7fb2a81581b0 2023-10-26T05:53:28.969 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:53:28.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb28c066470 msgr2=0x7fb28c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb28c066470 0x7fb28c068930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb2a81536b0 tx=0x7fb2a000c040 comp rx=0 tx=0).stop 2023-10-26T05:53:28.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 msgr2=0x7fb2a8158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:28.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 0x7fb2a8158660 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009b30 tx=0x7fb2a4005680 comp rx=0 tx=0).stop 2023-10-26T05:53:28.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 shutdown_connections 2023-10-26T05:53:28.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb28c066470 0x7fb28c068930 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2a81581b0 0x7fb2a8158660 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a813cd00 0x7fb2a813aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 --2- 172.21.15.71:0/3553123494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb294005190 0x7fb2a8071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:28.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 >> 172.21.15.71:0/3553123494 conn(0x7fb2a806b8c0 msgr2=0x7fb2a815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:28.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 shutdown_connections 2023-10-26T05:53:28.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:28.968+0000 7fb2b013e640 1 -- 172.21.15.71:0/3553123494 wait complete. 2023-10-26T05:53:28.977 INFO:tasks.ceph.mds.a.smithi071.stderr: -3147> 2023-10-26T05:53:12.368+0000 7f22f6239640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:53:28.991 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:53:29.014 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:53:29.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 -- 172.21.15.71:0/1713545629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac13c220 msgr2=0x7fadac15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:29.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1713545629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac13c220 0x7fadac15cce0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009840 tx=0x7fad9c01c700 comp rx=0 tx=0).stop 2023-10-26T05:53:29.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 -- 172.21.15.71:0/1713545629 shutdown_connections 2023-10-26T05:53:29.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1713545629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac13c220 0x7fadac15cce0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 -- 172.21.15.71:0/1713545629 >> 172.21.15.71:0/1713545629 conn(0x7fadac0aec60 msgr2=0x7fadac0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:29.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 -- 172.21.15.71:0/1713545629 shutdown_connections 2023-10-26T05:53:29.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 -- 172.21.15.71:0/1713545629 wait complete. 2023-10-26T05:53:29.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 Processor -- start 2023-10-26T05:53:29.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.100+0000 7fadb12fc640 1 -- start start 2023-10-26T05:53:29.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 0x7fadac1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:29.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadac13c220 0x7fadac0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:29.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadac15d220 0x7fadac134ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:29.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac14a650 con 0x7fadac0cbf30 2023-10-26T05:53:29.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac149a40 con 0x7fadac15d220 2023-10-26T05:53:29.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac149d20 con 0x7fadac13c220 2023-10-26T05:53:29.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 0x7fadac1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:29.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 0x7fadac1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37596/0 (socket says 172.21.15.71:37596) 2023-10-26T05:53:29.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 -- 172.21.15.71:0/1112313567 learned_addr learned my addr 172.21.15.71:0/1112313567 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:29.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 -- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadac13c220 msgr2=0x7fadac0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:29.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadac13c220 0x7fadac0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 -- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadac15d220 msgr2=0x7fadac134ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:29.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadac15d220 0x7fadac134ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 -- 172.21.15.71:0/1112313567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c0094a0 con 0x7fadac0cbf30 2023-10-26T05:53:29.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaaffd640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 0x7fadac1534e0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009790 tx=0x7fad9c0a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:29.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0042c0 con 0x7fadac0cbf30 2023-10-26T05:53:29.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fad9c004510 con 0x7fadac0cbf30 2023-10-26T05:53:29.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac14a000 con 0x7fadac0cbf30 2023-10-26T05:53:29.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c005280 con 0x7fadac0cbf30 2023-10-26T05:53:29.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac153a20 con 0x7fadac0cbf30 2023-10-26T05:53:29.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac066b10 con 0x7fadac0cbf30 2023-10-26T05:53:29.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fad9c0b4040 con 0x7fadac0cbf30 2023-10-26T05:53:29.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fad937fe640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad98066470 0x7fad98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fad9c11e230 con 0x7fadac0cbf30 2023-10-26T05:53:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.104+0000 7fadaa7fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad98066470 0x7fad98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.108+0000 7fadaa7fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad98066470 0x7fad98068930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fadac153070 tx=0x7fada0006bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.108+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fad9c0eb6e0 con 0x7fadac0cbf30 2023-10-26T05:53:29.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.284+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fadac06b0c0 con 0x7fadac0cbf30 2023-10-26T05:53:29.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.963+0000 7fad937fe640 1 -- 172.21.15.71:0/1112313567 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7fad9c0b20a0 con 0x7fadac0cbf30 2023-10-26T05:53:29.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad98066470 msgr2=0x7fad98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:29.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad98066470 0x7fad98068930 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fadac153070 tx=0x7fada0006bd0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 msgr2=0x7fadac1534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:29.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 0x7fadac1534e0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009790 tx=0x7fad9c0a57a0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 shutdown_connections 2023-10-26T05:53:29.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadac15d220 0x7fadac134ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadac13c220 0x7fadac0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fad98066470 0x7fad98068930 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 --2- 172.21.15.71:0/1112313567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadac0cbf30 0x7fadac1534e0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 >> 172.21.15.71:0/1112313567 conn(0x7fadac0aec60 msgr2=0x7fadac0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:29.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 shutdown_connections 2023-10-26T05:53:29.978 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:29.967+0000 7fadb12fc640 1 -- 172.21.15.71:0/1112313567 wait complete. 2023-10-26T05:53:29.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:53:30.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ebf7e640 1 -- 172.21.15.71:0/2443084278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d0004d10 msgr2=0x7f40d0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/2443084278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d0004d10 0x7f40d0005190 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f40e00016e0 tx=0x7f40e001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:53:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ea51b640 1 -- 172.21.15.71:0/2443084278 reap_dead start 2023-10-26T05:53:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ebf7e640 1 -- 172.21.15.71:0/2443084278 shutdown_connections 2023-10-26T05:53:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ebf7e640 1 -- 172.21.15.71:0/2443084278 >> 172.21.15.71:0/2443084278 conn(0x7f40e40aec60 msgr2=0x7f40e40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ebf7e640 1 -- 172.21.15.71:0/2443084278 shutdown_connections 2023-10-26T05:53:30.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.167+0000 7f40ebf7e640 1 -- 172.21.15.71:0/2443084278 wait complete. 2023-10-26T05:53:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 Processor -- start 2023-10-26T05:53:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- start start 2023-10-26T05:53:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40d0004d10 0x7f40e4136130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 0x7f40e4135580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40e415b460 0x7f40e4135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e413a3f0 con 0x7f40e415afe0 2023-10-26T05:53:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e41397e0 con 0x7f40e415b460 2023-10-26T05:53:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4139ac0 con 0x7f40d0004d10 2023-10-26T05:53:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 0x7f40e4135580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 0x7f40e4135580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37614/0 (socket says 172.21.15.71:37614) 2023-10-26T05:53:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 -- 172.21.15.71:0/3160573893 learned_addr learned my addr 172.21.15.71:0/3160573893 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9d1a640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40d0004d10 0x7f40e4136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ea51b640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40e415b460 0x7f40e4135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 -- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40d0004d10 msgr2=0x7f40e4136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40d0004d10 0x7f40e4136130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 -- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40e415b460 msgr2=0x7f40e4135ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40e415b460 0x7f40e4135ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 -- 172.21.15.71:0/3160573893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e4139da0 con 0x7f40e415afe0 2023-10-26T05:53:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40e9519640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 0x7f40e4135580 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f40d800d780 tx=0x7f40d800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d800de90 con 0x7f40e415afe0 2023-10-26T05:53:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f40d80052e0 con 0x7f40e415afe0 2023-10-26T05:53:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e413a080 con 0x7f40e415afe0 2023-10-26T05:53:30.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d800b920 con 0x7f40e415afe0 2023-10-26T05:53:30.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e4136840 con 0x7f40e415afe0 2023-10-26T05:53:30.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.171+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e4066b10 con 0x7f40e415afe0 2023-10-26T05:53:30.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.175+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f40d800ba80 con 0x7f40e415afe0 2023-10-26T05:53:30.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.175+0000 7f40d67fc640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f40c80664a0 0x7f40c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:30.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.175+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f40d8083b50 con 0x7f40e415afe0 2023-10-26T05:53:30.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.175+0000 7f40e9d1a640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f40c80664a0 0x7f40c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:30.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.175+0000 7f40e9d1a640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f40c80664a0 0x7f40c8068960 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f40dc005c50 tx=0x7f40dc0094a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:30.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.179+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f40d8051000 con 0x7f40e415afe0 2023-10-26T05:53:30.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:30.323+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f40e40721e0 con 0x7f40e415afe0 2023-10-26T05:53:31.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.275+0000 7f40d67fc640 1 -- 172.21.15.71:0/3160573893 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7f40d80190d0 con 0x7f40e415afe0 2023-10-26T05:53:31.284 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:53:31.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f40c80664a0 msgr2=0x7f40c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:31.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f40c80664a0 0x7f40c8068960 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f40dc005c50 tx=0x7f40dc0094a0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 msgr2=0x7f40e4135580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 0x7f40e4135580 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f40d800d780 tx=0x7f40d800dc40 comp rx=0 tx=0).stop 2023-10-26T05:53:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 shutdown_connections 2023-10-26T05:53:31.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40e415b460 0x7f40e4135ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f40c80664a0 0x7f40c8068960 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40e415afe0 0x7f40e4135580 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 --2- 172.21.15.71:0/3160573893 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40d0004d10 0x7f40e4136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.279+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 >> 172.21.15.71:0/3160573893 conn(0x7f40e40aec60 msgr2=0x7f40e4158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:31.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.283+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 shutdown_connections 2023-10-26T05:53:31.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.283+0000 7f40ebf7e640 1 -- 172.21.15.71:0/3160573893 wait complete. 2023-10-26T05:53:31.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:53:31.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 -- 172.21.15.71:0/1824323279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 msgr2=0x7f0be00c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 --2- 172.21.15.71:0/1824323279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be00c9970 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f0be0071450 tx=0x7f0bd401c830 comp rx=0 tx=0).stop 2023-10-26T05:53:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 -- 172.21.15.71:0/1824323279 shutdown_connections 2023-10-26T05:53:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 --2- 172.21.15.71:0/1824323279 >> v2:172.21.15.86:3300/0 conn(0x7f0be0070de0 0x7f0be00b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 --2- 172.21.15.71:0/1824323279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be00c9970 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 -- 172.21.15.71:0/1824323279 >> 172.21.15.71:0/1824323279 conn(0x7f0be00aec60 msgr2=0x7f0be00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 -- 172.21.15.71:0/1824323279 shutdown_connections 2023-10-26T05:53:31.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.435+0000 7f0be6f62640 1 -- 172.21.15.71:0/1824323279 wait complete. 2023-10-26T05:53:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 Processor -- start 2023-10-26T05:53:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 -- start start 2023-10-26T05:53:31.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be014c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0be0070de0 0x7f0be013fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be00c9eb0 0x7f0be013f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0140fc0 con 0x7f0be0069770 2023-10-26T05:53:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0140440 con 0x7f0be00c9eb0 2023-10-26T05:53:31.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0140720 con 0x7f0be0070de0 2023-10-26T05:53:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be014c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be014c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37630/0 (socket says 172.21.15.71:37630) 2023-10-26T05:53:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 -- 172.21.15.71:0/2516566633 learned_addr learned my addr 172.21.15.71:0/2516566633 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:31.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0bdffff640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0be0070de0 0x7f0be013fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 -- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0be0070de0 msgr2=0x7f0be013fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0be0070de0 0x7f0be013fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 -- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be00c9eb0 msgr2=0x7f0be013f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be00c9eb0 0x7f0be013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:31.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 -- 172.21.15.71:0/2516566633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4009df0 con 0x7f0be0069770 2023-10-26T05:53:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0bdffff640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0be0070de0 0x7f0be013fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be4cfe640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be014c5e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009340 tx=0x7f0bd40a7ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40a58f0 con 0x7f0be0069770 2023-10-26T05:53:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0bd40046d0 con 0x7f0be0069770 2023-10-26T05:53:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4004aa0 con 0x7f0be0069770 2023-10-26T05:53:31.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be0140a00 con 0x7f0be0069770 2023-10-26T05:53:31.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.439+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be0140150 con 0x7f0be0069770 2023-10-26T05:53:31.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.443+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be0066b10 con 0x7f0be0069770 2023-10-26T05:53:31.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.443+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0bd4004c00 con 0x7f0be0069770 2023-10-26T05:53:31.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.443+0000 7f0bdd7fa640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bc4066470 0x7f0bc4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:31.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.443+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f0bd4121090 con 0x7f0be0069770 2023-10-26T05:53:31.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.443+0000 7f0bdffff640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bc4066470 0x7f0bc4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:31.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.443+0000 7f0bdffff640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bc4066470 0x7f0bc4068930 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0be0070940 tx=0x7f0bcc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:31.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.447+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0bd40ee790 con 0x7f0be0069770 2023-10-26T05:53:31.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:31.599+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0be0066b10 con 0x7f0be0069770 2023-10-26T05:53:32.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.279+0000 7f0bdd7fa640 1 -- 172.21.15.71:0/2516566633 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0bd40b20d0 con 0x7f0be0069770 2023-10-26T05:53:32.287 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:53:32.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bc4066470 msgr2=0x7f0bc4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bc4066470 0x7f0bc4068930 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0be0070940 tx=0x7f0bcc008040 comp rx=0 tx=0).stop 2023-10-26T05:53:32.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 msgr2=0x7f0be014c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be014c5e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009340 tx=0x7f0bd40a7ee0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 shutdown_connections 2023-10-26T05:53:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be00c9eb0 0x7f0be013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0be0070de0 0x7f0be013fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0bc4066470 0x7f0bc4068930 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 --2- 172.21.15.71:0/2516566633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0be0069770 0x7f0be014c5e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 >> 172.21.15.71:0/2516566633 conn(0x7f0be00aec60 msgr2=0x7f0be0153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:32.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 shutdown_connections 2023-10-26T05:53:32.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.283+0000 7f0be6f62640 1 -- 172.21.15.71:0/2516566633 wait complete. 2023-10-26T05:53:32.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:53:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 -- 172.21.15.71:0/592996336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a0004d10 msgr2=0x7f03a0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/592996336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a0004d10 0x7f03a0005190 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c590 tx=0x7f03b001fa80 comp rx=0 tx=0).stop 2023-10-26T05:53:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03b7fff640 1 -- 172.21.15.71:0/592996336 reap_dead start 2023-10-26T05:53:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 -- 172.21.15.71:0/592996336 shutdown_connections 2023-10-26T05:53:32.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 -- 172.21.15.71:0/592996336 >> 172.21.15.71:0/592996336 conn(0x7f03b806fd90 msgr2=0x7f03b80b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 -- 172.21.15.71:0/592996336 shutdown_connections 2023-10-26T05:53:32.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 -- 172.21.15.71:0/592996336 wait complete. 2023-10-26T05:53:32.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 Processor -- start 2023-10-26T05:53:32.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.439+0000 7f03bdbd2640 1 -- start start 2023-10-26T05:53:32.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 0x7f03b80c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03b80c6aa0 0x7f03b81f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b80c8cb0 0x7f03b81f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c4ac0 con 0x7f03b80c88d0 2023-10-26T05:53:32.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c3e40 con 0x7f03b80c8cb0 2023-10-26T05:53:32.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c4120 con 0x7f03b80c6aa0 2023-10-26T05:53:32.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 0x7f03b80c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 0x7f03b80c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34490/0 (socket says 172.21.15.71:34490) 2023-10-26T05:53:32.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 -- 172.21.15.71:0/2174420702 learned_addr learned my addr 172.21.15.71:0/2174420702 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:32.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b7fff640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b80c8cb0 0x7f03b81f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b6ffd640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03b80c6aa0 0x7f03b81f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 -- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03b80c6aa0 msgr2=0x7f03b81f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03b80c6aa0 0x7f03b81f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 -- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b80c8cb0 msgr2=0x7f03b81f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b80c8cb0 0x7f03b81f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 -- 172.21.15.71:0/2174420702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80c4400 con 0x7f03b80c88d0 2023-10-26T05:53:32.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b77fe640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 0x7f03b80c3820 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f03a8010a10 tx=0x7f03a8010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:32.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800eda0 con 0x7f03b80c88d0 2023-10-26T05:53:32.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f03a8012450 con 0x7f03b80c88d0 2023-10-26T05:53:32.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b80c46e0 con 0x7f03b80c88d0 2023-10-26T05:53:32.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8012840 con 0x7f03b80c88d0 2023-10-26T05:53:32.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b81f4f40 con 0x7f03b80c88d0 2023-10-26T05:53:32.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b806a7b0 con 0x7f03b80c88d0 2023-10-26T05:53:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f03a80129a0 con 0x7f03b80c88d0 2023-10-26T05:53:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f039ffff640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03a40664a0 0x7f03a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03a8016070 con 0x7f03b80c88d0 2023-10-26T05:53:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b6ffd640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03a40664a0 0x7f03a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.443+0000 7f03b6ffd640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03a40664a0 0x7f03a4068960 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f03b806d2f0 tx=0x7f03ac006b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:32.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.447+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f03a80522c0 con 0x7f03b80c88d0 2023-10-26T05:53:32.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.591+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f03b804db60 con 0x7f03b80c88d0 2023-10-26T05:53:32.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.599+0000 7f039ffff640 1 -- 172.21.15.71:0/2174420702 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v30) v1 ==== 71+0+0 (secure 0 0 0) 0x7f03a80558e0 con 0x7f03b80c88d0 2023-10-26T05:53:32.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03a40664a0 msgr2=0x7f03a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03a40664a0 0x7f03a4068960 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f03b806d2f0 tx=0x7f03ac006b60 comp rx=0 tx=0).stop 2023-10-26T05:53:32.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 msgr2=0x7f03b80c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 0x7f03b80c3820 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f03a8010a10 tx=0x7f03a8010ed0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 shutdown_connections 2023-10-26T05:53:32.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03b80c8cb0 0x7f03b81f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03b80c6aa0 0x7f03b81f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03a40664a0 0x7f03a4068960 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 --2- 172.21.15.71:0/2174420702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03b80c88d0 0x7f03b80c3820 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 >> 172.21.15.71:0/2174420702 conn(0x7f03b806fd90 msgr2=0x7f03b80cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:32.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 shutdown_connections 2023-10-26T05:53:32.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.603+0000 7f03bdbd2640 1 -- 172.21.15.71:0/2174420702 wait complete. 2023-10-26T05:53:32.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-10-26T05:53:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 -- 172.21.15.71:0/3101934668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00d0850 msgr2=0x7f2ae015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 --2- 172.21.15.71:0/3101934668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00d0850 0x7f2ae015b3a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009840 tx=0x7f2ad401c770 comp rx=0 tx=0).stop 2023-10-26T05:53:32.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 -- 172.21.15.71:0/3101934668 shutdown_connections 2023-10-26T05:53:32.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 --2- 172.21.15.71:0/3101934668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae00cd880 0x7f2ae015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 --2- 172.21.15.71:0/3101934668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00d0850 0x7f2ae015b3a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 -- 172.21.15.71:0/3101934668 >> 172.21.15.71:0/3101934668 conn(0x7f2ae00cdd80 msgr2=0x7f2ae00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:32.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 -- 172.21.15.71:0/3101934668 shutdown_connections 2023-10-26T05:53:32.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.727+0000 7f2ae5442640 1 -- 172.21.15.71:0/3101934668 wait complete. 2023-10-26T05:53:32.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 Processor -- start 2023-10-26T05:53:32.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- start start 2023-10-26T05:53:32.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 0x7f2ae00c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae00d0470 0x7f2ae014f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae00d0850 0x7f2ae014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae00c80c0 con 0x7f2ae00cd880 2023-10-26T05:53:32.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae015c970 con 0x7f2ae00d0470 2023-10-26T05:53:32.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae01589f0 con 0x7f2ae00d0850 2023-10-26T05:53:32.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 0x7f2ae00c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 0x7f2ae00c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34504/0 (socket says 172.21.15.71:34504) 2023-10-26T05:53:32.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 -- 172.21.15.71:0/4193705230 learned_addr learned my addr 172.21.15.71:0/4193705230 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:32.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adf7fe640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae00d0850 0x7f2ae014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ade7fc640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae00d0470 0x7f2ae014f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 -- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae00d0850 msgr2=0x7f2ae014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae00d0850 0x7f2ae014f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 -- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae00d0470 msgr2=0x7f2ae014f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:32.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae00d0470 0x7f2ae014f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:32.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 -- 172.21.15.71:0/4193705230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad40094a0 con 0x7f2ae00cd880 2023-10-26T05:53:32.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ade7fc640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae00d0470 0x7f2ae014f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2adeffd640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 0x7f2ae00c7660 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000c500 tx=0x7f2ad000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0012830 con 0x7f2ae00cd880 2023-10-26T05:53:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ad0012aa0 con 0x7f2ae00cd880 2023-10-26T05:53:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae00c7ba0 con 0x7f2ae00cd880 2023-10-26T05:53:32.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0011380 con 0x7f2ae00cd880 2023-10-26T05:53:32.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae0158c60 con 0x7f2ae00cd880 2023-10-26T05:53:32.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.731+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae0066b10 con 0x7f2ae00cd880 2023-10-26T05:53:32.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.735+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2ad00114e0 con 0x7f2ae00cd880 2023-10-26T05:53:32.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.735+0000 7f2ac77fe640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2acc066470 0x7f2acc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:32.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.735+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ad0084740 con 0x7f2ae00cd880 2023-10-26T05:53:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.735+0000 7f2ade7fc640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2acc066470 0x7f2acc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.735+0000 7f2ade7fc640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2acc066470 0x7f2acc068930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40af000 tx=0x7f2ad401afb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:32.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.739+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2ad0051f50 con 0x7f2ae00cd880 2023-10-26T05:53:32.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:32.939+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f2ae00b5130 con 0x7f2ae00cd880 2023-10-26T05:53:33.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ac77fe640 1 -- 172.21.15.71:0/4193705230 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v335) v1 ==== 128+0+0 (secure 0 0 0) 0x7f2ad0055570 con 0x7f2ae00cd880 2023-10-26T05:53:33.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2acc066470 msgr2=0x7f2acc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2acc066470 0x7f2acc068930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ad40af000 tx=0x7f2ad401afb0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 msgr2=0x7f2ae00c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 0x7f2ae00c7660 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f2ad000c500 tx=0x7f2ad000c9c0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 shutdown_connections 2023-10-26T05:53:33.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ae00d0850 0x7f2ae014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ae00d0470 0x7f2ae014f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2acc066470 0x7f2acc068930 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 --2- 172.21.15.71:0/4193705230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ae00cd880 0x7f2ae00c7660 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.295+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 >> 172.21.15.71:0/4193705230 conn(0x7f2ae00cdd80 msgr2=0x7f2ae00b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:33.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.299+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 shutdown_connections 2023-10-26T05:53:33.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.299+0000 7f2ae5442640 1 -- 172.21.15.71:0/4193705230 wait complete. 2023-10-26T05:53:33.323 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-10-26T05:53:33.323 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T05:53:33.324 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:33.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/4208551825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 msgr2=0x7f8608071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 --2- 172.21.15.71:0/4208551825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608071440 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f8600009c80 tx=0x7f860001cd50 comp rx=0 tx=0).stop 2023-10-26T05:53:33.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/4208551825 shutdown_connections 2023-10-26T05:53:33.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 --2- 172.21.15.71:0/4208551825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608071440 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 --2- 172.21.15.71:0/4208551825 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8608140cc0 0x7f860815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/4208551825 >> 172.21.15.71:0/4208551825 conn(0x7f8608069770 msgr2=0x7f8608069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:33.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/4208551825 shutdown_connections 2023-10-26T05:53:33.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/4208551825 wait complete. 2023-10-26T05:53:33.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 Processor -- start 2023-10-26T05:53:33.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- start start 2023-10-26T05:53:33.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8608070fe0 0x7f86081348e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608162680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608140cc0 0x7f8608158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081355c0 con 0x7f86080b2370 2023-10-26T05:53:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86081358a0 con 0x7f8608140cc0 2023-10-26T05:53:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8608135b80 con 0x7f8608070fe0 2023-10-26T05:53:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608162680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608162680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34524/0 (socket says 172.21.15.71:34524) 2023-10-26T05:53:33.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 -- 172.21.15.71:0/1462863306 learned_addr learned my addr 172.21.15.71:0/1462863306 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:33.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860d095640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608140cc0 0x7f8608158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860c894640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8608070fe0 0x7f86081348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 -- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8608070fe0 msgr2=0x7f86081348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8608070fe0 0x7f86081348e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 -- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608140cc0 msgr2=0x7f8608158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608140cc0 0x7f8608158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 -- 172.21.15.71:0/1462863306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000098e0 con 0x7f86080b2370 2023-10-26T05:53:33.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860d095640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608140cc0 0x7f8608158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:33.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f8607fff640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608162680 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c4d0 tx=0x7f85f800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:33.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8005480 con 0x7f86080b2370 2023-10-26T05:53:33.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f85f80036f0 con 0x7f86080b2370 2023-10-26T05:53:33.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8003b00 con 0x7f86080b2370 2023-10-26T05:53:33.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8608135e60 con 0x7f86080b2370 2023-10-26T05:53:33.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8608134e20 con 0x7f86080b2370 2023-10-26T05:53:33.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.439+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8608066b10 con 0x7f86080b2370 2023-10-26T05:53:33.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.443+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f85f8003c60 con 0x7f86080b2370 2023-10-26T05:53:33.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.443+0000 7f86057fa640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f4066470 0x7f85f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.443+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f85f80833e0 con 0x7f86080b2370 2023-10-26T05:53:33.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.443+0000 7f860c894640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f4066470 0x7f85f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.443+0000 7f860c894640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f4066470 0x7f85f4068930 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00a790 tx=0x7f85fc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:33.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.447+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f85f8050c20 con 0x7f86080b2370 2023-10-26T05:53:33.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.591+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8608136380 con 0x7f86080b2370 2023-10-26T05:53:33.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.591+0000 7f86057fa640 1 -- 172.21.15.71:0/1462863306 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+12913 (secure 0 0 0) 0x7f8608136380 con 0x7f86080b2370 2023-10-26T05:53:33.601 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:33.601 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":130,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:32.280205+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/1526458745":"2023-10-27T05:53:28.963544+0000","172.21.15.71:6834/1526458745":"2023-10-27T05:53:28.963544+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f4066470 msgr2=0x7f85f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f4066470 0x7f85f4068930 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00a790 tx=0x7f85fc008040 comp rx=0 tx=0).stop 2023-10-26T05:53:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 msgr2=0x7f8608162680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608162680 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c4d0 tx=0x7f85f800c990 comp rx=0 tx=0).stop 2023-10-26T05:53:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 shutdown_connections 2023-10-26T05:53:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8608140cc0 0x7f8608158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85f4066470 0x7f85f4068930 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86080b2370 0x7f8608162680 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 --2- 172.21.15.71:0/1462863306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8608070fe0 0x7f86081348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 >> 172.21.15.71:0/1462863306 conn(0x7f8608069770 msgr2=0x7f86080b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 shutdown_connections 2023-10-26T05:53:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.595+0000 7f860eaf8640 1 -- 172.21.15.71:0/1462863306 wait complete. 2023-10-26T05:53:33.618 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:53:33.618 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:53:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 -- 172.21.15.71:0/503880518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5006c010 msgr2=0x7f1d5006c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 --2- 172.21.15.71:0/503880518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5006c010 0x7f1d5006c3f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009900 tx=0x7f1d4801c880 comp rx=0 tx=0).stop 2023-10-26T05:53:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 -- 172.21.15.71:0/503880518 shutdown_connections 2023-10-26T05:53:33.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 --2- 172.21.15.71:0/503880518 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d500b30b0 0x7f1d500b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 --2- 172.21.15.71:0/503880518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5006c010 0x7f1d5006c3f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 -- 172.21.15.71:0/503880518 >> 172.21.15.71:0/503880518 conn(0x7f1d500ca2c0 msgr2=0x7f1d500ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:33.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 -- 172.21.15.71:0/503880518 shutdown_connections 2023-10-26T05:53:33.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 -- 172.21.15.71:0/503880518 wait complete. 2023-10-26T05:53:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 Processor -- start 2023-10-26T05:53:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.727+0000 7f1d54fea640 1 -- start start 2023-10-26T05:53:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5006c010 0x7f1d50148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d500b30b0 0x7f1d5014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 0x7f1d5014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5014a3a0 con 0x7f1d5015cd40 2023-10-26T05:53:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d500af600 con 0x7f1d5006c010 2023-10-26T05:53:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d50149a40 con 0x7f1d500b30b0 2023-10-26T05:53:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 0x7f1d5014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 0x7f1d5014f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34548/0 (socket says 172.21.15.71:34548) 2023-10-26T05:53:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 -- 172.21.15.71:0/2852547737 learned_addr learned my addr 172.21.15.71:0/2852547737 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4e59c640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5006c010 0x7f1d50148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4dd9b640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d500b30b0 0x7f1d5014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 -- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d500b30b0 msgr2=0x7f1d5014ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d500b30b0 0x7f1d5014ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 -- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5006c010 msgr2=0x7f1d50148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5006c010 0x7f1d50148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 -- 172.21.15.71:0/2852547737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d48009560 con 0x7f1d5015cd40 2023-10-26T05:53:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4ed9d640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 0x7f1d5014f4a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000b970 tx=0x7f1d4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4000cb40 con 0x7f1d5015cd40 2023-10-26T05:53:33.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1d4000cd90 con 0x7f1d5015cd40 2023-10-26T05:53:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d50149d20 con 0x7f1d5015cd40 2023-10-26T05:53:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d400146a0 con 0x7f1d5015cd40 2023-10-26T05:53:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5014a610 con 0x7f1d5015cd40 2023-10-26T05:53:33.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d50066d00 con 0x7f1d5015cd40 2023-10-26T05:53:33.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1d4001e440 con 0x7f1d5015cd40 2023-10-26T05:53:33.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d3effd640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d30066470 0x7f1d30068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:33.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d40083ba0 con 0x7f1d5015cd40 2023-10-26T05:53:33.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4e59c640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d30066470 0x7f1d30068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:33.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.731+0000 7f1d4e59c640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d30066470 0x7f1d30068930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1d480af000 tx=0x7f1d4801b0c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:33.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.735+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1d400513b0 con 0x7f1d5015cd40 2023-10-26T05:53:33.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.879+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d50150840 con 0x7f1d5015cd40 2023-10-26T05:53:33.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.879+0000 7f1d3effd640 1 -- 172.21.15.71:0/2852547737 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d400549d0 con 0x7f1d5015cd40 2023-10-26T05:53:33.888 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:53:33.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d30066470 msgr2=0x7f1d30068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d30066470 0x7f1d30068930 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1d480af000 tx=0x7f1d4801b0c0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 msgr2=0x7f1d5014f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:33.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 0x7f1d5014f4a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000b970 tx=0x7f1d4000be30 comp rx=0 tx=0).stop 2023-10-26T05:53:33.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 shutdown_connections 2023-10-26T05:53:33.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1d30066470 0x7f1d30068930 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d5015cd40 0x7f1d5014f4a0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1d500b30b0 0x7f1d5014ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 --2- 172.21.15.71:0/2852547737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d5006c010 0x7f1d50148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 >> 172.21.15.71:0/2852547737 conn(0x7f1d500ca2c0 msgr2=0x7f1d500cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:33.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 shutdown_connections 2023-10-26T05:53:33.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:33.883+0000 7f1d54fea640 1 -- 172.21.15.71:0/2852547737 wait complete. 2023-10-26T05:53:33.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-26T05:53:34.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 -- 172.21.15.71:0/747042946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 msgr2=0x7f53a40aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 --2- 172.21.15.71:0/747042946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a40aa8e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f539c009c80 tx=0x7f539c01ccb0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 -- 172.21.15.71:0/747042946 shutdown_connections 2023-10-26T05:53:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 --2- 172.21.15.71:0/747042946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a40aa8e0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 --2- 172.21.15.71:0/747042946 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53a413e430 0x7f53a40b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 -- 172.21.15.71:0/747042946 >> 172.21.15.71:0/747042946 conn(0x7f53a4069770 msgr2=0x7f53a4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:34.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 -- 172.21.15.71:0/747042946 shutdown_connections 2023-10-26T05:53:34.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 -- 172.21.15.71:0/747042946 wait complete. 2023-10-26T05:53:34.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 Processor -- start 2023-10-26T05:53:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.007+0000 7f53a92bf640 1 -- start start 2023-10-26T05:53:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53a40b0580 0x7f53a4151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a413e430 0x7f53a4152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a4153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a415e7e0 con 0x7f53a415c460 2023-10-26T05:53:34.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4153660 con 0x7f53a413e430 2023-10-26T05:53:34.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41365a0 con 0x7f53a40b0580 2023-10-26T05:53:34.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a4153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a4153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34568/0 (socket says 172.21.15.71:34568) 2023-10-26T05:53:34.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 -- 172.21.15.71:0/2665018483 learned_addr learned my addr 172.21.15.71:0/2665018483 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a2ffd640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53a40b0580 0x7f53a4151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539a7fc640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a413e430 0x7f53a4152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 -- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53a40b0580 msgr2=0x7f53a4151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53a40b0580 0x7f53a4151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 -- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a413e430 msgr2=0x7f53a4152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a413e430 0x7f53a4152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 -- 172.21.15.71:0/2665018483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c0098e0 con 0x7f53a415c460 2023-10-26T05:53:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539a7fc640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a413e430 0x7f53a4152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a37fe640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a4153030 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f539c009bd0 tx=0x7f539c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:34.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c0ba070 con 0x7f53a415c460 2023-10-26T05:53:34.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f539c004210 con 0x7f53a415c460 2023-10-26T05:53:34.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a4136880 con 0x7f53a415c460 2023-10-26T05:53:34.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c004600 con 0x7f53a415c460 2023-10-26T05:53:34.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a4136e40 con 0x7f53a415c460 2023-10-26T05:53:34.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a4066b10 con 0x7f53a415c460 2023-10-26T05:53:34.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f539c0b4040 con 0x7f53a415c460 2023-10-26T05:53:34.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539bfff640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5384066470 0x7f5384068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f539c11e0d0 con 0x7f53a415c460 2023-10-26T05:53:34.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a2ffd640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5384066470 0x7f5384068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.011+0000 7f53a2ffd640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5384066470 0x7f5384068930 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f53940058d0 tx=0x7f5394005e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:34.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.015+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f539c0eb840 con 0x7f53a415c460 2023-10-26T05:53:34.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.207+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f53a40721e0 con 0x7f53a415c460 2023-10-26T05:53:34.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.303+0000 7f539bfff640 1 -- 172.21.15.71:0/2665018483 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v336) v1 ==== 143+0+0 (secure 0 0 0) 0x7f539c0b20d0 con 0x7f53a415c460 2023-10-26T05:53:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5384066470 msgr2=0x7f5384068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5384066470 0x7f5384068930 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f53940058d0 tx=0x7f5394005e30 comp rx=0 tx=0).stop 2023-10-26T05:53:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 msgr2=0x7f53a4153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a4153030 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f539c009bd0 tx=0x7f539c0093d0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 shutdown_connections 2023-10-26T05:53:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5384066470 0x7f5384068930 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53a415c460 0x7f53a4153030 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53a413e430 0x7f53a4152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 --2- 172.21.15.71:0/2665018483 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f53a40b0580 0x7f53a4151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 >> 172.21.15.71:0/2665018483 conn(0x7f53a4069770 msgr2=0x7f53a40ac660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 shutdown_connections 2023-10-26T05:53:34.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.307+0000 7f53a92bf640 1 -- 172.21.15.71:0/2665018483 wait complete. 2023-10-26T05:53:34.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:53:34.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- 172.21.15.71:0/1976302272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c0b16e0 msgr2=0x7fb01c06b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- 172.21.15.71:0/1976302272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c0b16e0 0x7fb01c06b7c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb01c071900 tx=0x7fb01001c8a0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- 172.21.15.71:0/1976302272 shutdown_connections 2023-10-26T05:53:34.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- 172.21.15.71:0/1976302272 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb01c06bd00 0x7fb01c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- 172.21.15.71:0/1976302272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c0b16e0 0x7fb01c06b7c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- 172.21.15.71:0/1976302272 >> v2:172.21.15.86:3300/0 conn(0x7fb01c142c90 0x7fb01c0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- 172.21.15.71:0/1976302272 >> 172.21.15.71:0/1976302272 conn(0x7fb01c0caa90 msgr2=0x7fb01c0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:34.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- 172.21.15.71:0/1976302272 shutdown_connections 2023-10-26T05:53:34.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- 172.21.15.71:0/1976302272 wait complete. 2023-10-26T05:53:34.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 Processor -- start 2023-10-26T05:53:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- start start 2023-10-26T05:53:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb01c06bd00 0x7fb01c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb01c0b16e0 0x7fb01c15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 0x7fb01c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c13d250 con 0x7fb01c142c90 2023-10-26T05:53:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c13d530 con 0x7fb01c06bd00 2023-10-26T05:53:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb020e0c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c13d810 con 0x7fb01c0b16e0 2023-10-26T05:53:34.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 0x7fb01c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 0x7fb01c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34594/0 (socket says 172.21.15.71:34594) 2023-10-26T05:53:34.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 -- 172.21.15.71:0/314674475 learned_addr learned my addr 172.21.15.71:0/314674475 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:34.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01a59c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb01c06bd00 0x7fb01c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb019d9b640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb01c0b16e0 0x7fb01c15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 -- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb01c0b16e0 msgr2=0x7fb01c15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb01c0b16e0 0x7fb01c15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 -- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb01c06bd00 msgr2=0x7fb01c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb01c06bd00 0x7fb01c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.447+0000 7fb01ad9d640 1 -- 172.21.15.71:0/314674475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010009560 con 0x7fb01c142c90 2023-10-26T05:53:34.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb01ad9d640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 0x7fb01c15bee0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b9e0 tx=0x7fb01400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:34.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400cb20 con 0x7fb01c142c90 2023-10-26T05:53:34.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb01400cd70 con 0x7fb01c142c90 2023-10-26T05:53:34.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c13daf0 con 0x7fb01c142c90 2023-10-26T05:53:34.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140146a0 con 0x7fb01c142c90 2023-10-26T05:53:34.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c13c020 con 0x7fb01c142c90 2023-10-26T05:53:34.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c066d00 con 0x7fb01c142c90 2023-10-26T05:53:34.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb01401d3b0 con 0x7fb01c142c90 2023-10-26T05:53:34.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb00affd640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faffc066470 0x7faffc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb014083cb0 con 0x7fb01c142c90 2023-10-26T05:53:34.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb01a59c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faffc066470 0x7faffc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.451+0000 7fb01a59c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faffc066470 0x7faffc068930 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006fd0 tx=0x7fb00c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:34.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.455+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb0140514c0 con 0x7fb01c142c90 2023-10-26T05:53:34.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.611+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fb01c13dfa0 con 0x7fb01c142c90 2023-10-26T05:53:34.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.611+0000 7fb00affd640 1 -- 172.21.15.71:0/314674475 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fb014019020 con 0x7fb01c142c90 2023-10-26T05:53:34.620 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:53:34.621 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:53:34.621 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:53:34.621 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:53:34.621 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:53:34.621 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:53:34.622 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:53:34.622 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:53:34.622 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:53:34.622 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:53:34.622 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:53:34.623 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:53:34.623 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:53:34.623 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:53:34.623 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:53:34.624 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:53:34.624 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:53:34.624 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:53:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faffc066470 msgr2=0x7faffc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faffc066470 0x7faffc068930 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006fd0 tx=0x7fb00c008040 comp rx=0 tx=0).stop 2023-10-26T05:53:34.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 msgr2=0x7fb01c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 0x7fb01c15bee0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b9e0 tx=0x7fb01400bea0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 shutdown_connections 2023-10-26T05:53:34.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faffc066470 0x7faffc068930 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb01c142c90 0x7fb01c15bee0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb01c0b16e0 0x7fb01c15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 --2- 172.21.15.71:0/314674475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb01c06bd00 0x7fb01c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 >> 172.21.15.71:0/314674475 conn(0x7fb01c0caa90 msgr2=0x7fb01c13eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:34.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 shutdown_connections 2023-10-26T05:53:34.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.615+0000 7fb020e0c640 1 -- 172.21.15.71:0/314674475 wait complete. 2023-10-26T05:53:34.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:53:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/1720074247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f677c005190 msgr2=0x7f677c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 --2- 172.21.15.71:0/1720074247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f677c005190 0x7f677c00b990 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f678c009e30 tx=0x7f678c01d190 comp rx=0 tx=0).stop 2023-10-26T05:53:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/1720074247 shutdown_connections 2023-10-26T05:53:34.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 --2- 172.21.15.71:0/1720074247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f677c005190 0x7f677c00b990 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/1720074247 >> 172.21.15.71:0/1720074247 conn(0x7f67940aec60 msgr2=0x7f67940af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/1720074247 shutdown_connections 2023-10-26T05:53:34.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/1720074247 wait complete. 2023-10-26T05:53:34.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 Processor -- start 2023-10-26T05:53:34.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- start start 2023-10-26T05:53:34.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c005190 0x7f6794156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6794069cc0 0x7f67941f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 0x7f679413a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794156fa0 con 0x7f679413ce20 2023-10-26T05:53:34.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67941f2e60 con 0x7f677c005190 2023-10-26T05:53:34.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679413ec60 con 0x7f6794069cc0 2023-10-26T05:53:34.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 0x7f679413a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 0x7f679413a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34608/0 (socket says 172.21.15.71:34608) 2023-10-26T05:53:34.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 -- 172.21.15.71:0/150715205 learned_addr learned my addr 172.21.15.71:0/150715205 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f6792ffd640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c005190 0x7f6794156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67927fc640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6794069cc0 0x7f67941f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 -- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6794069cc0 msgr2=0x7f67941f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6794069cc0 0x7f67941f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 -- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c005190 msgr2=0x7f6794156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c005190 0x7f6794156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 -- 172.21.15.71:0/150715205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679413ef40 con 0x7f679413ce20 2023-10-26T05:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f67937fe640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 0x7f679413a0d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f678c01cc00 tx=0x7f678c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c004c60 con 0x7f679413ce20 2023-10-26T05:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f678c003f60 con 0x7f679413ce20 2023-10-26T05:53:34.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679413f220 con 0x7f679413ce20 2023-10-26T05:53:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c0042f0 con 0x7f679413ce20 2023-10-26T05:53:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679413f7e0 con 0x7f679413ce20 2023-10-26T05:53:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.755+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794066b10 con 0x7f679413ce20 2023-10-26T05:53:34.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.759+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f678c004450 con 0x7f679413ce20 2023-10-26T05:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.759+0000 7f677b7fe640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67800664a0 0x7f6780068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:34.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.759+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f678c11f2b0 con 0x7f679413ce20 2023-10-26T05:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.759+0000 7f6792ffd640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67800664a0 0x7f6780068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.759+0000 7f6792ffd640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67800664a0 0x7f6780068960 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6784000f70 tx=0x7f6784006ad0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:34.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.763+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f678c0ecab0 con 0x7f679413ce20 2023-10-26T05:53:34.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.903+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f679413f9d0 con 0x7f679413ce20 2023-10-26T05:53:34.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f677b7fe640 1 -- 172.21.15.71:0/150715205 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f678c0f00d0 con 0x7f679413ce20 2023-10-26T05:53:34.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67800664a0 msgr2=0x7f6780068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67800664a0 0x7f6780068960 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f6784000f70 tx=0x7f6784006ad0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 msgr2=0x7f679413a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:34.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 0x7f679413a0d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f678c01cc00 tx=0x7f678c003ef0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 shutdown_connections 2023-10-26T05:53:34.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67800664a0 0x7f6780068960 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f679413ce20 0x7f679413a0d0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6794069cc0 0x7f67941f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 --2- 172.21.15.71:0/150715205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f677c005190 0x7f6794156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:34.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.907+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 >> 172.21.15.71:0/150715205 conn(0x7f67940aec60 msgr2=0x7f67941577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:34.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.911+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 shutdown_connections 2023-10-26T05:53:34.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:34.911+0000 7f679941e640 1 -- 172.21.15.71:0/150715205 wait complete. 2023-10-26T05:53:34.933 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:53:34.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:53:34.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:53:35.027 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:53:35.028 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:35.028 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:35.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:53:35.034 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:53:35.034 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:35.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:53:35.074 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:35.075 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:35.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:53:35.084 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:53:35.084 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:35.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:53:35.122 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:53:35.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:35.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.239+0000 7fe74de3c640 1 -- 172.21.15.71:0/3859757029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 msgr2=0x7fe74815c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.239+0000 7fe74de3c640 1 --2- 172.21.15.71:0/3859757029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe74815c0a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe7480b2bd0 tx=0x7fe73801c5b0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/3859757029 shutdown_connections 2023-10-26T05:53:35.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 --2- 172.21.15.71:0/3859757029 >> v2:172.21.15.86:3300/0 conn(0x7fe7480b1470 0x7fe7480b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.250 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 --2- 172.21.15.71:0/3859757029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe74815c0a0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/3859757029 >> 172.21.15.71:0/3859757029 conn(0x7fe7480afeb0 msgr2=0x7fe7480b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:35.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/3859757029 shutdown_connections 2023-10-26T05:53:35.251 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/3859757029 wait complete. 2023-10-26T05:53:35.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 Processor -- start 2023-10-26T05:53:35.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- start start 2023-10-26T05:53:35.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7480b1470 0x7fe74815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe748141a70 0x7fe74815d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe748139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74813a6e0 con 0x7fe74815bc10 2023-10-26T05:53:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748156100 con 0x7fe748141a70 2023-10-26T05:53:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7481563c0 con 0x7fe7480b1470 2023-10-26T05:53:35.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe748139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe748139240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34628/0 (socket says 172.21.15.71:34628) 2023-10-26T05:53:35.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 -- 172.21.15.71:0/2058271338 learned_addr learned my addr 172.21.15.71:0/2058271338 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:35.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe7477fe640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7480b1470 0x7fe74815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 -- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7480b1470 msgr2=0x7fe74815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe746ffd640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe748141a70 0x7fe74815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7480b1470 0x7fe74815a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 -- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe748141a70 msgr2=0x7fe74815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe748141a70 0x7fe74815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 -- 172.21.15.71:0/2058271338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380094a0 con 0x7fe74815bc10 2023-10-26T05:53:35.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe747fff640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe748139240 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe74000ccb0 tx=0x7fe74000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:35.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740004b30 con 0x7fe74815bc10 2023-10-26T05:53:35.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe740004da0 con 0x7fe74815bc10 2023-10-26T05:53:35.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7481566a0 con 0x7fe74815bc10 2023-10-26T05:53:35.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740005280 con 0x7fe74815bc10 2023-10-26T05:53:35.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748156c60 con 0x7fe74815bc10 2023-10-26T05:53:35.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.243+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748066d00 con 0x7fe74815bc10 2023-10-26T05:53:35.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.247+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7400053e0 con 0x7fe74815bc10 2023-10-26T05:53:35.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.247+0000 7fe72ffff640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe734066470 0x7fe734068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.247+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe740084440 con 0x7fe74815bc10 2023-10-26T05:53:35.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.247+0000 7fe7477fe640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe734066470 0x7fe734068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.247+0000 7fe7477fe640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe734066470 0x7fe734068930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0017f0 tx=0x7fe73c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:35.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.251+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe740051bc0 con 0x7fe74815bc10 2023-10-26T05:53:35.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe74815cad0 con 0x7fe74815bc10 2023-10-26T05:53:35.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe72ffff640 1 -- 172.21.15.71:0/2058271338 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fe74001d020 con 0x7fe74815bc10 2023-10-26T05:53:35.431 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:35.432 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5594,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/1969972219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":1969972219},{"type":"v1","addr":"172.21.15.71:6833","nonce":1969972219}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/1150297066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1150297066},{"type":"v1","addr":"172.21.15.86:6835","nonce":1150297066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":5609,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3584912508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":3584912508},{"type":"v1","addr":"172.21.15.86:6837","nonce":3584912508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":7911,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1315945082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1315945082},{"type":"v1","addr":"172.21.15.71:6835","nonce":1315945082}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2023-10-26T05:53:35.432 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 46 2023-10-26T05:53:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe734066470 msgr2=0x7fe734068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe734066470 0x7fe734068930 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0017f0 tx=0x7fe73c009040 comp rx=0 tx=0).stop 2023-10-26T05:53:35.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 msgr2=0x7fe748139240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe748139240 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe74000ccb0 tx=0x7fe74000a620 comp rx=0 tx=0).stop 2023-10-26T05:53:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 shutdown_connections 2023-10-26T05:53:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe734066470 0x7fe734068930 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe74815bc10 0x7fe748139240 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe748141a70 0x7fe74815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 --2- 172.21.15.71:0/2058271338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7480b1470 0x7fe74815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.423+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 >> 172.21.15.71:0/2058271338 conn(0x7fe7480afeb0 msgr2=0x7fe748159780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.427+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 shutdown_connections 2023-10-26T05:53:35.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.427+0000 7fe74de3c640 1 -- 172.21.15.71:0/2058271338 wait complete. 2023-10-26T05:53:35.449 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:53:35.449 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:53:35.449 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:53:35.450 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:35.450 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:53:35.450 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:53:35.451 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:53:35.451 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:35.451 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:53:35.451 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:53:35.452 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:53:35.452 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:35.452 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:53:35.452 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:53:35.453 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:53:35.453 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:53:35.458 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:53:35.459 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:53:35.460 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:53:35.460 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:53:35.465 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:53:35.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:53:35.467 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:53:35.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:53:35.502 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:53:35.503 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:35.503 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:53:35.504 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:35.510 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:53:35.510 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:35.511 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:53:35.512 DEBUG:teuthology.parallel:result is None 2023-10-26T05:53:35.512 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:53:35.560 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:53:35.563 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:53:35.580 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:53:35.581 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:53:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 -- 172.21.15.71:0/2415449061 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 msgr2=0x7fce3c0c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 --2- 172.21.15.71:0/2415449061 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 0x7fce3c0c9ad0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce30009900 tx=0x7fce3001c800 comp rx=0 tx=0).stop 2023-10-26T05:53:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 -- 172.21.15.71:0/2415449061 shutdown_connections 2023-10-26T05:53:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 --2- 172.21.15.71:0/2415449061 >> v2:172.21.15.86:3300/0 conn(0x7fce3c0b3300 0x7fce3c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 --2- 172.21.15.71:0/2415449061 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 0x7fce3c0c9ad0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 -- 172.21.15.71:0/2415449061 >> 172.21.15.71:0/2415449061 conn(0x7fce3c070da0 msgr2=0x7fce3c06bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 -- 172.21.15.71:0/2415449061 shutdown_connections 2023-10-26T05:53:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 -- 172.21.15.71:0/2415449061 wait complete. 2023-10-26T05:53:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 Processor -- start 2023-10-26T05:53:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.627+0000 7fce41ebf640 1 -- start start 2023-10-26T05:53:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 0x7fce3c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 0x7fce3c13ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c15d930 0x7fce3c141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c141120 con 0x7fce3c0b3300 2023-10-26T05:53:35.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c14d560 con 0x7fce3c15d930 2023-10-26T05:53:35.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15b860 con 0x7fce3c159310 2023-10-26T05:53:35.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 0x7fce3c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 0x7fce3c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34670/0 (socket says 172.21.15.71:34670) 2023-10-26T05:53:35.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 -- 172.21.15.71:0/3015326808 learned_addr learned my addr 172.21.15.71:0/3015326808 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:35.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3bfff640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c15d930 0x7fce3c141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3affd640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 0x7fce3c13ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 -- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 msgr2=0x7fce3c13ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 0x7fce3c13ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 -- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c15d930 msgr2=0x7fce3c141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:35.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c15d930 0x7fce3c141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:35.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 -- 172.21.15.71:0/3015326808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce30009560 con 0x7fce3c0b3300 2023-10-26T05:53:35.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3b7fe640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 0x7fce3c156b90 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fce30009850 tx=0x7fce30005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:35.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30005230 con 0x7fce3c0b3300 2023-10-26T05:53:35.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce30005480 con 0x7fce3c0b3300 2023-10-26T05:53:35.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce30004930 con 0x7fce3c0b3300 2023-10-26T05:53:35.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c140a00 con 0x7fce3c0b3300 2023-10-26T05:53:35.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c140500 con 0x7fce3c0b3300 2023-10-26T05:53:35.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c066b10 con 0x7fce3c0b3300 2023-10-26T05:53:35.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce300b4040 con 0x7fce3c0b3300 2023-10-26T05:53:35.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce23fff640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce28066470 0x7fce28068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:35.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fce3011db10 con 0x7fce3c0b3300 2023-10-26T05:53:35.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3affd640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce28066470 0x7fce28068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:35.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.631+0000 7fce3affd640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce28066470 0x7fce28068930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce3c06cd40 tx=0x7fce2c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:35.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.635+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce300eb2c0 con 0x7fce3c0b3300 2023-10-26T05:53:35.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:35.799+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fce3c06ddc0 con 0x7fce3c0b3300 2023-10-26T05:53:36.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.331+0000 7fce23fff640 1 -- 172.21.15.71:0/3015326808 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7fce300ee8e0 con 0x7fce3c0b3300 2023-10-26T05:53:36.340 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:53:36.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce28066470 msgr2=0x7fce28068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce28066470 0x7fce28068930 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce3c06cd40 tx=0x7fce2c009040 comp rx=0 tx=0).stop 2023-10-26T05:53:36.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 msgr2=0x7fce3c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 0x7fce3c156b90 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fce30009850 tx=0x7fce30005e70 comp rx=0 tx=0).stop 2023-10-26T05:53:36.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 shutdown_connections 2023-10-26T05:53:36.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce3c15d930 0x7fce3c141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce3c159310 0x7fce3c13ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce28066470 0x7fce28068930 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 --2- 172.21.15.71:0/3015326808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce3c0b3300 0x7fce3c156b90 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 >> 172.21.15.71:0/3015326808 conn(0x7fce3c070da0 msgr2=0x7fce3c15bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:36.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 shutdown_connections 2023-10-26T05:53:36.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.335+0000 7fce41ebf640 1 -- 172.21.15.71:0/3015326808 wait complete. 2023-10-26T05:53:36.364 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:53:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/1628138294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 msgr2=0x7fe94815b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 --2- 172.21.15.71:0/1628138294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe94815b8e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe9480ccd20 tx=0x7fe94001cbe0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/1628138294 shutdown_connections 2023-10-26T05:53:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 --2- 172.21.15.71:0/1628138294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe94815b8e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 --2- 172.21.15.71:0/1628138294 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 0x7fe94815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/1628138294 >> 172.21.15.71:0/1628138294 conn(0x7fe9480cdd80 msgr2=0x7fe9480ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/1628138294 shutdown_connections 2023-10-26T05:53:36.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/1628138294 wait complete. 2023-10-26T05:53:36.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 Processor -- start 2023-10-26T05:53:36.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- start start 2023-10-26T05:53:36.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe9481509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480d0470 0x7fe9480c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 0x7fe9480c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94815a9f0 con 0x7fe9480cd880 2023-10-26T05:53:36.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948159d70 con 0x7fe9480d0470 2023-10-26T05:53:36.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94815a050 con 0x7fe9480d0850 2023-10-26T05:53:36.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe9481509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe9481509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34688/0 (socket says 172.21.15.71:34688) 2023-10-26T05:53:36.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 -- 172.21.15.71:0/3516434095 learned_addr learned my addr 172.21.15.71:0/3516434095 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:36.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe9477fe640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480d0470 0x7fe9480c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94cc2f640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 0x7fe9480c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 -- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 msgr2=0x7fe9480c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 0x7fe9480c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 -- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480d0470 msgr2=0x7fe9480c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480d0470 0x7fe9480c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 -- 172.21.15.71:0/3516434095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400098e0 con 0x7fe9480cd880 2023-10-26T05:53:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe9477fe640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480d0470 0x7fe9480c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:36.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94cc2f640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 0x7fe9480c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe947fff640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe9481509a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c450 tx=0x7fe93c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c012830 con 0x7fe9480cd880 2023-10-26T05:53:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe93c012aa0 con 0x7fe9480cd880 2023-10-26T05:53:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94815a330 con 0x7fe9480cd880 2023-10-26T05:53:36.490 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011380 con 0x7fe9480cd880 2023-10-26T05:53:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.475+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe948159040 con 0x7fe9480cd880 2023-10-26T05:53:36.491 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.479+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948066b10 con 0x7fe9480cd880 2023-10-26T05:53:36.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.479+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe93c0114e0 con 0x7fe9480cd880 2023-10-26T05:53:36.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.479+0000 7fe944ff9640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe934066470 0x7fe934068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.479+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe93c083bc0 con 0x7fe9480cd880 2023-10-26T05:53:36.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.479+0000 7fe9477fe640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe934066470 0x7fe934068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.479+0000 7fe9477fe640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe934066470 0x7fe934068930 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe948158b70 tx=0x7fe938005d20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:36.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.483+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe93c051420 con 0x7fe9480cd880 2023-10-26T05:53:36.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.687+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe9480721e0 con 0x7fe9480cd880 2023-10-26T05:53:36.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.687+0000 7fe944ff9640 1 -- 172.21.15.71:0/3516434095 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe93c054a40 con 0x7fe9480cd880 2023-10-26T05:53:36.695 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:36.696 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:36.696 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:53:36.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.696 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:53:36.696 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:53:36.697 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.697 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.697 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:36.697 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:36.698 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.698 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.698 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:53:36.699 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:53:36.699 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.699 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.699 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:36.699 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:36.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.700 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.700 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:53:36.701 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:53:36.701 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.701 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.701 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:36.702 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:36.702 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.702 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.703 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:53:36.703 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:53:36.703 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.703 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.703 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:36.703 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:36.704 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.704 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.704 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:53:36.705 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:53:36.705 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.705 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.705 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.705 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.706 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.706 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.706 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:53:36.707 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:53:36.707 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.707 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.707 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.708 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.708 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:53:36.709 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:53:36.709 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.709 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.709 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.709 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.709 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.710 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:53:36.710 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:53:36.711 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.711 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.711 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.711 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.712 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:53:36.712 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:53:36.712 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.713 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.713 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.713 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.714 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:53:36.714 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:53:36.714 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.714 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.715 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.717 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:53:36.718 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:53:36.718 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.718 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.718 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.718 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.719 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.719 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:53:36.719 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:53:36.720 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.720 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:53:36.720 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:53:36.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.721 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.721 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:53:36.721 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:53:36.721 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.722 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.722 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:53:36.722 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:53:36.722 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.723 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:53:36.723 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:53:36.723 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.723 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:53:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:53:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:53:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.724 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.725 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:53:36.725 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:53:36.725 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.725 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.725 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:53:36.726 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:53:36.726 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:53:36.726 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.727 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.727 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:53:36.727 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:53:36.727 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.727 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:53:36.728 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:53:36.728 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:53:36.728 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:53:36.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.729 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:53:36.729 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:53:36.729 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.729 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:53:36.729 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:53:36.730 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:53:36.730 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.731 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:53:36.731 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:53:36.731 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.731 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:53:36.732 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:53:36.732 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:36.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:36.732 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:53:36.733 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:53:36.733 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:53:36.733 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:53:36.733 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:53:36.733 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:53:36.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.734 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:36.734 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:36.734 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:36.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe934066470 msgr2=0x7fe934068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe934066470 0x7fe934068930 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe948158b70 tx=0x7fe938005d20 comp rx=0 tx=0).stop 2023-10-26T05:53:36.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 msgr2=0x7fe9481509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe9481509a0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c450 tx=0x7fe93c00c910 comp rx=0 tx=0).stop 2023-10-26T05:53:36.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 shutdown_connections 2023-10-26T05:53:36.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9480d0850 0x7fe9480c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480d0470 0x7fe9480c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe934066470 0x7fe934068930 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 --2- 172.21.15.71:0/3516434095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9480cd880 0x7fe9481509a0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 >> 172.21.15.71:0/3516434095 conn(0x7fe9480cdd80 msgr2=0x7fe9480b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:36.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 shutdown_connections 2023-10-26T05:53:36.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.691+0000 7fe94e692640 1 -- 172.21.15.71:0/3516434095 wait complete. 2023-10-26T05:53:36.738 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:36.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3873724239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 msgr2=0x7f4a700b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3873724239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a700b2350 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4a7006f190 tx=0x7f4a6001c840 comp rx=0 tx=0).stop 2023-10-26T05:53:36.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3873724239 shutdown_connections 2023-10-26T05:53:36.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3873724239 >> v2:172.21.15.86:3300/0 conn(0x7f4a700cb1e0 0x7f4a70159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3873724239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a700b2350 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3873724239 >> 172.21.15.71:0/3873724239 conn(0x7f4a70069770 msgr2=0x7f4a70069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:36.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3873724239 shutdown_connections 2023-10-26T05:53:36.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3873724239 wait complete. 2023-10-26T05:53:36.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 Processor -- start 2023-10-26T05:53:36.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- start start 2023-10-26T05:53:36.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a700b2890 0x7f4a70158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a700cb1e0 0x7f4a700b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a7014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a700b1060 con 0x7f4a7013c2f0 2023-10-26T05:53:36.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7014fa40 con 0x7f4a700b2890 2023-10-26T05:53:36.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a700b04c0 con 0x7f4a700cb1e0 2023-10-26T05:53:36.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a7014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a7014cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34714/0 (socket says 172.21.15.71:34714) 2023-10-26T05:53:36.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 -- 172.21.15.71:0/3999129167 learned_addr learned my addr 172.21.15.71:0/3999129167 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:36.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6ed9d640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a700b2890 0x7f4a70158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 -- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a700cb1e0 msgr2=0x7f4a700b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a700cb1e0 0x7f4a700b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 -- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a700b2890 msgr2=0x7f4a70158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:36.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a700b2890 0x7f4a70158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:36.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 -- 172.21.15.71:0/3999129167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a60009560 con 0x7f4a7013c2f0 2023-10-26T05:53:36.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a6f59e640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a7014cc60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800cce0 tx=0x7f4a6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:36.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68004b30 con 0x7f4a7013c2f0 2023-10-26T05:53:36.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a68004d80 con 0x7f4a7013c2f0 2023-10-26T05:53:36.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a700b0730 con 0x7f4a7013c2f0 2023-10-26T05:53:36.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68005280 con 0x7f4a7013c2f0 2023-10-26T05:53:36.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a700b0cf0 con 0x7f4a7013c2f0 2023-10-26T05:53:36.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.851+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a70066b10 con 0x7f4a7013c2f0 2023-10-26T05:53:36.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.855+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4a680053e0 con 0x7f4a7013c2f0 2023-10-26T05:53:36.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.855+0000 7f4a67fff640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a50066470 0x7f4a50068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:36.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.855+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4a680847b0 con 0x7f4a7013c2f0 2023-10-26T05:53:36.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.855+0000 7f4a6ed9d640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a50066470 0x7f4a50068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:36.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.855+0000 7f4a6ed9d640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a50066470 0x7f4a50068930 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4a600af000 tx=0x7f4a6001b080 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:36.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.859+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4a68052060 con 0x7f4a7013c2f0 2023-10-26T05:53:37.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:36.999+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a700b1210 con 0x7f4a7013c2f0 2023-10-26T05:53:37.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a67fff640 1 -- 172.21.15.71:0/3999129167 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+12784 (secure 0 0 0) 0x7f4a700b1210 con 0x7f4a7013c2f0 2023-10-26T05:53:37.011 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:37.011 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":134,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:36.348983+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:37.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a50066470 msgr2=0x7f4a50068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a50066470 0x7f4a50068930 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4a600af000 tx=0x7f4a6001b080 comp rx=0 tx=0).stop 2023-10-26T05:53:37.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 msgr2=0x7f4a7014cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a7014cc60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800cce0 tx=0x7f4a6800a620 comp rx=0 tx=0).stop 2023-10-26T05:53:37.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 shutdown_connections 2023-10-26T05:53:37.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4a50066470 0x7f4a50068930 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7013c2f0 0x7f4a7014cc60 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a700cb1e0 0x7f4a700b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 --2- 172.21.15.71:0/3999129167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a700b2890 0x7f4a70158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 >> 172.21.15.71:0/3999129167 conn(0x7f4a70069770 msgr2=0x7f4a700ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:37.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 shutdown_connections 2023-10-26T05:53:37.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.003+0000 7f4a7513a640 1 -- 172.21.15.71:0/3999129167 wait complete. 2023-10-26T05:53:37.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:53:37.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:53:37.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3879651315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff00b0fa0 msgr2=0x7fbff0070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3879651315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff00b0fa0 0x7fbff0070dc0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009840 tx=0x7fbfe001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3879651315 shutdown_connections 2023-10-26T05:53:37.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3879651315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff00b0fa0 0x7fbff0070dc0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3879651315 >> 172.21.15.71:0/3879651315 conn(0x7fbff00aec60 msgr2=0x7fbff00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3879651315 shutdown_connections 2023-10-26T05:53:37.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3879651315 wait complete. 2023-10-26T05:53:37.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 Processor -- start 2023-10-26T05:53:37.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- start start 2023-10-26T05:53:37.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbff00b0fa0 0x7fbff0134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff015bb90 0x7fbff01537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 0x7fbff0150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff0154300 con 0x7fbff015bf70 2023-10-26T05:53:37.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff014d7c0 con 0x7fbff015bb90 2023-10-26T05:53:37.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff7e7a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff014daa0 con 0x7fbff00b0fa0 2023-10-26T05:53:37.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff6417640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 0x7fbff0150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff6417640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 0x7fbff0150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34734/0 (socket says 172.21.15.71:34734) 2023-10-26T05:53:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff6417640 1 -- 172.21.15.71:0/3630887444 learned_addr learned my addr 172.21.15.71:0/3630887444 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.151+0000 7fbff5415640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff015bb90 0x7fbff01537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff5c16640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbff00b0fa0 0x7fbff0134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff6417640 1 -- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbff00b0fa0 msgr2=0x7fbff0134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff6417640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbff00b0fa0 0x7fbff0134bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff6417640 1 -- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff015bb90 msgr2=0x7fbff01537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff6417640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff015bb90 0x7fbff01537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff6417640 1 -- 172.21.15.71:0/3630887444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe00094a0 con 0x7fbff015bf70 2023-10-26T05:53:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff5c16640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbff00b0fa0 0x7fbff0134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff6417640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 0x7fbff0150970 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800caa0 tx=0x7fbfe800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800a840 con 0x7fbff015bf70 2023-10-26T05:53:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbfe800aab0 con 0x7fbff015bf70 2023-10-26T05:53:37.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8012420 con 0x7fbff015bf70 2023-10-26T05:53:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff014dd80 con 0x7fbff015bf70 2023-10-26T05:53:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff014d090 con 0x7fbff015bf70 2023-10-26T05:53:37.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff0066b10 con 0x7fbff015bf70 2023-10-26T05:53:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbfe801a3b0 con 0x7fbff015bf70 2023-10-26T05:53:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbfe67fc640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd4066470 0x7fbfd4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbfe8084080 con 0x7fbff015bf70 2023-10-26T05:53:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff5c16640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd4066470 0x7fbfd4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.155+0000 7fbff5c16640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd4066470 0x7fbfd4068930 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbff00c9260 tx=0x7fbfec009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:37.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.159+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbfe80518e0 con 0x7fbff015bf70 2023-10-26T05:53:37.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.315+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbff0141e40 con 0x7fbff015bf70 2023-10-26T05:53:37.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbfe67fc640 1 -- 172.21.15.71:0/3630887444 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfe8016020 con 0x7fbff015bf70 2023-10-26T05:53:37.326 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:53:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd4066470 msgr2=0x7fbfd4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd4066470 0x7fbfd4068930 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fbff00c9260 tx=0x7fbfec009040 comp rx=0 tx=0).stop 2023-10-26T05:53:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 msgr2=0x7fbff0150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 0x7fbff0150970 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800caa0 tx=0x7fbfe800cb90 comp rx=0 tx=0).stop 2023-10-26T05:53:37.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 shutdown_connections 2023-10-26T05:53:37.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbfd4066470 0x7fbfd4068930 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbff015bf70 0x7fbff0150970 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff015bb90 0x7fbff01537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 --2- 172.21.15.71:0/3630887444 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbff00b0fa0 0x7fbff0134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 >> 172.21.15.71:0/3630887444 conn(0x7fbff00aec60 msgr2=0x7fbff00b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:37.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 shutdown_connections 2023-10-26T05:53:37.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.319+0000 7fbff7e7a640 1 -- 172.21.15.71:0/3630887444 wait complete. 2023-10-26T05:53:37.348 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:53:37.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/1348313170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80640039b0 msgr2=0x7f8064003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 --2- 172.21.15.71:0/1348313170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80640039b0 0x7f8064003dd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8070001ec0 tx=0x7f807001cf50 comp rx=0 tx=0).stop 2023-10-26T05:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f807ad9d640 1 -- 172.21.15.71:0/1348313170 reap_dead start 2023-10-26T05:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/1348313170 shutdown_connections 2023-10-26T05:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/1348313170 >> 172.21.15.71:0/1348313170 conn(0x7f807c0cdd80 msgr2=0x7f807c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:37.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/1348313170 shutdown_connections 2023-10-26T05:53:37.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/1348313170 wait complete. 2023-10-26T05:53:37.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 Processor -- start 2023-10-26T05:53:37.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- start start 2023-10-26T05:53:37.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80640039b0 0x7f807c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 0x7f807c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c1567f0 0x7f807c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c06a650 con 0x7f807c166160 2023-10-26T05:53:37.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c164de0 con 0x7f807c1567f0 2023-10-26T05:53:37.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c1650c0 con 0x7f80640039b0 2023-10-26T05:53:37.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 0x7f807c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 0x7f807c1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34742/0 (socket says 172.21.15.71:34742) 2023-10-26T05:53:37.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 -- 172.21.15.71:0/3174954385 learned_addr learned my addr 172.21.15.71:0/3174954385 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f807ad9d640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c1567f0 0x7f807c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f807a59c640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80640039b0 0x7f807c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 -- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80640039b0 msgr2=0x7f807c150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80640039b0 0x7f807c150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 -- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c1567f0 msgr2=0x7f807c15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c1567f0 0x7f807c15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 -- 172.21.15.71:0/3174954385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c1653a0 con 0x7f807c166160 2023-10-26T05:53:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f807ad9d640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c1567f0 0x7f807c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:37.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f807a59c640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80640039b0 0x7f807c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:37.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8079d9b640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 0x7f807c1562b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f806c00fa00 tx=0x7f806c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:37.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c00bcb0 con 0x7f807c166160 2023-10-26T05:53:37.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f806c004020 con 0x7f807c166160 2023-10-26T05:53:37.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c004410 con 0x7f807c166160 2023-10-26T05:53:37.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c165680 con 0x7f807c166160 2023-10-26T05:53:37.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.459+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c165c40 con 0x7f807c166160 2023-10-26T05:53:37.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.463+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c066b10 con 0x7f807c166160 2023-10-26T05:53:37.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.463+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f806c003600 con 0x7f807c166160 2023-10-26T05:53:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.463+0000 7f806affd640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f805c066470 0x7f805c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.463+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f806c084560 con 0x7f807c166160 2023-10-26T05:53:37.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.463+0000 7f807a59c640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f805c066470 0x7f805c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.463+0000 7f807a59c640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f805c066470 0x7f805c068930 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f80700ac000 tx=0x7f8070019cf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:37.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.467+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f806c051cd0 con 0x7f807c166160 2023-10-26T05:53:37.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f807c0721e0 con 0x7f807c166160 2023-10-26T05:53:37.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f806affd640 1 -- 172.21.15.71:0/3174954385 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) v1 ==== 84+0+65 (secure 0 0 0) 0x7f806c020090 con 0x7f807c166160 2023-10-26T05:53:37.631 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:37.631 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:53:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:53:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:53:37.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:37.633 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f805c066470 msgr2=0x7f805c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f805c066470 0x7f805c068930 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f80700ac000 tx=0x7f8070019cf0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 msgr2=0x7f807c1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 0x7f807c1562b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f806c00fa00 tx=0x7f806c00fec0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 shutdown_connections 2023-10-26T05:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f807c1567f0 0x7f807c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f805c066470 0x7f805c068930 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f807c166160 0x7f807c1562b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 --2- 172.21.15.71:0/3174954385 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f80640039b0 0x7f807c150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 >> 172.21.15.71:0/3174954385 conn(0x7f807c0cdd80 msgr2=0x7f807c164570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 shutdown_connections 2023-10-26T05:53:37.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.623+0000 7f8080c80640 1 -- 172.21.15.71:0/3174954385 wait complete. 2023-10-26T05:53:37.649 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:53:37.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:53:37.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 -- 172.21.15.71:0/1231806460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c070c60 msgr2=0x7f6e1c0ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1231806460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c070c60 0x7f6e1c0ca350 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f6e14009930 tx=0x7f6e1401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 -- 172.21.15.71:0/1231806460 shutdown_connections 2023-10-26T05:53:37.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1231806460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c070c60 0x7f6e1c0ca350 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1231806460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1c141370 0x7f6e1c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 -- 172.21.15.71:0/1231806460 >> 172.21.15.71:0/1231806460 conn(0x7f6e1c0afe40 msgr2=0x7f6e1c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:37.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 -- 172.21.15.71:0/1231806460 shutdown_connections 2023-10-26T05:53:37.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.763+0000 7f6e234b0640 1 -- 172.21.15.71:0/1231806460 wait complete. 2023-10-26T05:53:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 Processor -- start 2023-10-26T05:53:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- start start 2023-10-26T05:53:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e1c070c60 0x7f6e1c1f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 0x7f6e1c0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1c141370 0x7f6e1c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c138310 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c137700 con 0x7f6e1c070c60 2023-10-26T05:53:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c1379e0 con 0x7f6e1c141370 2023-10-26T05:53:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 0x7f6e1c0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 0x7f6e1c0ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34762/0 (socket says 172.21.15.71:34762) 2023-10-26T05:53:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 -- 172.21.15.71:0/1243420240 learned_addr learned my addr 172.21.15.71:0/1243420240 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e21a4d640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1c141370 0x7f6e1c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e2124c640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e1c070c60 0x7f6e1c1f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 -- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1c141370 msgr2=0x7f6e1c06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1c141370 0x7f6e1c06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 -- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e1c070c60 msgr2=0x7f6e1c1f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e1c070c60 0x7f6e1c1f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 -- 172.21.15.71:0/1243420240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009560 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e20a4b640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 0x7f6e1c0ade20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00c490 tx=0x7f6e0c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c003600 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c137cc0 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6e0c003780 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c003bd0 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c0ae360 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.767+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c138480 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.771+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6e0c003d30 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.771+0000 7f6e11ffb640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e00066470 0x7f6e00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:37.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.771+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6e0c083030 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.771+0000 7f6e2124c640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e00066470 0x7f6e00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:37.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.771+0000 7f6e2124c640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e00066470 0x7f6e00068930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009f20 tx=0x7f6e180094f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:37.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.775+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6e0c050780 con 0x7f6e1c0b0f70 2023-10-26T05:53:37.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:37.919+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6e1c066d00 con 0x7f6e1c0b0f70 2023-10-26T05:53:38.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.351+0000 7f6e11ffb640 1 -- 172.21.15.71:0/1243420240 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6e0c053da0 con 0x7f6e1c0b0f70 2023-10-26T05:53:38.358 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:53:38.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e00066470 msgr2=0x7f6e00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:38.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e00066470 0x7f6e00068930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009f20 tx=0x7f6e180094f0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 msgr2=0x7f6e1c0ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:38.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 0x7f6e1c0ade20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00c490 tx=0x7f6e0c001e80 comp rx=0 tx=0).stop 2023-10-26T05:53:38.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 shutdown_connections 2023-10-26T05:53:38.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6e1c141370 0x7f6e1c06b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6e00066470 0x7f6e00068930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1c0b0f70 0x7f6e1c0ade20 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 --2- 172.21.15.71:0/1243420240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e1c070c60 0x7f6e1c1f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 >> 172.21.15.71:0/1243420240 conn(0x7f6e1c0afe40 msgr2=0x7f6e1c13bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:38.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 shutdown_connections 2023-10-26T05:53:38.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.355+0000 7f6e234b0640 1 -- 172.21.15.71:0/1243420240 wait complete. 2023-10-26T05:53:38.387 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:53:38.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- 172.21.15.71:0/129984838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 msgr2=0x7f797800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:38.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 --2- 172.21.15.71:0/129984838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f797800b990 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7988001ec0 tx=0x7f798801cf10 comp rx=0 tx=0).stop 2023-10-26T05:53:38.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- 172.21.15.71:0/129984838 shutdown_connections 2023-10-26T05:53:38.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 --2- 172.21.15.71:0/129984838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f797800b990 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- 172.21.15.71:0/129984838 >> 172.21.15.71:0/129984838 conn(0x7f798c0b0c10 msgr2=0x7f798c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:38.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- 172.21.15.71:0/129984838 shutdown_connections 2023-10-26T05:53:38.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- 172.21.15.71:0/129984838 wait complete. 2023-10-26T05:53:38.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 Processor -- start 2023-10-26T05:53:38.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- start start 2023-10-26T05:53:38.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f798c14c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:38.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f798c06d580 0x7f798c0c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:38.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f798c0c66f0 0x7f798c14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0c6b90 con 0x7f79780052b0 2023-10-26T05:53:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0c6e70 con 0x7f798c0c66f0 2023-10-26T05:53:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f79946ba640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798c0c4390 con 0x7f798c06d580 2023-10-26T05:53:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f798c14c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:38.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f798c14c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34780/0 (socket says 172.21.15.71:34780) 2023-10-26T05:53:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 -- 172.21.15.71:0/1758756226 learned_addr learned my addr 172.21.15.71:0/1758756226 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992c57640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f798c0c66f0 0x7f798c14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7991c55640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f798c06d580 0x7f798c0c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 -- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f798c06d580 msgr2=0x7f798c0c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:38.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f798c06d580 0x7f798c0c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 -- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f798c0c66f0 msgr2=0x7f798c14dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:38.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.511+0000 7f7992456640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f798c0c66f0 0x7f798c14dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:38.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7992456640 1 -- 172.21.15.71:0/1758756226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0c4670 con 0x7f79780052b0 2023-10-26T05:53:38.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7992c57640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f798c0c66f0 0x7f798c14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:38.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7992456640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f798c14c3c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f798400c780 tx=0x7f798400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:38.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7984004b70 con 0x7f79780052b0 2023-10-26T05:53:38.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f798400ce90 con 0x7f79780052b0 2023-10-26T05:53:38.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798c0c4950 con 0x7f79780052b0 2023-10-26T05:53:38.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798400aab0 con 0x7f79780052b0 2023-10-26T05:53:38.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798c06da00 con 0x7f79780052b0 2023-10-26T05:53:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798c066b10 con 0x7f79780052b0 2023-10-26T05:53:38.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f798400ac10 con 0x7f79780052b0 2023-10-26T05:53:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7982ffd640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7970066470 0x7f7970068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7984084160 con 0x7f79780052b0 2023-10-26T05:53:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7991c55640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7970066470 0x7f7970068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:38.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.515+0000 7f7991c55640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7970066470 0x7f7970068930 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f798c0c22b0 tx=0x7f797c00eda0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:38.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.519+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7984051740 con 0x7f79780052b0 2023-10-26T05:53:38.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:38.663+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f798c06dc80 con 0x7f79780052b0 2023-10-26T05:53:39.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.363+0000 7f7982ffd640 1 -- 172.21.15.71:0/1758756226 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7984054d60 con 0x7f79780052b0 2023-10-26T05:53:39.373 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:53:39.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7970066470 msgr2=0x7f7970068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:39.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7970066470 0x7f7970068930 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f798c0c22b0 tx=0x7f797c00eda0 comp rx=0 tx=0).stop 2023-10-26T05:53:39.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 msgr2=0x7f798c14c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:39.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f798c14c3c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f798400c780 tx=0x7f798400cc40 comp rx=0 tx=0).stop 2023-10-26T05:53:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 shutdown_connections 2023-10-26T05:53:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f798c0c66f0 0x7f798c14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f798c06d580 0x7f798c0c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7970066470 0x7f7970068930 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:39.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 --2- 172.21.15.71:0/1758756226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79780052b0 0x7f798c14c3c0 secure :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f798400c780 tx=0x7f798400cc40 comp rx=0 tx=0).stop 2023-10-26T05:53:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 >> 172.21.15.71:0/1758756226 conn(0x7f798c0b0c10 msgr2=0x7f798c1593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:39.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.371+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 shutdown_connections 2023-10-26T05:53:39.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.379+0000 7f79946ba640 1 -- 172.21.15.71:0/1758756226 wait complete. 2023-10-26T05:53:39.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:53:39.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/1352018274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d40039b0 msgr2=0x7fe9d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:39.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/1352018274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9d40039b0 0x7fe9d4003dd0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fb0 tx=0x7fe9d801cf90 comp rx=0 tx=0).stop 2023-10-26T05:53:39.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ecd75640 1 -- 172.21.15.71:0/1352018274 reap_dead start 2023-10-26T05:53:39.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/1352018274 shutdown_connections 2023-10-26T05:53:39.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/1352018274 >> 172.21.15.71:0/1352018274 conn(0x7fe9e80cdd80 msgr2=0x7fe9e80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:39.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/1352018274 shutdown_connections 2023-10-26T05:53:39.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/1352018274 wait complete. 2023-10-26T05:53:39.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 Processor -- start 2023-10-26T05:53:39.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- start start 2023-10-26T05:53:39.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 0x7fe9e80c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:39.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9e80c9db0 0x7fe9e80c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:39.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9e80cb930 0x7fe9e80cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:39.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e806a650 con 0x7fe9e8159a80 2023-10-26T05:53:39.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e80cc2f0 con 0x7fe9e80c9db0 2023-10-26T05:53:39.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9ee7d8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e80c5580 con 0x7fe9e80cb930 2023-10-26T05:53:39.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 0x7fe9e80c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:39.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 0x7fe9e80c6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34790/0 (socket says 172.21.15.71:34790) 2023-10-26T05:53:39.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.531+0000 7fe9e7fff640 1 -- 172.21.15.71:0/585246749 learned_addr learned my addr 172.21.15.71:0/585246749 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:39.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e77fe640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9e80c9db0 0x7fe9e80c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:39.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9ecd75640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9e80cb930 0x7fe9e80cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:39.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e7fff640 1 -- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9e80cb930 msgr2=0x7fe9e80cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:39.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e7fff640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9e80cb930 0x7fe9e80cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:39.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e7fff640 1 -- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9e80c9db0 msgr2=0x7fe9e80c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:39.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e7fff640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9e80c9db0 0x7fe9e80c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:39.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e7fff640 1 -- 172.21.15.71:0/585246749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8159e60 con 0x7fe9e8159a80 2023-10-26T05:53:39.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e7fff640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 0x7fe9e80c6200 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe9d801cdc0 tx=0x7fe9d8000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:39.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8006370 con 0x7fe9e8159a80 2023-10-26T05:53:39.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9d80010d0 con 0x7fe9e8159a80 2023-10-26T05:53:39.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e80c6740 con 0x7fe9e8159a80 2023-10-26T05:53:39.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8003940 con 0x7fe9e8159a80 2023-10-26T05:53:39.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e80c4f30 con 0x7fe9e8159a80 2023-10-26T05:53:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e8066b10 con 0x7fe9e8159a80 2023-10-26T05:53:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9d8003aa0 con 0x7fe9e8159a80 2023-10-26T05:53:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e4ff9640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9c80664a0 0x7fe9c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe9d80b4040 con 0x7fe9e8159a80 2023-10-26T05:53:39.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.535+0000 7fe9e77fe640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9c80664a0 0x7fe9c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:39.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.539+0000 7fe9e77fe640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9c80664a0 0x7fe9c8068960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe9e81f8640 tx=0x7fe9d00091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:39.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.539+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe9d80ebdb0 con 0x7fe9e8159a80 2023-10-26T05:53:39.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:39.727+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe9e80721e0 con 0x7fe9e8159a80 2023-10-26T05:53:40.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.399+0000 7fe9e4ff9640 1 -- 172.21.15.71:0/585246749 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe9d80ef3d0 con 0x7fe9e8159a80 2023-10-26T05:53:40.409 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 16 and data pool 17 2023-10-26T05:53:40.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9c80664a0 msgr2=0x7fe9c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9c80664a0 0x7fe9c8068960 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe9e81f8640 tx=0x7fe9d00091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 msgr2=0x7fe9e80c6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 0x7fe9e80c6200 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe9d801cdc0 tx=0x7fe9d8000f60 comp rx=0 tx=0).stop 2023-10-26T05:53:40.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 shutdown_connections 2023-10-26T05:53:40.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9e80cb930 0x7fe9e80cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9e80c9db0 0x7fe9e80c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9c80664a0 0x7fe9c8068960 secure :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fe9e81f8640 tx=0x7fe9d00091e0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 --2- 172.21.15.71:0/585246749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9e8159a80 0x7fe9e80c6200 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 >> 172.21.15.71:0/585246749 conn(0x7fe9e80cdd80 msgr2=0x7fe9e8166680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:40.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 shutdown_connections 2023-10-26T05:53:40.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.407+0000 7fe9ee7d8640 1 -- 172.21.15.71:0/585246749 wait complete. 2023-10-26T05:53:40.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:40.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/2217360223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9944003980 msgr2=0x7f9944003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 --2- 172.21.15.71:0/2217360223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9944003980 0x7f9944003d80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f994c006fd0 tx=0x7f994c01cf90 comp rx=0 tx=0).stop 2023-10-26T05:53:40.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/2217360223 shutdown_connections 2023-10-26T05:53:40.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 --2- 172.21.15.71:0/2217360223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9944003980 0x7f9944003d80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/2217360223 >> 172.21.15.71:0/2217360223 conn(0x7f995c06b780 msgr2=0x7f995c06bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:40.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/2217360223 shutdown_connections 2023-10-26T05:53:40.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/2217360223 wait complete. 2023-10-26T05:53:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 Processor -- start 2023-10-26T05:53:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- start start 2023-10-26T05:53:40.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9944003980 0x7f995c13f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 0x7f995c0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f995c0ab9c0 0x7f995c1e8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0abe40 con 0x7f995c156d80 2023-10-26T05:53:40.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c140f30 con 0x7f9944003980 2023-10-26T05:53:40.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995c13fa20 con 0x7f995c0ab9c0 2023-10-26T05:53:40.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 0x7f995c0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 0x7f995c0ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34800/0 (socket says 172.21.15.71:34800) 2023-10-26T05:53:40.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 -- 172.21.15.71:0/978285660 learned_addr learned my addr 172.21.15.71:0/978285660 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:40.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995b7fe640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9944003980 0x7f995c13f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995bfff640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f995c0ab9c0 0x7f995c1e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 -- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f995c0ab9c0 msgr2=0x7f995c1e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f995c0ab9c0 0x7f995c1e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 -- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9944003980 msgr2=0x7f995c13f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9944003980 0x7f995c13f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 -- 172.21.15.71:0/978285660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c13fd00 con 0x7f995c156d80 2023-10-26T05:53:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995b7fe640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9944003980 0x7f995c13f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:53:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f995affd640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 0x7f995c0ab480 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f995000e6a0 tx=0x7f995000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000ca50 con 0x7f995c156d80 2023-10-26T05:53:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f995000cca0 con 0x7f995c156d80 2023-10-26T05:53:40.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c13ffe0 con 0x7f995c156d80 2023-10-26T05:53:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99500116d0 con 0x7f995c156d80 2023-10-26T05:53:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.551+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c1405a0 con 0x7f995c156d80 2023-10-26T05:53:40.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.555+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c066b10 con 0x7f995c156d80 2023-10-26T05:53:40.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.555+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9950011830 con 0x7f995c156d80 2023-10-26T05:53:40.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.555+0000 7f9943fff640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99480664a0 0x7f9948068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.555+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9950084850 con 0x7f995c156d80 2023-10-26T05:53:40.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.555+0000 7f995b7fe640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99480664a0 0x7f9948068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.555+0000 7f995b7fe640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99480664a0 0x7f9948068960 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ac000 tx=0x7f994c0b0040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:40.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.559+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9950051cd0 con 0x7f995c156d80 2023-10-26T05:53:40.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.715+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f995c0721e0 con 0x7f995c156d80 2023-10-26T05:53:40.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.715+0000 7f9943fff640 1 -- 172.21.15.71:0/978285660 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f99500552f0 con 0x7f995c156d80 2023-10-26T05:53:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:40.725 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":137,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:39.734713+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:40.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99480664a0 msgr2=0x7f9948068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99480664a0 0x7f9948068960 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ac000 tx=0x7f994c0b0040 comp rx=0 tx=0).stop 2023-10-26T05:53:40.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 msgr2=0x7f995c0ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 0x7f995c0ab480 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f995000e6a0 tx=0x7f995000eb60 comp rx=0 tx=0).stop 2023-10-26T05:53:40.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 shutdown_connections 2023-10-26T05:53:40.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f995c0ab9c0 0x7f995c1e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99480664a0 0x7f9948068960 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995c156d80 0x7f995c0ab480 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 --2- 172.21.15.71:0/978285660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9944003980 0x7f995c13f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 >> 172.21.15.71:0/978285660 conn(0x7f995c06b780 msgr2=0x7f995c0ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:40.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 shutdown_connections 2023-10-26T05:53:40.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.719+0000 7f9961f46640 1 -- 172.21.15.71:0/978285660 wait complete. 2023-10-26T05:53:40.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:40.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/2140080744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 msgr2=0x7f99c806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 --2- 172.21.15.71:0/2140080744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c806aa70 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009930 tx=0x7f99c401c800 comp rx=0 tx=0).stop 2023-10-26T05:53:40.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/2140080744 shutdown_connections 2023-10-26T05:53:40.874 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 --2- 172.21.15.71:0/2140080744 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80cd880 0x7f99c815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 --2- 172.21.15.71:0/2140080744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c806aa70 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/2140080744 >> 172.21.15.71:0/2140080744 conn(0x7f99c80cdd80 msgr2=0x7f99c80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:40.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/2140080744 shutdown_connections 2023-10-26T05:53:40.875 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/2140080744 wait complete. 2023-10-26T05:53:40.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 Processor -- start 2023-10-26T05:53:40.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- start start 2023-10-26T05:53:40.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c80cd880 0x7f99c81507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c81515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80d0850 0x7f99c8151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c815c8b0 con 0x7f99c80d0470 2023-10-26T05:53:40.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8150d10 con 0x7f99c80cd880 2023-10-26T05:53:40.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c815a2c0 con 0x7f99c80d0850 2023-10-26T05:53:40.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c81515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c81515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34818/0 (socket says 172.21.15.71:34818) 2023-10-26T05:53:40.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 -- 172.21.15.71:0/3301655613 learned_addr learned my addr 172.21.15.71:0/3301655613 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:40.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cd625640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c80cd880 0x7f99c81507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cde26640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80d0850 0x7f99c8151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 -- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80d0850 msgr2=0x7f99c8151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80d0850 0x7f99c8151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 -- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c80cd880 msgr2=0x7f99c81507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:40.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c80cd880 0x7f99c81507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:40.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 -- 172.21.15.71:0/3301655613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009560 con 0x7f99c80d0470 2023-10-26T05:53:40.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cde26640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80d0850 0x7f99c8151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:40.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cce24640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c81515e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c3a0 tx=0x7f99b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:40.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80052d0 con 0x7f99c80d0470 2023-10-26T05:53:40.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f99b8003620 con 0x7f99c80d0470 2023-10-26T05:53:40.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c815a5a0 con 0x7f99c80d0470 2023-10-26T05:53:40.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003a70 con 0x7f99c80d0470 2023-10-26T05:53:40.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c815ab60 con 0x7f99c80d0470 2023-10-26T05:53:40.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.867+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c8066b10 con 0x7f99c80d0470 2023-10-26T05:53:40.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.871+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f99b8003bd0 con 0x7f99c80d0470 2023-10-26T05:53:40.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.871+0000 7f99bdffb640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99ac0664a0 0x7f99ac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:40.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.871+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99b80835b0 con 0x7f99c80d0470 2023-10-26T05:53:40.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.871+0000 7f99cd625640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99ac0664a0 0x7f99ac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:40.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.871+0000 7f99cd625640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99ac0664a0 0x7f99ac068960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f99c40af000 tx=0x7f99c401b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:40.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:40.875+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f99b8050ac0 con 0x7f99c80d0470 2023-10-26T05:53:41.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.027+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c80721e0 con 0x7f99c80d0470 2023-10-26T05:53:41.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99bdffb640 1 -- 172.21.15.71:0/3301655613 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f99b80540e0 con 0x7f99c80d0470 2023-10-26T05:53:41.038 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:41.038 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":137,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:39.734713+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:41.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99ac0664a0 msgr2=0x7f99ac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99ac0664a0 0x7f99ac068960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f99c40af000 tx=0x7f99c401b040 comp rx=0 tx=0).stop 2023-10-26T05:53:41.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 msgr2=0x7f99c81515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c81515e0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c3a0 tx=0x7f99b800c860 comp rx=0 tx=0).stop 2023-10-26T05:53:41.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 shutdown_connections 2023-10-26T05:53:41.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99c80d0850 0x7f99c8151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99ac0664a0 0x7f99ac068960 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99c80d0470 0x7f99c81515e0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 --2- 172.21.15.71:0/3301655613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99c80cd880 0x7f99c81507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 >> 172.21.15.71:0/3301655613 conn(0x7f99c80cdd80 msgr2=0x7f99c80b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:41.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 shutdown_connections 2023-10-26T05:53:41.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.031+0000 7f99cf889640 1 -- 172.21.15.71:0/3301655613 wait complete. 2023-10-26T05:53:41.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:53:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- 172.21.15.71:0/481512565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80c9780 msgr2=0x7fb1d80b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/481512565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80c9780 0x7fb1d80b0bb0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009840 tx=0x7fb1c801c740 comp rx=0 tx=0).stop 2023-10-26T05:53:41.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- 172.21.15.71:0/481512565 shutdown_connections 2023-10-26T05:53:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/481512565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80c9780 0x7fb1d80b0bb0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/481512565 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1d81591f0 0x7fb1d806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- 172.21.15.71:0/481512565 >> 172.21.15.71:0/481512565 conn(0x7fb1d806fd20 msgr2=0x7fb1d80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- 172.21.15.71:0/481512565 shutdown_connections 2023-10-26T05:53:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- 172.21.15.71:0/481512565 wait complete. 2023-10-26T05:53:41.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 Processor -- start 2023-10-26T05:53:41.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- start start 2023-10-26T05:53:41.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 0x7fb1d814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1d80c9780 0x7fb1d80c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1d81591f0 0x7fb1d8157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d814d3b0 con 0x7fb1d80b10f0 2023-10-26T05:53:41.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80c5aa0 con 0x7fb1d80c9780 2023-10-26T05:53:41.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1df4fb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d8162400 con 0x7fb1d81591f0 2023-10-26T05:53:41.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 0x7fb1d814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 0x7fb1d814ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34844/0 (socket says 172.21.15.71:34844) 2023-10-26T05:53:41.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 -- 172.21.15.71:0/2232857446 learned_addr learned my addr 172.21.15.71:0/2232857446 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:41.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dca96640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1d80c9780 0x7fb1d80c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dda98640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1d81591f0 0x7fb1d8157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 -- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1d81591f0 msgr2=0x7fb1d8157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1d81591f0 0x7fb1d8157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 -- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1d80c9780 msgr2=0x7fb1d80c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1d80c9780 0x7fb1d80c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.171+0000 7fb1dd297640 1 -- 172.21.15.71:0/2232857446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80094a0 con 0x7fb1d80b10f0 2023-10-26T05:53:41.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1dd297640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 0x7fb1d814ce70 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40058d0 tx=0x7fb1d400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:41.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4009c30 con 0x7fb1d80b10f0 2023-10-26T05:53:41.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb1d4009e80 con 0x7fb1d80b10f0 2023-10-26T05:53:41.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d80c31a0 con 0x7fb1d80b10f0 2023-10-26T05:53:41.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d40106a0 con 0x7fb1d80b10f0 2023-10-26T05:53:41.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d80c57c0 con 0x7fb1d80b10f0 2023-10-26T05:53:41.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d8066d00 con 0x7fb1d80b10f0 2023-10-26T05:53:41.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb1d4010800 con 0x7fb1d80b10f0 2023-10-26T05:53:41.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1cdffb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb1bc066470 0x7fb1bc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1d4084370 con 0x7fb1d80b10f0 2023-10-26T05:53:41.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1dca96640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb1bc066470 0x7fb1bc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.175+0000 7fb1dca96640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb1bc066470 0x7fb1bc068930 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80af000 tx=0x7fb1c801af80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:41.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.179+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb1d40517f0 con 0x7fb1d80b10f0 2023-10-26T05:53:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.355+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb1d8153270 con 0x7fb1d80b10f0 2023-10-26T05:53:41.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1cdffb640 1 -- 172.21.15.71:0/2232857446 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb1d4054e10 con 0x7fb1d80b10f0 2023-10-26T05:53:41.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb1bc066470 msgr2=0x7fb1bc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb1bc066470 0x7fb1bc068930 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80af000 tx=0x7fb1c801af80 comp rx=0 tx=0).stop 2023-10-26T05:53:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 msgr2=0x7fb1d814ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 0x7fb1d814ce70 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fb1d40058d0 tx=0x7fb1d400dde0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 shutdown_connections 2023-10-26T05:53:41.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb1d81591f0 0x7fb1d8157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1d80c9780 0x7fb1d80c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb1bc066470 0x7fb1bc068930 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 --2- 172.21.15.71:0/2232857446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1d80b10f0 0x7fb1d814ce70 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 >> 172.21.15.71:0/2232857446 conn(0x7fb1d806fd20 msgr2=0x7fb1d80ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:41.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 shutdown_connections 2023-10-26T05:53:41.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.403+0000 7fb1df4fb640 1 -- 172.21.15.71:0/2232857446 wait complete. 2023-10-26T05:53:41.429 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:53:41.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:53:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 -- 172.21.15.71:0/832722676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 msgr2=0x7f4aac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/832722676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4aac004c50 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8005fd0 tx=0x7f4aa801d1e0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 -- 172.21.15.71:0/832722676 shutdown_connections 2023-10-26T05:53:41.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/832722676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4aac004c50 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 -- 172.21.15.71:0/832722676 >> 172.21.15.71:0/832722676 conn(0x7f4abc0ca2c0 msgr2=0x7f4abc0ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:41.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 -- 172.21.15.71:0/832722676 shutdown_connections 2023-10-26T05:53:41.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.539+0000 7f4ac2d84640 1 -- 172.21.15.71:0/832722676 wait complete. 2023-10-26T05:53:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 Processor -- start 2023-10-26T05:53:41.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- start start 2023-10-26T05:53:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4abc1f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4abc155810 0x7f4abc140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4abc0cc390 0x7f4abc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc0ccd50 con 0x7f4aac0047d0 2023-10-26T05:53:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc156a00 con 0x7f4abc0cc390 2023-10-26T05:53:41.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4abc1f7340 con 0x7f4abc155810 2023-10-26T05:53:41.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4abc1f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4abc1f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34866/0 (socket says 172.21.15.71:34866) 2023-10-26T05:53:41.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 -- 172.21.15.71:0/2357169039 learned_addr learned my addr 172.21.15.71:0/2357169039 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:41.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4abbfff640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4abc155810 0x7f4abc140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 -- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4abc155810 msgr2=0x7f4abc140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac1321640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4abc0cc390 0x7f4abc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4abc155810 0x7f4abc140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 -- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4abc0cc390 msgr2=0x7f4abc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:41.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4abc0cc390 0x7f4abc0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:41.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 -- 172.21.15.71:0/2357169039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc141b60 con 0x7f4aac0047d0 2023-10-26T05:53:41.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4abbfff640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4abc155810 0x7f4abc140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:41.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac1321640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4abc0cc390 0x7f4abc0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:41.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac0b20640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4abc1f6e00 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000d800 tx=0x7f4ab000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:41.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00060d0 con 0x7f4aac0047d0 2023-10-26T05:53:41.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4ab0006320 con 0x7f4aac0047d0 2023-10-26T05:53:41.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4abc155d80 con 0x7f4aac0047d0 2023-10-26T05:53:41.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000ba70 con 0x7f4aac0047d0 2023-10-26T05:53:41.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc156340 con 0x7f4aac0047d0 2023-10-26T05:53:41.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.543+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4abc066d00 con 0x7f4aac0047d0 2023-10-26T05:53:41.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.547+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ab000bbd0 con 0x7f4aac0047d0 2023-10-26T05:53:41.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.547+0000 7f4ab97fa640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4aa0066470 0x7f4aa0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:41.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.547+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ab0084cc0 con 0x7f4aac0047d0 2023-10-26T05:53:41.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.547+0000 7f4abbfff640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4aa0066470 0x7f4aa0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:41.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.547+0000 7f4abbfff640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4aa0066470 0x7f4aa0068930 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80ad000 tx=0x7f4aa80af040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:41.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.551+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ab00522b0 con 0x7f4aac0047d0 2023-10-26T05:53:41.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:41.727+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4abc06cec0 con 0x7f4aac0047d0 2023-10-26T05:53:42.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.423+0000 7f4ab97fa640 1 -- 172.21.15.71:0/2357169039 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v51) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ab00140c0 con 0x7f4aac0047d0 2023-10-26T05:53:42.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4aa0066470 msgr2=0x7f4aa0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4aa0066470 0x7f4aa0068930 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4aa80ad000 tx=0x7f4aa80af040 comp rx=0 tx=0).stop 2023-10-26T05:53:42.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 msgr2=0x7f4abc1f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4abc1f6e00 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000d800 tx=0x7f4ab000dcc0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 shutdown_connections 2023-10-26T05:53:42.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4abc0cc390 0x7f4abc0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4abc155810 0x7f4abc140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4aa0066470 0x7f4aa0068930 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 --2- 172.21.15.71:0/2357169039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4aac0047d0 0x7f4abc1f6e00 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 >> 172.21.15.71:0/2357169039 conn(0x7f4abc0ca2c0 msgr2=0x7f4abc0c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:42.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 shutdown_connections 2023-10-26T05:53:42.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.431+0000 7f4ac2d84640 1 -- 172.21.15.71:0/2357169039 wait complete. 2023-10-26T05:53:42.461 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:53:42.462 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:42.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/3387353712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 msgr2=0x7f58fc0ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 --2- 172.21.15.71:0/3387353712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc0ac740 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009900 tx=0x7f58ec01c8c0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/3387353712 shutdown_connections 2023-10-26T05:53:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 --2- 172.21.15.71:0/3387353712 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58fc15bbc0 0x7f58fc070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 --2- 172.21.15.71:0/3387353712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc0ac740 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/3387353712 >> 172.21.15.71:0/3387353712 conn(0x7f58fc069770 msgr2=0x7f58fc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:42.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/3387353712 shutdown_connections 2023-10-26T05:53:42.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/3387353712 wait complete. 2023-10-26T05:53:42.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 Processor -- start 2023-10-26T05:53:42.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- start start 2023-10-26T05:53:42.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc1357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58fc13d800 0x7f58fc1f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58fc15bbc0 0x7f58fc1f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0b10a0 con 0x7f58fc13d420 2023-10-26T05:53:42.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc137e90 con 0x7f58fc13d800 2023-10-26T05:53:42.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc06aef0 con 0x7f58fc15bbc0 2023-10-26T05:53:42.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc1357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:42.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc1357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34380/0 (socket says 172.21.15.71:34380) 2023-10-26T05:53:42.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 -- 172.21.15.71:0/1883424609 learned_addr learned my addr 172.21.15.71:0/1883424609 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:42.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 -- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58fc15bbc0 msgr2=0x7f58fc1f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58fc15bbc0 0x7f58fc1f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 -- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58fc13d800 msgr2=0x7f58fc1f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58fc13d800 0x7f58fc1f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 -- 172.21.15.71:0/1883424609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec009560 con 0x7f58fc13d420 2023-10-26T05:53:42.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58fb7fe640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc1357d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f58ec01a4f0 tx=0x7f58ec0a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:42.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003800 con 0x7f58fc13d420 2023-10-26T05:53:42.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f58ec003960 con 0x7f58fc13d420 2023-10-26T05:53:42.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58fc1f3910 con 0x7f58fc13d420 2023-10-26T05:53:42.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003d70 con 0x7f58fc13d420 2023-10-26T05:53:42.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.583+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58fc0b0cc0 con 0x7f58fc13d420 2023-10-26T05:53:42.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.587+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58fc066b10 con 0x7f58fc13d420 2023-10-26T05:53:42.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.587+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58ec0b4040 con 0x7f58fc13d420 2023-10-26T05:53:42.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.587+0000 7f58f8ff9640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f58dc066470 0x7f58dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.587+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58ec11ec20 con 0x7f58fc13d420 2023-10-26T05:53:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.587+0000 7f58f3fff640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f58dc066470 0x7f58dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.587+0000 7f58f3fff640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f58dc066470 0x7f58dc068930 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f58e4007fd0 tx=0x7f58e4007980 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:42.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.591+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f58ec0ec210 con 0x7f58fc13d420 2023-10-26T05:53:42.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58fc0721e0 con 0x7f58fc13d420 2023-10-26T05:53:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f58f8ff9640 1 -- 172.21.15.71:0/1883424609 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f58ec0ef830 con 0x7f58fc13d420 2023-10-26T05:53:42.771 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:42.771 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:53:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:53:42.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f58dc066470 msgr2=0x7f58dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f58dc066470 0x7f58dc068930 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f58e4007fd0 tx=0x7f58e4007980 comp rx=0 tx=0).stop 2023-10-26T05:53:42.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 msgr2=0x7f58fc1357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc1357d0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f58ec01a4f0 tx=0x7f58ec0a56f0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 shutdown_connections 2023-10-26T05:53:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58fc15bbc0 0x7f58fc1f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58fc13d800 0x7f58fc1f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f58dc066470 0x7f58dc068930 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 --2- 172.21.15.71:0/1883424609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58fc13d420 0x7f58fc1357d0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 >> 172.21.15.71:0/1883424609 conn(0x7f58fc069770 msgr2=0x7f58fc159890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 shutdown_connections 2023-10-26T05:53:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.763+0000 7f5901d5b640 1 -- 172.21.15.71:0/1883424609 wait complete. 2023-10-26T05:53:42.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:53:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/4291854176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 msgr2=0x7f61b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 --2- 172.21.15.71:0/4291854176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61b0003da0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f61b400ac70 tx=0x7f61b401e070 comp rx=0 tx=0).stop 2023-10-26T05:53:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/4291854176 shutdown_connections 2023-10-26T05:53:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 --2- 172.21.15.71:0/4291854176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61b0003da0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/4291854176 >> 172.21.15.71:0/4291854176 conn(0x7f61c4069770 msgr2=0x7f61c4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/4291854176 shutdown_connections 2023-10-26T05:53:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/4291854176 wait complete. 2023-10-26T05:53:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 Processor -- start 2023-10-26T05:53:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- start start 2023-10-26T05:53:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61c413b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c41547b0 0x7f61c4154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61c40b1630 0x7f61c40b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40b2020 con 0x7f61b0003980 2023-10-26T05:53:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4155170 con 0x7f61c41547b0 2023-10-26T05:53:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4153540 con 0x7f61c40b1630 2023-10-26T05:53:42.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61c413b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:42.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61c413b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34400/0 (socket says 172.21.15.71:34400) 2023-10-26T05:53:42.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 -- 172.21.15.71:0/962411019 learned_addr learned my addr 172.21.15.71:0/962411019 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:42.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c8f3b640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61c40b1630 0x7f61c40b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:42.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61bb7fe640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c41547b0 0x7f61c4154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:42.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 -- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61c40b1630 msgr2=0x7f61c40b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61c40b1630 0x7f61c40b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 -- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c41547b0 msgr2=0x7f61c4154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:42.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c41547b0 0x7f61c4154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:42.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 -- 172.21.15.71:0/962411019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4153790 con 0x7f61b0003980 2023-10-26T05:53:42.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c8f3b640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61c40b1630 0x7f61c40b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:42.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c3fff640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61c413b9e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f61b40b0040 tx=0x7f61b40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:42.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4004a90 con 0x7f61b0003980 2023-10-26T05:53:42.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61b4004ce0 con 0x7f61b0003980 2023-10-26T05:53:42.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40b76a0 con 0x7f61b0003980 2023-10-26T05:53:42.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4153a70 con 0x7f61b0003980 2023-10-26T05:53:42.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.903+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c4154030 con 0x7f61b0003980 2023-10-26T05:53:42.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.907+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c4066b10 con 0x7f61b0003980 2023-10-26T05:53:42.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.907+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61b40b7800 con 0x7f61b0003980 2023-10-26T05:53:42.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.907+0000 7f61c17fa640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61a4066470 0x7f61a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:42.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.907+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b411e7a0 con 0x7f61b0003980 2023-10-26T05:53:42.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.907+0000 7f61bb7fe640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61a4066470 0x7f61a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:42.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.907+0000 7f61bb7fe640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61a4066470 0x7f61a4068930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f61c40b29a0 tx=0x7f61ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:42.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:42.911+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61b40ebc20 con 0x7f61b0003980 2023-10-26T05:53:43.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.055+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61c4150010 con 0x7f61b0003980 2023-10-26T05:53:43.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61c17fa640 1 -- 172.21.15.71:0/962411019 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16444 (secure 0 0 0) 0x7f61c4150010 con 0x7f61b0003980 2023-10-26T05:53:43.066 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:43.066 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":138,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:53:41.372675+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:53:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61a4066470 msgr2=0x7f61a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61a4066470 0x7f61a4068930 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f61c40b29a0 tx=0x7f61ac009040 comp rx=0 tx=0).stop 2023-10-26T05:53:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 msgr2=0x7f61c413b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61c413b9e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f61b40b0040 tx=0x7f61b40056a0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 shutdown_connections 2023-10-26T05:53:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f61c40b1630 0x7f61c40b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61c41547b0 0x7f61c4154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61a4066470 0x7f61a4068930 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 --2- 172.21.15.71:0/962411019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61b0003980 0x7f61c413b9e0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 >> 172.21.15.71:0/962411019 conn(0x7f61c4069770 msgr2=0x7f61c40ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:43.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 shutdown_connections 2023-10-26T05:53:43.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.059+0000 7f61ca99e640 1 -- 172.21.15.71:0/962411019 wait complete. 2023-10-26T05:53:43.084 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:53:43.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:43.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- 172.21.15.71:0/2977762557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1440007720 msgr2=0x7f1440009b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 --2- 172.21.15.71:0/2977762557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1440007720 0x7f1440009b50 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f1444009b60 tx=0x7f144401d250 comp rx=0 tx=0).stop 2023-10-26T05:53:43.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 -- 172.21.15.71:0/2977762557 reap_dead start 2023-10-26T05:53:43.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- 172.21.15.71:0/2977762557 shutdown_connections 2023-10-26T05:53:43.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- 172.21.15.71:0/2977762557 >> 172.21.15.71:0/2977762557 conn(0x7f1454069770 msgr2=0x7f1454069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:43.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- 172.21.15.71:0/2977762557 shutdown_connections 2023-10-26T05:53:43.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- 172.21.15.71:0/2977762557 wait complete. 2023-10-26T05:53:43.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 Processor -- start 2023-10-26T05:53:43.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- start start 2023-10-26T05:53:43.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440007720 0x7f14541f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14540adcc0 0x7f1454150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 0x7f145413d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14541516c0 con 0x7f1454151230 2023-10-26T05:53:43.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14540accb0 con 0x7f1440007720 2023-10-26T05:53:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f145b265640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14540acf00 con 0x7f14540adcc0 2023-10-26T05:53:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 0x7f145413d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 0x7f145413d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34410/0 (socket says 172.21.15.71:34410) 2023-10-26T05:53:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 -- 172.21.15.71:0/2175516343 learned_addr learned my addr 172.21.15.71:0/2175516343 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:43.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1458800640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14540adcc0 0x7f1454150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459001640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440007720 0x7f14541f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 -- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14540adcc0 msgr2=0x7f1454150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14540adcc0 0x7f1454150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 -- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440007720 msgr2=0x7f14541f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440007720 0x7f14541f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 -- 172.21.15.71:0/2175516343 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14540ad1e0 con 0x7f1454151230 2023-10-26T05:53:43.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459001640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440007720 0x7f14541f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:43.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.199+0000 7f1459802640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 0x7f145413d800 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f144c000fc0 tx=0x7f144c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:43.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c015030 con 0x7f1454151230 2023-10-26T05:53:43.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f144c003600 con 0x7f1454151230 2023-10-26T05:53:43.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14540ad4c0 con 0x7f1454151230 2023-10-26T05:53:43.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c003a10 con 0x7f1454151230 2023-10-26T05:53:43.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14541f3f80 con 0x7f1454151230 2023-10-26T05:53:43.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1454066b10 con 0x7f1454151230 2023-10-26T05:53:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f144c018070 con 0x7f1454151230 2023-10-26T05:53:43.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1449ffb640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14380664a0 0x7f1438068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f144c016070 con 0x7f1454151230 2023-10-26T05:53:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1459001640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14380664a0 0x7f1438068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.203+0000 7f1459001640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14380664a0 0x7f1438068960 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1450009910 tx=0x7f1450009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:43.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.207+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f144c08a080 con 0x7f1454151230 2023-10-26T05:53:43.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.359+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14540721e0 con 0x7f14380664a0 2023-10-26T05:53:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.379+0000 7f1449ffb640 1 -- 172.21.15.71:0/2175516343 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278238 (secure 0 0 0) 0x7f14540721e0 con 0x7f14380664a0 2023-10-26T05:53:43.390 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14380664a0 msgr2=0x7f1438068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14380664a0 0x7f1438068960 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1450009910 tx=0x7f1450009500 comp rx=0 tx=0).stop 2023-10-26T05:53:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 msgr2=0x7f145413d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 0x7f145413d800 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f144c000fc0 tx=0x7f144c00da80 comp rx=0 tx=0).stop 2023-10-26T05:53:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 shutdown_connections 2023-10-26T05:53:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14380664a0 0x7f1438068960 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1454151230 0x7f145413d800 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14540adcc0 0x7f1454150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 --2- 172.21.15.71:0/2175516343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1440007720 0x7f14541f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 >> 172.21.15.71:0/2175516343 conn(0x7f1454069770 msgr2=0x7f145414d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 shutdown_connections 2023-10-26T05:53:43.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.383+0000 7f145b265640 1 -- 172.21.15.71:0/2175516343 wait complete. 2023-10-26T05:53:43.408 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2023-10-26T05:53:41.538112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6987,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555512,"statfs":{"total":1600000000,"available":1592844333,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001337"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382172+0000","last_change":"2023-10-26T05:53:39.382172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382172+0000","last_undegraded":"2023-10-26T05:53:39.382172+0000","last_fullsized":"2023-10-26T05:53:39.382172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.404252+0000","last_change":"2023-10-26T05:53:39.404252+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.404252+0000","last_undegraded":"2023-10-26T05:53:39.404252+0000","last_fullsized":"2023-10-26T05:53:39.404252+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386376+0000","last_change":"2023-10-26T05:53:39.386376+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386376+0000","last_undegraded":"2023-10-26T05:53:39.386376+0000","last_fullsized":"2023-10-26T05:53:39.386376+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383947+0000","last_change":"2023-10-26T05:53:39.383947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383947+0000","last_undegraded":"2023-10-26T05:53:39.383947+0000","last_fullsized":"2023-10-26T05:53:39.383947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375327+0000","last_change":"2023-10-26T05:53:39.375327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375327+0000","last_undegraded":"2023-10-26T05:53:39.375327+0000","last_fullsized":"2023-10-26T05:53:39.375327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385715+0000","last_change":"2023-10-26T05:53:39.385715+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385715+0000","last_undegraded":"2023-10-26T05:53:39.385715+0000","last_fullsized":"2023-10-26T05:53:39.385715+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.396242+0000","last_change":"2023-10-26T05:53:39.396242+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.396242+0000","last_undegraded":"2023-10-26T05:53:39.396242+0000","last_fullsized":"2023-10-26T05:53:39.396242+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384805+0000","last_change":"2023-10-26T05:53:39.384805+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384805+0000","last_undegraded":"2023-10-26T05:53:39.384805+0000","last_fullsized":"2023-10-26T05:53:39.384805+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383663+0000","last_change":"2023-10-26T05:53:39.383663+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383663+0000","last_undegraded":"2023-10-26T05:53:39.383663+0000","last_fullsized":"2023-10-26T05:53:39.383663+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382563+0000","last_change":"2023-10-26T05:53:39.382563+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382563+0000","last_undegraded":"2023-10-26T05:53:39.382563+0000","last_fullsized":"2023-10-26T05:53:39.382563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403072+0000","last_change":"2023-10-26T05:53:39.403072+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403072+0000","last_undegraded":"2023-10-26T05:53:39.403072+0000","last_fullsized":"2023-10-26T05:53:39.403072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380608+0000","last_change":"2023-10-26T05:53:39.380608+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380608+0000","last_undegraded":"2023-10-26T05:53:39.380608+0000","last_fullsized":"2023-10-26T05:53:39.380608+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385665+0000","last_change":"2023-10-26T05:53:39.385665+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385665+0000","last_undegraded":"2023-10-26T05:53:39.385665+0000","last_fullsized":"2023-10-26T05:53:39.385665+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.392675+0000","last_change":"2023-10-26T05:53:39.392675+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.392675+0000","last_undegraded":"2023-10-26T05:53:39.392675+0000","last_fullsized":"2023-10-26T05:53:39.392675+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385863+0000","last_change":"2023-10-26T05:53:39.385863+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385863+0000","last_undegraded":"2023-10-26T05:53:39.385863+0000","last_fullsized":"2023-10-26T05:53:39.385863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381797+0000","last_change":"2023-10-26T05:53:39.381797+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381797+0000","last_undegraded":"2023-10-26T05:53:39.381797+0000","last_fullsized":"2023-10-26T05:53:39.381797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403479+0000","last_change":"2023-10-26T05:53:39.403479+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403479+0000","last_undegraded":"2023-10-26T05:53:39.403479+0000","last_fullsized":"2023-10-26T05:53:39.403479+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388071+0000","last_change":"2023-10-26T05:53:39.388071+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388071+0000","last_undegraded":"2023-10-26T05:53:39.388071+0000","last_fullsized":"2023-10-26T05:53:39.388071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391924+0000","last_change":"2023-10-26T05:53:39.391924+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391924+0000","last_undegraded":"2023-10-26T05:53:39.391924+0000","last_fullsized":"2023-10-26T05:53:39.391924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.395104+0000","last_change":"2023-10-26T05:53:39.395104+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.395104+0000","last_undegraded":"2023-10-26T05:53:39.395104+0000","last_fullsized":"2023-10-26T05:53:39.395104+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383708+0000","last_change":"2023-10-26T05:53:39.383708+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383708+0000","last_undegraded":"2023-10-26T05:53:39.383708+0000","last_fullsized":"2023-10-26T05:53:39.383708+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379010+0000","last_change":"2023-10-26T05:53:39.379010+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379010+0000","last_undegraded":"2023-10-26T05:53:39.379010+0000","last_fullsized":"2023-10-26T05:53:39.379010+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383231+0000","last_change":"2023-10-26T05:53:39.383231+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383231+0000","last_undegraded":"2023-10-26T05:53:39.383231+0000","last_fullsized":"2023-10-26T05:53:39.383231+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378094+0000","last_change":"2023-10-26T05:53:39.378094+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378094+0000","last_undegraded":"2023-10-26T05:53:39.378094+0000","last_fullsized":"2023-10-26T05:53:39.378094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383350+0000","last_change":"2023-10-26T05:53:39.383350+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383350+0000","last_undegraded":"2023-10-26T05:53:39.383350+0000","last_fullsized":"2023-10-26T05:53:39.383350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381436+0000","last_change":"2023-10-26T05:53:39.381436+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381436+0000","last_undegraded":"2023-10-26T05:53:39.381436+0000","last_fullsized":"2023-10-26T05:53:39.381436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385544+0000","last_change":"2023-10-26T05:53:39.385544+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385544+0000","last_undegraded":"2023-10-26T05:53:39.385544+0000","last_fullsized":"2023-10-26T05:53:39.385544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380545+0000","last_change":"2023-10-26T05:53:39.380545+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380545+0000","last_undegraded":"2023-10-26T05:53:39.380545+0000","last_fullsized":"2023-10-26T05:53:39.380545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390037+0000","last_change":"2023-10-26T05:53:39.390037+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390037+0000","last_undegraded":"2023-10-26T05:53:39.390037+0000","last_fullsized":"2023-10-26T05:53:39.390037+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386407+0000","last_change":"2023-10-26T05:53:39.386407+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386407+0000","last_undegraded":"2023-10-26T05:53:39.386407+0000","last_fullsized":"2023-10-26T05:53:39.386407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381024+0000","last_change":"2023-10-26T05:53:39.381024+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381024+0000","last_undegraded":"2023-10-26T05:53:39.381024+0000","last_fullsized":"2023-10-26T05:53:39.381024+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385153+0000","last_change":"2023-10-26T05:53:39.385153+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385153+0000","last_undegraded":"2023-10-26T05:53:39.385153+0000","last_fullsized":"2023-10-26T05:53:39.385153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401005+0000","last_change":"2023-10-26T05:53:39.400795+0000","last_active":"2023-10-26T05:53:39.401005+0000","last_peered":"2023-10-26T05:53:39.401005+0000","last_clean":"2023-10-26T05:53:39.401005+0000","last_became_active":"2023-10-26T05:53:39.400245+0000","last_became_peered":"2023-10-26T05:53:39.400245+0000","last_unstale":"2023-10-26T05:53:39.401005+0000","last_undegraded":"2023-10-26T05:53:39.401005+0000","last_fullsized":"2023-10-26T05:53:39.401005+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:10:39.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387453+0000","last_change":"2023-10-26T05:53:39.387453+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387453+0000","last_undegraded":"2023-10-26T05:53:39.387453+0000","last_fullsized":"2023-10-26T05:53:39.387453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.398646+0000","last_change":"2023-10-26T05:53:39.398370+0000","last_active":"2023-10-26T05:53:39.398646+0000","last_peered":"2023-10-26T05:53:39.398646+0000","last_clean":"2023-10-26T05:53:39.398646+0000","last_became_active":"2023-10-26T05:53:39.397754+0000","last_became_peered":"2023-10-26T05:53:39.397754+0000","last_unstale":"2023-10-26T05:53:39.398646+0000","last_undegraded":"2023-10-26T05:53:39.398646+0000","last_fullsized":"2023-10-26T05:53:39.398646+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385078+0000","last_change":"2023-10-26T05:53:39.385078+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385078+0000","last_undegraded":"2023-10-26T05:53:39.385078+0000","last_fullsized":"2023-10-26T05:53:39.385078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.384194+0000","last_change":"2023-10-26T05:53:39.383805+0000","last_active":"2023-10-26T05:53:39.384194+0000","last_peered":"2023-10-26T05:53:39.384194+0000","last_clean":"2023-10-26T05:53:39.384194+0000","last_became_active":"2023-10-26T05:53:39.383392+0000","last_became_peered":"2023-10-26T05:53:39.383392+0000","last_unstale":"2023-10-26T05:53:39.384194+0000","last_undegraded":"2023-10-26T05:53:39.384194+0000","last_fullsized":"2023-10-26T05:53:39.384194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:23.746787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388327+0000","last_change":"2023-10-26T05:53:39.388327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388327+0000","last_undegraded":"2023-10-26T05:53:39.388327+0000","last_fullsized":"2023-10-26T05:53:39.388327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379011+0000","last_change":"2023-10-26T05:53:39.378932+0000","last_active":"2023-10-26T05:53:39.379011+0000","last_peered":"2023-10-26T05:53:39.379011+0000","last_clean":"2023-10-26T05:53:39.379011+0000","last_became_active":"2023-10-26T05:53:39.378663+0000","last_became_peered":"2023-10-26T05:53:39.378663+0000","last_unstale":"2023-10-26T05:53:39.379011+0000","last_undegraded":"2023-10-26T05:53:39.379011+0000","last_fullsized":"2023-10-26T05:53:39.379011+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:42.264838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380877+0000","last_change":"2023-10-26T05:53:39.380877+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380877+0000","last_undegraded":"2023-10-26T05:53:39.380877+0000","last_fullsized":"2023-10-26T05:53:39.380877+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.408090+0000","last_change":"2023-10-26T05:53:39.407951+0000","last_active":"2023-10-26T05:53:39.408090+0000","last_peered":"2023-10-26T05:53:39.408090+0000","last_clean":"2023-10-26T05:53:39.408090+0000","last_became_active":"2023-10-26T05:53:39.406096+0000","last_became_peered":"2023-10-26T05:53:39.406096+0000","last_unstale":"2023-10-26T05:53:39.408090+0000","last_undegraded":"2023-10-26T05:53:39.408090+0000","last_fullsized":"2023-10-26T05:53:39.408090+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:37.233001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383057+0000","last_change":"2023-10-26T05:53:39.383057+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383057+0000","last_undegraded":"2023-10-26T05:53:39.383057+0000","last_fullsized":"2023-10-26T05:53:39.383057+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389003+0000","last_change":"2023-10-26T05:53:39.388934+0000","last_active":"2023-10-26T05:53:39.389003+0000","last_peered":"2023-10-26T05:53:39.389003+0000","last_clean":"2023-10-26T05:53:39.389003+0000","last_became_active":"2023-10-26T05:53:39.388668+0000","last_became_peered":"2023-10-26T05:53:39.388668+0000","last_unstale":"2023-10-26T05:53:39.389003+0000","last_undegraded":"2023-10-26T05:53:39.389003+0000","last_fullsized":"2023-10-26T05:53:39.389003+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:40.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382253+0000","last_change":"2023-10-26T05:53:39.382253+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382253+0000","last_undegraded":"2023-10-26T05:53:39.382253+0000","last_fullsized":"2023-10-26T05:53:39.382253+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.405096+0000","last_change":"2023-10-26T05:53:39.404546+0000","last_active":"2023-10-26T05:53:39.405096+0000","last_peered":"2023-10-26T05:53:39.405096+0000","last_clean":"2023-10-26T05:53:39.405096+0000","last_became_active":"2023-10-26T05:53:39.403913+0000","last_became_peered":"2023-10-26T05:53:39.403913+0000","last_unstale":"2023-10-26T05:53:39.405096+0000","last_undegraded":"2023-10-26T05:53:39.405096+0000","last_fullsized":"2023-10-26T05:53:39.405096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:46:51.608953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388172+0000","last_change":"2023-10-26T05:53:39.388172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388172+0000","last_undegraded":"2023-10-26T05:53:39.388172+0000","last_fullsized":"2023-10-26T05:53:39.388172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389953+0000","last_change":"2023-10-26T05:53:39.389482+0000","last_active":"2023-10-26T05:53:39.389953+0000","last_peered":"2023-10-26T05:53:39.389953+0000","last_clean":"2023-10-26T05:53:39.389953+0000","last_became_active":"2023-10-26T05:53:39.389225+0000","last_became_peered":"2023-10-26T05:53:39.389225+0000","last_unstale":"2023-10-26T05:53:39.389953+0000","last_undegraded":"2023-10-26T05:53:39.389953+0000","last_fullsized":"2023-10-26T05:53:39.389953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.397757+0000","last_change":"2023-10-26T05:53:39.397757+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.397757+0000","last_undegraded":"2023-10-26T05:53:39.397757+0000","last_fullsized":"2023-10-26T05:53:39.397757+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378170+0000","last_change":"2023-10-26T05:53:39.376735+0000","last_active":"2023-10-26T05:53:39.378170+0000","last_peered":"2023-10-26T05:53:39.378170+0000","last_clean":"2023-10-26T05:53:39.378170+0000","last_became_active":"2023-10-26T05:53:39.376384+0000","last_became_peered":"2023-10-26T05:53:39.376384+0000","last_unstale":"2023-10-26T05:53:39.378170+0000","last_undegraded":"2023-10-26T05:53:39.378170+0000","last_fullsized":"2023-10-26T05:53:39.378170+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:22.802217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388362+0000","last_change":"2023-10-26T05:53:39.388362+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388362+0000","last_undegraded":"2023-10-26T05:53:39.388362+0000","last_fullsized":"2023-10-26T05:53:39.388362+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387881+0000","last_change":"2023-10-26T05:53:39.387783+0000","last_active":"2023-10-26T05:53:39.387881+0000","last_peered":"2023-10-26T05:53:39.387881+0000","last_clean":"2023-10-26T05:53:39.387881+0000","last_became_active":"2023-10-26T05:53:39.387261+0000","last_became_peered":"2023-10-26T05:53:39.387261+0000","last_unstale":"2023-10-26T05:53:39.387881+0000","last_undegraded":"2023-10-26T05:53:39.387881+0000","last_fullsized":"2023-10-26T05:53:39.387881+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:12.049420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.377275+0000","last_change":"2023-10-26T05:53:39.377275+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.377275+0000","last_undegraded":"2023-10-26T05:53:39.377275+0000","last_fullsized":"2023-10-26T05:53:39.377275+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.381673+0000","last_change":"2023-10-26T05:53:39.381571+0000","last_active":"2023-10-26T05:53:39.381673+0000","last_peered":"2023-10-26T05:53:39.381673+0000","last_clean":"2023-10-26T05:53:39.381673+0000","last_became_active":"2023-10-26T05:53:39.381177+0000","last_became_peered":"2023-10-26T05:53:39.381177+0000","last_unstale":"2023-10-26T05:53:39.381673+0000","last_undegraded":"2023-10-26T05:53:39.381673+0000","last_fullsized":"2023-10-26T05:53:39.381673+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385502+0000","last_change":"2023-10-26T05:53:39.385502+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385502+0000","last_undegraded":"2023-10-26T05:53:39.385502+0000","last_fullsized":"2023-10-26T05:53:39.385502+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387655+0000","last_change":"2023-10-26T05:53:39.387472+0000","last_active":"2023-10-26T05:53:39.387655+0000","last_peered":"2023-10-26T05:53:39.387655+0000","last_clean":"2023-10-26T05:53:39.387655+0000","last_became_active":"2023-10-26T05:53:39.386601+0000","last_became_peered":"2023-10-26T05:53:39.386601+0000","last_unstale":"2023-10-26T05:53:39.387655+0000","last_undegraded":"2023-10-26T05:53:39.387655+0000","last_fullsized":"2023-10-26T05:53:39.387655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:55.907120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379151+0000","last_change":"2023-10-26T05:53:39.379151+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379151+0000","last_undegraded":"2023-10-26T05:53:39.379151+0000","last_fullsized":"2023-10-26T05:53:39.379151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392144+0000","last_change":"2023-10-26T05:53:39.392081+0000","last_active":"2023-10-26T05:53:39.392144+0000","last_peered":"2023-10-26T05:53:39.392144+0000","last_clean":"2023-10-26T05:53:39.392144+0000","last_became_active":"2023-10-26T05:53:39.391881+0000","last_became_peered":"2023-10-26T05:53:39.391881+0000","last_unstale":"2023-10-26T05:53:39.392144+0000","last_undegraded":"2023-10-26T05:53:39.392144+0000","last_fullsized":"2023-10-26T05:53:39.392144+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:05:58.324723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382825+0000","last_change":"2023-10-26T05:53:39.382825+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382825+0000","last_undegraded":"2023-10-26T05:53:39.382825+0000","last_fullsized":"2023-10-26T05:53:39.382825+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392055+0000","last_change":"2023-10-26T05:53:39.391669+0000","last_active":"2023-10-26T05:53:39.392055+0000","last_peered":"2023-10-26T05:53:39.392055+0000","last_clean":"2023-10-26T05:53:39.392055+0000","last_became_active":"2023-10-26T05:53:39.390032+0000","last_became_peered":"2023-10-26T05:53:39.390032+0000","last_unstale":"2023-10-26T05:53:39.392055+0000","last_undegraded":"2023-10-26T05:53:39.392055+0000","last_fullsized":"2023-10-26T05:53:39.392055+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378569+0000","last_change":"2023-10-26T05:53:39.378569+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378569+0000","last_undegraded":"2023-10-26T05:53:39.378569+0000","last_fullsized":"2023-10-26T05:53:39.378569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.383455+0000","last_change":"2023-10-26T05:53:39.382861+0000","last_active":"2023-10-26T05:53:39.383455+0000","last_peered":"2023-10-26T05:53:39.383455+0000","last_clean":"2023-10-26T05:53:39.383455+0000","last_became_active":"2023-10-26T05:53:39.382479+0000","last_became_peered":"2023-10-26T05:53:39.382479+0000","last_unstale":"2023-10-26T05:53:39.383455+0000","last_undegraded":"2023-10-26T05:53:39.383455+0000","last_fullsized":"2023-10-26T05:53:39.383455+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:44:49.447349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375035+0000","last_change":"2023-10-26T05:53:39.375035+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375035+0000","last_undegraded":"2023-10-26T05:53:39.375035+0000","last_fullsized":"2023-10-26T05:53:39.375035+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.390084+0000","last_change":"2023-10-26T05:53:39.390022+0000","last_active":"2023-10-26T05:53:39.390084+0000","last_peered":"2023-10-26T05:53:39.390084+0000","last_clean":"2023-10-26T05:53:39.390084+0000","last_became_active":"2023-10-26T05:53:39.389733+0000","last_became_peered":"2023-10-26T05:53:39.389733+0000","last_unstale":"2023-10-26T05:53:39.390084+0000","last_undegraded":"2023-10-26T05:53:39.390084+0000","last_fullsized":"2023-10-26T05:53:39.390084+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:03:07.974034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.397807+0000","last_change":"2023-10-26T05:53:39.397727+0000","last_active":"2023-10-26T05:53:39.397807+0000","last_peered":"2023-10-26T05:53:39.397807+0000","last_clean":"2023-10-26T05:53:39.397807+0000","last_became_active":"2023-10-26T05:53:39.397564+0000","last_became_peered":"2023-10-26T05:53:39.397564+0000","last_unstale":"2023-10-26T05:53:39.397807+0000","last_undegraded":"2023-10-26T05:53:39.397807+0000","last_fullsized":"2023-10-26T05:53:39.397807+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:55.723431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387095+0000","last_change":"2023-10-26T05:53:39.387095+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387095+0000","last_undegraded":"2023-10-26T05:53:39.387095+0000","last_fullsized":"2023-10-26T05:53:39.387095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.393569+0000","last_change":"2023-10-26T05:53:39.393375+0000","last_active":"2023-10-26T05:53:39.393569+0000","last_peered":"2023-10-26T05:53:39.393569+0000","last_clean":"2023-10-26T05:53:39.393569+0000","last_became_active":"2023-10-26T05:53:39.392683+0000","last_became_peered":"2023-10-26T05:53:39.392683+0000","last_unstale":"2023-10-26T05:53:39.393569+0000","last_undegraded":"2023-10-26T05:53:39.393569+0000","last_fullsized":"2023-10-26T05:53:39.393569+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389034+0000","last_change":"2023-10-26T05:53:39.389034+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389034+0000","last_undegraded":"2023-10-26T05:53:39.389034+0000","last_fullsized":"2023-10-26T05:53:39.389034+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407208+0000","last_change":"2023-10-26T05:53:39.407074+0000","last_active":"2023-10-26T05:53:39.407208+0000","last_peered":"2023-10-26T05:53:39.407208+0000","last_clean":"2023-10-26T05:53:39.407208+0000","last_became_active":"2023-10-26T05:53:39.403999+0000","last_became_peered":"2023-10-26T05:53:39.403999+0000","last_unstale":"2023-10-26T05:53:39.407208+0000","last_undegraded":"2023-10-26T05:53:39.407208+0000","last_fullsized":"2023-10-26T05:53:39.407208+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:40.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390483+0000","last_change":"2023-10-26T05:53:39.390483+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390483+0000","last_undegraded":"2023-10-26T05:53:39.390483+0000","last_fullsized":"2023-10-26T05:53:39.390483+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.376787+0000","last_change":"2023-10-26T05:53:39.376685+0000","last_active":"2023-10-26T05:53:39.376787+0000","last_peered":"2023-10-26T05:53:39.376787+0000","last_clean":"2023-10-26T05:53:39.376787+0000","last_became_active":"2023-10-26T05:53:39.376380+0000","last_became_peered":"2023-10-26T05:53:39.376380+0000","last_unstale":"2023-10-26T05:53:39.376787+0000","last_undegraded":"2023-10-26T05:53:39.376787+0000","last_fullsized":"2023-10-26T05:53:39.376787+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:47:02.364249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380548+0000","last_change":"2023-10-26T05:53:39.380548+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380548+0000","last_undegraded":"2023-10-26T05:53:39.380548+0000","last_fullsized":"2023-10-26T05:53:39.380548+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379531+0000","last_change":"2023-10-26T05:53:39.379454+0000","last_active":"2023-10-26T05:53:39.379531+0000","last_peered":"2023-10-26T05:53:39.379531+0000","last_clean":"2023-10-26T05:53:39.379531+0000","last_became_active":"2023-10-26T05:53:39.379184+0000","last_became_peered":"2023-10-26T05:53:39.379184+0000","last_unstale":"2023-10-26T05:53:39.379531+0000","last_undegraded":"2023-10-26T05:53:39.379531+0000","last_fullsized":"2023-10-26T05:53:39.379531+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:28.216044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389712+0000","last_change":"2023-10-26T05:53:39.389712+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389712+0000","last_undegraded":"2023-10-26T05:53:39.389712+0000","last_fullsized":"2023-10-26T05:53:39.389712+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379179+0000","last_change":"2023-10-26T05:53:39.379041+0000","last_active":"2023-10-26T05:53:39.379179+0000","last_peered":"2023-10-26T05:53:39.379179+0000","last_clean":"2023-10-26T05:53:39.379179+0000","last_became_active":"2023-10-26T05:53:39.378384+0000","last_became_peered":"2023-10-26T05:53:39.378384+0000","last_unstale":"2023-10-26T05:53:39.379179+0000","last_undegraded":"2023-10-26T05:53:39.379179+0000","last_fullsized":"2023-10-26T05:53:39.379179+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:31.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379897+0000","last_change":"2023-10-26T05:53:39.379897+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379897+0000","last_undegraded":"2023-10-26T05:53:39.379897+0000","last_fullsized":"2023-10-26T05:53:39.379897+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387336+0000","last_change":"2023-10-26T05:53:39.387235+0000","last_active":"2023-10-26T05:53:39.387336+0000","last_peered":"2023-10-26T05:53:39.387336+0000","last_clean":"2023-10-26T05:53:39.387336+0000","last_became_active":"2023-10-26T05:53:39.386945+0000","last_became_peered":"2023-10-26T05:53:39.386945+0000","last_unstale":"2023-10-26T05:53:39.387336+0000","last_undegraded":"2023-10-26T05:53:39.387336+0000","last_fullsized":"2023-10-26T05:53:39.387336+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386653+0000","last_change":"2023-10-26T05:53:39.386653+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386653+0000","last_undegraded":"2023-10-26T05:53:39.386653+0000","last_fullsized":"2023-10-26T05:53:39.386653+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388010+0000","last_change":"2023-10-26T05:53:39.387929+0000","last_active":"2023-10-26T05:53:39.388010+0000","last_peered":"2023-10-26T05:53:39.388010+0000","last_clean":"2023-10-26T05:53:39.388010+0000","last_became_active":"2023-10-26T05:53:39.387571+0000","last_became_peered":"2023-10-26T05:53:39.387571+0000","last_unstale":"2023-10-26T05:53:39.388010+0000","last_undegraded":"2023-10-26T05:53:39.388010+0000","last_fullsized":"2023-10-26T05:53:39.388010+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:20.746112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380202+0000","last_change":"2023-10-26T05:53:39.380202+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380202+0000","last_undegraded":"2023-10-26T05:53:39.380202+0000","last_fullsized":"2023-10-26T05:53:39.380202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.403035+0000","last_change":"2023-10-26T05:53:39.402866+0000","last_active":"2023-10-26T05:53:39.403035+0000","last_peered":"2023-10-26T05:53:39.403035+0000","last_clean":"2023-10-26T05:53:39.403035+0000","last_became_active":"2023-10-26T05:53:39.402076+0000","last_became_peered":"2023-10-26T05:53:39.402076+0000","last_unstale":"2023-10-26T05:53:39.403035+0000","last_undegraded":"2023-10-26T05:53:39.403035+0000","last_fullsized":"2023-10-26T05:53:39.403035+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381954+0000","last_change":"2023-10-26T05:53:39.381954+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381954+0000","last_undegraded":"2023-10-26T05:53:39.381954+0000","last_fullsized":"2023-10-26T05:53:39.381954+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387678+0000","last_change":"2023-10-26T05:53:39.387605+0000","last_active":"2023-10-26T05:53:39.387678+0000","last_peered":"2023-10-26T05:53:39.387678+0000","last_clean":"2023-10-26T05:53:39.387678+0000","last_became_active":"2023-10-26T05:53:39.387254+0000","last_became_peered":"2023-10-26T05:53:39.387254+0000","last_unstale":"2023-10-26T05:53:39.387678+0000","last_undegraded":"2023-10-26T05:53:39.387678+0000","last_fullsized":"2023-10-26T05:53:39.387678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:34.447932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388864+0000","last_change":"2023-10-26T05:53:39.388864+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388864+0000","last_undegraded":"2023-10-26T05:53:39.388864+0000","last_fullsized":"2023-10-26T05:53:39.388864+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388559+0000","last_change":"2023-10-26T05:53:39.388401+0000","last_active":"2023-10-26T05:53:39.388559+0000","last_peered":"2023-10-26T05:53:39.388559+0000","last_clean":"2023-10-26T05:53:39.388559+0000","last_became_active":"2023-10-26T05:53:39.386956+0000","last_became_peered":"2023-10-26T05:53:39.386956+0000","last_unstale":"2023-10-26T05:53:39.388559+0000","last_undegraded":"2023-10-26T05:53:39.388559+0000","last_fullsized":"2023-10-26T05:53:39.388559+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:35.955038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381985+0000","last_change":"2023-10-26T05:53:39.381985+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381985+0000","last_undegraded":"2023-10-26T05:53:39.381985+0000","last_fullsized":"2023-10-26T05:53:39.381985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.406716+0000","last_change":"2023-10-26T05:53:39.406613+0000","last_active":"2023-10-26T05:53:39.406716+0000","last_peered":"2023-10-26T05:53:39.406716+0000","last_clean":"2023-10-26T05:53:39.406716+0000","last_became_active":"2023-10-26T05:53:39.406173+0000","last_became_peered":"2023-10-26T05:53:39.406173+0000","last_unstale":"2023-10-26T05:53:39.406716+0000","last_undegraded":"2023-10-26T05:53:39.406716+0000","last_fullsized":"2023-10-26T05:53:39.406716+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:34.920267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381947+0000","last_change":"2023-10-26T05:53:39.381947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381947+0000","last_undegraded":"2023-10-26T05:53:39.381947+0000","last_fullsized":"2023-10-26T05:53:39.381947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401196+0000","last_change":"2023-10-26T05:53:39.399323+0000","last_active":"2023-10-26T05:53:39.401196+0000","last_peered":"2023-10-26T05:53:39.401196+0000","last_clean":"2023-10-26T05:53:39.401196+0000","last_became_active":"2023-10-26T05:53:39.398652+0000","last_became_peered":"2023-10-26T05:53:39.398652+0000","last_unstale":"2023-10-26T05:53:39.401196+0000","last_undegraded":"2023-10-26T05:53:39.401196+0000","last_fullsized":"2023-10-26T05:53:39.401196+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391217+0000","last_change":"2023-10-26T05:53:39.391217+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391217+0000","last_undegraded":"2023-10-26T05:53:39.391217+0000","last_fullsized":"2023-10-26T05:53:39.391217+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378146+0000","last_change":"2023-10-26T05:53:39.376606+0000","last_active":"2023-10-26T05:53:39.378146+0000","last_peered":"2023-10-26T05:53:39.378146+0000","last_clean":"2023-10-26T05:53:39.378146+0000","last_became_active":"2023-10-26T05:53:39.376273+0000","last_became_peered":"2023-10-26T05:53:39.376273+0000","last_unstale":"2023-10-26T05:53:39.378146+0000","last_undegraded":"2023-10-26T05:53:39.378146+0000","last_fullsized":"2023-10-26T05:53:39.378146+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:01.830273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380160+0000","last_change":"2023-10-26T05:53:39.380160+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380160+0000","last_undegraded":"2023-10-26T05:53:39.380160+0000","last_fullsized":"2023-10-26T05:53:39.380160+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387314+0000","last_change":"2023-10-26T05:53:39.386889+0000","last_active":"2023-10-26T05:53:39.387314+0000","last_peered":"2023-10-26T05:53:39.387314+0000","last_clean":"2023-10-26T05:53:39.387314+0000","last_became_active":"2023-10-26T05:53:39.386241+0000","last_became_peered":"2023-10-26T05:53:39.386241+0000","last_unstale":"2023-10-26T05:53:39.387314+0000","last_undegraded":"2023-10-26T05:53:39.387314+0000","last_fullsized":"2023-10-26T05:53:39.387314+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:14.432402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380090+0000","last_change":"2023-10-26T05:53:39.380090+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380090+0000","last_undegraded":"2023-10-26T05:53:39.380090+0000","last_fullsized":"2023-10-26T05:53:39.380090+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407711+0000","last_change":"2023-10-26T05:53:39.407411+0000","last_active":"2023-10-26T05:53:39.407711+0000","last_peered":"2023-10-26T05:53:39.407711+0000","last_clean":"2023-10-26T05:53:39.407711+0000","last_became_active":"2023-10-26T05:53:39.404908+0000","last_became_peered":"2023-10-26T05:53:39.404908+0000","last_unstale":"2023-10-26T05:53:39.407711+0000","last_undegraded":"2023-10-26T05:53:39.407711+0000","last_fullsized":"2023-10-26T05:53:39.407711+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:04:52.683441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":274,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-26T05:53:38.367192+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:38.367192+0000","last_peered":"2023-10-26T05:53:38.367192+0000","last_clean":"2023-10-26T05:53:38.367192+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:38.367192+0000","last_undegraded":"2023-10-26T05:53:38.367192+0000","last_fullsized":"2023-10-26T05:53:38.367192+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194007,"statfs":{"total":200000000,"available":198663550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81699999999999995,"15min":0.85299999999999998},"min":{"1min":0.501,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0840000000000001,"5min":1.3100000000000001,"15min":1.347},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.44600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1140000000000001,"5min":1.286,"15min":1.3100000000000001},"last":0.872}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.024},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.60299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.24,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.236}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84599999999999997,"15min":0.81999999999999995},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.216,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.167,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.879,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.4330000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.375,"15min":1.375},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.91400000000000003,"15min":0.91900000000000004},"min":{"1min":0.61399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.639,"15min":1.639},"last":1.026},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95899999999999996,"15min":0.94199999999999995},"min":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.145,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.024,"15min":1.1100000000000001},"min":{"1min":0.70099999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2729999999999999,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.999,"5min":1.0169999999999999,"15min":1.079},"min":{"1min":0.76300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.25,"5min":1.702,"15min":1.702},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0820000000000001,"15min":1.1120000000000001},"min":{"1min":0.91400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.391,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.048,"15min":1.048},"min":{"1min":0.65400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.5469999999999999,"15min":1.6180000000000001},"last":1.1279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83999999999999997,"15min":0.81299999999999994},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85899999999999999,"15min":0.85699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.298,"15min":1.3},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1779999999999999,"15min":1.2609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.83999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.167,"5min":1.405,"15min":1.405},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90100000000000002,"15min":0.81499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89500000000000002,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.337,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.151}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.89400000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2629999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.054,"15min":0.97199999999999998},"min":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.502},"max":{"1min":1.2869999999999999,"5min":1.399,"15min":1.399},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":1.429,"15min":1.429},"last":1.014},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.98999999999999999,"15min":0.97599999999999998},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.024,"15min":1.044},"min":{"1min":0.59999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.554},"last":1.286},{"interface":"front","average":{"1min":1.024,"5min":0.95199999999999996,"15min":0.99199999999999999},"min":{"1min":0.53300000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.25}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.097,"15min":1.1160000000000001},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.278,"5min":1.579,"15min":1.579},"last":1.248},{"interface":"front","average":{"1min":0.87,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.272,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.591}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.89100000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.464,"15min":1.464},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.91400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1679999999999999,"5min":1.4219999999999999,"15min":1.5089999999999999},"last":1.006},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.3160000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90200000000000002,"15min":0.91800000000000004},"min":{"1min":0.69399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.448,"5min":1.448,"15min":1.548},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95699999999999996,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.288,"5min":1.4139999999999999,"15min":1.587},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93600000000000005,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.286,"5min":1.286,"15min":2.1909999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.0489999999999999,"15min":1.097},"min":{"1min":0.51700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.4950000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":1.018},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.321,"5min":1.393,"15min":1.5780000000000001},"last":1.744},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0409999999999999,"15min":1.0720000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.093,"15min":1.159},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.6830000000000001},"last":1.8939999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.85399999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1899999999999999,"5min":1.347,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.879,"15min":0.88200000000000001},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2909999999999999,"5min":1.502,"15min":1.502},"last":1.143}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.81699999999999995},"min":{"1min":0.626,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.097,"5min":1.25,"15min":1.375},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94299999999999995,"15min":0.89800000000000002},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3440000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1899999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.81799999999999995},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.55500000000000005,"15min":0.49199999999999999},"max":{"1min":1.2170000000000001,"5min":1.373,"15min":1.373},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":1.341,"15min":2.0190000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.97299999999999998,"15min":0.94299999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2350000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.673,"15min":1.673},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.012,"15min":1},"min":{"1min":0.65200000000000002,"5min":0.503,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.028,"15min":1.036},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.05}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.054,"15min":1.038},"min":{"1min":0.60799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.748,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4590000000000001,"5min":1.728,"15min":1.728},"last":1.0269999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.84299999999999997,"5min":1.385,"15min":1.385},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.2689999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85099999999999998,"5min":1.349,"15min":1.349},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.373}]},{"osd":2,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2170000000000001,"5min":1.367,"15min":1.367},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81000000000000005,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0389999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.441}]},{"osd":4,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.57399999999999995,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91300000000000003,"15min":0.92200000000000004},"min":{"1min":0.60899999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1719999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.748,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.413,"15min":1.478},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94799999999999995,"15min":0.90900000000000003},"min":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1830000000000001,"5min":1.742,"15min":1.742},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.99199999999999999},"min":{"1min":0.73399999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.266,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0249999999999999},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.196,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.92900000000000005},"min":{"1min":0.80000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.216,"5min":1.661,"15min":1.661},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.24,"5min":2.004,"15min":2.004},"last":0.752}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.433},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.60599999999999998},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.379,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96699999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.37}]},{"osd":3,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82099999999999995,"15min":0.73699999999999999},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.28299999999999997},"max":{"1min":1.1180000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.60099999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.022,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93799999999999994,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.49399999999999999},"max":{"1min":1.5189999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.92500000000000004,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4870000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1,"15min":0.91400000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.55200000000000005},"max":{"1min":1.5600000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.028,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.58199999999999996,"15min":0.51700000000000002},"max":{"1min":1.4790000000000001,"5min":1.653,"15min":1.653},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0209999999999999,"15min":0.92600000000000005},"min":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.6000000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.56599999999999995,"15min":0.5},"max":{"1min":1.7769999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97499999999999998,"15min":0.90800000000000003},"min":{"1min":0.746,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.2410000000000001,"15min":1.04},"min":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.51800000000000002},"max":{"1min":1.823,"5min":15.352,"15min":15.352},"last":0.64600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.254},"max":{"1min":0.98799999999999999,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.372},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.623},"min":{"1min":0.34200000000000003,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":1.038,"5min":1.157,"15min":1.157},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95399999999999996,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.72199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1160000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.137,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.27700000000000002},"max":{"1min":1.175,"5min":1.298,"15min":1.298},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88200000000000001,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.113,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2290000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.23,"5min":1.492,"15min":1.492},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.93300000000000005},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.071,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.012,"15min":1.0349999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.258,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.468,"15min":1.837},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0049999999999999,"15min":1.026},"min":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.321,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":1.026},{"interface":"front","average":{"1min":1.038,"5min":1.0840000000000001,"15min":1.0900000000000001},"min":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.319,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":0.87}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194001,"statfs":{"total":200000000,"available":198657275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.62},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.496}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.79100000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.095,"5min":1.1140000000000001,"15min":1.177},"last":0.755}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.26800000000000002},"max":{"1min":1.3520000000000001,"5min":1.409,"15min":1.409},"last":0.625},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.79400000000000004,"15min":0.70599999999999996},"min":{"1min":0.36899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.85599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46100000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.179,"5min":0.98999999999999999,"15min":0.91600000000000004},"min":{"1min":0.70799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.86699999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.234,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.94599999999999995,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94399999999999995,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59799999999999998},"max":{"1min":1.149,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.099},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.1279999999999999,"15min":1.002},"min":{"1min":0.82499999999999996,"5min":0.60499999999999998,"15min":0.54800000000000004},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"max":{"1min":1.214,"5min":1.669,"15min":1.6990000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.03,"15min":0.98899999999999999},"min":{"1min":0.80600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4790000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.98699999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:43.415 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:43.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.567+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/3821319465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14071470 msgr2=0x7fbd140b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.567+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/3821319465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14071470 0x7fbd140b1a70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009930 tx=0x7fbd0c01c890 comp rx=0 tx=0).stop 2023-10-26T05:53:43.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/3821319465 shutdown_connections 2023-10-26T05:53:43.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/3821319465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14071470 0x7fbd140b1a70 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/3821319465 >> 172.21.15.71:0/3821319465 conn(0x7fbd14069770 msgr2=0x7fbd14069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:43.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/3821319465 shutdown_connections 2023-10-26T05:53:43.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/3821319465 wait complete. 2023-10-26T05:53:43.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 Processor -- start 2023-10-26T05:53:43.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- start start 2023-10-26T05:53:43.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 0x7fbd1413c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd14071470 0x7fbd14156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14140cc0 0x7fbd1413e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1415a4c0 con 0x7fbd14070fe0 2023-10-26T05:53:43.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd14158b70 con 0x7fbd14140cc0 2023-10-26T05:53:43.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd141f2770 con 0x7fbd14071470 2023-10-26T05:53:43.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 0x7fbd1413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 0x7fbd1413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34432/0 (socket says 172.21.15.71:34432) 2023-10-26T05:53:43.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 -- 172.21.15.71:0/488331698 learned_addr learned my addr 172.21.15.71:0/488331698 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:43.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1a270640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14140cc0 0x7fbd1413e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 -- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd14071470 msgr2=0x7fbd14156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd14071470 0x7fbd14156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 -- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14140cc0 msgr2=0x7fbd1413e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14140cc0 0x7fbd1413e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 -- 172.21.15.71:0/488331698 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c009560 con 0x7fbd14070fe0 2023-10-26T05:53:43.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1a270640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14140cc0 0x7fbd1413e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:43.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd19a6f640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 0x7fbd1413c490 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c760 tx=0x7fbd0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:43.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04003db0 con 0x7fbd14070fe0 2023-10-26T05:53:43.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbd0400ce70 con 0x7fbd14070fe0 2023-10-26T05:53:43.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd14134520 con 0x7fbd14070fe0 2023-10-26T05:53:43.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd040052b0 con 0x7fbd14070fe0 2023-10-26T05:53:43.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd14134ae0 con 0x7fbd14070fe0 2023-10-26T05:53:43.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.571+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd14066b10 con 0x7fbd14070fe0 2023-10-26T05:53:43.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.575+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbd04005410 con 0x7fbd14070fe0 2023-10-26T05:53:43.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.575+0000 7fbd127fc640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd00066470 0x7fbd00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.575+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd04083f60 con 0x7fbd14070fe0 2023-10-26T05:53:43.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.575+0000 7fbd1926e640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd00066470 0x7fbd00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.575+0000 7fbd1926e640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd00066470 0x7fbd00068930 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbd08000f70 tx=0x7fbd080079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:43.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.579+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbd04051360 con 0x7fbd14070fe0 2023-10-26T05:53:43.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.719+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd140721e0 con 0x7fbd00066470 2023-10-26T05:53:43.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd127fc640 1 -- 172.21.15.71:0/488331698 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278238 (secure 0 0 0) 0x7fbd140721e0 con 0x7fbd00066470 2023-10-26T05:53:43.752 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:43.754 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:43.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd00066470 msgr2=0x7fbd00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd00066470 0x7fbd00068930 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbd08000f70 tx=0x7fbd080079c0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 msgr2=0x7fbd1413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 0x7fbd1413c490 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c760 tx=0x7fbd0400cc20 comp rx=0 tx=0).stop 2023-10-26T05:53:43.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 shutdown_connections 2023-10-26T05:53:43.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd14140cc0 0x7fbd1413e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbd14071470 0x7fbd14156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbd00066470 0x7fbd00068930 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 --2- 172.21.15.71:0/488331698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd14070fe0 0x7fbd1413c490 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 >> 172.21.15.71:0/488331698 conn(0x7fbd14069770 msgr2=0x7fbd140ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:43.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 shutdown_connections 2023-10-26T05:53:43.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.743+0000 7fbd1bcd3640 1 -- 172.21.15.71:0/488331698 wait complete. 2023-10-26T05:53:43.766 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-26T05:53:43.538669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6987,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555512,"statfs":{"total":1600000000,"available":1592844333,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.176615"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382172+0000","last_change":"2023-10-26T05:53:39.382172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382172+0000","last_undegraded":"2023-10-26T05:53:39.382172+0000","last_fullsized":"2023-10-26T05:53:39.382172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.404252+0000","last_change":"2023-10-26T05:53:39.404252+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.404252+0000","last_undegraded":"2023-10-26T05:53:39.404252+0000","last_fullsized":"2023-10-26T05:53:39.404252+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386376+0000","last_change":"2023-10-26T05:53:39.386376+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386376+0000","last_undegraded":"2023-10-26T05:53:39.386376+0000","last_fullsized":"2023-10-26T05:53:39.386376+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383947+0000","last_change":"2023-10-26T05:53:39.383947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383947+0000","last_undegraded":"2023-10-26T05:53:39.383947+0000","last_fullsized":"2023-10-26T05:53:39.383947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375327+0000","last_change":"2023-10-26T05:53:39.375327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375327+0000","last_undegraded":"2023-10-26T05:53:39.375327+0000","last_fullsized":"2023-10-26T05:53:39.375327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385715+0000","last_change":"2023-10-26T05:53:39.385715+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385715+0000","last_undegraded":"2023-10-26T05:53:39.385715+0000","last_fullsized":"2023-10-26T05:53:39.385715+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.396242+0000","last_change":"2023-10-26T05:53:39.396242+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.396242+0000","last_undegraded":"2023-10-26T05:53:39.396242+0000","last_fullsized":"2023-10-26T05:53:39.396242+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384805+0000","last_change":"2023-10-26T05:53:39.384805+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384805+0000","last_undegraded":"2023-10-26T05:53:39.384805+0000","last_fullsized":"2023-10-26T05:53:39.384805+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383663+0000","last_change":"2023-10-26T05:53:39.383663+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383663+0000","last_undegraded":"2023-10-26T05:53:39.383663+0000","last_fullsized":"2023-10-26T05:53:39.383663+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382563+0000","last_change":"2023-10-26T05:53:39.382563+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382563+0000","last_undegraded":"2023-10-26T05:53:39.382563+0000","last_fullsized":"2023-10-26T05:53:39.382563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403072+0000","last_change":"2023-10-26T05:53:39.403072+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403072+0000","last_undegraded":"2023-10-26T05:53:39.403072+0000","last_fullsized":"2023-10-26T05:53:39.403072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380608+0000","last_change":"2023-10-26T05:53:39.380608+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380608+0000","last_undegraded":"2023-10-26T05:53:39.380608+0000","last_fullsized":"2023-10-26T05:53:39.380608+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385665+0000","last_change":"2023-10-26T05:53:39.385665+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385665+0000","last_undegraded":"2023-10-26T05:53:39.385665+0000","last_fullsized":"2023-10-26T05:53:39.385665+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.392675+0000","last_change":"2023-10-26T05:53:39.392675+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.392675+0000","last_undegraded":"2023-10-26T05:53:39.392675+0000","last_fullsized":"2023-10-26T05:53:39.392675+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385863+0000","last_change":"2023-10-26T05:53:39.385863+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385863+0000","last_undegraded":"2023-10-26T05:53:39.385863+0000","last_fullsized":"2023-10-26T05:53:39.385863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381797+0000","last_change":"2023-10-26T05:53:39.381797+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381797+0000","last_undegraded":"2023-10-26T05:53:39.381797+0000","last_fullsized":"2023-10-26T05:53:39.381797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403479+0000","last_change":"2023-10-26T05:53:39.403479+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403479+0000","last_undegraded":"2023-10-26T05:53:39.403479+0000","last_fullsized":"2023-10-26T05:53:39.403479+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388071+0000","last_change":"2023-10-26T05:53:39.388071+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388071+0000","last_undegraded":"2023-10-26T05:53:39.388071+0000","last_fullsized":"2023-10-26T05:53:39.388071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391924+0000","last_change":"2023-10-26T05:53:39.391924+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391924+0000","last_undegraded":"2023-10-26T05:53:39.391924+0000","last_fullsized":"2023-10-26T05:53:39.391924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.395104+0000","last_change":"2023-10-26T05:53:39.395104+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.395104+0000","last_undegraded":"2023-10-26T05:53:39.395104+0000","last_fullsized":"2023-10-26T05:53:39.395104+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383708+0000","last_change":"2023-10-26T05:53:39.383708+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383708+0000","last_undegraded":"2023-10-26T05:53:39.383708+0000","last_fullsized":"2023-10-26T05:53:39.383708+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379010+0000","last_change":"2023-10-26T05:53:39.379010+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379010+0000","last_undegraded":"2023-10-26T05:53:39.379010+0000","last_fullsized":"2023-10-26T05:53:39.379010+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383231+0000","last_change":"2023-10-26T05:53:39.383231+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383231+0000","last_undegraded":"2023-10-26T05:53:39.383231+0000","last_fullsized":"2023-10-26T05:53:39.383231+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378094+0000","last_change":"2023-10-26T05:53:39.378094+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378094+0000","last_undegraded":"2023-10-26T05:53:39.378094+0000","last_fullsized":"2023-10-26T05:53:39.378094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383350+0000","last_change":"2023-10-26T05:53:39.383350+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383350+0000","last_undegraded":"2023-10-26T05:53:39.383350+0000","last_fullsized":"2023-10-26T05:53:39.383350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381436+0000","last_change":"2023-10-26T05:53:39.381436+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381436+0000","last_undegraded":"2023-10-26T05:53:39.381436+0000","last_fullsized":"2023-10-26T05:53:39.381436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385544+0000","last_change":"2023-10-26T05:53:39.385544+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385544+0000","last_undegraded":"2023-10-26T05:53:39.385544+0000","last_fullsized":"2023-10-26T05:53:39.385544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380545+0000","last_change":"2023-10-26T05:53:39.380545+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380545+0000","last_undegraded":"2023-10-26T05:53:39.380545+0000","last_fullsized":"2023-10-26T05:53:39.380545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390037+0000","last_change":"2023-10-26T05:53:39.390037+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390037+0000","last_undegraded":"2023-10-26T05:53:39.390037+0000","last_fullsized":"2023-10-26T05:53:39.390037+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386407+0000","last_change":"2023-10-26T05:53:39.386407+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386407+0000","last_undegraded":"2023-10-26T05:53:39.386407+0000","last_fullsized":"2023-10-26T05:53:39.386407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381024+0000","last_change":"2023-10-26T05:53:39.381024+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381024+0000","last_undegraded":"2023-10-26T05:53:39.381024+0000","last_fullsized":"2023-10-26T05:53:39.381024+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385153+0000","last_change":"2023-10-26T05:53:39.385153+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385153+0000","last_undegraded":"2023-10-26T05:53:39.385153+0000","last_fullsized":"2023-10-26T05:53:39.385153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401005+0000","last_change":"2023-10-26T05:53:39.400795+0000","last_active":"2023-10-26T05:53:39.401005+0000","last_peered":"2023-10-26T05:53:39.401005+0000","last_clean":"2023-10-26T05:53:39.401005+0000","last_became_active":"2023-10-26T05:53:39.400245+0000","last_became_peered":"2023-10-26T05:53:39.400245+0000","last_unstale":"2023-10-26T05:53:39.401005+0000","last_undegraded":"2023-10-26T05:53:39.401005+0000","last_fullsized":"2023-10-26T05:53:39.401005+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:10:39.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387453+0000","last_change":"2023-10-26T05:53:39.387453+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387453+0000","last_undegraded":"2023-10-26T05:53:39.387453+0000","last_fullsized":"2023-10-26T05:53:39.387453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.398646+0000","last_change":"2023-10-26T05:53:39.398370+0000","last_active":"2023-10-26T05:53:39.398646+0000","last_peered":"2023-10-26T05:53:39.398646+0000","last_clean":"2023-10-26T05:53:39.398646+0000","last_became_active":"2023-10-26T05:53:39.397754+0000","last_became_peered":"2023-10-26T05:53:39.397754+0000","last_unstale":"2023-10-26T05:53:39.398646+0000","last_undegraded":"2023-10-26T05:53:39.398646+0000","last_fullsized":"2023-10-26T05:53:39.398646+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385078+0000","last_change":"2023-10-26T05:53:39.385078+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385078+0000","last_undegraded":"2023-10-26T05:53:39.385078+0000","last_fullsized":"2023-10-26T05:53:39.385078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.384194+0000","last_change":"2023-10-26T05:53:39.383805+0000","last_active":"2023-10-26T05:53:39.384194+0000","last_peered":"2023-10-26T05:53:39.384194+0000","last_clean":"2023-10-26T05:53:39.384194+0000","last_became_active":"2023-10-26T05:53:39.383392+0000","last_became_peered":"2023-10-26T05:53:39.383392+0000","last_unstale":"2023-10-26T05:53:39.384194+0000","last_undegraded":"2023-10-26T05:53:39.384194+0000","last_fullsized":"2023-10-26T05:53:39.384194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:23.746787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388327+0000","last_change":"2023-10-26T05:53:39.388327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388327+0000","last_undegraded":"2023-10-26T05:53:39.388327+0000","last_fullsized":"2023-10-26T05:53:39.388327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379011+0000","last_change":"2023-10-26T05:53:39.378932+0000","last_active":"2023-10-26T05:53:39.379011+0000","last_peered":"2023-10-26T05:53:39.379011+0000","last_clean":"2023-10-26T05:53:39.379011+0000","last_became_active":"2023-10-26T05:53:39.378663+0000","last_became_peered":"2023-10-26T05:53:39.378663+0000","last_unstale":"2023-10-26T05:53:39.379011+0000","last_undegraded":"2023-10-26T05:53:39.379011+0000","last_fullsized":"2023-10-26T05:53:39.379011+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:42.264838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380877+0000","last_change":"2023-10-26T05:53:39.380877+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380877+0000","last_undegraded":"2023-10-26T05:53:39.380877+0000","last_fullsized":"2023-10-26T05:53:39.380877+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.408090+0000","last_change":"2023-10-26T05:53:39.407951+0000","last_active":"2023-10-26T05:53:39.408090+0000","last_peered":"2023-10-26T05:53:39.408090+0000","last_clean":"2023-10-26T05:53:39.408090+0000","last_became_active":"2023-10-26T05:53:39.406096+0000","last_became_peered":"2023-10-26T05:53:39.406096+0000","last_unstale":"2023-10-26T05:53:39.408090+0000","last_undegraded":"2023-10-26T05:53:39.408090+0000","last_fullsized":"2023-10-26T05:53:39.408090+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:37.233001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383057+0000","last_change":"2023-10-26T05:53:39.383057+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383057+0000","last_undegraded":"2023-10-26T05:53:39.383057+0000","last_fullsized":"2023-10-26T05:53:39.383057+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389003+0000","last_change":"2023-10-26T05:53:39.388934+0000","last_active":"2023-10-26T05:53:39.389003+0000","last_peered":"2023-10-26T05:53:39.389003+0000","last_clean":"2023-10-26T05:53:39.389003+0000","last_became_active":"2023-10-26T05:53:39.388668+0000","last_became_peered":"2023-10-26T05:53:39.388668+0000","last_unstale":"2023-10-26T05:53:39.389003+0000","last_undegraded":"2023-10-26T05:53:39.389003+0000","last_fullsized":"2023-10-26T05:53:39.389003+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:40.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382253+0000","last_change":"2023-10-26T05:53:39.382253+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382253+0000","last_undegraded":"2023-10-26T05:53:39.382253+0000","last_fullsized":"2023-10-26T05:53:39.382253+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.405096+0000","last_change":"2023-10-26T05:53:39.404546+0000","last_active":"2023-10-26T05:53:39.405096+0000","last_peered":"2023-10-26T05:53:39.405096+0000","last_clean":"2023-10-26T05:53:39.405096+0000","last_became_active":"2023-10-26T05:53:39.403913+0000","last_became_peered":"2023-10-26T05:53:39.403913+0000","last_unstale":"2023-10-26T05:53:39.405096+0000","last_undegraded":"2023-10-26T05:53:39.405096+0000","last_fullsized":"2023-10-26T05:53:39.405096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:46:51.608953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388172+0000","last_change":"2023-10-26T05:53:39.388172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388172+0000","last_undegraded":"2023-10-26T05:53:39.388172+0000","last_fullsized":"2023-10-26T05:53:39.388172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389953+0000","last_change":"2023-10-26T05:53:39.389482+0000","last_active":"2023-10-26T05:53:39.389953+0000","last_peered":"2023-10-26T05:53:39.389953+0000","last_clean":"2023-10-26T05:53:39.389953+0000","last_became_active":"2023-10-26T05:53:39.389225+0000","last_became_peered":"2023-10-26T05:53:39.389225+0000","last_unstale":"2023-10-26T05:53:39.389953+0000","last_undegraded":"2023-10-26T05:53:39.389953+0000","last_fullsized":"2023-10-26T05:53:39.389953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.397757+0000","last_change":"2023-10-26T05:53:39.397757+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.397757+0000","last_undegraded":"2023-10-26T05:53:39.397757+0000","last_fullsized":"2023-10-26T05:53:39.397757+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378170+0000","last_change":"2023-10-26T05:53:39.376735+0000","last_active":"2023-10-26T05:53:39.378170+0000","last_peered":"2023-10-26T05:53:39.378170+0000","last_clean":"2023-10-26T05:53:39.378170+0000","last_became_active":"2023-10-26T05:53:39.376384+0000","last_became_peered":"2023-10-26T05:53:39.376384+0000","last_unstale":"2023-10-26T05:53:39.378170+0000","last_undegraded":"2023-10-26T05:53:39.378170+0000","last_fullsized":"2023-10-26T05:53:39.378170+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:22.802217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388362+0000","last_change":"2023-10-26T05:53:39.388362+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388362+0000","last_undegraded":"2023-10-26T05:53:39.388362+0000","last_fullsized":"2023-10-26T05:53:39.388362+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387881+0000","last_change":"2023-10-26T05:53:39.387783+0000","last_active":"2023-10-26T05:53:39.387881+0000","last_peered":"2023-10-26T05:53:39.387881+0000","last_clean":"2023-10-26T05:53:39.387881+0000","last_became_active":"2023-10-26T05:53:39.387261+0000","last_became_peered":"2023-10-26T05:53:39.387261+0000","last_unstale":"2023-10-26T05:53:39.387881+0000","last_undegraded":"2023-10-26T05:53:39.387881+0000","last_fullsized":"2023-10-26T05:53:39.387881+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:12.049420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.377275+0000","last_change":"2023-10-26T05:53:39.377275+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.377275+0000","last_undegraded":"2023-10-26T05:53:39.377275+0000","last_fullsized":"2023-10-26T05:53:39.377275+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.381673+0000","last_change":"2023-10-26T05:53:39.381571+0000","last_active":"2023-10-26T05:53:39.381673+0000","last_peered":"2023-10-26T05:53:39.381673+0000","last_clean":"2023-10-26T05:53:39.381673+0000","last_became_active":"2023-10-26T05:53:39.381177+0000","last_became_peered":"2023-10-26T05:53:39.381177+0000","last_unstale":"2023-10-26T05:53:39.381673+0000","last_undegraded":"2023-10-26T05:53:39.381673+0000","last_fullsized":"2023-10-26T05:53:39.381673+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385502+0000","last_change":"2023-10-26T05:53:39.385502+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385502+0000","last_undegraded":"2023-10-26T05:53:39.385502+0000","last_fullsized":"2023-10-26T05:53:39.385502+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387655+0000","last_change":"2023-10-26T05:53:39.387472+0000","last_active":"2023-10-26T05:53:39.387655+0000","last_peered":"2023-10-26T05:53:39.387655+0000","last_clean":"2023-10-26T05:53:39.387655+0000","last_became_active":"2023-10-26T05:53:39.386601+0000","last_became_peered":"2023-10-26T05:53:39.386601+0000","last_unstale":"2023-10-26T05:53:39.387655+0000","last_undegraded":"2023-10-26T05:53:39.387655+0000","last_fullsized":"2023-10-26T05:53:39.387655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:55.907120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379151+0000","last_change":"2023-10-26T05:53:39.379151+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379151+0000","last_undegraded":"2023-10-26T05:53:39.379151+0000","last_fullsized":"2023-10-26T05:53:39.379151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392144+0000","last_change":"2023-10-26T05:53:39.392081+0000","last_active":"2023-10-26T05:53:39.392144+0000","last_peered":"2023-10-26T05:53:39.392144+0000","last_clean":"2023-10-26T05:53:39.392144+0000","last_became_active":"2023-10-26T05:53:39.391881+0000","last_became_peered":"2023-10-26T05:53:39.391881+0000","last_unstale":"2023-10-26T05:53:39.392144+0000","last_undegraded":"2023-10-26T05:53:39.392144+0000","last_fullsized":"2023-10-26T05:53:39.392144+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:05:58.324723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382825+0000","last_change":"2023-10-26T05:53:39.382825+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382825+0000","last_undegraded":"2023-10-26T05:53:39.382825+0000","last_fullsized":"2023-10-26T05:53:39.382825+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392055+0000","last_change":"2023-10-26T05:53:39.391669+0000","last_active":"2023-10-26T05:53:39.392055+0000","last_peered":"2023-10-26T05:53:39.392055+0000","last_clean":"2023-10-26T05:53:39.392055+0000","last_became_active":"2023-10-26T05:53:39.390032+0000","last_became_peered":"2023-10-26T05:53:39.390032+0000","last_unstale":"2023-10-26T05:53:39.392055+0000","last_undegraded":"2023-10-26T05:53:39.392055+0000","last_fullsized":"2023-10-26T05:53:39.392055+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378569+0000","last_change":"2023-10-26T05:53:39.378569+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378569+0000","last_undegraded":"2023-10-26T05:53:39.378569+0000","last_fullsized":"2023-10-26T05:53:39.378569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.383455+0000","last_change":"2023-10-26T05:53:39.382861+0000","last_active":"2023-10-26T05:53:39.383455+0000","last_peered":"2023-10-26T05:53:39.383455+0000","last_clean":"2023-10-26T05:53:39.383455+0000","last_became_active":"2023-10-26T05:53:39.382479+0000","last_became_peered":"2023-10-26T05:53:39.382479+0000","last_unstale":"2023-10-26T05:53:39.383455+0000","last_undegraded":"2023-10-26T05:53:39.383455+0000","last_fullsized":"2023-10-26T05:53:39.383455+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:44:49.447349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375035+0000","last_change":"2023-10-26T05:53:39.375035+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375035+0000","last_undegraded":"2023-10-26T05:53:39.375035+0000","last_fullsized":"2023-10-26T05:53:39.375035+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.390084+0000","last_change":"2023-10-26T05:53:39.390022+0000","last_active":"2023-10-26T05:53:39.390084+0000","last_peered":"2023-10-26T05:53:39.390084+0000","last_clean":"2023-10-26T05:53:39.390084+0000","last_became_active":"2023-10-26T05:53:39.389733+0000","last_became_peered":"2023-10-26T05:53:39.389733+0000","last_unstale":"2023-10-26T05:53:39.390084+0000","last_undegraded":"2023-10-26T05:53:39.390084+0000","last_fullsized":"2023-10-26T05:53:39.390084+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:03:07.974034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.397807+0000","last_change":"2023-10-26T05:53:39.397727+0000","last_active":"2023-10-26T05:53:39.397807+0000","last_peered":"2023-10-26T05:53:39.397807+0000","last_clean":"2023-10-26T05:53:39.397807+0000","last_became_active":"2023-10-26T05:53:39.397564+0000","last_became_peered":"2023-10-26T05:53:39.397564+0000","last_unstale":"2023-10-26T05:53:39.397807+0000","last_undegraded":"2023-10-26T05:53:39.397807+0000","last_fullsized":"2023-10-26T05:53:39.397807+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:55.723431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387095+0000","last_change":"2023-10-26T05:53:39.387095+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387095+0000","last_undegraded":"2023-10-26T05:53:39.387095+0000","last_fullsized":"2023-10-26T05:53:39.387095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.393569+0000","last_change":"2023-10-26T05:53:39.393375+0000","last_active":"2023-10-26T05:53:39.393569+0000","last_peered":"2023-10-26T05:53:39.393569+0000","last_clean":"2023-10-26T05:53:39.393569+0000","last_became_active":"2023-10-26T05:53:39.392683+0000","last_became_peered":"2023-10-26T05:53:39.392683+0000","last_unstale":"2023-10-26T05:53:39.393569+0000","last_undegraded":"2023-10-26T05:53:39.393569+0000","last_fullsized":"2023-10-26T05:53:39.393569+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389034+0000","last_change":"2023-10-26T05:53:39.389034+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389034+0000","last_undegraded":"2023-10-26T05:53:39.389034+0000","last_fullsized":"2023-10-26T05:53:39.389034+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407208+0000","last_change":"2023-10-26T05:53:39.407074+0000","last_active":"2023-10-26T05:53:39.407208+0000","last_peered":"2023-10-26T05:53:39.407208+0000","last_clean":"2023-10-26T05:53:39.407208+0000","last_became_active":"2023-10-26T05:53:39.403999+0000","last_became_peered":"2023-10-26T05:53:39.403999+0000","last_unstale":"2023-10-26T05:53:39.407208+0000","last_undegraded":"2023-10-26T05:53:39.407208+0000","last_fullsized":"2023-10-26T05:53:39.407208+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:40.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390483+0000","last_change":"2023-10-26T05:53:39.390483+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390483+0000","last_undegraded":"2023-10-26T05:53:39.390483+0000","last_fullsized":"2023-10-26T05:53:39.390483+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.376787+0000","last_change":"2023-10-26T05:53:39.376685+0000","last_active":"2023-10-26T05:53:39.376787+0000","last_peered":"2023-10-26T05:53:39.376787+0000","last_clean":"2023-10-26T05:53:39.376787+0000","last_became_active":"2023-10-26T05:53:39.376380+0000","last_became_peered":"2023-10-26T05:53:39.376380+0000","last_unstale":"2023-10-26T05:53:39.376787+0000","last_undegraded":"2023-10-26T05:53:39.376787+0000","last_fullsized":"2023-10-26T05:53:39.376787+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:47:02.364249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380548+0000","last_change":"2023-10-26T05:53:39.380548+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380548+0000","last_undegraded":"2023-10-26T05:53:39.380548+0000","last_fullsized":"2023-10-26T05:53:39.380548+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379531+0000","last_change":"2023-10-26T05:53:39.379454+0000","last_active":"2023-10-26T05:53:39.379531+0000","last_peered":"2023-10-26T05:53:39.379531+0000","last_clean":"2023-10-26T05:53:39.379531+0000","last_became_active":"2023-10-26T05:53:39.379184+0000","last_became_peered":"2023-10-26T05:53:39.379184+0000","last_unstale":"2023-10-26T05:53:39.379531+0000","last_undegraded":"2023-10-26T05:53:39.379531+0000","last_fullsized":"2023-10-26T05:53:39.379531+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:28.216044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389712+0000","last_change":"2023-10-26T05:53:39.389712+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389712+0000","last_undegraded":"2023-10-26T05:53:39.389712+0000","last_fullsized":"2023-10-26T05:53:39.389712+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379179+0000","last_change":"2023-10-26T05:53:39.379041+0000","last_active":"2023-10-26T05:53:39.379179+0000","last_peered":"2023-10-26T05:53:39.379179+0000","last_clean":"2023-10-26T05:53:39.379179+0000","last_became_active":"2023-10-26T05:53:39.378384+0000","last_became_peered":"2023-10-26T05:53:39.378384+0000","last_unstale":"2023-10-26T05:53:39.379179+0000","last_undegraded":"2023-10-26T05:53:39.379179+0000","last_fullsized":"2023-10-26T05:53:39.379179+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:31.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379897+0000","last_change":"2023-10-26T05:53:39.379897+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379897+0000","last_undegraded":"2023-10-26T05:53:39.379897+0000","last_fullsized":"2023-10-26T05:53:39.379897+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387336+0000","last_change":"2023-10-26T05:53:39.387235+0000","last_active":"2023-10-26T05:53:39.387336+0000","last_peered":"2023-10-26T05:53:39.387336+0000","last_clean":"2023-10-26T05:53:39.387336+0000","last_became_active":"2023-10-26T05:53:39.386945+0000","last_became_peered":"2023-10-26T05:53:39.386945+0000","last_unstale":"2023-10-26T05:53:39.387336+0000","last_undegraded":"2023-10-26T05:53:39.387336+0000","last_fullsized":"2023-10-26T05:53:39.387336+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386653+0000","last_change":"2023-10-26T05:53:39.386653+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386653+0000","last_undegraded":"2023-10-26T05:53:39.386653+0000","last_fullsized":"2023-10-26T05:53:39.386653+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388010+0000","last_change":"2023-10-26T05:53:39.387929+0000","last_active":"2023-10-26T05:53:39.388010+0000","last_peered":"2023-10-26T05:53:39.388010+0000","last_clean":"2023-10-26T05:53:39.388010+0000","last_became_active":"2023-10-26T05:53:39.387571+0000","last_became_peered":"2023-10-26T05:53:39.387571+0000","last_unstale":"2023-10-26T05:53:39.388010+0000","last_undegraded":"2023-10-26T05:53:39.388010+0000","last_fullsized":"2023-10-26T05:53:39.388010+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:20.746112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380202+0000","last_change":"2023-10-26T05:53:39.380202+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380202+0000","last_undegraded":"2023-10-26T05:53:39.380202+0000","last_fullsized":"2023-10-26T05:53:39.380202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.403035+0000","last_change":"2023-10-26T05:53:39.402866+0000","last_active":"2023-10-26T05:53:39.403035+0000","last_peered":"2023-10-26T05:53:39.403035+0000","last_clean":"2023-10-26T05:53:39.403035+0000","last_became_active":"2023-10-26T05:53:39.402076+0000","last_became_peered":"2023-10-26T05:53:39.402076+0000","last_unstale":"2023-10-26T05:53:39.403035+0000","last_undegraded":"2023-10-26T05:53:39.403035+0000","last_fullsized":"2023-10-26T05:53:39.403035+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381954+0000","last_change":"2023-10-26T05:53:39.381954+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381954+0000","last_undegraded":"2023-10-26T05:53:39.381954+0000","last_fullsized":"2023-10-26T05:53:39.381954+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387678+0000","last_change":"2023-10-26T05:53:39.387605+0000","last_active":"2023-10-26T05:53:39.387678+0000","last_peered":"2023-10-26T05:53:39.387678+0000","last_clean":"2023-10-26T05:53:39.387678+0000","last_became_active":"2023-10-26T05:53:39.387254+0000","last_became_peered":"2023-10-26T05:53:39.387254+0000","last_unstale":"2023-10-26T05:53:39.387678+0000","last_undegraded":"2023-10-26T05:53:39.387678+0000","last_fullsized":"2023-10-26T05:53:39.387678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:34.447932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388864+0000","last_change":"2023-10-26T05:53:39.388864+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388864+0000","last_undegraded":"2023-10-26T05:53:39.388864+0000","last_fullsized":"2023-10-26T05:53:39.388864+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388559+0000","last_change":"2023-10-26T05:53:39.388401+0000","last_active":"2023-10-26T05:53:39.388559+0000","last_peered":"2023-10-26T05:53:39.388559+0000","last_clean":"2023-10-26T05:53:39.388559+0000","last_became_active":"2023-10-26T05:53:39.386956+0000","last_became_peered":"2023-10-26T05:53:39.386956+0000","last_unstale":"2023-10-26T05:53:39.388559+0000","last_undegraded":"2023-10-26T05:53:39.388559+0000","last_fullsized":"2023-10-26T05:53:39.388559+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:35.955038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381985+0000","last_change":"2023-10-26T05:53:39.381985+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381985+0000","last_undegraded":"2023-10-26T05:53:39.381985+0000","last_fullsized":"2023-10-26T05:53:39.381985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.406716+0000","last_change":"2023-10-26T05:53:39.406613+0000","last_active":"2023-10-26T05:53:39.406716+0000","last_peered":"2023-10-26T05:53:39.406716+0000","last_clean":"2023-10-26T05:53:39.406716+0000","last_became_active":"2023-10-26T05:53:39.406173+0000","last_became_peered":"2023-10-26T05:53:39.406173+0000","last_unstale":"2023-10-26T05:53:39.406716+0000","last_undegraded":"2023-10-26T05:53:39.406716+0000","last_fullsized":"2023-10-26T05:53:39.406716+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:34.920267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381947+0000","last_change":"2023-10-26T05:53:39.381947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381947+0000","last_undegraded":"2023-10-26T05:53:39.381947+0000","last_fullsized":"2023-10-26T05:53:39.381947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401196+0000","last_change":"2023-10-26T05:53:39.399323+0000","last_active":"2023-10-26T05:53:39.401196+0000","last_peered":"2023-10-26T05:53:39.401196+0000","last_clean":"2023-10-26T05:53:39.401196+0000","last_became_active":"2023-10-26T05:53:39.398652+0000","last_became_peered":"2023-10-26T05:53:39.398652+0000","last_unstale":"2023-10-26T05:53:39.401196+0000","last_undegraded":"2023-10-26T05:53:39.401196+0000","last_fullsized":"2023-10-26T05:53:39.401196+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391217+0000","last_change":"2023-10-26T05:53:39.391217+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391217+0000","last_undegraded":"2023-10-26T05:53:39.391217+0000","last_fullsized":"2023-10-26T05:53:39.391217+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378146+0000","last_change":"2023-10-26T05:53:39.376606+0000","last_active":"2023-10-26T05:53:39.378146+0000","last_peered":"2023-10-26T05:53:39.378146+0000","last_clean":"2023-10-26T05:53:39.378146+0000","last_became_active":"2023-10-26T05:53:39.376273+0000","last_became_peered":"2023-10-26T05:53:39.376273+0000","last_unstale":"2023-10-26T05:53:39.378146+0000","last_undegraded":"2023-10-26T05:53:39.378146+0000","last_fullsized":"2023-10-26T05:53:39.378146+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:01.830273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380160+0000","last_change":"2023-10-26T05:53:39.380160+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380160+0000","last_undegraded":"2023-10-26T05:53:39.380160+0000","last_fullsized":"2023-10-26T05:53:39.380160+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387314+0000","last_change":"2023-10-26T05:53:39.386889+0000","last_active":"2023-10-26T05:53:39.387314+0000","last_peered":"2023-10-26T05:53:39.387314+0000","last_clean":"2023-10-26T05:53:39.387314+0000","last_became_active":"2023-10-26T05:53:39.386241+0000","last_became_peered":"2023-10-26T05:53:39.386241+0000","last_unstale":"2023-10-26T05:53:39.387314+0000","last_undegraded":"2023-10-26T05:53:39.387314+0000","last_fullsized":"2023-10-26T05:53:39.387314+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:14.432402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380090+0000","last_change":"2023-10-26T05:53:39.380090+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380090+0000","last_undegraded":"2023-10-26T05:53:39.380090+0000","last_fullsized":"2023-10-26T05:53:39.380090+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407711+0000","last_change":"2023-10-26T05:53:39.407411+0000","last_active":"2023-10-26T05:53:39.407711+0000","last_peered":"2023-10-26T05:53:39.407711+0000","last_clean":"2023-10-26T05:53:39.407711+0000","last_became_active":"2023-10-26T05:53:39.404908+0000","last_became_peered":"2023-10-26T05:53:39.404908+0000","last_unstale":"2023-10-26T05:53:39.407711+0000","last_undegraded":"2023-10-26T05:53:39.407711+0000","last_fullsized":"2023-10-26T05:53:39.407711+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:04:52.683441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":274,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-26T05:53:38.367192+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:38.367192+0000","last_peered":"2023-10-26T05:53:38.367192+0000","last_clean":"2023-10-26T05:53:38.367192+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:38.367192+0000","last_undegraded":"2023-10-26T05:53:38.367192+0000","last_fullsized":"2023-10-26T05:53:38.367192+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194007,"statfs":{"total":200000000,"available":198663550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81699999999999995,"15min":0.85299999999999998},"min":{"1min":0.501,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0840000000000001,"5min":1.3100000000000001,"15min":1.347},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.44600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1140000000000001,"5min":1.286,"15min":1.3100000000000001},"last":0.872}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.024},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.60299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.24,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.236}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84599999999999997,"15min":0.81999999999999995},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.216,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.167,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.879,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.4330000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.375,"15min":1.375},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.91400000000000003,"15min":0.91900000000000004},"min":{"1min":0.61399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.639,"15min":1.639},"last":1.026},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95899999999999996,"15min":0.94199999999999995},"min":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.145,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.024,"15min":1.1100000000000001},"min":{"1min":0.70099999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2729999999999999,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.999,"5min":1.0169999999999999,"15min":1.079},"min":{"1min":0.76300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.25,"5min":1.702,"15min":1.702},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0820000000000001,"15min":1.1120000000000001},"min":{"1min":0.91400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.391,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.048,"15min":1.048},"min":{"1min":0.65400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.5469999999999999,"15min":1.6180000000000001},"last":1.1279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83999999999999997,"15min":0.81299999999999994},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85899999999999999,"15min":0.85699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.298,"15min":1.3},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1779999999999999,"15min":1.2609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.83999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.167,"5min":1.405,"15min":1.405},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90100000000000002,"15min":0.81499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89500000000000002,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.337,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.151}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.89400000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2629999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.054,"15min":0.97199999999999998},"min":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.502},"max":{"1min":1.2869999999999999,"5min":1.399,"15min":1.399},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":1.429,"15min":1.429},"last":1.014},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.98999999999999999,"15min":0.97599999999999998},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.024,"15min":1.044},"min":{"1min":0.59999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.554},"last":1.286},{"interface":"front","average":{"1min":1.024,"5min":0.95199999999999996,"15min":0.99199999999999999},"min":{"1min":0.53300000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.25}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.097,"15min":1.1160000000000001},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.278,"5min":1.579,"15min":1.579},"last":1.248},{"interface":"front","average":{"1min":0.87,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.272,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.591}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.89100000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.464,"15min":1.464},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.91400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1679999999999999,"5min":1.4219999999999999,"15min":1.5089999999999999},"last":1.006},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.3160000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90200000000000002,"15min":0.91800000000000004},"min":{"1min":0.69399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.448,"5min":1.448,"15min":1.548},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95699999999999996,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.288,"5min":1.4139999999999999,"15min":1.587},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93600000000000005,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.286,"5min":1.286,"15min":2.1909999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.0489999999999999,"15min":1.097},"min":{"1min":0.51700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.4950000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":1.018},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.321,"5min":1.393,"15min":1.5780000000000001},"last":1.744},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0409999999999999,"15min":1.0720000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.093,"15min":1.159},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.6830000000000001},"last":1.8939999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.85399999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1899999999999999,"5min":1.347,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.879,"15min":0.88200000000000001},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2909999999999999,"5min":1.502,"15min":1.502},"last":1.143}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.81699999999999995},"min":{"1min":0.626,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.097,"5min":1.25,"15min":1.375},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94299999999999995,"15min":0.89800000000000002},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3440000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1899999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.81799999999999995},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.55500000000000005,"15min":0.49199999999999999},"max":{"1min":1.2170000000000001,"5min":1.373,"15min":1.373},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":1.341,"15min":2.0190000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.97299999999999998,"15min":0.94299999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2350000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.673,"15min":1.673},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.012,"15min":1},"min":{"1min":0.65200000000000002,"5min":0.503,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.028,"15min":1.036},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.05}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.054,"15min":1.038},"min":{"1min":0.60799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.748,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4590000000000001,"5min":1.728,"15min":1.728},"last":1.0269999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.84299999999999997,"5min":1.385,"15min":1.385},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.2689999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85099999999999998,"5min":1.349,"15min":1.349},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.373}]},{"osd":2,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2170000000000001,"5min":1.367,"15min":1.367},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81000000000000005,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0389999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.441}]},{"osd":4,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.57399999999999995,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91300000000000003,"15min":0.92200000000000004},"min":{"1min":0.60899999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1719999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.748,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.413,"15min":1.478},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94799999999999995,"15min":0.90900000000000003},"min":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1830000000000001,"5min":1.742,"15min":1.742},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.99199999999999999},"min":{"1min":0.73399999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.266,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0249999999999999},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.196,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.92900000000000005},"min":{"1min":0.80000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.216,"5min":1.661,"15min":1.661},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.24,"5min":2.004,"15min":2.004},"last":0.752}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.433},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.60599999999999998},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.379,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96699999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.37}]},{"osd":3,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82099999999999995,"15min":0.73699999999999999},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.28299999999999997},"max":{"1min":1.1180000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.60099999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.022,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93799999999999994,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.49399999999999999},"max":{"1min":1.5189999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.92500000000000004,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4870000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1,"15min":0.91400000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.55200000000000005},"max":{"1min":1.5600000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.028,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.58199999999999996,"15min":0.51700000000000002},"max":{"1min":1.4790000000000001,"5min":1.653,"15min":1.653},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0209999999999999,"15min":0.92600000000000005},"min":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.6000000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.56599999999999995,"15min":0.5},"max":{"1min":1.7769999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97499999999999998,"15min":0.90800000000000003},"min":{"1min":0.746,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.2410000000000001,"15min":1.04},"min":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.51800000000000002},"max":{"1min":1.823,"5min":15.352,"15min":15.352},"last":0.64600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.254},"max":{"1min":0.98799999999999999,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.372},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.623},"min":{"1min":0.34200000000000003,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":1.038,"5min":1.157,"15min":1.157},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95399999999999996,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.72199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1160000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.137,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.27700000000000002},"max":{"1min":1.175,"5min":1.298,"15min":1.298},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88200000000000001,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.113,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2290000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.23,"5min":1.492,"15min":1.492},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.93300000000000005},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.071,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.012,"15min":1.0349999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.258,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.468,"15min":1.837},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0049999999999999,"15min":1.026},"min":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.321,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":1.026},{"interface":"front","average":{"1min":1.038,"5min":1.0840000000000001,"15min":1.0900000000000001},"min":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.319,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":0.87}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194001,"statfs":{"total":200000000,"available":198657275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.62},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.496}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.79100000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.095,"5min":1.1140000000000001,"15min":1.177},"last":0.755}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.26800000000000002},"max":{"1min":1.3520000000000001,"5min":1.409,"15min":1.409},"last":0.625},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.79400000000000004,"15min":0.70599999999999996},"min":{"1min":0.36899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.85599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46100000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.179,"5min":0.98999999999999999,"15min":0.91600000000000004},"min":{"1min":0.70799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.86699999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.234,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.94599999999999995,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94399999999999995,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59799999999999998},"max":{"1min":1.149,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.099},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.1279999999999999,"15min":1.002},"min":{"1min":0.82499999999999996,"5min":0.60499999999999998,"15min":0.54800000000000004},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"max":{"1min":1.214,"5min":1.669,"15min":1.6990000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.03,"15min":0.98899999999999999},"min":{"1min":0.80600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4790000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.98699999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:43.773 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:43.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 -- 172.21.15.71:0/2230324502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa24013c2f0 msgr2=0x7fa2400b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 --2- 172.21.15.71:0/2230324502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa24013c2f0 0x7fa2400b2350 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009840 tx=0x7fa23c01c800 comp rx=0 tx=0).stop 2023-10-26T05:53:43.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 -- 172.21.15.71:0/2230324502 shutdown_connections 2023-10-26T05:53:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 --2- 172.21.15.71:0/2230324502 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2400b2890 0x7fa2400caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 --2- 172.21.15.71:0/2230324502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa24013c2f0 0x7fa2400b2350 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 -- 172.21.15.71:0/2230324502 >> 172.21.15.71:0/2230324502 conn(0x7fa240069770 msgr2=0x7fa240069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 -- 172.21.15.71:0/2230324502 shutdown_connections 2023-10-26T05:53:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 -- 172.21.15.71:0/2230324502 wait complete. 2023-10-26T05:53:43.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.923+0000 7fa247ff8640 1 Processor -- start 2023-10-26T05:53:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- start start 2023-10-26T05:53:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 0x7fa240151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2400cb010 0x7fa240131040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa24013c2f0 0x7fa240137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401526d0 con 0x7fa2400b2890 2023-10-26T05:53:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401529b0 con 0x7fa2400cb010 2023-10-26T05:53:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa240152c90 con 0x7fa24013c2f0 2023-10-26T05:53:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 0x7fa240151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 0x7fa240151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34458/0 (socket says 172.21.15.71:34458) 2023-10-26T05:53:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 -- 172.21.15.71:0/3053671036 learned_addr learned my addr 172.21.15.71:0/3053671036 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245593640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2400cb010 0x7fa240131040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa246595640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa24013c2f0 0x7fa240137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 -- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa24013c2f0 msgr2=0x7fa240137960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa24013c2f0 0x7fa240137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 -- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2400cb010 msgr2=0x7fa240131040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2400cb010 0x7fa240131040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 -- 172.21.15.71:0/3053671036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0094a0 con 0x7fa2400b2890 2023-10-26T05:53:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245593640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2400cb010 0x7fa240131040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa245d94640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 0x7fa240151da0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c10 tx=0x7fa23c0a8720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0b4040 con 0x7fa2400b2890 2023-10-26T05:53:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa23c004220 con 0x7fa2400b2890 2023-10-26T05:53:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa240152f70 con 0x7fa2400b2890 2023-10-26T05:53:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0051c0 con 0x7fa2400b2890 2023-10-26T05:53:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa240131580 con 0x7fa2400b2890 2023-10-26T05:53:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa240066b10 con 0x7fa2400b2890 2023-10-26T05:53:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa23c005320 con 0x7fa2400b2890 2023-10-26T05:53:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa2367fc640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa224066470 0x7fa224068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.927+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa23c11e8c0 con 0x7fa2400b2890 2023-10-26T05:53:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.931+0000 7fa245593640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa224066470 0x7fa224068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.931+0000 7fa245593640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa224066470 0x7fa224068930 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2400ac6e0 tx=0x7fa230008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:43.931+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa23c0ebeb0 con 0x7fa2400b2890 2023-10-26T05:53:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.083+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2401524b0 con 0x7fa224066470 2023-10-26T05:53:44.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.103+0000 7fa2367fc640 1 -- 172.21.15.71:0/3053671036 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278238 (secure 0 0 0) 0x7fa23000a4a0 con 0x7fa224066470 2023-10-26T05:53:44.114 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:44.116 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:44.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa224066470 msgr2=0x7fa224068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa224066470 0x7fa224068930 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2400ac6e0 tx=0x7fa230008040 comp rx=0 tx=0).stop 2023-10-26T05:53:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 msgr2=0x7fa240151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 0x7fa240151da0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c10 tx=0x7fa23c0a8720 comp rx=0 tx=0).stop 2023-10-26T05:53:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 shutdown_connections 2023-10-26T05:53:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa24013c2f0 0x7fa240137960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2400cb010 0x7fa240131040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa224066470 0x7fa224068930 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 --2- 172.21.15.71:0/3053671036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2400b2890 0x7fa240151da0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 >> 172.21.15.71:0/3053671036 conn(0x7fa240069770 msgr2=0x7fa2400adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:44.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 shutdown_connections 2023-10-26T05:53:44.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.107+0000 7fa247ff8640 1 -- 172.21.15.71:0/3053671036 wait complete. 2023-10-26T05:53:44.132 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-26T05:53:43.538669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6987,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555512,"statfs":{"total":1600000000,"available":1592844333,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.176615"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382172+0000","last_change":"2023-10-26T05:53:39.382172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382172+0000","last_undegraded":"2023-10-26T05:53:39.382172+0000","last_fullsized":"2023-10-26T05:53:39.382172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.404252+0000","last_change":"2023-10-26T05:53:39.404252+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.404252+0000","last_undegraded":"2023-10-26T05:53:39.404252+0000","last_fullsized":"2023-10-26T05:53:39.404252+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386376+0000","last_change":"2023-10-26T05:53:39.386376+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386376+0000","last_undegraded":"2023-10-26T05:53:39.386376+0000","last_fullsized":"2023-10-26T05:53:39.386376+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383947+0000","last_change":"2023-10-26T05:53:39.383947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383947+0000","last_undegraded":"2023-10-26T05:53:39.383947+0000","last_fullsized":"2023-10-26T05:53:39.383947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375327+0000","last_change":"2023-10-26T05:53:39.375327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375327+0000","last_undegraded":"2023-10-26T05:53:39.375327+0000","last_fullsized":"2023-10-26T05:53:39.375327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385715+0000","last_change":"2023-10-26T05:53:39.385715+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385715+0000","last_undegraded":"2023-10-26T05:53:39.385715+0000","last_fullsized":"2023-10-26T05:53:39.385715+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.396242+0000","last_change":"2023-10-26T05:53:39.396242+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.396242+0000","last_undegraded":"2023-10-26T05:53:39.396242+0000","last_fullsized":"2023-10-26T05:53:39.396242+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384805+0000","last_change":"2023-10-26T05:53:39.384805+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384805+0000","last_undegraded":"2023-10-26T05:53:39.384805+0000","last_fullsized":"2023-10-26T05:53:39.384805+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383663+0000","last_change":"2023-10-26T05:53:39.383663+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383663+0000","last_undegraded":"2023-10-26T05:53:39.383663+0000","last_fullsized":"2023-10-26T05:53:39.383663+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382563+0000","last_change":"2023-10-26T05:53:39.382563+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382563+0000","last_undegraded":"2023-10-26T05:53:39.382563+0000","last_fullsized":"2023-10-26T05:53:39.382563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403072+0000","last_change":"2023-10-26T05:53:39.403072+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403072+0000","last_undegraded":"2023-10-26T05:53:39.403072+0000","last_fullsized":"2023-10-26T05:53:39.403072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380608+0000","last_change":"2023-10-26T05:53:39.380608+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380608+0000","last_undegraded":"2023-10-26T05:53:39.380608+0000","last_fullsized":"2023-10-26T05:53:39.380608+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385665+0000","last_change":"2023-10-26T05:53:39.385665+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385665+0000","last_undegraded":"2023-10-26T05:53:39.385665+0000","last_fullsized":"2023-10-26T05:53:39.385665+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.392675+0000","last_change":"2023-10-26T05:53:39.392675+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.392675+0000","last_undegraded":"2023-10-26T05:53:39.392675+0000","last_fullsized":"2023-10-26T05:53:39.392675+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385863+0000","last_change":"2023-10-26T05:53:39.385863+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385863+0000","last_undegraded":"2023-10-26T05:53:39.385863+0000","last_fullsized":"2023-10-26T05:53:39.385863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381797+0000","last_change":"2023-10-26T05:53:39.381797+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381797+0000","last_undegraded":"2023-10-26T05:53:39.381797+0000","last_fullsized":"2023-10-26T05:53:39.381797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403479+0000","last_change":"2023-10-26T05:53:39.403479+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403479+0000","last_undegraded":"2023-10-26T05:53:39.403479+0000","last_fullsized":"2023-10-26T05:53:39.403479+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388071+0000","last_change":"2023-10-26T05:53:39.388071+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388071+0000","last_undegraded":"2023-10-26T05:53:39.388071+0000","last_fullsized":"2023-10-26T05:53:39.388071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391924+0000","last_change":"2023-10-26T05:53:39.391924+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391924+0000","last_undegraded":"2023-10-26T05:53:39.391924+0000","last_fullsized":"2023-10-26T05:53:39.391924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.395104+0000","last_change":"2023-10-26T05:53:39.395104+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.395104+0000","last_undegraded":"2023-10-26T05:53:39.395104+0000","last_fullsized":"2023-10-26T05:53:39.395104+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383708+0000","last_change":"2023-10-26T05:53:39.383708+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383708+0000","last_undegraded":"2023-10-26T05:53:39.383708+0000","last_fullsized":"2023-10-26T05:53:39.383708+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379010+0000","last_change":"2023-10-26T05:53:39.379010+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379010+0000","last_undegraded":"2023-10-26T05:53:39.379010+0000","last_fullsized":"2023-10-26T05:53:39.379010+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383231+0000","last_change":"2023-10-26T05:53:39.383231+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383231+0000","last_undegraded":"2023-10-26T05:53:39.383231+0000","last_fullsized":"2023-10-26T05:53:39.383231+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378094+0000","last_change":"2023-10-26T05:53:39.378094+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378094+0000","last_undegraded":"2023-10-26T05:53:39.378094+0000","last_fullsized":"2023-10-26T05:53:39.378094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383350+0000","last_change":"2023-10-26T05:53:39.383350+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383350+0000","last_undegraded":"2023-10-26T05:53:39.383350+0000","last_fullsized":"2023-10-26T05:53:39.383350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381436+0000","last_change":"2023-10-26T05:53:39.381436+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381436+0000","last_undegraded":"2023-10-26T05:53:39.381436+0000","last_fullsized":"2023-10-26T05:53:39.381436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385544+0000","last_change":"2023-10-26T05:53:39.385544+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385544+0000","last_undegraded":"2023-10-26T05:53:39.385544+0000","last_fullsized":"2023-10-26T05:53:39.385544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380545+0000","last_change":"2023-10-26T05:53:39.380545+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380545+0000","last_undegraded":"2023-10-26T05:53:39.380545+0000","last_fullsized":"2023-10-26T05:53:39.380545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390037+0000","last_change":"2023-10-26T05:53:39.390037+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390037+0000","last_undegraded":"2023-10-26T05:53:39.390037+0000","last_fullsized":"2023-10-26T05:53:39.390037+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386407+0000","last_change":"2023-10-26T05:53:39.386407+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386407+0000","last_undegraded":"2023-10-26T05:53:39.386407+0000","last_fullsized":"2023-10-26T05:53:39.386407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381024+0000","last_change":"2023-10-26T05:53:39.381024+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381024+0000","last_undegraded":"2023-10-26T05:53:39.381024+0000","last_fullsized":"2023-10-26T05:53:39.381024+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385153+0000","last_change":"2023-10-26T05:53:39.385153+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385153+0000","last_undegraded":"2023-10-26T05:53:39.385153+0000","last_fullsized":"2023-10-26T05:53:39.385153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401005+0000","last_change":"2023-10-26T05:53:39.400795+0000","last_active":"2023-10-26T05:53:39.401005+0000","last_peered":"2023-10-26T05:53:39.401005+0000","last_clean":"2023-10-26T05:53:39.401005+0000","last_became_active":"2023-10-26T05:53:39.400245+0000","last_became_peered":"2023-10-26T05:53:39.400245+0000","last_unstale":"2023-10-26T05:53:39.401005+0000","last_undegraded":"2023-10-26T05:53:39.401005+0000","last_fullsized":"2023-10-26T05:53:39.401005+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:10:39.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387453+0000","last_change":"2023-10-26T05:53:39.387453+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387453+0000","last_undegraded":"2023-10-26T05:53:39.387453+0000","last_fullsized":"2023-10-26T05:53:39.387453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.398646+0000","last_change":"2023-10-26T05:53:39.398370+0000","last_active":"2023-10-26T05:53:39.398646+0000","last_peered":"2023-10-26T05:53:39.398646+0000","last_clean":"2023-10-26T05:53:39.398646+0000","last_became_active":"2023-10-26T05:53:39.397754+0000","last_became_peered":"2023-10-26T05:53:39.397754+0000","last_unstale":"2023-10-26T05:53:39.398646+0000","last_undegraded":"2023-10-26T05:53:39.398646+0000","last_fullsized":"2023-10-26T05:53:39.398646+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385078+0000","last_change":"2023-10-26T05:53:39.385078+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385078+0000","last_undegraded":"2023-10-26T05:53:39.385078+0000","last_fullsized":"2023-10-26T05:53:39.385078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.384194+0000","last_change":"2023-10-26T05:53:39.383805+0000","last_active":"2023-10-26T05:53:39.384194+0000","last_peered":"2023-10-26T05:53:39.384194+0000","last_clean":"2023-10-26T05:53:39.384194+0000","last_became_active":"2023-10-26T05:53:39.383392+0000","last_became_peered":"2023-10-26T05:53:39.383392+0000","last_unstale":"2023-10-26T05:53:39.384194+0000","last_undegraded":"2023-10-26T05:53:39.384194+0000","last_fullsized":"2023-10-26T05:53:39.384194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:23.746787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388327+0000","last_change":"2023-10-26T05:53:39.388327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388327+0000","last_undegraded":"2023-10-26T05:53:39.388327+0000","last_fullsized":"2023-10-26T05:53:39.388327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379011+0000","last_change":"2023-10-26T05:53:39.378932+0000","last_active":"2023-10-26T05:53:39.379011+0000","last_peered":"2023-10-26T05:53:39.379011+0000","last_clean":"2023-10-26T05:53:39.379011+0000","last_became_active":"2023-10-26T05:53:39.378663+0000","last_became_peered":"2023-10-26T05:53:39.378663+0000","last_unstale":"2023-10-26T05:53:39.379011+0000","last_undegraded":"2023-10-26T05:53:39.379011+0000","last_fullsized":"2023-10-26T05:53:39.379011+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:42.264838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380877+0000","last_change":"2023-10-26T05:53:39.380877+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380877+0000","last_undegraded":"2023-10-26T05:53:39.380877+0000","last_fullsized":"2023-10-26T05:53:39.380877+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.408090+0000","last_change":"2023-10-26T05:53:39.407951+0000","last_active":"2023-10-26T05:53:39.408090+0000","last_peered":"2023-10-26T05:53:39.408090+0000","last_clean":"2023-10-26T05:53:39.408090+0000","last_became_active":"2023-10-26T05:53:39.406096+0000","last_became_peered":"2023-10-26T05:53:39.406096+0000","last_unstale":"2023-10-26T05:53:39.408090+0000","last_undegraded":"2023-10-26T05:53:39.408090+0000","last_fullsized":"2023-10-26T05:53:39.408090+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:37.233001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383057+0000","last_change":"2023-10-26T05:53:39.383057+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383057+0000","last_undegraded":"2023-10-26T05:53:39.383057+0000","last_fullsized":"2023-10-26T05:53:39.383057+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389003+0000","last_change":"2023-10-26T05:53:39.388934+0000","last_active":"2023-10-26T05:53:39.389003+0000","last_peered":"2023-10-26T05:53:39.389003+0000","last_clean":"2023-10-26T05:53:39.389003+0000","last_became_active":"2023-10-26T05:53:39.388668+0000","last_became_peered":"2023-10-26T05:53:39.388668+0000","last_unstale":"2023-10-26T05:53:39.389003+0000","last_undegraded":"2023-10-26T05:53:39.389003+0000","last_fullsized":"2023-10-26T05:53:39.389003+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:40.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382253+0000","last_change":"2023-10-26T05:53:39.382253+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382253+0000","last_undegraded":"2023-10-26T05:53:39.382253+0000","last_fullsized":"2023-10-26T05:53:39.382253+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.405096+0000","last_change":"2023-10-26T05:53:39.404546+0000","last_active":"2023-10-26T05:53:39.405096+0000","last_peered":"2023-10-26T05:53:39.405096+0000","last_clean":"2023-10-26T05:53:39.405096+0000","last_became_active":"2023-10-26T05:53:39.403913+0000","last_became_peered":"2023-10-26T05:53:39.403913+0000","last_unstale":"2023-10-26T05:53:39.405096+0000","last_undegraded":"2023-10-26T05:53:39.405096+0000","last_fullsized":"2023-10-26T05:53:39.405096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:46:51.608953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388172+0000","last_change":"2023-10-26T05:53:39.388172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388172+0000","last_undegraded":"2023-10-26T05:53:39.388172+0000","last_fullsized":"2023-10-26T05:53:39.388172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389953+0000","last_change":"2023-10-26T05:53:39.389482+0000","last_active":"2023-10-26T05:53:39.389953+0000","last_peered":"2023-10-26T05:53:39.389953+0000","last_clean":"2023-10-26T05:53:39.389953+0000","last_became_active":"2023-10-26T05:53:39.389225+0000","last_became_peered":"2023-10-26T05:53:39.389225+0000","last_unstale":"2023-10-26T05:53:39.389953+0000","last_undegraded":"2023-10-26T05:53:39.389953+0000","last_fullsized":"2023-10-26T05:53:39.389953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.397757+0000","last_change":"2023-10-26T05:53:39.397757+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.397757+0000","last_undegraded":"2023-10-26T05:53:39.397757+0000","last_fullsized":"2023-10-26T05:53:39.397757+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378170+0000","last_change":"2023-10-26T05:53:39.376735+0000","last_active":"2023-10-26T05:53:39.378170+0000","last_peered":"2023-10-26T05:53:39.378170+0000","last_clean":"2023-10-26T05:53:39.378170+0000","last_became_active":"2023-10-26T05:53:39.376384+0000","last_became_peered":"2023-10-26T05:53:39.376384+0000","last_unstale":"2023-10-26T05:53:39.378170+0000","last_undegraded":"2023-10-26T05:53:39.378170+0000","last_fullsized":"2023-10-26T05:53:39.378170+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:22.802217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388362+0000","last_change":"2023-10-26T05:53:39.388362+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388362+0000","last_undegraded":"2023-10-26T05:53:39.388362+0000","last_fullsized":"2023-10-26T05:53:39.388362+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387881+0000","last_change":"2023-10-26T05:53:39.387783+0000","last_active":"2023-10-26T05:53:39.387881+0000","last_peered":"2023-10-26T05:53:39.387881+0000","last_clean":"2023-10-26T05:53:39.387881+0000","last_became_active":"2023-10-26T05:53:39.387261+0000","last_became_peered":"2023-10-26T05:53:39.387261+0000","last_unstale":"2023-10-26T05:53:39.387881+0000","last_undegraded":"2023-10-26T05:53:39.387881+0000","last_fullsized":"2023-10-26T05:53:39.387881+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:12.049420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.377275+0000","last_change":"2023-10-26T05:53:39.377275+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.377275+0000","last_undegraded":"2023-10-26T05:53:39.377275+0000","last_fullsized":"2023-10-26T05:53:39.377275+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.381673+0000","last_change":"2023-10-26T05:53:39.381571+0000","last_active":"2023-10-26T05:53:39.381673+0000","last_peered":"2023-10-26T05:53:39.381673+0000","last_clean":"2023-10-26T05:53:39.381673+0000","last_became_active":"2023-10-26T05:53:39.381177+0000","last_became_peered":"2023-10-26T05:53:39.381177+0000","last_unstale":"2023-10-26T05:53:39.381673+0000","last_undegraded":"2023-10-26T05:53:39.381673+0000","last_fullsized":"2023-10-26T05:53:39.381673+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385502+0000","last_change":"2023-10-26T05:53:39.385502+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385502+0000","last_undegraded":"2023-10-26T05:53:39.385502+0000","last_fullsized":"2023-10-26T05:53:39.385502+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387655+0000","last_change":"2023-10-26T05:53:39.387472+0000","last_active":"2023-10-26T05:53:39.387655+0000","last_peered":"2023-10-26T05:53:39.387655+0000","last_clean":"2023-10-26T05:53:39.387655+0000","last_became_active":"2023-10-26T05:53:39.386601+0000","last_became_peered":"2023-10-26T05:53:39.386601+0000","last_unstale":"2023-10-26T05:53:39.387655+0000","last_undegraded":"2023-10-26T05:53:39.387655+0000","last_fullsized":"2023-10-26T05:53:39.387655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:55.907120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379151+0000","last_change":"2023-10-26T05:53:39.379151+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379151+0000","last_undegraded":"2023-10-26T05:53:39.379151+0000","last_fullsized":"2023-10-26T05:53:39.379151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392144+0000","last_change":"2023-10-26T05:53:39.392081+0000","last_active":"2023-10-26T05:53:39.392144+0000","last_peered":"2023-10-26T05:53:39.392144+0000","last_clean":"2023-10-26T05:53:39.392144+0000","last_became_active":"2023-10-26T05:53:39.391881+0000","last_became_peered":"2023-10-26T05:53:39.391881+0000","last_unstale":"2023-10-26T05:53:39.392144+0000","last_undegraded":"2023-10-26T05:53:39.392144+0000","last_fullsized":"2023-10-26T05:53:39.392144+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:05:58.324723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382825+0000","last_change":"2023-10-26T05:53:39.382825+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382825+0000","last_undegraded":"2023-10-26T05:53:39.382825+0000","last_fullsized":"2023-10-26T05:53:39.382825+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392055+0000","last_change":"2023-10-26T05:53:39.391669+0000","last_active":"2023-10-26T05:53:39.392055+0000","last_peered":"2023-10-26T05:53:39.392055+0000","last_clean":"2023-10-26T05:53:39.392055+0000","last_became_active":"2023-10-26T05:53:39.390032+0000","last_became_peered":"2023-10-26T05:53:39.390032+0000","last_unstale":"2023-10-26T05:53:39.392055+0000","last_undegraded":"2023-10-26T05:53:39.392055+0000","last_fullsized":"2023-10-26T05:53:39.392055+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378569+0000","last_change":"2023-10-26T05:53:39.378569+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378569+0000","last_undegraded":"2023-10-26T05:53:39.378569+0000","last_fullsized":"2023-10-26T05:53:39.378569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.383455+0000","last_change":"2023-10-26T05:53:39.382861+0000","last_active":"2023-10-26T05:53:39.383455+0000","last_peered":"2023-10-26T05:53:39.383455+0000","last_clean":"2023-10-26T05:53:39.383455+0000","last_became_active":"2023-10-26T05:53:39.382479+0000","last_became_peered":"2023-10-26T05:53:39.382479+0000","last_unstale":"2023-10-26T05:53:39.383455+0000","last_undegraded":"2023-10-26T05:53:39.383455+0000","last_fullsized":"2023-10-26T05:53:39.383455+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:44:49.447349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375035+0000","last_change":"2023-10-26T05:53:39.375035+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375035+0000","last_undegraded":"2023-10-26T05:53:39.375035+0000","last_fullsized":"2023-10-26T05:53:39.375035+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.390084+0000","last_change":"2023-10-26T05:53:39.390022+0000","last_active":"2023-10-26T05:53:39.390084+0000","last_peered":"2023-10-26T05:53:39.390084+0000","last_clean":"2023-10-26T05:53:39.390084+0000","last_became_active":"2023-10-26T05:53:39.389733+0000","last_became_peered":"2023-10-26T05:53:39.389733+0000","last_unstale":"2023-10-26T05:53:39.390084+0000","last_undegraded":"2023-10-26T05:53:39.390084+0000","last_fullsized":"2023-10-26T05:53:39.390084+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:03:07.974034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.397807+0000","last_change":"2023-10-26T05:53:39.397727+0000","last_active":"2023-10-26T05:53:39.397807+0000","last_peered":"2023-10-26T05:53:39.397807+0000","last_clean":"2023-10-26T05:53:39.397807+0000","last_became_active":"2023-10-26T05:53:39.397564+0000","last_became_peered":"2023-10-26T05:53:39.397564+0000","last_unstale":"2023-10-26T05:53:39.397807+0000","last_undegraded":"2023-10-26T05:53:39.397807+0000","last_fullsized":"2023-10-26T05:53:39.397807+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:55.723431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387095+0000","last_change":"2023-10-26T05:53:39.387095+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387095+0000","last_undegraded":"2023-10-26T05:53:39.387095+0000","last_fullsized":"2023-10-26T05:53:39.387095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.393569+0000","last_change":"2023-10-26T05:53:39.393375+0000","last_active":"2023-10-26T05:53:39.393569+0000","last_peered":"2023-10-26T05:53:39.393569+0000","last_clean":"2023-10-26T05:53:39.393569+0000","last_became_active":"2023-10-26T05:53:39.392683+0000","last_became_peered":"2023-10-26T05:53:39.392683+0000","last_unstale":"2023-10-26T05:53:39.393569+0000","last_undegraded":"2023-10-26T05:53:39.393569+0000","last_fullsized":"2023-10-26T05:53:39.393569+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389034+0000","last_change":"2023-10-26T05:53:39.389034+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389034+0000","last_undegraded":"2023-10-26T05:53:39.389034+0000","last_fullsized":"2023-10-26T05:53:39.389034+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407208+0000","last_change":"2023-10-26T05:53:39.407074+0000","last_active":"2023-10-26T05:53:39.407208+0000","last_peered":"2023-10-26T05:53:39.407208+0000","last_clean":"2023-10-26T05:53:39.407208+0000","last_became_active":"2023-10-26T05:53:39.403999+0000","last_became_peered":"2023-10-26T05:53:39.403999+0000","last_unstale":"2023-10-26T05:53:39.407208+0000","last_undegraded":"2023-10-26T05:53:39.407208+0000","last_fullsized":"2023-10-26T05:53:39.407208+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:40.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390483+0000","last_change":"2023-10-26T05:53:39.390483+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390483+0000","last_undegraded":"2023-10-26T05:53:39.390483+0000","last_fullsized":"2023-10-26T05:53:39.390483+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.376787+0000","last_change":"2023-10-26T05:53:39.376685+0000","last_active":"2023-10-26T05:53:39.376787+0000","last_peered":"2023-10-26T05:53:39.376787+0000","last_clean":"2023-10-26T05:53:39.376787+0000","last_became_active":"2023-10-26T05:53:39.376380+0000","last_became_peered":"2023-10-26T05:53:39.376380+0000","last_unstale":"2023-10-26T05:53:39.376787+0000","last_undegraded":"2023-10-26T05:53:39.376787+0000","last_fullsized":"2023-10-26T05:53:39.376787+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:47:02.364249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380548+0000","last_change":"2023-10-26T05:53:39.380548+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380548+0000","last_undegraded":"2023-10-26T05:53:39.380548+0000","last_fullsized":"2023-10-26T05:53:39.380548+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379531+0000","last_change":"2023-10-26T05:53:39.379454+0000","last_active":"2023-10-26T05:53:39.379531+0000","last_peered":"2023-10-26T05:53:39.379531+0000","last_clean":"2023-10-26T05:53:39.379531+0000","last_became_active":"2023-10-26T05:53:39.379184+0000","last_became_peered":"2023-10-26T05:53:39.379184+0000","last_unstale":"2023-10-26T05:53:39.379531+0000","last_undegraded":"2023-10-26T05:53:39.379531+0000","last_fullsized":"2023-10-26T05:53:39.379531+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:28.216044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389712+0000","last_change":"2023-10-26T05:53:39.389712+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389712+0000","last_undegraded":"2023-10-26T05:53:39.389712+0000","last_fullsized":"2023-10-26T05:53:39.389712+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379179+0000","last_change":"2023-10-26T05:53:39.379041+0000","last_active":"2023-10-26T05:53:39.379179+0000","last_peered":"2023-10-26T05:53:39.379179+0000","last_clean":"2023-10-26T05:53:39.379179+0000","last_became_active":"2023-10-26T05:53:39.378384+0000","last_became_peered":"2023-10-26T05:53:39.378384+0000","last_unstale":"2023-10-26T05:53:39.379179+0000","last_undegraded":"2023-10-26T05:53:39.379179+0000","last_fullsized":"2023-10-26T05:53:39.379179+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:31.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379897+0000","last_change":"2023-10-26T05:53:39.379897+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379897+0000","last_undegraded":"2023-10-26T05:53:39.379897+0000","last_fullsized":"2023-10-26T05:53:39.379897+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387336+0000","last_change":"2023-10-26T05:53:39.387235+0000","last_active":"2023-10-26T05:53:39.387336+0000","last_peered":"2023-10-26T05:53:39.387336+0000","last_clean":"2023-10-26T05:53:39.387336+0000","last_became_active":"2023-10-26T05:53:39.386945+0000","last_became_peered":"2023-10-26T05:53:39.386945+0000","last_unstale":"2023-10-26T05:53:39.387336+0000","last_undegraded":"2023-10-26T05:53:39.387336+0000","last_fullsized":"2023-10-26T05:53:39.387336+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386653+0000","last_change":"2023-10-26T05:53:39.386653+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386653+0000","last_undegraded":"2023-10-26T05:53:39.386653+0000","last_fullsized":"2023-10-26T05:53:39.386653+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388010+0000","last_change":"2023-10-26T05:53:39.387929+0000","last_active":"2023-10-26T05:53:39.388010+0000","last_peered":"2023-10-26T05:53:39.388010+0000","last_clean":"2023-10-26T05:53:39.388010+0000","last_became_active":"2023-10-26T05:53:39.387571+0000","last_became_peered":"2023-10-26T05:53:39.387571+0000","last_unstale":"2023-10-26T05:53:39.388010+0000","last_undegraded":"2023-10-26T05:53:39.388010+0000","last_fullsized":"2023-10-26T05:53:39.388010+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:20.746112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380202+0000","last_change":"2023-10-26T05:53:39.380202+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380202+0000","last_undegraded":"2023-10-26T05:53:39.380202+0000","last_fullsized":"2023-10-26T05:53:39.380202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.403035+0000","last_change":"2023-10-26T05:53:39.402866+0000","last_active":"2023-10-26T05:53:39.403035+0000","last_peered":"2023-10-26T05:53:39.403035+0000","last_clean":"2023-10-26T05:53:39.403035+0000","last_became_active":"2023-10-26T05:53:39.402076+0000","last_became_peered":"2023-10-26T05:53:39.402076+0000","last_unstale":"2023-10-26T05:53:39.403035+0000","last_undegraded":"2023-10-26T05:53:39.403035+0000","last_fullsized":"2023-10-26T05:53:39.403035+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381954+0000","last_change":"2023-10-26T05:53:39.381954+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381954+0000","last_undegraded":"2023-10-26T05:53:39.381954+0000","last_fullsized":"2023-10-26T05:53:39.381954+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387678+0000","last_change":"2023-10-26T05:53:39.387605+0000","last_active":"2023-10-26T05:53:39.387678+0000","last_peered":"2023-10-26T05:53:39.387678+0000","last_clean":"2023-10-26T05:53:39.387678+0000","last_became_active":"2023-10-26T05:53:39.387254+0000","last_became_peered":"2023-10-26T05:53:39.387254+0000","last_unstale":"2023-10-26T05:53:39.387678+0000","last_undegraded":"2023-10-26T05:53:39.387678+0000","last_fullsized":"2023-10-26T05:53:39.387678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:34.447932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388864+0000","last_change":"2023-10-26T05:53:39.388864+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388864+0000","last_undegraded":"2023-10-26T05:53:39.388864+0000","last_fullsized":"2023-10-26T05:53:39.388864+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388559+0000","last_change":"2023-10-26T05:53:39.388401+0000","last_active":"2023-10-26T05:53:39.388559+0000","last_peered":"2023-10-26T05:53:39.388559+0000","last_clean":"2023-10-26T05:53:39.388559+0000","last_became_active":"2023-10-26T05:53:39.386956+0000","last_became_peered":"2023-10-26T05:53:39.386956+0000","last_unstale":"2023-10-26T05:53:39.388559+0000","last_undegraded":"2023-10-26T05:53:39.388559+0000","last_fullsized":"2023-10-26T05:53:39.388559+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:35.955038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381985+0000","last_change":"2023-10-26T05:53:39.381985+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381985+0000","last_undegraded":"2023-10-26T05:53:39.381985+0000","last_fullsized":"2023-10-26T05:53:39.381985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.406716+0000","last_change":"2023-10-26T05:53:39.406613+0000","last_active":"2023-10-26T05:53:39.406716+0000","last_peered":"2023-10-26T05:53:39.406716+0000","last_clean":"2023-10-26T05:53:39.406716+0000","last_became_active":"2023-10-26T05:53:39.406173+0000","last_became_peered":"2023-10-26T05:53:39.406173+0000","last_unstale":"2023-10-26T05:53:39.406716+0000","last_undegraded":"2023-10-26T05:53:39.406716+0000","last_fullsized":"2023-10-26T05:53:39.406716+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:34.920267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381947+0000","last_change":"2023-10-26T05:53:39.381947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381947+0000","last_undegraded":"2023-10-26T05:53:39.381947+0000","last_fullsized":"2023-10-26T05:53:39.381947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401196+0000","last_change":"2023-10-26T05:53:39.399323+0000","last_active":"2023-10-26T05:53:39.401196+0000","last_peered":"2023-10-26T05:53:39.401196+0000","last_clean":"2023-10-26T05:53:39.401196+0000","last_became_active":"2023-10-26T05:53:39.398652+0000","last_became_peered":"2023-10-26T05:53:39.398652+0000","last_unstale":"2023-10-26T05:53:39.401196+0000","last_undegraded":"2023-10-26T05:53:39.401196+0000","last_fullsized":"2023-10-26T05:53:39.401196+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391217+0000","last_change":"2023-10-26T05:53:39.391217+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391217+0000","last_undegraded":"2023-10-26T05:53:39.391217+0000","last_fullsized":"2023-10-26T05:53:39.391217+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378146+0000","last_change":"2023-10-26T05:53:39.376606+0000","last_active":"2023-10-26T05:53:39.378146+0000","last_peered":"2023-10-26T05:53:39.378146+0000","last_clean":"2023-10-26T05:53:39.378146+0000","last_became_active":"2023-10-26T05:53:39.376273+0000","last_became_peered":"2023-10-26T05:53:39.376273+0000","last_unstale":"2023-10-26T05:53:39.378146+0000","last_undegraded":"2023-10-26T05:53:39.378146+0000","last_fullsized":"2023-10-26T05:53:39.378146+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:01.830273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380160+0000","last_change":"2023-10-26T05:53:39.380160+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380160+0000","last_undegraded":"2023-10-26T05:53:39.380160+0000","last_fullsized":"2023-10-26T05:53:39.380160+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387314+0000","last_change":"2023-10-26T05:53:39.386889+0000","last_active":"2023-10-26T05:53:39.387314+0000","last_peered":"2023-10-26T05:53:39.387314+0000","last_clean":"2023-10-26T05:53:39.387314+0000","last_became_active":"2023-10-26T05:53:39.386241+0000","last_became_peered":"2023-10-26T05:53:39.386241+0000","last_unstale":"2023-10-26T05:53:39.387314+0000","last_undegraded":"2023-10-26T05:53:39.387314+0000","last_fullsized":"2023-10-26T05:53:39.387314+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:14.432402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380090+0000","last_change":"2023-10-26T05:53:39.380090+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380090+0000","last_undegraded":"2023-10-26T05:53:39.380090+0000","last_fullsized":"2023-10-26T05:53:39.380090+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407711+0000","last_change":"2023-10-26T05:53:39.407411+0000","last_active":"2023-10-26T05:53:39.407711+0000","last_peered":"2023-10-26T05:53:39.407711+0000","last_clean":"2023-10-26T05:53:39.407711+0000","last_became_active":"2023-10-26T05:53:39.404908+0000","last_became_peered":"2023-10-26T05:53:39.404908+0000","last_unstale":"2023-10-26T05:53:39.407711+0000","last_undegraded":"2023-10-26T05:53:39.407711+0000","last_fullsized":"2023-10-26T05:53:39.407711+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:04:52.683441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":274,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-26T05:53:38.367192+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:38.367192+0000","last_peered":"2023-10-26T05:53:38.367192+0000","last_clean":"2023-10-26T05:53:38.367192+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:38.367192+0000","last_undegraded":"2023-10-26T05:53:38.367192+0000","last_fullsized":"2023-10-26T05:53:38.367192+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194007,"statfs":{"total":200000000,"available":198663550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81699999999999995,"15min":0.85299999999999998},"min":{"1min":0.501,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0840000000000001,"5min":1.3100000000000001,"15min":1.347},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.44600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1140000000000001,"5min":1.286,"15min":1.3100000000000001},"last":0.872}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.024},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.60299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.24,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.236}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84599999999999997,"15min":0.81999999999999995},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.216,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.167,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.879,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.4330000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.375,"15min":1.375},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.91400000000000003,"15min":0.91900000000000004},"min":{"1min":0.61399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.639,"15min":1.639},"last":1.026},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95899999999999996,"15min":0.94199999999999995},"min":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.145,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.024,"15min":1.1100000000000001},"min":{"1min":0.70099999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2729999999999999,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.999,"5min":1.0169999999999999,"15min":1.079},"min":{"1min":0.76300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.25,"5min":1.702,"15min":1.702},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0820000000000001,"15min":1.1120000000000001},"min":{"1min":0.91400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.391,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.048,"15min":1.048},"min":{"1min":0.65400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.5469999999999999,"15min":1.6180000000000001},"last":1.1279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83999999999999997,"15min":0.81299999999999994},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85899999999999999,"15min":0.85699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.298,"15min":1.3},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1779999999999999,"15min":1.2609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.83999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.167,"5min":1.405,"15min":1.405},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90100000000000002,"15min":0.81499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89500000000000002,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.337,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.151}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.89400000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2629999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.054,"15min":0.97199999999999998},"min":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.502},"max":{"1min":1.2869999999999999,"5min":1.399,"15min":1.399},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":1.429,"15min":1.429},"last":1.014},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.98999999999999999,"15min":0.97599999999999998},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.024,"15min":1.044},"min":{"1min":0.59999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.554},"last":1.286},{"interface":"front","average":{"1min":1.024,"5min":0.95199999999999996,"15min":0.99199999999999999},"min":{"1min":0.53300000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.25}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.097,"15min":1.1160000000000001},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.278,"5min":1.579,"15min":1.579},"last":1.248},{"interface":"front","average":{"1min":0.87,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.272,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.591}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.89100000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.464,"15min":1.464},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.91400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1679999999999999,"5min":1.4219999999999999,"15min":1.5089999999999999},"last":1.006},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.3160000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90200000000000002,"15min":0.91800000000000004},"min":{"1min":0.69399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.448,"5min":1.448,"15min":1.548},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95699999999999996,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.288,"5min":1.4139999999999999,"15min":1.587},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93600000000000005,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.286,"5min":1.286,"15min":2.1909999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.0489999999999999,"15min":1.097},"min":{"1min":0.51700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.4950000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":1.018},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.321,"5min":1.393,"15min":1.5780000000000001},"last":1.744},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0409999999999999,"15min":1.0720000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.093,"15min":1.159},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.6830000000000001},"last":1.8939999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.85399999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1899999999999999,"5min":1.347,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.879,"15min":0.88200000000000001},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2909999999999999,"5min":1.502,"15min":1.502},"last":1.143}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.81699999999999995},"min":{"1min":0.626,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.097,"5min":1.25,"15min":1.375},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94299999999999995,"15min":0.89800000000000002},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3440000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1899999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.81799999999999995},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.55500000000000005,"15min":0.49199999999999999},"max":{"1min":1.2170000000000001,"5min":1.373,"15min":1.373},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":1.341,"15min":2.0190000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.97299999999999998,"15min":0.94299999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2350000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.673,"15min":1.673},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.012,"15min":1},"min":{"1min":0.65200000000000002,"5min":0.503,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.028,"15min":1.036},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.05}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.054,"15min":1.038},"min":{"1min":0.60799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.748,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4590000000000001,"5min":1.728,"15min":1.728},"last":1.0269999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.84299999999999997,"5min":1.385,"15min":1.385},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.2689999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85099999999999998,"5min":1.349,"15min":1.349},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.373}]},{"osd":2,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2170000000000001,"5min":1.367,"15min":1.367},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81000000000000005,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0389999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.441}]},{"osd":4,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.57399999999999995,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91300000000000003,"15min":0.92200000000000004},"min":{"1min":0.60899999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1719999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.748,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.413,"15min":1.478},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94799999999999995,"15min":0.90900000000000003},"min":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1830000000000001,"5min":1.742,"15min":1.742},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.99199999999999999},"min":{"1min":0.73399999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.266,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0249999999999999},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.196,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.92900000000000005},"min":{"1min":0.80000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.216,"5min":1.661,"15min":1.661},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.24,"5min":2.004,"15min":2.004},"last":0.752}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.433},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.60599999999999998},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.379,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96699999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.37}]},{"osd":3,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82099999999999995,"15min":0.73699999999999999},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.28299999999999997},"max":{"1min":1.1180000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.60099999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.022,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93799999999999994,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.49399999999999999},"max":{"1min":1.5189999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.92500000000000004,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4870000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1,"15min":0.91400000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.55200000000000005},"max":{"1min":1.5600000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.028,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.58199999999999996,"15min":0.51700000000000002},"max":{"1min":1.4790000000000001,"5min":1.653,"15min":1.653},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0209999999999999,"15min":0.92600000000000005},"min":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.6000000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.56599999999999995,"15min":0.5},"max":{"1min":1.7769999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97499999999999998,"15min":0.90800000000000003},"min":{"1min":0.746,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.2410000000000001,"15min":1.04},"min":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.51800000000000002},"max":{"1min":1.823,"5min":15.352,"15min":15.352},"last":0.64600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.254},"max":{"1min":0.98799999999999999,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.372},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.623},"min":{"1min":0.34200000000000003,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":1.038,"5min":1.157,"15min":1.157},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95399999999999996,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.72199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1160000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.137,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.27700000000000002},"max":{"1min":1.175,"5min":1.298,"15min":1.298},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88200000000000001,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.113,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2290000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.23,"5min":1.492,"15min":1.492},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.93300000000000005},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.071,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.012,"15min":1.0349999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.258,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.468,"15min":1.837},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0049999999999999,"15min":1.026},"min":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.321,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":1.026},{"interface":"front","average":{"1min":1.038,"5min":1.0840000000000001,"15min":1.0900000000000001},"min":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.319,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":0.87}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194001,"statfs":{"total":200000000,"available":198657275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.62},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.496}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.79100000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.095,"5min":1.1140000000000001,"15min":1.177},"last":0.755}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.26800000000000002},"max":{"1min":1.3520000000000001,"5min":1.409,"15min":1.409},"last":0.625},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.79400000000000004,"15min":0.70599999999999996},"min":{"1min":0.36899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.85599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46100000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.179,"5min":0.98999999999999999,"15min":0.91600000000000004},"min":{"1min":0.70799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.86699999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.234,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.94599999999999995,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94399999999999995,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59799999999999998},"max":{"1min":1.149,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.099},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.1279999999999999,"15min":1.002},"min":{"1min":0.82499999999999996,"5min":0.60499999999999998,"15min":0.54800000000000004},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"max":{"1min":1.214,"5min":1.669,"15min":1.6990000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.03,"15min":0.98899999999999999},"min":{"1min":0.80600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4790000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.98699999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:44.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-10-26T05:53:44.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.382172+0000', 'last_change': '2023-10-26T05:53:39.382172+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.382172+0000', 'last_undegraded': '2023-10-26T05:53:39.382172+0000', 'last_fullsized': '2023-10-26T05:53:39.382172+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2023-10-26T05:53:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.404252+0000', 'last_change': '2023-10-26T05:53:39.404252+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.404252+0000', 'last_undegraded': '2023-10-26T05:53:39.404252+0000', 'last_fullsized': '2023-10-26T05:53:39.404252+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2023-10-26T05:53:44.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.386376+0000', 'last_change': '2023-10-26T05:53:39.386376+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.386376+0000', 'last_undegraded': '2023-10-26T05:53:39.386376+0000', 'last_fullsized': '2023-10-26T05:53:39.386376+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-10-26T05:53:44.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.383947+0000', 'last_change': '2023-10-26T05:53:39.383947+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.383947+0000', 'last_undegraded': '2023-10-26T05:53:39.383947+0000', 'last_fullsized': '2023-10-26T05:53:39.383947+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2023-10-26T05:53:44.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.375327+0000', 'last_change': '2023-10-26T05:53:39.375327+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.375327+0000', 'last_undegraded': '2023-10-26T05:53:39.375327+0000', 'last_fullsized': '2023-10-26T05:53:39.375327+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:53:44.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2023-10-26T05:53:44.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385715+0000', 'last_change': '2023-10-26T05:53:39.385715+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385715+0000', 'last_undegraded': '2023-10-26T05:53:39.385715+0000', 'last_fullsized': '2023-10-26T05:53:39.385715+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-10-26T05:53:44.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.396242+0000', 'last_change': '2023-10-26T05:53:39.396242+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.396242+0000', 'last_undegraded': '2023-10-26T05:53:39.396242+0000', 'last_fullsized': '2023-10-26T05:53:39.396242+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:53:44.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2023-10-26T05:53:44.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.384805+0000', 'last_change': '2023-10-26T05:53:39.384805+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.384805+0000', 'last_undegraded': '2023-10-26T05:53:39.384805+0000', 'last_fullsized': '2023-10-26T05:53:39.384805+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-10-26T05:53:44.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.383663+0000', 'last_change': '2023-10-26T05:53:39.383663+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.383663+0000', 'last_undegraded': '2023-10-26T05:53:39.383663+0000', 'last_fullsized': '2023-10-26T05:53:39.383663+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-10-26T05:53:44.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.382563+0000', 'last_change': '2023-10-26T05:53:39.382563+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.382563+0000', 'last_undegraded': '2023-10-26T05:53:39.382563+0000', 'last_fullsized': '2023-10-26T05:53:39.382563+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2023-10-26T05:53:44.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.403072+0000', 'last_change': '2023-10-26T05:53:39.403072+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.403072+0000', 'last_undegraded': '2023-10-26T05:53:39.403072+0000', 'last_fullsized': '2023-10-26T05:53:39.403072+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-10-26T05:53:44.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380608+0000', 'last_change': '2023-10-26T05:53:39.380608+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380608+0000', 'last_undegraded': '2023-10-26T05:53:39.380608+0000', 'last_fullsized': '2023-10-26T05:53:39.380608+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2023-10-26T05:53:44.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385665+0000', 'last_change': '2023-10-26T05:53:39.385665+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385665+0000', 'last_undegraded': '2023-10-26T05:53:39.385665+0000', 'last_fullsized': '2023-10-26T05:53:39.385665+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2023-10-26T05:53:44.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.392675+0000', 'last_change': '2023-10-26T05:53:39.392675+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.392675+0000', 'last_undegraded': '2023-10-26T05:53:39.392675+0000', 'last_fullsized': '2023-10-26T05:53:39.392675+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2023-10-26T05:53:44.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385863+0000', 'last_change': '2023-10-26T05:53:39.385863+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385863+0000', 'last_undegraded': '2023-10-26T05:53:39.385863+0000', 'last_fullsized': '2023-10-26T05:53:39.385863+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-10-26T05:53:44.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.381797+0000', 'last_change': '2023-10-26T05:53:39.381797+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.381797+0000', 'last_undegraded': '2023-10-26T05:53:39.381797+0000', 'last_fullsized': '2023-10-26T05:53:39.381797+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2023-10-26T05:53:44.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.403479+0000', 'last_change': '2023-10-26T05:53:39.403479+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.403479+0000', 'last_undegraded': '2023-10-26T05:53:39.403479+0000', 'last_fullsized': '2023-10-26T05:53:39.403479+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2023-10-26T05:53:44.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.388071+0000', 'last_change': '2023-10-26T05:53:39.388071+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.388071+0000', 'last_undegraded': '2023-10-26T05:53:39.388071+0000', 'last_fullsized': '2023-10-26T05:53:39.388071+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:53:44.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2023-10-26T05:53:44.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.391924+0000', 'last_change': '2023-10-26T05:53:39.391924+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.391924+0000', 'last_undegraded': '2023-10-26T05:53:39.391924+0000', 'last_fullsized': '2023-10-26T05:53:39.391924+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2023-10-26T05:53:44.159 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.395104+0000', 'last_change': '2023-10-26T05:53:39.395104+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.395104+0000', 'last_undegraded': '2023-10-26T05:53:39.395104+0000', 'last_fullsized': '2023-10-26T05:53:39.395104+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.159 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2023-10-26T05:53:44.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.383708+0000', 'last_change': '2023-10-26T05:53:39.383708+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.383708+0000', 'last_undegraded': '2023-10-26T05:53:39.383708+0000', 'last_fullsized': '2023-10-26T05:53:39.383708+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.160 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2023-10-26T05:53:44.160 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.379010+0000', 'last_change': '2023-10-26T05:53:39.379010+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.379010+0000', 'last_undegraded': '2023-10-26T05:53:39.379010+0000', 'last_fullsized': '2023-10-26T05:53:39.379010+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.161 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2023-10-26T05:53:44.161 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.383231+0000', 'last_change': '2023-10-26T05:53:39.383231+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.383231+0000', 'last_undegraded': '2023-10-26T05:53:39.383231+0000', 'last_fullsized': '2023-10-26T05:53:39.383231+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.162 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2023-10-26T05:53:44.162 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.378094+0000', 'last_change': '2023-10-26T05:53:39.378094+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.378094+0000', 'last_undegraded': '2023-10-26T05:53:39.378094+0000', 'last_fullsized': '2023-10-26T05:53:39.378094+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.163 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2023-10-26T05:53:44.163 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.383350+0000', 'last_change': '2023-10-26T05:53:39.383350+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.383350+0000', 'last_undegraded': '2023-10-26T05:53:39.383350+0000', 'last_fullsized': '2023-10-26T05:53:39.383350+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-10-26T05:53:44.164 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.381436+0000', 'last_change': '2023-10-26T05:53:39.381436+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.381436+0000', 'last_undegraded': '2023-10-26T05:53:39.381436+0000', 'last_fullsized': '2023-10-26T05:53:39.381436+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.164 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2023-10-26T05:53:44.165 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385544+0000', 'last_change': '2023-10-26T05:53:39.385544+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385544+0000', 'last_undegraded': '2023-10-26T05:53:39.385544+0000', 'last_fullsized': '2023-10-26T05:53:39.385544+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.165 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2023-10-26T05:53:44.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380545+0000', 'last_change': '2023-10-26T05:53:39.380545+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380545+0000', 'last_undegraded': '2023-10-26T05:53:39.380545+0000', 'last_fullsized': '2023-10-26T05:53:39.380545+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.166 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2023-10-26T05:53:44.166 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.390037+0000', 'last_change': '2023-10-26T05:53:39.390037+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.390037+0000', 'last_undegraded': '2023-10-26T05:53:39.390037+0000', 'last_fullsized': '2023-10-26T05:53:39.390037+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.167 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-10-26T05:53:44.167 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.386407+0000', 'last_change': '2023-10-26T05:53:39.386407+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.386407+0000', 'last_undegraded': '2023-10-26T05:53:39.386407+0000', 'last_fullsized': '2023-10-26T05:53:39.386407+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:53:44.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-10-26T05:53:44.168 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.381024+0000', 'last_change': '2023-10-26T05:53:39.381024+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.381024+0000', 'last_undegraded': '2023-10-26T05:53:39.381024+0000', 'last_fullsized': '2023-10-26T05:53:39.381024+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.168 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2023-10-26T05:53:44.169 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.384773+0000', 'last_change': '2023-10-26T05:53:39.384773+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.384773+0000', 'last_undegraded': '2023-10-26T05:53:39.384773+0000', 'last_fullsized': '2023-10-26T05:53:39.384773+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.169 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-10-26T05:53:44.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385153+0000', 'last_change': '2023-10-26T05:53:39.385153+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385153+0000', 'last_undegraded': '2023-10-26T05:53:39.385153+0000', 'last_fullsized': '2023-10-26T05:53:39.385153+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.170 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-10-26T05:53:44.170 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.387453+0000', 'last_change': '2023-10-26T05:53:39.387453+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.387453+0000', 'last_undegraded': '2023-10-26T05:53:39.387453+0000', 'last_fullsized': '2023-10-26T05:53:39.387453+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:53:44.171 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-10-26T05:53:44.171 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385078+0000', 'last_change': '2023-10-26T05:53:39.385078+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385078+0000', 'last_undegraded': '2023-10-26T05:53:39.385078+0000', 'last_fullsized': '2023-10-26T05:53:39.385078+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-10-26T05:53:44.172 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.388327+0000', 'last_change': '2023-10-26T05:53:39.388327+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.388327+0000', 'last_undegraded': '2023-10-26T05:53:39.388327+0000', 'last_fullsized': '2023-10-26T05:53:39.388327+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.172 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-10-26T05:53:44.173 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380877+0000', 'last_change': '2023-10-26T05:53:39.380877+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380877+0000', 'last_undegraded': '2023-10-26T05:53:39.380877+0000', 'last_fullsized': '2023-10-26T05:53:39.380877+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.173 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-10-26T05:53:44.174 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.383057+0000', 'last_change': '2023-10-26T05:53:39.383057+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.383057+0000', 'last_undegraded': '2023-10-26T05:53:39.383057+0000', 'last_fullsized': '2023-10-26T05:53:39.383057+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-10-26T05:53:44.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.382253+0000', 'last_change': '2023-10-26T05:53:39.382253+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.382253+0000', 'last_undegraded': '2023-10-26T05:53:39.382253+0000', 'last_fullsized': '2023-10-26T05:53:39.382253+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-10-26T05:53:44.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.388172+0000', 'last_change': '2023-10-26T05:53:39.388172+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.388172+0000', 'last_undegraded': '2023-10-26T05:53:39.388172+0000', 'last_fullsized': '2023-10-26T05:53:39.388172+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:53:44.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-10-26T05:53:44.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.397757+0000', 'last_change': '2023-10-26T05:53:39.397757+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.397757+0000', 'last_undegraded': '2023-10-26T05:53:39.397757+0000', 'last_fullsized': '2023-10-26T05:53:39.397757+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:53:44.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-10-26T05:53:44.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.388362+0000', 'last_change': '2023-10-26T05:53:39.388362+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.388362+0000', 'last_undegraded': '2023-10-26T05:53:39.388362+0000', 'last_fullsized': '2023-10-26T05:53:39.388362+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:53:44.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-10-26T05:53:44.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.377275+0000', 'last_change': '2023-10-26T05:53:39.377275+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.377275+0000', 'last_undegraded': '2023-10-26T05:53:39.377275+0000', 'last_fullsized': '2023-10-26T05:53:39.377275+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-10-26T05:53:44.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.385502+0000', 'last_change': '2023-10-26T05:53:39.385502+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.385502+0000', 'last_undegraded': '2023-10-26T05:53:39.385502+0000', 'last_fullsized': '2023-10-26T05:53:39.385502+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:53:44.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-10-26T05:53:44.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.379151+0000', 'last_change': '2023-10-26T05:53:39.379151+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.379151+0000', 'last_undegraded': '2023-10-26T05:53:39.379151+0000', 'last_fullsized': '2023-10-26T05:53:39.379151+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-10-26T05:53:44.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.382825+0000', 'last_change': '2023-10-26T05:53:39.382825+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.382825+0000', 'last_undegraded': '2023-10-26T05:53:39.382825+0000', 'last_fullsized': '2023-10-26T05:53:39.382825+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-10-26T05:53:44.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.378569+0000', 'last_change': '2023-10-26T05:53:39.378569+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.378569+0000', 'last_undegraded': '2023-10-26T05:53:39.378569+0000', 'last_fullsized': '2023-10-26T05:53:39.378569+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-10-26T05:53:44.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.375035+0000', 'last_change': '2023-10-26T05:53:39.375035+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.375035+0000', 'last_undegraded': '2023-10-26T05:53:39.375035+0000', 'last_fullsized': '2023-10-26T05:53:39.375035+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:53:44.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-10-26T05:53:44.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.384773+0000', 'last_change': '2023-10-26T05:53:39.384773+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.384773+0000', 'last_undegraded': '2023-10-26T05:53:39.384773+0000', 'last_fullsized': '2023-10-26T05:53:39.384773+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:53:44.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-10-26T05:53:44.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.387095+0000', 'last_change': '2023-10-26T05:53:39.387095+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.387095+0000', 'last_undegraded': '2023-10-26T05:53:39.387095+0000', 'last_fullsized': '2023-10-26T05:53:39.387095+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-10-26T05:53:44.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.389034+0000', 'last_change': '2023-10-26T05:53:39.389034+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.389034+0000', 'last_undegraded': '2023-10-26T05:53:39.389034+0000', 'last_fullsized': '2023-10-26T05:53:39.389034+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-10-26T05:53:44.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.390483+0000', 'last_change': '2023-10-26T05:53:39.390483+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.390483+0000', 'last_undegraded': '2023-10-26T05:53:39.390483+0000', 'last_fullsized': '2023-10-26T05:53:39.390483+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-10-26T05:53:44.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380548+0000', 'last_change': '2023-10-26T05:53:39.380548+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380548+0000', 'last_undegraded': '2023-10-26T05:53:39.380548+0000', 'last_fullsized': '2023-10-26T05:53:39.380548+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.187 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-10-26T05:53:44.187 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.389712+0000', 'last_change': '2023-10-26T05:53:39.389712+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.389712+0000', 'last_undegraded': '2023-10-26T05:53:39.389712+0000', 'last_fullsized': '2023-10-26T05:53:39.389712+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.188 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-10-26T05:53:44.188 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.379897+0000', 'last_change': '2023-10-26T05:53:39.379897+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.379897+0000', 'last_undegraded': '2023-10-26T05:53:39.379897+0000', 'last_fullsized': '2023-10-26T05:53:39.379897+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.189 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-10-26T05:53:44.189 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.386653+0000', 'last_change': '2023-10-26T05:53:39.386653+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.386653+0000', 'last_undegraded': '2023-10-26T05:53:39.386653+0000', 'last_fullsized': '2023-10-26T05:53:39.386653+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:53:44.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-10-26T05:53:44.190 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380202+0000', 'last_change': '2023-10-26T05:53:39.380202+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380202+0000', 'last_undegraded': '2023-10-26T05:53:39.380202+0000', 'last_fullsized': '2023-10-26T05:53:39.380202+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.190 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-10-26T05:53:44.191 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.381954+0000', 'last_change': '2023-10-26T05:53:39.381954+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.381954+0000', 'last_undegraded': '2023-10-26T05:53:39.381954+0000', 'last_fullsized': '2023-10-26T05:53:39.381954+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.191 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-10-26T05:53:44.192 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.388864+0000', 'last_change': '2023-10-26T05:53:39.388864+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.388864+0000', 'last_undegraded': '2023-10-26T05:53:39.388864+0000', 'last_fullsized': '2023-10-26T05:53:39.388864+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:53:44.193 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-10-26T05:53:44.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.381985+0000', 'last_change': '2023-10-26T05:53:39.381985+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.381985+0000', 'last_undegraded': '2023-10-26T05:53:39.381985+0000', 'last_fullsized': '2023-10-26T05:53:39.381985+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.194 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-10-26T05:53:44.194 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.381947+0000', 'last_change': '2023-10-26T05:53:39.381947+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.381947+0000', 'last_undegraded': '2023-10-26T05:53:39.381947+0000', 'last_fullsized': '2023-10-26T05:53:39.381947+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:53:44.195 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-10-26T05:53:44.195 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.391217+0000', 'last_change': '2023-10-26T05:53:39.391217+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.391217+0000', 'last_undegraded': '2023-10-26T05:53:39.391217+0000', 'last_fullsized': '2023-10-26T05:53:39.391217+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:53:44.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-10-26T05:53:44.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380160+0000', 'last_change': '2023-10-26T05:53:39.380160+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380160+0000', 'last_undegraded': '2023-10-26T05:53:39.380160+0000', 'last_fullsized': '2023-10-26T05:53:39.380160+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:53:44.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-10-26T05:53:44.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 136, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:53:39.380090+0000', 'last_change': '2023-10-26T05:53:39.380090+0000', 'last_active': '2023-10-26T05:53:39.355381+0000', 'last_peered': '2023-10-26T05:53:39.355381+0000', 'last_clean': '2023-10-26T05:53:39.355381+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:53:39.380090+0000', 'last_undegraded': '2023-10-26T05:53:39.380090+0000', 'last_fullsized': '2023-10-26T05:53:39.380090+0000', 'mapping_epoch': 136, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 136, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'last_clean_scrub_stamp': '2023-10-26T05:53:39.355381+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:53:44.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T05:53:44.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 -- 172.21.15.71:0/3730592633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 msgr2=0x7fa1140b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 --2- 172.21.15.71:0/3730592633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1140b2350 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009900 tx=0x7fa10c01c8c0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 -- 172.21.15.71:0/3730592633 shutdown_connections 2023-10-26T05:53:44.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 --2- 172.21.15.71:0/3730592633 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1140cb1e0 0x7fa114159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 --2- 172.21.15.71:0/3730592633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1140b2350 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 -- 172.21.15.71:0/3730592633 >> 172.21.15.71:0/3730592633 conn(0x7fa114069770 msgr2=0x7fa114069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:44.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 -- 172.21.15.71:0/3730592633 shutdown_connections 2023-10-26T05:53:44.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.315+0000 7fa11a33b640 1 -- 172.21.15.71:0/3730592633 wait complete. 2023-10-26T05:53:44.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 Processor -- start 2023-10-26T05:53:44.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- start start 2023-10-26T05:53:44.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1140b2890 0x7fa114156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1140cb1e0 0x7fa11413e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1141f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11413eb10 con 0x7fa11413c2f0 2023-10-26T05:53:44.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11413edf0 con 0x7fa1140b2890 2023-10-26T05:53:44.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11413f0d0 con 0x7fa1140cb1e0 2023-10-26T05:53:44.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1141f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1141f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34476/0 (socket says 172.21.15.71:34476) 2023-10-26T05:53:44.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 -- 172.21.15.71:0/315991679 learned_addr learned my addr 172.21.15.71:0/315991679 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:44.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1137fe640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1140cb1e0 0x7fa11413e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa113fff640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1140b2890 0x7fa114156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 -- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1140cb1e0 msgr2=0x7fa11413e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1140cb1e0 0x7fa11413e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 -- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1140b2890 msgr2=0x7fa114156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1140b2890 0x7fa114156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 -- 172.21.15.71:0/315991679 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c009560 con 0x7fa11413c2f0 2023-10-26T05:53:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa1188d8640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1141f6060 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa10400ea20 tx=0x7fa10400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10400cde0 con 0x7fa11413c2f0 2023-10-26T05:53:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa104004c40 con 0x7fa11413c2f0 2023-10-26T05:53:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11413f3b0 con 0x7fa11413c2f0 2023-10-26T05:53:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104005020 con 0x7fa11413c2f0 2023-10-26T05:53:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11413e730 con 0x7fa11413c2f0 2023-10-26T05:53:44.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa114066b10 con 0x7fa11413c2f0 2023-10-26T05:53:44.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa104005180 con 0x7fa11413c2f0 2023-10-26T05:53:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.319+0000 7fa110ff9640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa100066470 0x7fa100068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.323+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa104084820 con 0x7fa11413c2f0 2023-10-26T05:53:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.323+0000 7fa113fff640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa100066470 0x7fa100068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.323+0000 7fa113fff640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa100066470 0x7fa100068930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0af000 tx=0x7fa10c01b100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.327+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa104051d30 con 0x7fa11413c2f0 2023-10-26T05:53:44.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.527+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa1140703d0 con 0x7fa11413c2f0 2023-10-26T05:53:44.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa110ff9640 1 -- 172.21.15.71:0/315991679 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1447 (secure 0 0 0) 0x7fa10401f020 con 0x7fa11413c2f0 2023-10-26T05:53:44.538 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:44.538 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":402,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":138,"num_osds":8,"num_up_osds":8,"osd_up_since":1698299229,"num_in_osds":8,"osd_in_since":1698299226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":7155667,"bytes_avail":1592844333,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":51,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8007}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-10-26T05:53:29.534833+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-26T05:53:44.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa100066470 msgr2=0x7fa100068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa100066470 0x7fa100068930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0af000 tx=0x7fa10c01b100 comp rx=0 tx=0).stop 2023-10-26T05:53:44.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 msgr2=0x7fa1141f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1141f6060 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa10400ea20 tx=0x7fa10400eee0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 shutdown_connections 2023-10-26T05:53:44.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa100066470 0x7fa100068930 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa11413c2f0 0x7fa1141f6060 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1140cb1e0 0x7fa11413e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.541 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 --2- 172.21.15.71:0/315991679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1140b2890 0x7fa114156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 >> 172.21.15.71:0/315991679 conn(0x7fa114069770 msgr2=0x7fa1140ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:44.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 shutdown_connections 2023-10-26T05:53:44.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.531+0000 7fa11a33b640 1 -- 172.21.15.71:0/315991679 wait complete. 2023-10-26T05:53:44.555 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T05:53:44.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:44.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3054432939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb56c004d10 msgr2=0x7fb56c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3054432939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb56c004d10 0x7fb56c005190 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb56800bb10 tx=0x7fb56801cd60 comp rx=0 tx=0).stop 2023-10-26T05:53:44.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb586a7d640 1 -- 172.21.15.71:0/3054432939 reap_dead start 2023-10-26T05:53:44.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3054432939 shutdown_connections 2023-10-26T05:53:44.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3054432939 >> 172.21.15.71:0/3054432939 conn(0x7fb580069770 msgr2=0x7fb580069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:44.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3054432939 shutdown_connections 2023-10-26T05:53:44.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3054432939 wait complete. 2023-10-26T05:53:44.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 Processor -- start 2023-10-26T05:53:44.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- start start 2023-10-26T05:53:44.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb56c004d10 0x7fb58015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 0x7fb580132810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015e210 0x7fb58015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801549b0 con 0x7fb58013c4e0 2023-10-26T05:53:44.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58013c940 con 0x7fb58015e210 2023-10-26T05:53:44.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb580132d50 con 0x7fb56c004d10 2023-10-26T05:53:44.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 0x7fb580132810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 0x7fb580132810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34486/0 (socket says 172.21.15.71:34486) 2023-10-26T05:53:44.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 -- 172.21.15.71:0/3665711682 learned_addr learned my addr 172.21.15.71:0/3665711682 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:44.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb58627c640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb56c004d10 0x7fb58015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb586a7d640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015e210 0x7fb58015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 -- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb56c004d10 msgr2=0x7fb58015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb56c004d10 0x7fb58015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 -- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015e210 msgr2=0x7fb58015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015e210 0x7fb58015e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 -- 172.21.15.71:0/3665711682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58015d170 con 0x7fb58013c4e0 2023-10-26T05:53:44.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb585a7b640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 0x7fb580132810 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb5700119c0 tx=0x7fb570011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:44.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000fce0 con 0x7fb58013c4e0 2023-10-26T05:53:44.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb570013450 con 0x7fb58013c4e0 2023-10-26T05:53:44.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58015d450 con 0x7fb58013c4e0 2023-10-26T05:53:44.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570013860 con 0x7fb58013c4e0 2023-10-26T05:53:44.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.675+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58015da10 con 0x7fb58013c4e0 2023-10-26T05:53:44.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.679+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb580066b10 con 0x7fb58013c4e0 2023-10-26T05:53:44.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.679+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb5700139c0 con 0x7fb58013c4e0 2023-10-26T05:53:44.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.679+0000 7fb57effd640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb5600664a0 0x7fb560068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:44.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.679+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5700881d0 con 0x7fb58013c4e0 2023-10-26T05:53:44.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.679+0000 7fb58627c640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb5600664a0 0x7fb560068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:44.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.679+0000 7fb58627c640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb5600664a0 0x7fb560068960 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb578006fd0 tx=0x7fb578006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:44.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.683+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb5700555c0 con 0x7fb58013c4e0 2023-10-26T05:53:44.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.831+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb580164340 con 0x7fb5600664a0 2023-10-26T05:53:44.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.855+0000 7fb57effd640 1 -- 172.21.15.71:0/3665711682 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278238 (secure 0 0 0) 0x7fb578012500 con 0x7fb5600664a0 2023-10-26T05:53:44.864 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:44.866 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:44.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb5600664a0 msgr2=0x7fb560068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb5600664a0 0x7fb560068960 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb578006fd0 tx=0x7fb578006af0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 msgr2=0x7fb580132810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:44.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 0x7fb580132810 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb5700119c0 tx=0x7fb570011e80 comp rx=0 tx=0).stop 2023-10-26T05:53:44.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 shutdown_connections 2023-10-26T05:53:44.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015e210 0x7fb58015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb5600664a0 0x7fb560068960 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb58013c4e0 0x7fb580132810 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 --2- 172.21.15.71:0/3665711682 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb56c004d10 0x7fb58015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:44.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 >> 172.21.15.71:0/3665711682 conn(0x7fb580069770 msgr2=0x7fb58015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:44.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 shutdown_connections 2023-10-26T05:53:44.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:44.859+0000 7fb5884e0640 1 -- 172.21.15.71:0/3665711682 wait complete. 2023-10-26T05:53:44.882 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-26T05:53:43.538669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6987,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555512,"statfs":{"total":1600000000,"available":1592844333,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.176615"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382172+0000","last_change":"2023-10-26T05:53:39.382172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382172+0000","last_undegraded":"2023-10-26T05:53:39.382172+0000","last_fullsized":"2023-10-26T05:53:39.382172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.404252+0000","last_change":"2023-10-26T05:53:39.404252+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.404252+0000","last_undegraded":"2023-10-26T05:53:39.404252+0000","last_fullsized":"2023-10-26T05:53:39.404252+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386376+0000","last_change":"2023-10-26T05:53:39.386376+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386376+0000","last_undegraded":"2023-10-26T05:53:39.386376+0000","last_fullsized":"2023-10-26T05:53:39.386376+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383947+0000","last_change":"2023-10-26T05:53:39.383947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383947+0000","last_undegraded":"2023-10-26T05:53:39.383947+0000","last_fullsized":"2023-10-26T05:53:39.383947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375327+0000","last_change":"2023-10-26T05:53:39.375327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375327+0000","last_undegraded":"2023-10-26T05:53:39.375327+0000","last_fullsized":"2023-10-26T05:53:39.375327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385715+0000","last_change":"2023-10-26T05:53:39.385715+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385715+0000","last_undegraded":"2023-10-26T05:53:39.385715+0000","last_fullsized":"2023-10-26T05:53:39.385715+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.396242+0000","last_change":"2023-10-26T05:53:39.396242+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.396242+0000","last_undegraded":"2023-10-26T05:53:39.396242+0000","last_fullsized":"2023-10-26T05:53:39.396242+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384805+0000","last_change":"2023-10-26T05:53:39.384805+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384805+0000","last_undegraded":"2023-10-26T05:53:39.384805+0000","last_fullsized":"2023-10-26T05:53:39.384805+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383663+0000","last_change":"2023-10-26T05:53:39.383663+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383663+0000","last_undegraded":"2023-10-26T05:53:39.383663+0000","last_fullsized":"2023-10-26T05:53:39.383663+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382563+0000","last_change":"2023-10-26T05:53:39.382563+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382563+0000","last_undegraded":"2023-10-26T05:53:39.382563+0000","last_fullsized":"2023-10-26T05:53:39.382563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403072+0000","last_change":"2023-10-26T05:53:39.403072+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403072+0000","last_undegraded":"2023-10-26T05:53:39.403072+0000","last_fullsized":"2023-10-26T05:53:39.403072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380608+0000","last_change":"2023-10-26T05:53:39.380608+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380608+0000","last_undegraded":"2023-10-26T05:53:39.380608+0000","last_fullsized":"2023-10-26T05:53:39.380608+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385665+0000","last_change":"2023-10-26T05:53:39.385665+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385665+0000","last_undegraded":"2023-10-26T05:53:39.385665+0000","last_fullsized":"2023-10-26T05:53:39.385665+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.392675+0000","last_change":"2023-10-26T05:53:39.392675+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.392675+0000","last_undegraded":"2023-10-26T05:53:39.392675+0000","last_fullsized":"2023-10-26T05:53:39.392675+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385863+0000","last_change":"2023-10-26T05:53:39.385863+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385863+0000","last_undegraded":"2023-10-26T05:53:39.385863+0000","last_fullsized":"2023-10-26T05:53:39.385863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381797+0000","last_change":"2023-10-26T05:53:39.381797+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381797+0000","last_undegraded":"2023-10-26T05:53:39.381797+0000","last_fullsized":"2023-10-26T05:53:39.381797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.403479+0000","last_change":"2023-10-26T05:53:39.403479+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.403479+0000","last_undegraded":"2023-10-26T05:53:39.403479+0000","last_fullsized":"2023-10-26T05:53:39.403479+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388071+0000","last_change":"2023-10-26T05:53:39.388071+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388071+0000","last_undegraded":"2023-10-26T05:53:39.388071+0000","last_fullsized":"2023-10-26T05:53:39.388071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391924+0000","last_change":"2023-10-26T05:53:39.391924+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391924+0000","last_undegraded":"2023-10-26T05:53:39.391924+0000","last_fullsized":"2023-10-26T05:53:39.391924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.395104+0000","last_change":"2023-10-26T05:53:39.395104+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.395104+0000","last_undegraded":"2023-10-26T05:53:39.395104+0000","last_fullsized":"2023-10-26T05:53:39.395104+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383708+0000","last_change":"2023-10-26T05:53:39.383708+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383708+0000","last_undegraded":"2023-10-26T05:53:39.383708+0000","last_fullsized":"2023-10-26T05:53:39.383708+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379010+0000","last_change":"2023-10-26T05:53:39.379010+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379010+0000","last_undegraded":"2023-10-26T05:53:39.379010+0000","last_fullsized":"2023-10-26T05:53:39.379010+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383231+0000","last_change":"2023-10-26T05:53:39.383231+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383231+0000","last_undegraded":"2023-10-26T05:53:39.383231+0000","last_fullsized":"2023-10-26T05:53:39.383231+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378094+0000","last_change":"2023-10-26T05:53:39.378094+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378094+0000","last_undegraded":"2023-10-26T05:53:39.378094+0000","last_fullsized":"2023-10-26T05:53:39.378094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383350+0000","last_change":"2023-10-26T05:53:39.383350+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383350+0000","last_undegraded":"2023-10-26T05:53:39.383350+0000","last_fullsized":"2023-10-26T05:53:39.383350+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381436+0000","last_change":"2023-10-26T05:53:39.381436+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381436+0000","last_undegraded":"2023-10-26T05:53:39.381436+0000","last_fullsized":"2023-10-26T05:53:39.381436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385544+0000","last_change":"2023-10-26T05:53:39.385544+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385544+0000","last_undegraded":"2023-10-26T05:53:39.385544+0000","last_fullsized":"2023-10-26T05:53:39.385544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380545+0000","last_change":"2023-10-26T05:53:39.380545+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380545+0000","last_undegraded":"2023-10-26T05:53:39.380545+0000","last_fullsized":"2023-10-26T05:53:39.380545+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390037+0000","last_change":"2023-10-26T05:53:39.390037+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390037+0000","last_undegraded":"2023-10-26T05:53:39.390037+0000","last_fullsized":"2023-10-26T05:53:39.390037+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386407+0000","last_change":"2023-10-26T05:53:39.386407+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386407+0000","last_undegraded":"2023-10-26T05:53:39.386407+0000","last_fullsized":"2023-10-26T05:53:39.386407+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381024+0000","last_change":"2023-10-26T05:53:39.381024+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381024+0000","last_undegraded":"2023-10-26T05:53:39.381024+0000","last_fullsized":"2023-10-26T05:53:39.381024+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385153+0000","last_change":"2023-10-26T05:53:39.385153+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385153+0000","last_undegraded":"2023-10-26T05:53:39.385153+0000","last_fullsized":"2023-10-26T05:53:39.385153+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401005+0000","last_change":"2023-10-26T05:53:39.400795+0000","last_active":"2023-10-26T05:53:39.401005+0000","last_peered":"2023-10-26T05:53:39.401005+0000","last_clean":"2023-10-26T05:53:39.401005+0000","last_became_active":"2023-10-26T05:53:39.400245+0000","last_became_peered":"2023-10-26T05:53:39.400245+0000","last_unstale":"2023-10-26T05:53:39.401005+0000","last_undegraded":"2023-10-26T05:53:39.401005+0000","last_fullsized":"2023-10-26T05:53:39.401005+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:10:39.648246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387453+0000","last_change":"2023-10-26T05:53:39.387453+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387453+0000","last_undegraded":"2023-10-26T05:53:39.387453+0000","last_fullsized":"2023-10-26T05:53:39.387453+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.398646+0000","last_change":"2023-10-26T05:53:39.398370+0000","last_active":"2023-10-26T05:53:39.398646+0000","last_peered":"2023-10-26T05:53:39.398646+0000","last_clean":"2023-10-26T05:53:39.398646+0000","last_became_active":"2023-10-26T05:53:39.397754+0000","last_became_peered":"2023-10-26T05:53:39.397754+0000","last_unstale":"2023-10-26T05:53:39.398646+0000","last_undegraded":"2023-10-26T05:53:39.398646+0000","last_fullsized":"2023-10-26T05:53:39.398646+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385078+0000","last_change":"2023-10-26T05:53:39.385078+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385078+0000","last_undegraded":"2023-10-26T05:53:39.385078+0000","last_fullsized":"2023-10-26T05:53:39.385078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.384194+0000","last_change":"2023-10-26T05:53:39.383805+0000","last_active":"2023-10-26T05:53:39.384194+0000","last_peered":"2023-10-26T05:53:39.384194+0000","last_clean":"2023-10-26T05:53:39.384194+0000","last_became_active":"2023-10-26T05:53:39.383392+0000","last_became_peered":"2023-10-26T05:53:39.383392+0000","last_unstale":"2023-10-26T05:53:39.384194+0000","last_undegraded":"2023-10-26T05:53:39.384194+0000","last_fullsized":"2023-10-26T05:53:39.384194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:23.746787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388327+0000","last_change":"2023-10-26T05:53:39.388327+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388327+0000","last_undegraded":"2023-10-26T05:53:39.388327+0000","last_fullsized":"2023-10-26T05:53:39.388327+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379011+0000","last_change":"2023-10-26T05:53:39.378932+0000","last_active":"2023-10-26T05:53:39.379011+0000","last_peered":"2023-10-26T05:53:39.379011+0000","last_clean":"2023-10-26T05:53:39.379011+0000","last_became_active":"2023-10-26T05:53:39.378663+0000","last_became_peered":"2023-10-26T05:53:39.378663+0000","last_unstale":"2023-10-26T05:53:39.379011+0000","last_undegraded":"2023-10-26T05:53:39.379011+0000","last_fullsized":"2023-10-26T05:53:39.379011+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:42.264838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380877+0000","last_change":"2023-10-26T05:53:39.380877+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380877+0000","last_undegraded":"2023-10-26T05:53:39.380877+0000","last_fullsized":"2023-10-26T05:53:39.380877+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.408090+0000","last_change":"2023-10-26T05:53:39.407951+0000","last_active":"2023-10-26T05:53:39.408090+0000","last_peered":"2023-10-26T05:53:39.408090+0000","last_clean":"2023-10-26T05:53:39.408090+0000","last_became_active":"2023-10-26T05:53:39.406096+0000","last_became_peered":"2023-10-26T05:53:39.406096+0000","last_unstale":"2023-10-26T05:53:39.408090+0000","last_undegraded":"2023-10-26T05:53:39.408090+0000","last_fullsized":"2023-10-26T05:53:39.408090+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:37.233001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.383057+0000","last_change":"2023-10-26T05:53:39.383057+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.383057+0000","last_undegraded":"2023-10-26T05:53:39.383057+0000","last_fullsized":"2023-10-26T05:53:39.383057+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389003+0000","last_change":"2023-10-26T05:53:39.388934+0000","last_active":"2023-10-26T05:53:39.389003+0000","last_peered":"2023-10-26T05:53:39.389003+0000","last_clean":"2023-10-26T05:53:39.389003+0000","last_became_active":"2023-10-26T05:53:39.388668+0000","last_became_peered":"2023-10-26T05:53:39.388668+0000","last_unstale":"2023-10-26T05:53:39.389003+0000","last_undegraded":"2023-10-26T05:53:39.389003+0000","last_fullsized":"2023-10-26T05:53:39.389003+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:32:40.922879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382253+0000","last_change":"2023-10-26T05:53:39.382253+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382253+0000","last_undegraded":"2023-10-26T05:53:39.382253+0000","last_fullsized":"2023-10-26T05:53:39.382253+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.405096+0000","last_change":"2023-10-26T05:53:39.404546+0000","last_active":"2023-10-26T05:53:39.405096+0000","last_peered":"2023-10-26T05:53:39.405096+0000","last_clean":"2023-10-26T05:53:39.405096+0000","last_became_active":"2023-10-26T05:53:39.403913+0000","last_became_peered":"2023-10-26T05:53:39.403913+0000","last_unstale":"2023-10-26T05:53:39.405096+0000","last_undegraded":"2023-10-26T05:53:39.405096+0000","last_fullsized":"2023-10-26T05:53:39.405096+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:46:51.608953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388172+0000","last_change":"2023-10-26T05:53:39.388172+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388172+0000","last_undegraded":"2023-10-26T05:53:39.388172+0000","last_fullsized":"2023-10-26T05:53:39.388172+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.389953+0000","last_change":"2023-10-26T05:53:39.389482+0000","last_active":"2023-10-26T05:53:39.389953+0000","last_peered":"2023-10-26T05:53:39.389953+0000","last_clean":"2023-10-26T05:53:39.389953+0000","last_became_active":"2023-10-26T05:53:39.389225+0000","last_became_peered":"2023-10-26T05:53:39.389225+0000","last_unstale":"2023-10-26T05:53:39.389953+0000","last_undegraded":"2023-10-26T05:53:39.389953+0000","last_fullsized":"2023-10-26T05:53:39.389953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.397757+0000","last_change":"2023-10-26T05:53:39.397757+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.397757+0000","last_undegraded":"2023-10-26T05:53:39.397757+0000","last_fullsized":"2023-10-26T05:53:39.397757+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378170+0000","last_change":"2023-10-26T05:53:39.376735+0000","last_active":"2023-10-26T05:53:39.378170+0000","last_peered":"2023-10-26T05:53:39.378170+0000","last_clean":"2023-10-26T05:53:39.378170+0000","last_became_active":"2023-10-26T05:53:39.376384+0000","last_became_peered":"2023-10-26T05:53:39.376384+0000","last_unstale":"2023-10-26T05:53:39.378170+0000","last_undegraded":"2023-10-26T05:53:39.378170+0000","last_fullsized":"2023-10-26T05:53:39.378170+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:22.802217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388362+0000","last_change":"2023-10-26T05:53:39.388362+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388362+0000","last_undegraded":"2023-10-26T05:53:39.388362+0000","last_fullsized":"2023-10-26T05:53:39.388362+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387881+0000","last_change":"2023-10-26T05:53:39.387783+0000","last_active":"2023-10-26T05:53:39.387881+0000","last_peered":"2023-10-26T05:53:39.387881+0000","last_clean":"2023-10-26T05:53:39.387881+0000","last_became_active":"2023-10-26T05:53:39.387261+0000","last_became_peered":"2023-10-26T05:53:39.387261+0000","last_unstale":"2023-10-26T05:53:39.387881+0000","last_undegraded":"2023-10-26T05:53:39.387881+0000","last_fullsized":"2023-10-26T05:53:39.387881+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:12.049420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.377275+0000","last_change":"2023-10-26T05:53:39.377275+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.377275+0000","last_undegraded":"2023-10-26T05:53:39.377275+0000","last_fullsized":"2023-10-26T05:53:39.377275+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.381673+0000","last_change":"2023-10-26T05:53:39.381571+0000","last_active":"2023-10-26T05:53:39.381673+0000","last_peered":"2023-10-26T05:53:39.381673+0000","last_clean":"2023-10-26T05:53:39.381673+0000","last_became_active":"2023-10-26T05:53:39.381177+0000","last_became_peered":"2023-10-26T05:53:39.381177+0000","last_unstale":"2023-10-26T05:53:39.381673+0000","last_undegraded":"2023-10-26T05:53:39.381673+0000","last_fullsized":"2023-10-26T05:53:39.381673+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.385502+0000","last_change":"2023-10-26T05:53:39.385502+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.385502+0000","last_undegraded":"2023-10-26T05:53:39.385502+0000","last_fullsized":"2023-10-26T05:53:39.385502+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387655+0000","last_change":"2023-10-26T05:53:39.387472+0000","last_active":"2023-10-26T05:53:39.387655+0000","last_peered":"2023-10-26T05:53:39.387655+0000","last_clean":"2023-10-26T05:53:39.387655+0000","last_became_active":"2023-10-26T05:53:39.386601+0000","last_became_peered":"2023-10-26T05:53:39.386601+0000","last_unstale":"2023-10-26T05:53:39.387655+0000","last_undegraded":"2023-10-26T05:53:39.387655+0000","last_fullsized":"2023-10-26T05:53:39.387655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:55.907120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379151+0000","last_change":"2023-10-26T05:53:39.379151+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379151+0000","last_undegraded":"2023-10-26T05:53:39.379151+0000","last_fullsized":"2023-10-26T05:53:39.379151+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392144+0000","last_change":"2023-10-26T05:53:39.392081+0000","last_active":"2023-10-26T05:53:39.392144+0000","last_peered":"2023-10-26T05:53:39.392144+0000","last_clean":"2023-10-26T05:53:39.392144+0000","last_became_active":"2023-10-26T05:53:39.391881+0000","last_became_peered":"2023-10-26T05:53:39.391881+0000","last_unstale":"2023-10-26T05:53:39.392144+0000","last_undegraded":"2023-10-26T05:53:39.392144+0000","last_fullsized":"2023-10-26T05:53:39.392144+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:05:58.324723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.382825+0000","last_change":"2023-10-26T05:53:39.382825+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.382825+0000","last_undegraded":"2023-10-26T05:53:39.382825+0000","last_fullsized":"2023-10-26T05:53:39.382825+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.392055+0000","last_change":"2023-10-26T05:53:39.391669+0000","last_active":"2023-10-26T05:53:39.392055+0000","last_peered":"2023-10-26T05:53:39.392055+0000","last_clean":"2023-10-26T05:53:39.392055+0000","last_became_active":"2023-10-26T05:53:39.390032+0000","last_became_peered":"2023-10-26T05:53:39.390032+0000","last_unstale":"2023-10-26T05:53:39.392055+0000","last_undegraded":"2023-10-26T05:53:39.392055+0000","last_fullsized":"2023-10-26T05:53:39.392055+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:29.025894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.378569+0000","last_change":"2023-10-26T05:53:39.378569+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.378569+0000","last_undegraded":"2023-10-26T05:53:39.378569+0000","last_fullsized":"2023-10-26T05:53:39.378569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.383455+0000","last_change":"2023-10-26T05:53:39.382861+0000","last_active":"2023-10-26T05:53:39.383455+0000","last_peered":"2023-10-26T05:53:39.383455+0000","last_clean":"2023-10-26T05:53:39.383455+0000","last_became_active":"2023-10-26T05:53:39.382479+0000","last_became_peered":"2023-10-26T05:53:39.382479+0000","last_unstale":"2023-10-26T05:53:39.383455+0000","last_undegraded":"2023-10-26T05:53:39.383455+0000","last_fullsized":"2023-10-26T05:53:39.383455+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:44:49.447349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.375035+0000","last_change":"2023-10-26T05:53:39.375035+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.375035+0000","last_undegraded":"2023-10-26T05:53:39.375035+0000","last_fullsized":"2023-10-26T05:53:39.375035+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.390084+0000","last_change":"2023-10-26T05:53:39.390022+0000","last_active":"2023-10-26T05:53:39.390084+0000","last_peered":"2023-10-26T05:53:39.390084+0000","last_clean":"2023-10-26T05:53:39.390084+0000","last_became_active":"2023-10-26T05:53:39.389733+0000","last_became_peered":"2023-10-26T05:53:39.389733+0000","last_unstale":"2023-10-26T05:53:39.390084+0000","last_undegraded":"2023-10-26T05:53:39.390084+0000","last_fullsized":"2023-10-26T05:53:39.390084+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:03:07.974034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.384773+0000","last_change":"2023-10-26T05:53:39.384773+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.384773+0000","last_undegraded":"2023-10-26T05:53:39.384773+0000","last_fullsized":"2023-10-26T05:53:39.384773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.397807+0000","last_change":"2023-10-26T05:53:39.397727+0000","last_active":"2023-10-26T05:53:39.397807+0000","last_peered":"2023-10-26T05:53:39.397807+0000","last_clean":"2023-10-26T05:53:39.397807+0000","last_became_active":"2023-10-26T05:53:39.397564+0000","last_became_peered":"2023-10-26T05:53:39.397564+0000","last_unstale":"2023-10-26T05:53:39.397807+0000","last_undegraded":"2023-10-26T05:53:39.397807+0000","last_fullsized":"2023-10-26T05:53:39.397807+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:40:55.723431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.387095+0000","last_change":"2023-10-26T05:53:39.387095+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.387095+0000","last_undegraded":"2023-10-26T05:53:39.387095+0000","last_fullsized":"2023-10-26T05:53:39.387095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.393569+0000","last_change":"2023-10-26T05:53:39.393375+0000","last_active":"2023-10-26T05:53:39.393569+0000","last_peered":"2023-10-26T05:53:39.393569+0000","last_clean":"2023-10-26T05:53:39.393569+0000","last_became_active":"2023-10-26T05:53:39.392683+0000","last_became_peered":"2023-10-26T05:53:39.392683+0000","last_unstale":"2023-10-26T05:53:39.393569+0000","last_undegraded":"2023-10-26T05:53:39.393569+0000","last_fullsized":"2023-10-26T05:53:39.393569+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389034+0000","last_change":"2023-10-26T05:53:39.389034+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389034+0000","last_undegraded":"2023-10-26T05:53:39.389034+0000","last_fullsized":"2023-10-26T05:53:39.389034+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407208+0000","last_change":"2023-10-26T05:53:39.407074+0000","last_active":"2023-10-26T05:53:39.407208+0000","last_peered":"2023-10-26T05:53:39.407208+0000","last_clean":"2023-10-26T05:53:39.407208+0000","last_became_active":"2023-10-26T05:53:39.403999+0000","last_became_peered":"2023-10-26T05:53:39.403999+0000","last_unstale":"2023-10-26T05:53:39.407208+0000","last_undegraded":"2023-10-26T05:53:39.407208+0000","last_fullsized":"2023-10-26T05:53:39.407208+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:40.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.390483+0000","last_change":"2023-10-26T05:53:39.390483+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.390483+0000","last_undegraded":"2023-10-26T05:53:39.390483+0000","last_fullsized":"2023-10-26T05:53:39.390483+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.376787+0000","last_change":"2023-10-26T05:53:39.376685+0000","last_active":"2023-10-26T05:53:39.376787+0000","last_peered":"2023-10-26T05:53:39.376787+0000","last_clean":"2023-10-26T05:53:39.376787+0000","last_became_active":"2023-10-26T05:53:39.376380+0000","last_became_peered":"2023-10-26T05:53:39.376380+0000","last_unstale":"2023-10-26T05:53:39.376787+0000","last_undegraded":"2023-10-26T05:53:39.376787+0000","last_fullsized":"2023-10-26T05:53:39.376787+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:47:02.364249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380548+0000","last_change":"2023-10-26T05:53:39.380548+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380548+0000","last_undegraded":"2023-10-26T05:53:39.380548+0000","last_fullsized":"2023-10-26T05:53:39.380548+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379531+0000","last_change":"2023-10-26T05:53:39.379454+0000","last_active":"2023-10-26T05:53:39.379531+0000","last_peered":"2023-10-26T05:53:39.379531+0000","last_clean":"2023-10-26T05:53:39.379531+0000","last_became_active":"2023-10-26T05:53:39.379184+0000","last_became_peered":"2023-10-26T05:53:39.379184+0000","last_unstale":"2023-10-26T05:53:39.379531+0000","last_undegraded":"2023-10-26T05:53:39.379531+0000","last_fullsized":"2023-10-26T05:53:39.379531+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:28.216044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.389712+0000","last_change":"2023-10-26T05:53:39.389712+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.389712+0000","last_undegraded":"2023-10-26T05:53:39.389712+0000","last_fullsized":"2023-10-26T05:53:39.389712+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.379179+0000","last_change":"2023-10-26T05:53:39.379041+0000","last_active":"2023-10-26T05:53:39.379179+0000","last_peered":"2023-10-26T05:53:39.379179+0000","last_clean":"2023-10-26T05:53:39.379179+0000","last_became_active":"2023-10-26T05:53:39.378384+0000","last_became_peered":"2023-10-26T05:53:39.378384+0000","last_unstale":"2023-10-26T05:53:39.379179+0000","last_undegraded":"2023-10-26T05:53:39.379179+0000","last_fullsized":"2023-10-26T05:53:39.379179+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:59:31.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.379897+0000","last_change":"2023-10-26T05:53:39.379897+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.379897+0000","last_undegraded":"2023-10-26T05:53:39.379897+0000","last_fullsized":"2023-10-26T05:53:39.379897+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387336+0000","last_change":"2023-10-26T05:53:39.387235+0000","last_active":"2023-10-26T05:53:39.387336+0000","last_peered":"2023-10-26T05:53:39.387336+0000","last_clean":"2023-10-26T05:53:39.387336+0000","last_became_active":"2023-10-26T05:53:39.386945+0000","last_became_peered":"2023-10-26T05:53:39.386945+0000","last_unstale":"2023-10-26T05:53:39.387336+0000","last_undegraded":"2023-10-26T05:53:39.387336+0000","last_fullsized":"2023-10-26T05:53:39.387336+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.386653+0000","last_change":"2023-10-26T05:53:39.386653+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.386653+0000","last_undegraded":"2023-10-26T05:53:39.386653+0000","last_fullsized":"2023-10-26T05:53:39.386653+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388010+0000","last_change":"2023-10-26T05:53:39.387929+0000","last_active":"2023-10-26T05:53:39.388010+0000","last_peered":"2023-10-26T05:53:39.388010+0000","last_clean":"2023-10-26T05:53:39.388010+0000","last_became_active":"2023-10-26T05:53:39.387571+0000","last_became_peered":"2023-10-26T05:53:39.387571+0000","last_unstale":"2023-10-26T05:53:39.388010+0000","last_undegraded":"2023-10-26T05:53:39.388010+0000","last_fullsized":"2023-10-26T05:53:39.388010+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:20:20.746112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380202+0000","last_change":"2023-10-26T05:53:39.380202+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380202+0000","last_undegraded":"2023-10-26T05:53:39.380202+0000","last_fullsized":"2023-10-26T05:53:39.380202+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.403035+0000","last_change":"2023-10-26T05:53:39.402866+0000","last_active":"2023-10-26T05:53:39.403035+0000","last_peered":"2023-10-26T05:53:39.403035+0000","last_clean":"2023-10-26T05:53:39.403035+0000","last_became_active":"2023-10-26T05:53:39.402076+0000","last_became_peered":"2023-10-26T05:53:39.402076+0000","last_unstale":"2023-10-26T05:53:39.403035+0000","last_undegraded":"2023-10-26T05:53:39.403035+0000","last_fullsized":"2023-10-26T05:53:39.403035+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:06.879916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381954+0000","last_change":"2023-10-26T05:53:39.381954+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381954+0000","last_undegraded":"2023-10-26T05:53:39.381954+0000","last_fullsized":"2023-10-26T05:53:39.381954+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387678+0000","last_change":"2023-10-26T05:53:39.387605+0000","last_active":"2023-10-26T05:53:39.387678+0000","last_peered":"2023-10-26T05:53:39.387678+0000","last_clean":"2023-10-26T05:53:39.387678+0000","last_became_active":"2023-10-26T05:53:39.387254+0000","last_became_peered":"2023-10-26T05:53:39.387254+0000","last_unstale":"2023-10-26T05:53:39.387678+0000","last_undegraded":"2023-10-26T05:53:39.387678+0000","last_fullsized":"2023-10-26T05:53:39.387678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:34.447932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.388864+0000","last_change":"2023-10-26T05:53:39.388864+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.388864+0000","last_undegraded":"2023-10-26T05:53:39.388864+0000","last_fullsized":"2023-10-26T05:53:39.388864+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.388559+0000","last_change":"2023-10-26T05:53:39.388401+0000","last_active":"2023-10-26T05:53:39.388559+0000","last_peered":"2023-10-26T05:53:39.388559+0000","last_clean":"2023-10-26T05:53:39.388559+0000","last_became_active":"2023-10-26T05:53:39.386956+0000","last_became_peered":"2023-10-26T05:53:39.386956+0000","last_unstale":"2023-10-26T05:53:39.388559+0000","last_undegraded":"2023-10-26T05:53:39.388559+0000","last_fullsized":"2023-10-26T05:53:39.388559+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:35.955038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381985+0000","last_change":"2023-10-26T05:53:39.381985+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381985+0000","last_undegraded":"2023-10-26T05:53:39.381985+0000","last_fullsized":"2023-10-26T05:53:39.381985+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.406716+0000","last_change":"2023-10-26T05:53:39.406613+0000","last_active":"2023-10-26T05:53:39.406716+0000","last_peered":"2023-10-26T05:53:39.406716+0000","last_clean":"2023-10-26T05:53:39.406716+0000","last_became_active":"2023-10-26T05:53:39.406173+0000","last_became_peered":"2023-10-26T05:53:39.406173+0000","last_unstale":"2023-10-26T05:53:39.406716+0000","last_undegraded":"2023-10-26T05:53:39.406716+0000","last_fullsized":"2023-10-26T05:53:39.406716+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:34.920267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.381947+0000","last_change":"2023-10-26T05:53:39.381947+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.381947+0000","last_undegraded":"2023-10-26T05:53:39.381947+0000","last_fullsized":"2023-10-26T05:53:39.381947+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.401196+0000","last_change":"2023-10-26T05:53:39.399323+0000","last_active":"2023-10-26T05:53:39.401196+0000","last_peered":"2023-10-26T05:53:39.401196+0000","last_clean":"2023-10-26T05:53:39.401196+0000","last_became_active":"2023-10-26T05:53:39.398652+0000","last_became_peered":"2023-10-26T05:53:39.398652+0000","last_unstale":"2023-10-26T05:53:39.401196+0000","last_undegraded":"2023-10-26T05:53:39.401196+0000","last_fullsized":"2023-10-26T05:53:39.401196+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.391217+0000","last_change":"2023-10-26T05:53:39.391217+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.391217+0000","last_undegraded":"2023-10-26T05:53:39.391217+0000","last_fullsized":"2023-10-26T05:53:39.391217+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.378146+0000","last_change":"2023-10-26T05:53:39.376606+0000","last_active":"2023-10-26T05:53:39.378146+0000","last_peered":"2023-10-26T05:53:39.378146+0000","last_clean":"2023-10-26T05:53:39.378146+0000","last_became_active":"2023-10-26T05:53:39.376273+0000","last_became_peered":"2023-10-26T05:53:39.376273+0000","last_unstale":"2023-10-26T05:53:39.378146+0000","last_undegraded":"2023-10-26T05:53:39.378146+0000","last_fullsized":"2023-10-26T05:53:39.378146+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:41:01.830273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380160+0000","last_change":"2023-10-26T05:53:39.380160+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380160+0000","last_undegraded":"2023-10-26T05:53:39.380160+0000","last_fullsized":"2023-10-26T05:53:39.380160+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.387314+0000","last_change":"2023-10-26T05:53:39.386889+0000","last_active":"2023-10-26T05:53:39.387314+0000","last_peered":"2023-10-26T05:53:39.387314+0000","last_clean":"2023-10-26T05:53:39.387314+0000","last_became_active":"2023-10-26T05:53:39.386241+0000","last_became_peered":"2023-10-26T05:53:39.386241+0000","last_unstale":"2023-10-26T05:53:39.387314+0000","last_undegraded":"2023-10-26T05:53:39.387314+0000","last_fullsized":"2023-10-26T05:53:39.387314+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:14.432402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":1,"reported_epoch":136,"state":"creating+peering","last_fresh":"2023-10-26T05:53:39.380090+0000","last_change":"2023-10-26T05:53:39.380090+0000","last_active":"2023-10-26T05:53:39.355381+0000","last_peered":"2023-10-26T05:53:39.355381+0000","last_clean":"2023-10-26T05:53:39.355381+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:53:39.380090+0000","last_undegraded":"2023-10-26T05:53:39.380090+0000","last_fullsized":"2023-10-26T05:53:39.380090+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-26T05:53:39.407711+0000","last_change":"2023-10-26T05:53:39.407411+0000","last_active":"2023-10-26T05:53:39.407711+0000","last_peered":"2023-10-26T05:53:39.407711+0000","last_clean":"2023-10-26T05:53:39.407711+0000","last_became_active":"2023-10-26T05:53:39.404908+0000","last_became_peered":"2023-10-26T05:53:39.404908+0000","last_unstale":"2023-10-26T05:53:39.407711+0000","last_undegraded":"2023-10-26T05:53:39.407711+0000","last_fullsized":"2023-10-26T05:53:39.407711+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:04:52.683441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":274,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-26T05:53:38.367192+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:38.367192+0000","last_peered":"2023-10-26T05:53:38.367192+0000","last_clean":"2023-10-26T05:53:38.367192+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:38.367192+0000","last_undegraded":"2023-10-26T05:53:38.367192+0000","last_fullsized":"2023-10-26T05:53:38.367192+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194007,"statfs":{"total":200000000,"available":198663550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81699999999999995,"15min":0.85299999999999998},"min":{"1min":0.501,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0840000000000001,"5min":1.3100000000000001,"15min":1.347},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.44600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1140000000000001,"5min":1.286,"15min":1.3100000000000001},"last":0.872}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.024},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.60299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.24,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.236}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84599999999999997,"15min":0.81999999999999995},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.216,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.167,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.879,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.4330000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.375,"15min":1.375},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.91400000000000003,"15min":0.91900000000000004},"min":{"1min":0.61399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.639,"15min":1.639},"last":1.026},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95899999999999996,"15min":0.94199999999999995},"min":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.145,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.98799999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.024,"15min":1.1100000000000001},"min":{"1min":0.70099999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2729999999999999,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.999,"5min":1.0169999999999999,"15min":1.079},"min":{"1min":0.76300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.25,"5min":1.702,"15min":1.702},"last":1.1539999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0820000000000001,"15min":1.1120000000000001},"min":{"1min":0.91400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.391,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.048,"15min":1.048},"min":{"1min":0.65400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.5469999999999999,"15min":1.6180000000000001},"last":1.1279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83999999999999997,"15min":0.81299999999999994},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85899999999999999,"15min":0.85699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.298,"15min":1.3},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1779999999999999,"15min":1.2609999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.83999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.167,"5min":1.405,"15min":1.405},"last":1.2010000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90100000000000002,"15min":0.81499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89500000000000002,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.337,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.151}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.89400000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2629999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.054,"15min":0.97199999999999998},"min":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.502},"max":{"1min":1.2869999999999999,"5min":1.399,"15min":1.399},"last":1.1160000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":1.429,"15min":1.429},"last":1.014},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.98999999999999999,"15min":0.97599999999999998},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.024,"15min":1.044},"min":{"1min":0.59999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.554},"last":1.286},{"interface":"front","average":{"1min":1.024,"5min":0.95199999999999996,"15min":0.99199999999999999},"min":{"1min":0.53300000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.25}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.097,"15min":1.1160000000000001},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.339}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.278,"5min":1.579,"15min":1.579},"last":1.248},{"interface":"front","average":{"1min":0.87,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.272,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.591}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.89100000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.464,"15min":1.464},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.91400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1679999999999999,"5min":1.4219999999999999,"15min":1.5089999999999999},"last":1.006},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.3160000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90200000000000002,"15min":0.91800000000000004},"min":{"1min":0.69399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.448,"5min":1.448,"15min":1.548},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95699999999999996,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.288,"5min":1.4139999999999999,"15min":1.587},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93600000000000005,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.286,"5min":1.286,"15min":2.1909999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.0489999999999999,"15min":1.097},"min":{"1min":0.51700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.4950000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":1.018},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.321,"5min":1.393,"15min":1.5780000000000001},"last":1.744},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1930000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0409999999999999,"15min":1.0720000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.093,"15min":1.159},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.6830000000000001},"last":1.8939999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.85399999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1899999999999999,"5min":1.347,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.879,"15min":0.88200000000000001},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2909999999999999,"5min":1.502,"15min":1.502},"last":1.143}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.81699999999999995},"min":{"1min":0.626,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.097,"5min":1.25,"15min":1.375},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94299999999999995,"15min":0.89800000000000002},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3440000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":1.1899999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.81799999999999995},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.55500000000000005,"15min":0.49199999999999999},"max":{"1min":1.2170000000000001,"5min":1.373,"15min":1.373},"last":0.79100000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":1.341,"15min":2.0190000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.97299999999999998,"15min":0.94299999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2350000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0720000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.998},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.673,"15min":1.673},"last":0.86899999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.012,"15min":1},"min":{"1min":0.65200000000000002,"5min":0.503,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.028,"15min":1.036},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.05}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.054,"15min":1.038},"min":{"1min":0.60799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.748,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4590000000000001,"5min":1.728,"15min":1.728},"last":1.0269999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.84299999999999997,"5min":1.385,"15min":1.385},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.2689999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85099999999999998,"5min":1.349,"15min":1.349},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.373}]},{"osd":2,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2170000000000001,"5min":1.367,"15min":1.367},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81000000000000005,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0389999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.441}]},{"osd":4,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.57399999999999995,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91300000000000003,"15min":0.92200000000000004},"min":{"1min":0.60899999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1719999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.748,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.413,"15min":1.478},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94799999999999995,"15min":0.90900000000000003},"min":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1830000000000001,"5min":1.742,"15min":1.742},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.99199999999999999},"min":{"1min":0.73399999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.266,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0249999999999999},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.196,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.92900000000000005},"min":{"1min":0.80000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.216,"5min":1.661,"15min":1.661},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.24,"5min":2.004,"15min":2.004},"last":0.752}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.433},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.60599999999999998},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.379,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96699999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.37}]},{"osd":3,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82099999999999995,"15min":0.73699999999999999},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.28299999999999997},"max":{"1min":1.1180000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.60099999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.022,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93799999999999994,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.49399999999999999},"max":{"1min":1.5189999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.88,"5min":0.92500000000000004,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4870000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1,"15min":0.91400000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.55200000000000005},"max":{"1min":1.5600000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.028,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.58199999999999996,"15min":0.51700000000000002},"max":{"1min":1.4790000000000001,"5min":1.653,"15min":1.653},"last":0.60399999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0209999999999999,"15min":0.92600000000000005},"min":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.6000000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.56599999999999995,"15min":0.5},"max":{"1min":1.7769999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.626}]},{"osd":7,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97499999999999998,"15min":0.90800000000000003},"min":{"1min":0.746,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.2410000000000001,"15min":1.04},"min":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.51800000000000002},"max":{"1min":1.823,"5min":15.352,"15min":15.352},"last":0.64600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253918,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.254},"max":{"1min":0.98799999999999999,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.372},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.623},"min":{"1min":0.34200000000000003,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":1.038,"5min":1.157,"15min":1.157},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95399999999999996,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.72199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1160000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.137,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.27700000000000002},"max":{"1min":1.175,"5min":1.298,"15min":1.298},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88200000000000001,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.113,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2290000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.23,"5min":1.492,"15min":1.492},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.93300000000000005},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.071,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.012,"15min":1.0349999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.258,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.468,"15min":1.837},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0049999999999999,"15min":1.026},"min":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.321,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":1.026},{"interface":"front","average":{"1min":1.038,"5min":1.0840000000000001,"15min":1.0900000000000001},"min":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.319,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":0.87}]}]},{"osd":0,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1311,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194001,"statfs":{"total":200000000,"available":198657275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.62},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.496}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.79100000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.095,"5min":1.1140000000000001,"15min":1.177},"last":0.755}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.26800000000000002},"max":{"1min":1.3520000000000001,"5min":1.409,"15min":1.409},"last":0.625},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.79400000000000004,"15min":0.70599999999999996},"min":{"1min":0.36899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.68300000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.85599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46100000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.179,"5min":0.98999999999999999,"15min":0.91600000000000004},"min":{"1min":0.70799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.86699999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.234,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.94599999999999995,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94399999999999995,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59799999999999998},"max":{"1min":1.149,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.099},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.1279999999999999,"15min":1.002},"min":{"1min":0.82499999999999996,"5min":0.60499999999999998,"15min":0.54800000000000004},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.95499999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"max":{"1min":1.214,"5min":1.669,"15min":1.6990000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.03,"15min":0.98899999999999999},"min":{"1min":0.80600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4790000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.98699999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:45.794 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:53:45.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:53:46.334 INFO:tasks.ceph.mds.a.smithi071.stderr:2023-10-26T05:53:46.327+0000 7f8c9da52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:53:47.889 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:53:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 -- 172.21.15.71:0/3013360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f673c005190 msgr2=0x7f673c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 --2- 172.21.15.71:0/3013360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f673c005190 0x7f673c00b990 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f674c006fd0 tx=0x7f674c01cfa0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 -- 172.21.15.71:0/3013360735 shutdown_connections 2023-10-26T05:53:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 --2- 172.21.15.71:0/3013360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f673c005190 0x7f673c00b990 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 -- 172.21.15.71:0/3013360735 >> 172.21.15.71:0/3013360735 conn(0x7f6754069770 msgr2=0x7f6754069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:48.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 -- 172.21.15.71:0/3013360735 shutdown_connections 2023-10-26T05:53:48.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.019+0000 7f675b827640 1 -- 172.21.15.71:0/3013360735 wait complete. 2023-10-26T05:53:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 Processor -- start 2023-10-26T05:53:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- start start 2023-10-26T05:53:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f673c005190 0x7f6754137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67541387c0 0x7f6754138c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 0x7f6754152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754153450 con 0x7f6754151ed0 2023-10-26T05:53:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6754139180 con 0x7f67541387c0 2023-10-26T05:53:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6754152890 con 0x7f673c005190 2023-10-26T05:53:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 0x7f6754152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 0x7f6754152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34496/0 (socket says 172.21.15.71:34496) 2023-10-26T05:53:48.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 -- 172.21.15.71:0/540766727 learned_addr learned my addr 172.21.15.71:0/540766727 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6758dc2640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67541387c0 0x7f6754138c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f67595c3640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f673c005190 0x7f6754137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 -- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f673c005190 msgr2=0x7f6754137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f673c005190 0x7f6754137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 -- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67541387c0 msgr2=0x7f6754138c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67541387c0 0x7f6754138c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 -- 172.21.15.71:0/540766727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675415d680 con 0x7f6754151ed0 2023-10-26T05:53:48.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6759dc4640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 0x7f6754152350 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f674c007010 tx=0x7f674c001ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:48.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0016a0 con 0x7f6754151ed0 2023-10-26T05:53:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f674c006190 con 0x7f6754151ed0 2023-10-26T05:53:48.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c0065a0 con 0x7f6754151ed0 2023-10-26T05:53:48.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675413b720 con 0x7f6754151ed0 2023-10-26T05:53:48.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675413a980 con 0x7f6754151ed0 2023-10-26T05:53:48.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.023+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6754066b10 con 0x7f6754151ed0 2023-10-26T05:53:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.027+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f674c0aa030 con 0x7f6754151ed0 2023-10-26T05:53:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.027+0000 7f6749ffb640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67400664a0 0x7f6740068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.027+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f674c11cce0 con 0x7f6754151ed0 2023-10-26T05:53:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.027+0000 7f67595c3640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67400664a0 0x7f6740068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.027+0000 7f67595c3640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67400664a0 0x7f6740068960 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6744009f20 tx=0x7f6744009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:48.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.031+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f674c0ea160 con 0x7f6754151ed0 2023-10-26T05:53:48.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.183+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6754070b90 con 0x7f67400664a0 2023-10-26T05:53:48.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.203+0000 7f6749ffb640 1 -- 172.21.15.71:0/540766727 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283554 (secure 0 0 0) 0x7f6744016070 con 0x7f67400664a0 2023-10-26T05:53:48.213 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:48.215 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:53:48.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67400664a0 msgr2=0x7f6740068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67400664a0 0x7f6740068960 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6744009f20 tx=0x7f6744009500 comp rx=0 tx=0).stop 2023-10-26T05:53:48.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 msgr2=0x7f6754152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 0x7f6754152350 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f674c007010 tx=0x7f674c001ca0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 shutdown_connections 2023-10-26T05:53:48.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f67400664a0 0x7f6740068960 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6754151ed0 0x7f6754152350 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67541387c0 0x7f6754138c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 --2- 172.21.15.71:0/540766727 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f673c005190 0x7f6754137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 >> 172.21.15.71:0/540766727 conn(0x7f6754069770 msgr2=0x7f6754158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:48.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 shutdown_connections 2023-10-26T05:53:48.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.207+0000 7f675b827640 1 -- 172.21.15.71:0/540766727 wait complete. 2023-10-26T05:53:48.232 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-10-26T05:53:47.540232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555362,"statfs":{"total":1600000000,"available":1592690988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003009"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.742852+0000","last_change":"2023-10-26T05:53:40.407043+0000","last_active":"2023-10-26T05:53:41.742852+0000","last_peered":"2023-10-26T05:53:41.742852+0000","last_clean":"2023-10-26T05:53:41.742852+0000","last_became_active":"2023-10-26T05:53:40.404068+0000","last_became_peered":"2023-10-26T05:53:40.404068+0000","last_unstale":"2023-10-26T05:53:41.742852+0000","last_undegraded":"2023-10-26T05:53:41.742852+0000","last_fullsized":"2023-10-26T05:53:41.742852+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:41.556731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.384588+0000","last_change":"2023-10-26T05:53:40.388827+0000","last_active":"2023-10-26T05:53:41.384588+0000","last_peered":"2023-10-26T05:53:41.384588+0000","last_clean":"2023-10-26T05:53:41.384588+0000","last_became_active":"2023-10-26T05:53:40.386063+0000","last_became_peered":"2023-10-26T05:53:40.386063+0000","last_unstale":"2023-10-26T05:53:41.384588+0000","last_undegraded":"2023-10-26T05:53:41.384588+0000","last_fullsized":"2023-10-26T05:53:41.384588+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:11:21.892097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.750250+0000","last_change":"2023-10-26T05:53:40.418928+0000","last_active":"2023-10-26T05:53:41.750250+0000","last_peered":"2023-10-26T05:53:41.750250+0000","last_clean":"2023-10-26T05:53:41.750250+0000","last_became_active":"2023-10-26T05:53:40.412443+0000","last_became_peered":"2023-10-26T05:53:40.412443+0000","last_unstale":"2023-10-26T05:53:41.750250+0000","last_undegraded":"2023-10-26T05:53:41.750250+0000","last_fullsized":"2023-10-26T05:53:41.750250+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:52:55.600940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.742375+0000","last_change":"2023-10-26T05:53:40.409716+0000","last_active":"2023-10-26T05:53:41.742375+0000","last_peered":"2023-10-26T05:53:41.742375+0000","last_clean":"2023-10-26T05:53:41.742375+0000","last_became_active":"2023-10-26T05:53:40.406969+0000","last_became_peered":"2023-10-26T05:53:40.406969+0000","last_unstale":"2023-10-26T05:53:41.742375+0000","last_undegraded":"2023-10-26T05:53:41.742375+0000","last_fullsized":"2023-10-26T05:53:41.742375+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:40:53.124462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.379855+0000","last_change":"2023-10-26T05:53:40.397137+0000","last_active":"2023-10-26T05:53:41.379855+0000","last_peered":"2023-10-26T05:53:41.379855+0000","last_clean":"2023-10-26T05:53:41.379855+0000","last_became_active":"2023-10-26T05:53:40.396506+0000","last_became_peered":"2023-10-26T05:53:40.396506+0000","last_unstale":"2023-10-26T05:53:41.379855+0000","last_undegraded":"2023-10-26T05:53:41.379855+0000","last_fullsized":"2023-10-26T05:53:41.379855+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:50:25.999641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.376679+0000","last_change":"2023-10-26T05:53:40.383582+0000","last_active":"2023-10-26T05:53:41.376679+0000","last_peered":"2023-10-26T05:53:41.376679+0000","last_clean":"2023-10-26T05:53:41.376679+0000","last_became_active":"2023-10-26T05:53:40.382117+0000","last_became_peered":"2023-10-26T05:53:40.382117+0000","last_unstale":"2023-10-26T05:53:41.376679+0000","last_undegraded":"2023-10-26T05:53:41.376679+0000","last_fullsized":"2023-10-26T05:53:41.376679+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:40:50.437759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.749676+0000","last_change":"2023-10-26T05:53:40.406866+0000","last_active":"2023-10-26T05:53:41.749676+0000","last_peered":"2023-10-26T05:53:41.749676+0000","last_clean":"2023-10-26T05:53:41.749676+0000","last_became_active":"2023-10-26T05:53:40.405054+0000","last_became_peered":"2023-10-26T05:53:40.405054+0000","last_unstale":"2023-10-26T05:53:41.749676+0000","last_undegraded":"2023-10-26T05:53:41.749676+0000","last_fullsized":"2023-10-26T05:53:41.749676+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:59.768990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.751079+0000","last_change":"2023-10-26T05:53:40.413679+0000","last_active":"2023-10-26T05:53:41.751079+0000","last_peered":"2023-10-26T05:53:41.751079+0000","last_clean":"2023-10-26T05:53:41.751079+0000","last_became_active":"2023-10-26T05:53:40.411296+0000","last_became_peered":"2023-10-26T05:53:40.411296+0000","last_unstale":"2023-10-26T05:53:41.751079+0000","last_undegraded":"2023-10-26T05:53:41.751079+0000","last_fullsized":"2023-10-26T05:53:41.751079+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:22:42.468545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.742496+0000","last_change":"2023-10-26T05:53:40.409899+0000","last_active":"2023-10-26T05:53:41.742496+0000","last_peered":"2023-10-26T05:53:41.742496+0000","last_clean":"2023-10-26T05:53:41.742496+0000","last_became_active":"2023-10-26T05:53:40.407884+0000","last_became_peered":"2023-10-26T05:53:40.407884+0000","last_unstale":"2023-10-26T05:53:41.742496+0000","last_undegraded":"2023-10-26T05:53:41.742496+0000","last_fullsized":"2023-10-26T05:53:41.742496+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:47:39.051859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743231+0000","last_change":"2023-10-26T05:53:40.409242+0000","last_active":"2023-10-26T05:53:41.743231+0000","last_peered":"2023-10-26T05:53:41.743231+0000","last_clean":"2023-10-26T05:53:41.743231+0000","last_became_active":"2023-10-26T05:53:40.400023+0000","last_became_peered":"2023-10-26T05:53:40.400023+0000","last_unstale":"2023-10-26T05:53:41.743231+0000","last_undegraded":"2023-10-26T05:53:41.743231+0000","last_fullsized":"2023-10-26T05:53:41.743231+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:03.561387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.384341+0000","last_change":"2023-10-26T05:53:40.398172+0000","last_active":"2023-10-26T05:53:41.384341+0000","last_peered":"2023-10-26T05:53:41.384341+0000","last_clean":"2023-10-26T05:53:41.384341+0000","last_became_active":"2023-10-26T05:53:40.397537+0000","last_became_peered":"2023-10-26T05:53:40.397537+0000","last_unstale":"2023-10-26T05:53:41.384341+0000","last_undegraded":"2023-10-26T05:53:41.384341+0000","last_fullsized":"2023-10-26T05:53:41.384341+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:15.447207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.742837+0000","last_change":"2023-10-26T05:53:40.404598+0000","last_active":"2023-10-26T05:53:41.742837+0000","last_peered":"2023-10-26T05:53:41.742837+0000","last_clean":"2023-10-26T05:53:41.742837+0000","last_became_active":"2023-10-26T05:53:40.399882+0000","last_became_peered":"2023-10-26T05:53:40.399882+0000","last_unstale":"2023-10-26T05:53:41.742837+0000","last_undegraded":"2023-10-26T05:53:41.742837+0000","last_fullsized":"2023-10-26T05:53:41.742837+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:29:59.029057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.750574+0000","last_change":"2023-10-26T05:53:40.413160+0000","last_active":"2023-10-26T05:53:41.750574+0000","last_peered":"2023-10-26T05:53:41.750574+0000","last_clean":"2023-10-26T05:53:41.750574+0000","last_became_active":"2023-10-26T05:53:40.410311+0000","last_became_peered":"2023-10-26T05:53:40.410311+0000","last_unstale":"2023-10-26T05:53:41.750574+0000","last_undegraded":"2023-10-26T05:53:41.750574+0000","last_fullsized":"2023-10-26T05:53:41.750574+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:04:25.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748779+0000","last_change":"2023-10-26T05:53:40.411197+0000","last_active":"2023-10-26T05:53:41.748779+0000","last_peered":"2023-10-26T05:53:41.748779+0000","last_clean":"2023-10-26T05:53:41.748779+0000","last_became_active":"2023-10-26T05:53:40.405406+0000","last_became_peered":"2023-10-26T05:53:40.405406+0000","last_unstale":"2023-10-26T05:53:41.748779+0000","last_undegraded":"2023-10-26T05:53:41.748779+0000","last_fullsized":"2023-10-26T05:53:41.748779+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:35.462737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.377304+0000","last_change":"2023-10-26T05:53:40.383531+0000","last_active":"2023-10-26T05:53:41.377304+0000","last_peered":"2023-10-26T05:53:41.377304+0000","last_clean":"2023-10-26T05:53:41.377304+0000","last_became_active":"2023-10-26T05:53:40.382869+0000","last_became_peered":"2023-10-26T05:53:40.382869+0000","last_unstale":"2023-10-26T05:53:41.377304+0000","last_undegraded":"2023-10-26T05:53:41.377304+0000","last_fullsized":"2023-10-26T05:53:41.377304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:25:13.497323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743078+0000","last_change":"2023-10-26T05:53:40.407211+0000","last_active":"2023-10-26T05:53:41.743078+0000","last_peered":"2023-10-26T05:53:41.743078+0000","last_clean":"2023-10-26T05:53:41.743078+0000","last_became_active":"2023-10-26T05:53:40.405088+0000","last_became_peered":"2023-10-26T05:53:40.405088+0000","last_unstale":"2023-10-26T05:53:41.743078+0000","last_undegraded":"2023-10-26T05:53:41.743078+0000","last_fullsized":"2023-10-26T05:53:41.743078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:34:37.664086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.384965+0000","last_change":"2023-10-26T05:53:40.400296+0000","last_active":"2023-10-26T05:53:41.384965+0000","last_peered":"2023-10-26T05:53:41.384965+0000","last_clean":"2023-10-26T05:53:41.384965+0000","last_became_active":"2023-10-26T05:53:40.398914+0000","last_became_peered":"2023-10-26T05:53:40.398914+0000","last_unstale":"2023-10-26T05:53:41.384965+0000","last_undegraded":"2023-10-26T05:53:41.384965+0000","last_fullsized":"2023-10-26T05:53:41.384965+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:39.886243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748017+0000","last_change":"2023-10-26T05:53:40.418664+0000","last_active":"2023-10-26T05:53:41.748017+0000","last_peered":"2023-10-26T05:53:41.748017+0000","last_clean":"2023-10-26T05:53:41.748017+0000","last_became_active":"2023-10-26T05:53:40.417633+0000","last_became_peered":"2023-10-26T05:53:40.417633+0000","last_unstale":"2023-10-26T05:53:41.748017+0000","last_undegraded":"2023-10-26T05:53:41.748017+0000","last_fullsized":"2023-10-26T05:53:41.748017+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:04:25.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.749056+0000","last_change":"2023-10-26T05:53:40.415080+0000","last_active":"2023-10-26T05:53:41.749056+0000","last_peered":"2023-10-26T05:53:41.749056+0000","last_clean":"2023-10-26T05:53:41.749056+0000","last_became_active":"2023-10-26T05:53:40.405974+0000","last_became_peered":"2023-10-26T05:53:40.405974+0000","last_unstale":"2023-10-26T05:53:41.749056+0000","last_undegraded":"2023-10-26T05:53:41.749056+0000","last_fullsized":"2023-10-26T05:53:41.749056+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:53.202414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.385673+0000","last_change":"2023-10-26T05:53:40.408377+0000","last_active":"2023-10-26T05:53:41.385673+0000","last_peered":"2023-10-26T05:53:41.385673+0000","last_clean":"2023-10-26T05:53:41.385673+0000","last_became_active":"2023-10-26T05:53:40.407800+0000","last_became_peered":"2023-10-26T05:53:40.407800+0000","last_unstale":"2023-10-26T05:53:41.385673+0000","last_undegraded":"2023-10-26T05:53:41.385673+0000","last_fullsized":"2023-10-26T05:53:41.385673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:22:42.468545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.384906+0000","last_change":"2023-10-26T05:53:40.389256+0000","last_active":"2023-10-26T05:53:41.384906+0000","last_peered":"2023-10-26T05:53:41.384906+0000","last_clean":"2023-10-26T05:53:41.384906+0000","last_became_active":"2023-10-26T05:53:40.388640+0000","last_became_peered":"2023-10-26T05:53:40.388640+0000","last_unstale":"2023-10-26T05:53:41.384906+0000","last_undegraded":"2023-10-26T05:53:41.384906+0000","last_fullsized":"2023-10-26T05:53:41.384906+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:27.446546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743098+0000","last_change":"2023-10-26T05:53:40.399376+0000","last_active":"2023-10-26T05:53:41.743098+0000","last_peered":"2023-10-26T05:53:41.743098+0000","last_clean":"2023-10-26T05:53:41.743098+0000","last_became_active":"2023-10-26T05:53:40.398254+0000","last_became_peered":"2023-10-26T05:53:40.398254+0000","last_unstale":"2023-10-26T05:53:41.743098+0000","last_undegraded":"2023-10-26T05:53:41.743098+0000","last_fullsized":"2023-10-26T05:53:41.743098+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:09:00.374768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.384422+0000","last_change":"2023-10-26T05:53:40.412254+0000","last_active":"2023-10-26T05:53:41.384422+0000","last_peered":"2023-10-26T05:53:41.384422+0000","last_clean":"2023-10-26T05:53:41.384422+0000","last_became_active":"2023-10-26T05:53:40.411617+0000","last_became_peered":"2023-10-26T05:53:40.411617+0000","last_unstale":"2023-10-26T05:53:41.384422+0000","last_undegraded":"2023-10-26T05:53:41.384422+0000","last_fullsized":"2023-10-26T05:53:41.384422+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:53.202414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743667+0000","last_change":"2023-10-26T05:53:40.403242+0000","last_active":"2023-10-26T05:53:41.743667+0000","last_peered":"2023-10-26T05:53:41.743667+0000","last_clean":"2023-10-26T05:53:41.743667+0000","last_became_active":"2023-10-26T05:53:40.402079+0000","last_became_peered":"2023-10-26T05:53:40.402079+0000","last_unstale":"2023-10-26T05:53:41.743667+0000","last_undegraded":"2023-10-26T05:53:41.743667+0000","last_fullsized":"2023-10-26T05:53:41.743667+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:03:08.988839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.386025+0000","last_change":"2023-10-26T05:53:40.398213+0000","last_active":"2023-10-26T05:53:41.386025+0000","last_peered":"2023-10-26T05:53:41.386025+0000","last_clean":"2023-10-26T05:53:41.386025+0000","last_became_active":"2023-10-26T05:53:40.397654+0000","last_became_peered":"2023-10-26T05:53:40.397654+0000","last_unstale":"2023-10-26T05:53:41.386025+0000","last_undegraded":"2023-10-26T05:53:41.386025+0000","last_fullsized":"2023-10-26T05:53:41.386025+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:09.535434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743255+0000","last_change":"2023-10-26T05:53:40.410281+0000","last_active":"2023-10-26T05:53:41.743255+0000","last_peered":"2023-10-26T05:53:41.743255+0000","last_clean":"2023-10-26T05:53:41.743255+0000","last_became_active":"2023-10-26T05:53:40.408109+0000","last_became_peered":"2023-10-26T05:53:40.408109+0000","last_unstale":"2023-10-26T05:53:41.743255+0000","last_undegraded":"2023-10-26T05:53:41.743255+0000","last_fullsized":"2023-10-26T05:53:41.743255+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:53:43.683003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.376885+0000","last_change":"2023-10-26T05:53:40.396162+0000","last_active":"2023-10-26T05:53:41.376885+0000","last_peered":"2023-10-26T05:53:41.376885+0000","last_clean":"2023-10-26T05:53:41.376885+0000","last_became_active":"2023-10-26T05:53:40.395175+0000","last_became_peered":"2023-10-26T05:53:40.395175+0000","last_unstale":"2023-10-26T05:53:41.376885+0000","last_undegraded":"2023-10-26T05:53:41.376885+0000","last_fullsized":"2023-10-26T05:53:41.376885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:06.619355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743759+0000","last_change":"2023-10-26T05:53:40.400948+0000","last_active":"2023-10-26T05:53:41.743759+0000","last_peered":"2023-10-26T05:53:41.743759+0000","last_clean":"2023-10-26T05:53:41.743759+0000","last_became_active":"2023-10-26T05:53:40.399308+0000","last_became_peered":"2023-10-26T05:53:40.399308+0000","last_unstale":"2023-10-26T05:53:41.743759+0000","last_undegraded":"2023-10-26T05:53:41.743759+0000","last_fullsized":"2023-10-26T05:53:41.743759+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:27.787630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.384507+0000","last_change":"2023-10-26T05:53:40.412163+0000","last_active":"2023-10-26T05:53:41.384507+0000","last_peered":"2023-10-26T05:53:41.384507+0000","last_clean":"2023-10-26T05:53:41.384507+0000","last_became_active":"2023-10-26T05:53:40.410046+0000","last_became_peered":"2023-10-26T05:53:40.410046+0000","last_unstale":"2023-10-26T05:53:41.384507+0000","last_undegraded":"2023-10-26T05:53:41.384507+0000","last_fullsized":"2023-10-26T05:53:41.384507+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:57.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.750490+0000","last_change":"2023-10-26T05:53:40.404317+0000","last_active":"2023-10-26T05:53:41.750490+0000","last_peered":"2023-10-26T05:53:41.750490+0000","last_clean":"2023-10-26T05:53:41.750490+0000","last_became_active":"2023-10-26T05:53:40.403785+0000","last_became_peered":"2023-10-26T05:53:40.403785+0000","last_unstale":"2023-10-26T05:53:41.750490+0000","last_undegraded":"2023-10-26T05:53:41.750490+0000","last_fullsized":"2023-10-26T05:53:41.750490+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:25:11.338928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743383+0000","last_change":"2023-10-26T05:53:40.410125+0000","last_active":"2023-10-26T05:53:41.743383+0000","last_peered":"2023-10-26T05:53:41.743383+0000","last_clean":"2023-10-26T05:53:41.743383+0000","last_became_active":"2023-10-26T05:53:40.407751+0000","last_became_peered":"2023-10-26T05:53:40.407751+0000","last_unstale":"2023-10-26T05:53:41.743383+0000","last_undegraded":"2023-10-26T05:53:41.743383+0000","last_fullsized":"2023-10-26T05:53:41.743383+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:57:55.789413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.377167+0000","last_change":"2023-10-26T05:53:40.388837+0000","last_active":"2023-10-26T05:53:41.377167+0000","last_peered":"2023-10-26T05:53:41.377167+0000","last_clean":"2023-10-26T05:53:41.377167+0000","last_became_active":"2023-10-26T05:53:40.388206+0000","last_became_peered":"2023-10-26T05:53:40.388206+0000","last_unstale":"2023-10-26T05:53:41.377167+0000","last_undegraded":"2023-10-26T05:53:41.377167+0000","last_fullsized":"2023-10-26T05:53:41.377167+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:59.532149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.376594+0000","last_change":"2023-10-26T05:53:40.397418+0000","last_active":"2023-10-26T05:53:41.376594+0000","last_peered":"2023-10-26T05:53:41.376594+0000","last_clean":"2023-10-26T05:53:41.376594+0000","last_became_active":"2023-10-26T05:53:40.396449+0000","last_became_peered":"2023-10-26T05:53:40.396449+0000","last_unstale":"2023-10-26T05:53:41.376594+0000","last_undegraded":"2023-10-26T05:53:41.376594+0000","last_fullsized":"2023-10-26T05:53:41.376594+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:42:38.247806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"137'4","reported_seq":16,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.431086+0000","last_change":"2023-10-26T05:53:39.400795+0000","last_active":"2023-10-26T05:53:40.431086+0000","last_peered":"2023-10-26T05:53:40.431086+0000","last_clean":"2023-10-26T05:53:40.431086+0000","last_became_active":"2023-10-26T05:53:39.400245+0000","last_became_peered":"2023-10-26T05:53:39.400245+0000","last_unstale":"2023-10-26T05:53:40.431086+0000","last_undegraded":"2023-10-26T05:53:40.431086+0000","last_fullsized":"2023-10-26T05:53:40.431086+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:05:58.517344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748380+0000","last_change":"2023-10-26T05:53:40.418239+0000","last_active":"2023-10-26T05:53:41.748380+0000","last_peered":"2023-10-26T05:53:41.748380+0000","last_clean":"2023-10-26T05:53:41.748380+0000","last_became_active":"2023-10-26T05:53:40.415696+0000","last_became_peered":"2023-10-26T05:53:40.415696+0000","last_unstale":"2023-10-26T05:53:41.748380+0000","last_undegraded":"2023-10-26T05:53:41.748380+0000","last_fullsized":"2023-10-26T05:53:41.748380+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:03.451684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.428795+0000","last_change":"2023-10-26T05:53:39.398370+0000","last_active":"2023-10-26T05:53:40.428795+0000","last_peered":"2023-10-26T05:53:40.428795+0000","last_clean":"2023-10-26T05:53:40.428795+0000","last_became_active":"2023-10-26T05:53:39.397754+0000","last_became_peered":"2023-10-26T05:53:39.397754+0000","last_unstale":"2023-10-26T05:53:40.428795+0000","last_undegraded":"2023-10-26T05:53:40.428795+0000","last_fullsized":"2023-10-26T05:53:40.428795+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:26.772825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.376805+0000","last_change":"2023-10-26T05:53:40.383951+0000","last_active":"2023-10-26T05:53:41.376805+0000","last_peered":"2023-10-26T05:53:41.376805+0000","last_clean":"2023-10-26T05:53:41.376805+0000","last_became_active":"2023-10-26T05:53:40.382965+0000","last_became_peered":"2023-10-26T05:53:40.382965+0000","last_unstale":"2023-10-26T05:53:41.376805+0000","last_undegraded":"2023-10-26T05:53:41.376805+0000","last_fullsized":"2023-10-26T05:53:41.376805+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:38:59.742946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.418803+0000","last_change":"2023-10-26T05:53:39.383805+0000","last_active":"2023-10-26T05:53:40.418803+0000","last_peered":"2023-10-26T05:53:40.418803+0000","last_clean":"2023-10-26T05:53:40.418803+0000","last_became_active":"2023-10-26T05:53:39.383392+0000","last_became_peered":"2023-10-26T05:53:39.383392+0000","last_unstale":"2023-10-26T05:53:40.418803+0000","last_undegraded":"2023-10-26T05:53:40.418803+0000","last_fullsized":"2023-10-26T05:53:40.418803+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:59:51.218101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748535+0000","last_change":"2023-10-26T05:53:40.408522+0000","last_active":"2023-10-26T05:53:41.748535+0000","last_peered":"2023-10-26T05:53:41.748535+0000","last_clean":"2023-10-26T05:53:41.748535+0000","last_became_active":"2023-10-26T05:53:40.405937+0000","last_became_peered":"2023-10-26T05:53:40.405937+0000","last_unstale":"2023-10-26T05:53:41.748535+0000","last_undegraded":"2023-10-26T05:53:41.748535+0000","last_fullsized":"2023-10-26T05:53:41.748535+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:30.040699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.418272+0000","last_change":"2023-10-26T05:53:39.378932+0000","last_active":"2023-10-26T05:53:40.418272+0000","last_peered":"2023-10-26T05:53:40.418272+0000","last_clean":"2023-10-26T05:53:40.418272+0000","last_became_active":"2023-10-26T05:53:39.378663+0000","last_became_peered":"2023-10-26T05:53:39.378663+0000","last_unstale":"2023-10-26T05:53:40.418272+0000","last_undegraded":"2023-10-26T05:53:40.418272+0000","last_fullsized":"2023-10-26T05:53:40.418272+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:12:42.458344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743032+0000","last_change":"2023-10-26T05:53:40.409367+0000","last_active":"2023-10-26T05:53:41.743032+0000","last_peered":"2023-10-26T05:53:41.743032+0000","last_clean":"2023-10-26T05:53:41.743032+0000","last_became_active":"2023-10-26T05:53:40.407991+0000","last_became_peered":"2023-10-26T05:53:40.407991+0000","last_unstale":"2023-10-26T05:53:41.743032+0000","last_undegraded":"2023-10-26T05:53:41.743032+0000","last_fullsized":"2023-10-26T05:53:41.743032+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:32:49.714355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.428453+0000","last_change":"2023-10-26T05:53:39.407951+0000","last_active":"2023-10-26T05:53:40.428453+0000","last_peered":"2023-10-26T05:53:40.428453+0000","last_clean":"2023-10-26T05:53:40.428453+0000","last_became_active":"2023-10-26T05:53:39.406096+0000","last_became_peered":"2023-10-26T05:53:39.406096+0000","last_unstale":"2023-10-26T05:53:40.428453+0000","last_undegraded":"2023-10-26T05:53:40.428453+0000","last_fullsized":"2023-10-26T05:53:40.428453+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:27:42.979004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.385296+0000","last_change":"2023-10-26T05:53:40.392402+0000","last_active":"2023-10-26T05:53:41.385296+0000","last_peered":"2023-10-26T05:53:41.385296+0000","last_clean":"2023-10-26T05:53:41.385296+0000","last_became_active":"2023-10-26T05:53:40.388162+0000","last_became_peered":"2023-10-26T05:53:40.388162+0000","last_unstale":"2023-10-26T05:53:41.385296+0000","last_undegraded":"2023-10-26T05:53:41.385296+0000","last_fullsized":"2023-10-26T05:53:41.385296+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:04:25.060987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.418304+0000","last_change":"2023-10-26T05:53:39.388934+0000","last_active":"2023-10-26T05:53:40.418304+0000","last_peered":"2023-10-26T05:53:40.418304+0000","last_clean":"2023-10-26T05:53:40.418304+0000","last_became_active":"2023-10-26T05:53:39.388668+0000","last_became_peered":"2023-10-26T05:53:39.388668+0000","last_unstale":"2023-10-26T05:53:40.418304+0000","last_undegraded":"2023-10-26T05:53:40.418304+0000","last_fullsized":"2023-10-26T05:53:40.418304+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:02:15.560171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743568+0000","last_change":"2023-10-26T05:53:40.410006+0000","last_active":"2023-10-26T05:53:41.743568+0000","last_peered":"2023-10-26T05:53:41.743568+0000","last_clean":"2023-10-26T05:53:41.743568+0000","last_became_active":"2023-10-26T05:53:40.408972+0000","last_became_peered":"2023-10-26T05:53:40.408972+0000","last_unstale":"2023-10-26T05:53:41.743568+0000","last_undegraded":"2023-10-26T05:53:41.743568+0000","last_fullsized":"2023-10-26T05:53:41.743568+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:02:51.652532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.382980+0000","last_change":"2023-10-26T05:53:39.404546+0000","last_active":"2023-10-26T05:53:40.382980+0000","last_peered":"2023-10-26T05:53:40.382980+0000","last_clean":"2023-10-26T05:53:40.382980+0000","last_became_active":"2023-10-26T05:53:39.403913+0000","last_became_peered":"2023-10-26T05:53:39.403913+0000","last_unstale":"2023-10-26T05:53:40.382980+0000","last_undegraded":"2023-10-26T05:53:40.382980+0000","last_fullsized":"2023-10-26T05:53:40.382980+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:20.405388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.747756+0000","last_change":"2023-10-26T05:53:40.404498+0000","last_active":"2023-10-26T05:53:41.747756+0000","last_peered":"2023-10-26T05:53:41.747756+0000","last_clean":"2023-10-26T05:53:41.747756+0000","last_became_active":"2023-10-26T05:53:40.403818+0000","last_became_peered":"2023-10-26T05:53:40.403818+0000","last_unstale":"2023-10-26T05:53:41.747756+0000","last_undegraded":"2023-10-26T05:53:41.747756+0000","last_fullsized":"2023-10-26T05:53:41.747756+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:18:55.100582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.390450+0000","last_change":"2023-10-26T05:53:39.389482+0000","last_active":"2023-10-26T05:53:40.390450+0000","last_peered":"2023-10-26T05:53:40.390450+0000","last_clean":"2023-10-26T05:53:40.390450+0000","last_became_active":"2023-10-26T05:53:39.389225+0000","last_became_peered":"2023-10-26T05:53:39.389225+0000","last_unstale":"2023-10-26T05:53:40.390450+0000","last_undegraded":"2023-10-26T05:53:40.390450+0000","last_fullsized":"2023-10-26T05:53:40.390450+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:43.779382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.749368+0000","last_change":"2023-10-26T05:53:40.407242+0000","last_active":"2023-10-26T05:53:41.749368+0000","last_peered":"2023-10-26T05:53:41.749368+0000","last_clean":"2023-10-26T05:53:41.749368+0000","last_became_active":"2023-10-26T05:53:40.406009+0000","last_became_peered":"2023-10-26T05:53:40.406009+0000","last_unstale":"2023-10-26T05:53:41.749368+0000","last_undegraded":"2023-10-26T05:53:41.749368+0000","last_fullsized":"2023-10-26T05:53:41.749368+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:35:15.545898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.388604+0000","last_change":"2023-10-26T05:53:39.376735+0000","last_active":"2023-10-26T05:53:40.388604+0000","last_peered":"2023-10-26T05:53:40.388604+0000","last_clean":"2023-10-26T05:53:40.388604+0000","last_became_active":"2023-10-26T05:53:39.376384+0000","last_became_peered":"2023-10-26T05:53:39.376384+0000","last_unstale":"2023-10-26T05:53:40.388604+0000","last_undegraded":"2023-10-26T05:53:40.388604+0000","last_fullsized":"2023-10-26T05:53:40.388604+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:05:58.324723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748690+0000","last_change":"2023-10-26T05:53:40.414963+0000","last_active":"2023-10-26T05:53:41.748690+0000","last_peered":"2023-10-26T05:53:41.748690+0000","last_clean":"2023-10-26T05:53:41.748690+0000","last_became_active":"2023-10-26T05:53:40.414205+0000","last_became_peered":"2023-10-26T05:53:40.414205+0000","last_unstale":"2023-10-26T05:53:41.748690+0000","last_undegraded":"2023-10-26T05:53:41.748690+0000","last_fullsized":"2023-10-26T05:53:41.748690+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:09.535434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.380640+0000","last_change":"2023-10-26T05:53:39.387783+0000","last_active":"2023-10-26T05:53:40.380640+0000","last_peered":"2023-10-26T05:53:40.380640+0000","last_clean":"2023-10-26T05:53:40.380640+0000","last_became_active":"2023-10-26T05:53:39.387261+0000","last_became_peered":"2023-10-26T05:53:39.387261+0000","last_unstale":"2023-10-26T05:53:40.380640+0000","last_undegraded":"2023-10-26T05:53:40.380640+0000","last_fullsized":"2023-10-26T05:53:40.380640+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:23:21.720096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.744013+0000","last_change":"2023-10-26T05:53:40.385225+0000","last_active":"2023-10-26T05:53:41.744013+0000","last_peered":"2023-10-26T05:53:41.744013+0000","last_clean":"2023-10-26T05:53:41.744013+0000","last_became_active":"2023-10-26T05:53:40.384639+0000","last_became_peered":"2023-10-26T05:53:40.384639+0000","last_unstale":"2023-10-26T05:53:41.744013+0000","last_undegraded":"2023-10-26T05:53:41.744013+0000","last_fullsized":"2023-10-26T05:53:41.744013+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:35.462737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"137'3","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.424660+0000","last_change":"2023-10-26T05:53:39.381571+0000","last_active":"2023-10-26T05:53:40.424660+0000","last_peered":"2023-10-26T05:53:40.424660+0000","last_clean":"2023-10-26T05:53:40.424660+0000","last_became_active":"2023-10-26T05:53:39.381177+0000","last_became_peered":"2023-10-26T05:53:39.381177+0000","last_unstale":"2023-10-26T05:53:40.424660+0000","last_undegraded":"2023-10-26T05:53:40.424660+0000","last_fullsized":"2023-10-26T05:53:40.424660+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:24:20.470510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.747801+0000","last_change":"2023-10-26T05:53:40.412472+0000","last_active":"2023-10-26T05:53:41.747801+0000","last_peered":"2023-10-26T05:53:41.747801+0000","last_clean":"2023-10-26T05:53:41.747801+0000","last_became_active":"2023-10-26T05:53:40.411877+0000","last_became_peered":"2023-10-26T05:53:40.411877+0000","last_unstale":"2023-10-26T05:53:41.747801+0000","last_undegraded":"2023-10-26T05:53:41.747801+0000","last_fullsized":"2023-10-26T05:53:41.747801+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:37:20.755604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"137'6","reported_seq":18,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.426051+0000","last_change":"2023-10-26T05:53:39.387472+0000","last_active":"2023-10-26T05:53:40.426051+0000","last_peered":"2023-10-26T05:53:40.426051+0000","last_clean":"2023-10-26T05:53:40.426051+0000","last_became_active":"2023-10-26T05:53:39.386601+0000","last_became_peered":"2023-10-26T05:53:39.386601+0000","last_unstale":"2023-10-26T05:53:40.426051+0000","last_undegraded":"2023-10-26T05:53:40.426051+0000","last_fullsized":"2023-10-26T05:53:40.426051+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:14:11.640342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743808+0000","last_change":"2023-10-26T05:53:40.400766+0000","last_active":"2023-10-26T05:53:41.743808+0000","last_peered":"2023-10-26T05:53:41.743808+0000","last_clean":"2023-10-26T05:53:41.743808+0000","last_became_active":"2023-10-26T05:53:40.399181+0000","last_became_peered":"2023-10-26T05:53:40.399181+0000","last_unstale":"2023-10-26T05:53:41.743808+0000","last_undegraded":"2023-10-26T05:53:41.743808+0000","last_fullsized":"2023-10-26T05:53:41.743808+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:28:07.894721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.423438+0000","last_change":"2023-10-26T05:53:39.392081+0000","last_active":"2023-10-26T05:53:40.423438+0000","last_peered":"2023-10-26T05:53:40.423438+0000","last_clean":"2023-10-26T05:53:40.423438+0000","last_became_active":"2023-10-26T05:53:39.391881+0000","last_became_peered":"2023-10-26T05:53:39.391881+0000","last_unstale":"2023-10-26T05:53:40.423438+0000","last_undegraded":"2023-10-26T05:53:40.423438+0000","last_fullsized":"2023-10-26T05:53:40.423438+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:56:52.766235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743379+0000","last_change":"2023-10-26T05:53:40.403711+0000","last_active":"2023-10-26T05:53:41.743379+0000","last_peered":"2023-10-26T05:53:41.743379+0000","last_clean":"2023-10-26T05:53:41.743379+0000","last_became_active":"2023-10-26T05:53:40.402379+0000","last_became_peered":"2023-10-26T05:53:40.402379+0000","last_unstale":"2023-10-26T05:53:41.743379+0000","last_undegraded":"2023-10-26T05:53:41.743379+0000","last_fullsized":"2023-10-26T05:53:41.743379+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:52:21.420193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.394164+0000","last_change":"2023-10-26T05:53:39.391669+0000","last_active":"2023-10-26T05:53:40.394164+0000","last_peered":"2023-10-26T05:53:40.394164+0000","last_clean":"2023-10-26T05:53:40.394164+0000","last_became_active":"2023-10-26T05:53:39.390032+0000","last_became_peered":"2023-10-26T05:53:39.390032+0000","last_unstale":"2023-10-26T05:53:40.394164+0000","last_undegraded":"2023-10-26T05:53:40.394164+0000","last_fullsized":"2023-10-26T05:53:40.394164+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:52.187609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743220+0000","last_change":"2023-10-26T05:53:40.398996+0000","last_active":"2023-10-26T05:53:41.743220+0000","last_peered":"2023-10-26T05:53:41.743220+0000","last_clean":"2023-10-26T05:53:41.743220+0000","last_became_active":"2023-10-26T05:53:40.396950+0000","last_became_peered":"2023-10-26T05:53:40.396950+0000","last_unstale":"2023-10-26T05:53:41.743220+0000","last_undegraded":"2023-10-26T05:53:41.743220+0000","last_fullsized":"2023-10-26T05:53:41.743220+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:50:04.805439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"137'8","reported_seq":20,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.423179+0000","last_change":"2023-10-26T05:53:39.382861+0000","last_active":"2023-10-26T05:53:40.423179+0000","last_peered":"2023-10-26T05:53:40.423179+0000","last_clean":"2023-10-26T05:53:40.423179+0000","last_became_active":"2023-10-26T05:53:39.382479+0000","last_became_peered":"2023-10-26T05:53:39.382479+0000","last_unstale":"2023-10-26T05:53:40.423179+0000","last_undegraded":"2023-10-26T05:53:40.423179+0000","last_fullsized":"2023-10-26T05:53:40.423179+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:19:43.008985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.380001+0000","last_change":"2023-10-26T05:53:40.401041+0000","last_active":"2023-10-26T05:53:41.380001+0000","last_peered":"2023-10-26T05:53:41.380001+0000","last_clean":"2023-10-26T05:53:41.380001+0000","last_became_active":"2023-10-26T05:53:40.400136+0000","last_became_peered":"2023-10-26T05:53:40.400136+0000","last_unstale":"2023-10-26T05:53:41.380001+0000","last_undegraded":"2023-10-26T05:53:41.380001+0000","last_fullsized":"2023-10-26T05:53:41.380001+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:16:40.877623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.420591+0000","last_change":"2023-10-26T05:53:39.390022+0000","last_active":"2023-10-26T05:53:40.420591+0000","last_peered":"2023-10-26T05:53:40.420591+0000","last_clean":"2023-10-26T05:53:40.420591+0000","last_became_active":"2023-10-26T05:53:39.389733+0000","last_became_peered":"2023-10-26T05:53:39.389733+0000","last_unstale":"2023-10-26T05:53:40.420591+0000","last_undegraded":"2023-10-26T05:53:40.420591+0000","last_fullsized":"2023-10-26T05:53:40.420591+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:43:55.019882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.376907+0000","last_change":"2023-10-26T05:53:40.397054+0000","last_active":"2023-10-26T05:53:41.376907+0000","last_peered":"2023-10-26T05:53:41.376907+0000","last_clean":"2023-10-26T05:53:41.376907+0000","last_became_active":"2023-10-26T05:53:40.395516+0000","last_became_peered":"2023-10-26T05:53:40.395516+0000","last_unstale":"2023-10-26T05:53:41.376907+0000","last_undegraded":"2023-10-26T05:53:41.376907+0000","last_fullsized":"2023-10-26T05:53:41.376907+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:50:02.725711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.419578+0000","last_change":"2023-10-26T05:53:39.397727+0000","last_active":"2023-10-26T05:53:40.419578+0000","last_peered":"2023-10-26T05:53:40.419578+0000","last_clean":"2023-10-26T05:53:40.419578+0000","last_became_active":"2023-10-26T05:53:39.397564+0000","last_became_peered":"2023-10-26T05:53:39.397564+0000","last_unstale":"2023-10-26T05:53:40.419578+0000","last_undegraded":"2023-10-26T05:53:40.419578+0000","last_fullsized":"2023-10-26T05:53:40.419578+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:24:18.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748813+0000","last_change":"2023-10-26T05:53:40.403277+0000","last_active":"2023-10-26T05:53:41.748813+0000","last_peered":"2023-10-26T05:53:41.748813+0000","last_clean":"2023-10-26T05:53:41.748813+0000","last_became_active":"2023-10-26T05:53:40.402557+0000","last_became_peered":"2023-10-26T05:53:40.402557+0000","last_unstale":"2023-10-26T05:53:41.748813+0000","last_undegraded":"2023-10-26T05:53:41.748813+0000","last_fullsized":"2023-10-26T05:53:41.748813+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:15.447207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"137'1","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.428785+0000","last_change":"2023-10-26T05:53:39.393375+0000","last_active":"2023-10-26T05:53:40.428785+0000","last_peered":"2023-10-26T05:53:40.428785+0000","last_clean":"2023-10-26T05:53:40.428785+0000","last_became_active":"2023-10-26T05:53:39.392683+0000","last_became_peered":"2023-10-26T05:53:39.392683+0000","last_unstale":"2023-10-26T05:53:40.428785+0000","last_undegraded":"2023-10-26T05:53:40.428785+0000","last_fullsized":"2023-10-26T05:53:40.428785+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:56.465903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.750063+0000","last_change":"2023-10-26T05:53:40.403599+0000","last_active":"2023-10-26T05:53:41.750063+0000","last_peered":"2023-10-26T05:53:41.750063+0000","last_clean":"2023-10-26T05:53:41.750063+0000","last_became_active":"2023-10-26T05:53:40.402404+0000","last_became_peered":"2023-10-26T05:53:40.402404+0000","last_unstale":"2023-10-26T05:53:41.750063+0000","last_undegraded":"2023-10-26T05:53:41.750063+0000","last_fullsized":"2023-10-26T05:53:41.750063+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:39.886243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.404317+0000","last_change":"2023-10-26T05:53:39.407074+0000","last_active":"2023-10-26T05:53:40.404317+0000","last_peered":"2023-10-26T05:53:40.404317+0000","last_clean":"2023-10-26T05:53:40.404317+0000","last_became_active":"2023-10-26T05:53:39.403999+0000","last_became_peered":"2023-10-26T05:53:39.403999+0000","last_unstale":"2023-10-26T05:53:40.404317+0000","last_undegraded":"2023-10-26T05:53:40.404317+0000","last_fullsized":"2023-10-26T05:53:40.404317+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:46:29.124863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.749690+0000","last_change":"2023-10-26T05:53:40.420943+0000","last_active":"2023-10-26T05:53:41.749690+0000","last_peered":"2023-10-26T05:53:41.749690+0000","last_clean":"2023-10-26T05:53:41.749690+0000","last_became_active":"2023-10-26T05:53:40.419874+0000","last_became_peered":"2023-10-26T05:53:40.419874+0000","last_unstale":"2023-10-26T05:53:41.749690+0000","last_undegraded":"2023-10-26T05:53:41.749690+0000","last_fullsized":"2023-10-26T05:53:41.749690+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:11:21.892097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.422825+0000","last_change":"2023-10-26T05:53:39.376685+0000","last_active":"2023-10-26T05:53:40.422825+0000","last_peered":"2023-10-26T05:53:40.422825+0000","last_clean":"2023-10-26T05:53:40.422825+0000","last_became_active":"2023-10-26T05:53:39.376380+0000","last_became_peered":"2023-10-26T05:53:39.376380+0000","last_unstale":"2023-10-26T05:53:40.422825+0000","last_undegraded":"2023-10-26T05:53:40.422825+0000","last_fullsized":"2023-10-26T05:53:40.422825+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:24:18.090840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743697+0000","last_change":"2023-10-26T05:53:40.410230+0000","last_active":"2023-10-26T05:53:41.743697+0000","last_peered":"2023-10-26T05:53:41.743697+0000","last_clean":"2023-10-26T05:53:41.743697+0000","last_became_active":"2023-10-26T05:53:40.409877+0000","last_became_peered":"2023-10-26T05:53:40.409877+0000","last_unstale":"2023-10-26T05:53:41.743697+0000","last_undegraded":"2023-10-26T05:53:41.743697+0000","last_fullsized":"2023-10-26T05:53:41.743697+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:47:56.320274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.386970+0000","last_change":"2023-10-26T05:53:39.379454+0000","last_active":"2023-10-26T05:53:40.386970+0000","last_peered":"2023-10-26T05:53:40.386970+0000","last_clean":"2023-10-26T05:53:40.386970+0000","last_became_active":"2023-10-26T05:53:39.379184+0000","last_became_peered":"2023-10-26T05:53:39.379184+0000","last_unstale":"2023-10-26T05:53:40.386970+0000","last_undegraded":"2023-10-26T05:53:40.386970+0000","last_fullsized":"2023-10-26T05:53:40.386970+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:54.923322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.749732+0000","last_change":"2023-10-26T05:53:40.403206+0000","last_active":"2023-10-26T05:53:41.749732+0000","last_peered":"2023-10-26T05:53:41.749732+0000","last_clean":"2023-10-26T05:53:41.749732+0000","last_became_active":"2023-10-26T05:53:40.401737+0000","last_became_peered":"2023-10-26T05:53:40.401737+0000","last_unstale":"2023-10-26T05:53:41.749732+0000","last_undegraded":"2023-10-26T05:53:41.749732+0000","last_fullsized":"2023-10-26T05:53:41.749732+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:27.446546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.377192+0000","last_change":"2023-10-26T05:53:39.379041+0000","last_active":"2023-10-26T05:53:40.377192+0000","last_peered":"2023-10-26T05:53:40.377192+0000","last_clean":"2023-10-26T05:53:40.377192+0000","last_became_active":"2023-10-26T05:53:39.378384+0000","last_became_peered":"2023-10-26T05:53:39.378384+0000","last_unstale":"2023-10-26T05:53:40.377192+0000","last_undegraded":"2023-10-26T05:53:40.377192+0000","last_fullsized":"2023-10-26T05:53:40.377192+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:52:40.450471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743491+0000","last_change":"2023-10-26T05:53:40.403227+0000","last_active":"2023-10-26T05:53:41.743491+0000","last_peered":"2023-10-26T05:53:41.743491+0000","last_clean":"2023-10-26T05:53:41.743491+0000","last_became_active":"2023-10-26T05:53:40.399238+0000","last_became_peered":"2023-10-26T05:53:40.399238+0000","last_unstale":"2023-10-26T05:53:41.743491+0000","last_undegraded":"2023-10-26T05:53:41.743491+0000","last_fullsized":"2023-10-26T05:53:41.743491+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:56.921925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.390915+0000","last_change":"2023-10-26T05:53:39.387235+0000","last_active":"2023-10-26T05:53:40.390915+0000","last_peered":"2023-10-26T05:53:40.390915+0000","last_clean":"2023-10-26T05:53:40.390915+0000","last_became_active":"2023-10-26T05:53:39.386945+0000","last_became_peered":"2023-10-26T05:53:39.386945+0000","last_unstale":"2023-10-26T05:53:40.390915+0000","last_undegraded":"2023-10-26T05:53:40.390915+0000","last_fullsized":"2023-10-26T05:53:40.390915+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:08:47.573389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748918+0000","last_change":"2023-10-26T05:53:40.414561+0000","last_active":"2023-10-26T05:53:41.748918+0000","last_peered":"2023-10-26T05:53:41.748918+0000","last_clean":"2023-10-26T05:53:41.748918+0000","last_became_active":"2023-10-26T05:53:40.410903+0000","last_became_peered":"2023-10-26T05:53:40.410903+0000","last_unstale":"2023-10-26T05:53:41.748918+0000","last_undegraded":"2023-10-26T05:53:41.748918+0000","last_fullsized":"2023-10-26T05:53:41.748918+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:35:49.689256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"137'2","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.420313+0000","last_change":"2023-10-26T05:53:39.387929+0000","last_active":"2023-10-26T05:53:40.420313+0000","last_peered":"2023-10-26T05:53:40.420313+0000","last_clean":"2023-10-26T05:53:40.420313+0000","last_became_active":"2023-10-26T05:53:39.387571+0000","last_became_peered":"2023-10-26T05:53:39.387571+0000","last_unstale":"2023-10-26T05:53:40.420313+0000","last_undegraded":"2023-10-26T05:53:40.420313+0000","last_fullsized":"2023-10-26T05:53:40.420313+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:42.563068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.742980+0000","last_change":"2023-10-26T05:53:40.410073+0000","last_active":"2023-10-26T05:53:41.742980+0000","last_peered":"2023-10-26T05:53:41.742980+0000","last_clean":"2023-10-26T05:53:41.742980+0000","last_became_active":"2023-10-26T05:53:40.408053+0000","last_became_peered":"2023-10-26T05:53:40.408053+0000","last_unstale":"2023-10-26T05:53:41.742980+0000","last_undegraded":"2023-10-26T05:53:41.742980+0000","last_fullsized":"2023-10-26T05:53:41.742980+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:22:07.591178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.399942+0000","last_change":"2023-10-26T05:53:39.402866+0000","last_active":"2023-10-26T05:53:40.399942+0000","last_peered":"2023-10-26T05:53:40.399942+0000","last_clean":"2023-10-26T05:53:40.399942+0000","last_became_active":"2023-10-26T05:53:39.402076+0000","last_became_peered":"2023-10-26T05:53:39.402076+0000","last_unstale":"2023-10-26T05:53:40.399942+0000","last_undegraded":"2023-10-26T05:53:40.399942+0000","last_fullsized":"2023-10-26T05:53:40.399942+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:52:54.586135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743863+0000","last_change":"2023-10-26T05:53:40.409720+0000","last_active":"2023-10-26T05:53:41.743863+0000","last_peered":"2023-10-26T05:53:41.743863+0000","last_clean":"2023-10-26T05:53:41.743863+0000","last_became_active":"2023-10-26T05:53:40.405833+0000","last_became_peered":"2023-10-26T05:53:40.405833+0000","last_unstale":"2023-10-26T05:53:41.743863+0000","last_undegraded":"2023-10-26T05:53:41.743863+0000","last_fullsized":"2023-10-26T05:53:41.743863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:10:42.465277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.391596+0000","last_change":"2023-10-26T05:53:39.387605+0000","last_active":"2023-10-26T05:53:40.391596+0000","last_peered":"2023-10-26T05:53:40.391596+0000","last_clean":"2023-10-26T05:53:40.391596+0000","last_became_active":"2023-10-26T05:53:39.387254+0000","last_became_peered":"2023-10-26T05:53:39.387254+0000","last_unstale":"2023-10-26T05:53:40.391596+0000","last_undegraded":"2023-10-26T05:53:40.391596+0000","last_fullsized":"2023-10-26T05:53:40.391596+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:33:32.825291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.748156+0000","last_change":"2023-10-26T05:53:40.400201+0000","last_active":"2023-10-26T05:53:41.748156+0000","last_peered":"2023-10-26T05:53:41.748156+0000","last_clean":"2023-10-26T05:53:41.748156+0000","last_became_active":"2023-10-26T05:53:40.398631+0000","last_became_peered":"2023-10-26T05:53:40.398631+0000","last_unstale":"2023-10-26T05:53:41.748156+0000","last_undegraded":"2023-10-26T05:53:41.748156+0000","last_fullsized":"2023-10-26T05:53:41.748156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:22:42.468545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.380351+0000","last_change":"2023-10-26T05:53:39.388401+0000","last_active":"2023-10-26T05:53:40.380351+0000","last_peered":"2023-10-26T05:53:40.380351+0000","last_clean":"2023-10-26T05:53:40.380351+0000","last_became_active":"2023-10-26T05:53:39.386956+0000","last_became_peered":"2023-10-26T05:53:39.386956+0000","last_unstale":"2023-10-26T05:53:40.380351+0000","last_undegraded":"2023-10-26T05:53:40.380351+0000","last_fullsized":"2023-10-26T05:53:40.380351+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:49.789827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.385698+0000","last_change":"2023-10-26T05:53:40.412627+0000","last_active":"2023-10-26T05:53:41.385698+0000","last_peered":"2023-10-26T05:53:41.385698+0000","last_clean":"2023-10-26T05:53:41.385698+0000","last_became_active":"2023-10-26T05:53:40.411927+0000","last_became_peered":"2023-10-26T05:53:40.411927+0000","last_unstale":"2023-10-26T05:53:41.385698+0000","last_undegraded":"2023-10-26T05:53:41.385698+0000","last_fullsized":"2023-10-26T05:53:41.385698+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:03.451684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.391313+0000","last_change":"2023-10-26T05:53:39.406613+0000","last_active":"2023-10-26T05:53:40.391313+0000","last_peered":"2023-10-26T05:53:40.391313+0000","last_clean":"2023-10-26T05:53:40.391313+0000","last_became_active":"2023-10-26T05:53:39.406173+0000","last_became_peered":"2023-10-26T05:53:39.406173+0000","last_unstale":"2023-10-26T05:53:40.391313+0000","last_undegraded":"2023-10-26T05:53:40.391313+0000","last_fullsized":"2023-10-26T05:53:40.391313+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:05.604550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.385249+0000","last_change":"2023-10-26T05:53:40.399751+0000","last_active":"2023-10-26T05:53:41.385249+0000","last_peered":"2023-10-26T05:53:41.385249+0000","last_clean":"2023-10-26T05:53:41.385249+0000","last_became_active":"2023-10-26T05:53:40.399070+0000","last_became_peered":"2023-10-26T05:53:40.399070+0000","last_unstale":"2023-10-26T05:53:41.385249+0000","last_undegraded":"2023-10-26T05:53:41.385249+0000","last_fullsized":"2023-10-26T05:53:41.385249+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:52:55.600940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.385314+0000","last_change":"2023-10-26T05:53:39.399323+0000","last_active":"2023-10-26T05:53:40.385314+0000","last_peered":"2023-10-26T05:53:40.385314+0000","last_clean":"2023-10-26T05:53:40.385314+0000","last_became_active":"2023-10-26T05:53:39.398652+0000","last_became_peered":"2023-10-26T05:53:39.398652+0000","last_unstale":"2023-10-26T05:53:40.385314+0000","last_undegraded":"2023-10-26T05:53:40.385314+0000","last_fullsized":"2023-10-26T05:53:40.385314+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:35:34.447932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.749363+0000","last_change":"2023-10-26T05:53:40.416005+0000","last_active":"2023-10-26T05:53:41.749363+0000","last_peered":"2023-10-26T05:53:41.749363+0000","last_clean":"2023-10-26T05:53:41.749363+0000","last_became_active":"2023-10-26T05:53:40.407793+0000","last_became_peered":"2023-10-26T05:53:40.407793+0000","last_unstale":"2023-10-26T05:53:41.749363+0000","last_undegraded":"2023-10-26T05:53:41.749363+0000","last_fullsized":"2023-10-26T05:53:41.749363+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:57.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.386389+0000","last_change":"2023-10-26T05:53:39.376606+0000","last_active":"2023-10-26T05:53:40.386389+0000","last_peered":"2023-10-26T05:53:40.386389+0000","last_clean":"2023-10-26T05:53:40.386389+0000","last_became_active":"2023-10-26T05:53:39.376273+0000","last_became_peered":"2023-10-26T05:53:39.376273+0000","last_unstale":"2023-10-26T05:53:40.386389+0000","last_undegraded":"2023-10-26T05:53:40.386389+0000","last_fullsized":"2023-10-26T05:53:40.386389+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:41:28.923900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743371+0000","last_change":"2023-10-26T05:53:40.405649+0000","last_active":"2023-10-26T05:53:41.743371+0000","last_peered":"2023-10-26T05:53:41.743371+0000","last_clean":"2023-10-26T05:53:41.743371+0000","last_became_active":"2023-10-26T05:53:40.399963+0000","last_became_peered":"2023-10-26T05:53:40.399963+0000","last_unstale":"2023-10-26T05:53:41.743371+0000","last_undegraded":"2023-10-26T05:53:41.743371+0000","last_fullsized":"2023-10-26T05:53:41.743371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:11:26.219719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.391930+0000","last_change":"2023-10-26T05:53:39.386889+0000","last_active":"2023-10-26T05:53:40.391930+0000","last_peered":"2023-10-26T05:53:40.391930+0000","last_clean":"2023-10-26T05:53:40.391930+0000","last_became_active":"2023-10-26T05:53:39.386241+0000","last_became_peered":"2023-10-26T05:53:39.386241+0000","last_unstale":"2023-10-26T05:53:40.391930+0000","last_undegraded":"2023-10-26T05:53:40.391930+0000","last_fullsized":"2023-10-26T05:53:40.391930+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:08:59.451124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-26T05:53:41.743873+0000","last_change":"2023-10-26T05:53:40.407163+0000","last_active":"2023-10-26T05:53:41.743873+0000","last_peered":"2023-10-26T05:53:41.743873+0000","last_clean":"2023-10-26T05:53:41.743873+0000","last_became_active":"2023-10-26T05:53:40.406901+0000","last_became_peered":"2023-10-26T05:53:40.406901+0000","last_unstale":"2023-10-26T05:53:41.743873+0000","last_undegraded":"2023-10-26T05:53:41.743873+0000","last_fullsized":"2023-10-26T05:53:41.743873+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:39.355381+0000","last_clean_scrub_stamp":"2023-10-26T05:53:39.355381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:56:30.040699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.403995+0000","last_change":"2023-10-26T05:53:39.407411+0000","last_active":"2023-10-26T05:53:40.403995+0000","last_peered":"2023-10-26T05:53:40.403995+0000","last_clean":"2023-10-26T05:53:40.403995+0000","last_became_active":"2023-10-26T05:53:39.404908+0000","last_became_peered":"2023-10-26T05:53:39.404908+0000","last_unstale":"2023-10-26T05:53:40.403995+0000","last_undegraded":"2023-10-26T05:53:40.403995+0000","last_fullsized":"2023-10-26T05:53:40.403995+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:53:38.340576+0000","last_clean_scrub_stamp":"2023-10-26T05:53:38.340576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:50:02.357965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":276,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-26T05:53:40.404296+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:53:40.404296+0000","last_peered":"2023-10-26T05:53:40.404296+0000","last_clean":"2023-10-26T05:53:40.404296+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:53:40.404296+0000","last_undegraded":"2023-10-26T05:53:40.404296+0000","last_fullsized":"2023-10-26T05:53:40.404296+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1324,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193987,"statfs":{"total":200000000,"available":198643584,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.81699999999999995,"15min":0.85299999999999998},"min":{"1min":0.501,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0840000000000001,"5min":1.3100000000000001,"15min":1.347},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.83299999999999996},"min":{"1min":0.44600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1140000000000001,"5min":1.286,"15min":1.3100000000000001},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93600000000000005,"15min":1.024},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.149,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.96699999999999997,"15min":0.97299999999999998},"min":{"1min":0.60299999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.24,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.621}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.84599999999999997,"15min":0.81999999999999995},"min":{"1min":0.48799999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.216,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.91100000000000003,"15min":0.93400000000000005},"min":{"1min":0.51300000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.167,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0509999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.879,"15min":0.88600000000000001},"min":{"1min":0.55600000000000005,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.4330000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.96699999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1080000000000001,"5min":1.375,"15min":1.375},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.91400000000000003,"15min":0.91900000000000004},"min":{"1min":0.61399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.2470000000000001,"5min":1.639,"15min":1.639},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95899999999999996,"15min":0.94199999999999995},"min":{"1min":0.68200000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.145,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.024,"15min":1.1100000000000001},"min":{"1min":0.70099999999999996,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2729999999999999,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":1.145},{"interface":"front","average":{"1min":0.999,"5min":1.0169999999999999,"15min":1.079},"min":{"1min":0.76300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.25,"5min":1.702,"15min":1.702},"last":0.877}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0820000000000001,"15min":1.1120000000000001},"min":{"1min":0.91400000000000003,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.391,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.048,"15min":1.048},"min":{"1min":0.65400000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.28,"5min":1.5469999999999999,"15min":1.6180000000000001},"last":1.02}]}]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194564,"statfs":{"total":200000000,"available":199234498,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83999999999999997,"15min":0.81299999999999994},"min":{"1min":0.51200000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.85899999999999999,"15min":0.85699999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.206,"5min":1.298,"15min":1.3},"last":0.879}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.77800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1779999999999999,"15min":1.2609999999999999},"last":0.623},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.89000000000000001,"15min":0.83999999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.167,"5min":1.405,"15min":1.405},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90100000000000002,"15min":0.81499999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1599999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89500000000000002,"15min":0.84399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.337,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.89400000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2629999999999999,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.054,"15min":0.97199999999999998},"min":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.502},"max":{"1min":1.2869999999999999,"5min":1.399,"15min":1.399},"last":1.0620000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93400000000000005,"15min":0.95599999999999996},"min":{"1min":0.57599999999999996,"5min":0.502,"15min":0.502},"max":{"1min":1.407,"5min":1.429,"15min":1.429},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.98999999999999999,"15min":0.97599999999999998},"min":{"1min":0.63900000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":1.137}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.024,"15min":1.044},"min":{"1min":0.59999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.3280000000000001,"5min":1.3480000000000001,"15min":1.554},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.024,"5min":0.95199999999999996,"15min":0.99199999999999999},"min":{"1min":0.53300000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.099}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.107,"15min":1.1020000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4219999999999999,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.097,"15min":1.1160000000000001},"min":{"1min":0.49399999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.171}]}]},{"osd":5,"up_from":13,"seq":55834574929,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":748,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194564,"statfs":{"total":200000000,"available":199233964,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93700000000000006,"15min":0.95499999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.278,"5min":1.579,"15min":1.579},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.272,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84799999999999998,"15min":0.89100000000000001},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85499999999999998},"min":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.242,"5min":1.464,"15min":1.464},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.85399999999999998,"15min":0.91400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1679999999999999,"5min":1.4219999999999999,"15min":1.5089999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.92400000000000004,"15min":0.93300000000000005},"min":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.274,"5min":1.274,"15min":1.3160000000000001},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90200000000000002,"15min":0.91800000000000004},"min":{"1min":0.69399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.448,"5min":1.448,"15min":1.548},"last":0.753},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.95699999999999996,"15min":0.998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.288,"5min":1.4139999999999999,"15min":1.587},"last":1.032}]},{"osd":4,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.93600000000000005,"15min":0.999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.286,"5min":1.286,"15min":2.1909999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.014,"5min":1.0489999999999999,"15min":1.097},"min":{"1min":0.51700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.087}]},{"osd":6,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93400000000000005,"15min":1.018},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.321,"5min":1.393,"15min":1.5780000000000001},"last":1.29},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.99399999999999999,"15min":1.0289999999999999},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.367,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.97999999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:53:17 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.0409999999999999,"15min":1.0720000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.357,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.093,"15min":1.159},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.6830000000000001},"last":1.393}]}]},{"osd":4,"up_from":13,"seq":55834574929,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194565,"statfs":{"total":200000000,"available":199234568,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.85399999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1899999999999999,"5min":1.347,"15min":1.347},"last":1.046},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.879,"15min":0.88200000000000001},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2909999999999999,"5min":1.502,"15min":1.502},"last":0.94799999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.86299999999999999,"15min":0.81699999999999995},"min":{"1min":0.626,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.097,"5min":1.25,"15min":1.375},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.94299999999999995,"15min":0.89800000000000002},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.3440000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.91400000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.872,"15min":0.81799999999999995},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.238},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.86799999999999999},"min":{"1min":0.63200000000000001,"5min":0.55500000000000005,"15min":0.49199999999999999},"max":{"1min":1.2170000000000001,"5min":1.373,"15min":1.373},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92500000000000004,"15min":0.94199999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.2709999999999999,"5min":1.341,"15min":2.0190000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.97299999999999998,"15min":0.94299999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2350000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.90200000000000002,"15min":0.88600000000000001},"min":{"1min":0.56799999999999995,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.87},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.61399999999999999,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.673,"15min":1.673},"last":1.323}]},{"osd":6,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.012,"15min":1},"min":{"1min":0.65200000000000002,"5min":0.503,"15min":0.42099999999999999},"max":{"1min":1.3680000000000001,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.206},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.028,"15min":1.036},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3360000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.1719999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:53:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.054,"15min":1.038},"min":{"1min":0.60799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.292,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.119,"15min":1.115},"min":{"1min":0.748,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4590000000000001,"5min":1.728,"15min":1.728},"last":1.4019999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574929,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":749,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194563,"statfs":{"total":200000000,"available":199232886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.59899999999999998,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.84299999999999997,"5min":1.385,"15min":1.385},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.58499999999999996},"min":{"1min":0.35399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.2689999999999999},"last":0.33800000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.627,"15min":0.65100000000000002},"min":{"1min":0.46800000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.85099999999999998,"5min":1.349,"15min":1.349},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.60499999999999998},"min":{"1min":0.41199999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.496}]},{"osd":2,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.443,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2170000000000001,"5min":1.367,"15min":1.367},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.81000000000000005,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.0389999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88700000000000001,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.57399999999999995,"15min":0.40000000000000002},"max":{"1min":1.093,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.91300000000000003,"15min":0.92200000000000004},"min":{"1min":0.60899999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1719999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.748,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.413,"15min":1.478},"last":0.622},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94799999999999995,"15min":0.90900000000000003},"min":{"1min":0.63900000000000001,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1830000000000001,"5min":1.742,"15min":1.742},"last":0.78700000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93200000000000005,"15min":0.99199999999999999},"min":{"1min":0.73399999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.266,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0249999999999999},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.196,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:53:21 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98499999999999999,"15min":0.92900000000000005},"min":{"1min":0.80000000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.216,"5min":1.661,"15min":1.661},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0189999999999999,"15min":1.0389999999999999},"min":{"1min":0.68600000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.24,"5min":2.004,"15min":2.004},"last":0.73099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194565,"statfs":{"total":200000000,"available":199234568,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60199999999999998,"15min":0.54000000000000004},"min":{"1min":0.39800000000000002,"5min":0.317,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.313},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71299999999999997,"15min":0.60599999999999998},"min":{"1min":0.34100000000000003,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.379,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73199999999999998,"15min":0.67000000000000004},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":1.1899999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.68700000000000006,"15min":0.59899999999999998},"min":{"1min":0.29299999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.96699999999999997,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.36799999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.82099999999999995,"15min":0.73699999999999999},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.28299999999999997},"max":{"1min":1.1180000000000001,"5min":2.8809999999999998,"15min":2.8809999999999998},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69499999999999995,"15min":0.60099999999999998},"min":{"1min":0.313,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.022,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93799999999999994,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.53200000000000003,"15min":0.49399999999999999},"max":{"1min":1.5189999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.92500000000000004,"15min":0.84499999999999997},"min":{"1min":0.64100000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4870000000000001,"5min":5.9550000000000001,"15min":5.9550000000000001},"last":0.626}]},{"osd":5,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1,"15min":0.91400000000000003},"min":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.55200000000000005},"max":{"1min":1.5600000000000001,"5min":2.8319999999999999,"15min":2.8319999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.028,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.58199999999999996,"15min":0.51700000000000002},"max":{"1min":1.4790000000000001,"5min":1.653,"15min":1.653},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0209999999999999,"15min":0.92600000000000005},"min":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.54000000000000004},"max":{"1min":1.6000000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1339999999999999,"15min":0.98499999999999999},"min":{"1min":0.76000000000000001,"5min":0.56599999999999995,"15min":0.5},"max":{"1min":1.7769999999999999,"5min":6.7729999999999997,"15min":6.7729999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:53:19 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.97499999999999998,"15min":0.90800000000000003},"min":{"1min":0.746,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.51500000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.2410000000000001,"15min":1.04},"min":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.51800000000000002},"max":{"1min":1.823,"5min":15.352,"15min":15.352},"last":0.752}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194564,"statfs":{"total":200000000,"available":199234444,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56999999999999995,"15min":0.56599999999999995},"min":{"1min":0.315,"5min":0.29099999999999998,"15min":0.254},"max":{"1min":0.98799999999999999,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.629,"15min":0.623},"min":{"1min":0.34200000000000003,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":1.038,"5min":1.157,"15min":1.157},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66200000000000003,"15min":0.65000000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95399999999999996,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.72199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.1160000000000001,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.997}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.72199999999999998,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.137,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70299999999999996,"15min":0.68300000000000005},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.27700000000000002},"max":{"1min":1.175,"5min":1.298,"15min":1.298},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.88200000000000001,"15min":0.86699999999999999},"min":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.113,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.97499999999999998,"15min":0.94699999999999995},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2290000000000001,"5min":3.7440000000000002,"15min":3.7440000000000002},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95799999999999996,"15min":0.96299999999999997},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.23,"5min":1.492,"15min":1.492},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.93300000000000005},"min":{"1min":0.63100000000000001,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.071,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":1.012,"15min":1.0349999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.258,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.073},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.159,"5min":1.468,"15min":1.837},"last":1.1850000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.0049999999999999,"15min":1.026},"min":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.321,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":1.05},{"interface":"front","average":{"1min":1.038,"5min":1.0840000000000001,"15min":1.0900000000000001},"min":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.319,"5min":5.2430000000000003,"15min":5.2430000000000003},"last":1.0229999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1325,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193986,"statfs":{"total":200000000,"available":198642476,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.54300000000000004},"min":{"1min":0.33700000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.70199999999999996,"15min":0.62},"min":{"1min":0.318,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.498}]},{"osd":2,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.79100000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.67400000000000004,"15min":0.61299999999999999},"min":{"1min":0.34200000000000003,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.095,"5min":1.1140000000000001,"15min":1.177},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.66400000000000003,"15min":0.626},"min":{"1min":0.39300000000000002,"5min":0.318,"15min":0.26800000000000002},"max":{"1min":1.3520000000000001,"5min":1.409,"15min":1.409},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.79400000000000004,"15min":0.70599999999999996},"min":{"1min":0.36899999999999999,"5min":0.247,"15min":0.247},"max":{"1min":2.6509999999999998,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91300000000000003,"15min":0.85599999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.46100000000000002},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.629},{"interface":"front","average":{"1min":1.179,"5min":0.98999999999999999,"15min":0.91600000000000004},"min":{"1min":0.70799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.92800000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.93200000000000005,"15min":0.92200000000000004},"min":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.234,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.94599999999999995,"15min":0.91000000000000003},"min":{"1min":0.59199999999999997,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.873}]},{"osd":6,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.94399999999999995,"15min":0.90800000000000003},"min":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.59799999999999998},"max":{"1min":1.149,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.097},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.1279999999999999,"15min":1.002},"min":{"1min":0.82499999999999996,"5min":0.60499999999999998,"15min":0.54800000000000004},"max":{"1min":6.4690000000000003,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.95599999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:53:22 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1,"15min":0.997},"min":{"1min":0.65000000000000002,"5min":0.62,"15min":0.61499999999999999},"max":{"1min":1.214,"5min":1.669,"15min":1.6990000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.03,"15min":0.98899999999999999},"min":{"1min":0.80600000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4790000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.80600000000000005}]}]}],"pool_statfs":[]}} 2023-10-26T05:53:48.238 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:53:48.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:53:48.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 -- 172.21.15.71:0/1583737947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 msgr2=0x7fca4013e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 --2- 172.21.15.71:0/1583737947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca4013e850 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fca4006c780 tx=0x7fca2801c7b0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 -- 172.21.15.71:0/1583737947 shutdown_connections 2023-10-26T05:53:48.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 --2- 172.21.15.71:0/1583737947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca4013e850 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 --2- 172.21.15.71:0/1583737947 >> v2:172.21.15.86:3300/0 conn(0x7fca4013c120 0x7fca400cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 -- 172.21.15.71:0/1583737947 >> 172.21.15.71:0/1583737947 conn(0x7fca400aec60 msgr2=0x7fca400af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 -- 172.21.15.71:0/1583737947 shutdown_connections 2023-10-26T05:53:48.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.395+0000 7fca45e6d640 1 -- 172.21.15.71:0/1583737947 wait complete. 2023-10-26T05:53:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 Processor -- start 2023-10-26T05:53:48.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- start start 2023-10-26T05:53:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca40132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4013c120 0x7fca4006fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca4015a3f0 0x7fca40134880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4013c9e0 con 0x7fca400cbe00 2023-10-26T05:53:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca40135840 con 0x7fca4013c120 2023-10-26T05:53:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca40135b20 con 0x7fca4015a3f0 2023-10-26T05:53:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca40132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca40132c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34522/0 (socket says 172.21.15.71:34522) 2023-10-26T05:53:48.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 -- 172.21.15.71:0/390420095 learned_addr learned my addr 172.21.15.71:0/390420095 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3ffff640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca4015a3f0 0x7fca40134880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3effd640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4013c120 0x7fca4006fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 -- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca4015a3f0 msgr2=0x7fca40134880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca4015a3f0 0x7fca40134880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 -- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4013c120 msgr2=0x7fca4006fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4013c120 0x7fca4006fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 -- 172.21.15.71:0/390420095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280094a0 con 0x7fca400cbe00 2023-10-26T05:53:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca3f7fe640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca40132c40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fca3000d950 tx=0x7fca3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30009c30 con 0x7fca400cbe00 2023-10-26T05:53:48.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fca30009e80 con 0x7fca400cbe00 2023-10-26T05:53:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca40135e00 con 0x7fca400cbe00 2023-10-26T05:53:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300106c0 con 0x7fca400cbe00 2023-10-26T05:53:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca40070370 con 0x7fca400cbe00 2023-10-26T05:53:48.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca40066b10 con 0x7fca400cbe00 2023-10-26T05:53:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca30010820 con 0x7fca400cbe00 2023-10-26T05:53:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.399+0000 7fca37fff640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca20066470 0x7fca20068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.403+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca30084340 con 0x7fca400cbe00 2023-10-26T05:53:48.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.403+0000 7fca3effd640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca20066470 0x7fca20068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.403+0000 7fca3effd640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca20066470 0x7fca20068930 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca280af000 tx=0x7fca2801aff0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:48.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.407+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fca300517c0 con 0x7fca400cbe00 2023-10-26T05:53:48.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.615+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fca400705c0 con 0x7fca400cbe00 2023-10-26T05:53:48.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca37fff640 1 -- 172.21.15.71:0/390420095 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) v1 ==== 190+0+147 (secure 0 0 0) 0x7fca30054de0 con 0x7fca400cbe00 2023-10-26T05:53:48.627 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:53:48.627 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:53:48.627 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:53:48.628 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:53:48.628 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:53:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:53:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca20066470 msgr2=0x7fca20068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca20066470 0x7fca20068930 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca280af000 tx=0x7fca2801aff0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 msgr2=0x7fca40132c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca40132c40 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fca3000d950 tx=0x7fca3000de10 comp rx=0 tx=0).stop 2023-10-26T05:53:48.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 shutdown_connections 2023-10-26T05:53:48.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca4015a3f0 0x7fca40134880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca4013c120 0x7fca4006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fca20066470 0x7fca20068930 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 --2- 172.21.15.71:0/390420095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca400cbe00 0x7fca40132c40 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 >> 172.21.15.71:0/390420095 conn(0x7fca400aec60 msgr2=0x7fca400cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:48.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 shutdown_connections 2023-10-26T05:53:48.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.619+0000 7fca45e6d640 1 -- 172.21.15.71:0/390420095 wait complete. 2023-10-26T05:53:48.645 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:48.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/2556965150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec06b010 msgr2=0x7fe7ec06b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/2556965150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec06b010 0x7fe7ec06b4a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0009930 tx=0x7fe7e001c7f0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/2556965150 shutdown_connections 2023-10-26T05:53:48.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/2556965150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ec1595a0 0x7fe7ec159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/2556965150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec06b010 0x7fe7ec06b4a0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/2556965150 >> 172.21.15.71:0/2556965150 conn(0x7fe7ec070170 msgr2=0x7fe7ec070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:48.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/2556965150 shutdown_connections 2023-10-26T05:53:48.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/2556965150 wait complete. 2023-10-26T05:53:48.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 Processor -- start 2023-10-26T05:53:48.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- start start 2023-10-26T05:53:48.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ec06b010 0x7fe7ec0b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec1591c0 0x7fe7ec144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 0x7fe7ec157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b0c70 con 0x7fe7ec1595a0 2023-10-26T05:53:48.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b0660 con 0x7fe7ec1591c0 2023-10-26T05:53:48.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec142b90 con 0x7fe7ec06b010 2023-10-26T05:53:48.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 0x7fe7ec157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 0x7fe7ec157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34536/0 (socket says 172.21.15.71:34536) 2023-10-26T05:53:48.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 -- 172.21.15.71:0/3277900093 learned_addr learned my addr 172.21.15.71:0/3277900093 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:48.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f0b7c640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ec06b010 0x7fe7ec0b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7ebfff640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec1591c0 0x7fe7ec144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 -- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ec06b010 msgr2=0x7fe7ec0b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ec06b010 0x7fe7ec0b2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 -- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec1591c0 msgr2=0x7fe7ec144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec1591c0 0x7fe7ec144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 -- 172.21.15.71:0/3277900093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e0009560 con 0x7fe7ec1595a0 2023-10-26T05:53:48.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f137d640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 0x7fe7ec157f70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ca60 tx=0x7fe7e400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:48.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4007d90 con 0x7fe7ec1595a0 2023-10-26T05:53:48.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe7e4004b30 con 0x7fe7ec1595a0 2023-10-26T05:53:48.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec142e70 con 0x7fe7ec1595a0 2023-10-26T05:53:48.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4005020 con 0x7fe7ec1595a0 2023-10-26T05:53:48.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.759+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec1433d0 con 0x7fe7ec1595a0 2023-10-26T05:53:48.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.763+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec066d00 con 0x7fe7ec1595a0 2023-10-26T05:53:48.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.763+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7e4005180 con 0x7fe7ec1595a0 2023-10-26T05:53:48.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.763+0000 7fe7e97fa640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7d80664a0 0x7fe7d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:48.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.763+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7e4084650 con 0x7fe7ec1595a0 2023-10-26T05:53:48.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.763+0000 7fe7f0b7c640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7d80664a0 0x7fe7d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.763+0000 7fe7f0b7c640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7d80664a0 0x7fe7d8068960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc007fd0 tx=0x7fe7dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:48.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.767+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe7e4051ac0 con 0x7fe7ec1595a0 2023-10-26T05:53:48.954 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7ec1562d0 con 0x7fe7ec1595a0 2023-10-26T05:53:48.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7e97fa640 1 -- 172.21.15.71:0/3277900093 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fe7e401d090 con 0x7fe7ec1595a0 2023-10-26T05:53:48.955 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:48.956 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:53:48.956 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:53:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7d80664a0 msgr2=0x7fe7d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.957 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7d80664a0 0x7fe7d8068960 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc007fd0 tx=0x7fe7dc009040 comp rx=0 tx=0).stop 2023-10-26T05:53:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 msgr2=0x7fe7ec157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 0x7fe7ec157f70 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400ca60 tx=0x7fe7e400cf20 comp rx=0 tx=0).stop 2023-10-26T05:53:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 shutdown_connections 2023-10-26T05:53:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe7d80664a0 0x7fe7d8068960 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7ec1595a0 0x7fe7ec157f70 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ec1591c0 0x7fe7ec144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 --2- 172.21.15.71:0/3277900093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ec06b010 0x7fe7ec0b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 >> 172.21.15.71:0/3277900093 conn(0x7fe7ec070170 msgr2=0x7fe7ec15b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:48.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 shutdown_connections 2023-10-26T05:53:48.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:48.947+0000 7fe7f2de0640 1 -- 172.21.15.71:0/3277900093 wait complete. 2023-10-26T05:53:49.375 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:53:39.735661+0000', 'modified': '2023-10-26T05:53:41.733046+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8007': {'gid': 8007, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/1300756647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 1300756647}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 1300756647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:53:49.375 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:53:49.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T05:53:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb8180f7640 1 -- 172.21.15.71:0/3356638500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 msgr2=0x7fb7fc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3356638500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb7fc003da0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb808009ee0 tx=0x7fb80801d5d0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb816694640 1 -- 172.21.15.71:0/3356638500 reap_dead start 2023-10-26T05:53:49.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb8180f7640 1 -- 172.21.15.71:0/3356638500 shutdown_connections 2023-10-26T05:53:49.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb8180f7640 1 -- 172.21.15.71:0/3356638500 >> 172.21.15.71:0/3356638500 conn(0x7fb8100aff00 msgr2=0x7fb81006fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb8180f7640 1 -- 172.21.15.71:0/3356638500 shutdown_connections 2023-10-26T05:53:49.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.499+0000 7fb8180f7640 1 -- 172.21.15.71:0/3356638500 wait complete. 2023-10-26T05:53:49.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 Processor -- start 2023-10-26T05:53:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 -- start start 2023-10-26T05:53:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb8101f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8100c28c0 0x7fb81014e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8101e8850 0x7fb8101e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8101e7590 con 0x7fb7fc003980 2023-10-26T05:53:49.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8100c3060 con 0x7fb8101e8850 2023-10-26T05:53:49.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8101e83c0 con 0x7fb8100c28c0 2023-10-26T05:53:49.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb8101f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb8101f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34554/0 (socket says 172.21.15.71:34554) 2023-10-26T05:53:49.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 -- 172.21.15.71:0/3813690361 learned_addr learned my addr 172.21.15.71:0/3813690361 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:49.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb816694640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8101e8850 0x7fb8101e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815692640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8100c28c0 0x7fb81014e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 -- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8100c28c0 msgr2=0x7fb81014e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8100c28c0 0x7fb81014e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 -- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8101e8850 msgr2=0x7fb8101e94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8101e8850 0x7fb8101e94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 -- 172.21.15.71:0/3813690361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8101f5cb0 con 0x7fb7fc003980 2023-10-26T05:53:49.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815692640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8100c28c0 0x7fb81014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:49.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb815e93640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb8101f5770 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb808009ee0 tx=0x7fb808009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8067fc640 1 -- 172.21.15.71:0/3813690361 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb808003b80 con 0x7fb7fc003980 2023-10-26T05:53:49.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8067fc640 1 -- 172.21.15.71:0/3813690361 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb8080bb070 con 0x7fb7fc003980 2023-10-26T05:53:49.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8100c2d80 con 0x7fb7fc003980 2023-10-26T05:53:49.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8067fc640 1 -- 172.21.15.71:0/3813690361 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8080ac5c0 con 0x7fb7fc003980 2023-10-26T05:53:49.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8101e8050 con 0x7fb7fc003980 2023-10-26T05:53:49.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.503+0000 7fb8067fc640 1 -- 172.21.15.71:0/3813690361 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb8080bd070 con 0x7fb7fc003980 2023-10-26T05:53:49.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb8067fc640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7f40664a0 0x7fb7f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb8067fc640 1 -- 172.21.15.71:0/3813690361 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb80811f930 con 0x7fb7fc003980 2023-10-26T05:53:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7f40664a0 0x7fb7f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 Processor -- start 2023-10-26T05:53:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- start start 2023-10-26T05:53:49.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7f40664a0 0x7fb7f4068960 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb800005d10 tx=0x7fb800005c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965295da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x56096524ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x56096524c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965040cc0 con 0x5609652962e0 2023-10-26T05:53:49.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609650a3920 con 0x56096524bf80 2023-10-26T05:53:49.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609650a0e50 con 0x5609652959c0 2023-10-26T05:53:49.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x56096524ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x56096524ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34558/0 (socket says 172.21.15.71:34558) 2023-10-26T05:53:49.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 -- 172.21.15.71:0/3513054455 learned_addr learned my addr 172.21.15.71:0/3513054455 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:49.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb816694640 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x56096524c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815e93640 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965295da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 -- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 msgr2=0x560965295da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965295da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 -- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 msgr2=0x56096524c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x56096524c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 -- 172.21.15.71:0/3513054455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560964d95890 con 0x5609652962e0 2023-10-26T05:53:49.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb815692640 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x56096524ba40 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb800010d50 tx=0x7fb800009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ecb83e02b6be27 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb7f37fe640 1 -- 172.21.15.71:0/3513054455 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800018040 con 0x5609652962e0 2023-10-26T05:53:49.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb7f37fe640 1 -- 172.21.15.71:0/3513054455 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb800003ba0 con 0x5609652962e0 2023-10-26T05:53:49.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb7f37fe640 1 -- 172.21.15.71:0/3513054455 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800020460 con 0x5609652962e0 2023-10-26T05:53:49.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- 172.21.15.71:0/3513054455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 msgr2=0x56096524ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x56096524ba40 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb800010d50 tx=0x7fb800009f90 comp rx=0 tx=0).stop 2023-10-26T05:53:49.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- 172.21.15.71:0/3513054455 shutdown_connections 2023-10-26T05:53:49.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x56096524c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x56096524ba40 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 --2- 172.21.15.71:0/3513054455 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965295da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- 172.21.15.71:0/3513054455 >> 172.21.15.71:0/3513054455 conn(0x5609652432f0 msgr2=0x56096524a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- 172.21.15.71:0/3513054455 shutdown_connections 2023-10-26T05:53:49.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- 172.21.15.71:0/3513054455 wait complete. 2023-10-26T05:53:49.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 -- start start 2023-10-26T05:53:49.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.507+0000 7fb81b241000 1 Processor -- start 2023-10-26T05:53:49.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x560965295410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965251350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x560965251890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096509c740 con 0x5609652962e0 2023-10-26T05:53:49.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096505bd10 con 0x56096524bf80 2023-10-26T05:53:49.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096505b9b0 con 0x5609652959c0 2023-10-26T05:53:49.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 10 client.0 _ll_register_callbacks cb 0x5609652376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:53:49.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x560965251890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x560965251890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34564/0 (socket says 172.21.15.71:34564) 2023-10-26T05:53:49.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 -- 172.21.15.71:0/197903278 learned_addr learned my addr 172.21.15.71:0/197903278 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:49.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 -- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 msgr2=0x560965251350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965251350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 -- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 msgr2=0x560965295410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb815e93640 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x560965295410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x560965295410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 -- 172.21.15.71:0/197903278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb800010a00 con 0x5609652962e0 2023-10-26T05:53:49.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb816694640 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x560965251890 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb8100b0610 tx=0x7fb80c010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712973fe6c587f63 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:53:49.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/197903278 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00b040 con 0x5609652962e0 2023-10-26T05:53:49.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/197903278 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb80c014740 con 0x5609652962e0 2023-10-26T05:53:49.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560965050370 con 0x5609652962e0 2023-10-26T05:53:49.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/197903278 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80c014900 con 0x5609652962e0 2023-10-26T05:53:49.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 10 client.8181 fetch_fsmap learned FSMap version 51 2023-10-26T05:53:49.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x56096505e460 con 0x5609652962e0 2023-10-26T05:53:49.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/197903278 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x560965050370 con 0x5609652962e0 2023-10-26T05:53:49.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/197903278 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb80c014a60 con 0x5609652962e0 2023-10-26T05:53:49.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 10 client.8181 fetch_fsmap finished waiting for FSMap version 51 2023-10-26T05:53:49.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 10 client.8181 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:53:49.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 20 client.8181 populate_metadata read hostname 'smithi071' 2023-10-26T05:53:49.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x560965258440 0x5609652a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 4 client.8181 mds_command: new command op to 8007 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:53:49.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 --> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609650362d0 con 0x560965258440 2023-10-26T05:53:49.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb815e93640 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x560965258440 0x5609652a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb815e93640 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x560965258440 0x5609652a7a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a286ff680fc3fb5 server_cookie=9e8bf65bf388b8c4 in_seq=0 out_seq=0 2023-10-26T05:53:49.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 10 client.8181 ms_handle_connect on v2:172.21.15.71:6834/1300756647 2023-10-26T05:53:49.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/197903278 <== mds.0 v2:172.21.15.71:6834/1300756647 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5609650362d0 con 0x560965258440 2023-10-26T05:53:49.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 10 client.8181 handle_command_reply: tid=0 2023-10-26T05:53:49.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 client.8181 shutdown 2023-10-26T05:53:49.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 msgr2=0x560965251890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x560965251890 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb8100b0610 tx=0x7fb80c010910 comp rx=0 tx=0).stop 2023-10-26T05:53:49.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 shutdown_connections 2023-10-26T05:53:49.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x560965258440 0x5609652a7a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652962e0 0x560965251890 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652959c0 0x560965251350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 --2- 172.21.15.71:0/197903278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x560965295410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb7f0ff9640 0 client.8181 ms_handle_reset on v2:172.21.15.71:6834/1300756647 2023-10-26T05:53:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 >> 172.21.15.71:0/197903278 conn(0x5609652432f0 msgr2=0x56096524aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 shutdown_connections 2023-10-26T05:53:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 1 -- 172.21.15.71:0/197903278 wait complete. 2023-10-26T05:53:49.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.511+0000 7fb81b241000 20 client.8181 trim_cache size 0 max 16384 2023-10-26T05:53:49.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 Processor -- start 2023-10-26T05:53:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- start start 2023-10-26T05:53:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 0x560965293a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609652959c0 0x560965293f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560964d56940 con 0x56096524bf80 2023-10-26T05:53:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096509b5d0 con 0x5609652959c0 2023-10-26T05:53:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x560964d961b0 con 0x5609652962e0 2023-10-26T05:53:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 0x560965293a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 0x560965293a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34566/0 (socket says 172.21.15.71:34566) 2023-10-26T05:53:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 -- 172.21.15.71:0/721973990 learned_addr learned my addr 172.21.15.71:0/721973990 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 -- 172.21.15.71:0/721973990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 msgr2=0x5609652a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 -- 172.21.15.71:0/721973990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609652959c0 msgr2=0x560965293f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609652959c0 0x560965293f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 -- 172.21.15.71:0/721973990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560965044760 con 0x56096524bf80 2023-10-26T05:53:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb815e93640 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 0x560965293a00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb80801ecb0 tx=0x7fb80811dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c27b7a3169e3d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb7f3fff640 1 -- 172.21.15.71:0/721973990 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8080fb070 con 0x56096524bf80 2023-10-26T05:53:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb7f3fff640 1 -- 172.21.15.71:0/721973990 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb8080f0b90 con 0x56096524bf80 2023-10-26T05:53:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb7f3fff640 1 -- 172.21.15.71:0/721973990 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8080c4050 con 0x56096524bf80 2023-10-26T05:53:49.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- 172.21.15.71:0/721973990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 msgr2=0x560965293a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 0x560965293a00 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb80801ecb0 tx=0x7fb80811dd30 comp rx=0 tx=0).stop 2023-10-26T05:53:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- 172.21.15.71:0/721973990 shutdown_connections 2023-10-26T05:53:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609652959c0 0x560965293f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 --2- 172.21.15.71:0/721973990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56096524bf80 0x560965293a00 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- 172.21.15.71:0/721973990 >> 172.21.15.71:0/721973990 conn(0x5609652432f0 msgr2=0x560965256b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- 172.21.15.71:0/721973990 shutdown_connections 2023-10-26T05:53:49.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- 172.21.15.71:0/721973990 wait complete. 2023-10-26T05:53:49.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 -- start start 2023-10-26T05:53:49.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.527+0000 7fb81b241000 1 Processor -- start 2023-10-26T05:53:49.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x5609652fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 0x5609652fc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560965065720 con 0x5609652959c0 2023-10-26T05:53:49.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609652503b0 con 0x56096524bf80 2023-10-26T05:53:49.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609650362d0 con 0x5609652962e0 2023-10-26T05:53:49.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 10 client.0 _ll_register_callbacks cb 0x56096509aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:53:49.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 0x5609652fc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 0x5609652fc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34570/0 (socket says 172.21.15.71:34570) 2023-10-26T05:53:49.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 -- 172.21.15.71:0/1614712691 learned_addr learned my addr 172.21.15.71:0/1614712691 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:49.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 -- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 msgr2=0x5609652a7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb816694640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815e93640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x5609652fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 -- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 msgr2=0x5609652fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:53:49.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x5609652fc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 -- 172.21.15.71:0/1614712691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8101e8050 con 0x5609652959c0 2023-10-26T05:53:49.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815692640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 0x5609652fc9c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb800009fd0 tx=0x7fb800003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cbc7cc14257cfa server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:53:49.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/1614712691 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800020660 con 0x5609652959c0 2023-10-26T05:53:49.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/1614712691 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8000207c0 con 0x5609652959c0 2023-10-26T05:53:49.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609650346f0 con 0x5609652959c0 2023-10-26T05:53:49.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/1614712691 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb800020980 con 0x5609652959c0 2023-10-26T05:53:49.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 10 client.8187 fetch_fsmap learned FSMap version 51 2023-10-26T05:53:49.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x560964d95cc0 con 0x5609652959c0 2023-10-26T05:53:49.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/1614712691 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5609650346f0 con 0x5609652959c0 2023-10-26T05:53:49.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/1614712691 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb80002f020 con 0x5609652959c0 2023-10-26T05:53:49.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 10 client.8187 fetch_fsmap finished waiting for FSMap version 51 2023-10-26T05:53:49.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 10 client.8187 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:53:49.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 20 client.8187 populate_metadata read hostname 'smithi071' 2023-10-26T05:53:49.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x5609652a8920 0x5609652a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 4 client.8187 mds_command: new command op to 8007 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:53:49.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 --> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560965095720 con 0x5609652a8920 2023-10-26T05:53:49.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815e93640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x5609652a8920 0x5609652a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb815e93640 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x5609652a8920 0x5609652a8d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff7b5dcbfab6dcc server_cookie=eaf0d0bca4708f1d in_seq=0 out_seq=0 2023-10-26T05:53:49.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 10 client.8187 ms_handle_connect on v2:172.21.15.71:6834/1300756647 2023-10-26T05:53:49.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 1 -- 172.21.15.71:0/1614712691 <== mds.0 v2:172.21.15.71:6834/1300756647 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7fb80811c160 con 0x5609652a8920 2023-10-26T05:53:49.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 10 client.8187 handle_command_reply: tid=0 2023-10-26T05:53:49.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 client.8187 shutdown 2023-10-26T05:53:49.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 msgr2=0x5609652fc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 0x5609652fc9c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fb800009fd0 tx=0x7fb800003ba0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 shutdown_connections 2023-10-26T05:53:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:6834/1300756647,v1:172.21.15.71:6835/1300756647] conn(0x5609652a8920 0x5609652a8d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5609652962e0 0x5609652a7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609652959c0 0x5609652fc9c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 --2- 172.21.15.71:0/1614712691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56096524bf80 0x5609652fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb7f0ff9640 0 client.8187 ms_handle_reset on v2:172.21.15.71:6834/1300756647 2023-10-26T05:53:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 >> 172.21.15.71:0/1614712691 conn(0x5609652432f0 msgr2=0x560965257420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 shutdown_connections 2023-10-26T05:53:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 1 -- 172.21.15.71:0/1614712691 wait complete. 2023-10-26T05:53:49.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb81b241000 20 client.8187 trim_cache size 0 max 16384 2023-10-26T05:53:49.561 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:53:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:53:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8007, 2023-10-26T05:53:49.561 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:53:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:53:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:53:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 9.1283107660000002, 2023-10-26T05:53:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 51, 2023-10-26T05:53:49.562 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 138, 2023-10-26T05:53:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 138, 2023-10-26T05:53:49.563 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 13.974991918000001 2023-10-26T05:53:49.563 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7f40664a0 msgr2=0x7fb7f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7f40664a0 0x7fb7f4068960 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb800005d10 tx=0x7fb800005c60 comp rx=0 tx=0).stop 2023-10-26T05:53:49.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 msgr2=0x7fb8101f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb8101f5770 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb808009ee0 tx=0x7fb808009280 comp rx=0 tx=0).stop 2023-10-26T05:53:49.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 shutdown_connections 2023-10-26T05:53:49.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8101e8850 0x7fb8101e94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb8100c28c0 0x7fb81014e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb7f40664a0 0x7fb7f4068960 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 --2- 172.21.15.71:0/3813690361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb7fc003980 0x7fb8101f5770 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 >> 172.21.15.71:0/3813690361 conn(0x7fb8100aff00 msgr2=0x7fb8100adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 shutdown_connections 2023-10-26T05:53:49.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.531+0000 7fb8180f7640 1 -- 172.21.15.71:0/3813690361 wait complete. 2023-10-26T05:53:49.567 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:53:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/3993439949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f00047d0 msgr2=0x7ff3f0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/3993439949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3f00047d0 0x7ff3f0004c50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4005fd0 tx=0x7ff3f401cd80 comp rx=0 tx=0).stop 2023-10-26T05:53:49.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 -- 172.21.15.71:0/3993439949 reap_dead start 2023-10-26T05:53:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/3993439949 shutdown_connections 2023-10-26T05:53:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/3993439949 >> 172.21.15.71:0/3993439949 conn(0x7ff404069770 msgr2=0x7ff404069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/3993439949 shutdown_connections 2023-10-26T05:53:49.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/3993439949 wait complete. 2023-10-26T05:53:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 Processor -- start 2023-10-26T05:53:49.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- start start 2023-10-26T05:53:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f00047d0 0x7ff404160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff40413c350 0x7ff4041383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 0x7ff40413d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40413d700 con 0x7ff40413cd30 2023-10-26T05:53:49.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404161bf0 con 0x7ff3f00047d0 2023-10-26T05:53:49.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff404152d60 con 0x7ff40413c350 2023-10-26T05:53:49.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 0x7ff40413d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 0x7ff40413d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34588/0 (socket says 172.21.15.71:34588) 2023-10-26T05:53:49.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 -- 172.21.15.71:0/2284592092 learned_addr learned my addr 172.21.15.71:0/2284592092 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:49.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff409d97640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff40413c350 0x7ff4041383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40a598640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f00047d0 0x7ff404160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 -- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff40413c350 msgr2=0x7ff4041383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff40413c350 0x7ff4041383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 -- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f00047d0 msgr2=0x7ff404160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f00047d0 0x7ff404160c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 -- 172.21.15.71:0/2284592092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404161350 con 0x7ff40413cd30 2023-10-26T05:53:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40ad99640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 0x7ff40413d1c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff4040b1110 tx=0x7ff3fc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc018070 con 0x7ff40413cd30 2023-10-26T05:53:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff3fc004b30 con 0x7ff40413cd30 2023-10-26T05:53:49.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00ba20 con 0x7ff40413cd30 2023-10-26T05:53:49.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404161630 con 0x7ff40413cd30 2023-10-26T05:53:49.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff404153140 con 0x7ff40413cd30 2023-10-26T05:53:49.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.671+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404066b10 con 0x7ff40413cd30 2023-10-26T05:53:49.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.675+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff3fc00bb80 con 0x7ff40413cd30 2023-10-26T05:53:49.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.675+0000 7ff3faffd640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3e8066470 0x7ff3e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:49.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.675+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3fc083b10 con 0x7ff40413cd30 2023-10-26T05:53:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.675+0000 7ff40a598640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3e8066470 0x7ff3e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.675+0000 7ff40a598640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3e8066470 0x7ff3e8068930 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff400000fc0 tx=0x7ff400001690 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:49.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.679+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff3fc050f90 con 0x7ff40413cd30 2023-10-26T05:53:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.859+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff404152fa0 con 0x7ff40413cd30 2023-10-26T05:53:49.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.859+0000 7ff3faffd640 1 -- 172.21.15.71:0/2284592092 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff404152fa0 con 0x7ff40413cd30 2023-10-26T05:53:49.869 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:53:49.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3e8066470 msgr2=0x7ff3e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3e8066470 0x7ff3e8068930 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff400000fc0 tx=0x7ff400001690 comp rx=0 tx=0).stop 2023-10-26T05:53:49.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 msgr2=0x7ff40413d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:49.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 0x7ff40413d1c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff4040b1110 tx=0x7ff3fc00dd10 comp rx=0 tx=0).stop 2023-10-26T05:53:49.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 shutdown_connections 2023-10-26T05:53:49.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3e8066470 0x7ff3e8068930 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff40413cd30 0x7ff40413d1c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff40413c350 0x7ff4041383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 --2- 172.21.15.71:0/2284592092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3f00047d0 0x7ff404160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 >> 172.21.15.71:0/2284592092 conn(0x7ff404069770 msgr2=0x7ff4040ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 shutdown_connections 2023-10-26T05:53:49.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:49.863+0000 7ff40c7fc640 1 -- 172.21.15.71:0/2284592092 wait complete. 2023-10-26T05:53:49.886 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:53:49.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:53:49.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:53:49.887 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:53:49.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:49.888 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:53:49.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:53:49.889 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:53:49.889 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:53:49.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:49.890 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:53:49.896 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:53:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:53:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:53:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:53:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.897 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:49.897 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:53:49.897 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:53:49.898 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:49.898 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:49.899 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:53:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:53:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42147sec preferred_lft 42147sec 2023-10-26T05:53:49.899 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:53:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.900 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:53:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:53:49.900 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.900 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:53:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:53:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:53:49.901 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.902 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:53:49.902 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.902 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:53:49.903 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:49.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:49.904 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:53:49.947 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:53:49.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:49.949 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:53:49.997 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:53:50.014 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:53:50.015 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:53:50.015 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:53:50.015 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:53:50.015 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:53:50.015 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:50.016 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:50.016 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:50.016 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:53:50.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:50.017 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:53:50.017 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:53:50.017 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:53:50.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:53:50.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:53:50.019 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:53:50.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:50.072 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:53:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:53:50.117 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:53:50.117 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:53:50.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:50.118 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:53:50.172 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:53:50.172 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:50.173 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:50.218 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:53:50.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:50.218 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:50.265 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:50.315 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:53:50.315 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:53:50.315 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:53:50.315 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:53:50.316 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:53:50.316 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:53:50.316 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:53:50.316 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:53:50.316 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:53:50.317 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:53:50.317 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:53:50.317 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:53:50.317 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:50.317 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:53:50.318 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:53:50.318 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:53:50.318 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:53:50.318 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:53:50.318 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:50.319 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:53:50.319 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:50.319 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:53:50.319 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:53:50.319 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:53:50.319 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:53:50.320 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:53:50.320 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:53:50.320 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:53:50.320 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:53:50.320 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:50.321 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:50.321 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:50.321 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:50.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:50.322 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:53:50.370 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:53:50.371 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:53:50.419 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:50.426 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:53:50.426 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:53:50.426 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:53:50.426 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:53:50.427 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:53:50.427 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:53:50.427 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:53:50.427 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:53:50.427 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:53:50.428 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:53:50.428 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:53:50.428 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:53:50.428 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:50.428 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:53:50.429 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:53:50.429 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:53:50.429 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:53:50.429 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:53:50.429 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:50.429 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:53:50.430 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:50.430 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:53:50.430 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:53:50.430 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:53:50.430 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:53:50.431 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:53:50.431 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:53:50.431 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:53:50.431 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:53:50.431 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:50.432 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:50.432 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:50.432 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:50.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:50.433 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:53:50.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:53:50.495+0000 7fbd9bd02580 -1 init, newargv = 0x56444cfb7250 newargc=15 2023-10-26T05:53:50.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[49371]: starting ceph client 2023-10-26T05:53:50.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[49371]: starting fuse 2023-10-26T05:53:51.583 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:51.591 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:53:51.592 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:53:51.592 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:53:51.592 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:53:51.593 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:53:51.593 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:53:51.593 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:53:51.593 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:53:51.593 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:53:51.594 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:53:51.594 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:53:51.594 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:53:51.594 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:53:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:53:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:53:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:53:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:53:51.596 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:51.596 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:53:51.596 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:51.596 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:53:51.596 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:53:51.597 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:53:51.597 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:53:51.597 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:53:51.597 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:53:51.597 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:53:51.598 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:53:51.598 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:51.598 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:53:51.598 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:51.599 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:53:51.599 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:51.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:51.600 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:53:51.638 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:53:51.639 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:53:51.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:51.640 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:53:51.640 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:53:51.640 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:53:51.641 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:53:51.641 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:51.642 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:53:51.642 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:53:51.642 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:53:51.643 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:53:51.643 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:51.644 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:53:51.644 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:53:51.645 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:53:51.645 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:53:51.645 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:51.646 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:53:51.646 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:51.647 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:51.647 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:51.647 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:53:51.648 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:53:51.648 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:53:51.649 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:51.649 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:53:51.649 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:53:51.650 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:53:51.763 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.49371.asok 2023-10-26T05:53:51.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:51.774 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49371.asok 2023-10-26T05:53:51.775 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:51.775 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.49371.asok status 2023-10-26T05:53:51.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:51.975 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:51.976 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:53:51.976 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:51.976 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:51.976 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:53:51.976 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:53:51.977 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:53:51.977 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "49371", 2023-10-26T05:53:51.977 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:53:51.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:51.977 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:53:51.978 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:51.978 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8199, 2023-10-26T05:53:51.978 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:53:51.978 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:53:51.978 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:53:51.978 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 8199 2023-10-26T05:53:51.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:51.979 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:51.979 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:51.979 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:51.979 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 33362412 2023-10-26T05:53:51.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:51.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:51.980 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:53:51.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:51.980 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:51.980 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 33362412 2023-10-26T05:53:51.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:51.981 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.8199 192.168.0.1:0/33362412", 2023-10-26T05:53:51.981 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/33362412", 2023-10-26T05:53:51.981 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:53:51.981 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 51, 2023-10-26T05:53:51.982 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 138, 2023-10-26T05:53:51.982 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:53:51.982 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:53:51.982 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:53:51.982 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:51.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:51.996 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:51.996 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:53:51.996 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:52.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:52.042 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:53:52.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:52.049 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:52.050 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:52.050 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:53:52.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:52.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:52.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:53:52.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/196661095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 msgr2=0x7fe738005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:52.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 --2- 172.21.15.71:0/196661095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe738005190 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe748001d70 tx=0x7fe74801cae0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe754e0f640 1 -- 172.21.15.71:0/196661095 reap_dead start 2023-10-26T05:53:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/196661095 shutdown_connections 2023-10-26T05:53:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/196661095 >> 172.21.15.71:0/196661095 conn(0x7fe75006f440 msgr2=0x7fe75006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:52.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/196661095 shutdown_connections 2023-10-26T05:53:52.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/196661095 wait complete. 2023-10-26T05:53:52.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 Processor -- start 2023-10-26T05:53:52.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- start start 2023-10-26T05:53:52.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe750137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:52.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe75015d900 0x7fe7500ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:52.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7500ca640 0x7fe7500cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:52.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501f25a0 con 0x7fe738004d10 2023-10-26T05:53:52.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501f1d10 con 0x7fe7500ca640 2023-10-26T05:53:52.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500cae50 con 0x7fe75015d900 2023-10-26T05:53:52.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe750137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:52.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe750137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38076/0 (socket says 172.21.15.71:38076) 2023-10-26T05:53:52.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 -- 172.21.15.71:0/199713284 learned_addr learned my addr 172.21.15.71:0/199713284 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:52.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74f7fe640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe75015d900 0x7fe7500ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe754e0f640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7500ca640 0x7fe7500cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 -- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe75015d900 msgr2=0x7fe7500ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe75015d900 0x7fe7500ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 -- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7500ca640 msgr2=0x7fe7500cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7500ca640 0x7fe7500cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 -- 172.21.15.71:0/199713284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7501f1f20 con 0x7fe738004d10 2023-10-26T05:53:52.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74ffff640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe750137ba0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe740011a50 tx=0x7fe740011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:52.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000fd30 con 0x7fe738004d10 2023-10-26T05:53:52.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe740012450 con 0x7fe738004d10 2023-10-26T05:53:52.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7500cab70 con 0x7fe738004d10 2023-10-26T05:53:52.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740012860 con 0x7fe738004d10 2023-10-26T05:53:52.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.263+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7501380e0 con 0x7fe738004d10 2023-10-26T05:53:52.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.267+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe750066b10 con 0x7fe738004d10 2023-10-26T05:53:52.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.267+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7400129c0 con 0x7fe738004d10 2023-10-26T05:53:52.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.267+0000 7fe74cff9640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe73c0664a0 0x7fe73c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:52.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.267+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7400854d0 con 0x7fe738004d10 2023-10-26T05:53:52.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.267+0000 7fe74f7fe640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe73c0664a0 0x7fe73c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:52.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.267+0000 7fe74f7fe640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe73c0664a0 0x7fe73c068960 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe75014c950 tx=0x7fe744011410 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:52.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.271+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe740052b70 con 0x7fe738004d10 2023-10-26T05:53:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe7500721e0 con 0x7fe738004d10 2023-10-26T05:53:52.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe74cff9640 1 -- 172.21.15.71:0/199713284 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe74001f070 con 0x7fe738004d10 2023-10-26T05:53:52.471 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:53:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe73c0664a0 msgr2=0x7fe73c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe73c0664a0 0x7fe73c068960 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe75014c950 tx=0x7fe744011410 comp rx=0 tx=0).stop 2023-10-26T05:53:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 msgr2=0x7fe750137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe750137ba0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe740011a50 tx=0x7fe740011f10 comp rx=0 tx=0).stop 2023-10-26T05:53:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 shutdown_connections 2023-10-26T05:53:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7500ca640 0x7fe7500cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe75015d900 0x7fe7500ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe73c0664a0 0x7fe73c068960 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 --2- 172.21.15.71:0/199713284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe738004d10 0x7fe750137ba0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 >> 172.21.15.71:0/199713284 conn(0x7fe75006f440 msgr2=0x7fe75006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 shutdown_connections 2023-10-26T05:53:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:52.463+0000 7fe756872640 1 -- 172.21.15.71:0/199713284 wait complete. 2023-10-26T05:53:52.490 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:53:52.491 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:53:52.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:53:52.491 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:53:52.492 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:52.492 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:53:52.492 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:53:52.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:53:52.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:53:52.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.494 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:53:52.501 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:53:52.502 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:53:52.502 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:53:52.502 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.502 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:53:52.503 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.503 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:52.503 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:52.503 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:53:52.503 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:52.504 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:52.504 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:53:52.504 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:53:52.504 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:52.504 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:53:52.504 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:52.505 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:53:52.505 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42138sec preferred_lft 42138sec 2023-10-26T05:53:52.505 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:53:52.505 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.505 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:53:52.505 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:52.506 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:53:52.506 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.506 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:53:52.506 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:53:52.506 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:53:52.507 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.507 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:53:52.507 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.507 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:52.507 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:52.508 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:53:52.508 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.508 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:53:52.508 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:52.508 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:53:52.509 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:53:52.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.509 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:53:52.557 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:53:52.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.559 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:53:52.610 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:53:52.610 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:53:52.611 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:53:52.611 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:53:52.611 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:53:52.611 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:53:52.611 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:52.612 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:52.612 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:53:52.612 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:53:52.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.613 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:53:52.613 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:53:52.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:53:52.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:53:52.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:53:52.614 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:53:52.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.692 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:53:52.742 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:53:52.743 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:53:52.743 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:53:52.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:53:52.798 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:53:52.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:52.799 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:52.845 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:53:52.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:53:52.846 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:52.894 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:52.942 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:53:52.943 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:53:52.943 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:53:52.943 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:53:52.943 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:53:52.943 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:53:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:53:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:53:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:53:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:53:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:53:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:53:52.945 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:52.945 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:53:52.945 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:53:52.945 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:53:52.945 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:53:52.946 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:53:52.946 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:52.946 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:53:52.946 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:52.946 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:53:52.947 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:53:52.947 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:53:52.947 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:53:52.947 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:53:52.947 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:53:52.948 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:53:52.948 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:53:52.948 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:52.948 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:52.948 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:52.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:52.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:52.950 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:53:53.002 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:53:53.002 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:53:53.047 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:53.053 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:53:53.053 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:53:53.054 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:53:53.054 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:53:53.054 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:53:53.054 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:53:53.054 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:53:53.055 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:53:53.055 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:53:53.055 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:53:53.055 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:53:53.056 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:53:53.056 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:53.056 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:53:53.056 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:53:53.056 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:53:53.056 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:53:53.057 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:53:53.057 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:53.057 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:53:53.057 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:53.057 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:53:53.058 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:53:53.058 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:53:53.058 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:53:53.058 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:53:53.058 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:53:53.058 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:53:53.059 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:53:53.059 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:53.059 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:53.059 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:53.059 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:53.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:53.061 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:53:53.129 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:53:53.124+0000 7f5593806580 -1 init, newargv = 0x562e598aa380 newargc=15 2023-10-26T05:53:53.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35050]: starting ceph client 2023-10-26T05:53:53.135 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35050]: starting fuse 2023-10-26T05:53:54.210 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:53:54.218 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:53:54.219 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:53:54.219 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:53:54.219 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:53:54.219 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:53:54.220 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:53:54.220 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:53:54.220 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:53:54.220 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:53:54.220 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:53:54.221 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:53:54.221 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:53:54.221 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:53:54.221 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:53:54.221 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:53:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:53:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:53:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:53:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:53:54.222 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:53:54.223 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:53:54.223 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:53:54.223 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:53:54.223 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:53:54.223 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:53:54.224 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:53:54.224 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:53:54.224 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:53:54.224 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:53:54.224 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:54.225 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:53:54.225 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:54.225 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:53:54.225 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:54.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:54.227 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:53:54.274 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:53:54.274 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:53:54.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:53:54.275 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:53:54.275 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:53:54.276 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:53:54.276 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:53:54.277 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:54.277 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:53:54.277 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:53:54.278 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:53:54.278 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:53:54.278 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:54.279 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:53:54.279 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:53:54.280 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:53:54.280 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:53:54.280 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:54.281 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:53:54.281 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:53:54.281 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:53:54.282 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:53:54.282 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:53:54.282 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:53:54.283 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:53:54.283 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:53:54.283 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:53:54.284 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:53:54.284 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:53:54.398 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.35050.asok 2023-10-26T05:53:54.402 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:54.410 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35050.asok 2023-10-26T05:53:54.411 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:54.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.35050.asok status 2023-10-26T05:53:54.503 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:54.606 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:53:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:53:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:53:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:53:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:53:54.607 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:53:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:53:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "35050", 2023-10-26T05:53:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:53:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:54.608 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:53:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:53:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5834, 2023-10-26T05:53:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:53:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:53:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:53:54.609 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 5834 2023-10-26T05:53:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:53:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:53:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:54.610 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1121414841 2023-10-26T05:53:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:53:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:53:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:53:54.611 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:53:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1121414841 2023-10-26T05:53:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:53:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.5834 192.168.0.1:0/1121414841", 2023-10-26T05:53:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1121414841", 2023-10-26T05:53:54.612 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:53:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 51, 2023-10-26T05:53:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 138, 2023-10-26T05:53:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:53:54.613 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:53:54.614 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:53:54.614 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:53:54.618 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:54.625 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:54.626 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:53:54.626 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:54.669 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:54.670 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:53:54.670 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:54.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:54.677 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:53:54.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:53:54.766 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:53:54.767 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:53:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:53:54.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:53:54.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/554972851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6d80 msgr2=0x7f7dc406a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:54.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 --2- 172.21.15.71:0/554972851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6d80 0x7f7dc406a570 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40d0e70 tx=0x7f7db401c720 comp rx=0 tx=0).stop 2023-10-26T05:53:54.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/554972851 shutdown_connections 2023-10-26T05:53:54.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 --2- 172.21.15.71:0/554972851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6d80 0x7f7dc406a570 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:54.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 --2- 172.21.15.71:0/554972851 >> v2:172.21.15.86:3300/0 conn(0x7f7dc415b5c0 0x7f7dc40b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:54.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/554972851 >> 172.21.15.71:0/554972851 conn(0x7f7dc40d12d0 msgr2=0x7f7dc40d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:54.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/554972851 shutdown_connections 2023-10-26T05:53:54.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/554972851 wait complete. 2023-10-26T05:53:54.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 Processor -- start 2023-10-26T05:53:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- start start 2023-10-26T05:53:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 0x7f7dc41535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc40b6d80 0x7f7dc40c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:54.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc415b5c0 0x7f7dc4140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc4159340 con 0x7f7dc40b6910 2023-10-26T05:53:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc40c9ec0 con 0x7f7dc415b5c0 2023-10-26T05:53:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc4158a10 con 0x7f7dc40b6d80 2023-10-26T05:53:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 0x7f7dc41535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:54.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 0x7f7dc41535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38100/0 (socket says 172.21.15.71:38100) 2023-10-26T05:53:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 -- 172.21.15.71:0/812050597 learned_addr learned my addr 172.21.15.71:0/812050597 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc97b3640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc415b5c0 0x7f7dc4140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 -- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc40b6d80 msgr2=0x7f7dc40c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc3fff640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc40b6d80 0x7f7dc40c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:54.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc40b6d80 0x7f7dc40c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:54.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 -- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc415b5c0 msgr2=0x7f7dc4140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:54.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc415b5c0 0x7f7dc4140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:54.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 -- 172.21.15.71:0/812050597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40094a0 con 0x7f7dc40b6910 2023-10-26T05:53:54.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc97b3640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc415b5c0 0x7f7dc4140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:54.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc8fb2640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 0x7f7dc41535b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00eaa0 tx=0x7f7dbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:54.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc00ce10 con 0x7f7dc40b6910 2023-10-26T05:53:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7dbc011470 con 0x7f7dc40b6910 2023-10-26T05:53:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc40cdb70 con 0x7f7dc40b6910 2023-10-26T05:53:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc011860 con 0x7f7dc40b6910 2023-10-26T05:53:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.903+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc4158fd0 con 0x7f7dc40b6910 2023-10-26T05:53:54.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.907+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc4066b10 con 0x7f7dc40b6910 2023-10-26T05:53:54.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.907+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7dbc01a400 con 0x7f7dc40b6910 2023-10-26T05:53:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.907+0000 7f7dc17fa640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7db00664a0 0x7f7db0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.907+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dbc084ec0 con 0x7f7dc40b6910 2023-10-26T05:53:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.907+0000 7f7dc3fff640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7db00664a0 0x7f7db0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.907+0000 7f7dc3fff640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7db00664a0 0x7f7db0068960 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7db8001380 tx=0x7f7db8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:54.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:54.911+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7dbc052560 con 0x7f7dc40b6910 2023-10-26T05:53:55.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.091+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dc40721e0 con 0x7f7dc40b6910 2023-10-26T05:53:55.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.091+0000 7f7dc17fa640 1 -- 172.21.15.71:0/812050597 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7dbc055b80 con 0x7f7dc40b6910 2023-10-26T05:53:55.100 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:55.100 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:53:55.100 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:53:55.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7db00664a0 msgr2=0x7f7db0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7db00664a0 0x7f7db0068960 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7db8001380 tx=0x7f7db8009040 comp rx=0 tx=0).stop 2023-10-26T05:53:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 msgr2=0x7f7dc41535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 0x7f7dc41535b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc00eaa0 tx=0x7f7dbc00ef60 comp rx=0 tx=0).stop 2023-10-26T05:53:55.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 shutdown_connections 2023-10-26T05:53:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc415b5c0 0x7f7dc4140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc40b6d80 0x7f7dc40c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7db00664a0 0x7f7db0068960 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 --2- 172.21.15.71:0/812050597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc40b6910 0x7f7dc41535b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 >> 172.21.15.71:0/812050597 conn(0x7f7dc40d12d0 msgr2=0x7f7dc40b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 shutdown_connections 2023-10-26T05:53:55.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.095+0000 7f7dcb216640 1 -- 172.21.15.71:0/812050597 wait complete. 2023-10-26T05:53:55.117 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T05:53:55.237 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.813543979,"mdsmap_epoch":51,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":19.660238066000002} 2023-10-26T05:53:55.238 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 8007, "mdsmap_epoch": 51, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 14.813543979, "state": "up:active", "uptime": 19.660238066, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:53:55.238 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T05:53:55.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa2067fc640 1 -- 172.21.15.71:0/1614970098 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2000ad940 con 0x7fa20815bf70 2023-10-26T05:53:55.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 -- 172.21.15.71:0/1614970098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bf70 msgr2=0x7fa2080ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/1614970098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bf70 0x7fa2080ca6d0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa200009c80 tx=0x7fa20001cd50 comp rx=0 tx=0).stop 2023-10-26T05:53:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 -- 172.21.15.71:0/1614970098 shutdown_connections 2023-10-26T05:53:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/1614970098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bf70 0x7fa2080ca6d0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/1614970098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2080b0fa0 0x7fa208070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 -- 172.21.15.71:0/1614970098 >> 172.21.15.71:0/1614970098 conn(0x7fa2080aec60 msgr2=0x7fa2080af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 -- 172.21.15.71:0/1614970098 shutdown_connections 2023-10-26T05:53:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 -- 172.21.15.71:0/1614970098 wait complete. 2023-10-26T05:53:55.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 Processor -- start 2023-10-26T05:53:55.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 -- start start 2023-10-26T05:53:55.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.359+0000 7fa20d6d8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2080b0fa0 0x7fa2081348d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 0x7fa208135200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20815bf70 0x7fa208140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa208134e10 con 0x7fa20815bb90 2023-10-26T05:53:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20813fdc0 con 0x7fa20815bf70 2023-10-26T05:53:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa208141150 con 0x7fa2080b0fa0 2023-10-26T05:53:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 0x7fa208135200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 0x7fa208135200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38114/0 (socket says 172.21.15.71:38114) 2023-10-26T05:53:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 -- 172.21.15.71:0/2763914145 learned_addr learned my addr 172.21.15.71:0/2763914145 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa206ffd640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2080b0fa0 0x7fa2081348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 -- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2080b0fa0 msgr2=0x7fa2081348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2080b0fa0 0x7fa2081348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 -- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20815bf70 msgr2=0x7fa208140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:55.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20815bf70 0x7fa208140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:55.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 -- 172.21.15.71:0/2763914145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000098e0 con 0x7fa20815bb90 2023-10-26T05:53:55.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fe7fc640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 0x7fa208135200 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000d980 tx=0x7fa1f000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:55.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0009c30 con 0x7fa20815bb90 2023-10-26T05:53:55.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1f0005020 con 0x7fa20815bb90 2023-10-26T05:53:55.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa208155d40 con 0x7fa20815bb90 2023-10-26T05:53:55.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0005430 con 0x7fa20815bb90 2023-10-26T05:53:55.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa208140de0 con 0x7fa20815bb90 2023-10-26T05:53:55.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa208066b10 con 0x7fa20815bb90 2023-10-26T05:53:55.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa1f00047b0 con 0x7fa20815bb90 2023-10-26T05:53:55.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fffff640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1e8066470 0x7fa1e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:55.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f00840f0 con 0x7fa20815bb90 2023-10-26T05:53:55.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa206ffd640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1e8066470 0x7fa1e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:55.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.363+0000 7fa206ffd640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1e8066470 0x7fa1e8068930 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8001650 tx=0x7fa1f8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:55.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.367+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa1f0051570 con 0x7fa20815bb90 2023-10-26T05:53:55.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:55.523+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fa20813ff30 con 0x7fa20815bb90 2023-10-26T05:53:56.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.187+0000 7fa1fffff640 1 -- 172.21.15.71:0/2763914145 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v139) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa1f00200c0 con 0x7fa20815bb90 2023-10-26T05:53:56.196 INFO:teuthology.orchestra.run.smithi071.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T05:53:56.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1e8066470 msgr2=0x7fa1e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:56.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1e8066470 0x7fa1e8068930 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8001650 tx=0x7fa1f8009040 comp rx=0 tx=0).stop 2023-10-26T05:53:56.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 msgr2=0x7fa208135200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 0x7fa208135200 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000d980 tx=0x7fa1f000de40 comp rx=0 tx=0).stop 2023-10-26T05:53:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 shutdown_connections 2023-10-26T05:53:56.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa20815bf70 0x7fa208140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1e8066470 0x7fa1e8068930 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20815bb90 0x7fa208135200 secure :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000d980 tx=0x7fa1f000de40 comp rx=0 tx=0).stop 2023-10-26T05:53:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 --2- 172.21.15.71:0/2763914145 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa2080b0fa0 0x7fa2081348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 >> 172.21.15.71:0/2763914145 conn(0x7fa2080aec60 msgr2=0x7fa208156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:56.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 shutdown_connections 2023-10-26T05:53:56.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.191+0000 7fa20d6d8640 1 -- 172.21.15.71:0/2763914145 wait complete. 2023-10-26T05:53:56.219 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:53:56.354 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:53:56.354 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:53:56.355 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-26T05:53:56.355 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:56.356 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:56.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:56.362 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-26T05:53:56.530 INFO:teuthology.orchestra.run.smithi071.stderr:16+0 records in 2023-10-26T05:53:56.530 INFO:teuthology.orchestra.run.smithi071.stderr:16+0 records out 2023-10-26T05:53:56.530 INFO:teuthology.orchestra.run.smithi071.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.112144 s, 150 MB/s 2023-10-26T05:53:56.531 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:53:56.531 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:53:56.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:53:56.537 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-26T05:53:56.726 INFO:teuthology.orchestra.run.smithi071.stderr:17+0 records in 2023-10-26T05:53:56.726 INFO:teuthology.orchestra.run.smithi071.stderr:17+0 records out 2023-10-26T05:53:56.727 INFO:teuthology.orchestra.run.smithi071.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.137139 s, 130 MB/s 2023-10-26T05:53:56.728 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T05:53:56.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:53:56.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 -- 172.21.15.71:0/689064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0415bbc0 msgr2=0x7f4e04070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:56.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 --2- 172.21.15.71:0/689064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0415bbc0 0x7f4e04070690 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009900 tx=0x7f4dfc01c880 comp rx=0 tx=0).stop 2023-10-26T05:53:56.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 -- 172.21.15.71:0/689064 shutdown_connections 2023-10-26T05:53:56.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 --2- 172.21.15.71:0/689064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0415bbc0 0x7f4e04070690 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 --2- 172.21.15.71:0/689064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d420 0x7f4e040ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 -- 172.21.15.71:0/689064 >> 172.21.15.71:0/689064 conn(0x7f4e04069770 msgr2=0x7f4e04069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:56.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 -- 172.21.15.71:0/689064 shutdown_connections 2023-10-26T05:53:56.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 -- 172.21.15.71:0/689064 wait complete. 2023-10-26T05:53:56.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 Processor -- start 2023-10-26T05:53:56.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.851+0000 7f4e08f68640 1 -- start start 2023-10-26T05:53:56.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 0x7f4e041359d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:56.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d800 0x7f4e04156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:56.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e0415bbc0 0x7f4e04137c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:56.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e04135f10 con 0x7f4e0413d420 2023-10-26T05:53:56.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e04134f80 con 0x7f4e0415bbc0 2023-10-26T05:53:56.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e04135540 con 0x7f4e0413d800 2023-10-26T05:53:56.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 0x7f4e041359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:56.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 0x7f4e041359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38136/0 (socket says 172.21.15.71:38136) 2023-10-26T05:53:56.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 -- 172.21.15.71:0/1963921806 learned_addr learned my addr 172.21.15.71:0/1963921806 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:53:56.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e02d9d640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e0415bbc0 0x7f4e04137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:56.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e01d9b640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d800 0x7f4e04156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:56.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 -- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d800 msgr2=0x7f4e04156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:56.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d800 0x7f4e04156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 -- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e0415bbc0 msgr2=0x7f4e04137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:56.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e0415bbc0 0x7f4e04137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:56.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 -- 172.21.15.71:0/1963921806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc009560 con 0x7f4e0413d420 2023-10-26T05:53:56.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e01d9b640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d800 0x7f4e04156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:53:56.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e0259c640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 0x7f4e041359d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4df400ccb0 tx=0x7f4df400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:56.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4004b30 con 0x7f4e0413d420 2023-10-26T05:53:56.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4df4004d80 con 0x7f4e0413d420 2023-10-26T05:53:56.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0413a880 con 0x7f4e0413d420 2023-10-26T05:53:56.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4005280 con 0x7f4e0413d420 2023-10-26T05:53:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e041351d0 con 0x7f4e0413d420 2023-10-26T05:53:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e04066b10 con 0x7f4e0413d420 2023-10-26T05:53:56.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4df40053e0 con 0x7f4e0413d420 2023-10-26T05:53:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4df2ffd640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4de4066470 0x7f4de4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:53:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4df4084880 con 0x7f4e0413d420 2023-10-26T05:53:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.855+0000 7f4e01d9b640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4de4066470 0x7f4de4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:53:56.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.859+0000 7f4e01d9b640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4de4066470 0x7f4de4068930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4e0415b330 tx=0x7f4df8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:53:56.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:56.859+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4df4051c60 con 0x7f4e0413d420 2023-10-26T05:53:57.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.015+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4e040721e0 con 0x7f4e0413d420 2023-10-26T05:53:57.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.019+0000 7f4df2ffd640 1 -- 172.21.15.71:0/1963921806 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) v1 ==== 81+0+33886 (secure 0 0 0) 0x7f4df401d020 con 0x7f4e0413d420 2023-10-26T05:53:57.028 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:53:57.028 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:53:57.028 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 139, 2023-10-26T05:53:57.028 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:53:57.029 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:53:57.029 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:53:56.181377+0000", 2023-10-26T05:53:57.029 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:53:57.029 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:53:57.030 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:53:57.030 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:53:57.030 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:53:57.030 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:53:57.030 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:53:57.031 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:53:57.031 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:53:57.031 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:53:57.031 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:53:57.031 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:53:57.032 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:53:57.032 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:53:57.032 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:53:57.032 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:53:57.032 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:53:57.033 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:53:57.033 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:53:57.033 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:53:57.033 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:53:57.033 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:53:57.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.034 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:53:57.034 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:53:57.034 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:53:57.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:53:57.035 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:53:57.035 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:53:57.035 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:53:57.035 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:53:57.035 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:53:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:53:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:53:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:53:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:53:57.036 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:53:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:53:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:53:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:53:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:53:57.037 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:53:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:53:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:53:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:53:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:53:57.038 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:53:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:53:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:53:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:53:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.039 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:53:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:53:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:53:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:53:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:53:57.040 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:53:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:53:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:53:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:53:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:53:57.041 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:53:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:53:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:53:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:53:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:53:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:53:57.042 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:53:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:53:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:53:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:53:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:53:57.043 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:53:57.044 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:53:57.044 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:53:57.044 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:53:57.044 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:53:57.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:53:57.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.045 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:53:57.045 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:53:57.045 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:53:57.046 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:53:57.046 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:53:57.046 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:53:57.046 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:53:57.046 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:53:57.047 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:53:57.047 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:53:57.047 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:53:57.047 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:53:57.047 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:53:57.048 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:53:57.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.048 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:53:57.048 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:53:57.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.049 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:53:57.049 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:53:57.049 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:53:57.049 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:53:57.049 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:53:57.049 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:53:57.050 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:53:57.050 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:53:57.050 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:53:57.050 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.051 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:53:57.051 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:53:57.051 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:53:57.051 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:53:57.052 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:53:57.052 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:53:57.052 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:53:57.052 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:53:57.052 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:53:57.053 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:53:57.053 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:53:57.053 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:53:57.053 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:53:57.053 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:53:57.054 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:53:57.054 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:53:57.054 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:53:57.054 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:53:57.054 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:53:57.055 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:53:57.055 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:53:57.055 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:53:57.055 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:53:57.055 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:53:57.056 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:53:57.056 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:53:57.056 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:53:57.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.056 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:53:57.056 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:53:57.057 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:53:57.057 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:53:57.057 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:53:57.057 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:53:57.057 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:53:57.058 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:53:57.058 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:53:57.058 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:53:57.058 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:53:57.058 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:53:57.059 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:53:57.059 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:53:57.059 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:53:57.059 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:53:57.059 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:53:57.059 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:53:57.060 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:53:57.060 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:53:57.060 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:53:57.060 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:53:57.060 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:53:57.061 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:53:57.061 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:53:57.061 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:53:57.061 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:53:57.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.062 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:53:57.062 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:53:57.062 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:53:57.062 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:53:57.062 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:53:57.063 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:53:57.063 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:53:57.063 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:53:57.063 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:53:57.063 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:53:57.063 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:53:57.064 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:53:57.064 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:53:57.064 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:53:57.064 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:53:57.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:53:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:53:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:53:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:53:57.066 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:53:57.066 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:53:57.066 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:53:57.066 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:53:57.066 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:53:57.066 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:53:57.067 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:53:57.067 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:53:57.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.067 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:53:57.068 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:53:57.068 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:53:57.068 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:53:57.068 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:53:57.068 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:53:57.069 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:53:57.069 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:53:57.069 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:53:57.069 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:53:57.069 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:53:57.069 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:53:57.070 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:53:57.070 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:53:57.070 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:53:57.070 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:53:57.070 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:53:57.071 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:53:57.071 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:53:57.071 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:53:57.071 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:53:57.071 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:53:57.071 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:53:57.072 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:53:57.072 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:53:57.072 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:53:57.072 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:53:57.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.073 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "139", 2023-10-26T05:53:57.073 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:53:57.073 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:53:57.073 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:53:57.073 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:53:57.074 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:53:57.074 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:53:57.074 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:53:57.074 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:53:57.074 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:53:57.075 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:53:57.075 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:53:57.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:53:57.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:53:57.075 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:53:57.075 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:53:57.076 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:53:57.076 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:53:57.076 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:53:57.076 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:53:57.076 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:53:57.077 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:53:57.077 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:53:57.077 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:53:57.077 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:53:57.077 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:53:57.077 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:53:57.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.078 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:53:57.078 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:53:57.078 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:53:57.078 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:53:57.079 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:53:57.079 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:53:57.079 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:53:57.079 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:53:57.079 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:53:57.079 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:53:57.080 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:53:57.080 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:53:57.080 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:53:57.080 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:53:57.080 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.081 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:53:57.081 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:53:57.081 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:53:57.081 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.082 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:53:57.082 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:53:57.082 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:53:57.082 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:53:57.082 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:53:57.083 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:53:57.083 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:53:57.083 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:53:57.083 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:53:57.083 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.084 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.084 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:53:57.084 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:53:57.084 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.084 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:53:57.084 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.085 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.085 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.085 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.085 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.085 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.085 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.086 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.086 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.086 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.086 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.086 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.087 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.087 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.087 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:53:57.087 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.088 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.088 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:53:57.088 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.088 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.088 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.089 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:53:57.090 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.090 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.090 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.090 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.090 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:53:57.091 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.091 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.091 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.091 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:53:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.093 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.093 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:53:57.093 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.093 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.093 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.094 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.094 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.094 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.094 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.095 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:53:57.095 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.095 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.095 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.095 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:53:57.096 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:53:57.096 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.096 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.096 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:53:57.096 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:53:57.097 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:53:57.097 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:53:57.097 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.097 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.097 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.097 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.098 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.098 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:53:57.098 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.098 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.099 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.099 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.099 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.099 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.099 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.099 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.100 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.100 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.100 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.100 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.100 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.101 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.101 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.101 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:53:57.101 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.102 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.102 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.102 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:53:57.102 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.102 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.103 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.103 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.103 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.103 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.103 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.103 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:53:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.104 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:53:57.105 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.105 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.105 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.105 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.105 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.106 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.106 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.106 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:53:57.106 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.107 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.107 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.107 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:53:57.107 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.107 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.108 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.108 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.108 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.108 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.109 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.109 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:53:57.109 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.109 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.110 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.110 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:53:57.110 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:53:57.110 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.110 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.111 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.111 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:53:57.111 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:53:57.111 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:53:57.111 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:53:57.111 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.112 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.112 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.112 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.112 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.113 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:53:57.113 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.113 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.113 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.113 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.114 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.115 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.115 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.115 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.115 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.115 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.116 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:53:57.116 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.116 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.116 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:53:57.117 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.117 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.117 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.117 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.117 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.118 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.118 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.118 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:53:57.118 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.118 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.119 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.119 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:53:57.119 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.120 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.120 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.120 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.120 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:53:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.121 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.122 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.122 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:53:57.122 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.122 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.122 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:53:57.124 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.124 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.124 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.124 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:53:57.124 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:53:57.125 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.125 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.125 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:53:57.125 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:53:57.126 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:53:57.126 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:53:57.126 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.126 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.126 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:53:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.127 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.128 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.128 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.128 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.128 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.129 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.129 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.129 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.129 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.129 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.129 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.130 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.130 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.130 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:53:57.130 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.131 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.131 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.131 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:53:57.131 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.131 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.132 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.132 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.132 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.132 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.133 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:53:57.133 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.133 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.133 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.134 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.134 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:53:57.134 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.134 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.134 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.135 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.135 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.135 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.135 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.135 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:53:57.136 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.136 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.136 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.136 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:53:57.136 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.137 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.137 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.137 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.137 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.138 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.138 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.138 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:53:57.138 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.138 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.139 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.139 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.139 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:53:57.139 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:53:57.139 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.140 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.140 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:53:57.140 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:53:57.140 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:53:57.141 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:53:57.141 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.141 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.141 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.141 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.142 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.142 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:53:57.142 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.142 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.142 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.143 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.143 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.143 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.143 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.144 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.144 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.144 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.144 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.144 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.145 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.145 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:53:57.145 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.145 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.146 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.146 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:53:57.146 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.146 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.146 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.147 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.147 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.147 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.148 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:53:57.148 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.148 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.148 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.149 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:53:57.149 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.149 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.149 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.149 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.150 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.150 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.150 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:53:57.151 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.151 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.151 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:53:57.151 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.152 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.152 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.152 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.152 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.153 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.153 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.153 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:53:57.153 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.153 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:53:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:53:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.154 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.155 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:53:57.155 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:53:57.155 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:53:57.155 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:53:57.155 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.156 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.156 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.156 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.157 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:53:57.157 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.157 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.157 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.157 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.157 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.158 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.158 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.158 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.158 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.158 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.159 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:53:57.160 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.160 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.160 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.160 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:53:57.161 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.161 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.161 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.161 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.161 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.161 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.162 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.162 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:53:57.162 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.163 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.163 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:53:57.163 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.163 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.164 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.164 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.164 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.164 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.165 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:53:57.165 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.166 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:53:57.166 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.166 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.166 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.167 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.167 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.167 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.167 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:53:57.168 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.168 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.168 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:53:57.168 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:53:57.169 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.169 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.169 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:53:57.169 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:53:57.169 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:53:57.170 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:53:57.170 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.170 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.170 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.170 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.171 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.171 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:53:57.171 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.171 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.172 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.173 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.173 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.173 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.173 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.173 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.174 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.174 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.174 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:53:57.174 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:53:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.175 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.176 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.176 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.177 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:53:57.177 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.177 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.178 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:53:57.178 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.178 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.178 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.178 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.178 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.179 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.179 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:53:57.179 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.180 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.180 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.180 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:53:57.180 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.181 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.181 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.181 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.181 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.182 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.182 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:53:57.182 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.183 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:53:57.183 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:53:57.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.183 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.184 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:53:57.184 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:53:57.184 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:53:57.184 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:53:57.185 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.185 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.185 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.185 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.185 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.185 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.186 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:53:57.186 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:53:57.186 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:53:57.186 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:53:57.186 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:53:57.187 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:53:57.187 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:53:57.187 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:53:57.187 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:53:57.187 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:53:57.188 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:53:57.188 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:53:57.188 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:53:57.188 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.188 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.188 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.189 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:53:57.189 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.189 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.190 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:53:57.190 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.190 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.190 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.191 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:53:57.191 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.191 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.191 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:53:57.192 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.192 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.192 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:53:57.192 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.193 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.193 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:53:57.193 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.194 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:53:57.194 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.195 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.195 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:53:57.195 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.195 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.196 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.196 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:53:57.196 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:53:57.196 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:53:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:53:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:53:57.198 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:53:57.198 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:53:57.198 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.198 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.199 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:53:57.199 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:53:57.199 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:53:57.199 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:53:57.199 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:53:57.200 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:53:57.200 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:53:57.200 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:53:57.200 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.200 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:53:57.201 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:53:57.201 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.201 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:53:57.201 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.201 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.201 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:53:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.203 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.203 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:53:57.203 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.203 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.203 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.204 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.204 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.204 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:53:57.204 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.204 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.205 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:53:57.205 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.205 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.205 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.205 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.206 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.206 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:53:57.206 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.206 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.207 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:53:57.207 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.207 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.207 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.207 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.208 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.208 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:53:57.208 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.208 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.208 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:53:57.209 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.209 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.209 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.209 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.209 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.210 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:53:57.210 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.210 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.210 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:53:57.210 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.211 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.211 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.211 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.211 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.211 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:53:57.212 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.212 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.212 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:53:57.212 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.213 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.213 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.213 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.213 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.213 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:53:57.213 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.214 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:53:57.214 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:53:57.214 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:53:57.214 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:53:57.215 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:53:57.215 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:53:57.215 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:53:57.215 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:53:57.215 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:53:57.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.216 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:53:57.216 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:53:57.216 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:53:57.216 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:53:57.216 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:53:57.217 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:53:57.217 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:53:57.217 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:53:57.217 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:53:57.217 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:53:57.218 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:53:57.218 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:53:57.218 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:53:57.218 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:53:57.218 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:53:57.219 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:53:57.219 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:53:57.219 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:53:57.219 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:53:57.219 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:53:57.220 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:53:57.220 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:53:57.220 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:53:57.220 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:53:57.220 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:53:57.221 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:53:57.221 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:53:57.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:53:57.221 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:53:57.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4de4066470 msgr2=0x7f4de4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:57.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4de4066470 0x7f4de4068930 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4e0415b330 tx=0x7f4df8009040 comp rx=0 tx=0).stop 2023-10-26T05:53:57.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 msgr2=0x7f4e041359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:53:57.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 0x7f4e041359d0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f4df400ccb0 tx=0x7f4df400a620 comp rx=0 tx=0).stop 2023-10-26T05:53:57.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 shutdown_connections 2023-10-26T05:53:57.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e0415bbc0 0x7f4e04137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:57.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e0413d800 0x7f4e04156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:57.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4de4066470 0x7f4de4068930 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:57.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 --2- 172.21.15.71:0/1963921806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e0413d420 0x7f4e041359d0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:53:57.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 >> 172.21.15.71:0/1963921806 conn(0x7f4e04069770 msgr2=0x7f4e040cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:53:57.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 shutdown_connections 2023-10-26T05:53:57.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:53:57.023+0000 7f4e08f68640 1 -- 172.21.15.71:0/1963921806 wait complete. 2023-10-26T05:53:57.227 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:54:02.229 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:54:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/983774832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180b0040 msgr2=0x7f4a180b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/983774832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180b0040 0x7f4a180b04b0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4a1806ed70 tx=0x7f4a1001cd30 comp rx=0 tx=0).stop 2023-10-26T05:54:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/983774832 shutdown_connections 2023-10-26T05:54:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/983774832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a18070fe0 0x7f4a18071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/983774832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a180b0040 0x7f4a180b04b0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/983774832 >> v2:172.21.15.86:3300/0 conn(0x7f4a18140cc0 0x7f4a1815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/983774832 >> 172.21.15.71:0/983774832 conn(0x7f4a18069770 msgr2=0x7f4a18069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/983774832 shutdown_connections 2023-10-26T05:54:02.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/983774832 wait complete. 2023-10-26T05:54:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 Processor -- start 2023-10-26T05:54:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- start start 2023-10-26T05:54:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 0x7f4a18151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a180b0040 0x7f4a1813b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a18140cc0 0x7f4a18069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1813fc80 con 0x7f4a18070fe0 2023-10-26T05:54:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1813f070 con 0x7f4a18140cc0 2023-10-26T05:54:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1813f350 con 0x7f4a180b0040 2023-10-26T05:54:02.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 0x7f4a18151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:02.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 0x7f4a18151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51514/0 (socket says 172.21.15.71:51514) 2023-10-26T05:54:02.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 -- 172.21.15.71:0/4163002078 learned_addr learned my addr 172.21.15.71:0/4163002078 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:02.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1d320640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a180b0040 0x7f4a1813b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:02.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1e322640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a18140cc0 0x7f4a18069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:02.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 -- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a180b0040 msgr2=0x7f4a1813b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:02.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a180b0040 0x7f4a1813b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 -- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a18140cc0 msgr2=0x7f4a18069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:02.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a18140cc0 0x7f4a18069b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 -- 172.21.15.71:0/4163002078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a100098e0 con 0x7f4a18070fe0 2023-10-26T05:54:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1db21640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 0x7f4a18151a20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4a140058d0 tx=0x7f4a1400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14009c30 con 0x7f4a18070fe0 2023-10-26T05:54:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a14009e80 con 0x7f4a18070fe0 2023-10-26T05:54:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140106a0 con 0x7f4a18070fe0 2023-10-26T05:54:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1813f630 con 0x7f4a18070fe0 2023-10-26T05:54:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.359+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1813ecd0 con 0x7f4a18070fe0 2023-10-26T05:54:02.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18066b10 con 0x7f4a18070fe0 2023-10-26T05:54:02.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4a14010800 con 0x7f4a18070fe0 2023-10-26T05:54:02.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a0e7fc640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc0664a0 0x7f49fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:02.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a140843c0 con 0x7f4a18070fe0 2023-10-26T05:54:02.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f49fc06dfd0 con 0x7f4a18070fe0 2023-10-26T05:54:02.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a1d320640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc0664a0 0x7f49fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:02.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.363+0000 7f4a1d320640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc0664a0 0x7f49fc068960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005fd0 tx=0x7f4a08005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:02.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.367+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4a140517c0 con 0x7f4a18070fe0 2023-10-26T05:54:02.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.535+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4a180721e0 con 0x7f4a18070fe0 2023-10-26T05:54:02.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.535+0000 7f4a0e7fc640 1 -- 172.21.15.71:0/4163002078 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+33905 (secure 0 0 0) 0x7f4a14021090 con 0x7f4a18070fe0 2023-10-26T05:54:02.546 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:02.546 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:02.546 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 140, 2023-10-26T05:54:02.547 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:02.547 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:54:02.547 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:54:02.174155+0000", 2023-10-26T05:54:02.547 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:54:02.547 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:54:02.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:54:02.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:54:02.548 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:54:02.548 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:54:02.548 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:54:02.548 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:54:02.549 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:54:02.549 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:02.549 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:54:02.549 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:54:02.549 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:54:02.550 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:54:02.550 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:54:02.550 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:54:02.550 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:54:02.550 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:54:02.551 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:54:02.551 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:54:02.551 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:54:02.551 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:54:02.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.552 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:54:02.552 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:54:02.552 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:54:02.552 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:02.552 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:02.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:02.553 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:02.553 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:02.553 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:02.553 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:02.553 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:02.554 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:02.554 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:02.554 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:02.554 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:02.554 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:54:02.555 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:54:02.555 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:54:02.555 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:54:02.555 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:54:02.555 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:02.556 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:02.556 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:02.556 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:02.556 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:02.557 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:02.557 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:02.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.557 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:54:02.558 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:02.558 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:02.558 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:02.558 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:02.559 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:02.559 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:02.559 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:02.560 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:02.560 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:02.560 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:02.560 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:02.561 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:02.561 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:02.561 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:02.562 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:02.562 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:02.563 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:02.563 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:02.563 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:02.564 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:02.564 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:02.564 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:02.565 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:02.565 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:02.565 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:02.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:02.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.566 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:02.567 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:02.567 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:02.568 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:02.568 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:02.568 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:02.568 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:02.569 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:02.569 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:02.569 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:02.569 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:02.569 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:02.570 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:54:02.570 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:54:02.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.570 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:02.570 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:54:02.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.571 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:02.571 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:54:02.571 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:54:02.571 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:54:02.572 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:54:02.572 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:54:02.572 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:02.572 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:02.572 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:02.573 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.573 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.573 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:54:02.573 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:54:02.573 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:54:02.574 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:02.574 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:02.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:02.574 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:02.574 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:02.575 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:02.575 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:02.575 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:02.575 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:02.575 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:02.576 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:02.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:02.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:54:02.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:54:02.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:54:02.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:54:02.577 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:54:02.577 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:02.577 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:02.577 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:02.577 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:02.578 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:02.578 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:02.578 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:02.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.578 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:54:02.579 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:02.579 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:02.579 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:02.579 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:02.579 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:02.580 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:02.580 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:02.580 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:02.580 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:02.580 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:02.581 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:02.581 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:02.581 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:02.581 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:02.581 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:02.581 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:02.582 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:02.582 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:02.582 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:02.582 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:02.582 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:02.583 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:02.583 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:02.583 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:02.583 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:02.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:02.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.584 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:02.584 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:02.584 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:02.584 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:02.584 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:02.585 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:02.585 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:02.585 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:02.585 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:02.585 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:02.586 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:02.586 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:02.586 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:54:02.586 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:54:02.586 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:54:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:54:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:02.588 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:02.589 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:02.589 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:02.589 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:02.589 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:02.589 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:02.590 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.590 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:54:02.590 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:54:02.591 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:54:02.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:54:02.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:54:02.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:02.593 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:02.593 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:02.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:02.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:54:02.593 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:54:02.594 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:54:02.594 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:54:02.594 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:54:02.594 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:02.594 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:02.595 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:02.595 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:02.595 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:02.595 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:02.596 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:02.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.596 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "140", 2023-10-26T05:54:02.596 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:02.596 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:02.597 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:02.598 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:02.598 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:54:02.598 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:02.599 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:02.599 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:02.599 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:02.600 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:02.600 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:02.600 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:02.601 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:02.601 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:02.601 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:02.602 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:02.602 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:02.602 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:02.603 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:02.603 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:02.603 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:02.605 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:02.605 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:02.605 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:02.606 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:02.606 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:02.606 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:02.607 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:02.607 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:02.607 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:02.608 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:02.608 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:02.608 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:54:02.609 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:54:02.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.610 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:02.610 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:02.610 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:54:02.611 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.611 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:02.611 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:02.612 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:02.612 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:02.612 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:02.612 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:02.613 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:02.613 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:02.613 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:02.613 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.614 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.614 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:02.614 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:54:02.615 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.615 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:02.615 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.615 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.616 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.617 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.617 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.617 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.617 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.617 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.618 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.618 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.618 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:54:02.618 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.619 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.619 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:54:02.619 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.620 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.620 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.620 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.620 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.621 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:54:02.621 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.621 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.622 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:54:02.622 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.622 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.623 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.623 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.623 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.623 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:54:02.623 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.624 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.624 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.624 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:54:02.624 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.625 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.625 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.625 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.625 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.626 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.626 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:54:02.626 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.626 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.627 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.627 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:54:02.627 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:02.627 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.627 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.628 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:54:02.628 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:54:02.628 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:54:02.628 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:54:02.628 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.629 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.629 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.629 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.629 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.630 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:02.630 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.630 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.630 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.630 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.631 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.631 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.631 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.631 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.631 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.632 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.632 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.632 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.632 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.632 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.633 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.633 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:54:02.633 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.633 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.633 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.634 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:54:02.634 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.634 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.634 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.635 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.635 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.635 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.635 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.636 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:54:02.636 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.636 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.636 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:54:02.637 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.637 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.637 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.637 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.638 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.638 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.638 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:54:02.638 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.639 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.639 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.639 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.639 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:54:02.639 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.639 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.640 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.640 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.640 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.640 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.641 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:54:02.641 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.641 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.641 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.642 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:54:02.642 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:02.642 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.642 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.643 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:54:02.643 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:54:02.643 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:54:02.643 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:54:02.643 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.644 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:02.645 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.645 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.645 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.645 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.645 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.646 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.646 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.646 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.646 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.646 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.647 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.647 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.647 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.647 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.647 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:54:02.648 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.648 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.648 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.648 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:54:02.649 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.649 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.649 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.649 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.649 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.650 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.650 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.650 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:54:02.650 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.650 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.651 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.651 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:54:02.651 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.651 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.652 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.652 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.652 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.652 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.652 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.653 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:54:02.653 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.653 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.653 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:54:02.654 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.654 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.654 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.654 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.655 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.655 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.655 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.655 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:54:02.655 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.656 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.656 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.656 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:54:02.656 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:02.656 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.657 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.657 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:54:02.657 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:54:02.657 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:54:02.658 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:54:02.658 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.658 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.658 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.658 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.658 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.659 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.659 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:02.659 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.659 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.659 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.660 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.660 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.660 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.660 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.660 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.661 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.661 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.661 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.661 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.661 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.662 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:54:02.662 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.663 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.663 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:54:02.663 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.663 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.663 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.664 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.664 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.664 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.664 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:54:02.664 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:54:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.665 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.666 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.666 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.666 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.666 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.667 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:54:02.667 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.667 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.667 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.668 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:54:02.668 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.668 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.668 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.669 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.669 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.669 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.669 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:54:02.669 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.670 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.670 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:54:02.670 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:02.671 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.671 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.671 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.671 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:54:02.672 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:54:02.672 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:54:02.672 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:54:02.672 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.672 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.672 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.673 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.673 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.673 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.673 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:02.673 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.674 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.674 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.674 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.674 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.674 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.675 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.675 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.675 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.675 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.675 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.675 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.676 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.676 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.676 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:54:02.676 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.677 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.677 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.677 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:54:02.677 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.677 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.678 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.678 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.678 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.678 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.678 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.679 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:54:02.679 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.679 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.680 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.680 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:54:02.680 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.680 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.680 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.681 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.681 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.681 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:54:02.682 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.682 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.682 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.682 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.682 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:54:02.682 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.683 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.683 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.683 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.683 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.684 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:54:02.684 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.684 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.685 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:54:02.685 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:02.685 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.685 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.686 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:54:02.686 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:54:02.686 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:54:02.686 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:54:02.687 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.687 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.687 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.687 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.688 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.688 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:02.688 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.688 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.688 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.689 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.689 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.689 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.689 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.690 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.690 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.690 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.690 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.691 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.691 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.691 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:54:02.692 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.692 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:54:02.693 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.693 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.693 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.694 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:54:02.694 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.695 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.695 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.695 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.695 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:54:02.695 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.696 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.697 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.697 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.698 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:54:02.698 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.698 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.699 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:54:02.699 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.700 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.700 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.701 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.701 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.701 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.702 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:54:02.703 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.703 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.704 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:54:02.704 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:02.704 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.705 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.706 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:54:02.706 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:54:02.706 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:54:02.707 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:54:02.707 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.707 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.707 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.708 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.708 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:02.708 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.709 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.709 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.709 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.709 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.709 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.710 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.710 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.710 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.710 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.710 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.711 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.711 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.711 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:54:02.711 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.712 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.712 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:54:02.712 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.713 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.713 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.713 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.713 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.714 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.714 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:54:02.714 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.715 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:54:02.715 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.715 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.716 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.716 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.716 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:54:02.717 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.717 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.717 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:54:02.718 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.718 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.718 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.718 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.719 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.719 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:54:02.719 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.720 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.720 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:54:02.720 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:02.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.721 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.721 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.721 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:54:02.721 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:54:02.721 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:54:02.722 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:54:02.722 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.722 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.722 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.722 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.723 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.723 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:02.723 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:02.723 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:02.723 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:02.724 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:02.724 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:02.724 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:02.724 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:02.724 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:02.724 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:02.725 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:02.725 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:02.725 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:02.725 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.725 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.726 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.726 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:54:02.726 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.726 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.727 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:54:02.727 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.728 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.728 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.728 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.728 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:02.728 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.728 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:54:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:54:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:54:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:54:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:54:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:54:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:54:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:54:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:54:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:54:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:54:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:54:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:54:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:54:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:54:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:54:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:54:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:54:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:54:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:54:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:54:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:54:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:54:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:54:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:54:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:54:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:54:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:54:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:54:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:54:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:54:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:54:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:54:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:54:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:54:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:54:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:54:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:54:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:54:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:54:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:54:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:54:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:54:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:54:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:02.760 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc0664a0 msgr2=0x7f49fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc0664a0 0x7f49fc068960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005fd0 tx=0x7f4a08005e60 comp rx=0 tx=0).stop 2023-10-26T05:54:02.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 msgr2=0x7f4a18151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:02.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 0x7f4a18151a20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4a140058d0 tx=0x7f4a1400dde0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 shutdown_connections 2023-10-26T05:54:02.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a18140cc0 0x7f4a18069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a180b0040 0x7f4a1813b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc0664a0 0x7f49fc068960 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 --2- 172.21.15.71:0/4163002078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a18070fe0 0x7f4a18151a20 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:02.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 >> 172.21.15.71:0/4163002078 conn(0x7f4a18069770 msgr2=0x7f4a180b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:02.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 shutdown_connections 2023-10-26T05:54:02.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:02.539+0000 7f4a1fd85640 1 -- 172.21.15.71:0/4163002078 wait complete. 2023-10-26T05:54:02.765 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-10-26T05:54:02.765 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:54:02.766 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:02.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:02.773 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-26T05:54:02.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:54:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:dd: error writing 'large_file_b': No space left on device 2023-10-26T05:54:02.892 INFO:teuthology.orchestra.run.smithi071.stderr:dd: closing output file 'large_file_b': No space left on device 2023-10-26T05:54:02.893 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:54:03.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- 172.21.15.71:0/248149718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580cd240 msgr2=0x7f425815ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- 172.21.15.71:0/248149718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580cd240 0x7f425815ce40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f42580aff30 tx=0x7f425001ca10 comp rx=0 tx=0).stop 2023-10-26T05:54:03.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- 172.21.15.71:0/248149718 shutdown_connections 2023-10-26T05:54:03.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- 172.21.15.71:0/248149718 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f42580b0420 0x7f42580ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- 172.21.15.71:0/248149718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580cd240 0x7f425815ce40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- 172.21.15.71:0/248149718 >> v2:172.21.15.86:3300/0 conn(0x7f4258159210 0x7f42580ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- 172.21.15.71:0/248149718 >> 172.21.15.71:0/248149718 conn(0x7f42580b0c80 msgr2=0x7f42580b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:03.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- 172.21.15.71:0/248149718 shutdown_connections 2023-10-26T05:54:03.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- 172.21.15.71:0/248149718 wait complete. 2023-10-26T05:54:03.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 Processor -- start 2023-10-26T05:54:03.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- start start 2023-10-26T05:54:03.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 0x7f425806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42580cd240 0x7f42580c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4258159210 0x7f42581503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258153e00 con 0x7f42580b0420 2023-10-26T05:54:03.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258153180 con 0x7f42580cd240 2023-10-26T05:54:03.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425f9cb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4258153460 con 0x7f4258159210 2023-10-26T05:54:03.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 0x7f425806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 0x7f425806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51536/0 (socket says 172.21.15.71:51536) 2023-10-26T05:54:03.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 -- 172.21.15.71:0/1812411838 learned_addr learned my addr 172.21.15.71:0/1812411838 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:03.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425cf66640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42580cd240 0x7f42580c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425df68640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4258159210 0x7f42581503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 -- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4258159210 msgr2=0x7f42581503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4258159210 0x7f42581503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 -- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42580cd240 msgr2=0x7f42580c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42580cd240 0x7f42580c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 -- 172.21.15.71:0/1812411838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500098e0 con 0x7f42580b0420 2023-10-26T05:54:03.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.011+0000 7f425d767640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 0x7f425806e440 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f425400c940 tx=0x7f425400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:03.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254004220 con 0x7f42580b0420 2023-10-26T05:54:03.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4254004470 con 0x7f42580b0420 2023-10-26T05:54:03.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258153740 con 0x7f42580b0420 2023-10-26T05:54:03.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254007bf0 con 0x7f42580b0420 2023-10-26T05:54:03.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425806e980 con 0x7f42580b0420 2023-10-26T05:54:03.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4258066b10 con 0x7f42580b0420 2023-10-26T05:54:03.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4254007d50 con 0x7f42580b0420 2023-10-26T05:54:03.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c0664a0 0x7f423c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4254083f90 con 0x7f42580b0420 2023-10-26T05:54:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7f423c06dfd0 con 0x7f42580b0420 2023-10-26T05:54:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f425cf66640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c0664a0 0x7f423c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.015+0000 7f425cf66640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c0664a0 0x7f423c068960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4248006fd0 tx=0x7f4248008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.019+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f42540b5090 con 0x7f42580b0420 2023-10-26T05:54:03.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.171+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f42580721e0 con 0x7f42580b0420 2023-10-26T05:54:03.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f424dffb640 1 -- 172.21.15.71:0/1812411838 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+33905 (secure 0 0 0) 0x7f425401d020 con 0x7f42580b0420 2023-10-26T05:54:03.183 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:03.183 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 140, 2023-10-26T05:54:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:54:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:54:02.174155+0000", 2023-10-26T05:54:03.184 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:54:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:54:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:54:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:54:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:54:03.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:54:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:54:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:54:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:54:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:03.186 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:54:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:54:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:54:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:54:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:54:03.187 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:54:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:54:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:54:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:54:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:54:03.188 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:54:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:54:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:54:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:54:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:54:03.189 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:03.190 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:03.191 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:54:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:54:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:54:03.192 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:54:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:54:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:03.193 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.194 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:54:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:03.195 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:03.196 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:03.197 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:03.198 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:03.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:03.200 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:03.201 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:54:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:54:03.202 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:54:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:03.203 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:54:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:54:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:54:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:54:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:54:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:03.204 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:54:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:54:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:54:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:03.206 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:03.207 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:54:03.208 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:54:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:54:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:54:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:54:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:03.209 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:03.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:54:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:03.211 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:03.212 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:03.213 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:03.214 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:03.215 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:03.216 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:03.217 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:54:03.218 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:54:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:54:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:03.219 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:54:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:03.220 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:03.221 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:54:03.222 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:54:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:54:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:54:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:54:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:03.223 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:54:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:54:03.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:54:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:54:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:54:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:03.226 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:03.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "140", 2023-10-26T05:54:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:03.228 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:03.229 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:54:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:03.230 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:03.231 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:03.232 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:03.233 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:03.234 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:03.235 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:54:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:54:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.236 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:54:03.237 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.238 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:03.239 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:03.240 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:03.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:03.242 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:54:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.243 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.244 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.245 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.246 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.247 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:54:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:54:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.252 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:54:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:54:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.256 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:54:03.258 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:54:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.260 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.261 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:54:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:54:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.263 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:54:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:54:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:54:03.264 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:54:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.266 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.267 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:54:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:54:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.270 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:54:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:54:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:54:03.274 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:54:03.275 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.276 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:54:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:54:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.278 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:54:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:54:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:54:03.279 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:54:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.281 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.282 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:54:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:54:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.286 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:54:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:54:03.288 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.288 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.288 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.288 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.289 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.289 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.289 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:54:03.289 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.290 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:54:03.290 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.291 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.291 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.291 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.291 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.292 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:54:03.292 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.293 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:54:03.293 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:03.293 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.293 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.294 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:54:03.294 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:54:03.294 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:54:03.294 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:54:03.295 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.295 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.295 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.295 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.296 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.297 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.297 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.297 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.297 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.297 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.297 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.299 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:54:03.299 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.299 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:54:03.300 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.300 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.300 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.300 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.301 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.301 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.301 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:54:03.301 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.302 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:54:03.302 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.303 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.303 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.303 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.304 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:54:03.304 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.304 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.304 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:54:03.305 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.305 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.305 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.305 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.305 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.306 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.306 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:54:03.306 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.307 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.307 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:54:03.307 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:03.307 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.307 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.308 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:54:03.308 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:54:03.308 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:54:03.308 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:54:03.309 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.309 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.309 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.309 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.310 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.310 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:03.310 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.310 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.310 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.310 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.311 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.311 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.311 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.311 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.311 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.311 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.312 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.312 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.312 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.312 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:54:03.313 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.313 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:54:03.314 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.314 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.314 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.314 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.314 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.315 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:54:03.315 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.316 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.316 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:54:03.316 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.316 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.316 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.317 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.317 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.318 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:54:03.318 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.318 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.319 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:54:03.319 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.319 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.319 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.319 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.320 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.320 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:54:03.320 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.321 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:54:03.321 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:03.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.322 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.322 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:54:03.322 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:54:03.322 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:54:03.323 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:54:03.323 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.323 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.323 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.323 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.324 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:03.324 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.324 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.324 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.325 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.325 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.325 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.325 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.325 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.325 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.326 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.326 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.326 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.326 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.329 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:54:03.330 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.330 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:54:03.331 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.331 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.331 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.332 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.332 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:54:03.332 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.333 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:54:03.333 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.333 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.334 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.334 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.335 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:54:03.335 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.335 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:54:03.336 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.336 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.336 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.337 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.337 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:54:03.337 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.338 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:54:03.338 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:03.338 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.339 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.339 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:54:03.339 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:54:03.339 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:54:03.339 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:54:03.340 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.340 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.340 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.340 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.341 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:03.341 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.341 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.341 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.341 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.342 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.342 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.342 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.343 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.343 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.343 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.343 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.344 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:54:03.344 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.345 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:54:03.345 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.345 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.345 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.346 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.346 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.346 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:54:03.347 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.347 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:54:03.347 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.348 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.348 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.348 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:54:03.349 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.350 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:54:03.350 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.350 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.350 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.351 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.351 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.351 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:54:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.352 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:54:03.353 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:03.353 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.353 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.353 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:54:03.353 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:54:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:54:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:54:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.354 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:03.355 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:03.356 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:03.357 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:03.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:54:03.358 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.359 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:54:03.359 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.360 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.360 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.360 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:03.360 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:54:03.361 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:54:03.362 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.362 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.363 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:03.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.363 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:54:03.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.364 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:54:03.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.365 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.365 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:03.365 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:03.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:03.366 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:54:03.366 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:03.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:54:03.367 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:03.367 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.368 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:54:03.368 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:54:03.368 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:54:03.368 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:54:03.368 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:03.368 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.369 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:03.370 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.370 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.370 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.370 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.370 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.371 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:54:03.371 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.371 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:03.371 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:54:03.372 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.373 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:03.373 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.373 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.374 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.374 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.374 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.374 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:54:03.374 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.375 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.375 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:03.375 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.375 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.375 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.376 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.376 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.376 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:54:03.376 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.377 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:03.377 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.377 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.377 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.377 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.377 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.378 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:54:03.378 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.378 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:03.379 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.379 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.379 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.379 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.379 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.379 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:54:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.380 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.381 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.381 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.381 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.381 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:54:03.381 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:03.382 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:03.382 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:03.382 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:03.382 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:03.383 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:03.383 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:03.383 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:54:03.383 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:03.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.384 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:03.384 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:54:03.384 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:54:03.384 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:54:03.384 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:54:03.384 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:54:03.385 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:54:03.385 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:54:03.385 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:54:03.385 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:54:03.385 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:54:03.386 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:54:03.386 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:54:03.386 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:54:03.386 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:54:03.386 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:03.387 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:54:03.387 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:54:03.387 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:54:03.387 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:54:03.387 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:54:03.387 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:54:03.388 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:54:03.388 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:54:03.388 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:54:03.388 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:54:03.388 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:54:03.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:03.389 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:03.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c0664a0 msgr2=0x7f423c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c0664a0 0x7f423c068960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4248006fd0 tx=0x7f4248008040 comp rx=0 tx=0).stop 2023-10-26T05:54:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 msgr2=0x7f425806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 0x7f425806e440 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f425400c940 tx=0x7f425400ce00 comp rx=0 tx=0).stop 2023-10-26T05:54:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 shutdown_connections 2023-10-26T05:54:03.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4258159210 0x7f42581503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f42580cd240 0x7f42580c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f423c0664a0 0x7f423c068960 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 --2- 172.21.15.71:0/1812411838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42580b0420 0x7f425806e440 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 >> 172.21.15.71:0/1812411838 conn(0x7f42580b0c80 msgr2=0x7f42580b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:03.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 shutdown_connections 2023-10-26T05:54:03.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.175+0000 7f425f9cb640 1 -- 172.21.15.71:0/1812411838 wait complete. 2023-10-26T05:54:03.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:03.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/670147694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 msgr2=0x7fbc9815aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/670147694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc9815aa20 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009900 tx=0x7fbc9001c7c0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/670147694 shutdown_connections 2023-10-26T05:54:03.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/670147694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc9815aa20 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/670147694 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 0x7fbc9815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/670147694 >> 172.21.15.71:0/670147694 conn(0x7fbc980b0f20 msgr2=0x7fbc980b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:03.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/670147694 shutdown_connections 2023-10-26T05:54:03.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.511+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/670147694 wait complete. 2023-10-26T05:54:03.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 Processor -- start 2023-10-26T05:54:03.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- start start 2023-10-26T05:54:03.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 0x7fbc98150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc9815c720 0x7fbc9814f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc980cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98144480 con 0x7fbc9815d390 2023-10-26T05:54:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9814f8e0 con 0x7fbc9815c720 2023-10-26T05:54:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc981474d0 con 0x7fbc980cede0 2023-10-26T05:54:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc980cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc980cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51552/0 (socket says 172.21.15.71:51552) 2023-10-26T05:54:03.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 -- 172.21.15.71:0/282983394 learned_addr learned my addr 172.21.15.71:0/282983394 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc967fc640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc9815c720 0x7fbc9814f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 0x7fbc98150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 -- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 msgr2=0x7fbc98150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 0x7fbc98150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 -- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc9815c720 msgr2=0x7fbc9814f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc9815c720 0x7fbc9814f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 -- 172.21.15.71:0/282983394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009560 con 0x7fbc9815d390 2023-10-26T05:54:03.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc967fc640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc9815c720 0x7fbc9814f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:54:03.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 0x7fbc98150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:54:03.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc977fe640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc980cacf0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbc900b0040 tx=0x7fbc900a5640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:03.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90003800 con 0x7fbc9815d390 2023-10-26T05:54:03.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc900a5830 con 0x7fbc9815d390 2023-10-26T05:54:03.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90003cd0 con 0x7fbc9815d390 2023-10-26T05:54:03.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc98146be0 con 0x7fbc9815d390 2023-10-26T05:54:03.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc98147160 con 0x7fbc9815d390 2023-10-26T05:54:03.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.515+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc98147680 con 0x7fbc9815d390 2023-10-26T05:54:03.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.519+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc900b4070 con 0x7fbc9815d390 2023-10-26T05:54:03.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.519+0000 7fbc8f7fe640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc7c066470 0x7fbc7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:03.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.519+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc9011e7d0 con 0x7fbc9815d390 2023-10-26T05:54:03.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.519+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7fbc7c06dfa0 con 0x7fbc9815d390 2023-10-26T05:54:03.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.519+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc7c066470 0x7fbc7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:03.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.519+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc7c066470 0x7fbc7c068930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbc88003ec0 tx=0x7fbc88009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:03.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.523+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc900b2020 con 0x7fbc9815d390 2023-10-26T05:54:03.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.695+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc98066d00 con 0x7fbc9815d390 2023-10-26T05:54:03.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/282983394 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fbc98147680 con 0x7fbc9815d390 2023-10-26T05:54:03.706 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:03.706 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:54:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:54:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc7c066470 msgr2=0x7fbc7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc7c066470 0x7fbc7c068930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbc88003ec0 tx=0x7fbc88009040 comp rx=0 tx=0).stop 2023-10-26T05:54:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 msgr2=0x7fbc980cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc980cacf0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbc900b0040 tx=0x7fbc900a5640 comp rx=0 tx=0).stop 2023-10-26T05:54:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 shutdown_connections 2023-10-26T05:54:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc7c066470 0x7fbc7c068930 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc9815d390 0x7fbc980cacf0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc9815c720 0x7fbc9814f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 --2- 172.21.15.71:0/282983394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cede0 0x7fbc98150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 >> 172.21.15.71:0/282983394 conn(0x7fbc980b0f20 msgr2=0x7fbc980b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 shutdown_connections 2023-10-26T05:54:03.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:03.699+0000 7fbc9d5c8640 1 -- 172.21.15.71:0/282983394 wait complete. 2023-10-26T05:54:03.724 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T05:54:03.856 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.431430424999999,"mdsmap_epoch":51,"osdmap_epoch":140,"osdmap_epoch_barrier":138,"uptime":28.278113652999998} 2023-10-26T05:54:03.857 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 8007, "mdsmap_epoch": 51, "osdmap_epoch": 140, "osdmap_epoch_barrier": 138, "rank_uptime": 23.431430425, "state": "up:active", "uptime": 28.278113653, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:54:03.857 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:54:03.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:03.858 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T05:54:03.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:03.869 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T05:54:03.921 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:54:04.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- 172.21.15.71:0/1989034238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 msgr2=0x7fc50406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:04.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 --2- 172.21.15.71:0/1989034238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50406aa70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- 172.21.15.71:0/1989034238 shutdown_connections 2023-10-26T05:54:04.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 --2- 172.21.15.71:0/1989034238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50406aa70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- 172.21.15.71:0/1989034238 >> 172.21.15.71:0/1989034238 conn(0x7fc5040cdd80 msgr2=0x7fc5040ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:04.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- 172.21.15.71:0/1989034238 shutdown_connections 2023-10-26T05:54:04.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- 172.21.15.71:0/1989034238 wait complete. 2023-10-26T05:54:04.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 Processor -- start 2023-10-26T05:54:04.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- start start 2023-10-26T05:54:04.102 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5040cd880 0x7fc5040c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:04.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:04.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5040d0850 0x7fc5040ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:04.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50406a650 con 0x7fc5040d0470 2023-10-26T05:54:04.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc504150d10 con 0x7fc5040d0850 2023-10-26T05:54:04.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc509958640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5040cc3d0 con 0x7fc5040cd880 2023-10-26T05:54:04.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:04.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50414fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51566/0 (socket says 172.21.15.71:51566) 2023-10-26T05:54:04.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 -- 172.21.15.71:0/3661221639 learned_addr learned my addr 172.21.15.71:0/3661221639 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:04.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc502ffd640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5040cd880 0x7fc5040c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:04.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5037fe640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5040d0850 0x7fc5040ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 -- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5040cd880 msgr2=0x7fc5040c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5040cd880 0x7fc5040c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 -- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5040d0850 msgr2=0x7fc5040ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5040d0850 0x7fc5040ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5027fc640 1 -- 172.21.15.71:0/3661221639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f4009560 con 0x7fc5040d0470 2023-10-26T05:54:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc502ffd640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5040cd880 0x7fc5040c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.091+0000 7fc5037fe640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5040d0850 0x7fc5040ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc5027fc640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50414fb00 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b440 tx=0x7fc4f800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8011830 con 0x7fc5040d0470 2023-10-26T05:54:04.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc4f8011a80 con 0x7fc5040d0470 2023-10-26T05:54:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc504150040 con 0x7fc5040d0470 2023-10-26T05:54:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8011e90 con 0x7fc5040d0470 2023-10-26T05:54:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc504150650 con 0x7fc5040d0470 2023-10-26T05:54:04.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc504066b10 con 0x7fc5040d0470 2023-10-26T05:54:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc4f801a400 con 0x7fc5040d0470 2023-10-26T05:54:04.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4f0066470 0x7fc4f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4f8083f40 con 0x7fc5040d0470 2023-10-26T05:54:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7fc4f006dfa0 con 0x7fc5040d0470 2023-10-26T05:54:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc502ffd640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4f0066470 0x7fc4f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.095+0000 7fc502ffd640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4f0066470 0x7fc4f0068930 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40af000 tx=0x7fc4f401b000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:04.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.099+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc4f80513c0 con 0x7fc5040d0470 2023-10-26T05:54:04.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.255+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc504150e50 con 0x7fc5040d0470 2023-10-26T05:54:04.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.255+0000 7fc4ebfff640 1 -- 172.21.15.71:0/3661221639 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+33905 (secure 0 0 0) 0x7fc504150e50 con 0x7fc5040d0470 2023-10-26T05:54:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:04.266 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:04.266 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 140, 2023-10-26T05:54:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:54:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:54:02.174155+0000", 2023-10-26T05:54:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:54:04.267 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:54:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:54:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:54:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:54:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:54:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:54:04.268 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:54:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:54:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:54:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:54:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:54:04.269 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:54:04.270 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:54:04.270 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:54:04.270 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:54:04.270 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:54:04.270 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:54:04.271 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:54:04.271 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:54:04.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:54:04.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:54:04.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:54:04.272 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:54:04.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:04.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:04.272 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:04.272 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:04.273 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:04.273 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:04.273 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:04.273 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:04.273 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:04.273 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:04.274 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:04.274 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:04.274 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:54:04.274 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:54:04.274 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:54:04.275 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:54:04.275 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:54:04.275 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:04.275 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:04.275 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:04.275 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:04.276 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:04.276 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:04.276 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:04.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.276 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:54:04.277 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:04.277 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:04.277 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:04.277 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:04.277 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:04.278 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:04.279 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:04.280 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:04.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.282 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:04.282 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:04.282 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:04.282 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:04.282 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:04.283 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:04.283 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:04.283 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:04.283 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:04.283 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:04.284 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:04.284 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:04.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:54:04.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:54:04.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.284 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:04.285 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:54:04.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.285 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:04.285 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:54:04.286 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:54:04.286 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:54:04.286 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:54:04.286 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:54:04.287 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:04.287 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:04.287 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:04.287 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.288 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:54:04.288 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:54:04.288 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:54:04.288 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:04.289 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:04.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:04.289 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:04.289 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:04.289 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:04.289 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:04.290 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:04.290 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:04.290 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:04.290 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:04.290 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:54:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:54:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:54:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:54:04.291 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:54:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:04.292 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:54:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:04.293 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:04.294 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:04.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:04.296 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:04.297 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.298 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:04.299 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:04.300 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:54:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:54:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:54:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.301 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:54:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.302 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:54:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:54:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:54:04.305 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:54:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:54:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:04.306 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:04.307 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:54:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:54:04.308 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:54:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:54:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:54:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:04.309 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:04.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "140", 2023-10-26T05:54:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:04.311 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:04.312 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:54:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:04.313 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:04.314 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:04.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:04.316 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:04.317 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:54:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:54:04.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:54:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.319 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:04.320 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:54:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:04.322 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.323 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.324 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:54:04.325 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:54:04.326 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.327 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:54:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:54:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:54:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.331 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:54:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.332 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.333 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:54:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.334 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:54:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.335 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:54:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:54:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:54:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:54:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.336 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.338 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.339 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:54:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:54:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.342 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.343 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:54:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:54:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.345 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:54:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:54:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.348 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:54:04.349 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:54:04.350 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:54:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:54:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:54:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:54:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.352 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.353 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.354 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.355 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:54:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:54:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:54:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:54:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.361 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:54:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.362 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:54:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.363 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:54:04.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:54:04.365 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:54:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:54:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:54:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:54:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.368 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.369 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.370 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:54:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:54:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.372 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:54:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.374 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:54:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.375 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:54:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:54:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.377 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:54:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:54:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.380 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:54:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:54:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:54:04.381 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:54:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.383 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:54:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:54:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:54:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:54:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:54:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:54:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:54:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:54:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:54:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:54:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:54:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:54:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:54:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:54:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:54:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:54:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:54:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:54:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:54:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:54:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:54:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:54:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:54:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:54:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:54:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:54:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:54:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:54:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:54:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:54:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:54:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:54:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:54:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:54:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:54:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:54:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:54:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:54:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:54:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:54:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:54:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:54:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:54:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:54:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:54:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:54:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:54:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:54:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:54:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:54:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:54:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:54:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:54:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:54:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:54:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.456 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.456 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:54:04.456 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:04.457 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:04.457 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:04.457 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:04.457 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:04.457 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:04.457 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:04.458 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:54:04.458 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:04.458 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.458 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:04.458 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:54:04.459 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:54:04.459 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:54:04.459 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:54:04.459 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:54:04.459 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:54:04.459 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:54:04.460 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:54:04.460 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:54:04.460 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:54:04.460 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:54:04.460 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:54:04.461 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:54:04.461 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:54:04.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:04.461 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:54:04.462 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:54:04.462 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:54:04.462 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:54:04.462 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:54:04.462 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:54:04.463 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:54:04.463 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:54:04.463 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:54:04.463 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:54:04.463 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:54:04.463 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:04.464 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:04.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4f0066470 msgr2=0x7fc4f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:04.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4f0066470 0x7fc4f0068930 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40af000 tx=0x7fc4f401b000 comp rx=0 tx=0).stop 2023-10-26T05:54:04.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 msgr2=0x7fc50414fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:04.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50414fb00 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800b440 tx=0x7fc4f800b900 comp rx=0 tx=0).stop 2023-10-26T05:54:04.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 shutdown_connections 2023-10-26T05:54:04.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5040d0850 0x7fc5040ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc4f0066470 0x7fc4f0068930 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc5040d0470 0x7fc50414fb00 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 --2- 172.21.15.71:0/3661221639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc5040cd880 0x7fc5040c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:04.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 >> 172.21.15.71:0/3661221639 conn(0x7fc5040cdd80 msgr2=0x7fc50415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:04.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 shutdown_connections 2023-10-26T05:54:04.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:04.259+0000 7fc509958640 1 -- 172.21.15.71:0/3661221639 wait complete. 2023-10-26T05:54:04.468 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:54:09.469 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:54:09.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- 172.21.15.71:0/3751348974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8f8003980 msgr2=0x7fb8f8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:09.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 --2- 172.21.15.71:0/3751348974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8f8003980 0x7fb8f8003da0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fb904009b60 tx=0x7fb90401d250 comp rx=0 tx=0).stop 2023-10-26T05:54:09.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb90f7fe640 1 -- 172.21.15.71:0/3751348974 reap_dead start 2023-10-26T05:54:09.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- 172.21.15.71:0/3751348974 shutdown_connections 2023-10-26T05:54:09.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- 172.21.15.71:0/3751348974 >> 172.21.15.71:0/3751348974 conn(0x7fb910069770 msgr2=0x7fb910069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:09.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- 172.21.15.71:0/3751348974 shutdown_connections 2023-10-26T05:54:09.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- 172.21.15.71:0/3751348974 wait complete. 2023-10-26T05:54:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 Processor -- start 2023-10-26T05:54:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- start start 2023-10-26T05:54:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 0x7fb91013f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:09.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9101372c0 0x7fb91013d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910163580 0x7fb910163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb910163f50 con 0x7fb910136ee0 2023-10-26T05:54:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9101412e0 con 0x7fb910163580 2023-10-26T05:54:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb91571b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91013d930 con 0x7fb9101372c0 2023-10-26T05:54:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb90effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 0x7fb91013f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:09.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb90effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 0x7fb91013f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51584/0 (socket says 172.21.15.71:51584) 2023-10-26T05:54:09.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.595+0000 7fb90effd640 1 -- 172.21.15.71:0/1833132387 learned_addr learned my addr 172.21.15.71:0/1833132387 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:09.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90f7fe640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910163580 0x7fb910163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:09.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90e7fc640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9101372c0 0x7fb91013d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:09.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90effd640 1 -- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9101372c0 msgr2=0x7fb91013d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:09.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90effd640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9101372c0 0x7fb91013d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90effd640 1 -- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910163580 msgr2=0x7fb910163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90effd640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910163580 0x7fb910163a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90effd640 1 -- 172.21.15.71:0/1833132387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91013dc10 con 0x7fb910136ee0 2023-10-26T05:54:09.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90e7fc640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9101372c0 0x7fb91013d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90effd640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 0x7fb91013f220 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb9040af040 tx=0x7fb904005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904004a10 con 0x7fb910136ee0 2023-10-26T05:54:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb904004c60 con 0x7fb910136ee0 2023-10-26T05:54:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb910140c90 con 0x7fb910136ee0 2023-10-26T05:54:09.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9040b6410 con 0x7fb910136ee0 2023-10-26T05:54:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91013d620 con 0x7fb910136ee0 2023-10-26T05:54:09.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb910066b10 con 0x7fb910136ee0 2023-10-26T05:54:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb9040b3070 con 0x7fb910136ee0 2023-10-26T05:54:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb8fc066490 0x7fb8fc068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb90411f330 con 0x7fb910136ee0 2023-10-26T05:54:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=141}) v3 -- 0x7fb8fc06dfc0 con 0x7fb910136ee0 2023-10-26T05:54:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90e7fc640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb8fc066490 0x7fb8fc068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.599+0000 7fb90e7fc640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb8fc066490 0x7fb8fc068950 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb900000c10 tx=0x7fb900000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.603+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb9040b1060 con 0x7fb910136ee0 2023-10-26T05:54:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.755+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb910163f50 con 0x7fb910136ee0 2023-10-26T05:54:09.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.759+0000 7fb8f77fe640 1 -- 172.21.15.71:0/1833132387 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) v1 ==== 81+0+33905 (secure 0 0 0) 0x7fb9040ec790 con 0x7fb910136ee0 2023-10-26T05:54:09.769 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:09.769 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:09.769 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 140, 2023-10-26T05:54:09.770 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:09.770 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:54:09.770 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:54:02.174155+0000", 2023-10-26T05:54:09.770 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:54:09.770 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:54:09.771 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:54:09.771 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:54:09.771 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:54:09.771 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:54:09.771 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:54:09.772 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:54:09.772 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:54:09.772 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:09.772 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:54:09.772 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:54:09.773 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:54:09.773 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:54:09.773 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:54:09.773 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:54:09.773 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:54:09.773 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:54:09.774 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:54:09.774 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:54:09.774 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:54:09.774 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:54:09.774 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:54:09.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:54:09.775 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:54:09.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:09.775 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:09.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:09.776 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:09.776 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:09.776 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:09.776 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:09.776 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:09.777 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:09.777 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:09.777 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:09.777 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:09.777 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:54:09.777 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:54:09.778 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:54:09.778 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:54:09.778 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:54:09.778 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:09.778 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:09.779 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:09.779 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:09.779 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:09.779 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:09.779 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:09.780 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.780 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:54:09.780 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:09.780 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:09.780 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:09.780 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:09.781 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:09.781 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:09.781 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:09.781 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:09.781 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:09.782 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:09.783 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:09.783 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:09.783 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:09.783 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:09.783 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:09.783 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:09.784 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:09.784 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:09.784 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:09.784 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:09.784 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:09.785 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:09.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.785 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:09.785 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:09.785 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:09.786 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:09.786 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:09.786 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:09.786 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:09.786 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:09.787 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:09.787 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:09.787 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:09.787 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:09.787 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:54:09.787 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:54:09.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.788 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:09.788 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:54:09.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.788 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:09.789 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:54:09.789 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:54:09.789 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:54:09.789 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:54:09.789 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:54:09.790 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:09.790 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:09.790 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:09.790 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.791 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.791 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:54:09.791 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:54:09.791 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:54:09.791 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:09.792 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:09.792 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:09.792 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:09.792 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:09.792 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:09.792 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:09.793 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:09.793 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:09.793 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:09.793 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:09.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:09.794 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:54:09.794 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:54:09.794 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:54:09.794 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:54:09.794 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:54:09.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:09.795 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:09.795 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:09.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:09.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:09.796 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:09.796 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:09.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.796 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:54:09.796 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:09.797 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:09.797 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:09.797 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:09.797 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:09.797 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:09.798 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:09.798 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:09.798 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:09.798 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:09.798 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:09.799 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:09.799 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:09.799 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:09.799 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:09.799 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:09.799 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:09.800 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:09.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:09.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:09.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:09.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:09.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:09.801 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:09.801 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:09.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:09.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.801 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:09.802 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:09.802 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:09.802 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:09.802 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:09.802 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:09.802 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:09.803 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:09.803 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:09.803 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:09.803 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:09.804 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:09.804 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:54:09.804 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:54:09.804 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:54:09.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.804 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:09.805 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:09.805 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:54:09.805 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.805 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:09.806 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:09.807 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:09.807 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:09.807 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.808 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:54:09.808 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:54:09.808 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:54:09.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:54:09.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:54:09.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:09.809 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:09.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:09.809 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:09.809 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:09.809 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:09.810 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:09.810 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:09.810 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:09.810 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:09.810 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:54:09.811 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:54:09.811 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:54:09.811 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:54:09.811 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:54:09.811 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:09.812 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:09.812 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:09.812 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:09.812 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:09.812 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:09.813 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:09.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.813 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "140", 2023-10-26T05:54:09.813 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:09.813 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:09.814 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:09.814 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:09.814 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:09.814 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:09.814 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:09.815 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:09.815 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:09.815 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:54:09.815 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:09.815 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:09.816 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:09.816 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:09.816 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:09.816 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:09.816 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:09.816 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:09.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:09.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:09.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:09.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:09.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:09.818 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:09.818 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:09.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:09.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.818 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:09.819 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:09.819 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:09.819 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:09.819 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:09.819 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:09.820 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:09.820 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:09.820 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:09.820 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:09.820 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:09.820 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:09.821 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:54:09.821 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:54:09.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.821 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:09.821 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:09.822 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:54:09.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.822 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:09.822 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:09.823 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:09.823 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:09.823 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:09.823 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:09.823 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:09.824 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:09.824 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:09.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.824 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:09.825 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:54:09.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.825 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:09.825 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.825 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.826 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.826 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.826 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.826 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.826 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.827 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.827 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.827 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.827 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.827 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.827 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.828 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.828 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.828 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:54:09.828 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.829 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:54:09.829 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.829 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.829 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.830 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.830 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.831 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:54:09.831 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.831 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.831 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:54:09.832 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.832 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.832 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.832 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.832 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.833 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:54:09.833 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.834 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:54:09.834 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.835 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.835 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.835 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.835 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.836 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:54:09.836 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.837 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:54:09.837 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:09.837 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.837 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.837 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:54:09.838 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:54:09.838 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:54:09.838 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:54:09.838 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.838 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.839 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.839 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.839 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:09.839 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.840 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.840 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.840 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.840 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.840 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.840 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.841 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.841 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.841 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.841 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.841 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.842 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.842 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:54:09.842 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.843 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:54:09.843 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.843 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:54:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:54:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.847 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:54:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.848 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:54:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.849 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:54:09.850 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:54:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:09.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:54:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:54:09.852 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:54:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:54:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.854 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.855 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.856 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:54:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:54:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:54:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:54:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.861 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:54:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:54:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:54:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:54:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.866 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:54:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:54:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:54:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:54:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.867 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:09.868 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.869 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.870 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:54:09.871 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:54:09.872 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.873 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:54:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:54:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.875 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:54:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.877 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:54:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.878 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:54:09.879 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:54:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:09.880 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:54:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:54:09.881 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:54:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:54:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.882 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.883 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.884 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.885 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:54:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:54:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:54:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.890 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:54:09.890 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.890 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.890 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.890 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.891 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.891 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.891 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:54:09.891 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.892 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:54:09.892 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.893 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.893 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.893 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.894 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:54:09.894 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.895 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.895 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:54:09.895 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:09.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.895 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.896 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:54:09.896 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:54:09.896 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:54:09.896 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:54:09.896 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.897 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.897 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.897 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.898 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:09.898 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.898 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.898 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.898 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.899 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.899 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.899 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.899 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.899 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.899 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.900 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.900 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.900 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.900 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:54:09.901 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:54:09.902 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.902 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:54:09.903 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.904 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:54:09.904 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.905 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.905 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.905 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.905 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.906 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:54:09.906 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.907 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:54:09.907 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.907 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.907 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.908 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.908 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:54:09.908 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.909 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:54:09.909 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:09.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.910 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:54:09.910 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:54:09.911 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:54:09.911 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:54:09.911 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.911 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.911 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.912 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.912 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:09.912 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.913 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.913 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.913 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.913 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.913 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.914 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.914 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.914 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.914 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.914 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.914 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:54:09.915 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.916 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:54:09.916 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.917 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.917 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.917 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.918 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:54:09.918 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.919 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:54:09.919 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.919 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.919 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.920 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.920 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.921 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:54:09.921 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.922 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:54:09.922 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.922 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.923 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.923 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.923 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.923 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:54:09.924 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.924 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.924 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:54:09.925 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:09.925 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.925 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.925 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:54:09.925 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:54:09.926 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:54:09.926 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:54:09.926 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.926 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:09.928 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:09.928 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:09.928 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:09.928 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:09.928 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:09.929 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:09.929 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:09.929 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:09.929 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:09.929 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:09.929 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:09.930 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.930 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:54:09.931 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.931 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:54:09.931 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.932 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.932 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:09.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:54:09.933 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:54:09.934 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:09.935 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.935 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.936 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:54:09.936 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.936 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:54:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:09.937 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:54:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.939 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:09.939 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:54:09.939 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:09.939 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.940 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.940 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:54:09.940 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:54:09.940 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:54:09.941 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:54:09.941 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:09.941 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:09.941 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:09.941 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:09.942 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.942 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:09.942 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:54:09.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.943 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:09.943 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.943 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.943 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.943 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.943 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.944 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:54:09.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.944 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:09.944 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.945 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.945 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.945 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.945 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.945 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:54:09.946 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.946 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.946 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:09.946 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.947 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.947 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.947 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.947 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.947 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:54:09.947 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.948 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:09.948 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.948 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.948 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.949 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.949 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.949 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:54:09.949 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.950 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:09.950 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.950 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.950 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.951 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.951 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.951 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:54:09.951 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.952 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:09.952 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.952 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.952 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.952 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.953 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.953 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:54:09.953 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.953 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:09.954 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.954 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.954 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.954 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.954 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.955 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:54:09.955 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:09.955 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:09.956 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:09.956 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:09.956 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:09.956 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:09.956 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:09.957 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:54:09.957 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:09.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.957 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:09.957 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:54:09.958 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:54:09.958 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:54:09.958 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:54:09.958 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:54:09.958 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:54:09.959 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:54:09.959 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:54:09.959 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:54:09.959 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:54:09.959 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:54:09.959 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:54:09.960 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:54:09.960 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:54:09.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:09.960 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:54:09.961 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:54:09.961 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:54:09.961 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:54:09.961 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:54:09.961 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:54:09.961 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:54:09.962 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:54:09.962 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:54:09.962 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:54:09.962 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:54:09.962 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:09.962 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:09.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb8fc066490 msgr2=0x7fb8fc068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:09.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb8fc066490 0x7fb8fc068950 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb900000c10 tx=0x7fb900000ee0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 msgr2=0x7fb91013f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:09.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 0x7fb91013f220 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb9040af040 tx=0x7fb904005620 comp rx=0 tx=0).stop 2023-10-26T05:54:09.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 shutdown_connections 2023-10-26T05:54:09.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb910163580 0x7fb910163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9101372c0 0x7fb91013d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb8fc066490 0x7fb8fc068950 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 --2- 172.21.15.71:0/1833132387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910136ee0 0x7fb91013f220 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:09.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 >> 172.21.15.71:0/1833132387 conn(0x7fb910069770 msgr2=0x7fb910069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:09.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 shutdown_connections 2023-10-26T05:54:09.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:09.763+0000 7fb91571b640 1 -- 172.21.15.71:0/1833132387 wait complete. 2023-10-26T05:54:09.967 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:54:14.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:54:15.103 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 -- 172.21.15.71:0/1982528404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c070fe0 msgr2=0x7f7d9c071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 --2- 172.21.15.71:0/1982528404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c071440 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009900 tx=0x7f7d9001c890 comp rx=0 tx=0).stop 2023-10-26T05:54:15.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 -- 172.21.15.71:0/1982528404 shutdown_connections 2023-10-26T05:54:15.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 --2- 172.21.15.71:0/1982528404 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c071440 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.104 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 --2- 172.21.15.71:0/1982528404 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d9c140cc0 0x7f7d9c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 -- 172.21.15.71:0/1982528404 >> 172.21.15.71:0/1982528404 conn(0x7f7d9c069770 msgr2=0x7f7d9c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:15.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 -- 172.21.15.71:0/1982528404 shutdown_connections 2023-10-26T05:54:15.105 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 -- 172.21.15.71:0/1982528404 wait complete. 2023-10-26T05:54:15.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 Processor -- start 2023-10-26T05:54:15.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.095+0000 7f7da2891640 1 -- start start 2023-10-26T05:54:15.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c13b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c0b0040 0x7f7d9c13c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d9c140cc0 0x7f7d9c14a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c134680 con 0x7f7d9c070fe0 2023-10-26T05:54:15.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c1616e0 con 0x7f7d9c0b0040 2023-10-26T05:54:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c1619c0 con 0x7f7d9c140cc0 2023-10-26T05:54:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c13b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c13b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38626/0 (socket says 172.21.15.71:38626) 2023-10-26T05:54:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 -- 172.21.15.71:0/2048730240 learned_addr learned my addr 172.21.15.71:0/2048730240 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:15.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9b7fe640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c0b0040 0x7f7d9c13c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da0e2e640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d9c140cc0 0x7f7d9c14a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 -- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d9c140cc0 msgr2=0x7f7d9c14a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d9c140cc0 0x7f7d9c14a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 -- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c0b0040 msgr2=0x7f7d9c13c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c0b0040 0x7f7d9c13c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 -- 172.21.15.71:0/2048730240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009560 con 0x7f7d9c070fe0 2023-10-26T05:54:15.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9bfff640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c13b730 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e7c0 tx=0x7f7d8c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:15.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c0095b0 con 0x7f7d9c070fe0 2023-10-26T05:54:15.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c161ca0 con 0x7f7d9c070fe0 2023-10-26T05:54:15.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d8c00eed0 con 0x7f7d9c070fe0 2023-10-26T05:54:15.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c005020 con 0x7f7d9c070fe0 2023-10-26T05:54:15.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c162260 con 0x7f7d9c070fe0 2023-10-26T05:54:15.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c066b10 con 0x7f7d9c070fe0 2023-10-26T05:54:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d8c005180 con 0x7f7d9c070fe0 2023-10-26T05:54:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d98ff9640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d88066470 0x7f7d88068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7d8c014070 con 0x7f7d9c070fe0 2023-10-26T05:54:15.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.099+0000 7f7d9b7fe640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d88066470 0x7f7d88068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.103+0000 7f7d9b7fe640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d88066470 0x7f7d88068930 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7d900af000 tx=0x7f7d9001b0d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:15.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.103+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d8c055810 con 0x7f7d9c070fe0 2023-10-26T05:54:15.260 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.251+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7d9c0721e0 con 0x7f7d9c070fe0 2023-10-26T05:54:15.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7d98ff9640 1 -- 172.21.15.71:0/2048730240 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v143) v1 ==== 81+0+34258 (secure 0 0 0) 0x7f7d8c0553a0 con 0x7f7d9c070fe0 2023-10-26T05:54:15.263 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:15.263 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:15.263 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 143, 2023-10-26T05:54:15.263 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:15.264 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:54:15.264 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:54:12.178962+0000", 2023-10-26T05:54:15.264 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:54:15.264 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:54:15.264 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:54:15.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:54:15.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:54:15.265 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:54:15.265 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:54:15.265 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:54:15.265 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:54:15.266 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.266 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:54:15.266 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:54:15.266 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:54:15.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:54:15.267 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:54:15.267 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:54:15.267 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:54:15.267 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:54:15.267 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:54:15.268 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:54:15.268 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:54:15.268 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:54:15.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.268 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:54:15.268 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:54:15.269 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:54:15.269 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:15.269 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:15.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:15.269 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:15.270 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:15.270 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:15.270 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:15.270 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:15.270 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:15.270 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:15.271 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:15.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:15.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:54:15.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:54:15.271 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:54:15.272 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:54:15.272 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:54:15.272 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:15.272 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:15.272 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:15.272 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:15.273 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:15.273 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:15.273 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:15.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.273 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:54:15.273 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:15.274 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:15.274 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:15.274 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:15.274 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:15.274 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:15.275 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:15.275 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:15.275 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:15.275 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:15.275 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:15.275 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:15.276 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:15.276 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:15.276 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:15.276 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:15.276 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:15.276 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:15.277 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:15.277 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:15.277 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:15.277 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:15.277 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:15.278 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:15.278 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:15.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:15.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.278 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:15.278 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:15.279 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:15.279 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:15.279 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:15.279 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:15.280 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:15.280 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:15.280 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:15.280 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:15.280 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:15.280 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:15.281 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:54:15.281 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:54:15.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.281 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:15.281 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:54:15.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.282 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:15.282 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:54:15.282 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:54:15.282 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:54:15.282 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:54:15.283 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:54:15.283 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:15.283 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:15.283 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:15.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:54:15.284 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:54:15.284 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:54:15.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:15.285 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:15.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:15.285 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:15.285 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:15.285 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:15.285 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:15.286 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:15.286 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:15.286 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:15.286 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:15.286 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:15.286 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:54:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:54:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:54:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:54:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:54:15.287 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:15.288 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:15.288 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:15.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:15.288 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:15.288 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:15.288 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:15.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.289 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:54:15.289 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:15.289 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:15.289 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:15.289 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:15.290 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:15.290 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:15.290 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:15.290 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:15.290 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:15.291 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:15.291 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:15.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:15.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:15.292 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:15.293 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:15.293 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:15.293 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:15.293 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:15.293 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:15.294 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:15.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:15.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.294 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:15.294 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:15.295 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:15.295 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:15.295 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:15.295 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:15.295 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:15.295 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:15.296 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:15.296 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:15.296 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:15.296 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:15.296 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:54:15.296 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:54:15.297 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:54:15.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.297 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:15.297 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:15.297 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:54:15.297 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.298 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:15.298 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:15.298 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:15.298 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:15.299 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:15.299 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:15.299 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:15.299 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:15.299 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:15.300 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.300 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:54:15.300 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:54:15.301 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:54:15.301 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:15.301 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:15.301 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:15.301 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:15.302 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:15.302 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:15.302 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:15.302 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:15.302 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:15.303 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:15.303 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:15.303 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:15.303 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:54:15.303 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:54:15.304 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:54:15.304 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:54:15.304 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:54:15.304 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:15.304 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:15.304 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:15.305 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:15.305 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:15.305 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:15.305 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:15.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "143", 2023-10-26T05:54:15.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:15.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:15.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:15.306 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:15.307 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:15.307 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:15.307 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:15.307 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:15.307 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:15.308 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:54:15.308 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:15.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:15.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:15.308 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:15.309 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:15.309 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:15.309 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:15.309 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:15.309 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:15.309 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:15.310 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:15.310 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:15.310 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:15.310 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:15.310 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:15.311 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:15.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.311 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:15.311 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:15.311 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:15.312 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:54:15.313 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:54:15.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.314 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:15.314 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:15.314 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:54:15.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.315 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:15.315 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:54:15.315 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:54:15.316 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:15.316 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:54:15.316 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:54:15.316 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:15.316 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:15.316 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:15.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.317 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.317 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:54:15.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.318 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:15.318 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.318 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.318 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.319 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.319 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.319 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.319 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.319 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.319 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.320 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.320 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.320 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.320 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.321 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:54:15.321 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.322 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:54:15.322 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.322 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.322 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.323 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:54:15.323 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:54:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.324 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.325 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.325 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.326 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:54:15.326 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.327 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:54:15.327 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.327 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.327 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.328 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.328 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:54:15.328 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.329 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:54:15.329 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.329 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.330 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:54:15.330 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:54:15.330 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:54:15.330 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:54:15.331 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.331 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.331 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.331 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.332 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:15.332 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.332 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.332 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.333 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.333 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.333 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.333 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.333 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.333 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.334 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.334 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.334 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.334 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:54:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.336 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:54:15.336 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.336 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.336 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:54:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.339 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:54:15.339 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.339 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.340 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.340 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.340 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:54:15.340 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.341 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:54:15.341 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.342 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.342 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.342 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.343 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.343 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:54:15.343 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.344 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.344 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:54:15.344 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.344 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.345 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:54:15.345 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:54:15.345 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:54:15.345 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:54:15.345 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.346 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.346 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.346 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.346 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.347 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:15.347 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.347 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.347 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.347 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.347 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.348 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.348 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.348 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.348 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 141, 2023-10-26T05:54:15.348 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.349 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.349 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.349 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:54:15.350 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.350 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.351 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:54:15.351 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.351 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.351 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.352 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.352 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.352 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:54:15.352 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.353 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.353 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:54:15.353 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.354 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.354 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.354 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.355 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:54:15.355 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.356 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.356 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:54:15.356 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.357 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.357 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.357 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.358 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:54:15.358 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.358 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.359 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:54:15.359 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.359 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.359 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.360 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:54:15.360 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:54:15.360 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:54:15.360 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:54:15.360 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.360 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.361 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.361 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.361 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:15.362 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.362 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.362 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.362 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.362 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.363 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.363 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.363 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.363 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.363 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.363 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.364 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.364 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.364 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:54:15.364 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.365 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.365 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:54:15.365 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.366 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.366 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.366 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.366 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.367 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:54:15.367 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.367 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.368 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:54:15.368 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.368 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.368 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.369 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.369 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.369 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.369 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:54:15.370 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.370 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.370 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:54:15.371 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.371 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.371 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.371 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.371 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.372 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:54:15.372 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.373 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.373 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:54:15.373 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.373 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.374 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.374 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:54:15.374 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:54:15.374 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:54:15.375 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:54:15.375 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.375 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.375 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.375 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.376 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:15.376 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.376 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.376 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.377 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.377 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.377 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.377 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.377 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.378 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.378 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.378 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.378 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.378 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.379 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:54:15.379 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.379 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.380 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:54:15.380 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.380 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.381 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.381 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.381 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.381 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:54:15.382 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.382 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.382 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:54:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.383 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.384 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:54:15.384 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.384 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.385 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:54:15.385 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.385 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.386 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.386 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.386 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:54:15.387 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.387 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:54:15.388 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.388 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.388 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:54:15.389 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:54:15.389 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:54:15.389 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:54:15.389 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.389 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.390 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.390 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.390 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.390 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:15.391 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.391 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.391 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.391 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.391 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.391 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.392 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.392 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.392 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 141, 2023-10-26T05:54:15.392 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.392 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.392 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.393 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.393 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.393 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:54:15.393 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.394 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.394 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:54:15.394 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.395 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.395 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.395 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.396 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:54:15.396 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.397 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.397 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:54:15.397 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.397 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.397 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.398 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.398 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.399 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:54:15.399 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.399 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.400 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:54:15.400 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.400 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.400 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.400 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.401 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.401 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.401 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:54:15.401 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.402 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.402 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:54:15.402 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.403 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.403 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:54:15.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:54:15.404 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:54:15.404 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:54:15.404 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.404 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.404 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.404 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.405 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:15.405 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.405 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.405 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.406 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.406 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.406 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.406 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.406 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.407 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.407 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.407 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.407 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.407 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.408 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.408 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:54:15.408 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.409 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:54:15.409 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.410 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.410 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.410 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.410 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.411 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:54:15.411 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.412 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:54:15.412 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.412 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.413 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.413 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.414 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:54:15.414 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.414 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.414 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.414 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:54:15.415 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.415 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.415 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.416 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.416 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.416 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:54:15.417 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.417 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:54:15.418 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.418 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.418 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:54:15.421 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:54:15.421 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:54:15.421 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:54:15.421 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.422 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.422 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.422 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.423 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:15.423 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.423 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.423 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.423 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.423 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.424 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.424 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.424 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.424 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.424 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.425 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.425 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.425 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:54:15.426 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.427 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:54:15.427 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:54:15.428 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.429 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:54:15.429 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.430 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.430 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.430 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:54:15.431 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.432 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:54:15.432 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.432 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.433 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.433 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.433 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.433 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.433 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:54:15.434 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.434 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:54:15.435 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.435 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:54:15.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:54:15.436 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:54:15.436 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:54:15.436 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.436 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.436 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.437 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.437 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:54:15.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:15.438 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.438 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.438 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.438 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.439 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.439 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:54:15.439 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.439 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:15.440 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.440 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.440 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.440 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.440 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.441 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:54:15.441 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.441 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:15.442 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.442 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.442 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.442 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.442 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.443 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:54:15.443 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:15.444 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.444 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.444 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.444 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.444 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.445 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:54:15.445 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:15.446 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.446 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.446 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.446 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.446 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.446 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:54:15.447 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:15.447 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.447 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.448 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.448 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.448 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.448 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:54:15.448 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.449 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:15.449 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.449 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.449 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.450 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.450 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.450 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:54:15.450 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.451 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:15.451 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.451 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.451 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.452 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.452 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.452 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:54:15.452 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.453 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.453 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:54:15.453 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:54:15.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.453 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "17.7", 2023-10-26T05:54:15.454 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:54:15.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.454 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:54:15.454 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:54:15.454 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.455 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.455 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "17.3c", 2023-10-26T05:54:15.455 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:54:15.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.456 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:54:15.456 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 2 2023-10-26T05:54:15.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.456 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.456 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.457 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:54:15.457 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:54:15.457 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:54:15.457 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:54:15.457 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:54:15.457 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:54:15.458 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:54:15.458 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:54:15.458 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:54:15.458 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:54:15.458 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:54:15.458 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:54:15.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.459 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:54:15.459 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:54:15.459 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:54:15.460 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:54:15.460 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:54:15.460 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:54:15.460 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:54:15.460 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:54:15.460 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:54:15.461 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:54:15.461 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:54:15.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.461 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:15.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d88066470 msgr2=0x7f7d88068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d88066470 0x7f7d88068930 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7d900af000 tx=0x7f7d9001b0d0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 msgr2=0x7f7d9c13b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c13b730 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00e7c0 tx=0x7f7d8c00ec80 comp rx=0 tx=0).stop 2023-10-26T05:54:15.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 shutdown_connections 2023-10-26T05:54:15.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d9c140cc0 0x7f7d9c14a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d9c0b0040 0x7f7d9c13c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d88066470 0x7f7d88068930 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 --2- 172.21.15.71:0/2048730240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d9c070fe0 0x7f7d9c13b730 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 >> 172.21.15.71:0/2048730240 conn(0x7f7d9c069770 msgr2=0x7f7d9c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:15.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 shutdown_connections 2023-10-26T05:54:15.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.255+0000 7f7da2891640 1 -- 172.21.15.71:0/2048730240 wait complete. 2023-10-26T05:54:15.466 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T05:54:15.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:54:15.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- 172.21.15.71:0/3000393664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ffc003f00 msgr2=0x7f4ffc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 --2- 172.21.15.71:0/3000393664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ffc003f00 0x7f4ffc0047d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f500000c290 tx=0x7f500001f780 comp rx=0 tx=0).stop 2023-10-26T05:54:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f50160ca640 1 -- 172.21.15.71:0/3000393664 reap_dead start 2023-10-26T05:54:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- 172.21.15.71:0/3000393664 shutdown_connections 2023-10-26T05:54:15.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- 172.21.15.71:0/3000393664 >> 172.21.15.71:0/3000393664 conn(0x7f5010069770 msgr2=0x7f5010069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- 172.21.15.71:0/3000393664 shutdown_connections 2023-10-26T05:54:15.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- 172.21.15.71:0/3000393664 wait complete. 2023-10-26T05:54:15.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 Processor -- start 2023-10-26T05:54:15.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- start start 2023-10-26T05:54:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ffc003f00 0x7f501013abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 0x7f50100ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50101f0cb0 0x7f50101f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501012c9b0 con 0x7f50100abbc0 2023-10-26T05:54:15.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501013e970 con 0x7f50101f0cb0 2023-10-26T05:54:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f5017b2d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501013ec50 con 0x7f4ffc003f00 2023-10-26T05:54:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f50150c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 0x7f50100ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f50150c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 0x7f50100ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38638/0 (socket says 172.21.15.71:38638) 2023-10-26T05:54:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.591+0000 7f50150c8640 1 -- 172.21.15.71:0/1856663761 learned_addr learned my addr 172.21.15.71:0/1856663761 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50158c9640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ffc003f00 0x7f501013abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50160ca640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50101f0cb0 0x7f50101f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50150c8640 1 -- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ffc003f00 msgr2=0x7f501013abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50150c8640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ffc003f00 0x7f501013abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50150c8640 1 -- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50101f0cb0 msgr2=0x7f50101f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50150c8640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50101f0cb0 0x7f50101f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.606 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50150c8640 1 -- 172.21.15.71:0/1856663761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501013ef30 con 0x7f50100abbc0 2023-10-26T05:54:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50150c8640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 0x7f50100ac020 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f50000263f0 tx=0x7f5000000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000003600 con 0x7f50100abbc0 2023-10-26T05:54:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5000003850 con 0x7f50100abbc0 2023-10-26T05:54:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501013f210 con 0x7f50100abbc0 2023-10-26T05:54:15.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000003c60 con 0x7f50100abbc0 2023-10-26T05:54:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501013e540 con 0x7f50100abbc0 2023-10-26T05:54:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5010066b10 con 0x7f50100abbc0 2023-10-26T05:54:15.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f50000c4460 con 0x7f50100abbc0 2023-10-26T05:54:15.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50067fc640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ff4066470 0x7f4ff4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:15.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5000122240 con 0x7f50100abbc0 2023-10-26T05:54:15.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50158c9640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ff4066470 0x7f4ff4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:15.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.595+0000 7f50158c9640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ff4066470 0x7f4ff4068930 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f500c009a60 tx=0x7f500c0091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:15.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.599+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f50000ef5e0 con 0x7f50100abbc0 2023-10-26T05:54:15.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.755+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f50100721e0 con 0x7f50100abbc0 2023-10-26T05:54:15.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f50067fc640 1 -- 172.21.15.71:0/1856663761 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v143) v1 ==== 81+0+34258 (secure 0 0 0) 0x7f50000f2c00 con 0x7f50100abbc0 2023-10-26T05:54:15.767 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:15.767 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:15.767 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 143, 2023-10-26T05:54:15.767 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:15.768 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:54:15.768 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:54:12.178962+0000", 2023-10-26T05:54:15.768 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:54:15.768 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:54:15.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:54:15.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:54:15.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:54:15.769 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:54:15.769 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:54:15.770 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:54:15.770 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:54:15.770 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.770 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:54:15.771 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:54:15.771 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:54:15.771 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:54:15.771 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:54:15.771 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 17, 2023-10-26T05:54:15.771 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:54:15.772 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:54:15.772 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "luminous", 2023-10-26T05:54:15.772 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:54:15.772 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:54:15.772 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:54:15.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.773 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:54:15.773 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:54:15.773 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:54:15.773 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:15.774 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:15.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:15.774 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:15.774 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:15.774 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:15.775 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:15.775 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:15.775 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:15.775 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:15.775 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:15.775 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:15.776 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:54:15.776 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:54:15.776 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:54:15.776 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:54:15.776 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:54:15.776 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:15.777 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:15.777 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:15.777 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:15.777 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:15.777 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:15.777 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:15.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.778 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:54:15.778 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:15.778 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:15.778 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:15.779 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:15.779 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:15.779 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:15.779 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:15.779 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:15.779 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:15.780 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:15.780 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:15.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:15.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:15.780 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:15.780 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:15.781 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:15.781 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:15.781 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:15.781 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:15.781 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:15.782 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:15.782 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:15.782 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:15.782 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:15.782 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:15.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:15.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.783 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:15.783 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:15.783 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:15.783 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:15.784 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:15.784 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:15.784 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:15.784 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:15.784 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:15.785 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:15.785 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:15.785 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:15.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:54:15.786 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:54:15.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.786 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:15.786 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:54:15.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.787 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:15.787 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:54:15.787 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:54:15.787 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:54:15.787 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:54:15.788 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:54:15.788 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:15.788 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:15.788 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:15.789 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.789 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 16, 2023-10-26T05:54:15.789 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:54:15.790 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:37.960720+0000", 2023-10-26T05:54:15.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:15.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:15.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:15.790 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:15.791 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:15.791 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:15.791 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:15.791 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:15.791 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:15.792 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:15.792 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:15.792 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:15.792 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:54:15.792 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:54:15.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:54:15.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:54:15.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:54:15.793 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:15.793 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:15.794 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:15.794 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:15.794 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:15.794 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:15.795 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:15.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "137", 2023-10-26T05:54:15.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:15.795 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:15.796 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:15.796 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:15.796 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:15.796 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:15.797 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:15.797 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:15.797 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:15.797 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:54:15.797 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:15.798 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:15.798 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:15.798 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:15.798 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:15.798 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:15.799 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:15.799 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:15.799 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:15.799 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:15.799 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:15.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:15.800 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:15.800 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:15.800 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:15.800 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:15.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.801 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:15.801 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:15.801 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:15.801 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:15.801 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:15.802 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:15.802 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:15.802 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:15.802 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:15.802 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:15.803 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:15.803 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:15.803 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:54:15.803 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:54:15.803 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:54:15.803 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.804 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:15.804 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:15.804 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:54:15.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.804 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:15.805 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.75, 2023-10-26T05:54:15.805 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.75, 2023-10-26T05:54:15.805 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:15.805 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.75, 2023-10-26T05:54:15.805 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.75, 2023-10-26T05:54:15.805 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:15.806 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:15.806 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:15.806 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.806 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.807 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 17, 2023-10-26T05:54:15.807 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:54:15.807 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:53:38.704883+0000", 2023-10-26T05:54:15.807 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:54:15.807 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:54:15.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:54:15.808 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:54:15.808 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:54:15.808 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:54:15.808 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:54:15.809 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:54:15.809 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:54:15.809 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:54:15.809 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:54:15.809 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:54:15.809 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:54:15.810 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:54:15.810 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:54:15.810 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:54:15.810 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:54:15.810 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:54:15.811 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:54:15.811 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:54:15.811 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:54:15.811 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:54:15.811 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:54:15.812 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:54:15.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.812 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "143", 2023-10-26T05:54:15.812 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:54:15.812 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:54:15.813 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:54:15.813 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:54:15.813 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:54:15.813 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:54:15.813 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:54:15.814 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:54:15.814 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:54:15.814 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:54:15.814 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:54:15.814 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:54:15.815 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:54:15.815 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:54:15.815 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:54:15.815 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:54:15.815 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:54:15.816 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:54:15.816 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:54:15.816 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:54:15.816 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:54:15.816 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:54:15.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:54:15.817 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:54:15.817 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:54:15.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:54:15.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.818 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:54:15.818 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:54:15.818 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:54:15.818 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:54:15.818 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:54:15.819 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:54:15.819 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:54:15.819 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:54:15.819 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:54:15.819 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:54:15.819 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:54:15.820 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:54:15.820 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:54:15.820 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:54:15.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.820 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:54:15.820 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:54:15.821 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:54:15.821 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.821 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:54:15.821 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.5, 2023-10-26T05:54:15.822 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.5, 2023-10-26T05:54:15.822 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:54:15.822 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.5, 2023-10-26T05:54:15.822 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.5, 2023-10-26T05:54:15.822 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:54:15.823 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:54:15.823 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:54:15.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.823 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.823 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.823 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:54:15.824 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.824 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:15.824 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.824 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.824 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.825 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.825 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.825 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.825 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.825 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.826 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.826 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.826 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.826 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.826 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.826 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.827 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:54:15.827 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.828 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.828 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:54:15.828 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.828 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.828 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.829 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.829 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.829 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:54:15.830 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.830 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:54:15.831 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.831 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.831 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:54:15.832 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.833 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.833 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:54:15.833 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.834 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.834 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.834 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.834 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.834 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.835 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.835 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:54:15.835 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.836 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:54:15.836 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:54:15.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.836 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.837 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:54:15.837 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:54:15.837 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:54:15.837 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:54:15.837 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.837 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.838 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.838 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.838 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:15.839 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.839 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.839 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.839 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.839 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.839 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.840 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.840 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.840 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.840 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.840 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.840 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.841 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.841 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:54:15.841 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.842 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:54:15.842 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.843 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.843 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.843 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:54:15.844 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.845 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:54:15.845 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.845 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.845 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.845 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:54:15.846 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.847 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:54:15.847 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.848 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.848 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.848 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.848 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.849 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.849 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:54:15.849 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.849 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.850 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:54:15.850 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:54:15.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.851 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.851 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:54:15.851 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:54:15.851 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:54:15.851 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:54:15.852 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.852 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.852 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.852 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.853 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:15.853 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.853 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.854 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.854 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.854 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.854 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.854 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.855 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.855 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 141, 2023-10-26T05:54:15.855 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.855 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.855 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.856 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.856 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.856 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:54:15.856 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.857 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.857 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:54:15.857 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.858 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.858 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.858 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.859 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:54:15.859 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.860 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:54:15.860 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.860 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.861 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.861 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.861 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.862 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:54:15.862 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.863 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.863 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:54:15.863 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.863 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.863 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.864 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.864 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.864 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:54:15.865 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.865 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.865 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:54:15.866 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:54:15.866 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.866 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.866 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:54:15.867 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:54:15.867 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:54:15.867 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:54:15.867 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.867 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.868 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.868 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.868 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:15.869 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.869 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.869 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.869 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.869 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.869 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.870 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.870 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.870 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.870 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.871 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.871 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.871 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:54:15.872 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.872 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:54:15.873 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.873 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.873 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.874 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.874 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.874 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:54:15.874 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.875 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:54:15.875 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.876 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.876 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.876 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.877 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:54:15.877 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.877 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.878 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:54:15.878 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.878 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.878 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.879 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.879 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.879 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.879 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.879 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:54:15.879 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.880 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:54:15.880 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:54:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:54:15.881 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:54:15.882 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:54:15.882 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:54:15.882 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.882 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.882 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.883 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.883 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:15.883 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.883 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.884 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.884 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.884 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.884 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.884 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.884 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.885 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.885 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.885 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.885 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.885 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.886 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.886 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:54:15.886 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.887 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:54:15.887 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.887 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:54:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.889 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:54:15.889 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.890 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.890 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.890 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.890 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.891 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.891 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:54:15.891 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.891 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.892 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:54:15.892 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.892 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.893 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.893 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.893 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.893 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:54:15.894 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.894 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.894 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.894 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.894 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:54:15.895 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:54:15.895 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.895 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.895 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:54:15.896 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:54:15.896 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:54:15.896 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:54:15.896 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.897 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.897 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.897 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.898 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:15.898 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.898 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.898 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.898 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 141, 2023-10-26T05:54:15.899 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.900 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.900 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.900 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.900 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.900 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:54:15.901 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:54:15.902 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.902 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:54:15.903 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.904 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:54:15.904 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.904 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.904 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.905 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.905 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.905 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.905 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.905 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:54:15.906 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.906 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:54:15.907 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.907 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.907 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.908 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.908 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.908 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:54:15.908 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.909 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:54:15.909 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:54:15.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.910 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.910 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:54:15.910 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:54:15.910 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:54:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:54:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.912 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:15.912 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.912 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.912 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.913 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.913 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.913 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.913 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.913 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.913 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.914 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.914 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.914 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.914 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:54:15.915 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.916 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:54:15.916 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.916 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:54:15.917 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.918 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:54:15.918 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.918 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.919 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.919 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.920 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:54:15.920 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.920 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.920 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:54:15.921 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.921 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.921 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.921 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.921 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.922 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.922 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:54:15.922 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.923 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.923 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.923 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:54:15.923 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:54:15.923 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.923 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.924 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:54:15.924 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:54:15.924 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:54:15.924 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:54:15.924 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.925 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.925 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.925 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.926 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:15.926 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:54:15.926 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:54:15.926 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:54:15.926 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:54:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:54:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:54:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:54:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:54:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 136, 2023-10-26T05:54:15.927 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:54:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:54:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:54:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:54:15.929 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:54:15.929 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.930 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:54:15.930 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.931 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:54:15.931 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:54:15.932 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.932 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.933 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:54:15.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:54:15.934 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:54:15.935 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.935 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:54:15.935 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:54:15.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:54:15.936 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:54:15.936 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:15.937 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:54:15.937 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:54:15.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.937 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.938 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:54:15.938 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:54:15.938 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:54:15.938 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:54:15.938 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:54:15.939 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:54:15.939 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:54:15.939 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.939 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.939 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.940 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:54:15.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.940 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:54:15.940 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.940 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.940 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.941 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.941 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.941 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:54:15.941 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.941 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.942 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:54:15.942 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.942 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.942 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.942 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.943 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.943 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:54:15.943 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.943 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:54:15.944 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.944 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.944 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.944 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.944 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.944 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:54:15.945 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.945 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:54:15.945 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.946 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.946 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.946 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.946 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.946 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:54:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:54:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:54:15.949 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.949 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:54:15.949 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.949 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.950 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.950 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.950 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.950 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:54:15.950 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.951 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:54:15.951 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.951 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.951 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.952 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.952 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.952 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:54:15.952 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.953 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:54:15.953 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:54:15.953 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:54:15.953 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:54:15.953 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:54:15.953 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:54:15.954 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:54:15.954 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:54:15.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.954 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.954 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:54:15.955 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [ 2023-10-26T05:54:15.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.955 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "17.7", 2023-10-26T05:54:15.955 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:54:15.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.955 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:54:15.956 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 5 2023-10-26T05:54:15.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.956 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.956 INFO:teuthology.orchestra.run.smithi071.stdout: "pgid": "17.3c", 2023-10-26T05:54:15.957 INFO:teuthology.orchestra.run.smithi071.stdout: "mappings": [ 2023-10-26T05:54:15.957 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:15.957 INFO:teuthology.orchestra.run.smithi071.stdout: "from": 0, 2023-10-26T05:54:15.957 INFO:teuthology.orchestra.run.smithi071.stdout: "to": 2 2023-10-26T05:54:15.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.957 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:15.958 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.958 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:54:15.958 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:54:15.958 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:54:15.959 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:54:15.959 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:54:15.959 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:54:15.959 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:54:15.959 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:54:15.959 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:54:15.960 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:54:15.960 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:54:15.960 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:54:15.960 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:54:15.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:15.961 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:54:15.961 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:54:15.961 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:54:15.961 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:54:15.961 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:54:15.962 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:54:15.962 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:54:15.962 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:54:15.962 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:54:15.962 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:54:15.963 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:54:15.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:15.963 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:15.964 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:54:15.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ff4066470 msgr2=0x7f4ff4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ff4066470 0x7f4ff4068930 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f500c009a60 tx=0x7f500c0091e0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 msgr2=0x7f50100ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:15.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 0x7f50100ac020 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f50000263f0 tx=0x7f5000000f60 comp rx=0 tx=0).stop 2023-10-26T05:54:15.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 shutdown_connections 2023-10-26T05:54:15.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f50101f0cb0 0x7f50101f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ff4066470 0x7f4ff4068930 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50100abbc0 0x7f50100ac020 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 --2- 172.21.15.71:0/1856663761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ffc003f00 0x7f501013abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:15.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 >> 172.21.15.71:0/1856663761 conn(0x7f5010069770 msgr2=0x7f5010158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:15.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 shutdown_connections 2023-10-26T05:54:15.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:15.759+0000 7f5017b2d640 1 -- 172.21.15.71:0/1856663761 wait complete. 2023-10-26T05:54:15.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:16.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:54:16.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/2443310944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d74003fe0 msgr2=0x7f8d740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:16.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/2443310944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d74003fe0 0x7f8d740047d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000c260 tx=0x7f8d8001f740 comp rx=0 tx=0).stop 2023-10-26T05:54:16.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 -- 172.21.15.71:0/2443310944 reap_dead start 2023-10-26T05:54:16.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/2443310944 shutdown_connections 2023-10-26T05:54:16.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/2443310944 >> 172.21.15.71:0/2443310944 conn(0x7f8d8c069770 msgr2=0x7f8d8c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:16.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/2443310944 shutdown_connections 2023-10-26T05:54:16.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/2443310944 wait complete. 2023-10-26T05:54:16.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 Processor -- start 2023-10-26T05:54:16.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- start start 2023-10-26T05:54:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d74003fe0 0x7f8d8c1359b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0ac1f0 0x7f8d8c1f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 0x7f8d8c157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:16.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c135ef0 con 0x7f8d8c0b0140 2023-10-26T05:54:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0aa8e0 con 0x7f8d8c0ac1f0 2023-10-26T05:54:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c1f24b0 con 0x7f8d74003fe0 2023-10-26T05:54:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 0x7f8d8c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 0x7f8d8c157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38662/0 (socket says 172.21.15.71:38662) 2023-10-26T05:54:16.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 -- 172.21.15.71:0/3858931105 learned_addr learned my addr 172.21.15.71:0/3858931105 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:16.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d8bfff640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d74003fe0 0x7f8d8c1359b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:16.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d8b7fe640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0ac1f0 0x7f8d8c1f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:16.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 -- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d74003fe0 msgr2=0x7f8d8c1359b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:16.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d74003fe0 0x7f8d8c1359b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:16.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 -- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0ac1f0 msgr2=0x7f8d8c1f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0ac1f0 0x7f8d8c1f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 -- 172.21.15.71:0/3858931105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c1f2790 con 0x7f8d8c0b0140 2023-10-26T05:54:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d8bfff640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d74003fe0 0x7f8d8c1359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:54:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d90fe9640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 0x7f8d8c157d60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c072f40 tx=0x7f8d7c00da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c006440 con 0x7f8d8c0b0140 2023-10-26T05:54:16.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d7c003600 con 0x7f8d8c0b0140 2023-10-26T05:54:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c1f2a70 con 0x7f8d8c0b0140 2023-10-26T05:54:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c003a30 con 0x7f8d8c0b0140 2023-10-26T05:54:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.123+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c1f2130 con 0x7f8d8c0b0140 2023-10-26T05:54:16.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.127+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c066b10 con 0x7f8d8c0b0140 2023-10-26T05:54:16.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.127+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d7c003b90 con 0x7f8d8c0b0140 2023-10-26T05:54:16.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.127+0000 7f8d88ff9640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d780664a0 0x7f8d78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:16.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.127+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8d7c0836d0 con 0x7f8d8c0b0140 2023-10-26T05:54:16.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.127+0000 7f8d8bfff640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d780664a0 0x7f8d78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:16.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.127+0000 7f8d8bfff640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d780664a0 0x7f8d78068960 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009e70 tx=0x7f8d84009460 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:16.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.131+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d7c050b10 con 0x7f8d8c0b0140 2023-10-26T05:54:16.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d8c0721e0 con 0x7f8d8c0b0140 2023-10-26T05:54:16.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d88ff9640 1 -- 172.21.15.71:0/3858931105 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f8d7c016090 con 0x7f8d8c0b0140 2023-10-26T05:54:16.315 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:16.315 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:54:16.316 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:54:16.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d780664a0 msgr2=0x7f8d78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:16.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d780664a0 0x7f8d78068960 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009e70 tx=0x7f8d84009460 comp rx=0 tx=0).stop 2023-10-26T05:54:16.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 msgr2=0x7f8d8c157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:16.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 0x7f8d8c157d60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c072f40 tx=0x7f8d7c00da10 comp rx=0 tx=0).stop 2023-10-26T05:54:16.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 shutdown_connections 2023-10-26T05:54:16.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d780664a0 0x7f8d78068960 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:16.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b0140 0x7f8d8c157d60 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:16.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0ac1f0 0x7f8d8c1f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:16.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 --2- 172.21.15.71:0/3858931105 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d74003fe0 0x7f8d8c1359b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:16.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 >> 172.21.15.71:0/3858931105 conn(0x7f8d8c069770 msgr2=0x7f8d8c0ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:16.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 shutdown_connections 2023-10-26T05:54:16.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:16.307+0000 7f8d92a4c640 1 -- 172.21.15.71:0/3858931105 wait complete. 2023-10-26T05:54:16.333 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T05:54:16.503 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":8007,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":36.077997695999997,"mdsmap_epoch":51,"osdmap_epoch":143,"osdmap_epoch_barrier":140,"uptime":40.924699424000003} 2023-10-26T05:54:16.504 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 8007, "mdsmap_epoch": 51, "osdmap_epoch": 143, "osdmap_epoch_barrier": 140, "rank_uptime": 36.077997696, "state": "up:active", "uptime": 40.924699424, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:54:16.504 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:54:16.504 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:54:16.505 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:16.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:16.511 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T05:54:16.926 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records in 2023-10-26T05:54:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records out 2023-10-26T05:54:16.927 INFO:teuthology.orchestra.run.smithi071.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.366698 s, 143 MB/s 2023-10-26T05:54:16.928 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:54:16.928 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:54:16.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:54:17.014 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:54:17.015 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:54:17.015 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:17.021 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:17.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/1290148405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee7413c2f0 msgr2=0x7fee740b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/1290148405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee7413c2f0 0x7fee740b2350 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fee740ac6e0 tx=0x7fee7001c880 comp rx=0 tx=0).stop 2023-10-26T05:54:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/1290148405 shutdown_connections 2023-10-26T05:54:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/1290148405 >> v2:172.21.15.86:3300/0 conn(0x7fee740cb1e0 0x7fee74159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/1290148405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee7413c2f0 0x7fee740b2350 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/1290148405 >> 172.21.15.71:0/1290148405 conn(0x7fee74069770 msgr2=0x7fee74069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/1290148405 shutdown_connections 2023-10-26T05:54:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/1290148405 wait complete. 2023-10-26T05:54:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 Processor -- start 2023-10-26T05:54:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- start start 2023-10-26T05:54:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 0x7fee7413e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee740cb1e0 0x7fee741377a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7413c2f0 0x7fee74135e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7413d5e0 con 0x7fee740b2890 2023-10-26T05:54:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7413d8c0 con 0x7fee7413c2f0 2023-10-26T05:54:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7413dba0 con 0x7fee740cb1e0 2023-10-26T05:54:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 0x7fee7413e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 0x7fee7413e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38680/0 (socket says 172.21.15.71:38680) 2023-10-26T05:54:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 -- 172.21.15.71:0/3042048098 learned_addr learned my addr 172.21.15.71:0/3042048098 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7894c640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee740cb1e0 0x7fee741377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 -- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee740cb1e0 msgr2=0x7fee741377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7994e640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7413c2f0 0x7fee74135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee740cb1e0 0x7fee741377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 -- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7413c2f0 msgr2=0x7fee74135e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7413c2f0 0x7fee74135e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 -- 172.21.15.71:0/3042048098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70009560 con 0x7fee740b2890 2023-10-26T05:54:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7894c640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee740cb1e0 0x7fee741377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7994e640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7413c2f0 0x7fee74135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:17.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7914d640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 0x7fee7413e680 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fee7001cd60 tx=0x7fee70005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:17.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70004240 con 0x7fee740b2890 2023-10-26T05:54:17.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee70004490 con 0x7fee740b2890 2023-10-26T05:54:17.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7413de80 con 0x7fee740b2890 2023-10-26T05:54:17.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70003840 con 0x7fee740b2890 2023-10-26T05:54:17.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.191+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee74137ce0 con 0x7fee740b2890 2023-10-26T05:54:17.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.195+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee74066b10 con 0x7fee740b2890 2023-10-26T05:54:17.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.195+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fee700b4040 con 0x7fee740b2890 2023-10-26T05:54:17.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.195+0000 7fee69ffb640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066470 0x7fee58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.195+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fee7011e6e0 con 0x7fee740b2890 2023-10-26T05:54:17.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.195+0000 7fee7894c640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066470 0x7fee58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.195+0000 7fee7894c640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066470 0x7fee58068930 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fee740ac650 tx=0x7fee64009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:17.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.199+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fee700ebaa0 con 0x7fee740b2890 2023-10-26T05:54:17.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.355+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee740721e0 con 0x7fee740b2890 2023-10-26T05:54:17.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee69ffb640 1 -- 172.21.15.71:0/3042048098 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16544 (secure 0 0 0) 0x7fee700ef0c0 con 0x7fee740b2890 2023-10-26T05:54:17.367 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:17.367 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":143,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:12.178962+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066470 msgr2=0x7fee58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066470 0x7fee58068930 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fee740ac650 tx=0x7fee64009040 comp rx=0 tx=0).stop 2023-10-26T05:54:17.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 msgr2=0x7fee7413e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 0x7fee7413e680 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fee7001cd60 tx=0x7fee70005e70 comp rx=0 tx=0).stop 2023-10-26T05:54:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 shutdown_connections 2023-10-26T05:54:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7413c2f0 0x7fee74135e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee740cb1e0 0x7fee741377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee58066470 0x7fee58068930 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 --2- 172.21.15.71:0/3042048098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee740b2890 0x7fee7413e680 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 >> 172.21.15.71:0/3042048098 conn(0x7fee74069770 msgr2=0x7fee740b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 shutdown_connections 2023-10-26T05:54:17.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.359+0000 7fee7b3b1640 1 -- 172.21.15.71:0/3042048098 wait complete. 2023-10-26T05:54:17.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:53:37.960720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:53:38.704883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:54:17.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:54:17.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3261148818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f80cc240 msgr2=0x7fe9f80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3261148818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f80cc240 0x7fe9f80cc6d0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009900 tx=0x7fe9f001c730 comp rx=0 tx=0).stop 2023-10-26T05:54:17.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3261148818 shutdown_connections 2023-10-26T05:54:17.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3261148818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f80cc240 0x7fe9f80cc6d0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3261148818 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9f815bbb0 0x7fe9f815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3261148818 >> 172.21.15.71:0/3261148818 conn(0x7fe9f80caaa0 msgr2=0x7fe9f80caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:17.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3261148818 shutdown_connections 2023-10-26T05:54:17.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3261148818 wait complete. 2023-10-26T05:54:17.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 Processor -- start 2023-10-26T05:54:17.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- start start 2023-10-26T05:54:17.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9f806b2e0 0x7fe9f814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f80cc240 0x7fe9f81473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 0x7fe9f8147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f815b5c0 con 0x7fe9f815bbb0 2023-10-26T05:54:17.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8148a70 con 0x7fe9f80cc240 2023-10-26T05:54:17.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f8148d50 con 0x7fe9f806b2e0 2023-10-26T05:54:17.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 0x7fe9f8147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 0x7fe9f8147960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38694/0 (socket says 172.21.15.71:38694) 2023-10-26T05:54:17.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 -- 172.21.15.71:0/3922272314 learned_addr learned my addr 172.21.15.71:0/3922272314 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:17.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fca58640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9f806b2e0 0x7fe9f814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9f7fff640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f80cc240 0x7fe9f81473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 -- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9f806b2e0 msgr2=0x7fe9f814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9f806b2e0 0x7fe9f814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 -- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f80cc240 msgr2=0x7fe9f81473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f80cc240 0x7fe9f81473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 -- 172.21.15.71:0/3922272314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009560 con 0x7fe9f815bbb0 2023-10-26T05:54:17.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fd259640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 0x7fe9f8147960 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009850 tx=0x7fe9f00a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:17.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00043c0 con 0x7fe9f815bbb0 2023-10-26T05:54:17.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe9f0005020 con 0x7fe9f815bbb0 2023-10-26T05:54:17.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f815ef40 con 0x7fe9f815bbb0 2023-10-26T05:54:17.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0005410 con 0x7fe9f815bbb0 2023-10-26T05:54:17.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.507+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f815f500 con 0x7fe9f815bbb0 2023-10-26T05:54:17.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.511+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f814b280 con 0x7fe9f815bbb0 2023-10-26T05:54:17.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.511+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9f00b4040 con 0x7fe9f815bbb0 2023-10-26T05:54:17.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.511+0000 7fe9f57fa640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9e4066470 0x7fe9e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.511+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe9f011de40 con 0x7fe9f815bbb0 2023-10-26T05:54:17.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.511+0000 7fe9fca58640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9e4066470 0x7fe9e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.511+0000 7fe9fca58640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9e4066470 0x7fe9e4068930 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe9f81f5d60 tx=0x7fe9e800a780 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:17.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.515+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe9f814b280 con 0x7fe9f815bbb0 2023-10-26T05:54:17.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.667+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9f814b280 con 0x7fe9f815bbb0 2023-10-26T05:54:17.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.667+0000 7fe9f57fa640 1 -- 172.21.15.71:0/3922272314 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9f00b2020 con 0x7fe9f815bbb0 2023-10-26T05:54:17.676 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:54:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9e4066470 msgr2=0x7fe9e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9e4066470 0x7fe9e4068930 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe9f81f5d60 tx=0x7fe9e800a780 comp rx=0 tx=0).stop 2023-10-26T05:54:17.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 msgr2=0x7fe9f8147960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 0x7fe9f8147960 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009850 tx=0x7fe9f00a5870 comp rx=0 tx=0).stop 2023-10-26T05:54:17.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 shutdown_connections 2023-10-26T05:54:17.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe9e4066470 0x7fe9e4068930 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f815bbb0 0x7fe9f8147960 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9f80cc240 0x7fe9f81473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 --2- 172.21.15.71:0/3922272314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9f806b2e0 0x7fe9f814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 >> 172.21.15.71:0/3922272314 conn(0x7fe9f80caaa0 msgr2=0x7fe9f8070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:17.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 shutdown_connections 2023-10-26T05:54:17.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.671+0000 7fe9fecbc640 1 -- 172.21.15.71:0/3922272314 wait complete. 2023-10-26T05:54:17.694 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:54:17.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- 172.21.15.71:0/3052581044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 msgr2=0x7f4e4815bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/3052581044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815bab0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f4e4808a140 tx=0x7f4e4001caa0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- 172.21.15.71:0/3052581044 shutdown_connections 2023-10-26T05:54:17.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/3052581044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815bab0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/3052581044 >> v2:172.21.15.86:3300/0 conn(0x7f4e480d0470 0x7f4e4806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- 172.21.15.71:0/3052581044 >> 172.21.15.71:0/3052581044 conn(0x7f4e480cdd80 msgr2=0x7f4e480ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:17.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- 172.21.15.71:0/3052581044 shutdown_connections 2023-10-26T05:54:17.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- 172.21.15.71:0/3052581044 wait complete. 2023-10-26T05:54:17.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 Processor -- start 2023-10-26T05:54:17.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- start start 2023-10-26T05:54:17.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e480d0470 0x7f4e480c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e480d0850 0x7f4e480c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480c5ac0 con 0x7f4e480cd880 2023-10-26T05:54:17.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480cb520 con 0x7f4e480d0850 2023-10-26T05:54:17.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4f7de640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e480ca9d0 con 0x7f4e480d0470 2023-10-26T05:54:17.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38708/0 (socket says 172.21.15.71:38708) 2023-10-26T05:54:17.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 -- 172.21.15.71:0/2171653440 learned_addr learned my addr 172.21.15.71:0/2171653440 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:17.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4dd7b640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e480d0850 0x7f4e480c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4cd79640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e480d0470 0x7f4e480c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 -- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e480d0470 msgr2=0x7f4e480c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e480d0470 0x7f4e480c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 -- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e480d0850 msgr2=0x7f4e480c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e480d0850 0x7f4e480c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.811+0000 7f4e4d57a640 1 -- 172.21.15.71:0/2171653440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009820 con 0x7f4e480cd880 2023-10-26T05:54:17.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e4d57a640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815e870 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c970 tx=0x7f4e4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:17.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44007c70 con 0x7f4e480cd880 2023-10-26T05:54:17.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4e4400f450 con 0x7f4e480cd880 2023-10-26T05:54:17.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400f860 con 0x7f4e480cd880 2023-10-26T05:54:17.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e480ca6f0 con 0x7f4e480cd880 2023-10-26T05:54:17.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e480c56e0 con 0x7f4e480cd880 2023-10-26T05:54:17.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e48066b10 con 0x7f4e480cd880 2023-10-26T05:54:17.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4e4400f9c0 con 0x7f4e480cd880 2023-10-26T05:54:17.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e3dffb640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e2c066470 0x7f4e2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:17.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4e44084740 con 0x7f4e480cd880 2023-10-26T05:54:17.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e4cd79640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e2c066470 0x7f4e2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:17.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.815+0000 7f4e4cd79640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e2c066470 0x7f4e2c068930 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4e480b2210 tx=0x7f4e38009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:17.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.819+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4e44051c10 con 0x7f4e480cd880 2023-10-26T05:54:17.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e480721e0 con 0x7f4e480cd880 2023-10-26T05:54:17.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e3dffb640 1 -- 172.21.15.71:0/2171653440 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e44055230 con 0x7f4e480cd880 2023-10-26T05:54:17.983 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:54:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e2c066470 msgr2=0x7f4e2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e2c066470 0x7f4e2c068930 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4e480b2210 tx=0x7f4e38009040 comp rx=0 tx=0).stop 2023-10-26T05:54:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 msgr2=0x7f4e4815e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815e870 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400c970 tx=0x7f4e4400ce30 comp rx=0 tx=0).stop 2023-10-26T05:54:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 shutdown_connections 2023-10-26T05:54:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e480d0850 0x7f4e480c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4e480d0470 0x7f4e480c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4e2c066470 0x7f4e2c068930 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 --2- 172.21.15.71:0/2171653440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e480cd880 0x7f4e4815e870 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 >> 172.21.15.71:0/2171653440 conn(0x7f4e480cdd80 msgr2=0x7f4e481549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 shutdown_connections 2023-10-26T05:54:17.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:17.975+0000 7f4e4f7de640 1 -- 172.21.15.71:0/2171653440 wait complete. 2023-10-26T05:54:18.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:54:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- 172.21.15.71:0/2739826183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46013c220 msgr2=0x7fc46015cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 --2- 172.21.15.71:0/2739826183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46013c220 0x7fc46015cce0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc46006c770 tx=0x7fc45c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- 172.21.15.71:0/2739826183 shutdown_connections 2023-10-26T05:54:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 --2- 172.21.15.71:0/2739826183 >> v2:172.21.15.86:3300/0 conn(0x7fc46015d220 0x7fc4600cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 --2- 172.21.15.71:0/2739826183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46013c220 0x7fc46015cce0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- 172.21.15.71:0/2739826183 >> 172.21.15.71:0/2739826183 conn(0x7fc4600aec60 msgr2=0x7fc4600af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:18.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- 172.21.15.71:0/2739826183 shutdown_connections 2023-10-26T05:54:18.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- 172.21.15.71:0/2739826183 wait complete. 2023-10-26T05:54:18.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 Processor -- start 2023-10-26T05:54:18.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- start start 2023-10-26T05:54:18.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4600cbd60 0x7fc460135c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc46013c220 0x7fc46013ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 0x7fc460137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc460139af0 con 0x7fc46015d220 2023-10-26T05:54:18.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc460139dd0 con 0x7fc46013c220 2023-10-26T05:54:18.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc4680d6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46013a0b0 con 0x7fc4600cbd60 2023-10-26T05:54:18.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 0x7fc460137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 0x7fc460137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38720/0 (socket says 172.21.15.71:38720) 2023-10-26T05:54:18.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 -- 172.21.15.71:0/1496450794 learned_addr learned my addr 172.21.15.71:0/1496450794 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:18.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc465e72640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4600cbd60 0x7fc460135c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc465671640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc46013c220 0x7fc46013ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 -- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4600cbd60 msgr2=0x7fc460135c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4600cbd60 0x7fc460135c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 -- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc46013c220 msgr2=0x7fc46013ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc46013c220 0x7fc46013ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.127+0000 7fc466673640 1 -- 172.21.15.71:0/1496450794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c009560 con 0x7fc46015d220 2023-10-26T05:54:18.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc466673640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 0x7fc460137220 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc45800c780 tx=0x7fc45800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:18.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458003d90 con 0x7fc46015d220 2023-10-26T05:54:18.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc45800ce90 con 0x7fc46015d220 2023-10-26T05:54:18.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580052d0 con 0x7fc46015d220 2023-10-26T05:54:18.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46013a390 con 0x7fc46015d220 2023-10-26T05:54:18.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46013a950 con 0x7fc46015d220 2023-10-26T05:54:18.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc460066b10 con 0x7fc46015d220 2023-10-26T05:54:18.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc458005430 con 0x7fc46015d220 2023-10-26T05:54:18.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4567fc640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc444066470 0x7fc444068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc458083640 con 0x7fc46015d220 2023-10-26T05:54:18.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc465e72640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc444066470 0x7fc444068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.131+0000 7fc465e72640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc444066470 0x7fc444068930 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0af000 tx=0x7fc45c01afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:18.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.135+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc458050a80 con 0x7fc46015d220 2023-10-26T05:54:18.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.283+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4600721e0 con 0x7fc46015d220 2023-10-26T05:54:18.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.283+0000 7fc4567fc640 1 -- 172.21.15.71:0/1496450794 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4580540a0 con 0x7fc46015d220 2023-10-26T05:54:18.292 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:54:18.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc444066470 msgr2=0x7fc444068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc444066470 0x7fc444068930 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0af000 tx=0x7fc45c01afe0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 msgr2=0x7fc460137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 0x7fc460137220 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc45800c780 tx=0x7fc45800cc40 comp rx=0 tx=0).stop 2023-10-26T05:54:18.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 shutdown_connections 2023-10-26T05:54:18.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc444066470 0x7fc444068930 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc46015d220 0x7fc460137220 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc46013c220 0x7fc46013ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 --2- 172.21.15.71:0/1496450794 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc4600cbd60 0x7fc460135c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 >> 172.21.15.71:0/1496450794 conn(0x7fc4600aec60 msgr2=0x7fc4600cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:18.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 shutdown_connections 2023-10-26T05:54:18.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.287+0000 7fc4680d6640 1 -- 172.21.15.71:0/1496450794 wait complete. 2023-10-26T05:54:18.312 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:54:18.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/430477712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f263c003980 msgr2=0x7f263c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 --2- 172.21.15.71:0/430477712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f263c003980 0x7f263c003da0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f26440016b0 tx=0x7f264401c750 comp rx=0 tx=0).stop 2023-10-26T05:54:18.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/430477712 shutdown_connections 2023-10-26T05:54:18.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 --2- 172.21.15.71:0/430477712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f263c003980 0x7f263c003da0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/430477712 >> 172.21.15.71:0/430477712 conn(0x7f265406b8c0 msgr2=0x7f265406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:18.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/430477712 shutdown_connections 2023-10-26T05:54:18.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/430477712 wait complete. 2023-10-26T05:54:18.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 Processor -- start 2023-10-26T05:54:18.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- start start 2023-10-26T05:54:18.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f263c003980 0x7f2654152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265414f730 0x7f2654152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 0x7f2654157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654157060 con 0x7f26541502a0 2023-10-26T05:54:18.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654152670 con 0x7f265414f730 2023-10-26T05:54:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2654156470 con 0x7f263c003980 2023-10-26T05:54:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 0x7f2654157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 0x7f2654157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38734/0 (socket says 172.21.15.71:38734) 2023-10-26T05:54:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 -- 172.21.15.71:0/94958668 learned_addr learned my addr 172.21.15.71:0/94958668 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2652ffd640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265414f730 0x7f2654152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f26537fe640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f263c003980 0x7f2654152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 -- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f263c003980 msgr2=0x7f2654152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f263c003980 0x7f2654152cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 -- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265414f730 msgr2=0x7f2654152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265414f730 0x7f2654152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 -- 172.21.15.71:0/94958668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265414fbb0 con 0x7f26541502a0 2023-10-26T05:54:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2652ffd640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265414f730 0x7f2654152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f2653fff640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 0x7f2654157480 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f265406b110 tx=0x7f264c00fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c003d90 con 0x7f26541502a0 2023-10-26T05:54:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f264c004b30 con 0x7f26541502a0 2023-10-26T05:54:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26541566c0 con 0x7f26541502a0 2023-10-26T05:54:18.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c00d900 con 0x7f26541502a0 2023-10-26T05:54:18.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.431+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2654156c80 con 0x7f26541502a0 2023-10-26T05:54:18.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.435+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2654066b10 con 0x7f26541502a0 2023-10-26T05:54:18.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.435+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f264c019070 con 0x7f26541502a0 2023-10-26T05:54:18.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.435+0000 7f263bfff640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2640066470 0x7f2640068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.435+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f264c084290 con 0x7f26541502a0 2023-10-26T05:54:18.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.435+0000 7f26537fe640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2640066470 0x7f2640068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.435+0000 7f26537fe640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2640066470 0x7f2640068930 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2644001680 tx=0x7f26440194f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:18.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.439+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f264c051810 con 0x7f26541502a0 2023-10-26T05:54:18.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.603+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26541571a0 con 0x7f26541502a0 2023-10-26T05:54:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f263bfff640 1 -- 172.21.15.71:0/94958668 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v143) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26541571a0 con 0x7f26541502a0 2023-10-26T05:54:18.614 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:54:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2640066470 msgr2=0x7f2640068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2640066470 0x7f2640068930 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2644001680 tx=0x7f26440194f0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 msgr2=0x7f2654157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 0x7f2654157480 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f265406b110 tx=0x7f264c00fe20 comp rx=0 tx=0).stop 2023-10-26T05:54:18.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 shutdown_connections 2023-10-26T05:54:18.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2640066470 0x7f2640068930 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26541502a0 0x7f2654157480 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265414f730 0x7f2654152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 --2- 172.21.15.71:0/94958668 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f263c003980 0x7f2654152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 >> 172.21.15.71:0/94958668 conn(0x7f265406b8c0 msgr2=0x7f26540b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:18.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 shutdown_connections 2023-10-26T05:54:18.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.607+0000 7f265a0a7640 1 -- 172.21.15.71:0/94958668 wait complete. 2023-10-26T05:54:18.632 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:54:18.632 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:18.632 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:18.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:18.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[49371]: fuse finished with error 0 and tester_r 0 2023-10-26T05:54:18.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:18.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:54:18.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:18.697 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:54:18.725 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:54:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:18.733 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:54:18.733 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:18.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:18.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 -- 172.21.15.71:0/1938245509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 msgr2=0x7f745c159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 --2- 172.21.15.71:0/1938245509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c159810 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f7454009c80 tx=0x7f745401cd40 comp rx=0 tx=0).stop 2023-10-26T05:54:18.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 -- 172.21.15.71:0/1938245509 shutdown_connections 2023-10-26T05:54:18.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 --2- 172.21.15.71:0/1938245509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c159810 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 --2- 172.21.15.71:0/1938245509 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f745c13c2f0 0x7f745c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 -- 172.21.15.71:0/1938245509 >> 172.21.15.71:0/1938245509 conn(0x7f745c069770 msgr2=0x7f745c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:18.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 -- 172.21.15.71:0/1938245509 shutdown_connections 2023-10-26T05:54:18.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 -- 172.21.15.71:0/1938245509 wait complete. 2023-10-26T05:54:18.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.879+0000 7f7460a65640 1 Processor -- start 2023-10-26T05:54:18.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- start start 2023-10-26T05:54:18.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c0b2890 0x7f745c136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c13c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f745c13c2f0 0x7f745c13cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c153ed0 con 0x7f745c0cb1e0 2023-10-26T05:54:18.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745c1374e0 con 0x7f745c0b2890 2023-10-26T05:54:18.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f745c137e20 con 0x7f745c13c2f0 2023-10-26T05:54:18.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c13c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c13c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38750/0 (socket says 172.21.15.71:38750) 2023-10-26T05:54:18.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 -- 172.21.15.71:0/1739054877 learned_addr learned my addr 172.21.15.71:0/1739054877 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:18.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f745ad9d640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f745c13c2f0 0x7f745c13cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f745a59c640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c0b2890 0x7f745c136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 -- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f745c13c2f0 msgr2=0x7f745c13cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f745c13c2f0 0x7f745c13cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 -- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c0b2890 msgr2=0x7f745c136fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:18.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c0b2890 0x7f745c136fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:18.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 -- 172.21.15.71:0/1739054877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540098e0 con 0x7f745c0cb1e0 2023-10-26T05:54:18.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7459d9b640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c13c820 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f745000c450 tx=0x7f745000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:18.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450012830 con 0x7f745c0cb1e0 2023-10-26T05:54:18.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7450012aa0 con 0x7f745c0cb1e0 2023-10-26T05:54:18.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c154700 con 0x7f745c0cb1e0 2023-10-26T05:54:18.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450011380 con 0x7f745c0cb1e0 2023-10-26T05:54:18.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745c1540b0 con 0x7f745c0cb1e0 2023-10-26T05:54:18.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c066b10 con 0x7f745c0cb1e0 2023-10-26T05:54:18.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f74500114e0 con 0x7f745c0cb1e0 2023-10-26T05:54:18.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f744affd640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f743c0664a0 0x7f743c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:18.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7450084100 con 0x7f745c0cb1e0 2023-10-26T05:54:18.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.883+0000 7f745a59c640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f743c0664a0 0x7f743c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:18.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.887+0000 7f745a59c640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f743c0664a0 0x7f743c068960 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f744c005fd0 tx=0x7f744c005d10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:18.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:18.887+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7450051410 con 0x7f745c0cb1e0 2023-10-26T05:54:19.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.043+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f745c13f7a0 con 0x7f745c0cb1e0 2023-10-26T05:54:19.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.047+0000 7f744affd640 1 -- 172.21.15.71:0/1739054877 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16544 (secure 0 0 0) 0x7f7450054a30 con 0x7f745c0cb1e0 2023-10-26T05:54:19.056 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:19.056 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":143,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:12.178962+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f743c0664a0 msgr2=0x7f743c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f743c0664a0 0x7f743c068960 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f744c005fd0 tx=0x7f744c005d10 comp rx=0 tx=0).stop 2023-10-26T05:54:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 msgr2=0x7f745c13c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c13c820 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f745000c450 tx=0x7f745000c910 comp rx=0 tx=0).stop 2023-10-26T05:54:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 shutdown_connections 2023-10-26T05:54:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f745c13c2f0 0x7f745c13cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f743c0664a0 0x7f743c068960 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f745c0cb1e0 0x7f745c13c820 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 --2- 172.21.15.71:0/1739054877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f745c0b2890 0x7f745c136fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 >> 172.21.15.71:0/1739054877 conn(0x7f745c069770 msgr2=0x7f745c137fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 shutdown_connections 2023-10-26T05:54:19.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.051+0000 7f7460a65640 1 -- 172.21.15.71:0/1739054877 wait complete. 2023-10-26T05:54:19.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:53:37.960720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:53:38.704883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:54:19.075 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:54:19.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/3837858885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba806a710 msgr2=0x7fbba80ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 --2- 172.21.15.71:0/3837858885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba806a710 0x7fbba80ca420 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbba80afc50 tx=0x7fbba001cca0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/3837858885 shutdown_connections 2023-10-26T05:54:19.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 --2- 172.21.15.71:0/3837858885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba806a710 0x7fbba80ca420 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 --2- 172.21.15.71:0/3837858885 >> v2:172.21.15.86:3300/0 conn(0x7fbba8157620 0x7fbba806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/3837858885 >> 172.21.15.71:0/3837858885 conn(0x7fbba80aec60 msgr2=0x7fbba80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/3837858885 shutdown_connections 2023-10-26T05:54:19.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/3837858885 wait complete. 2023-10-26T05:54:19.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 Processor -- start 2023-10-26T05:54:19.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- start start 2023-10-26T05:54:19.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbba806a710 0x7fbba814d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba80affd0 0x7fbba8153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 0x7fbba80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba814d5e0 con 0x7fbba8157620 2023-10-26T05:54:19.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8154d80 con 0x7fbba80affd0 2023-10-26T05:54:19.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba81f7bd0 con 0x7fbba806a710 2023-10-26T05:54:19.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 0x7fbba80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 0x7fbba80cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38776/0 (socket says 172.21.15.71:38776) 2023-10-26T05:54:19.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 -- 172.21.15.71:0/156079132 learned_addr learned my addr 172.21.15.71:0/156079132 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:19.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbad618640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba80affd0 0x7fbba8153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbade19640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbba806a710 0x7fbba814d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 -- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbba806a710 msgr2=0x7fbba814d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbba806a710 0x7fbba814d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 -- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba80affd0 msgr2=0x7fbba8153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba80affd0 0x7fbba8153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 -- 172.21.15.71:0/156079132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba00098e0 con 0x7fbba8157620 2023-10-26T05:54:19.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbade19640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbba806a710 0x7fbba814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:19.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbad618640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba80affd0 0x7fbba8153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:19.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbae61a640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 0x7fbba80cced0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbba001c710 tx=0x7fbba0009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:19.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0003d90 con 0x7fbba8157620 2023-10-26T05:54:19.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbba00a5cb0 con 0x7fbba8157620 2023-10-26T05:54:19.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba81f7eb0 con 0x7fbba8157620 2023-10-26T05:54:19.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004290 con 0x7fbba8157620 2023-10-26T05:54:19.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba81f8470 con 0x7fbba8157620 2023-10-26T05:54:19.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.199+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba8066b10 con 0x7fbba8157620 2023-10-26T05:54:19.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.203+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbba000a040 con 0x7fbba8157620 2023-10-26T05:54:19.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.203+0000 7fbb9e7fc640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb8c0664a0 0x7fbb8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.203+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbba011f0d0 con 0x7fbba8157620 2023-10-26T05:54:19.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.203+0000 7fbbade19640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb8c0664a0 0x7fbb8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.203+0000 7fbbade19640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb8c0664a0 0x7fbb8c068960 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005fd0 tx=0x7fbba4005dc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:19.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.207+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbba00ec480 con 0x7fbba8157620 2023-10-26T05:54:19.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbba815ac90 con 0x7fbba8157620 2023-10-26T05:54:19.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbb9e7fc640 1 -- 172.21.15.71:0/156079132 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbba00b20a0 con 0x7fbba8157620 2023-10-26T05:54:19.371 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:54:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb8c0664a0 msgr2=0x7fbb8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb8c0664a0 0x7fbb8c068960 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbba4005fd0 tx=0x7fbba4005dc0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 msgr2=0x7fbba80cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 0x7fbba80cced0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbba001c710 tx=0x7fbba0009340 comp rx=0 tx=0).stop 2023-10-26T05:54:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 shutdown_connections 2023-10-26T05:54:19.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb8c0664a0 0x7fbb8c068960 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbba8157620 0x7fbba80cced0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbba80affd0 0x7fbba8153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 --2- 172.21.15.71:0/156079132 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbba806a710 0x7fbba814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 >> 172.21.15.71:0/156079132 conn(0x7fbba80aec60 msgr2=0x7fbba806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 shutdown_connections 2023-10-26T05:54:19.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.363+0000 7fbbb007d640 1 -- 172.21.15.71:0/156079132 wait complete. 2023-10-26T05:54:19.391 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:54:19.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- 172.21.15.71:0/2630697161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 msgr2=0x7f49800cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 --2- 172.21.15.71:0/2630697161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 0x7f49800cb8c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4970009840 tx=0x7f497001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- 172.21.15.71:0/2630697161 shutdown_connections 2023-10-26T05:54:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 --2- 172.21.15.71:0/2630697161 >> v2:172.21.15.86:3300/0 conn(0x7f498013ef60 0x7f4980069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 --2- 172.21.15.71:0/2630697161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 0x7f49800cb8c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- 172.21.15.71:0/2630697161 >> 172.21.15.71:0/2630697161 conn(0x7f49800aec60 msgr2=0x7f49800af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- 172.21.15.71:0/2630697161 shutdown_connections 2023-10-26T05:54:19.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- 172.21.15.71:0/2630697161 wait complete. 2023-10-26T05:54:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 Processor -- start 2023-10-26T05:54:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- start start 2023-10-26T05:54:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49800cbe00 0x7f498014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 0x7f498014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 0x7f49800b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4980071520 con 0x7f498013ef60 2023-10-26T05:54:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800aba70 con 0x7f49800cbe00 2023-10-26T05:54:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49863d2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498014f730 con 0x7f498013c120 2023-10-26T05:54:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f498496f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 0x7f49800b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f498496f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 0x7f49800b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38792/0 (socket says 172.21.15.71:38792) 2023-10-26T05:54:19.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f498496f640 1 -- 172.21.15.71:0/720920663 learned_addr learned my addr 172.21.15.71:0/720920663 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f497ffff640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49800cbe00 0x7f498014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.511+0000 7f49777fe640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 0x7f498014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f498496f640 1 -- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 msgr2=0x7f498014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f498496f640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 0x7f498014f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f498496f640 1 -- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49800cbe00 msgr2=0x7f498014f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f498496f640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49800cbe00 0x7f498014f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f498496f640 1 -- 172.21.15.71:0/720920663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700094a0 con 0x7f498013ef60 2023-10-26T05:54:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f498496f640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 0x7f49800b1710 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f497800eae0 tx=0x7f4978009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800ce40 con 0x7f498013ef60 2023-10-26T05:54:19.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4978004c20 con 0x7f498013ef60 2023-10-26T05:54:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4980071240 con 0x7f498013ef60 2023-10-26T05:54:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978005020 con 0x7f498013ef60 2023-10-26T05:54:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49800ab700 con 0x7f498013ef60 2023-10-26T05:54:19.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4980066b10 con 0x7f498013ef60 2023-10-26T05:54:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4978005180 con 0x7f498013ef60 2023-10-26T05:54:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497d7fa640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4960066470 0x7f4960068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4978014070 con 0x7f498013ef60 2023-10-26T05:54:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497ffff640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4960066470 0x7f4960068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.515+0000 7f497ffff640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4960066470 0x7f4960068930 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f49700af000 tx=0x7f497001b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:19.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.519+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49780520c0 con 0x7f498013ef60 2023-10-26T05:54:19.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.671+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49800721e0 con 0x7f498013ef60 2023-10-26T05:54:19.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f497d7fa640 1 -- 172.21.15.71:0/720920663 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f497801d020 con 0x7f498013ef60 2023-10-26T05:54:19.682 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:54:19.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4960066470 msgr2=0x7f4960068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4960066470 0x7f4960068930 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f49700af000 tx=0x7f497001b010 comp rx=0 tx=0).stop 2023-10-26T05:54:19.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 msgr2=0x7f49800b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 0x7f49800b1710 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f497800eae0 tx=0x7f4978009f70 comp rx=0 tx=0).stop 2023-10-26T05:54:19.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 shutdown_connections 2023-10-26T05:54:19.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4960066470 0x7f4960068930 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f498013ef60 0x7f49800b1710 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f498013c120 0x7f498014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 --2- 172.21.15.71:0/720920663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49800cbe00 0x7f498014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 >> 172.21.15.71:0/720920663 conn(0x7f49800aec60 msgr2=0x7f498015b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 shutdown_connections 2023-10-26T05:54:19.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.675+0000 7f49863d2640 1 -- 172.21.15.71:0/720920663 wait complete. 2023-10-26T05:54:19.700 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:54:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/964687936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 msgr2=0x7fd508003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 --2- 172.21.15.71:0/964687936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd508003dd0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd510001680 tx=0x7fd51001c760 comp rx=0 tx=0).stop 2023-10-26T05:54:19.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd524be4640 1 -- 172.21.15.71:0/964687936 reap_dead start 2023-10-26T05:54:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/964687936 shutdown_connections 2023-10-26T05:54:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/964687936 >> 172.21.15.71:0/964687936 conn(0x7fd5200aec60 msgr2=0x7fd5200af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/964687936 shutdown_connections 2023-10-26T05:54:19.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/964687936 wait complete. 2023-10-26T05:54:19.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 Processor -- start 2023-10-26T05:54:19.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- start start 2023-10-26T05:54:19.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd5200ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd520140b60 0x7fd520158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200ad370 0x7fd520139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd520140140 con 0x7fd5080039b0 2023-10-26T05:54:19.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52013f930 con 0x7fd5200ad370 2023-10-26T05:54:19.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52015a310 con 0x7fd520140b60 2023-10-26T05:54:19.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd5200ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd5200ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38816/0 (socket says 172.21.15.71:38816) 2023-10-26T05:54:19.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 -- 172.21.15.71:0/4076984171 learned_addr learned my addr 172.21.15.71:0/4076984171 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:19.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd524be4640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200ad370 0x7fd520139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 -- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd520140b60 msgr2=0x7fd520158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51f7fe640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd520140b60 0x7fd520158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd520140b60 0x7fd520158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 -- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200ad370 msgr2=0x7fd520139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200ad370 0x7fd520139450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 -- 172.21.15.71:0/4076984171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5200acbc0 con 0x7fd5080039b0 2023-10-26T05:54:19.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51ffff640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd5200ac680 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd51001c590 tx=0x7fd510003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:19.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100a9040 con 0x7fd5080039b0 2023-10-26T05:54:19.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd510003e50 con 0x7fd5080039b0 2023-10-26T05:54:19.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52013fb80 con 0x7fd5080039b0 2023-10-26T05:54:19.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100a57a0 con 0x7fd5080039b0 2023-10-26T05:54:19.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.823+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52015a5b0 con 0x7fd5080039b0 2023-10-26T05:54:19.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.827+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd520066b10 con 0x7fd5080039b0 2023-10-26T05:54:19.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.827+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd5100b5020 con 0x7fd5080039b0 2023-10-26T05:54:19.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.827+0000 7fd51cff9640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd50c066470 0x7fd50c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:19.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.827+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd5100aa070 con 0x7fd5080039b0 2023-10-26T05:54:19.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.827+0000 7fd51f7fe640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd50c066470 0x7fd50c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:19.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.827+0000 7fd51f7fe640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd50c066470 0x7fd50c068930 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd520158310 tx=0x7fd514001120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:19.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.831+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd5100eb610 con 0x7fd5080039b0 2023-10-26T05:54:19.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.983+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5200721e0 con 0x7fd5080039b0 2023-10-26T05:54:19.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.983+0000 7fd51cff9640 1 -- 172.21.15.71:0/4076984171 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5100bd020 con 0x7fd5080039b0 2023-10-26T05:54:19.992 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:54:19.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd50c066470 msgr2=0x7fd50c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd50c066470 0x7fd50c068930 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd520158310 tx=0x7fd514001120 comp rx=0 tx=0).stop 2023-10-26T05:54:19.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 msgr2=0x7fd5200ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:19.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd5200ac680 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd51001c590 tx=0x7fd510003c60 comp rx=0 tx=0).stop 2023-10-26T05:54:19.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 shutdown_connections 2023-10-26T05:54:19.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5200ad370 0x7fd520139450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd520140b60 0x7fd520158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd50c066470 0x7fd50c068930 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 --2- 172.21.15.71:0/4076984171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5080039b0 0x7fd5200ac680 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 >> 172.21.15.71:0/4076984171 conn(0x7fd5200aec60 msgr2=0x7fd5200cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:19.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 shutdown_connections 2023-10-26T05:54:19.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:19.987+0000 7fd526647640 1 -- 172.21.15.71:0/4076984171 wait complete. 2023-10-26T05:54:20.011 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:54:20.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/4258675459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf04003980 msgr2=0x7fcf04003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 --2- 172.21.15.71:0/4258675459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf04003980 0x7fcf04003da0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400c260 tx=0x7fcf1401cf50 comp rx=0 tx=0).stop 2023-10-26T05:54:20.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 -- 172.21.15.71:0/4258675459 reap_dead start 2023-10-26T05:54:20.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/4258675459 shutdown_connections 2023-10-26T05:54:20.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/4258675459 >> 172.21.15.71:0/4258675459 conn(0x7fcf180b2d90 msgr2=0x7fcf180b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:20.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/4258675459 shutdown_connections 2023-10-26T05:54:20.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/4258675459 wait complete. 2023-10-26T05:54:20.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 Processor -- start 2023-10-26T05:54:20.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- start start 2023-10-26T05:54:20.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf04003980 0x7fcf181f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf181f7e60 0x7fcf18160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 0x7fcf180cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181f6880 con 0x7fcf18160f90 2023-10-26T05:54:20.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181f6b60 con 0x7fcf181f7e60 2023-10-26T05:54:20.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181f6e40 con 0x7fcf04003980 2023-10-26T05:54:20.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 0x7fcf180cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 0x7fcf180cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38836/0 (socket says 172.21.15.71:38836) 2023-10-26T05:54:20.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 -- 172.21.15.71:0/2790343040 learned_addr learned my addr 172.21.15.71:0/2790343040 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:20.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1df19640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf04003980 0x7fcf181f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1d718640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf181f7e60 0x7fcf18160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 -- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf04003980 msgr2=0x7fcf181f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf04003980 0x7fcf181f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 -- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf181f7e60 msgr2=0x7fcf18160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf181f7e60 0x7fcf18160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 -- 172.21.15.71:0/2790343040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf181f7120 con 0x7fcf18160f90 2023-10-26T05:54:20.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf1e71a640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 0x7fcf180cc900 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fcf180896c0 tx=0x7fcf1000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:20.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10004b30 con 0x7fcf18160f90 2023-10-26T05:54:20.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf10004d80 con 0x7fcf18160f90 2023-10-26T05:54:20.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf181f7400 con 0x7fcf18160f90 2023-10-26T05:54:20.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000bc80 con 0x7fcf18160f90 2023-10-26T05:54:20.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.131+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf180cd010 con 0x7fcf18160f90 2023-10-26T05:54:20.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.135+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18066b10 con 0x7fcf18160f90 2023-10-26T05:54:20.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.135+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf10010460 con 0x7fcf18160f90 2023-10-26T05:54:20.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.135+0000 7fcf0e7fc640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcefc066470 0x7fcefc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.135+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcf100837e0 con 0x7fcf18160f90 2023-10-26T05:54:20.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.135+0000 7fcf1df19640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcefc066470 0x7fcefc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.135+0000 7fcf1df19640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcefc066470 0x7fcefc068930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcf140ac000 tx=0x7fcf14019d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:20.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.139+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf10050cd0 con 0x7fcf18160f90 2023-10-26T05:54:20.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.311+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcf1804da80 con 0x7fcf18160f90 2023-10-26T05:54:20.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.311+0000 7fcf0e7fc640 1 -- 172.21.15.71:0/2790343040 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v143) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcf100542f0 con 0x7fcf18160f90 2023-10-26T05:54:20.320 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:54:20.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcefc066470 msgr2=0x7fcefc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcefc066470 0x7fcefc068930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcf140ac000 tx=0x7fcf14019d50 comp rx=0 tx=0).stop 2023-10-26T05:54:20.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 msgr2=0x7fcf180cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 0x7fcf180cc900 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fcf180896c0 tx=0x7fcf1000dd10 comp rx=0 tx=0).stop 2023-10-26T05:54:20.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 shutdown_connections 2023-10-26T05:54:20.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcefc066470 0x7fcefc068930 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf18160f90 0x7fcf180cc900 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf181f7e60 0x7fcf18160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 --2- 172.21.15.71:0/2790343040 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf04003980 0x7fcf181f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 >> 172.21.15.71:0/2790343040 conn(0x7fcf180b2d90 msgr2=0x7fcf1815e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 shutdown_connections 2023-10-26T05:54:20.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.315+0000 7fcf2017d640 1 -- 172.21.15.71:0/2790343040 wait complete. 2023-10-26T05:54:20.340 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:54:20.341 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:20.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:20.397 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:20.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35050]: fuse finished with error 0 and tester_r 0 2023-10-26T05:54:20.399 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:20.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:54:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:20.407 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:54:20.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:20.408 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:54:20.415 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:54:20.416 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:20.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:54:20.422 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:54:20.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:20.423 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:54:20.469 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:54:20.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:54:20.471 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:54:20.471 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:20.478 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:54:20.478 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:54:20.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:20.479 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:54:20.525 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:54:20.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:54:20.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:20.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- 172.21.15.71:0/3791473207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0d0850 msgr2=0x7f793c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3791473207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0d0850 0x7f793c15b3a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7924009900 tx=0x7f792401c770 comp rx=0 tx=0).stop 2023-10-26T05:54:20.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- 172.21.15.71:0/3791473207 shutdown_connections 2023-10-26T05:54:20.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3791473207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0d0850 0x7f793c15b3a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3791473207 >> v2:172.21.15.86:3300/0 conn(0x7f793c0d0470 0x7f793c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- 172.21.15.71:0/3791473207 >> 172.21.15.71:0/3791473207 conn(0x7f793c0cdd80 msgr2=0x7f793c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:20.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- 172.21.15.71:0/3791473207 shutdown_connections 2023-10-26T05:54:20.664 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- 172.21.15.71:0/3791473207 wait complete. 2023-10-26T05:54:20.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 Processor -- start 2023-10-26T05:54:20.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- start start 2023-10-26T05:54:20.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0cd880 0x7f793c14a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c0d0470 0x7f793c154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 0x7f793c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c154ab0 con 0x7f793c0d0850 2023-10-26T05:54:20.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c1541a0 con 0x7f793c0d0470 2023-10-26T05:54:20.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7942ac8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c157460 con 0x7f793c0cd880 2023-10-26T05:54:20.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7941065640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 0x7f793c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7941065640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 0x7f793c150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38846/0 (socket says 172.21.15.71:38846) 2023-10-26T05:54:20.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7941065640 1 -- 172.21.15.71:0/3988762695 learned_addr learned my addr 172.21.15.71:0/3988762695 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:20.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f793bfff640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c0d0470 0x7f793c154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.655+0000 7f7940864640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0cd880 0x7f793c14a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7941065640 1 -- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0cd880 msgr2=0x7f793c14a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7941065640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0cd880 0x7f793c14a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7941065640 1 -- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c0d0470 msgr2=0x7f793c154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7941065640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c0d0470 0x7f793c154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7941065640 1 -- 172.21.15.71:0/3988762695 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924009560 con 0x7f793c0d0850 2023-10-26T05:54:20.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7941065640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 0x7f793c150dc0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f792c00c430 tx=0x7f792c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:20.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c005220 con 0x7f793c0d0850 2023-10-26T05:54:20.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f792c005470 con 0x7f793c0d0850 2023-10-26T05:54:20.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c007290 con 0x7f793c0d0850 2023-10-26T05:54:20.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c1543f0 con 0x7f793c0d0850 2023-10-26T05:54:20.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c151300 con 0x7f793c0d0850 2023-10-26T05:54:20.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c066b10 con 0x7f793c0d0850 2023-10-26T05:54:20.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f792c0073f0 con 0x7f793c0d0850 2023-10-26T05:54:20.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f79397fa640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79200664a0 0x7f7920068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:20.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f792c0834c0 con 0x7f793c0d0850 2023-10-26T05:54:20.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.659+0000 7f7940864640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79200664a0 0x7f7920068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:20.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.663+0000 7f7940864640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79200664a0 0x7f7920068960 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7934009f60 tx=0x7f7934009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:20.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.663+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f792c0509c0 con 0x7f793c0d0850 2023-10-26T05:54:20.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.847+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f793c154bf0 con 0x7f793c0d0850 2023-10-26T05:54:20.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.847+0000 7f79397fa640 1 -- 172.21.15.71:0/3988762695 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f793c154bf0 con 0x7f793c0d0850 2023-10-26T05:54:20.856 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:20.857 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:54:20.857 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:54:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79200664a0 msgr2=0x7f7920068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79200664a0 0x7f7920068960 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7934009f60 tx=0x7f7934009500 comp rx=0 tx=0).stop 2023-10-26T05:54:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 msgr2=0x7f793c150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 0x7f793c150dc0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f792c00c430 tx=0x7f792c00c8f0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 shutdown_connections 2023-10-26T05:54:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79200664a0 0x7f7920068960 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f793c0d0850 0x7f793c150dc0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c0d0470 0x7f793c154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 --2- 172.21.15.71:0/3988762695 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f793c0cd880 0x7f793c14a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 >> 172.21.15.71:0/3988762695 conn(0x7f793c0cdd80 msgr2=0x7f793c0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:20.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 shutdown_connections 2023-10-26T05:54:20.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.851+0000 7f7942ac8640 1 -- 172.21.15.71:0/3988762695 wait complete. 2023-10-26T05:54:20.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:21.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3593820968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e20158160 msgr2=0x7f8e200cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 --2- 172.21.15.71:0/3593820968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e20158160 0x7f8e200cc2c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009bc0 tx=0x7f8e1001cb80 comp rx=0 tx=0).stop 2023-10-26T05:54:21.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3593820968 shutdown_connections 2023-10-26T05:54:21.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 --2- 172.21.15.71:0/3593820968 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e200b08e0 0x7f8e2006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 --2- 172.21.15.71:0/3593820968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e20158160 0x7f8e200cc2c0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3593820968 >> 172.21.15.71:0/3593820968 conn(0x7f8e20070d80 msgr2=0x7f8e2006c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3593820968 shutdown_connections 2023-10-26T05:54:21.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3593820968 wait complete. 2023-10-26T05:54:21.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 Processor -- start 2023-10-26T05:54:21.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- start start 2023-10-26T05:54:21.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 0x7f8e201390c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e200b08e0 0x7f8e20146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e20158160 0x7f8e201545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20139600 con 0x7f8e2006b650 2023-10-26T05:54:21.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e201520d0 con 0x7f8e200b08e0 2023-10-26T05:54:21.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e201475f0 con 0x7f8e20158160 2023-10-26T05:54:21.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 0x7f8e201390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 0x7f8e201390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38862/0 (socket says 172.21.15.71:38862) 2023-10-26T05:54:21.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 -- 172.21.15.71:0/3880022134 learned_addr learned my addr 172.21.15.71:0/3880022134 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:21.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1e7fc640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e200b08e0 0x7f8e20146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 -- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e20158160 msgr2=0x7f8e201545d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1f7fe640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e20158160 0x7f8e201545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e20158160 0x7f8e201545d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 -- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e200b08e0 msgr2=0x7f8e20146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e200b08e0 0x7f8e20146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 -- 172.21.15.71:0/3880022134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009820 con 0x7f8e2006b650 2023-10-26T05:54:21.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1f7fe640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e20158160 0x7f8e201545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:21.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e1effd640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 0x7f8e201390c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8e1001c5f0 tx=0x7f8e100a5a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100bc070 con 0x7f8e2006b650 2023-10-26T05:54:21.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8e10005040 con 0x7f8e2006b650 2023-10-26T05:54:21.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e20147310 con 0x7f8e2006b650 2023-10-26T05:54:21.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10005430 con 0x7f8e2006b650 2023-10-26T05:54:21.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.995+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e20152650 con 0x7f8e2006b650 2023-10-26T05:54:21.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.999+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e20066b10 con 0x7f8e2006b650 2023-10-26T05:54:21.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.999+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8e100b4040 con 0x7f8e2006b650 2023-10-26T05:54:21.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.999+0000 7f8e177fe640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e00066470 0x7f8e00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.999+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8e1011e5b0 con 0x7f8e2006b650 2023-10-26T05:54:21.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.999+0000 7f8e1e7fc640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e00066470 0x7f8e00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:20.999+0000 7f8e1e7fc640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e00066470 0x7f8e00068930 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e200cc720 tx=0x7f8e08005e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.003+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8e100ebb10 con 0x7f8e2006b650 2023-10-26T05:54:21.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.151+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e20151f30 con 0x7f8e2006b650 2023-10-26T05:54:21.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e177fe640 1 -- 172.21.15.71:0/3880022134 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16544 (secure 0 0 0) 0x7f8e20151f30 con 0x7f8e2006b650 2023-10-26T05:54:21.162 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:21.162 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":143,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:12.178962+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:21.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e00066470 msgr2=0x7f8e00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e00066470 0x7f8e00068930 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e200cc720 tx=0x7f8e08005e30 comp rx=0 tx=0).stop 2023-10-26T05:54:21.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 msgr2=0x7f8e201390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 0x7f8e201390c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f8e1001c5f0 tx=0x7f8e100a5a70 comp rx=0 tx=0).stop 2023-10-26T05:54:21.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 shutdown_connections 2023-10-26T05:54:21.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e20158160 0x7f8e201545d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e200b08e0 0x7f8e20146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e00066470 0x7f8e00068930 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 --2- 172.21.15.71:0/3880022134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2006b650 0x7f8e201390c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 >> 172.21.15.71:0/3880022134 conn(0x7f8e20070d80 msgr2=0x7f8e20162610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 shutdown_connections 2023-10-26T05:54:21.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.155+0000 7f8e25462640 1 -- 172.21.15.71:0/3880022134 wait complete. 2023-10-26T05:54:21.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:53:37.960720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:53:38.704883+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:54:21.180 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:54:21.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2569442086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 msgr2=0x7fb6e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2569442086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6e0003dd0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000c260 tx=0x7fb6f001cf90 comp rx=0 tx=0).stop 2023-10-26T05:54:21.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6f987d640 1 -- 172.21.15.71:0/2569442086 reap_dead start 2023-10-26T05:54:21.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2569442086 shutdown_connections 2023-10-26T05:54:21.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2569442086 >> 172.21.15.71:0/2569442086 conn(0x7fb6f40aec60 msgr2=0x7fb6f40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2569442086 shutdown_connections 2023-10-26T05:54:21.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2569442086 wait complete. 2023-10-26T05:54:21.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 Processor -- start 2023-10-26T05:54:21.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- start start 2023-10-26T05:54:21.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6f40b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f40b2c90 0x7fb6f406f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f4135690 0x7fb6f406fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4136b40 con 0x7fb6e00039b0 2023-10-26T05:54:21.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40b1c70 con 0x7fb6f40b2c90 2023-10-26T05:54:21.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6fb2e0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40b1f50 con 0x7fb6f4135690 2023-10-26T05:54:21.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6f907c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6f40b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6f907c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6f40b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38886/0 (socket says 172.21.15.71:38886) 2023-10-26T05:54:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.299+0000 7fb6f907c640 1 -- 172.21.15.71:0/2118905598 learned_addr learned my addr 172.21.15.71:0/2118905598 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f887b640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f40b2c90 0x7fb6f406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f987d640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f4135690 0x7fb6f406fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f907c640 1 -- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f4135690 msgr2=0x7fb6f406fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f907c640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f4135690 0x7fb6f406fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f907c640 1 -- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f40b2c90 msgr2=0x7fb6f406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f907c640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f40b2c90 0x7fb6f406f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f907c640 1 -- 172.21.15.71:0/2118905598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40b2230 con 0x7fb6e00039b0 2023-10-26T05:54:21.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f887b640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f40b2c90 0x7fb6f406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:54:21.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f907c640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6f40b2750 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0001940 tx=0x7fb6f0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.314 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00b3070 con 0x7fb6e00039b0 2023-10-26T05:54:21.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb6f0006320 con 0x7fb6e00039b0 2023-10-26T05:54:21.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f4136e20 con 0x7fb6e00039b0 2023-10-26T05:54:21.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0003600 con 0x7fb6e00039b0 2023-10-26T05:54:21.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f41387c0 con 0x7fb6e00039b0 2023-10-26T05:54:21.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f4066b10 con 0x7fb6e00039b0 2023-10-26T05:54:21.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb6f00ab020 con 0x7fb6e00039b0 2023-10-26T05:54:21.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6e9ffb640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb6d80664a0 0x7fb6d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb6f011d9e0 con 0x7fb6e00039b0 2023-10-26T05:54:21.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f887b640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb6d80664a0 0x7fb6d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.303+0000 7fb6f887b640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb6d80664a0 0x7fb6d8068960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005b70 tx=0x7fb6e4005ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.317 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.307+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb6f00eaeb0 con 0x7fb6e00039b0 2023-10-26T05:54:21.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.459+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6f40b1ad0 con 0x7fb6e00039b0 2023-10-26T05:54:21.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.459+0000 7fb6e9ffb640 1 -- 172.21.15.71:0/2118905598 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6f40b1ad0 con 0x7fb6e00039b0 2023-10-26T05:54:21.469 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:54:21.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb6d80664a0 msgr2=0x7fb6d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb6d80664a0 0x7fb6d8068960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005b70 tx=0x7fb6e4005ac0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 msgr2=0x7fb6f40b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6f40b2750 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0001940 tx=0x7fb6f0000ee0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 shutdown_connections 2023-10-26T05:54:21.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb6f4135690 0x7fb6f406fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6f40b2c90 0x7fb6f406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb6d80664a0 0x7fb6d8068960 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 --2- 172.21.15.71:0/2118905598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6e00039b0 0x7fb6f40b2750 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 >> 172.21.15.71:0/2118905598 conn(0x7fb6f40aec60 msgr2=0x7fb6f40ae450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 shutdown_connections 2023-10-26T05:54:21.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.463+0000 7fb6fb2e0640 1 -- 172.21.15.71:0/2118905598 wait complete. 2023-10-26T05:54:21.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:54:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- 172.21.15.71:0/3358897073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99013c220 msgr2=0x7fe99015ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 --2- 172.21.15.71:0/3358897073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99013c220 0x7fe99015ccc0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009900 tx=0x7fe98c01c850 comp rx=0 tx=0).stop 2023-10-26T05:54:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- 172.21.15.71:0/3358897073 shutdown_connections 2023-10-26T05:54:21.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 --2- 172.21.15.71:0/3358897073 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99015d200 0x7fe9900cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 --2- 172.21.15.71:0/3358897073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99013c220 0x7fe99015ccc0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- 172.21.15.71:0/3358897073 >> 172.21.15.71:0/3358897073 conn(0x7fe9900aec60 msgr2=0x7fe9900af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- 172.21.15.71:0/3358897073 shutdown_connections 2023-10-26T05:54:21.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- 172.21.15.71:0/3358897073 wait complete. 2023-10-26T05:54:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 Processor -- start 2023-10-26T05:54:21.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- start start 2023-10-26T05:54:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 0x7fe99013c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99013c220 0x7fe99013cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99015d200 0x7fe990137420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990137f60 con 0x7fe9900cbcd0 2023-10-26T05:54:21.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99013cb60 con 0x7fe99013c220 2023-10-26T05:54:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe997af7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900ab990 con 0x7fe99015d200 2023-10-26T05:54:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 0x7fe99013c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 0x7fe99013c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38914/0 (socket says 172.21.15.71:38914) 2023-10-26T05:54:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 -- 172.21.15.71:0/4167888756 learned_addr learned my addr 172.21.15.71:0/4167888756 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:21.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995092640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99013c220 0x7fe99013cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe996094640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99015d200 0x7fe990137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 -- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99015d200 msgr2=0x7fe990137420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99015d200 0x7fe990137420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.610 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 -- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99013c220 msgr2=0x7fe99013cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99013c220 0x7fe99013cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 -- 172.21.15.71:0/4167888756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009560 con 0x7fe9900cbcd0 2023-10-26T05:54:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995092640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99013c220 0x7fe99013cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:54:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.595+0000 7fe995893640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 0x7fe99013c620 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009850 tx=0x7fe98c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0044b0 con 0x7fe9900cbcd0 2023-10-26T05:54:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe98c0036f0 con 0x7fe9900cbcd0 2023-10-26T05:54:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c003b00 con 0x7fe9900cbcd0 2023-10-26T05:54:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99013d2d0 con 0x7fe9900cbcd0 2023-10-26T05:54:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9900abc30 con 0x7fe9900cbcd0 2023-10-26T05:54:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990066b10 con 0x7fe9900cbcd0 2023-10-26T05:54:21.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe98c0b4040 con 0x7fe9900cbcd0 2023-10-26T05:54:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe9867fc640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe974066470 0x7fe974068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe98c11e790 con 0x7fe9900cbcd0 2023-10-26T05:54:21.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe995092640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe974066470 0x7fe974068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.599+0000 7fe995092640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe974066470 0x7fe974068930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe990138100 tx=0x7fe980009570 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.603+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe98c0ebbe0 con 0x7fe9900cbcd0 2023-10-26T05:54:21.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.743+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe990066b10 con 0x7fe9900cbcd0 2023-10-26T05:54:21.754 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe9867fc640 1 -- 172.21.15.71:0/4167888756 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe98c0ef200 con 0x7fe9900cbcd0 2023-10-26T05:54:21.754 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:54:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe974066470 msgr2=0x7fe974068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe974066470 0x7fe974068930 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe990138100 tx=0x7fe980009570 comp rx=0 tx=0).stop 2023-10-26T05:54:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 msgr2=0x7fe99013c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 0x7fe99013c620 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009850 tx=0x7fe98c0040c0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 shutdown_connections 2023-10-26T05:54:21.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe99015d200 0x7fe990137420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe99013c220 0x7fe99013cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe974066470 0x7fe974068930 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 --2- 172.21.15.71:0/4167888756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9900cbcd0 0x7fe99013c620 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 >> 172.21.15.71:0/4167888756 conn(0x7fe9900aec60 msgr2=0x7fe990069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 shutdown_connections 2023-10-26T05:54:21.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.747+0000 7fe997af7640 1 -- 172.21.15.71:0/4167888756 wait complete. 2023-10-26T05:54:21.771 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:54:21.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/4233112218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3780047d0 msgr2=0x7fe378004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 --2- 172.21.15.71:0/4233112218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3780047d0 0x7fe378004c50 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe3840016b0 tx=0x7fe38401c770 comp rx=0 tx=0).stop 2023-10-26T05:54:21.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38f7fe640 1 -- 172.21.15.71:0/4233112218 reap_dead start 2023-10-26T05:54:21.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/4233112218 shutdown_connections 2023-10-26T05:54:21.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/4233112218 >> 172.21.15.71:0/4233112218 conn(0x7fe390070d10 msgr2=0x7fe39006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:21.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/4233112218 shutdown_connections 2023-10-26T05:54:21.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/4233112218 wait complete. 2023-10-26T05:54:21.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 Processor -- start 2023-10-26T05:54:21.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- start start 2023-10-26T05:54:21.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3780047d0 0x7fe390139c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 0x7fe390159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3900ad370 0x7fe3900ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39015a350 con 0x7fe390159990 2023-10-26T05:54:21.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900ae2a0 con 0x7fe3780047d0 2023-10-26T05:54:21.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3900add60 con 0x7fe3900ad370 2023-10-26T05:54:21.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 0x7fe390159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 0x7fe390159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44328/0 (socket says 172.21.15.71:44328) 2023-10-26T05:54:21.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 -- 172.21.15.71:0/697457939 learned_addr learned my addr 172.21.15.71:0/697457939 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:21.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38effd640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3780047d0 0x7fe390139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38f7fe640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3900ad370 0x7fe3900ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 -- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3900ad370 msgr2=0x7fe3900ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3900ad370 0x7fe3900ad820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 -- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3780047d0 msgr2=0x7fe390139c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:21.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3780047d0 0x7fe390139c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:21.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 -- 172.21.15.71:0/697457939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3901577e0 con 0x7fe390159990 2023-10-26T05:54:21.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe38e7fc640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 0x7fe390159e10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe3840016b0 tx=0x7fe384003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3840a9040 con 0x7fe390159990 2023-10-26T05:54:21.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe384003e30 con 0x7fe390159990 2023-10-26T05:54:21.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3900adfc0 con 0x7fe390159990 2023-10-26T05:54:21.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3840a57b0 con 0x7fe390159990 2023-10-26T05:54:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.887+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe390160900 con 0x7fe390159990 2023-10-26T05:54:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.891+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe390066b10 con 0x7fe390159990 2023-10-26T05:54:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.891+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe3840b5020 con 0x7fe390159990 2023-10-26T05:54:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.891+0000 7fe377fff640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe37c0664a0 0x7fe37c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:21.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.891+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe3840aa070 con 0x7fe390159990 2023-10-26T05:54:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.891+0000 7fe38effd640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe37c0664a0 0x7fe37c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.891+0000 7fe38effd640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe37c0664a0 0x7fe37c068960 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe39006dd80 tx=0x7fe38000d550 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:21.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:21.895+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe3840eb1c0 con 0x7fe390159990 2023-10-26T05:54:22.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3900721e0 con 0x7fe390159990 2023-10-26T05:54:22.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe377fff640 1 -- 172.21.15.71:0/697457939 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3840bd020 con 0x7fe390159990 2023-10-26T05:54:22.059 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:54:22.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe37c0664a0 msgr2=0x7fe37c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe37c0664a0 0x7fe37c068960 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe39006dd80 tx=0x7fe38000d550 comp rx=0 tx=0).stop 2023-10-26T05:54:22.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 msgr2=0x7fe390159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 0x7fe390159e10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe3840016b0 tx=0x7fe384003c40 comp rx=0 tx=0).stop 2023-10-26T05:54:22.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 shutdown_connections 2023-10-26T05:54:22.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3900ad370 0x7fe3900ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe37c0664a0 0x7fe37c068960 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390159990 0x7fe390159e10 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 --2- 172.21.15.71:0/697457939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3780047d0 0x7fe390139c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 >> 172.21.15.71:0/697457939 conn(0x7fe390070d10 msgr2=0x7fe39013c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:22.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 shutdown_connections 2023-10-26T05:54:22.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.051+0000 7fe395a59640 1 -- 172.21.15.71:0/697457939 wait complete. 2023-10-26T05:54:22.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:22.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 -- 172.21.15.71:0/3436690260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e00b16a0 msgr2=0x7f65e0158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 --2- 172.21.15.71:0/3436690260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e00b16a0 0x7f65e0158ce0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009900 tx=0x7f65d401c890 comp rx=0 tx=0).stop 2023-10-26T05:54:22.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 -- 172.21.15.71:0/3436690260 shutdown_connections 2023-10-26T05:54:22.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 --2- 172.21.15.71:0/3436690260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e00b16a0 0x7f65e0158ce0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 --2- 172.21.15.71:0/3436690260 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65e006b820 0x7f65e01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 -- 172.21.15.71:0/3436690260 >> 172.21.15.71:0/3436690260 conn(0x7f65e00af5e0 msgr2=0x7f65e00af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:22.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 -- 172.21.15.71:0/3436690260 shutdown_connections 2023-10-26T05:54:22.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.191+0000 7f65e7125640 1 -- 172.21.15.71:0/3436690260 wait complete. 2023-10-26T05:54:22.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 Processor -- start 2023-10-26T05:54:22.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 -- start start 2023-10-26T05:54:22.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65e006b820 0x7f65e014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65e00b16a0 0x7f65e0146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 0x7f65e014dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0147b00 con 0x7f65e0158030 2023-10-26T05:54:22.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0147630 con 0x7f65e00b16a0 2023-10-26T05:54:22.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e7125640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e014f2d0 con 0x7f65e006b820 2023-10-26T05:54:22.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 0x7f65e014dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 0x7f65e014dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44346/0 (socket says 172.21.15.71:44346) 2023-10-26T05:54:22.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 -- 172.21.15.71:0/135737845 learned_addr learned my addr 172.21.15.71:0/135737845 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:22.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 -- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65e006b820 msgr2=0x7f65e014f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65e006b820 0x7f65e014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 -- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65e00b16a0 msgr2=0x7f65e0146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65dffff640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65e00b16a0 0x7f65e0146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65e00b16a0 0x7f65e0146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 -- 172.21.15.71:0/135737845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009560 con 0x7f65e0158030 2023-10-26T05:54:22.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.195+0000 7f65e56c2640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 0x7f65e014dc50 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009850 tx=0x7f65d4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:22.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4003800 con 0x7f65e0158030 2023-10-26T05:54:22.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f65d4003a50 con 0x7f65e0158030 2023-10-26T05:54:22.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e014e9a0 con 0x7f65e0158030 2023-10-26T05:54:22.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4003e20 con 0x7f65e0158030 2023-10-26T05:54:22.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e014ef60 con 0x7f65e0158030 2023-10-26T05:54:22.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e0066d00 con 0x7f65e0158030 2023-10-26T05:54:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f65d40b4040 con 0x7f65e0158030 2023-10-26T05:54:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65cc066470 0x7f65cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f65d411ea10 con 0x7f65e0158030 2023-10-26T05:54:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.199+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f65cc06e060 con 0x7f65e0158030 2023-10-26T05:54:22.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.203+0000 7f65e4ec1640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65cc066470 0x7f65cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.203+0000 7f65e4ec1640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65cc066470 0x7f65cc068930 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f65e01f5560 tx=0x7f65d8007cc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:22.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.207+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f65d40ebee0 con 0x7f65e0158030 2023-10-26T05:54:22.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.387+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e014f410 con 0x7f65e0158030 2023-10-26T05:54:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65dd7fa640 1 -- 172.21.15.71:0/135737845 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f65e014f410 con 0x7f65e0158030 2023-10-26T05:54:22.398 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:22.398 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:54:22.399 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:54:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65cc066470 msgr2=0x7f65cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65cc066470 0x7f65cc068930 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f65e01f5560 tx=0x7f65d8007cc0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 msgr2=0x7f65e014dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 0x7f65e014dc50 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009850 tx=0x7f65d4005760 comp rx=0 tx=0).stop 2023-10-26T05:54:22.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 shutdown_connections 2023-10-26T05:54:22.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f65cc066470 0x7f65cc068930 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65e0158030 0x7f65e014dc50 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65e00b16a0 0x7f65e0146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 --2- 172.21.15.71:0/135737845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65e006b820 0x7f65e014f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 >> 172.21.15.71:0/135737845 conn(0x7f65e00af5e0 msgr2=0x7f65e006ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 shutdown_connections 2023-10-26T05:54:22.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.391+0000 7f65e7125640 1 -- 172.21.15.71:0/135737845 wait complete. 2023-10-26T05:54:22.415 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:22.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/3725430765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8406d720 msgr2=0x7f0e8415c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/3725430765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8406d720 0x7f0e8415c3b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009840 tx=0x7f0e840cd140 comp rx=0 tx=0).stop 2023-10-26T05:54:22.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/3725430765 shutdown_connections 2023-10-26T05:54:22.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/3725430765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8406d720 0x7f0e8415c3b0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/3725430765 >> 172.21.15.71:0/3725430765 conn(0x7f0e84070170 msgr2=0x7f0e84070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:22.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/3725430765 shutdown_connections 2023-10-26T05:54:22.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/3725430765 wait complete. 2023-10-26T05:54:22.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 Processor -- start 2023-10-26T05:54:22.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- start start 2023-10-26T05:54:22.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e8406d340 0x7f0e84155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e8406d720 0x7f0e840ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 0x7f0e84139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840adc50 con 0x7f0e840ca8c0 2023-10-26T05:54:22.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840afb30 con 0x7f0e8406d720 2023-10-26T05:54:22.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e8a8d5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e841541d0 con 0x7f0e8406d340 2023-10-26T05:54:22.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e88e72640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 0x7f0e84139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e88e72640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 0x7f0e84139ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44362/0 (socket says 172.21.15.71:44362) 2023-10-26T05:54:22.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.535+0000 7f0e88e72640 1 -- 172.21.15.71:0/984130634 learned_addr learned my addr 172.21.15.71:0/984130634 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:22.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e837fe640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e8406d720 0x7f0e840ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e83fff640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e8406d340 0x7f0e84155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e88e72640 1 -- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e8406d340 msgr2=0x7f0e84155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e88e72640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e8406d340 0x7f0e84155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e88e72640 1 -- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e8406d720 msgr2=0x7f0e840ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e88e72640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e8406d720 0x7f0e840ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e88e72640 1 -- 172.21.15.71:0/984130634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780094a0 con 0x7f0e840ca8c0 2023-10-26T05:54:22.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e83fff640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e8406d340 0x7f0e84155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:22.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e88e72640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 0x7f0e84139ac0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c730 tx=0x7f0e7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:22.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c003b80 con 0x7f0e840ca8c0 2023-10-26T05:54:22.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0e7c003ce0 con 0x7f0e840ca8c0 2023-10-26T05:54:22.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e840afd40 con 0x7f0e840ca8c0 2023-10-26T05:54:22.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00aa50 con 0x7f0e840ca8c0 2023-10-26T05:54:22.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e840b0300 con 0x7f0e840ca8c0 2023-10-26T05:54:22.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e84154360 con 0x7f0e840ca8c0 2023-10-26T05:54:22.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0e7c00abb0 con 0x7f0e840ca8c0 2023-10-26T05:54:22.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0e70066470 0x7f0e70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e7c083d50 con 0x7f0e840ca8c0 2023-10-26T05:54:22.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f0e7006e060 con 0x7f0e840ca8c0 2023-10-26T05:54:22.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e83fff640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0e70066470 0x7f0e70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.539+0000 7f0e83fff640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0e70066470 0x7f0e70068930 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0e740011f0 tx=0x7f0e74009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:22.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.543+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0e7c087030 con 0x7f0e840ca8c0 2023-10-26T05:54:22.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.687+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e84066d00 con 0x7f0e840ca8c0 2023-10-26T05:54:22.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e80ff9640 1 -- 172.21.15.71:0/984130634 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16563 (secure 0 0 0) 0x7f0e7c051220 con 0x7f0e840ca8c0 2023-10-26T05:54:22.698 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:22.698 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":144,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:22.182961+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:22.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0e70066470 msgr2=0x7f0e70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0e70066470 0x7f0e70068930 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0e740011f0 tx=0x7f0e74009040 comp rx=0 tx=0).stop 2023-10-26T05:54:22.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 msgr2=0x7f0e84139ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 0x7f0e84139ac0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c730 tx=0x7f0e7c00cbf0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 shutdown_connections 2023-10-26T05:54:22.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0e70066470 0x7f0e70068930 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e840ca8c0 0x7f0e84139ac0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e8406d720 0x7f0e840ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 --2- 172.21.15.71:0/984130634 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e8406d340 0x7f0e84155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 >> 172.21.15.71:0/984130634 conn(0x7f0e84070170 msgr2=0x7f0e84070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 shutdown_connections 2023-10-26T05:54:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.691+0000 7f0e8a8d5640 1 -- 172.21.15.71:0/984130634 wait complete. 2023-10-26T05:54:22.716 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:54:22.716 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:22.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 -- 172.21.15.71:0/3854748159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f764813c2f0 msgr2=0x7f76480b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 --2- 172.21.15.71:0/3854748159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f764813c2f0 0x7f76480b2350 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f76480b32d0 tx=0x7f763801c5e0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 -- 172.21.15.71:0/3854748159 shutdown_connections 2023-10-26T05:54:22.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 --2- 172.21.15.71:0/3854748159 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f76480b2890 0x7f76480caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 --2- 172.21.15.71:0/3854748159 >> v2:172.21.15.86:3300/0 conn(0x7f76480cb010 0x7f764806f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 --2- 172.21.15.71:0/3854748159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f764813c2f0 0x7f76480b2350 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 -- 172.21.15.71:0/3854748159 >> 172.21.15.71:0/3854748159 conn(0x7f7648069770 msgr2=0x7f7648069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:22.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 -- 172.21.15.71:0/3854748159 shutdown_connections 2023-10-26T05:54:22.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.815+0000 7f764c9f5640 1 -- 172.21.15.71:0/3854748159 wait complete. 2023-10-26T05:54:22.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 Processor -- start 2023-10-26T05:54:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- start start 2023-10-26T05:54:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 0x7f764814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76480cb010 0x7f764815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f764813c2f0 0x7f7648150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648131b20 con 0x7f76480b2890 2023-10-26T05:54:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7648151820 con 0x7f76480cb010 2023-10-26T05:54:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7648151b00 con 0x7f764813c2f0 2023-10-26T05:54:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 0x7f764814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 0x7f764814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44380/0 (socket says 172.21.15.71:44380) 2023-10-26T05:54:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 -- 172.21.15.71:0/1228835278 learned_addr learned my addr 172.21.15.71:0/1228835278 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:22.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7645d9b640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76480cb010 0x7f764815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7646d9d640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f764813c2f0 0x7f7648150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 -- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f764813c2f0 msgr2=0x7f7648150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f764813c2f0 0x7f7648150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 -- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76480cb010 msgr2=0x7f764815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:22.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76480cb010 0x7f764815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 -- 172.21.15.71:0/1228835278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638009560 con 0x7f76480b2890 2023-10-26T05:54:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7645d9b640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76480cb010 0x7f764815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7646d9d640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f764813c2f0 0x7f7648150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764659c640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 0x7f764814e910 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f76380ac040 tx=0x7f76380aa410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:22.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380aaa60 con 0x7f76480b2890 2023-10-26T05:54:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f76380b8070 con 0x7f76480b2890 2023-10-26T05:54:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7648151de0 con 0x7f76480b2890 2023-10-26T05:54:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76380aa600 con 0x7f76480b2890 2023-10-26T05:54:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764814ee50 con 0x7f76480b2890 2023-10-26T05:54:22.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7648066b10 con 0x7f76480b2890 2023-10-26T05:54:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7638003600 con 0x7f76480b2890 2023-10-26T05:54:22.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7628066470 0x7f7628068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f76380ba070 con 0x7f76480b2890 2023-10-26T05:54:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.819+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f762806e060 con 0x7f76480b2890 2023-10-26T05:54:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.823+0000 7f7645d9b640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7628066470 0x7f7628068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:22.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.823+0000 7f7645d9b640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7628066470 0x7f7628068930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f763c005d90 tx=0x7f763c005d20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:22.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.823+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f76380ec320 con 0x7f76480b2890 2023-10-26T05:54:23.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.995+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7648151c40 con 0x7f76480b2890 2023-10-26T05:54:23.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f7636ffd640 1 -- 172.21.15.71:0/1228835278 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7648151c40 con 0x7f76480b2890 2023-10-26T05:54:23.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:23.007 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:54:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:54:23.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7628066470 msgr2=0x7f7628068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7628066470 0x7f7628068930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f763c005d90 tx=0x7f763c005d20 comp rx=0 tx=0).stop 2023-10-26T05:54:23.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 msgr2=0x7f764814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 0x7f764814e910 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f76380ac040 tx=0x7f76380aa410 comp rx=0 tx=0).stop 2023-10-26T05:54:23.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 shutdown_connections 2023-10-26T05:54:23.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f764813c2f0 0x7f7648150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76480cb010 0x7f764815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7628066470 0x7f7628068930 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 --2- 172.21.15.71:0/1228835278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76480b2890 0x7f764814e910 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 >> 172.21.15.71:0/1228835278 conn(0x7f7648069770 msgr2=0x7f7648137d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 shutdown_connections 2023-10-26T05:54:23.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:22.999+0000 7f764c9f5640 1 -- 172.21.15.71:0/1228835278 wait complete. 2023-10-26T05:54:23.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:23.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2096516979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 msgr2=0x7f734015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 --2- 172.21.15.71:0/2096516979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015c760 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f7330009840 tx=0x7f733001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2096516979 shutdown_connections 2023-10-26T05:54:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 --2- 172.21.15.71:0/2096516979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7340070fe0 0x7f7340071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 --2- 172.21.15.71:0/2096516979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015c760 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2096516979 >> 172.21.15.71:0/2096516979 conn(0x7f7340069770 msgr2=0x7f7340069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2096516979 shutdown_connections 2023-10-26T05:54:23.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2096516979 wait complete. 2023-10-26T05:54:23.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 Processor -- start 2023-10-26T05:54:23.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- start start 2023-10-26T05:54:23.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7340070fe0 0x7f734015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73400b0040 0x7f7340156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734015e830 con 0x7f7340140cc0 2023-10-26T05:54:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7340137270 con 0x7f73400b0040 2023-10-26T05:54:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7340136d40 con 0x7f7340070fe0 2023-10-26T05:54:23.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44400/0 (socket says 172.21.15.71:44400) 2023-10-26T05:54:23.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 -- 172.21.15.71:0/2462558350 learned_addr learned my addr 172.21.15.71:0/2462558350 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:23.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733e59c640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73400b0040 0x7f7340156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733ed9d640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7340070fe0 0x7f734015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 -- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7340070fe0 msgr2=0x7f734015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7340070fe0 0x7f734015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 -- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73400b0040 msgr2=0x7f7340156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73400b0040 0x7f7340156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 -- 172.21.15.71:0/2462558350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73300094a0 con 0x7f7340140cc0 2023-10-26T05:54:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733e59c640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73400b0040 0x7f7340156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f733f59e640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015b430 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7338001570 tx=0x7f733800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338003db0 con 0x7f7340140cc0 2023-10-26T05:54:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7338005070 con 0x7f7340140cc0 2023-10-26T05:54:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734015a990 con 0x7f7340140cc0 2023-10-26T05:54:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338005480 con 0x7f7340140cc0 2023-10-26T05:54:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.139+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734015af10 con 0x7f7340140cc0 2023-10-26T05:54:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7340066b10 con 0x7f7340140cc0 2023-10-26T05:54:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f73380055e0 con 0x7f7340140cc0 2023-10-26T05:54:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f73277fe640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f732c066470 0x7f732c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f73380839c0 con 0x7f7340140cc0 2023-10-26T05:54:23.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f732c06e060 con 0x7f7340140cc0 2023-10-26T05:54:23.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f733ed9d640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f732c066470 0x7f732c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.143+0000 7f733ed9d640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f732c066470 0x7f732c068930 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f73300af000 tx=0x7f733001b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:23.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.147+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7338017070 con 0x7f7340140cc0 2023-10-26T05:54:23.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.319+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f734015b130 con 0x7f7340140cc0 2023-10-26T05:54:23.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.319+0000 7f73277fe640 1 -- 172.21.15.71:0/2462558350 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7338050e00 con 0x7f7340140cc0 2023-10-26T05:54:23.329 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:23.330 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":51,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:53:39.735661+0000","modified":"2023-10-26T05:53:41.733046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/1300756647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1300756647},{"type":"v1","addr":"172.21.15.71:6835","nonce":1300756647}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-26T05:54:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 51 2023-10-26T05:54:23.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f732c066470 msgr2=0x7f732c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f732c066470 0x7f732c068930 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f73300af000 tx=0x7f733001b010 comp rx=0 tx=0).stop 2023-10-26T05:54:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 msgr2=0x7f734015b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015b430 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7338001570 tx=0x7f733800cd20 comp rx=0 tx=0).stop 2023-10-26T05:54:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 shutdown_connections 2023-10-26T05:54:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f732c066470 0x7f732c068930 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7340140cc0 0x7f734015b430 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73400b0040 0x7f7340156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 --2- 172.21.15.71:0/2462558350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7340070fe0 0x7f734015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 >> 172.21.15.71:0/2462558350 conn(0x7f7340069770 msgr2=0x7f734006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 shutdown_connections 2023-10-26T05:54:23.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.323+0000 7f7345181640 1 -- 172.21.15.71:0/2462558350 wait complete. 2023-10-26T05:54:23.346 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:23.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 -- 172.21.15.71:0/73481486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe24140cc0 msgr2=0x7fbe2415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 --2- 172.21.15.71:0/73481486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe24140cc0 0x7fbe2415c760 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009930 tx=0x7fbe1401c850 comp rx=0 tx=0).stop 2023-10-26T05:54:23.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 -- 172.21.15.71:0/73481486 shutdown_connections 2023-10-26T05:54:23.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 --2- 172.21.15.71:0/73481486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe24140cc0 0x7fbe2415c760 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 -- 172.21.15.71:0/73481486 >> 172.21.15.71:0/73481486 conn(0x7fbe24069770 msgr2=0x7fbe24069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 -- 172.21.15.71:0/73481486 shutdown_connections 2023-10-26T05:54:23.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.459+0000 7fbe2980b640 1 -- 172.21.15.71:0/73481486 wait complete. 2023-10-26T05:54:23.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 Processor -- start 2023-10-26T05:54:23.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- start start 2023-10-26T05:54:23.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe24070fe0 0x7fbe241358f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 0x7fbe2413c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe24140cc0 0x7fbe24158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2413c930 con 0x7fbe240b2370 2023-10-26T05:54:23.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24137b90 con 0x7fbe24070fe0 2023-10-26T05:54:23.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24137de0 con 0x7fbe24140cc0 2023-10-26T05:54:23.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 0x7fbe2413c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 0x7fbe2413c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44412/0 (socket says 172.21.15.71:44412) 2023-10-26T05:54:23.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 -- 172.21.15.71:0/2895552291 learned_addr learned my addr 172.21.15.71:0/2895552291 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:23.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe22ffd640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe24070fe0 0x7fbe241358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe237fe640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe24140cc0 0x7fbe24158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 -- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe24140cc0 msgr2=0x7fbe24158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe24140cc0 0x7fbe24158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 -- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe24070fe0 msgr2=0x7fbe241358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe24070fe0 0x7fbe241358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 -- 172.21.15.71:0/2895552291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14009560 con 0x7fbe240b2370 2023-10-26T05:54:23.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe22ffd640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe24070fe0 0x7fbe241358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe227fc640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 0x7fbe2413c3f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800c4d0 tx=0x7fbe1800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18005480 con 0x7fbe240b2370 2023-10-26T05:54:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe180036f0 con 0x7fbe240b2370 2023-10-26T05:54:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe241380c0 con 0x7fbe240b2370 2023-10-26T05:54:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18003b00 con 0x7fbe240b2370 2023-10-26T05:54:23.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe24138680 con 0x7fbe240b2370 2023-10-26T05:54:23.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe24066b10 con 0x7fbe240b2370 2023-10-26T05:54:23.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.463+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe18003c60 con 0x7fbe240b2370 2023-10-26T05:54:23.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.467+0000 7fbe0bfff640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe10066470 0x7fbe10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.467+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbe18083670 con 0x7fbe240b2370 2023-10-26T05:54:23.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.467+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7fbe1006e060 con 0x7fbe240b2370 2023-10-26T05:54:23.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.467+0000 7fbe22ffd640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe10066470 0x7fbe10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.467+0000 7fbe22ffd640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe10066470 0x7fbe10068930 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbe140af000 tx=0x7fbe1401b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:23.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.471+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe18017020 con 0x7fbe240b2370 2023-10-26T05:54:23.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.615+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe240721e0 con 0x7fbe240b2370 2023-10-26T05:54:23.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe0bfff640 1 -- 172.21.15.71:0/2895552291 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16563 (secure 0 0 0) 0x7fbe18050c00 con 0x7fbe240b2370 2023-10-26T05:54:23.626 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:23.626 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":144,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:22.182961+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:53:37.960720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-26T05:53:38.704883+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.7","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe10066470 msgr2=0x7fbe10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe10066470 0x7fbe10068930 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbe140af000 tx=0x7fbe1401b090 comp rx=0 tx=0).stop 2023-10-26T05:54:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 msgr2=0x7fbe2413c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 0x7fbe2413c3f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800c4d0 tx=0x7fbe1800c990 comp rx=0 tx=0).stop 2023-10-26T05:54:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 shutdown_connections 2023-10-26T05:54:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe24140cc0 0x7fbe24158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe10066470 0x7fbe10068930 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe240b2370 0x7fbe2413c3f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 --2- 172.21.15.71:0/2895552291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe24070fe0 0x7fbe241358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 >> 172.21.15.71:0/2895552291 conn(0x7fbe24069770 msgr2=0x7fbe2415a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 shutdown_connections 2023-10-26T05:54:23.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.619+0000 7fbe2980b640 1 -- 172.21.15.71:0/2895552291 wait complete. 2023-10-26T05:54:23.644 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:54:23.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 -- 172.21.15.71:0/4036242190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b4159220 msgr2=0x7f38b40b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 --2- 172.21.15.71:0/4036242190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b4159220 0x7f38b40b1f80 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f38b40b1d40 tx=0x7f38a401c7e0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 -- 172.21.15.71:0/4036242190 shutdown_connections 2023-10-26T05:54:23.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 --2- 172.21.15.71:0/4036242190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38b40cd1c0 0x7f38b40cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 --2- 172.21.15.71:0/4036242190 >> v2:172.21.15.86:3300/0 conn(0x7f38b40ce9d0 0x7f38b40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 --2- 172.21.15.71:0/4036242190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b4159220 0x7f38b40b1f80 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 -- 172.21.15.71:0/4036242190 >> 172.21.15.71:0/4036242190 conn(0x7f38b40b0c80 msgr2=0x7f38b40b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 -- 172.21.15.71:0/4036242190 shutdown_connections 2023-10-26T05:54:23.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.763+0000 7f38ba772640 1 -- 172.21.15.71:0/4036242190 wait complete. 2023-10-26T05:54:23.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 Processor -- start 2023-10-26T05:54:23.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- start start 2023-10-26T05:54:23.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b40cd1c0 0x7f38b406bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 0x7f38b414a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38b4159220 0x7f38b4161d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b4150580 con 0x7f38b40ce9d0 2023-10-26T05:54:23.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b414ab30 con 0x7f38b40cd1c0 2023-10-26T05:54:23.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b406c1b0 con 0x7f38b4159220 2023-10-26T05:54:23.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 0x7f38b414a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 0x7f38b414a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44434/0 (socket says 172.21.15.71:44434) 2023-10-26T05:54:23.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 -- 172.21.15.71:0/306873843 learned_addr learned my addr 172.21.15.71:0/306873843 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:23.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b3fff640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b40cd1c0 0x7f38b406bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 -- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38b4159220 msgr2=0x7f38b4161d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b8d0f640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38b4159220 0x7f38b4161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38b4159220 0x7f38b4161d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 -- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b40cd1c0 msgr2=0x7f38b406bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b40cd1c0 0x7f38b406bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 -- 172.21.15.71:0/306873843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a4009560 con 0x7f38b40ce9d0 2023-10-26T05:54:23.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b3fff640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b40cd1c0 0x7f38b406bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:23.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b37fe640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 0x7f38b414a5f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c970 tx=0x7f38a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:23.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8007c70 con 0x7f38b40ce9d0 2023-10-26T05:54:23.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38a800f450 con 0x7f38b40ce9d0 2023-10-26T05:54:23.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b406c490 con 0x7f38b40ce9d0 2023-10-26T05:54:23.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800f860 con 0x7f38b40ce9d0 2023-10-26T05:54:23.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b406ca50 con 0x7f38b40ce9d0 2023-10-26T05:54:23.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b4066b10 con 0x7f38b40ce9d0 2023-10-26T05:54:23.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.767+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f38a800f9c0 con 0x7f38b40ce9d0 2023-10-26T05:54:23.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.771+0000 7f38b0ff9640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38a0066470 0x7f38a0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:23.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.771+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f38a80847d0 con 0x7f38b40ce9d0 2023-10-26T05:54:23.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.771+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=145}) v3 -- 0x7f38a006e060 con 0x7f38b40ce9d0 2023-10-26T05:54:23.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.771+0000 7f38b3fff640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38a0066470 0x7f38a0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:23.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.771+0000 7f38b3fff640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38a0066470 0x7f38a0068930 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f38a40af000 tx=0x7f38a401b020 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:23.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.775+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f38a8051c10 con 0x7f38b40ce9d0 2023-10-26T05:54:23.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.959+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f38b4150730 con 0x7f38b40ce9d0 2023-10-26T05:54:23.969 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:54:23.963+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:54:23.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.963+0000 7f38b0ff9640 1 -- 172.21.15.71:0/306873843 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f38b4150730 con 0x7f38b40ce9d0 2023-10-26T05:54:23.972 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:54:23.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38a0066470 msgr2=0x7f38a0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38a0066470 0x7f38a0068930 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f38a40af000 tx=0x7f38a401b020 comp rx=0 tx=0).stop 2023-10-26T05:54:23.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 msgr2=0x7f38b414a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:23.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 0x7f38b414a5f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f38a800c970 tx=0x7f38a800ce30 comp rx=0 tx=0).stop 2023-10-26T05:54:23.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 shutdown_connections 2023-10-26T05:54:23.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38b4159220 0x7f38b4161d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38a0066470 0x7f38a0068930 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38b40ce9d0 0x7f38b414a5f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 --2- 172.21.15.71:0/306873843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b40cd1c0 0x7f38b406bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:23.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 >> 172.21.15.71:0/306873843 conn(0x7f38b40b0c80 msgr2=0x7f38b40b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:23.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.967+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 shutdown_connections 2023-10-26T05:54:23.978 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:23.971+0000 7f38ba772640 1 -- 172.21.15.71:0/306873843 wait complete. 2023-10-26T05:54:23.982 INFO:tasks.ceph.mds.a.smithi071.stderr: -7755> 2023-10-26T05:53:46.327+0000 7f8c9da52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:54:23.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:54:24.024 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:54:24.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 -- 172.21.15.71:0/3316027323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0003fe0 msgr2=0x7fdfc00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:24.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/3316027323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0003fe0 0x7fdfc00047d0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc006fb0 tx=0x7fdfcc01cf90 comp rx=0 tx=0).stop 2023-10-26T05:54:24.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdd083640 1 -- 172.21.15.71:0/3316027323 reap_dead start 2023-10-26T05:54:24.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 -- 172.21.15.71:0/3316027323 shutdown_connections 2023-10-26T05:54:24.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 -- 172.21.15.71:0/3316027323 >> 172.21.15.71:0/3316027323 conn(0x7fdfd80aec60 msgr2=0x7fdfd80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:24.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 -- 172.21.15.71:0/3316027323 shutdown_connections 2023-10-26T05:54:24.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 -- 172.21.15.71:0/3316027323 wait complete. 2023-10-26T05:54:24.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 Processor -- start 2023-10-26T05:54:24.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.119+0000 7fdfdeae6640 1 -- start start 2023-10-26T05:54:24.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfc0003fe0 0x7fdfd8137750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:24.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 0x7fdfd8139c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:24.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfd8152da0 0x7fdfd8153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:24.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd81548c0 con 0x7fdfd81397c0 2023-10-26T05:54:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd8153760 con 0x7fdfc0003fe0 2023-10-26T05:54:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd8154e40 con 0x7fdfd8152da0 2023-10-26T05:54:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 0x7fdfd8139c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 0x7fdfd8139c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44478/0 (socket says 172.21.15.71:44478) 2023-10-26T05:54:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 -- 172.21.15.71:0/626502311 learned_addr learned my addr 172.21.15.71:0/626502311 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:24.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdc882640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfc0003fe0 0x7fdfd8137750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdd083640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfd8152da0 0x7fdfd8153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 -- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfd8152da0 msgr2=0x7fdfd8153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfd8152da0 0x7fdfd8153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 -- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfc0003fe0 msgr2=0x7fdfd8137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:24.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfc0003fe0 0x7fdfd8137750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:24.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 -- 172.21.15.71:0/626502311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd8151220 con 0x7fdfd81397c0 2023-10-26T05:54:24.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd7fff640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 0x7fdfd8139c20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000c10 tx=0x7fdfcc000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:24.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0b3070 con 0x7fdfd81397c0 2023-10-26T05:54:24.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdfcc001640 con 0x7fdfd81397c0 2023-10-26T05:54:24.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc006420 con 0x7fdfd81397c0 2023-10-26T05:54:24.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd8151500 con 0x7fdfd81397c0 2023-10-26T05:54:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd81552d0 con 0x7fdfd81397c0 2023-10-26T05:54:24.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd8066b10 con 0x7fdfd81397c0 2023-10-26T05:54:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdfcc0ab020 con 0x7fdfd81397c0 2023-10-26T05:54:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdfc4066470 0x7fdfc4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdfcc11dc70 con 0x7fdfd81397c0 2023-10-26T05:54:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7fdfc406e170 con 0x7fdfd81397c0 2023-10-26T05:54:24.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.123+0000 7fdfdc882640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdfc4066470 0x7fdfc4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:24.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.127+0000 7fdfdc882640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdfc4066470 0x7fdfc4068930 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80058d0 tx=0x7fdfc8005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:24.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.127+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdfcc0be020 con 0x7fdfd81397c0 2023-10-26T05:54:24.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.311+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdfd80721e0 con 0x7fdfd81397c0 2023-10-26T05:54:24.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.967+0000 7fdfd57fa640 1 -- 172.21.15.71:0/626502311 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdfcc0eafd0 con 0x7fdfd81397c0 2023-10-26T05:54:24.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdfc4066470 msgr2=0x7fdfc4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:24.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdfc4066470 0x7fdfc4068930 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80058d0 tx=0x7fdfc8005860 comp rx=0 tx=0).stop 2023-10-26T05:54:24.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 msgr2=0x7fdfd8139c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:24.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 0x7fdfd8139c20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000c10 tx=0x7fdfcc000f70 comp rx=0 tx=0).stop 2023-10-26T05:54:24.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 shutdown_connections 2023-10-26T05:54:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfd8152da0 0x7fdfd8153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdfc4066470 0x7fdfc4068930 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd81397c0 0x7fdfd8139c20 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 --2- 172.21.15.71:0/626502311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfc0003fe0 0x7fdfd8137750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:24.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 >> 172.21.15.71:0/626502311 conn(0x7fdfd80aec60 msgr2=0x7fdfd8159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:24.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 shutdown_connections 2023-10-26T05:54:24.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:24.971+0000 7fdfdeae6640 1 -- 172.21.15.71:0/626502311 wait complete. 2023-10-26T05:54:25.002 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:54:25.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f373e107640 1 -- 172.21.15.71:0/845534935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3720004d10 msgr2=0x7f3720005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:25.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f373e107640 1 --2- 172.21.15.71:0/845534935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3720004d10 0x7f3720005190 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f373000c290 tx=0x7f373001cfc0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f3737fff640 1 -- 172.21.15.71:0/845534935 reap_dead start 2023-10-26T05:54:25.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f373e107640 1 -- 172.21.15.71:0/845534935 shutdown_connections 2023-10-26T05:54:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f373e107640 1 -- 172.21.15.71:0/845534935 >> 172.21.15.71:0/845534935 conn(0x7f3738069b70 msgr2=0x7f37380b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f373e107640 1 -- 172.21.15.71:0/845534935 shutdown_connections 2023-10-26T05:54:25.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.171+0000 7f373e107640 1 -- 172.21.15.71:0/845534935 wait complete. 2023-10-26T05:54:25.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 Processor -- start 2023-10-26T05:54:25.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- start start 2023-10-26T05:54:25.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3738143840 0x7f3738143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 0x7f37380c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37380b22d0 0x7f37380b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380c8e10 con 0x7f37380c75c0 2023-10-26T05:54:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37380c7f80 con 0x7f37380b22d0 2023-10-26T05:54:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37380b2c90 con 0x7f3738143840 2023-10-26T05:54:25.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 0x7f37380c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 0x7f37380c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44500/0 (socket says 172.21.15.71:44500) 2023-10-26T05:54:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 -- 172.21.15.71:0/1403920979 learned_addr learned my addr 172.21.15.71:0/1403920979 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3737fff640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37380b22d0 0x7f37380b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f37377fe640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3738143840 0x7f3738143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:25.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 -- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3738143840 msgr2=0x7f3738143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3738143840 0x7f3738143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 -- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37380b22d0 msgr2=0x7f37380b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37380b22d0 0x7f37380b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37380cc520 con 0x7f37380c75c0 2023-10-26T05:54:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3737fff640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37380b22d0 0x7f37380b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:54:25.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f3736ffd640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 0x7f37380c7a40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f372c00fa00 tx=0x7f372c00fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c004460 con 0x7f37380c75c0 2023-10-26T05:54:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f372c00b8a0 con 0x7f37380c75c0 2023-10-26T05:54:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3738144240 con 0x7f37380c75c0 2023-10-26T05:54:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00bc90 con 0x7f37380c75c0 2023-10-26T05:54:25.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3738144800 con 0x7f37380c75c0 2023-10-26T05:54:25.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3738066b10 con 0x7f37380c75c0 2023-10-26T05:54:25.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f372c003600 con 0x7f37380c75c0 2023-10-26T05:54:25.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.175+0000 7f371ffff640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3724066470 0x7f3724068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:25.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.179+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f372c084bd0 con 0x7f37380c75c0 2023-10-26T05:54:25.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.179+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=146}) v3 -- 0x7f372406e170 con 0x7f37380c75c0 2023-10-26T05:54:25.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.179+0000 7f37377fe640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3724066470 0x7f3724068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:25.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.179+0000 7f37377fe640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3724066470 0x7f3724068930 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3728006230 tx=0x7f3728006e30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:25.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.183+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f372c051fd0 con 0x7f37380c75c0 2023-10-26T05:54:25.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.327+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37380721e0 con 0x7f37380c75c0 2023-10-26T05:54:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.971+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v146) v1 ==== 181+0+0 (secure 0 0 0) 0x7f372c0555f0 con 0x7f37380c75c0 2023-10-26T05:54:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:54:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.971+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 <== mon.0 v2:172.21.15.71:3300/0 8 ==== osd_map(146..146 src has 1..146) v4 ==== 354+0+0 (secure 0 0 0) 0x7f372c056390 con 0x7f37380c75c0 2023-10-26T05:54:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.971+0000 7f371ffff640 1 -- 172.21.15.71:0/1403920979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7f372406c5a0 con 0x7f37380c75c0 2023-10-26T05:54:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3724066470 msgr2=0x7f3724068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:25.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3724066470 0x7f3724068930 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3728006230 tx=0x7f3728006e30 comp rx=0 tx=0).stop 2023-10-26T05:54:25.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 msgr2=0x7f37380c7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:25.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 0x7f37380c7a40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f372c00fa00 tx=0x7f372c00fec0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 shutdown_connections 2023-10-26T05:54:25.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37380b22d0 0x7f37380b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3724066470 0x7f3724068930 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37380c75c0 0x7f37380c7a40 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 --2- 172.21.15.71:0/1403920979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3738143840 0x7f3738143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:25.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 >> 172.21.15.71:0/1403920979 conn(0x7f3738069b70 msgr2=0x7f37380c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:25.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 shutdown_connections 2023-10-26T05:54:25.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:25.983+0000 7f373e107640 1 -- 172.21.15.71:0/1403920979 wait complete. 2023-10-26T05:54:26.012 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:54:26.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 -- 172.21.15.71:0/3329069900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c06a7e0 msgr2=0x7fc85c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:26.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 --2- 172.21.15.71:0/3329069900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c06a7e0 0x7fc85c06ac90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0cef70 tx=0x7fc85001c870 comp rx=0 tx=0).stop 2023-10-26T05:54:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 -- 172.21.15.71:0/3329069900 shutdown_connections 2023-10-26T05:54:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 --2- 172.21.15.71:0/3329069900 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc85c0cf2f0 0x7fc85c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 --2- 172.21.15.71:0/3329069900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c06a7e0 0x7fc85c06ac90 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:26.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 --2- 172.21.15.71:0/3329069900 >> v2:172.21.15.86:3300/0 conn(0x7fc85c15b4d0 0x7fc85c0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:26.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 -- 172.21.15.71:0/3329069900 >> 172.21.15.71:0/3329069900 conn(0x7fc85c0d13f0 msgr2=0x7fc85c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:26.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 -- 172.21.15.71:0/3329069900 shutdown_connections 2023-10-26T05:54:26.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 -- 172.21.15.71:0/3329069900 wait complete. 2023-10-26T05:54:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 Processor -- start 2023-10-26T05:54:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.135+0000 7fc8623e2640 1 -- start start 2023-10-26T05:54:26.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc85c06a7e0 0x7fc85c159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:26.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc85c0cf2f0 0x7fc85c0c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:26.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 0x7fc85c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:26.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c1534b0 con 0x7fc85c15b4d0 2023-10-26T05:54:26.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0c7f30 con 0x7fc85c06a7e0 2023-10-26T05:54:26.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0c7980 con 0x7fc85c0cf2f0 2023-10-26T05:54:26.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 0x7fc85c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:26.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 0x7fc85c14bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44518/0 (socket says 172.21.15.71:44518) 2023-10-26T05:54:26.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 -- 172.21.15.71:0/2335664436 learned_addr learned my addr 172.21.15.71:0/2335664436 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:26.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc85b7fe640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc85c0cf2f0 0x7fc85c0c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc85bfff640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc85c06a7e0 0x7fc85c159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 -- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc85c0cf2f0 msgr2=0x7fc85c0c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc85c0cf2f0 0x7fc85c0c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 -- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc85c06a7e0 msgr2=0x7fc85c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:26.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc85c06a7e0 0x7fc85c159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850009560 con 0x7fc85c15b4d0 2023-10-26T05:54:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc86097f640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 0x7fc85c14bf50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00c940 tx=0x7fc84c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004220 con 0x7fc85c15b4d0 2023-10-26T05:54:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc84c004470 con 0x7fc85c15b4d0 2023-10-26T05:54:26.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c0c7c50 con 0x7fc85c15b4d0 2023-10-26T05:54:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c007bf0 con 0x7fc85c15b4d0 2023-10-26T05:54:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1596a0 con 0x7fc85c15b4d0 2023-10-26T05:54:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c066b10 con 0x7fc85c15b4d0 2023-10-26T05:54:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc84c007d50 con 0x7fc85c15b4d0 2023-10-26T05:54:26.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc848066470 0x7fc848068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fc84c084220 con 0x7fc85c15b4d0 2023-10-26T05:54:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=147}) v3 -- 0x7fc84806d300 con 0x7fc85c15b4d0 2023-10-26T05:54:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.139+0000 7fc85bfff640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc848066470 0x7fc848068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.143+0000 7fc85bfff640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc848066470 0x7fc848068930 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc854005c50 tx=0x7fc85400a390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:26.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.143+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc84c01d020 con 0x7fc85c15b4d0 2023-10-26T05:54:26.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.303+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc85c0721e0 con 0x7fc85c15b4d0 2023-10-26T05:54:26.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.983+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v147) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc84c0517c0 con 0x7fc85c15b4d0 2023-10-26T05:54:26.992 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:54:26.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.983+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 <== mon.0 v2:172.21.15.71:3300/0 8 ==== osd_map(147..147 src has 1..147) v4 ==== 330+0+0 (secure 0 0 0) 0x7fc84c055480 con 0x7fc85c15b4d0 2023-10-26T05:54:26.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.983+0000 7fc858ff9640 1 -- 172.21.15.71:0/2335664436 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=148}) v3 -- 0x7fc84806c090 con 0x7fc85c15b4d0 2023-10-26T05:54:27.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.995+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc848066470 msgr2=0x7fc848068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.995+0000 7fc8623e2640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc848066470 0x7fc848068930 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc854005c50 tx=0x7fc85400a390 comp rx=0 tx=0).stop 2023-10-26T05:54:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.995+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 msgr2=0x7fc85c14bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.995+0000 7fc8623e2640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 0x7fc85c14bf50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00c940 tx=0x7fc84c00ce00 comp rx=0 tx=0).stop 2023-10-26T05:54:27.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 shutdown_connections 2023-10-26T05:54:27.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc848066470 0x7fc848068930 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc85c15b4d0 0x7fc85c14bf50 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc85c0cf2f0 0x7fc85c0c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 --2- 172.21.15.71:0/2335664436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc85c06a7e0 0x7fc85c159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 >> 172.21.15.71:0/2335664436 conn(0x7fc85c0d13f0 msgr2=0x7fc85c15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:27.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 shutdown_connections 2023-10-26T05:54:27.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:26.999+0000 7fc8623e2640 1 -- 172.21.15.71:0/2335664436 wait complete. 2023-10-26T05:54:27.028 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:54:27.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2237651079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff18c0039b0 msgr2=0x7ff18c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2237651079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff18c0039b0 0x7ff18c003dd0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7ff1980016b0 tx=0x7ff19801c770 comp rx=0 tx=0).stop 2023-10-26T05:54:27.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a359e640 1 -- 172.21.15.71:0/2237651079 reap_dead start 2023-10-26T05:54:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2237651079 shutdown_connections 2023-10-26T05:54:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2237651079 >> 172.21.15.71:0/2237651079 conn(0x7ff1a40d1330 msgr2=0x7ff1a40d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2237651079 shutdown_connections 2023-10-26T05:54:27.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2237651079 wait complete. 2023-10-26T05:54:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 Processor -- start 2023-10-26T05:54:27.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.147+0000 7ff1a91ce640 1 -- start start 2023-10-26T05:54:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff18c0039b0 0x7ff1a40cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 0x7ff1a415e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a41f8860 0x7ff1a41f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40c9130 con 0x7ff1a40c8400 2023-10-26T05:54:27.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40c8890 con 0x7ff1a41f8860 2023-10-26T05:54:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40c8b70 con 0x7ff18c0039b0 2023-10-26T05:54:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 0x7ff1a415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 0x7ff1a415e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44528/0 (socket says 172.21.15.71:44528) 2023-10-26T05:54:27.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 -- 172.21.15.71:0/2261059580 learned_addr learned my addr 172.21.15.71:0/2261059580 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:27.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a2d9d640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff18c0039b0 0x7ff1a40cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a359e640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a41f8860 0x7ff1a41f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 -- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff18c0039b0 msgr2=0x7ff1a40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff18c0039b0 0x7ff1a40cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 -- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a41f8860 msgr2=0x7ff1a41f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a41f8860 0x7ff1a41f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 -- 172.21.15.71:0/2261059580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40c8e50 con 0x7ff1a40c8400 2023-10-26T05:54:27.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a259c640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 0x7ff1a415e570 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff194007fd0 tx=0x7ff19400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:27.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194005270 con 0x7ff1a40c8400 2023-10-26T05:54:27.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff19400a640 con 0x7ff1a40c8400 2023-10-26T05:54:27.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a41f8340 con 0x7ff1a40c8400 2023-10-26T05:54:27.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940054c0 con 0x7ff1a40c8400 2023-10-26T05:54:27.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a415eab0 con 0x7ff1a40c8400 2023-10-26T05:54:27.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4066b10 con 0x7ff1a40c8400 2023-10-26T05:54:27.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff19400ac60 con 0x7ff1a40c8400 2023-10-26T05:54:27.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff18b7fe640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1900664a0 0x7ff190068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff194083b00 con 0x7ff1a40c8400 2023-10-26T05:54:27.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.151+0000 7ff1a2d9d640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1900664a0 0x7ff190068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.155+0000 7ff1a2d9d640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1900664a0 0x7ff190068960 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff1980aa000 tx=0x7ff1980a9040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:27.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.155+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff194051360 con 0x7ff1a40c8400 2023-10-26T05:54:27.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.315+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7ff1a41f81a0 con 0x7ff1a40c8400 2023-10-26T05:54:27.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.323+0000 7ff18b7fe640 1 -- 172.21.15.71:0/2261059580 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v32) v1 ==== 71+0+0 (secure 0 0 0) 0x7ff1a41f81a0 con 0x7ff1a40c8400 2023-10-26T05:54:27.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1900664a0 msgr2=0x7ff190068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1900664a0 0x7ff190068960 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff1980aa000 tx=0x7ff1980a9040 comp rx=0 tx=0).stop 2023-10-26T05:54:27.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 msgr2=0x7ff1a415e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 0x7ff1a415e570 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7ff194007fd0 tx=0x7ff19400cb90 comp rx=0 tx=0).stop 2023-10-26T05:54:27.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 shutdown_connections 2023-10-26T05:54:27.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a41f8860 0x7ff1a41f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1900664a0 0x7ff190068960 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1a40c8400 0x7ff1a415e570 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 --2- 172.21.15.71:0/2261059580 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff18c0039b0 0x7ff1a40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 >> 172.21.15.71:0/2261059580 conn(0x7ff1a40d1330 msgr2=0x7ff1a40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:27.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 shutdown_connections 2023-10-26T05:54:27.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.327+0000 7ff1a91ce640 1 -- 172.21.15.71:0/2261059580 wait complete. 2023-10-26T05:54:27.351 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-10-26T05:54:27.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- 172.21.15.71:0/1235597203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 msgr2=0x7f140c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 --2- 172.21.15.71:0/1235597203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c15c760 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1408009900 tx=0x7f140801c850 comp rx=0 tx=0).stop 2023-10-26T05:54:27.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- 172.21.15.71:0/1235597203 shutdown_connections 2023-10-26T05:54:27.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 --2- 172.21.15.71:0/1235597203 >> v2:172.21.15.86:3300/0 conn(0x7f140c070fe0 0x7f140c071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 --2- 172.21.15.71:0/1235597203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c15c760 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- 172.21.15.71:0/1235597203 >> 172.21.15.71:0/1235597203 conn(0x7f140c069770 msgr2=0x7f140c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:27.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- 172.21.15.71:0/1235597203 shutdown_connections 2023-10-26T05:54:27.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- 172.21.15.71:0/1235597203 wait complete. 2023-10-26T05:54:27.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 Processor -- start 2023-10-26T05:54:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- start start 2023-10-26T05:54:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f140c070fe0 0x7f140c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f140c0b0040 0x7f140c13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c13ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c140140 con 0x7f140c140cc0 2023-10-26T05:54:27.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140c156bf0 con 0x7f140c0b0040 2023-10-26T05:54:27.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f141357b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140c13f530 con 0x7f140c070fe0 2023-10-26T05:54:27.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c13ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c13ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44546/0 (socket says 172.21.15.71:44546) 2023-10-26T05:54:27.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 -- 172.21.15.71:0/2978258364 learned_addr learned my addr 172.21.15.71:0/2978258364 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:27.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1410b16640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f140c0b0040 0x7f140c13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411317640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f140c070fe0 0x7f140c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 -- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f140c070fe0 msgr2=0x7f140c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f140c070fe0 0x7f140c1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 -- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f140c0b0040 msgr2=0x7f140c13e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f140c0b0040 0x7f140c13e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 -- 172.21.15.71:0/2978258364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1408009560 con 0x7f140c140cc0 2023-10-26T05:54:27.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411317640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f140c070fe0 0x7f140c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:27.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f1411b18640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c13ef00 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f140400c740 tx=0x7f140400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:27.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404006d60 con 0x7f140c140cc0 2023-10-26T05:54:27.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f140400ce50 con 0x7f140c140cc0 2023-10-26T05:54:27.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.467+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404003be0 con 0x7f140c140cc0 2023-10-26T05:54:27.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f140c13f810 con 0x7f140c140cc0 2023-10-26T05:54:27.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140c13fdd0 con 0x7f140c140cc0 2023-10-26T05:54:27.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140c066b10 con 0x7f140c140cc0 2023-10-26T05:54:27.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1404003d40 con 0x7f140c140cc0 2023-10-26T05:54:27.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f13fdffb640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13f0066470 0x7f13f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:27.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1404083960 con 0x7f140c140cc0 2023-10-26T05:54:27.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f1411317640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13f0066470 0x7f13f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:27.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.471+0000 7f1411317640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13f0066470 0x7f13f0068930 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f14080af000 tx=0x7f140801b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:27.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.475+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1404051170 con 0x7f140c140cc0 2023-10-26T05:54:27.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.687+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f140c0721e0 con 0x7f140c140cc0 2023-10-26T05:54:27.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.975+0000 7f13fdffb640 1 -- 172.21.15.71:0/2978258364 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v381) v1 ==== 140+0+0 (secure 0 0 0) 0x7f1404054790 con 0x7f140c140cc0 2023-10-26T05:54:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13f0066470 msgr2=0x7f13f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13f0066470 0x7f13f0068930 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f14080af000 tx=0x7f140801b090 comp rx=0 tx=0).stop 2023-10-26T05:54:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 msgr2=0x7f140c13ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c13ef00 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f140400c740 tx=0x7f140400cc00 comp rx=0 tx=0).stop 2023-10-26T05:54:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 shutdown_connections 2023-10-26T05:54:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f13f0066470 0x7f13f0068930 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f140c140cc0 0x7f140c13ef00 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f140c0b0040 0x7f140c13e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 --2- 172.21.15.71:0/2978258364 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f140c070fe0 0x7f140c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 >> 172.21.15.71:0/2978258364 conn(0x7f140c069770 msgr2=0x7f140c13a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 shutdown_connections 2023-10-26T05:54:27.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:27.979+0000 7f141357b640 1 -- 172.21.15.71:0/2978258364 wait complete. 2023-10-26T05:54:28.009 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-26T05:54:28.009 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T05:54:28.010 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/2250487108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93580ca8c0 msgr2=0x7f935813e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 --2- 172.21.15.71:0/2250487108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93580ca8c0 0x7f935813e1d0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9350009900 tx=0x7f935001c7f0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/2250487108 shutdown_connections 2023-10-26T05:54:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 --2- 172.21.15.71:0/2250487108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93580ca8c0 0x7f935813e1d0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 --2- 172.21.15.71:0/2250487108 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f935806d720 0x7f935815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/2250487108 >> 172.21.15.71:0/2250487108 conn(0x7f9358070170 msgr2=0x7f9358070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:28.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/2250487108 shutdown_connections 2023-10-26T05:54:28.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/2250487108 wait complete. 2023-10-26T05:54:28.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 Processor -- start 2023-10-26T05:54:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- start start 2023-10-26T05:54:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f935806d340 0x7f93581567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 0x7f93580ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93580ca8c0 0x7f9358158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580b0c70 con 0x7f935806d720 2023-10-26T05:54:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93580b0660 con 0x7f935806d340 2023-10-26T05:54:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93580aff50 con 0x7f93580ca8c0 2023-10-26T05:54:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 0x7f93580ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 0x7f93580ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44552/0 (socket says 172.21.15.71:44552) 2023-10-26T05:54:28.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 -- 172.21.15.71:0/1723305395 learned_addr learned my addr 172.21.15.71:0/1723305395 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f9356ffd640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f935806d340 0x7f93581567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 -- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93580ca8c0 msgr2=0x7f9358158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93577fe640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93580ca8c0 0x7f9358158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93580ca8c0 0x7f9358158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 -- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f935806d340 msgr2=0x7f93581567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f935806d340 0x7f93581567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 -- 172.21.15.71:0/1723305395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009560 con 0x7f935806d720 2023-10-26T05:54:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f9356ffd640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f935806d340 0x7f93581567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93577fe640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93580ca8c0 0x7f9358158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f93567fc640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 0x7f93580ad420 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b550 tx=0x7f934c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c010870 con 0x7f935806d720 2023-10-26T05:54:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f934c010ae0 con 0x7f935806d720 2023-10-26T05:54:28.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93580cd020 con 0x7f935806d720 2023-10-26T05:54:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c00f3c0 con 0x7f935806d720 2023-10-26T05:54:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.131+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93580b0900 con 0x7f935806d720 2023-10-26T05:54:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.135+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9358066d00 con 0x7f935806d720 2023-10-26T05:54:28.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.135+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f934c00f520 con 0x7f935806d720 2023-10-26T05:54:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.135+0000 7f933ffff640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9344066470 0x7f9344068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.135+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f934c083dd0 con 0x7f935806d720 2023-10-26T05:54:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.135+0000 7f9356ffd640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9344066470 0x7f9344068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.135+0000 7f9356ffd640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9344066470 0x7f9344068930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9348007fd0 tx=0x7f9348009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:28.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.139+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f934c0515e0 con 0x7f935806d720 2023-10-26T05:54:28.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.291+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9358162a80 con 0x7f935806d720 2023-10-26T05:54:28.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.291+0000 7f933ffff640 1 -- 172.21.15.71:0/1723305395 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+12913 (secure 0 0 0) 0x7f934c01e090 con 0x7f935806d720 2023-10-26T05:54:28.301 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":147,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:26.976061+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/1300756647":"2023-10-27T05:54:23.966331+0000","172.21.15.71:6834/1300756647":"2023-10-27T05:54:23.966331+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9344066470 msgr2=0x7f9344068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9344066470 0x7f9344068930 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9348007fd0 tx=0x7f9348009040 comp rx=0 tx=0).stop 2023-10-26T05:54:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 msgr2=0x7f93580ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 0x7f93580ad420 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f934c00b550 tx=0x7f934c00ba10 comp rx=0 tx=0).stop 2023-10-26T05:54:28.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 shutdown_connections 2023-10-26T05:54:28.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f93580ca8c0 0x7f9358158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9344066470 0x7f9344068930 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f935806d720 0x7f93580ad420 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 --2- 172.21.15.71:0/1723305395 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f935806d340 0x7f93581567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 >> 172.21.15.71:0/1723305395 conn(0x7f9358070170 msgr2=0x7f9358070550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:28.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 shutdown_connections 2023-10-26T05:54:28.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.295+0000 7f935d8b6640 1 -- 172.21.15.71:0/1723305395 wait complete. 2023-10-26T05:54:28.320 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:54:28.320 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:54:28.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4141529992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 msgr2=0x7f329c0ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4141529992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c0ca780 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3294009c80 tx=0x7f329401f540 comp rx=0 tx=0).stop 2023-10-26T05:54:28.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4141529992 shutdown_connections 2023-10-26T05:54:28.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4141529992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c0ca780 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4141529992 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f329c0b1230 0x7f329c0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4141529992 >> 172.21.15.71:0/4141529992 conn(0x7f329c0aec60 msgr2=0x7f329c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:28.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4141529992 shutdown_connections 2023-10-26T05:54:28.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4141529992 wait complete. 2023-10-26T05:54:28.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 Processor -- start 2023-10-26T05:54:28.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- start start 2023-10-26T05:54:28.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f329c0b1230 0x7f329c1ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c13a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c141a40 0x7f329c13a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c0b0900 con 0x7f329c0b21a0 2023-10-26T05:54:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c15a150 con 0x7f329c141a40 2023-10-26T05:54:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f32a1c1f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329c15a3a0 con 0x7f329c0b1230 2023-10-26T05:54:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c13a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c13a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44568/0 (socket says 172.21.15.71:44568) 2023-10-26T05:54:28.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 -- 172.21.15.71:0/4119495372 learned_addr learned my addr 172.21.15.71:0/4119495372 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:28.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329bfff640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c141a40 0x7f329c13a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 -- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f329c0b1230 msgr2=0x7f329c1ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f329c0b1230 0x7f329c1ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 -- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c141a40 msgr2=0x7f329c13a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c141a40 0x7f329c13a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.431+0000 7f329affd640 1 -- 172.21.15.71:0/4119495372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940098e0 con 0x7f329c0b21a0 2023-10-26T05:54:28.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f329affd640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c13a380 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f329000b550 tx=0x7f329000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:28.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3290010870 con 0x7f329c0b21a0 2023-10-26T05:54:28.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3290010ae0 con 0x7f329c0b21a0 2023-10-26T05:54:28.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c15a680 con 0x7f329c0b21a0 2023-10-26T05:54:28.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329000f3c0 con 0x7f329c0b21a0 2023-10-26T05:54:28.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c15ac40 con 0x7f329c0b21a0 2023-10-26T05:54:28.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c066b10 con 0x7f329c0b21a0 2023-10-26T05:54:28.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f329000f520 con 0x7f329c0b21a0 2023-10-26T05:54:28.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f3283fff640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3288066470 0x7f3288068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f32900841c0 con 0x7f329c0b21a0 2023-10-26T05:54:28.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f329b7fe640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3288066470 0x7f3288068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.435+0000 7f329b7fe640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3288066470 0x7f3288068930 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f329c15b810 tx=0x7f328c006bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:28.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.439+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3290051a20 con 0x7f329c0b21a0 2023-10-26T05:54:28.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.587+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f329c0721e0 con 0x7f329c0b21a0 2023-10-26T05:54:28.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.587+0000 7f3283fff640 1 -- 172.21.15.71:0/4119495372 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32900140a0 con 0x7f329c0b21a0 2023-10-26T05:54:28.597 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:54:28.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3288066470 msgr2=0x7f3288068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3288066470 0x7f3288068930 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f329c15b810 tx=0x7f328c006bd0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 msgr2=0x7f329c13a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c13a380 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f329000b550 tx=0x7f329000ba10 comp rx=0 tx=0).stop 2023-10-26T05:54:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 shutdown_connections 2023-10-26T05:54:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f329c141a40 0x7f329c13a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3288066470 0x7f3288068930 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f329c0b21a0 0x7f329c13a380 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 --2- 172.21.15.71:0/4119495372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f329c0b1230 0x7f329c1ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 >> 172.21.15.71:0/4119495372 conn(0x7f329c0aec60 msgr2=0x7f329c0c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:28.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 shutdown_connections 2023-10-26T05:54:28.602 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.591+0000 7f32a1c1f640 1 -- 172.21.15.71:0/4119495372 wait complete. 2023-10-26T05:54:28.615 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-26T05:54:28.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/1331173073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c070fe0 msgr2=0x7fbf4c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 --2- 172.21.15.71:0/1331173073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c070fe0 0x7fbf4c071470 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009900 tx=0x7fbf4001c590 comp rx=0 tx=0).stop 2023-10-26T05:54:28.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/1331173073 shutdown_connections 2023-10-26T05:54:28.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 --2- 172.21.15.71:0/1331173073 >> v2:172.21.15.86:3300/0 conn(0x7fbf4c0b1360 0x7fbf4c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 --2- 172.21.15.71:0/1331173073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c070fe0 0x7fbf4c071470 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/1331173073 >> 172.21.15.71:0/1331173073 conn(0x7fbf4c069770 msgr2=0x7fbf4c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:28.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/1331173073 shutdown_connections 2023-10-26T05:54:28.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/1331173073 wait complete. 2023-10-26T05:54:28.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 Processor -- start 2023-10-26T05:54:28.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- start start 2023-10-26T05:54:28.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf4c070fe0 0x7fbf4c138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4c0b1360 0x7fbf4c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 0x7fbf4c13b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c15eca0 con 0x7fbf4c140cc0 2023-10-26T05:54:28.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c15b170 con 0x7fbf4c0b1360 2023-10-26T05:54:28.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c15b410 con 0x7fbf4c070fe0 2023-10-26T05:54:28.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 0x7fbf4c13b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 0x7fbf4c13b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44584/0 (socket says 172.21.15.71:44584) 2023-10-26T05:54:28.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 -- 172.21.15.71:0/3027649031 learned_addr learned my addr 172.21.15.71:0/3027649031 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:28.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf4b7fe640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4c0b1360 0x7fbf4c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf4bfff640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf4c070fe0 0x7fbf4c138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 -- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf4c070fe0 msgr2=0x7fbf4c138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf4c070fe0 0x7fbf4c138630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 -- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4c0b1360 msgr2=0x7fbf4c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4c0b1360 0x7fbf4c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 -- 172.21.15.71:0/3027649031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40009560 con 0x7fbf4c140cc0 2023-10-26T05:54:28.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf4bfff640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf4c070fe0 0x7fbf4c138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:28.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf4b7fe640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4c0b1360 0x7fbf4c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:28.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf509e6640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 0x7fbf4c13b3b0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400ea20 tx=0x7fbf4400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:28.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cde0 con 0x7fbf4c140cc0 2023-10-26T05:54:28.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf44004c40 con 0x7fbf4c140cc0 2023-10-26T05:54:28.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c15e650 con 0x7fbf4c140cc0 2023-10-26T05:54:28.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44005020 con 0x7fbf4c140cc0 2023-10-26T05:54:28.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.731+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c15b6f0 con 0x7fbf4c140cc0 2023-10-26T05:54:28.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.735+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c066b10 con 0x7fbf4c140cc0 2023-10-26T05:54:28.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.735+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbf44005180 con 0x7fbf4c140cc0 2023-10-26T05:54:28.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.735+0000 7fbf48ff9640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf380664a0 0x7fbf38068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:28.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.735+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf44084660 con 0x7fbf4c140cc0 2023-10-26T05:54:28.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.735+0000 7fbf4bfff640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf380664a0 0x7fbf38068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:28.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.735+0000 7fbf4bfff640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf380664a0 0x7fbf38068960 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c000f70 tx=0x7fbf3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:28.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.739+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbf44051d50 con 0x7fbf4c140cc0 2023-10-26T05:54:28.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.931+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fbf4c0721e0 con 0x7fbf4c140cc0 2023-10-26T05:54:28.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.979+0000 7fbf48ff9640 1 -- 172.21.15.71:0/3027649031 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v382) v1 ==== 135+0+0 (secure 0 0 0) 0x7fbf4401f020 con 0x7fbf4c140cc0 2023-10-26T05:54:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf380664a0 msgr2=0x7fbf38068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf380664a0 0x7fbf38068960 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c000f70 tx=0x7fbf3c009040 comp rx=0 tx=0).stop 2023-10-26T05:54:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 msgr2=0x7fbf4c13b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 0x7fbf4c13b3b0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400ea20 tx=0x7fbf4400eee0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 shutdown_connections 2023-10-26T05:54:28.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf380664a0 0x7fbf38068960 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbf4c140cc0 0x7fbf4c13b3b0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf4c0b1360 0x7fbf4c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 --2- 172.21.15.71:0/3027649031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf4c070fe0 0x7fbf4c138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:28.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 >> 172.21.15.71:0/3027649031 conn(0x7fbf4c069770 msgr2=0x7fbf4c13c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:28.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 shutdown_connections 2023-10-26T05:54:28.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:28.983+0000 7fbf52449640 1 -- 172.21.15.71:0/3027649031 wait complete. 2023-10-26T05:54:29.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:54:29.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 -- 172.21.15.71:0/1798560538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734070fe0 msgr2=0x7f2734071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 --2- 172.21.15.71:0/1798560538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734070fe0 0x7f2734071470 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2728009870 tx=0x7f272801c790 comp rx=0 tx=0).stop 2023-10-26T05:54:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 -- 172.21.15.71:0/1798560538 shutdown_connections 2023-10-26T05:54:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 --2- 172.21.15.71:0/1798560538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734070fe0 0x7f2734071470 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 -- 172.21.15.71:0/1798560538 >> 172.21.15.71:0/1798560538 conn(0x7f2734069770 msgr2=0x7f2734069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:29.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 -- 172.21.15.71:0/1798560538 shutdown_connections 2023-10-26T05:54:29.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.107+0000 7f273becb640 1 -- 172.21.15.71:0/1798560538 wait complete. 2023-10-26T05:54:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 Processor -- start 2023-10-26T05:54:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- start start 2023-10-26T05:54:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2734070fe0 0x7f273413a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27340b1360 0x7f273413ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 0x7f2734157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273413cf50 con 0x7f2734140cc0 2023-10-26T05:54:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27340a6d40 con 0x7f27340b1360 2023-10-26T05:54:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27340a6f90 con 0x7f2734070fe0 2023-10-26T05:54:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 0x7f2734157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 0x7f2734157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44608/0 (socket says 172.21.15.71:44608) 2023-10-26T05:54:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 -- 172.21.15.71:0/577953075 learned_addr learned my addr 172.21.15.71:0/577953075 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f2739466640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27340b1360 0x7f273413ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f2739c67640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2734070fe0 0x7f273413a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 -- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2734070fe0 msgr2=0x7f273413a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2734070fe0 0x7f273413a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 -- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27340b1360 msgr2=0x7f273413ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27340b1360 0x7f273413ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 -- 172.21.15.71:0/577953075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280094a0 con 0x7f2734140cc0 2023-10-26T05:54:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f2739466640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27340b1360 0x7f273413ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273a468640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 0x7f2734157b80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f272400c780 tx=0x7f272400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724003d90 con 0x7f2734140cc0 2023-10-26T05:54:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f272400ce90 con 0x7f2734140cc0 2023-10-26T05:54:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27340a7270 con 0x7f2734140cc0 2023-10-26T05:54:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240052d0 con 0x7f2734140cc0 2023-10-26T05:54:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27340a7830 con 0x7f2734140cc0 2023-10-26T05:54:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.111+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2734066b10 con 0x7f2734140cc0 2023-10-26T05:54:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.115+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2724005430 con 0x7f2734140cc0 2023-10-26T05:54:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.115+0000 7f27327fc640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2720066470 0x7f2720068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.115+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2724083180 con 0x7f2734140cc0 2023-10-26T05:54:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.115+0000 7f2739c67640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2720066470 0x7f2720068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.115+0000 7f2739c67640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2720066470 0x7f2720068930 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f272c007fd0 tx=0x7f272c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:29.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.119+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2724050990 con 0x7f2734140cc0 2023-10-26T05:54:29.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7f2734141ec0 con 0x7f2734140cc0 2023-10-26T05:54:29.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f27327fc640 1 -- 172.21.15.71:0/577953075 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v32) v1 ==== 59+0+1116 (secure 0 0 0) 0x7f2724053fb0 con 0x7f2734140cc0 2023-10-26T05:54:29.271 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:54:29.271 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:54:29.271 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:54:29.272 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:54:29.272 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:54:29.272 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:54:29.272 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:54:29.272 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:54:29.273 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:54:29.273 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:54:29.273 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:54:29.273 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:54:29.273 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:54:29.273 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:54:29.274 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:54:29.274 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:54:29.274 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:54:29.274 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:54:29.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2720066470 msgr2=0x7f2720068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2720066470 0x7f2720068930 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f272c007fd0 tx=0x7f272c009040 comp rx=0 tx=0).stop 2023-10-26T05:54:29.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 msgr2=0x7f2734157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 0x7f2734157b80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f272400c780 tx=0x7f272400cc40 comp rx=0 tx=0).stop 2023-10-26T05:54:29.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 shutdown_connections 2023-10-26T05:54:29.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2720066470 0x7f2720068930 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2734140cc0 0x7f2734157b80 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27340b1360 0x7f273413ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 --2- 172.21.15.71:0/577953075 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2734070fe0 0x7f273413a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 >> 172.21.15.71:0/577953075 conn(0x7f2734069770 msgr2=0x7f27340b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:29.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 shutdown_connections 2023-10-26T05:54:29.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.263+0000 7f273becb640 1 -- 172.21.15.71:0/577953075 wait complete. 2023-10-26T05:54:29.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:54:29.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/1411237016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778005010 msgr2=0x7f177800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 --2- 172.21.15.71:0/1411237016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1778005010 0x7f177800b990 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f177c009eb0 tx=0x7f177c01d5d0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 -- 172.21.15.71:0/1411237016 reap_dead start 2023-10-26T05:54:29.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/1411237016 shutdown_connections 2023-10-26T05:54:29.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/1411237016 >> 172.21.15.71:0/1411237016 conn(0x7f178c06fd20 msgr2=0x7f178c0cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:29.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/1411237016 shutdown_connections 2023-10-26T05:54:29.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/1411237016 wait complete. 2023-10-26T05:54:29.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 Processor -- start 2023-10-26T05:54:29.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- start start 2023-10-26T05:54:29.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1778005010 0x7f178c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178c0c5690 0x7f178c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 0x7f178c0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0b4dc0 con 0x7f178c0b43d0 2023-10-26T05:54:29.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c15ab00 con 0x7f178c0c5690 2023-10-26T05:54:29.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c15ade0 con 0x7f1778005010 2023-10-26T05:54:29.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 0x7f178c0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 0x7f178c0b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44622/0 (socket says 172.21.15.71:44622) 2023-10-26T05:54:29.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 -- 172.21.15.71:0/3558437579 learned_addr learned my addr 172.21.15.71:0/3558437579 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:29.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f178bfff640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178c0c5690 0x7f178c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 -- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1778005010 msgr2=0x7f178c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1790cad640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1778005010 0x7f178c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1778005010 0x7f178c151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 -- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178c0c5690 msgr2=0x7f178c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178c0c5690 0x7f178c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 -- 172.21.15.71:0/3558437579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c15b0c0 con 0x7f178c0b43d0 2023-10-26T05:54:29.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f178bfff640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178c0c5690 0x7f178c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:29.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17914ae640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 0x7f178c0b4880 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f177c009eb0 tx=0x7f177c004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:29.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0b6070 con 0x7f178c0b43d0 2023-10-26T05:54:29.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f177c009200 con 0x7f178c0b43d0 2023-10-26T05:54:29.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c15b3a0 con 0x7f178c0b43d0 2023-10-26T05:54:29.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0a9710 con 0x7f178c0b43d0 2023-10-26T05:54:29.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.399+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c1520e0 con 0x7f178c0b43d0 2023-10-26T05:54:29.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.403+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c066d00 con 0x7f178c0b43d0 2023-10-26T05:54:29.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.403+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f177c0a99e0 con 0x7f178c0b43d0 2023-10-26T05:54:29.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.403+0000 7f17897fa640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f176c066470 0x7f176c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.403+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f177c11e830 con 0x7f178c0b43d0 2023-10-26T05:54:29.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.403+0000 7f1790cad640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f176c066470 0x7f176c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.403+0000 7f1790cad640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f176c066470 0x7f176c068930 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1784005d10 tx=0x7f1784005c80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:29.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.407+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f177c0ebfa0 con 0x7f178c0b43d0 2023-10-26T05:54:29.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.551+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f178c15b850 con 0x7f178c0b43d0 2023-10-26T05:54:29.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.555+0000 7f17897fa640 1 -- 172.21.15.71:0/3558437579 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v33)=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f177c0c00c0 con 0x7f178c0b43d0 2023-10-26T05:54:29.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f176c066470 msgr2=0x7f176c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f176c066470 0x7f176c068930 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1784005d10 tx=0x7f1784005c80 comp rx=0 tx=0).stop 2023-10-26T05:54:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 msgr2=0x7f178c0b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 0x7f178c0b4880 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f177c009eb0 tx=0x7f177c004f50 comp rx=0 tx=0).stop 2023-10-26T05:54:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 shutdown_connections 2023-10-26T05:54:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f176c066470 0x7f176c068930 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f178c0b43d0 0x7f178c0b4880 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f178c0c5690 0x7f178c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 --2- 172.21.15.71:0/3558437579 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1778005010 0x7f178c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 >> 172.21.15.71:0/3558437579 conn(0x7f178c06fd20 msgr2=0x7f178c0ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 shutdown_connections 2023-10-26T05:54:29.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.559+0000 7f1792f11640 1 -- 172.21.15.71:0/3558437579 wait complete. 2023-10-26T05:54:29.582 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:54:29.583 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:54:29.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:54:29.699 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:54:29.700 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:54:29.700 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:29.706 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:54:29.706 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:54:29.707 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:29.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:54:29.746 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:54:29.747 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:29.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:54:29.754 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:54:29.754 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:54:29.794 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:54:29.794 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:54:29.795 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:29.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- 172.21.15.71:0/1398268741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 msgr2=0x7fc3640b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 --2- 172.21.15.71:0/1398268741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc3640b2350 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009c80 tx=0x7fc35c01cd40 comp rx=0 tx=0).stop 2023-10-26T05:54:29.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- 172.21.15.71:0/1398268741 shutdown_connections 2023-10-26T05:54:29.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 --2- 172.21.15.71:0/1398268741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3640b2890 0x7fc3640caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 --2- 172.21.15.71:0/1398268741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc3640b2350 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- 172.21.15.71:0/1398268741 >> 172.21.15.71:0/1398268741 conn(0x7fc364069770 msgr2=0x7fc364069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:29.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- 172.21.15.71:0/1398268741 shutdown_connections 2023-10-26T05:54:29.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- 172.21.15.71:0/1398268741 wait complete. 2023-10-26T05:54:29.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 Processor -- start 2023-10-26T05:54:29.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- start start 2023-10-26T05:54:29.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3640b2890 0x7fc36415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3640cb010 0x7fc36415c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc36414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415a980 con 0x7fc36413c2f0 2023-10-26T05:54:29.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415cbd0 con 0x7fc3640cb010 2023-10-26T05:54:29.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36415b1e0 con 0x7fc3640b2890 2023-10-26T05:54:29.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc36414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc36414fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44632/0 (socket says 172.21.15.71:44632) 2023-10-26T05:54:29.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 -- 172.21.15.71:0/2629526378 learned_addr learned my addr 172.21.15.71:0/2629526378 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:29.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc369430640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3640cb010 0x7fc36415c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc369c31640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3640b2890 0x7fc36415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 -- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3640b2890 msgr2=0x7fc36415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3640b2890 0x7fc36415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 -- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3640cb010 msgr2=0x7fc36415c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:29.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3640cb010 0x7fc36415c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:29.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 -- 172.21.15.71:0/2629526378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0098e0 con 0x7fc36413c2f0 2023-10-26T05:54:29.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36a432640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc36414fa10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c3f0 tx=0x7fc35400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:29.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354011790 con 0x7fc36413c2f0 2023-10-26T05:54:29.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc3540119e0 con 0x7fc36413c2f0 2023-10-26T05:54:29.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc354011df0 con 0x7fc36413c2f0 2023-10-26T05:54:29.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.919+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36415c0d0 con 0x7fc36413c2f0 2023-10-26T05:54:29.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36415ae70 con 0x7fc36413c2f0 2023-10-26T05:54:29.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364066b10 con 0x7fc36413c2f0 2023-10-26T05:54:29.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc35401c460 con 0x7fc36413c2f0 2023-10-26T05:54:29.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc3627fc640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc350066470 0x7fc350068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:29.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc354083c10 con 0x7fc36413c2f0 2023-10-26T05:54:29.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc369c31640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc350066470 0x7fc350068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:29.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.923+0000 7fc369c31640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc350066470 0x7fc350068930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0af000 tx=0x7fc35c01b580 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:29.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:29.927+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc354051420 con 0x7fc36413c2f0 2023-10-26T05:54:30.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.119+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3640721e0 con 0x7fc36413c2f0 2023-10-26T05:54:30.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.119+0000 7fc3627fc640 1 -- 172.21.15.71:0/2629526378 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fc354003810 con 0x7fc36413c2f0 2023-10-26T05:54:30.128 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:30.128 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3116082064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3116082064},{"type":"v1","addr":"172.21.15.86:6835","nonce":3116082064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2036125395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2036125395},{"type":"v1","addr":"172.21.15.86:6837","nonce":2036125395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3737560297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3737560297},{"type":"v1","addr":"172.21.15.71:6833","nonce":3737560297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8427,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/3235703057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":3235703057},{"type":"v1","addr":"172.21.15.71:6835","nonce":3235703057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[]} 2023-10-26T05:54:30.129 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 53 2023-10-26T05:54:30.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc350066470 msgr2=0x7fc350068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc350066470 0x7fc350068930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0af000 tx=0x7fc35c01b580 comp rx=0 tx=0).stop 2023-10-26T05:54:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 msgr2=0x7fc36414fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc36414fa10 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c3f0 tx=0x7fc35400c8b0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 shutdown_connections 2023-10-26T05:54:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc350066470 0x7fc350068930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc36413c2f0 0x7fc36414fa10 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3640cb010 0x7fc36415c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 --2- 172.21.15.71:0/2629526378 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3640b2890 0x7fc36415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 >> 172.21.15.71:0/2629526378 conn(0x7fc364069770 msgr2=0x7fc364159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:30.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 shutdown_connections 2023-10-26T05:54:30.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.123+0000 7fc36be95640 1 -- 172.21.15.71:0/2629526378 wait complete. 2023-10-26T05:54:30.147 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:54:30.147 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:54:30.148 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:54:30.148 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:54:30.148 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:54:30.148 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:54:30.149 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:54:30.149 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:54:30.149 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:54:30.150 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:54:30.150 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:54:30.150 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:54:30.150 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:54:30.151 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:54:30.151 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:54:30.151 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:54:30.158 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:54:30.158 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:54:30.159 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:54:30.159 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:54:30.164 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:54:30.164 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:54:30.167 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:54:30.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:54:30.206 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:54:30.207 DEBUG:teuthology.parallel:result is None 2023-10-26T05:54:30.208 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:54:30.208 DEBUG:teuthology.parallel:result is None 2023-10-26T05:54:30.209 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:54:30.210 DEBUG:teuthology.parallel:result is None 2023-10-26T05:54:30.211 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:54:30.211 DEBUG:teuthology.parallel:result is None 2023-10-26T05:54:30.211 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:54:30.266 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:54:30.266 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:54:30.285 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:54:30.285 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:54:30.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/2575758595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 msgr2=0x7f3eb40b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:30.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 --2- 172.21.15.71:0/2575758595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40b43e0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009900 tx=0x7f3ea401f070 comp rx=0 tx=0).stop 2023-10-26T05:54:30.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/2575758595 shutdown_connections 2023-10-26T05:54:30.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 --2- 172.21.15.71:0/2575758595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40b43e0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/2575758595 >> 172.21.15.71:0/2575758595 conn(0x7f3eb40d12d0 msgr2=0x7f3eb40d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:30.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/2575758595 shutdown_connections 2023-10-26T05:54:30.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/2575758595 wait complete. 2023-10-26T05:54:30.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 Processor -- start 2023-10-26T05:54:30.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- start start 2023-10-26T05:54:30.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb406a390 0x7f3eb4157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:30.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eb40cced0 0x7f3eb40ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:30.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:30.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb4157e60 con 0x7f3eb415b3b0 2023-10-26T05:54:30.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb40c9460 con 0x7f3eb406a390 2023-10-26T05:54:30.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb41578b0 con 0x7f3eb40cced0 2023-10-26T05:54:30.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:30.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44670/0 (socket says 172.21.15.71:44670) 2023-10-26T05:54:30.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 -- 172.21.15.71:0/1680113572 learned_addr learned my addr 172.21.15.71:0/1680113572 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:30.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 -- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eb40cced0 msgr2=0x7f3eb40ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:30.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb2ffd640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb406a390 0x7f3eb4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:30.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eb40cced0 0x7f3eb40ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 -- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb406a390 msgr2=0x7f3eb4157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:30.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb406a390 0x7f3eb4157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:30.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 -- 172.21.15.71:0/1680113572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea4009560 con 0x7f3eb415b3b0 2023-10-26T05:54:30.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb37fe640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40ca9b0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c7a0 tx=0x7f3eac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:30.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac003b80 con 0x7f3eb415b3b0 2023-10-26T05:54:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3eac003ce0 con 0x7f3eb415b3b0 2023-10-26T05:54:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb40c96b0 con 0x7f3eb415b3b0 2023-10-26T05:54:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00aab0 con 0x7f3eb415b3b0 2023-10-26T05:54:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.319+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb40c9c70 con 0x7f3eb415b3b0 2023-10-26T05:54:30.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.323+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb4066b10 con 0x7f3eb415b3b0 2023-10-26T05:54:30.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.323+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3eac00ac10 con 0x7f3eb415b3b0 2023-10-26T05:54:30.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.323+0000 7f3eab7fe640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e94066470 0x7f3e94068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:30.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.323+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3eac083ac0 con 0x7f3eb415b3b0 2023-10-26T05:54:30.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.323+0000 7f3eb2ffd640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e94066470 0x7f3e94068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:30.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.323+0000 7f3eb2ffd640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e94066470 0x7f3e94068930 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40b2000 tx=0x7f3ea401d8b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:30.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.327+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3eac0512d0 con 0x7f3eb415b3b0 2023-10-26T05:54:30.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:30.491+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3eb406c350 con 0x7f3eb415b3b0 2023-10-26T05:54:31.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.019+0000 7f3eab7fe640 1 -- 172.21.15.71:0/1680113572 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v149) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3eac0548f0 con 0x7f3eb415b3b0 2023-10-26T05:54:31.029 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:54:31.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e94066470 msgr2=0x7f3e94068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e94066470 0x7f3e94068930 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40b2000 tx=0x7f3ea401d8b0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 msgr2=0x7f3eb40ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40ca9b0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00c7a0 tx=0x7f3eac00cc60 comp rx=0 tx=0).stop 2023-10-26T05:54:31.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 shutdown_connections 2023-10-26T05:54:31.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3e94066470 0x7f3e94068930 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3eb415b3b0 0x7f3eb40ca9b0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eb40cced0 0x7f3eb40ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 --2- 172.21.15.71:0/1680113572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb406a390 0x7f3eb4157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 >> 172.21.15.71:0/1680113572 conn(0x7f3eb40d12d0 msgr2=0x7f3eb40cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:31.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 shutdown_connections 2023-10-26T05:54:31.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.023+0000 7f3eb955a640 1 -- 172.21.15.71:0/1680113572 wait complete. 2023-10-26T05:54:31.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:54:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc95ffb640 1 -- 172.21.15.71:0/3977190362 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900ad6a0 con 0x7fbc980d0470 2023-10-26T05:54:31.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/3977190362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0470 msgr2=0x7fbc9806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/3977190362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0470 0x7fbc9806aa70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009bf0 tx=0x7fbc9001cac0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/3977190362 shutdown_connections 2023-10-26T05:54:31.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/3977190362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980cd880 0x7fbc9815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/3977190362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0470 0x7fbc9806aa70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/3977190362 >> 172.21.15.71:0/3977190362 conn(0x7fbc980cdd80 msgr2=0x7fbc980ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:31.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/3977190362 shutdown_connections 2023-10-26T05:54:31.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.159+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/3977190362 wait complete. 2023-10-26T05:54:31.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 Processor -- start 2023-10-26T05:54:31.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- start start 2023-10-26T05:54:31.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 0x7fbc981586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980d0470 0x7fbc9814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0850 0x7fbc98153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980c7780 con 0x7fbc980cd880 2023-10-26T05:54:31.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc981542e0 con 0x7fbc980d0850 2023-10-26T05:54:31.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980b6a40 con 0x7fbc980d0470 2023-10-26T05:54:31.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 0x7fbc981586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 0x7fbc981586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44682/0 (socket says 172.21.15.71:44682) 2023-10-26T05:54:31.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 -- 172.21.15.71:0/4218958475 learned_addr learned my addr 172.21.15.71:0/4218958475 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:31.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc967fc640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980d0470 0x7fbc9814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc977fe640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0850 0x7fbc98153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 -- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980d0470 msgr2=0x7fbc9814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980d0470 0x7fbc9814a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 -- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0850 msgr2=0x7fbc98153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0850 0x7fbc98153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 -- 172.21.15.71:0/4218958475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009820 con 0x7fbc980cd880 2023-10-26T05:54:31.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc96ffd640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 0x7fbc981586e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fbc9001a6f0 tx=0x7fbc900a5a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:31.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900bc070 con 0x7fbc980cd880 2023-10-26T05:54:31.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc980b64c0 con 0x7fbc980cd880 2023-10-26T05:54:31.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc90005130 con 0x7fbc980cd880 2023-10-26T05:54:31.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90005540 con 0x7fbc980cd880 2023-10-26T05:54:31.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc980c6f80 con 0x7fbc980cd880 2023-10-26T05:54:31.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc98066b10 con 0x7fbc980cd880 2023-10-26T05:54:31.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc900b4040 con 0x7fbc980cd880 2023-10-26T05:54:31.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.163+0000 7fbc8f7fe640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc780664a0 0x7fbc78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.167+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc9011e360 con 0x7fbc980cd880 2023-10-26T05:54:31.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.167+0000 7fbc967fc640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc780664a0 0x7fbc78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.167+0000 7fbc967fc640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc780664a0 0x7fbc78068960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc980c4920 tx=0x7fbc80009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:31.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.167+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc900ebab0 con 0x7fbc980cd880 2023-10-26T05:54:31.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.375+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbc980721e0 con 0x7fbc980cd880 2023-10-26T05:54:31.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc8f7fe640 1 -- 172.21.15.71:0/4218958475 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbc900ef0d0 con 0x7fbc980cd880 2023-10-26T05:54:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:31.386 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:54:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:54:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:54:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:54:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:54:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:54:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:54:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:54:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:54:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:54:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:54:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:54:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:54:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:54:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:54:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:54:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:54:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:54:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:54:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:54:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:54:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:54:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:54:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:54:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:54:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:54:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:54:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:54:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:54:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:54:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:54:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:54:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:54:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:54:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:54:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:54:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:54:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:54:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:54:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:54:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:54:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:54:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:54:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:54:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:54:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:54:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:54:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:54:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:54:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:54:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:54:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:54:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:54:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:54:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:54:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:54:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:54:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:54:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:54:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:54:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:54:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:54:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:54:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:54:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:54:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:54:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:54:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:54:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:54:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:54:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:54:31.424 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:31.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc780664a0 msgr2=0x7fbc78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc780664a0 0x7fbc78068960 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc980c4920 tx=0x7fbc80009040 comp rx=0 tx=0).stop 2023-10-26T05:54:31.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 msgr2=0x7fbc981586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 0x7fbc981586e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fbc9001a6f0 tx=0x7fbc900a5a30 comp rx=0 tx=0).stop 2023-10-26T05:54:31.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 shutdown_connections 2023-10-26T05:54:31.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc980d0850 0x7fbc98153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc980d0470 0x7fbc9814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc780664a0 0x7fbc78068960 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 --2- 172.21.15.71:0/4218958475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc980cd880 0x7fbc981586e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 >> 172.21.15.71:0/4218958475 conn(0x7fbc980cdd80 msgr2=0x7fbc980cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:31.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 shutdown_connections 2023-10-26T05:54:31.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.379+0000 7fbc9d3a1640 1 -- 172.21.15.71:0/4218958475 wait complete. 2023-10-26T05:54:31.429 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:31.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.547+0000 7f837bfff640 1 -- 172.21.15.71:0/2766408624 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740b08a0 con 0x7f83840d0850 2023-10-26T05:54:31.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2766408624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 msgr2=0x7f838415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2766408624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f838415b3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8374009840 tx=0x7f837401c690 comp rx=0 tx=0).stop 2023-10-26T05:54:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2766408624 shutdown_connections 2023-10-26T05:54:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2766408624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83840cd880 0x7f838415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2766408624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f838415b3a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2766408624 >> 172.21.15.71:0/2766408624 conn(0x7f83840cdd80 msgr2=0x7f83840ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2766408624 shutdown_connections 2023-10-26T05:54:31.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2766408624 wait complete. 2023-10-26T05:54:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 Processor -- start 2023-10-26T05:54:31.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- start start 2023-10-26T05:54:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 0x7f838415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83840d0470 0x7f83841589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f8384153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838415ca10 con 0x7f83840cd880 2023-10-26T05:54:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83840cb3c0 con 0x7f83840d0850 2023-10-26T05:54:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83840c6df0 con 0x7f83840d0470 2023-10-26T05:54:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 0x7f838415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 0x7f838415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44690/0 (socket says 172.21.15.71:44690) 2023-10-26T05:54:31.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 -- 172.21.15.71:0/2323929540 learned_addr learned my addr 172.21.15.71:0/2323929540 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8388874640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83840d0470 0x7f83841589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389876640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f8384153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 -- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83840d0470 msgr2=0x7f83841589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83840d0470 0x7f83841589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 -- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 msgr2=0x7f8384153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f8384153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 -- 172.21.15.71:0/2323929540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740094a0 con 0x7f83840cd880 2023-10-26T05:54:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389876640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f8384153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8389075640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 0x7f838415c4d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f838000c780 tx=0x7f838000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380005270 con 0x7f83840cd880 2023-10-26T05:54:31.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8380003600 con 0x7f83840cd880 2023-10-26T05:54:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8384152af0 con 0x7f83840cd880 2023-10-26T05:54:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380003a10 con 0x7f83840cd880 2023-10-26T05:54:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.551+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83841530b0 con 0x7f83840cd880 2023-10-26T05:54:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.555+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8384066b10 con 0x7f83840cd880 2023-10-26T05:54:31.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.555+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8380003b70 con 0x7f83840cd880 2023-10-26T05:54:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.555+0000 7f8379ffb640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8368066470 0x7f8368068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.555+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8380083490 con 0x7f83840cd880 2023-10-26T05:54:31.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.555+0000 7f8388874640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8368066470 0x7f8368068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.555+0000 7f8388874640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8368066470 0x7f8368068930 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f83740af000 tx=0x7f837401aed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:31.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.559+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8380050c60 con 0x7f83840cd880 2023-10-26T05:54:31.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.711+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83840b5130 con 0x7f83840cd880 2023-10-26T05:54:31.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.711+0000 7f8379ffb640 1 -- 172.21.15.71:0/2323929540 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+12784 (secure 0 0 0) 0x7f8380017020 con 0x7f83840cd880 2023-10-26T05:54:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:31.720 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":151,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:31.045018+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:31.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8368066470 msgr2=0x7f8368068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8368066470 0x7f8368068930 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f83740af000 tx=0x7f837401aed0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 msgr2=0x7f838415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 0x7f838415c4d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f838000c780 tx=0x7f838000cc40 comp rx=0 tx=0).stop 2023-10-26T05:54:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 shutdown_connections 2023-10-26T05:54:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83840d0850 0x7f8384153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83840d0470 0x7f83841589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8368066470 0x7f8368068930 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 --2- 172.21.15.71:0/2323929540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83840cd880 0x7f838415c4d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 >> 172.21.15.71:0/2323929540 conn(0x7f83840cdd80 msgr2=0x7f838415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 shutdown_connections 2023-10-26T05:54:31.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.715+0000 7f838b2d9640 1 -- 172.21.15.71:0/2323929540 wait complete. 2023-10-26T05:54:31.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:54:31.738 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:54:31.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/4276556240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c10003980 msgr2=0x7f3c10003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/4276556240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c10003980 0x7f3c10003da0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009b90 tx=0x7f3c2001d260 comp rx=0 tx=0).stop 2023-10-26T05:54:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2a68f640 1 -- 172.21.15.71:0/4276556240 reap_dead start 2023-10-26T05:54:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/4276556240 shutdown_connections 2023-10-26T05:54:31.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/4276556240 >> 172.21.15.71:0/4276556240 conn(0x7f3c240aec60 msgr2=0x7f3c240af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/4276556240 shutdown_connections 2023-10-26T05:54:31.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/4276556240 wait complete. 2023-10-26T05:54:31.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 Processor -- start 2023-10-26T05:54:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- start start 2023-10-26T05:54:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10003980 0x7f3c24154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c24159710 0x7f3c24139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 0x7f3c24139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241553c0 con 0x7f3c24159b90 2023-10-26T05:54:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2415a0f0 con 0x7f3c10003980 2023-10-26T05:54:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.847+0000 7f3c2c0f2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2415a3d0 con 0x7f3c24159710 2023-10-26T05:54:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 0x7f3c24139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 0x7f3c24139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39654/0 (socket says 172.21.15.71:39654) 2023-10-26T05:54:31.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 -- 172.21.15.71:0/3313816332 learned_addr learned my addr 172.21.15.71:0/3313816332 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2968d640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c24159710 0x7f3c24139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c29e8e640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10003980 0x7f3c24154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 -- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c24159710 msgr2=0x7f3c24139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c24159710 0x7f3c24139120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 -- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10003980 msgr2=0x7f3c24154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10003980 0x7f3c24154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 -- 172.21.15.71:0/3313816332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2415a6b0 con 0x7f3c24159b90 2023-10-26T05:54:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2a68f640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 0x7f3c24139660 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f3c240b0e70 tx=0x7f3c1c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00dda0 con 0x7f3c24159b90 2023-10-26T05:54:31.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3c1c011440 con 0x7f3c24159b90 2023-10-26T05:54:31.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2415a990 con 0x7f3c24159b90 2023-10-26T05:54:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c011870 con 0x7f3c24159b90 2023-10-26T05:54:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2415b190 con 0x7f3c24159b90 2023-10-26T05:54:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c24066b10 con 0x7f3c24159b90 2023-10-26T05:54:31.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c1c0119d0 con 0x7f3c24159b90 2023-10-26T05:54:31.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c1a7fc640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c080664a0 0x7f3c08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:31.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c1c084390 con 0x7f3c24159b90 2023-10-26T05:54:31.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c29e8e640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c080664a0 0x7f3c08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:31.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.851+0000 7f3c29e8e640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c080664a0 0x7f3c08068960 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3c200ae000 tx=0x7f3c2001a000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:31.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.855+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3c1c051bf0 con 0x7f3c24159b90 2023-10-26T05:54:32.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.999+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c240721e0 con 0x7f3c24159b90 2023-10-26T05:54:32.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:31.999+0000 7f3c1a7fc640 1 -- 172.21.15.71:0/3313816332 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c1c055210 con 0x7f3c24159b90 2023-10-26T05:54:32.009 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:54:32.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c080664a0 msgr2=0x7f3c08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c080664a0 0x7f3c08068960 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3c200ae000 tx=0x7f3c2001a000 comp rx=0 tx=0).stop 2023-10-26T05:54:32.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 msgr2=0x7f3c24139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 0x7f3c24139660 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f3c240b0e70 tx=0x7f3c1c00fed0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 shutdown_connections 2023-10-26T05:54:32.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3c080664a0 0x7f3c08068960 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c24159b90 0x7f3c24139660 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c24159710 0x7f3c24139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 --2- 172.21.15.71:0/3313816332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10003980 0x7f3c24154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 >> 172.21.15.71:0/3313816332 conn(0x7f3c240aec60 msgr2=0x7f3c240ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:32.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 shutdown_connections 2023-10-26T05:54:32.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.003+0000 7f3c2c0f2640 1 -- 172.21.15.71:0/3313816332 wait complete. 2023-10-26T05:54:32.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:54:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/4188641610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 msgr2=0x7fe71815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 --2- 172.21.15.71:0/4188641610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe71815b3a0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe714009840 tx=0x7fe71401c770 comp rx=0 tx=0).stop 2023-10-26T05:54:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/4188641610 shutdown_connections 2023-10-26T05:54:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 --2- 172.21.15.71:0/4188641610 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7180cd880 0x7fe71815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 --2- 172.21.15.71:0/4188641610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe71815b3a0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/4188641610 >> 172.21.15.71:0/4188641610 conn(0x7fe7180cdd80 msgr2=0x7fe7180ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:32.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/4188641610 shutdown_connections 2023-10-26T05:54:32.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/4188641610 wait complete. 2023-10-26T05:54:32.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 Processor -- start 2023-10-26T05:54:32.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- start start 2023-10-26T05:54:32.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7180cd880 0x7fe718152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7180d0470 0x7fe718150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe718150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe718150f90 con 0x7fe7180d0850 2023-10-26T05:54:32.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71815ce80 con 0x7fe7180cd880 2023-10-26T05:54:32.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71815c270 con 0x7fe7180d0470 2023-10-26T05:54:32.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe718150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe718150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39668/0 (socket says 172.21.15.71:39668) 2023-10-26T05:54:32.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 -- 172.21.15.71:0/560818414 learned_addr learned my addr 172.21.15.71:0/560818414 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:32.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71d3cf640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7180d0470 0x7fe718150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 -- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7180d0470 msgr2=0x7fe718150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71dbd0640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7180cd880 0x7fe718152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7180d0470 0x7fe718150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 -- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7180cd880 msgr2=0x7fe718152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7180cd880 0x7fe718152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 -- 172.21.15.71:0/560818414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140094a0 con 0x7fe7180d0850 2023-10-26T05:54:32.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71e3d1640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe718150a50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe71000b9a0 tx=0x7fe71000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:32.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000cba0 con 0x7fe7180d0850 2023-10-26T05:54:32.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe71000ce10 con 0x7fe7180d0850 2023-10-26T05:54:32.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71815c550 con 0x7fe7180d0850 2023-10-26T05:54:32.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7100146a0 con 0x7fe7180d0850 2023-10-26T05:54:32.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.131+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71815cb10 con 0x7fe7180d0850 2023-10-26T05:54:32.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.135+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe718066b10 con 0x7fe7180d0850 2023-10-26T05:54:32.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.135+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe71001d3b0 con 0x7fe7180d0850 2023-10-26T05:54:32.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.135+0000 7fe70e7fc640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6fc0664a0 0x7fe6fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.135+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe710083d50 con 0x7fe7180d0850 2023-10-26T05:54:32.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.135+0000 7fe71dbd0640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6fc0664a0 0x7fe6fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.135+0000 7fe71dbd0640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6fc0664a0 0x7fe6fc068960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe7080046c0 tx=0x7fe708009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:32.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.139+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe710087030 con 0x7fe7180d0850 2023-10-26T05:54:32.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.283+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe7180b5150 con 0x7fe7180d0850 2023-10-26T05:54:32.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.283+0000 7fe70e7fc640 1 -- 172.21.15.71:0/560818414 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v151) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe710019020 con 0x7fe7180d0850 2023-10-26T05:54:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:32.294 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:54:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:54:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:54:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:54:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:32.295 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:54:32.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6fc0664a0 msgr2=0x7fe6fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6fc0664a0 0x7fe6fc068960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe7080046c0 tx=0x7fe708009040 comp rx=0 tx=0).stop 2023-10-26T05:54:32.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 msgr2=0x7fe718150a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.296 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe718150a50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe71000b9a0 tx=0x7fe71000be60 comp rx=0 tx=0).stop 2023-10-26T05:54:32.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 shutdown_connections 2023-10-26T05:54:32.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe6fc0664a0 0x7fe6fc068960 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe7180d0850 0x7fe718150a50 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7180d0470 0x7fe718150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 --2- 172.21.15.71:0/560818414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7180cd880 0x7fe718152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 >> 172.21.15.71:0/560818414 conn(0x7fe7180cdd80 msgr2=0x7fe71815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:32.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 shutdown_connections 2023-10-26T05:54:32.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.287+0000 7fe71fe34640 1 -- 172.21.15.71:0/560818414 wait complete. 2023-10-26T05:54:32.310 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:54:32.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:54:32.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe3218b1640 1 -- 172.21.15.71:0/2910204974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3080058e0 msgr2=0x7fe308005d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe3218b1640 1 --2- 172.21.15.71:0/2910204974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3080058e0 0x7fe308005d60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0cc950 tx=0x7fe30c01cc40 comp rx=0 tx=0).stop 2023-10-26T05:54:32.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe31b7fe640 1 -- 172.21.15.71:0/2910204974 reap_dead start 2023-10-26T05:54:32.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe3218b1640 1 -- 172.21.15.71:0/2910204974 shutdown_connections 2023-10-26T05:54:32.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe3218b1640 1 -- 172.21.15.71:0/2910204974 >> 172.21.15.71:0/2910204974 conn(0x7fe31c0aec60 msgr2=0x7fe31c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:32.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe3218b1640 1 -- 172.21.15.71:0/2910204974 shutdown_connections 2023-10-26T05:54:32.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.427+0000 7fe3218b1640 1 -- 172.21.15.71:0/2910204974 wait complete. 2023-10-26T05:54:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 Processor -- start 2023-10-26T05:54:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- start start 2023-10-26T05:54:32.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3080058e0 0x7fe31c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 0x7fe31c13afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe31c13b510 0x7fe31c0ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c13ea30 con 0x7fe31c13e470 2023-10-26T05:54:32.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1f3860 con 0x7fe3080058e0 2023-10-26T05:54:32.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c141880 con 0x7fe31c13b510 2023-10-26T05:54:32.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 0x7fe31c13afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 0x7fe31c13afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39678/0 (socket says 172.21.15.71:39678) 2023-10-26T05:54:32.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 -- 172.21.15.71:0/3744397687 learned_addr learned my addr 172.21.15.71:0/3744397687 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:32.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31affd640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3080058e0 0x7fe31c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 -- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe31c13b510 msgr2=0x7fe31c0ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe31c13b510 0x7fe31c0ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 -- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3080058e0 msgr2=0x7fe31c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3080058e0 0x7fe31c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 -- 172.21.15.71:0/3744397687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c15a9d0 con 0x7fe31c13e470 2023-10-26T05:54:32.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe31a7fc640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 0x7fe31c13afd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0b1040 tx=0x7fe30c0a5f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c0bd070 con 0x7fe31c13e470 2023-10-26T05:54:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe30c000e90 con 0x7fe31c13e470 2023-10-26T05:54:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c1f3cf0 con 0x7fe31c13e470 2023-10-26T05:54:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c005610 con 0x7fe31c13e470 2023-10-26T05:54:32.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1f3210 con 0x7fe31c13e470 2023-10-26T05:54:32.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c066b10 con 0x7fe31c13e470 2023-10-26T05:54:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe30c0046d0 con 0x7fe31c13e470 2023-10-26T05:54:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.431+0000 7fe313fff640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2fc066470 0x7fe2fc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.435+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe30c11edf0 con 0x7fe31c13e470 2023-10-26T05:54:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.435+0000 7fe31affd640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2fc066470 0x7fe2fc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:32.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.435+0000 7fe31affd640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2fc066470 0x7fe2fc068930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe3040065b0 tx=0x7fe304006af0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:32.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.435+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe30c0ec730 con 0x7fe31c13e470 2023-10-26T05:54:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:32.587+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe31c1f39f0 con 0x7fe31c13e470 2023-10-26T05:54:33.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.191+0000 7fe313fff640 1 -- 172.21.15.71:0/3744397687 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v152) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe30c0ab020 con 0x7fe31c13e470 2023-10-26T05:54:33.198 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:54:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2fc066470 msgr2=0x7fe2fc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2fc066470 0x7fe2fc068930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe3040065b0 tx=0x7fe304006af0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 msgr2=0x7fe31c13afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 0x7fe31c13afd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0b1040 tx=0x7fe30c0a5f70 comp rx=0 tx=0).stop 2023-10-26T05:54:33.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 shutdown_connections 2023-10-26T05:54:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe31c13b510 0x7fe31c0ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe2fc066470 0x7fe2fc068930 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe31c13e470 0x7fe31c13afd0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 --2- 172.21.15.71:0/3744397687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3080058e0 0x7fe31c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 >> 172.21.15.71:0/3744397687 conn(0x7fe31c0aec60 msgr2=0x7fe31c0ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:33.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 shutdown_connections 2023-10-26T05:54:33.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.195+0000 7fe3218b1640 1 -- 172.21.15.71:0/3744397687 wait complete. 2023-10-26T05:54:33.226 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:54:33.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/4144524515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900d0470 msgr2=0x7f1a9006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:33.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 --2- 172.21.15.71:0/4144524515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900d0470 0x7f1a9006aa70 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f1a900c4dd0 tx=0x7f1a8001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/4144524515 shutdown_connections 2023-10-26T05:54:33.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 --2- 172.21.15.71:0/4144524515 >> v2:172.21.15.86:3300/0 conn(0x7f1a900d0850 0x7f1a9015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 --2- 172.21.15.71:0/4144524515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900d0470 0x7f1a9006aa70 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/4144524515 >> 172.21.15.71:0/4144524515 conn(0x7f1a900cdd80 msgr2=0x7f1a900ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/4144524515 shutdown_connections 2023-10-26T05:54:33.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/4144524515 wait complete. 2023-10-26T05:54:33.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 Processor -- start 2023-10-26T05:54:33.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- start start 2023-10-26T05:54:33.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 0x7f1a90159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:33.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a900d0470 0x7f1a9015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:33.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d0850 0x7f1a9015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:33.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a90159f20 con 0x7f1a900cd880 2023-10-26T05:54:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9015a200 con 0x7f1a900d0850 2023-10-26T05:54:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a9015a4e0 con 0x7f1a900d0470 2023-10-26T05:54:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 0x7f1a90159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 0x7f1a90159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39694/0 (socket says 172.21.15.71:39694) 2023-10-26T05:54:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 -- 172.21.15.71:0/2389057402 learned_addr learned my addr 172.21.15.71:0/2389057402 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:33.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8effd640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a900d0470 0x7f1a9015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8ffff640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d0850 0x7f1a9015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 -- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a900d0470 msgr2=0x7f1a9015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a900d0470 0x7f1a9015c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 -- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d0850 msgr2=0x7f1a9015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:33.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d0850 0x7f1a9015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:33.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 -- 172.21.15.71:0/2389057402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80009560 con 0x7f1a900cd880 2023-10-26T05:54:33.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a8f7fe640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 0x7f1a90159460 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f1a800b0040 tx=0x7f1a800a5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:33.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80005230 con 0x7f1a900cd880 2023-10-26T05:54:33.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1a800054a0 con 0x7f1a900cd880 2023-10-26T05:54:33.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a9015a7c0 con 0x7f1a900cd880 2023-10-26T05:54:33.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004930 con 0x7f1a900cd880 2023-10-26T05:54:33.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.351+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a90154e90 con 0x7f1a900cd880 2023-10-26T05:54:33.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.355+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a90066b10 con 0x7f1a900cd880 2023-10-26T05:54:33.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.355+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1a800b4070 con 0x7f1a900cd880 2023-10-26T05:54:33.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.355+0000 7f1a77fff640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a7c066470 0x7f1a7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:33.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.355+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1a8011e470 con 0x7f1a900cd880 2023-10-26T05:54:33.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.355+0000 7f1a8effd640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a7c066470 0x7f1a7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:33.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.355+0000 7f1a8effd640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a7c066470 0x7f1a7c068930 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1a900c7960 tx=0x7f1a8400a250 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:33.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.359+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1a800ebb70 con 0x7f1a900cd880 2023-10-26T05:54:33.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:33.503+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a90066b10 con 0x7f1a900cd880 2023-10-26T05:54:34.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.203+0000 7f1a77fff640 1 -- 172.21.15.71:0/2389057402 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v153) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a800ef190 con 0x7f1a900cd880 2023-10-26T05:54:34.212 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:54:34.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a7c066470 msgr2=0x7f1a7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:34.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a7c066470 0x7f1a7c068930 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1a900c7960 tx=0x7f1a8400a250 comp rx=0 tx=0).stop 2023-10-26T05:54:34.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 msgr2=0x7f1a90159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:34.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 0x7f1a90159460 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f1a800b0040 tx=0x7f1a800a5840 comp rx=0 tx=0).stop 2023-10-26T05:54:34.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 shutdown_connections 2023-10-26T05:54:34.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a900d0850 0x7f1a9015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1a900d0470 0x7f1a9015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1a7c066470 0x7f1a7c068930 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 --2- 172.21.15.71:0/2389057402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a900cd880 0x7f1a90159460 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 >> 172.21.15.71:0/2389057402 conn(0x7f1a900cdd80 msgr2=0x7f1a9015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:34.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 shutdown_connections 2023-10-26T05:54:34.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.207+0000 7f1a96153640 1 -- 172.21.15.71:0/2389057402 wait complete. 2023-10-26T05:54:34.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:54:34.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.359+0000 7f83b5898640 1 -- 172.21.15.71:0/2983646982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b013c220 msgr2=0x7f83b015cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:34.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.359+0000 7f83b5898640 1 --2- 172.21.15.71:0/2983646982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b013c220 0x7f83b015cce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83a0009900 tx=0x7f83a001c800 comp rx=0 tx=0).stop 2023-10-26T05:54:34.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- 172.21.15.71:0/2983646982 shutdown_connections 2023-10-26T05:54:34.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 --2- 172.21.15.71:0/2983646982 >> v2:172.21.15.86:3300/0 conn(0x7f83b015d220 0x7f83b00cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 --2- 172.21.15.71:0/2983646982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b013c220 0x7f83b015cce0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- 172.21.15.71:0/2983646982 >> 172.21.15.71:0/2983646982 conn(0x7f83b00aec60 msgr2=0x7f83b00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:34.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- 172.21.15.71:0/2983646982 shutdown_connections 2023-10-26T05:54:34.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- 172.21.15.71:0/2983646982 wait complete. 2023-10-26T05:54:34.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 Processor -- start 2023-10-26T05:54:34.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- start start 2023-10-26T05:54:34.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b00cbd60 0x7f83b01f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:34.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b013c220 0x7f83b0158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:34.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 0x7f83b0158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:34.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b01f83b0 con 0x7f83b015d220 2023-10-26T05:54:34.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b014bd00 con 0x7f83b013c220 2023-10-26T05:54:34.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b014e100 con 0x7f83b00cbd60 2023-10-26T05:54:34.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 0x7f83b0158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:34.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 0x7f83b0158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39696/0 (socket says 172.21.15.71:39696) 2023-10-26T05:54:34.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 -- 172.21.15.71:0/3900407281 learned_addr learned my addr 172.21.15.71:0/3900407281 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:34.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83ae7fc640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b013c220 0x7f83b0158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:34.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83aeffd640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b00cbd60 0x7f83b01f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:34.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 -- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b00cbd60 msgr2=0x7f83b01f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b00cbd60 0x7f83b01f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 -- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b013c220 msgr2=0x7f83b0158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b013c220 0x7f83b0158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 -- 172.21.15.71:0/3900407281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a0009560 con 0x7f83b015d220 2023-10-26T05:54:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83af7fe640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 0x7f83b0158670 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c770 tx=0x7f83a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:34.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8003b80 con 0x7f83b015d220 2023-10-26T05:54:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f83a8003ce0 con 0x7f83b015d220 2023-10-26T05:54:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b014e3e0 con 0x7f83b015d220 2023-10-26T05:54:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800aab0 con 0x7f83b015d220 2023-10-26T05:54:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.363+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b014c2a0 con 0x7f83b015d220 2023-10-26T05:54:34.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.367+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b0066b10 con 0x7f83b015d220 2023-10-26T05:54:34.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.367+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f83a800ac10 con 0x7f83b015d220 2023-10-26T05:54:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.367+0000 7f83a7fff640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8398066470 0x7f8398068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.367+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f83a8084720 con 0x7f83b015d220 2023-10-26T05:54:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.367+0000 7f83aeffd640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8398066470 0x7f8398068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.367+0000 7f83aeffd640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8398066470 0x7f8398068930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f83a00af000 tx=0x7f83a001b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:34.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.371+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f83a8051d90 con 0x7f83b015d220 2023-10-26T05:54:34.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:34.543+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f83b00721e0 con 0x7f83b015d220 2023-10-26T05:54:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.207+0000 7f83a7fff640 1 -- 172.21.15.71:0/3900407281 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v56) v1 ==== 174+0+0 (secure 0 0 0) 0x7f83a80553b0 con 0x7f83b015d220 2023-10-26T05:54:35.216 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 18 and data pool 19 2023-10-26T05:54:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8398066470 msgr2=0x7f8398068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8398066470 0x7f8398068930 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f83a00af000 tx=0x7f83a001b040 comp rx=0 tx=0).stop 2023-10-26T05:54:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 msgr2=0x7f83b0158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 0x7f83b0158670 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c770 tx=0x7f83a800cc30 comp rx=0 tx=0).stop 2023-10-26T05:54:35.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 shutdown_connections 2023-10-26T05:54:35.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8398066470 0x7f8398068930 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83b015d220 0x7f83b0158670 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83b013c220 0x7f83b0158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 --2- 172.21.15.71:0/3900407281 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83b00cbd60 0x7f83b01f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 >> 172.21.15.71:0/3900407281 conn(0x7f83b00aec60 msgr2=0x7f83b00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:35.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 shutdown_connections 2023-10-26T05:54:35.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.211+0000 7f83b5898640 1 -- 172.21.15.71:0/3900407281 wait complete. 2023-10-26T05:54:35.249 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:35.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 -- 172.21.15.71:0/1309823010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 msgr2=0x7efd400b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 --2- 172.21.15.71:0/1309823010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd400b1ff0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7efd400cfc30 tx=0x7efd2801c7a0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 -- 172.21.15.71:0/1309823010 shutdown_connections 2023-10-26T05:54:35.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 --2- 172.21.15.71:0/1309823010 >> v2:172.21.15.86:3300/0 conn(0x7efd400cc220 0x7efd400cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 --2- 172.21.15.71:0/1309823010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd400b1ff0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 -- 172.21.15.71:0/1309823010 >> 172.21.15.71:0/1309823010 conn(0x7efd4006f440 msgr2=0x7efd4006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:35.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 -- 172.21.15.71:0/1309823010 shutdown_connections 2023-10-26T05:54:35.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.363+0000 7efd44f78640 1 -- 172.21.15.71:0/1309823010 wait complete. 2023-10-26T05:54:35.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 Processor -- start 2023-10-26T05:54:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- start start 2023-10-26T05:54:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd400af990 0x7efd40137420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd400cc220 0x7efd4014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd401386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd40137960 con 0x7efd40142b30 2023-10-26T05:54:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd40150280 con 0x7efd400af990 2023-10-26T05:54:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4014fd00 con 0x7efd400cc220 2023-10-26T05:54:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd401386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd401386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39716/0 (socket says 172.21.15.71:39716) 2023-10-26T05:54:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 -- 172.21.15.71:0/2100899549 learned_addr learned my addr 172.21.15.71:0/2100899549 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 -- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd400cc220 msgr2=0x7efd4014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3dd9b640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd400cc220 0x7efd4014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd400cc220 0x7efd4014f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3e59c640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd400af990 0x7efd40137420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 -- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd400af990 msgr2=0x7efd40137420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd400af990 0x7efd40137420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 -- 172.21.15.71:0/2100899549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28009560 con 0x7efd40142b30 2023-10-26T05:54:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3dd9b640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd400cc220 0x7efd4014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd3ed9d640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd401386e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7efd3000b970 tx=0x7efd3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:35.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000cb40 con 0x7efd40142b30 2023-10-26T05:54:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd3000cdb0 con 0x7efd40142b30 2023-10-26T05:54:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4014ffa0 con 0x7efd40142b30 2023-10-26T05:54:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300146a0 con 0x7efd40142b30 2023-10-26T05:54:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd40137ee0 con 0x7efd40142b30 2023-10-26T05:54:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd40066b10 con 0x7efd40142b30 2023-10-26T05:54:35.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd3001d3b0 con 0x7efd40142b30 2023-10-26T05:54:35.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.367+0000 7efd36ffd640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd20066470 0x7efd20068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.371+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd30083ff0 con 0x7efd40142b30 2023-10-26T05:54:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.371+0000 7efd3e59c640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd20066470 0x7efd20068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.371+0000 7efd3e59c640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd20066470 0x7efd20068930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efd280af000 tx=0x7efd2801afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:35.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.375+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd30051500 con 0x7efd40142b30 2023-10-26T05:54:35.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd400721e0 con 0x7efd40142b30 2023-10-26T05:54:35.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd36ffd640 1 -- 172.21.15.71:0/2100899549 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16509 (secure 0 0 0) 0x7efd30019090 con 0x7efd40142b30 2023-10-26T05:54:35.527 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:35.528 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":154,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:34.549121+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:35.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd20066470 msgr2=0x7efd20068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd20066470 0x7efd20068930 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efd280af000 tx=0x7efd2801afe0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 msgr2=0x7efd401386e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd401386e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7efd3000b970 tx=0x7efd3000be30 comp rx=0 tx=0).stop 2023-10-26T05:54:35.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 shutdown_connections 2023-10-26T05:54:35.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd20066470 0x7efd20068930 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40142b30 0x7efd401386e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd400cc220 0x7efd4014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 --2- 172.21.15.71:0/2100899549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd400af990 0x7efd40137420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.519+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 >> 172.21.15.71:0/2100899549 conn(0x7efd4006f440 msgr2=0x7efd40149350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:35.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.523+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 shutdown_connections 2023-10-26T05:54:35.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.523+0000 7efd44f78640 1 -- 172.21.15.71:0/2100899549 wait complete. 2023-10-26T05:54:35.544 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:35.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 -- 172.21.15.71:0/3326981952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340d01d0 msgr2=0x7f32340d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 --2- 172.21.15.71:0/3326981952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340d01d0 0x7f32340d05e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f32340c8670 tx=0x7f322401c7b0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 -- 172.21.15.71:0/3326981952 shutdown_connections 2023-10-26T05:54:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 --2- 172.21.15.71:0/3326981952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340d01d0 0x7f32340d05e0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 --2- 172.21.15.71:0/3326981952 >> v2:172.21.15.86:3300/0 conn(0x7f323415b5b0 0x7f32340b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 -- 172.21.15.71:0/3326981952 >> 172.21.15.71:0/3326981952 conn(0x7f32340ce070 msgr2=0x7f32340ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:35.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 -- 172.21.15.71:0/3326981952 shutdown_connections 2023-10-26T05:54:35.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 -- 172.21.15.71:0/3326981952 wait complete. 2023-10-26T05:54:35.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 Processor -- start 2023-10-26T05:54:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.659+0000 7f323a59f640 1 -- start start 2023-10-26T05:54:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32340d01d0 0x7f32340c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f323415b5b0 0x7f3234150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 0x7f32340ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32340c9780 con 0x7f32340ca140 2023-10-26T05:54:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32340c8db0 con 0x7f32340d01d0 2023-10-26T05:54:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32340cac00 con 0x7f323415b5b0 2023-10-26T05:54:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 0x7f32340ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 0x7f32340ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39742/0 (socket says 172.21.15.71:39742) 2023-10-26T05:54:35.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 -- 172.21.15.71:0/1478189060 learned_addr learned my addr 172.21.15.71:0/1478189060 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:35.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3233fff640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32340d01d0 0x7f32340c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 -- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f323415b5b0 msgr2=0x7f3234150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f32337fe640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f323415b5b0 0x7f3234150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f323415b5b0 0x7f3234150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 -- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32340d01d0 msgr2=0x7f32340c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32340d01d0 0x7f32340c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 -- 172.21.15.71:0/1478189060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240094a0 con 0x7f32340ca140 2023-10-26T05:54:35.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3233fff640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32340d01d0 0x7f32340c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:35.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3238b3c640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 0x7f32340ca5d0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f3224009790 tx=0x7f32240a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:35.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224005230 con 0x7f32340ca140 2023-10-26T05:54:35.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3224005480 con 0x7f32340ca140 2023-10-26T05:54:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224004930 con 0x7f32340ca140 2023-10-26T05:54:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32340c8760 con 0x7f32340ca140 2023-10-26T05:54:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323414bd20 con 0x7f32340ca140 2023-10-26T05:54:35.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32340c8f20 con 0x7f32340ca140 2023-10-26T05:54:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f32240b4040 con 0x7f32340ca140 2023-10-26T05:54:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3230ff9640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3214066470 0x7f3214068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f322411dea0 con 0x7f32340ca140 2023-10-26T05:54:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.663+0000 7f3233fff640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3214066470 0x7f3214068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.667+0000 7f3233fff640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3214066470 0x7f3214068930 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f322c009d90 tx=0x7f322c009320 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:35.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.667+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f32240eb330 con 0x7f32340ca140 2023-10-26T05:54:35.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.823+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3234066d00 con 0x7f32340ca140 2023-10-26T05:54:35.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f3230ff9640 1 -- 172.21.15.71:0/1478189060 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+16509 (secure 0 0 0) 0x7f32240ee950 con 0x7f32340ca140 2023-10-26T05:54:35.834 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:35.834 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":154,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:34.549121+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3214066470 msgr2=0x7f3214068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3214066470 0x7f3214068930 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f322c009d90 tx=0x7f322c009320 comp rx=0 tx=0).stop 2023-10-26T05:54:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 msgr2=0x7f32340ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 0x7f32340ca5d0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f3224009790 tx=0x7f32240a57d0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 shutdown_connections 2023-10-26T05:54:35.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3214066470 0x7f3214068930 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32340ca140 0x7f32340ca5d0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f323415b5b0 0x7f3234150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 --2- 172.21.15.71:0/1478189060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32340d01d0 0x7f32340c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 >> 172.21.15.71:0/1478189060 conn(0x7f32340ce070 msgr2=0x7f32340c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:35.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 shutdown_connections 2023-10-26T05:54:35.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.827+0000 7f323a59f640 1 -- 172.21.15.71:0/1478189060 wait complete. 2023-10-26T05:54:35.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:54:35.981 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 -- 172.21.15.71:0/3714436341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 msgr2=0x7f50740699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 --2- 172.21.15.71:0/3714436341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f50740699f0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f5070009900 tx=0x7f507001c7c0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 -- 172.21.15.71:0/3714436341 shutdown_connections 2023-10-26T05:54:35.982 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 --2- 172.21.15.71:0/3714436341 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f507415f100 0x7f507415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 --2- 172.21.15.71:0/3714436341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f50740699f0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 -- 172.21.15.71:0/3714436341 >> 172.21.15.71:0/3714436341 conn(0x7f50740d1440 msgr2=0x7f50740d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:35.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 -- 172.21.15.71:0/3714436341 shutdown_connections 2023-10-26T05:54:35.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.971+0000 7f507c74e640 1 -- 172.21.15.71:0/3714436341 wait complete. 2023-10-26T05:54:35.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 Processor -- start 2023-10-26T05:54:35.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- start start 2023-10-26T05:54:35.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f5074154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5074152000 0x7f50740c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f507415f100 0x7f507414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5074154760 con 0x7f50740cc810 2023-10-26T05:54:35.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c7f00 con 0x7f507415f100 2023-10-26T05:54:35.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50740c81e0 con 0x7f5074152000 2023-10-26T05:54:35.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f5074154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f5074154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39752/0 (socket says 172.21.15.71:39752) 2023-10-26T05:54:35.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 -- 172.21.15.71:0/447890974 learned_addr learned my addr 172.21.15.71:0/447890974 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:35.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 -- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5074152000 msgr2=0x7f50740c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5074152000 0x7f50740c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 -- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f507415f100 msgr2=0x7f507414df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f507415f100 0x7f507414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 -- 172.21.15.71:0/447890974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5070009560 con 0x7f50740cc810 2023-10-26T05:54:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507a4ea640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f5074154bf0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5070009850 tx=0x7f50700a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070005230 con 0x7f50740cc810 2023-10-26T05:54:35.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5070005480 con 0x7f50740cc810 2023-10-26T05:54:35.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5074154110 con 0x7f50740cc810 2023-10-26T05:54:35.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070004930 con 0x7f50740cc810 2023-10-26T05:54:35.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50740c4b40 con 0x7f50740cc810 2023-10-26T05:54:35.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.975+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5074066b10 con 0x7f50740cc810 2023-10-26T05:54:35.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.979+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f50700b4070 con 0x7f50740cc810 2023-10-26T05:54:35.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.979+0000 7f506affd640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5058066470 0x7f5058068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:35.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.979+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f507011e710 con 0x7f50740cc810 2023-10-26T05:54:35.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.979+0000 7f5079ce9640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5058066470 0x7f5058068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:35.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.979+0000 7f5079ce9640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5058066470 0x7f5058068930 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5074167470 tx=0x7f5064005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:35.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:35.983+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f50700ebc20 con 0x7f50740cc810 2023-10-26T05:54:36.177 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.167+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f50740721e0 con 0x7f50740cc810 2023-10-26T05:54:36.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.259+0000 7f506affd640 1 -- 172.21.15.71:0/447890974 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7f50700ef240 con 0x7f50740cc810 2023-10-26T05:54:36.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.259+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5058066470 msgr2=0x7f5058068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:36.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.259+0000 7f507c74e640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5058066470 0x7f5058068930 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5074167470 tx=0x7f5064005b60 comp rx=0 tx=0).stop 2023-10-26T05:54:36.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.259+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 msgr2=0x7f5074154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:36.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.259+0000 7f507c74e640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f5074154bf0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5070009850 tx=0x7f50700a5750 comp rx=0 tx=0).stop 2023-10-26T05:54:36.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 shutdown_connections 2023-10-26T05:54:36.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f507415f100 0x7f507414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5074152000 0x7f50740c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5058066470 0x7f5058068930 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 --2- 172.21.15.71:0/447890974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50740cc810 0x7f5074154bf0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 >> 172.21.15.71:0/447890974 conn(0x7f50740d1440 msgr2=0x7f50740b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:36.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 shutdown_connections 2023-10-26T05:54:36.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.263+0000 7f507c74e640 1 -- 172.21.15.71:0/447890974 wait complete. 2023-10-26T05:54:36.289 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:54:36.290 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:54:36.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- 172.21.15.71:0/913849408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb4141a10 msgr2=0x7f3cb4070930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:36.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/913849408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb4141a10 0x7f3cb4070930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009840 tx=0x7f3ca401c7d0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- 172.21.15.71:0/913849408 shutdown_connections 2023-10-26T05:54:36.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/913849408 >> v2:172.21.15.86:3300/0 conn(0x7f3cb4070e70 0x7f3cb40b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/913849408 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb4141a10 0x7f3cb4070930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- 172.21.15.71:0/913849408 >> 172.21.15.71:0/913849408 conn(0x7f3cb40aec60 msgr2=0x7f3cb40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:36.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- 172.21.15.71:0/913849408 shutdown_connections 2023-10-26T05:54:36.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- 172.21.15.71:0/913849408 wait complete. 2023-10-26T05:54:36.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 Processor -- start 2023-10-26T05:54:36.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- start start 2023-10-26T05:54:36.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb4070e70 0x7f3cb413c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:36.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb40b11c0 0x7f3cb41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:36.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 0x7f3cb406a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:36.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb413c810 con 0x7f3cb4141a10 2023-10-26T05:54:36.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4159260 con 0x7f3cb4070e70 2023-10-26T05:54:36.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cba0fd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4159540 con 0x7f3cb40b11c0 2023-10-26T05:54:36.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 0x7f3cb406a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:36.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 0x7f3cb406a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39768/0 (socket says 172.21.15.71:39768) 2023-10-26T05:54:36.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 -- 172.21.15.71:0/3846370566 learned_addr learned my addr 172.21.15.71:0/3846370566 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:36.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb37fe640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb4070e70 0x7f3cb413c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:36.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb2ffd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb40b11c0 0x7f3cb41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:36.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 -- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb40b11c0 msgr2=0x7f3cb41570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb40b11c0 0x7f3cb41570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 -- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb4070e70 msgr2=0x7f3cb413c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb4070e70 0x7f3cb413c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.411+0000 7f3cb3fff640 1 -- 172.21.15.71:0/3846370566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca40094a0 con 0x7f3cb4141a10 2023-10-26T05:54:36.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3cb3fff640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 0x7f3cb406a510 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00cb10 tx=0x7f3cac00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00aa50 con 0x7f3cb4141a10 2023-10-26T05:54:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4159820 con 0x7f3cb4141a10 2023-10-26T05:54:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cac00aca0 con 0x7f3cb4141a10 2023-10-26T05:54:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac012680 con 0x7f3cb4141a10 2023-10-26T05:54:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb41575f0 con 0x7f3cb4141a10 2023-10-26T05:54:36.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4066b10 con 0x7f3cb4141a10 2023-10-26T05:54:36.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3cac01a3b0 con 0x7f3cb4141a10 2023-10-26T05:54:36.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3c9bfff640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ca0066470 0x7f3ca0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cac084320 con 0x7f3cb4141a10 2023-10-26T05:54:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3cb37fe640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ca0066470 0x7f3ca0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.415+0000 7f3cb37fe640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ca0066470 0x7f3ca0068930 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40af000 tx=0x7f3ca401b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:36.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.419+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3cac051720 con 0x7f3cb4141a10 2023-10-26T05:54:36.612 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:36.603+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3cb4159d40 con 0x7f3cb4141a10 2023-10-26T05:54:37.036 INFO:tasks.ceph.mds.c.smithi071.stderr:2023-10-26T05:54:37.031+0000 7fea3f163640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:54:37.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3c9bfff640 1 -- 172.21.15.71:0/3846370566 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3cb4159d40 con 0x7f3cb4141a10 2023-10-26T05:54:37.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ca0066470 msgr2=0x7f3ca0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ca0066470 0x7f3ca0068930 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40af000 tx=0x7f3ca401b010 comp rx=0 tx=0).stop 2023-10-26T05:54:37.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 msgr2=0x7f3cb406a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 0x7f3cb406a510 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00cb10 tx=0x7f3cac00cc00 comp rx=0 tx=0).stop 2023-10-26T05:54:37.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 shutdown_connections 2023-10-26T05:54:37.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ca0066470 0x7f3ca0068930 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cb4141a10 0x7f3cb406a510 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cb40b11c0 0x7f3cb41570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 --2- 172.21.15.71:0/3846370566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cb4070e70 0x7f3cb413c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 >> 172.21.15.71:0/3846370566 conn(0x7f3cb40aec60 msgr2=0x7f3cb40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 shutdown_connections 2023-10-26T05:54:37.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.271+0000 7f3cba0fd640 1 -- 172.21.15.71:0/3846370566 wait complete. 2023-10-26T05:54:37.302 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:54:37.302 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:37.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- 172.21.15.71:0/3158480589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d74005190 msgr2=0x7f8d7400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3158480589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d74005190 0x7f8d7400b990 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8d84009e30 tx=0x7f8d8401d190 comp rx=0 tx=0).stop 2023-10-26T05:54:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d8ad9d640 1 -- 172.21.15.71:0/3158480589 reap_dead start 2023-10-26T05:54:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- 172.21.15.71:0/3158480589 shutdown_connections 2023-10-26T05:54:37.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- 172.21.15.71:0/3158480589 >> 172.21.15.71:0/3158480589 conn(0x7f8d8c0cdd80 msgr2=0x7f8d8c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- 172.21.15.71:0/3158480589 shutdown_connections 2023-10-26T05:54:37.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- 172.21.15.71:0/3158480589 wait complete. 2023-10-26T05:54:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 Processor -- start 2023-10-26T05:54:37.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- start start 2023-10-26T05:54:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0c7ba0 0x7f8d8c0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 0x7f8d8c150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d8c151480 0x7f8d8c153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0b1b00 con 0x7f8d8c0b1680 2023-10-26T05:54:37.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c160660 con 0x7f8d8c0c7ba0 2023-10-26T05:54:37.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d90b24640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0c7f80 con 0x7f8d8c151480 2023-10-26T05:54:37.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.427+0000 7f8d89d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 0x7f8d8c150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 0x7f8d8c150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39784/0 (socket says 172.21.15.71:39784) 2023-10-26T05:54:37.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 -- 172.21.15.71:0/3407826077 learned_addr learned my addr 172.21.15.71:0/3407826077 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:37.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d8a59c640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0c7ba0 0x7f8d8c0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d8ad9d640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d8c151480 0x7f8d8c153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 -- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d8c151480 msgr2=0x7f8d8c153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d8c151480 0x7f8d8c153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 -- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0c7ba0 msgr2=0x7f8d8c0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0c7ba0 0x7f8d8c0b1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 -- 172.21.15.71:0/3407826077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c15fd30 con 0x7f8d8c0b1680 2023-10-26T05:54:37.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d89d9b640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 0x7f8d8c150f40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000f9b0 tx=0x7f8d8000fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:37.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000bc50 con 0x7f8d8c0b1680 2023-10-26T05:54:37.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d8000bea0 con 0x7f8d8c0b1680 2023-10-26T05:54:37.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c160010 con 0x7f8d8c0b1680 2023-10-26T05:54:37.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800116a0 con 0x7f8d8c0b1680 2023-10-26T05:54:37.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c151900 con 0x7f8d8c0b1680 2023-10-26T05:54:37.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c066b10 con 0x7f8d8c0b1680 2023-10-26T05:54:37.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d80011800 con 0x7f8d8c0b1680 2023-10-26T05:54:37.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d7affd640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d6c0664a0 0x7f8d6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d80084a80 con 0x7f8d8c0b1680 2023-10-26T05:54:37.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d8a59c640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d6c0664a0 0x7f8d6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.431+0000 7f8d8a59c640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d6c0664a0 0x7f8d6c068960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c001230 tx=0x7f8d7c001140 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:37.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.435+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d80051f90 con 0x7f8d8c0b1680 2023-10-26T05:54:37.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.623+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d8c0721e0 con 0x7f8d8c0b1680 2023-10-26T05:54:37.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.623+0000 7f8d7affd640 1 -- 172.21.15.71:0/3407826077 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f8d80020020 con 0x7f8d8c0b1680 2023-10-26T05:54:37.633 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:37.633 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:54:37.633 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:54:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d6c0664a0 msgr2=0x7f8d6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d6c0664a0 0x7f8d6c068960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c001230 tx=0x7f8d7c001140 comp rx=0 tx=0).stop 2023-10-26T05:54:37.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 msgr2=0x7f8d8c150f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 0x7f8d8c150f40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000f9b0 tx=0x7f8d8000fe70 comp rx=0 tx=0).stop 2023-10-26T05:54:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 shutdown_connections 2023-10-26T05:54:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d8c151480 0x7f8d8c153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d6c0664a0 0x7f8d6c068960 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8c0b1680 0x7f8d8c150f40 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 --2- 172.21.15.71:0/3407826077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d8c0c7ba0 0x7f8d8c0b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 >> 172.21.15.71:0/3407826077 conn(0x7f8d8c0cdd80 msgr2=0x7f8d8c156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:37.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 shutdown_connections 2023-10-26T05:54:37.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.627+0000 7f8d90b24640 1 -- 172.21.15.71:0/3407826077 wait complete. 2023-10-26T05:54:37.652 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:54:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 -- 172.21.15.71:0/1267412764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 msgr2=0x7f114006aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 --2- 172.21.15.71:0/1267412764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f114006aa20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f114006e350 tx=0x7f113401c7b0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 -- 172.21.15.71:0/1267412764 shutdown_connections 2023-10-26T05:54:37.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 --2- 172.21.15.71:0/1267412764 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11400b2f00 0x7f11400ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 --2- 172.21.15.71:0/1267412764 >> v2:172.21.15.86:3300/0 conn(0x7f11400afa20 0x7f11400c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 --2- 172.21.15.71:0/1267412764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f114006aa20 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 -- 172.21.15.71:0/1267412764 >> 172.21.15.71:0/1267412764 conn(0x7f11400aef00 msgr2=0x7f11400af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 -- 172.21.15.71:0/1267412764 shutdown_connections 2023-10-26T05:54:37.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 -- 172.21.15.71:0/1267412764 wait complete. 2023-10-26T05:54:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 Processor -- start 2023-10-26T05:54:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.763+0000 7f1145c16640 1 -- start start 2023-10-26T05:54:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11400afa20 0x7f11400c0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11400b2f00 0x7f11400ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f11400c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140151330 con 0x7f114015b5f0 2023-10-26T05:54:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140151000 con 0x7f11400afa20 2023-10-26T05:54:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11400ac9a0 con 0x7f11400b2f00 2023-10-26T05:54:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f11400c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f11400c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39814/0 (socket says 172.21.15.71:39814) 2023-10-26T05:54:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 -- 172.21.15.71:0/1027485631 learned_addr learned my addr 172.21.15.71:0/1027485631 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113effd640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11400b2f00 0x7f11400ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113f7fe640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11400afa20 0x7f11400c0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 -- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11400b2f00 msgr2=0x7f11400ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11400b2f00 0x7f11400ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 -- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11400afa20 msgr2=0x7f11400c0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11400afa20 0x7f11400c0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 -- 172.21.15.71:0/1027485631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134009560 con 0x7f114015b5f0 2023-10-26T05:54:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113f7fe640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11400afa20 0x7f11400c0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113ffff640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f11400c4420 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f113800ccb0 tx=0x7f113800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:37.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138004b30 con 0x7f114015b5f0 2023-10-26T05:54:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1138004da0 con 0x7f114015b5f0 2023-10-26T05:54:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11400c14d0 con 0x7f114015b5f0 2023-10-26T05:54:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138005280 con 0x7f114015b5f0 2023-10-26T05:54:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11400c1a90 con 0x7f114015b5f0 2023-10-26T05:54:37.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11400c1cb0 con 0x7f114015b5f0 2023-10-26T05:54:37.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11380053e0 con 0x7f114015b5f0 2023-10-26T05:54:37.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1127fff640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f112c0664a0 0x7f112c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:37.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11380846e0 con 0x7f114015b5f0 2023-10-26T05:54:37.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.767+0000 7f113f7fe640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f112c0664a0 0x7f112c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:37.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.771+0000 7f113f7fe640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f112c0664a0 0x7f112c068960 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f11340af000 tx=0x7f113401b050 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:37.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.771+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f11400c1cb0 con 0x7f114015b5f0 2023-10-26T05:54:37.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.919+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11400c1cb0 con 0x7f114015b5f0 2023-10-26T05:54:37.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.919+0000 7f1127fff640 1 -- 172.21.15.71:0/1027485631 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+16496 (secure 0 0 0) 0x7f11400c1cb0 con 0x7f114015b5f0 2023-10-26T05:54:37.929 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:37.929 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":155,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:54:36.212101+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:54:37.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f112c0664a0 msgr2=0x7f112c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f112c0664a0 0x7f112c068960 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f11340af000 tx=0x7f113401b050 comp rx=0 tx=0).stop 2023-10-26T05:54:37.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 msgr2=0x7f11400c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:37.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f11400c4420 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f113800ccb0 tx=0x7f113800a620 comp rx=0 tx=0).stop 2023-10-26T05:54:37.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 shutdown_connections 2023-10-26T05:54:37.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f112c0664a0 0x7f112c068960 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f114015b5f0 0x7f11400c4420 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11400b2f00 0x7f11400ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 --2- 172.21.15.71:0/1027485631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11400afa20 0x7f11400c0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:37.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 >> 172.21.15.71:0/1027485631 conn(0x7f11400aef00 msgr2=0x7f11400ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:37.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 shutdown_connections 2023-10-26T05:54:37.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:37.923+0000 7f1145c16640 1 -- 172.21.15.71:0/1027485631 wait complete. 2023-10-26T05:54:37.947 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:54:37.947 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:54:38.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3987379569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe600052b0 msgr2=0x7fbe6000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3987379569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe600052b0 0x7fbe6000b990 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fbe70002240 tx=0x7fbe7001fa80 comp rx=0 tx=0).stop 2023-10-26T05:54:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3987379569 shutdown_connections 2023-10-26T05:54:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3987379569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe600052b0 0x7fbe6000b990 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3987379569 >> 172.21.15.71:0/3987379569 conn(0x7fbe780cdd80 msgr2=0x7fbe780ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3987379569 shutdown_connections 2023-10-26T05:54:38.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3987379569 wait complete. 2023-10-26T05:54:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 Processor -- start 2023-10-26T05:54:38.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- start start 2023-10-26T05:54:38.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe600052b0 0x7fbe7814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe780a7e70 0x7fbe7814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 0x7fbe7814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7806a650 con 0x7fbe7814d330 2023-10-26T05:54:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7815b1b0 con 0x7fbe780a7e70 2023-10-26T05:54:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe7ca1e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7815e190 con 0x7fbe600052b0 2023-10-26T05:54:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe76d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 0x7fbe7814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe76d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 0x7fbe7814d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39826/0 (socket says 172.21.15.71:39826) 2023-10-26T05:54:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.059+0000 7fbe76d9d640 1 -- 172.21.15.71:0/3990996514 learned_addr learned my addr 172.21.15.71:0/3990996514 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe75d9b640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe780a7e70 0x7fbe7814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe7659c640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe600052b0 0x7fbe7814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe76d9d640 1 -- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe600052b0 msgr2=0x7fbe7814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe76d9d640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe600052b0 0x7fbe7814a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe76d9d640 1 -- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe780a7e70 msgr2=0x7fbe7814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe76d9d640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe780a7e70 0x7fbe7814cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe76d9d640 1 -- 172.21.15.71:0/3990996514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7815deb0 con 0x7fbe7814d330 2023-10-26T05:54:38.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe76d9d640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 0x7fbe7814d7e0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbe70020320 tx=0x7fbe70001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:38.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700b6070 con 0x7fbe7814d330 2023-10-26T05:54:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe70001840 con 0x7fbe7814d330 2023-10-26T05:54:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7814af60 con 0x7fbe7814d330 2023-10-26T05:54:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70001c50 con 0x7fbe7814d330 2023-10-26T05:54:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7814b520 con 0x7fbe7814d330 2023-10-26T05:54:38.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78066b10 con 0x7fbe7814d330 2023-10-26T05:54:38.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe700af070 con 0x7fbe7814d330 2023-10-26T05:54:38.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe66ffd640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe580664a0 0x7fbe58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe70120760 con 0x7fbe7814d330 2023-10-26T05:54:38.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe7659c640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe580664a0 0x7fbe58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.063+0000 7fbe7659c640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe580664a0 0x7fbe58068960 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c10 tx=0x7fbe68006050 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:38.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.067+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe700edb50 con 0x7fbe7814d330 2023-10-26T05:54:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.219+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe7806aa70 con 0x7fbe580664a0 2023-10-26T05:54:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe66ffd640 1 -- 172.21.15.71:0/3990996514 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278822 (secure 0 0 0) 0x7fbe680069b0 con 0x7fbe580664a0 2023-10-26T05:54:38.251 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:54:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe580664a0 msgr2=0x7fbe58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe580664a0 0x7fbe58068960 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000c10 tx=0x7fbe68006050 comp rx=0 tx=0).stop 2023-10-26T05:54:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 msgr2=0x7fbe7814d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 0x7fbe7814d7e0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbe70020320 tx=0x7fbe70001260 comp rx=0 tx=0).stop 2023-10-26T05:54:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 shutdown_connections 2023-10-26T05:54:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe580664a0 0x7fbe58068960 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7814d330 0x7fbe7814d7e0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe780a7e70 0x7fbe7814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 --2- 172.21.15.71:0/3990996514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe600052b0 0x7fbe7814a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 >> 172.21.15.71:0/3990996514 conn(0x7fbe780cdd80 msgr2=0x7fbe780b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:38.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 shutdown_connections 2023-10-26T05:54:38.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.243+0000 7fbe7ca1e640 1 -- 172.21.15.71:0/3990996514 wait complete. 2023-10-26T05:54:38.269 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-10-26T05:54:37.556776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554723,"statfs":{"total":1600000000,"available":1592036621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.353828"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218733+0000","last_change":"2023-10-26T05:54:34.218733+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218733+0000","last_undegraded":"2023-10-26T05:54:34.218733+0000","last_fullsized":"2023-10-26T05:54:34.218733+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224954+0000","last_change":"2023-10-26T05:54:34.224954+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224954+0000","last_undegraded":"2023-10-26T05:54:34.224954+0000","last_fullsized":"2023-10-26T05:54:34.224954+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235808+0000","last_change":"2023-10-26T05:54:34.235808+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235808+0000","last_undegraded":"2023-10-26T05:54:34.235808+0000","last_fullsized":"2023-10-26T05:54:34.235808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217369+0000","last_change":"2023-10-26T05:54:34.217369+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217369+0000","last_undegraded":"2023-10-26T05:54:34.217369+0000","last_fullsized":"2023-10-26T05:54:34.217369+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222328+0000","last_change":"2023-10-26T05:54:34.222328+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222328+0000","last_undegraded":"2023-10-26T05:54:34.222328+0000","last_fullsized":"2023-10-26T05:54:34.222328+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219374+0000","last_change":"2023-10-26T05:54:34.219374+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219374+0000","last_undegraded":"2023-10-26T05:54:34.219374+0000","last_fullsized":"2023-10-26T05:54:34.219374+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223861+0000","last_change":"2023-10-26T05:54:34.223861+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223861+0000","last_undegraded":"2023-10-26T05:54:34.223861+0000","last_fullsized":"2023-10-26T05:54:34.223861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235033+0000","last_change":"2023-10-26T05:54:34.235033+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235033+0000","last_undegraded":"2023-10-26T05:54:34.235033+0000","last_fullsized":"2023-10-26T05:54:34.235033+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235409+0000","last_change":"2023-10-26T05:54:34.235409+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235409+0000","last_undegraded":"2023-10-26T05:54:34.235409+0000","last_fullsized":"2023-10-26T05:54:34.235409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222253+0000","last_change":"2023-10-26T05:54:34.222253+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222253+0000","last_undegraded":"2023-10-26T05:54:34.222253+0000","last_fullsized":"2023-10-26T05:54:34.222253+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224745+0000","last_change":"2023-10-26T05:54:34.224745+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224745+0000","last_undegraded":"2023-10-26T05:54:34.224745+0000","last_fullsized":"2023-10-26T05:54:34.224745+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234523+0000","last_change":"2023-10-26T05:54:34.234523+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234523+0000","last_undegraded":"2023-10-26T05:54:34.234523+0000","last_fullsized":"2023-10-26T05:54:34.234523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216288+0000","last_change":"2023-10-26T05:54:34.216288+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216288+0000","last_undegraded":"2023-10-26T05:54:34.216288+0000","last_fullsized":"2023-10-26T05:54:34.216288+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223176+0000","last_change":"2023-10-26T05:54:34.223176+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223176+0000","last_undegraded":"2023-10-26T05:54:34.223176+0000","last_fullsized":"2023-10-26T05:54:34.223176+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218575+0000","last_change":"2023-10-26T05:54:34.218575+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218575+0000","last_undegraded":"2023-10-26T05:54:34.218575+0000","last_fullsized":"2023-10-26T05:54:34.218575+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221557+0000","last_change":"2023-10-26T05:54:34.221557+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221557+0000","last_undegraded":"2023-10-26T05:54:34.221557+0000","last_fullsized":"2023-10-26T05:54:34.221557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223087+0000","last_change":"2023-10-26T05:54:34.223087+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223087+0000","last_undegraded":"2023-10-26T05:54:34.223087+0000","last_fullsized":"2023-10-26T05:54:34.223087+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223894+0000","last_change":"2023-10-26T05:54:34.223894+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223894+0000","last_undegraded":"2023-10-26T05:54:34.223894+0000","last_fullsized":"2023-10-26T05:54:34.223894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225892+0000","last_change":"2023-10-26T05:54:34.225892+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225892+0000","last_undegraded":"2023-10-26T05:54:34.225892+0000","last_fullsized":"2023-10-26T05:54:34.225892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221998+0000","last_change":"2023-10-26T05:54:34.221998+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221998+0000","last_undegraded":"2023-10-26T05:54:34.221998+0000","last_fullsized":"2023-10-26T05:54:34.221998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222347+0000","last_change":"2023-10-26T05:54:34.222347+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222347+0000","last_undegraded":"2023-10-26T05:54:34.222347+0000","last_fullsized":"2023-10-26T05:54:34.222347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225197+0000","last_change":"2023-10-26T05:54:34.225197+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225197+0000","last_undegraded":"2023-10-26T05:54:34.225197+0000","last_fullsized":"2023-10-26T05:54:34.225197+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234208+0000","last_change":"2023-10-26T05:54:34.234208+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234208+0000","last_undegraded":"2023-10-26T05:54:34.234208+0000","last_fullsized":"2023-10-26T05:54:34.234208+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225474+0000","last_change":"2023-10-26T05:54:34.225474+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225474+0000","last_undegraded":"2023-10-26T05:54:34.225474+0000","last_fullsized":"2023-10-26T05:54:34.225474+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221861+0000","last_change":"2023-10-26T05:54:34.221861+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221861+0000","last_undegraded":"2023-10-26T05:54:34.221861+0000","last_fullsized":"2023-10-26T05:54:34.221861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221231+0000","last_change":"2023-10-26T05:54:34.221231+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221231+0000","last_undegraded":"2023-10-26T05:54:34.221231+0000","last_fullsized":"2023-10-26T05:54:34.221231+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217314+0000","last_change":"2023-10-26T05:54:34.217314+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217314+0000","last_undegraded":"2023-10-26T05:54:34.217314+0000","last_fullsized":"2023-10-26T05:54:34.217314+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.242658+0000","last_change":"2023-10-26T05:54:34.242658+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.242658+0000","last_undegraded":"2023-10-26T05:54:34.242658+0000","last_fullsized":"2023-10-26T05:54:34.242658+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216909+0000","last_change":"2023-10-26T05:54:34.216909+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216909+0000","last_undegraded":"2023-10-26T05:54:34.216909+0000","last_fullsized":"2023-10-26T05:54:34.216909+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217829+0000","last_change":"2023-10-26T05:54:34.217829+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217829+0000","last_undegraded":"2023-10-26T05:54:34.217829+0000","last_fullsized":"2023-10-26T05:54:34.217829+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221485+0000","last_change":"2023-10-26T05:54:34.221485+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221485+0000","last_undegraded":"2023-10-26T05:54:34.221485+0000","last_fullsized":"2023-10-26T05:54:34.221485+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222189+0000","last_change":"2023-10-26T05:54:34.222189+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222189+0000","last_undegraded":"2023-10-26T05:54:34.222189+0000","last_fullsized":"2023-10-26T05:54:34.222189+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221314+0000","last_change":"2023-10-26T05:54:34.221314+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221314+0000","last_undegraded":"2023-10-26T05:54:34.221314+0000","last_fullsized":"2023-10-26T05:54:34.221314+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.239407+0000","last_change":"2023-10-26T05:54:34.238405+0000","last_active":"2023-10-26T05:54:34.239407+0000","last_peered":"2023-10-26T05:54:34.239407+0000","last_clean":"2023-10-26T05:54:34.239407+0000","last_became_active":"2023-10-26T05:54:34.234974+0000","last_became_peered":"2023-10-26T05:54:34.234974+0000","last_unstale":"2023-10-26T05:54:34.239407+0000","last_undegraded":"2023-10-26T05:54:34.239407+0000","last_fullsized":"2023-10-26T05:54:34.239407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:49.174594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.233445+0000","last_change":"2023-10-26T05:54:34.233445+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.233445+0000","last_undegraded":"2023-10-26T05:54:34.233445+0000","last_fullsized":"2023-10-26T05:54:34.233445+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238557+0000","last_change":"2023-10-26T05:54:34.238394+0000","last_active":"2023-10-26T05:54:34.238557+0000","last_peered":"2023-10-26T05:54:34.238557+0000","last_clean":"2023-10-26T05:54:34.238557+0000","last_became_active":"2023-10-26T05:54:34.237301+0000","last_became_peered":"2023-10-26T05:54:34.237301+0000","last_unstale":"2023-10-26T05:54:34.238557+0000","last_undegraded":"2023-10-26T05:54:34.238557+0000","last_fullsized":"2023-10-26T05:54:34.238557+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:49:47.037386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220728+0000","last_change":"2023-10-26T05:54:34.220728+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220728+0000","last_undegraded":"2023-10-26T05:54:34.220728+0000","last_fullsized":"2023-10-26T05:54:34.220728+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228507+0000","last_change":"2023-10-26T05:54:34.228217+0000","last_active":"2023-10-26T05:54:34.228507+0000","last_peered":"2023-10-26T05:54:34.228507+0000","last_clean":"2023-10-26T05:54:34.228507+0000","last_became_active":"2023-10-26T05:54:34.227446+0000","last_became_peered":"2023-10-26T05:54:34.227446+0000","last_unstale":"2023-10-26T05:54:34.228507+0000","last_undegraded":"2023-10-26T05:54:34.228507+0000","last_fullsized":"2023-10-26T05:54:34.228507+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:56.388499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221549+0000","last_change":"2023-10-26T05:54:34.221549+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221549+0000","last_undegraded":"2023-10-26T05:54:34.221549+0000","last_fullsized":"2023-10-26T05:54:34.221549+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.224414+0000","last_change":"2023-10-26T05:54:34.219413+0000","last_active":"2023-10-26T05:54:34.224414+0000","last_peered":"2023-10-26T05:54:34.224414+0000","last_clean":"2023-10-26T05:54:34.224414+0000","last_became_active":"2023-10-26T05:54:34.219109+0000","last_became_peered":"2023-10-26T05:54:34.219109+0000","last_unstale":"2023-10-26T05:54:34.224414+0000","last_undegraded":"2023-10-26T05:54:34.224414+0000","last_fullsized":"2023-10-26T05:54:34.224414+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:04:02.823811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223568+0000","last_change":"2023-10-26T05:54:34.223568+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223568+0000","last_undegraded":"2023-10-26T05:54:34.223568+0000","last_fullsized":"2023-10-26T05:54:34.223568+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.225568+0000","last_change":"2023-10-26T05:54:34.225466+0000","last_active":"2023-10-26T05:54:34.225568+0000","last_peered":"2023-10-26T05:54:34.225568+0000","last_clean":"2023-10-26T05:54:34.225568+0000","last_became_active":"2023-10-26T05:54:34.225226+0000","last_became_peered":"2023-10-26T05:54:34.225226+0000","last_unstale":"2023-10-26T05:54:34.225568+0000","last_undegraded":"2023-10-26T05:54:34.225568+0000","last_fullsized":"2023-10-26T05:54:34.225568+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:09:54.209739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221407+0000","last_change":"2023-10-26T05:54:34.221407+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221407+0000","last_undegraded":"2023-10-26T05:54:34.221407+0000","last_fullsized":"2023-10-26T05:54:34.221407+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.231347+0000","last_change":"2023-10-26T05:54:34.231290+0000","last_active":"2023-10-26T05:54:34.231347+0000","last_peered":"2023-10-26T05:54:34.231347+0000","last_clean":"2023-10-26T05:54:34.231347+0000","last_became_active":"2023-10-26T05:54:34.230708+0000","last_became_peered":"2023-10-26T05:54:34.230708+0000","last_unstale":"2023-10-26T05:54:34.231347+0000","last_undegraded":"2023-10-26T05:54:34.231347+0000","last_fullsized":"2023-10-26T05:54:34.231347+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:45:26.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.236168+0000","last_change":"2023-10-26T05:54:34.236168+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.236168+0000","last_undegraded":"2023-10-26T05:54:34.236168+0000","last_fullsized":"2023-10-26T05:54:34.236168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.229222+0000","last_change":"2023-10-26T05:54:34.229066+0000","last_active":"2023-10-26T05:54:34.229222+0000","last_peered":"2023-10-26T05:54:34.229222+0000","last_clean":"2023-10-26T05:54:34.229222+0000","last_became_active":"2023-10-26T05:54:34.228464+0000","last_became_peered":"2023-10-26T05:54:34.228464+0000","last_unstale":"2023-10-26T05:54:34.229222+0000","last_undegraded":"2023-10-26T05:54:34.229222+0000","last_fullsized":"2023-10-26T05:54:34.229222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:39.551427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222740+0000","last_change":"2023-10-26T05:54:34.222740+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222740+0000","last_undegraded":"2023-10-26T05:54:34.222740+0000","last_fullsized":"2023-10-26T05:54:34.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237238+0000","last_change":"2023-10-26T05:54:34.237160+0000","last_active":"2023-10-26T05:54:34.237238+0000","last_peered":"2023-10-26T05:54:34.237238+0000","last_clean":"2023-10-26T05:54:34.237238+0000","last_became_active":"2023-10-26T05:54:34.236869+0000","last_became_peered":"2023-10-26T05:54:34.236869+0000","last_unstale":"2023-10-26T05:54:34.237238+0000","last_undegraded":"2023-10-26T05:54:34.237238+0000","last_fullsized":"2023-10-26T05:54:34.237238+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:15.255164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217012+0000","last_change":"2023-10-26T05:54:34.217012+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217012+0000","last_undegraded":"2023-10-26T05:54:34.217012+0000","last_fullsized":"2023-10-26T05:54:34.217012+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228303+0000","last_change":"2023-10-26T05:54:34.227954+0000","last_active":"2023-10-26T05:54:34.228303+0000","last_peered":"2023-10-26T05:54:34.228303+0000","last_clean":"2023-10-26T05:54:34.228303+0000","last_became_active":"2023-10-26T05:54:34.227315+0000","last_became_peered":"2023-10-26T05:54:34.227315+0000","last_unstale":"2023-10-26T05:54:34.228303+0000","last_undegraded":"2023-10-26T05:54:34.228303+0000","last_fullsized":"2023-10-26T05:54:34.228303+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:36:29.297709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.244373+0000","last_change":"2023-10-26T05:54:34.244373+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.244373+0000","last_undegraded":"2023-10-26T05:54:34.244373+0000","last_fullsized":"2023-10-26T05:54:34.244373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228819+0000","last_change":"2023-10-26T05:54:34.228604+0000","last_active":"2023-10-26T05:54:34.228819+0000","last_peered":"2023-10-26T05:54:34.228819+0000","last_clean":"2023-10-26T05:54:34.228819+0000","last_became_active":"2023-10-26T05:54:34.228363+0000","last_became_peered":"2023-10-26T05:54:34.228363+0000","last_unstale":"2023-10-26T05:54:34.228819+0000","last_undegraded":"2023-10-26T05:54:34.228819+0000","last_fullsized":"2023-10-26T05:54:34.228819+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:08:08.211694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224664+0000","last_change":"2023-10-26T05:54:34.224664+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224664+0000","last_undegraded":"2023-10-26T05:54:34.224664+0000","last_fullsized":"2023-10-26T05:54:34.224664+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.241631+0000","last_change":"2023-10-26T05:54:34.241288+0000","last_active":"2023-10-26T05:54:34.241631+0000","last_peered":"2023-10-26T05:54:34.241631+0000","last_clean":"2023-10-26T05:54:34.241631+0000","last_became_active":"2023-10-26T05:54:34.240585+0000","last_became_peered":"2023-10-26T05:54:34.240585+0000","last_unstale":"2023-10-26T05:54:34.241631+0000","last_undegraded":"2023-10-26T05:54:34.241631+0000","last_fullsized":"2023-10-26T05:54:34.241631+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:17:51.315680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222024+0000","last_change":"2023-10-26T05:54:34.222024+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222024+0000","last_undegraded":"2023-10-26T05:54:34.222024+0000","last_fullsized":"2023-10-26T05:54:34.222024+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.246204+0000","last_change":"2023-10-26T05:54:34.245988+0000","last_active":"2023-10-26T05:54:34.246204+0000","last_peered":"2023-10-26T05:54:34.246204+0000","last_clean":"2023-10-26T05:54:34.246204+0000","last_became_active":"2023-10-26T05:54:34.245363+0000","last_became_peered":"2023-10-26T05:54:34.245363+0000","last_unstale":"2023-10-26T05:54:34.246204+0000","last_undegraded":"2023-10-26T05:54:34.246204+0000","last_fullsized":"2023-10-26T05:54:34.246204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:10:53.581294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216915+0000","last_change":"2023-10-26T05:54:34.216915+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216915+0000","last_undegraded":"2023-10-26T05:54:34.216915+0000","last_fullsized":"2023-10-26T05:54:34.216915+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226894+0000","last_change":"2023-10-26T05:54:34.226642+0000","last_active":"2023-10-26T05:54:34.226894+0000","last_peered":"2023-10-26T05:54:34.226894+0000","last_clean":"2023-10-26T05:54:34.226894+0000","last_became_active":"2023-10-26T05:54:34.226063+0000","last_became_peered":"2023-10-26T05:54:34.226063+0000","last_unstale":"2023-10-26T05:54:34.226894+0000","last_undegraded":"2023-10-26T05:54:34.226894+0000","last_fullsized":"2023-10-26T05:54:34.226894+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:29.737511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224031+0000","last_change":"2023-10-26T05:54:34.224031+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224031+0000","last_undegraded":"2023-10-26T05:54:34.224031+0000","last_fullsized":"2023-10-26T05:54:34.224031+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.230277+0000","last_change":"2023-10-26T05:54:34.230187+0000","last_active":"2023-10-26T05:54:34.230277+0000","last_peered":"2023-10-26T05:54:34.230277+0000","last_clean":"2023-10-26T05:54:34.230277+0000","last_became_active":"2023-10-26T05:54:34.229848+0000","last_became_peered":"2023-10-26T05:54:34.229848+0000","last_unstale":"2023-10-26T05:54:34.230277+0000","last_undegraded":"2023-10-26T05:54:34.230277+0000","last_fullsized":"2023-10-26T05:54:34.230277+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:50.756897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.233720+0000","last_change":"2023-10-26T05:54:34.233720+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.233720+0000","last_undegraded":"2023-10-26T05:54:34.233720+0000","last_fullsized":"2023-10-26T05:54:34.233720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.242840+0000","last_change":"2023-10-26T05:54:34.242272+0000","last_active":"2023-10-26T05:54:34.242840+0000","last_peered":"2023-10-26T05:54:34.242840+0000","last_clean":"2023-10-26T05:54:34.242840+0000","last_became_active":"2023-10-26T05:54:34.240219+0000","last_became_peered":"2023-10-26T05:54:34.240219+0000","last_unstale":"2023-10-26T05:54:34.242840+0000","last_undegraded":"2023-10-26T05:54:34.242840+0000","last_fullsized":"2023-10-26T05:54:34.242840+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:23.700023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234009+0000","last_change":"2023-10-26T05:54:34.234009+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234009+0000","last_undegraded":"2023-10-26T05:54:34.234009+0000","last_fullsized":"2023-10-26T05:54:34.234009+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238519+0000","last_change":"2023-10-26T05:54:34.237939+0000","last_active":"2023-10-26T05:54:34.238519+0000","last_peered":"2023-10-26T05:54:34.238519+0000","last_clean":"2023-10-26T05:54:34.238519+0000","last_became_active":"2023-10-26T05:54:34.236275+0000","last_became_peered":"2023-10-26T05:54:34.236275+0000","last_unstale":"2023-10-26T05:54:34.238519+0000","last_undegraded":"2023-10-26T05:54:34.238519+0000","last_fullsized":"2023-10-26T05:54:34.238519+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:12:15.727069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.242318+0000","last_change":"2023-10-26T05:54:34.242318+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.242318+0000","last_undegraded":"2023-10-26T05:54:34.242318+0000","last_fullsized":"2023-10-26T05:54:34.242318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238966+0000","last_change":"2023-10-26T05:54:34.238890+0000","last_active":"2023-10-26T05:54:34.238966+0000","last_peered":"2023-10-26T05:54:34.238966+0000","last_clean":"2023-10-26T05:54:34.238966+0000","last_became_active":"2023-10-26T05:54:34.238622+0000","last_became_peered":"2023-10-26T05:54:34.238622+0000","last_unstale":"2023-10-26T05:54:34.238966+0000","last_undegraded":"2023-10-26T05:54:34.238966+0000","last_fullsized":"2023-10-26T05:54:34.238966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:26:48.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222516+0000","last_change":"2023-10-26T05:54:34.222516+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222516+0000","last_undegraded":"2023-10-26T05:54:34.222516+0000","last_fullsized":"2023-10-26T05:54:34.222516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.240565+0000","last_change":"2023-10-26T05:54:34.239412+0000","last_active":"2023-10-26T05:54:34.240565+0000","last_peered":"2023-10-26T05:54:34.240565+0000","last_clean":"2023-10-26T05:54:34.240565+0000","last_became_active":"2023-10-26T05:54:34.238306+0000","last_became_peered":"2023-10-26T05:54:34.238306+0000","last_unstale":"2023-10-26T05:54:34.240565+0000","last_undegraded":"2023-10-26T05:54:34.240565+0000","last_fullsized":"2023-10-26T05:54:34.240565+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:09.950751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.226710+0000","last_change":"2023-10-26T05:54:34.226710+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.226710+0000","last_undegraded":"2023-10-26T05:54:34.226710+0000","last_fullsized":"2023-10-26T05:54:34.226710+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237616+0000","last_change":"2023-10-26T05:54:34.237449+0000","last_active":"2023-10-26T05:54:34.237616+0000","last_peered":"2023-10-26T05:54:34.237616+0000","last_clean":"2023-10-26T05:54:34.237616+0000","last_became_active":"2023-10-26T05:54:34.236646+0000","last_became_peered":"2023-10-26T05:54:34.236646+0000","last_unstale":"2023-10-26T05:54:34.237616+0000","last_undegraded":"2023-10-26T05:54:34.237616+0000","last_fullsized":"2023-10-26T05:54:34.237616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:09.282179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216456+0000","last_change":"2023-10-26T05:54:34.216456+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216456+0000","last_undegraded":"2023-10-26T05:54:34.216456+0000","last_fullsized":"2023-10-26T05:54:34.216456+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.230169+0000","last_change":"2023-10-26T05:54:34.229691+0000","last_active":"2023-10-26T05:54:34.230169+0000","last_peered":"2023-10-26T05:54:34.230169+0000","last_clean":"2023-10-26T05:54:34.230169+0000","last_became_active":"2023-10-26T05:54:34.229093+0000","last_became_peered":"2023-10-26T05:54:34.229093+0000","last_unstale":"2023-10-26T05:54:34.230169+0000","last_undegraded":"2023-10-26T05:54:34.230169+0000","last_fullsized":"2023-10-26T05:54:34.230169+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:21.281518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219762+0000","last_change":"2023-10-26T05:54:34.219762+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219762+0000","last_undegraded":"2023-10-26T05:54:34.219762+0000","last_fullsized":"2023-10-26T05:54:34.219762+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.231966+0000","last_change":"2023-10-26T05:54:34.231906+0000","last_active":"2023-10-26T05:54:34.231966+0000","last_peered":"2023-10-26T05:54:34.231966+0000","last_clean":"2023-10-26T05:54:34.231966+0000","last_became_active":"2023-10-26T05:54:34.231677+0000","last_became_peered":"2023-10-26T05:54:34.231677+0000","last_unstale":"2023-10-26T05:54:34.231966+0000","last_undegraded":"2023-10-26T05:54:34.231966+0000","last_fullsized":"2023-10-26T05:54:34.231966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:31.782805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.226387+0000","last_change":"2023-10-26T05:54:34.226387+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.226387+0000","last_undegraded":"2023-10-26T05:54:34.226387+0000","last_fullsized":"2023-10-26T05:54:34.226387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.236138+0000","last_change":"2023-10-26T05:54:34.236055+0000","last_active":"2023-10-26T05:54:34.236138+0000","last_peered":"2023-10-26T05:54:34.236138+0000","last_clean":"2023-10-26T05:54:34.236138+0000","last_became_active":"2023-10-26T05:54:34.235715+0000","last_became_peered":"2023-10-26T05:54:34.235715+0000","last_unstale":"2023-10-26T05:54:34.236138+0000","last_undegraded":"2023-10-26T05:54:34.236138+0000","last_fullsized":"2023-10-26T05:54:34.236138+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:51:26.940485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220755+0000","last_change":"2023-10-26T05:54:34.220755+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220755+0000","last_undegraded":"2023-10-26T05:54:34.220755+0000","last_fullsized":"2023-10-26T05:54:34.220755+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.235237+0000","last_change":"2023-10-26T05:54:34.235155+0000","last_active":"2023-10-26T05:54:34.235237+0000","last_peered":"2023-10-26T05:54:34.235237+0000","last_clean":"2023-10-26T05:54:34.235237+0000","last_became_active":"2023-10-26T05:54:34.234905+0000","last_became_peered":"2023-10-26T05:54:34.234905+0000","last_unstale":"2023-10-26T05:54:34.235237+0000","last_undegraded":"2023-10-26T05:54:34.235237+0000","last_fullsized":"2023-10-26T05:54:34.235237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:33.288741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218889+0000","last_change":"2023-10-26T05:54:34.218889+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218889+0000","last_undegraded":"2023-10-26T05:54:34.218889+0000","last_fullsized":"2023-10-26T05:54:34.218889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226868+0000","last_change":"2023-10-26T05:54:34.226691+0000","last_active":"2023-10-26T05:54:34.226868+0000","last_peered":"2023-10-26T05:54:34.226868+0000","last_clean":"2023-10-26T05:54:34.226868+0000","last_became_active":"2023-10-26T05:54:34.225872+0000","last_became_peered":"2023-10-26T05:54:34.225872+0000","last_unstale":"2023-10-26T05:54:34.226868+0000","last_undegraded":"2023-10-26T05:54:34.226868+0000","last_fullsized":"2023-10-26T05:54:34.226868+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:37.373754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216671+0000","last_change":"2023-10-26T05:54:34.216671+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216671+0000","last_undegraded":"2023-10-26T05:54:34.216671+0000","last_fullsized":"2023-10-26T05:54:34.216671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.239361+0000","last_change":"2023-10-26T05:54:34.235620+0000","last_active":"2023-10-26T05:54:34.239361+0000","last_peered":"2023-10-26T05:54:34.239361+0000","last_clean":"2023-10-26T05:54:34.239361+0000","last_became_active":"2023-10-26T05:54:34.231181+0000","last_became_peered":"2023-10-26T05:54:34.231181+0000","last_unstale":"2023-10-26T05:54:34.239361+0000","last_undegraded":"2023-10-26T05:54:34.239361+0000","last_fullsized":"2023-10-26T05:54:34.239361+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:12.750236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221409+0000","last_change":"2023-10-26T05:54:34.221409+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221409+0000","last_undegraded":"2023-10-26T05:54:34.221409+0000","last_fullsized":"2023-10-26T05:54:34.221409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.234344+0000","last_change":"2023-10-26T05:54:34.234275+0000","last_active":"2023-10-26T05:54:34.234344+0000","last_peered":"2023-10-26T05:54:34.234344+0000","last_clean":"2023-10-26T05:54:34.234344+0000","last_became_active":"2023-10-26T05:54:34.234042+0000","last_became_peered":"2023-10-26T05:54:34.234042+0000","last_unstale":"2023-10-26T05:54:34.234344+0000","last_undegraded":"2023-10-26T05:54:34.234344+0000","last_fullsized":"2023-10-26T05:54:34.234344+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:50:58.640410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218977+0000","last_change":"2023-10-26T05:54:34.218977+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218977+0000","last_undegraded":"2023-10-26T05:54:34.218977+0000","last_fullsized":"2023-10-26T05:54:34.218977+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228533+0000","last_change":"2023-10-26T05:54:34.228407+0000","last_active":"2023-10-26T05:54:34.228533+0000","last_peered":"2023-10-26T05:54:34.228533+0000","last_clean":"2023-10-26T05:54:34.228533+0000","last_became_active":"2023-10-26T05:54:34.227889+0000","last_became_peered":"2023-10-26T05:54:34.227889+0000","last_unstale":"2023-10-26T05:54:34.228533+0000","last_undegraded":"2023-10-26T05:54:34.228533+0000","last_fullsized":"2023-10-26T05:54:34.228533+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:42:05.162676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219774+0000","last_change":"2023-10-26T05:54:34.219774+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219774+0000","last_undegraded":"2023-10-26T05:54:34.219774+0000","last_fullsized":"2023-10-26T05:54:34.219774+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.227621+0000","last_change":"2023-10-26T05:54:34.227337+0000","last_active":"2023-10-26T05:54:34.227621+0000","last_peered":"2023-10-26T05:54:34.227621+0000","last_clean":"2023-10-26T05:54:34.227621+0000","last_became_active":"2023-10-26T05:54:34.227083+0000","last_became_peered":"2023-10-26T05:54:34.227083+0000","last_unstale":"2023-10-26T05:54:34.227621+0000","last_undegraded":"2023-10-26T05:54:34.227621+0000","last_fullsized":"2023-10-26T05:54:34.227621+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:21.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.243298+0000","last_change":"2023-10-26T05:54:34.243298+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.243298+0000","last_undegraded":"2023-10-26T05:54:34.243298+0000","last_fullsized":"2023-10-26T05:54:34.243298+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237935+0000","last_change":"2023-10-26T05:54:34.236735+0000","last_active":"2023-10-26T05:54:34.237935+0000","last_peered":"2023-10-26T05:54:34.237935+0000","last_clean":"2023-10-26T05:54:34.237935+0000","last_became_active":"2023-10-26T05:54:34.233447+0000","last_became_peered":"2023-10-26T05:54:34.233447+0000","last_unstale":"2023-10-26T05:54:34.237935+0000","last_undegraded":"2023-10-26T05:54:34.237935+0000","last_fullsized":"2023-10-26T05:54:34.237935+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:14.747480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220883+0000","last_change":"2023-10-26T05:54:34.220883+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220883+0000","last_undegraded":"2023-10-26T05:54:34.220883+0000","last_fullsized":"2023-10-26T05:54:34.220883+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.234328+0000","last_change":"2023-10-26T05:54:34.234238+0000","last_active":"2023-10-26T05:54:34.234328+0000","last_peered":"2023-10-26T05:54:34.234328+0000","last_clean":"2023-10-26T05:54:34.234328+0000","last_became_active":"2023-10-26T05:54:34.234006+0000","last_became_peered":"2023-10-26T05:54:34.234006+0000","last_unstale":"2023-10-26T05:54:34.234328+0000","last_undegraded":"2023-10-26T05:54:34.234328+0000","last_fullsized":"2023-10-26T05:54:34.234328+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:45.551886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223205+0000","last_change":"2023-10-26T05:54:34.223205+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223205+0000","last_undegraded":"2023-10-26T05:54:34.223205+0000","last_fullsized":"2023-10-26T05:54:34.223205+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226797+0000","last_change":"2023-10-26T05:54:34.226096+0000","last_active":"2023-10-26T05:54:34.226797+0000","last_peered":"2023-10-26T05:54:34.226797+0000","last_clean":"2023-10-26T05:54:34.226797+0000","last_became_active":"2023-10-26T05:54:34.224494+0000","last_became_peered":"2023-10-26T05:54:34.224494+0000","last_unstale":"2023-10-26T05:54:34.226797+0000","last_undegraded":"2023-10-26T05:54:34.226797+0000","last_fullsized":"2023-10-26T05:54:34.226797+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:09.983718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219971+0000","last_change":"2023-10-26T05:54:34.219971+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219971+0000","last_undegraded":"2023-10-26T05:54:34.219971+0000","last_fullsized":"2023-10-26T05:54:34.219971+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.243333+0000","last_change":"2023-10-26T05:54:34.241834+0000","last_active":"2023-10-26T05:54:34.243333+0000","last_peered":"2023-10-26T05:54:34.243333+0000","last_clean":"2023-10-26T05:54:34.243333+0000","last_became_active":"2023-10-26T05:54:34.232265+0000","last_became_peered":"2023-10-26T05:54:34.232265+0000","last_unstale":"2023-10-26T05:54:34.243333+0000","last_undegraded":"2023-10-26T05:54:34.243333+0000","last_fullsized":"2023-10-26T05:54:34.243333+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:19.460844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-26T05:54:33.207194+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:54:33.207194+0000","last_peered":"2023-10-26T05:54:33.207194+0000","last_clean":"2023-10-26T05:54:33.207194+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:54:33.207194+0000","last_undegraded":"2023-10-26T05:54:33.207194+0000","last_fullsized":"2023-10-26T05:54:33.207194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1409,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193902,"statfs":{"total":200000000,"available":198556311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.39600000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.169,"5min":1.286,"15min":1.3100000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94799999999999995,"15min":1.018},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.236,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.93100000000000005},"min":{"1min":0.67100000000000004,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.2849999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69199999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.214,"15min":1.4330000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3129999999999999,"5min":1.375,"15min":1.375},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.1679999999999999,"5min":1.639,"15min":1.639},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96399999999999997,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3149999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.877}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0169999999999999,"15min":1.1000000000000001},"min":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.58699999999999997},"max":{"1min":1.266,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0720000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.53800000000000003},"max":{"1min":1.3029999999999999,"5min":1.702,"15min":1.702},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.081,"15min":1.1100000000000001},"min":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.59599999999999997},"max":{"1min":1.3640000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.0609999999999999,"15min":1.0509999999999999},"min":{"1min":0.66700000000000004,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.6180000000000001},"last":1.0740000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.298,"5min":1.298,"15min":1.3},"last":1.0660000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.62,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.405},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.82099999999999995},"min":{"1min":0.54800000000000004,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1990000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.435,"15min":0.435},"max":{"1min":1.3,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.621}]},{"osd":3,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97999999999999998,"15min":0.90400000000000003},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2250000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.056,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.3700000000000001,"5min":1.399,"15min":1.399},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.85699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.002,"5min":0.97799999999999998,"15min":0.97899999999999998},"min":{"1min":0.64500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.351,"5min":1.375,"15min":1.375},"last":1.143}]},{"osd":5,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0469999999999999},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.76800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.25,"5min":1.512,"15min":1.512},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1060000000000001},"min":{"1min":0.96799999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4550000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.92100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":0.94999999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.381,"5min":1.579,"15min":1.579},"last":0.879},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.53300000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.089,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.75},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.464,"15min":1.464},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.2110000000000001,"5min":1.319,"15min":1.5089999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.92700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2190000000000001,"5min":1.274,"15min":1.3160000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.59299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.333,"5min":1.448,"15min":1.548},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.238,"5min":1.4139999999999999,"15min":1.587},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94099999999999995,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2769999999999999,"5min":1.286,"15min":2.1909999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.091,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.536,"15min":1.536},"last":1.117}]},{"osd":6,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.96899999999999997,"15min":1.018},"min":{"1min":0.63500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0169999999999999,"15min":1.032},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.4199999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.442,"15min":0.442},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.155}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.85999999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.248,"5min":1.2569999999999999,"15min":1.347},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.53100000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.143,"5min":1.502,"15min":1.502},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.878,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.25,"15min":1.375},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.96199999999999997,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":1.3440000000000001,"15min":1.4390000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90700000000000003,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.107,"5min":1.373,"15min":1.373},"last":0.877}]},{"osd":3,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.397,"5min":1.397,"15min":2.0190000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.71099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.232,"5min":1.3300000000000001,"15min":1.4359999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.88900000000000001},"min":{"1min":0.65600000000000003,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3440000000000001,"15min":1.3859999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.323,"5min":1.4179999999999999,"15min":1.673},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0289999999999999,"15min":1.004},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.42099999999999999},"max":{"1min":1.762,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.73799999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3540000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.68600000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.7509999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.137,"15min":1.119},"min":{"1min":0.74299999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"max":{"1min":1.484,"5min":1.728,"15min":1.728},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.29599999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.76800000000000002,"5min":1.385,"15min":1.385},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.94699999999999995,"5min":1.115,"15min":1.2689999999999999},"last":0.309}]},{"osd":1,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80100000000000005,"5min":1.349,"15min":1.349},"last":0.373},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65900000000000003,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.99299999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":1.1619999999999999,"5min":1.236,"15min":1.4199999999999999},"last":0.498}]},{"osd":4,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":0.83499999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.40000000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.073,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.95599999999999996},"min":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55800000000000005},"max":{"1min":1.113,"5min":1.413,"15min":1.478},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.91200000000000003},"min":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.59799999999999998},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93300000000000005,"15min":0.97999999999999998},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.165,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.254,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":0.93200000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.232,"5min":2.004,"15min":2.004},"last":0.81999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.54400000000000004},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.88300000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71799999999999997,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1120000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.67600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.198,"5min":1.226,"15min":1.2609999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68999999999999995,"15min":0.60199999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.157,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.74399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.149,"5min":1.333,"15min":2.8809999999999998},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.089,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.83899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1060000000000001,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.749},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.54400000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1459999999999999,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.91900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.476,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.052},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.99299999999999999,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.653,"15min":1.653},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.002,"15min":0.92700000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.077,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.419,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.135}]},{"osd":7,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.97799999999999998,"15min":0.91000000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.173,"5min":1.581,"15min":1.581},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0900000000000001,"15min":1.0449999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.367,"5min":1.823,"15min":15.352},"last":1.2130000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.157,"15min":1.157},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83599999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27700000000000002},"max":{"1min":1.087,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.378}]},{"osd":3,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.89500000000000002,"5min":1.298,"15min":1.298},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.64500000000000002,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1659999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.27,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.621,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3620000000000001,"5min":1.492,"15min":1.492},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.92400000000000004,"15min":0.92900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3029999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96699999999999997},"min":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.468,"15min":1.837},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.012,"15min":1.022},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.2330000000000001,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.046,"15min":1.0840000000000001},"min":{"1min":0.63200000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4099999999999999,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193908,"statfs":{"total":200000000,"available":198562586,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78100000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.69899999999999995,"15min":0.62},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85899999999999999,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.093,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.437},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1140000000000001,"15min":1.177},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.626},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.99299999999999999,"5min":1.409,"15min":1.409},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80500000000000005,"15min":0.70999999999999996},"min":{"1min":0.33400000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.2629999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.495}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.92100000000000004},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2769999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1399999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.90900000000000003},"min":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.123,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.004,"15min":0.996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4399999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":0.98999999999999999},"min":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.78300000000000003}]}]}],"pool_statfs":[]}} 2023-10-26T05:54:38.275 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:54:38.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 -- 172.21.15.71:0/1426724063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00d0850 msgr2=0x7f60d015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1426724063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00d0850 0x7f60d015b3a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f60c0009840 tx=0x7f60c001c770 comp rx=0 tx=0).stop 2023-10-26T05:54:38.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 -- 172.21.15.71:0/1426724063 shutdown_connections 2023-10-26T05:54:38.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1426724063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60d00cd880 0x7f60d015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1426724063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00d0850 0x7f60d015b3a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 -- 172.21.15.71:0/1426724063 >> 172.21.15.71:0/1426724063 conn(0x7f60d00cdd80 msgr2=0x7f60d00ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 -- 172.21.15.71:0/1426724063 shutdown_connections 2023-10-26T05:54:38.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 -- 172.21.15.71:0/1426724063 wait complete. 2023-10-26T05:54:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.431+0000 7f60d77d7640 1 Processor -- start 2023-10-26T05:54:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- start start 2023-10-26T05:54:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00cd880 0x7f60d0159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.444 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60d00d0470 0x7f60d015def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 0x7f60d00c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0158650 con 0x7f60d00d0850 2023-10-26T05:54:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d0159e10 con 0x7f60d00cd880 2023-10-26T05:54:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d01588a0 con 0x7f60d00d0470 2023-10-26T05:54:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 0x7f60d00c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 0x7f60d00c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39828/0 (socket says 172.21.15.71:39828) 2023-10-26T05:54:38.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 -- 172.21.15.71:0/1183084318 learned_addr learned my addr 172.21.15.71:0/1183084318 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:38.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d4d72640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60d00d0470 0x7f60d015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5573640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00cd880 0x7f60d0159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 -- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60d00d0470 msgr2=0x7f60d015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60d00d0470 0x7f60d015def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 -- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00cd880 msgr2=0x7f60d0159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00cd880 0x7f60d0159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 -- 172.21.15.71:0/1183084318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8009820 con 0x7f60d00d0850 2023-10-26T05:54:38.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5573640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00cd880 0x7f60d0159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:38.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d5d74640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 0x7f60d00c7b10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f60c800efd0 tx=0x7f60c800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:38.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8009b80 con 0x7f60d00d0850 2023-10-26T05:54:38.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f60c8003b80 con 0x7f60d00d0850 2023-10-26T05:54:38.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c00094a0 con 0x7f60d00d0850 2023-10-26T05:54:38.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8004020 con 0x7f60d00d0850 2023-10-26T05:54:38.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d0158e60 con 0x7f60d00d0850 2023-10-26T05:54:38.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d0066b10 con 0x7f60d00d0850 2023-10-26T05:54:38.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f60c8004180 con 0x7f60d00d0850 2023-10-26T05:54:38.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60c5ffb640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60b4066470 0x7f60b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.435+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60c8014070 con 0x7f60d00d0850 2023-10-26T05:54:38.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.439+0000 7f60d5573640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60b4066470 0x7f60b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.439+0000 7f60d5573640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60b4066470 0x7f60b4068930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0046c0 tx=0x7f60cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:38.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.439+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f60c80522a0 con 0x7f60d00d0850 2023-10-26T05:54:38.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.591+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60d00b5130 con 0x7f60b4066470 2023-10-26T05:54:38.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.611+0000 7f60c5ffb640 1 -- 172.21.15.71:0/1183084318 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278822 (secure 0 0 0) 0x7f60d00b5130 con 0x7f60b4066470 2023-10-26T05:54:38.624 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:38.625 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:54:38.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60b4066470 msgr2=0x7f60b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60b4066470 0x7f60b4068930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0046c0 tx=0x7f60cc009040 comp rx=0 tx=0).stop 2023-10-26T05:54:38.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 msgr2=0x7f60d00c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 0x7f60d00c7b10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f60c800efd0 tx=0x7f60c800c6a0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 shutdown_connections 2023-10-26T05:54:38.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f60b4066470 0x7f60b4068930 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60d00d0850 0x7f60d00c7b10 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f60d00d0470 0x7f60d015def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 --2- 172.21.15.71:0/1183084318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60d00cd880 0x7f60d0159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 >> 172.21.15.71:0/1183084318 conn(0x7f60d00cdd80 msgr2=0x7f60d00b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:38.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 shutdown_connections 2023-10-26T05:54:38.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.615+0000 7f60d77d7640 1 -- 172.21.15.71:0/1183084318 wait complete. 2023-10-26T05:54:38.643 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-10-26T05:54:37.556776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554723,"statfs":{"total":1600000000,"available":1592036621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.353828"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218733+0000","last_change":"2023-10-26T05:54:34.218733+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218733+0000","last_undegraded":"2023-10-26T05:54:34.218733+0000","last_fullsized":"2023-10-26T05:54:34.218733+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224954+0000","last_change":"2023-10-26T05:54:34.224954+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224954+0000","last_undegraded":"2023-10-26T05:54:34.224954+0000","last_fullsized":"2023-10-26T05:54:34.224954+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235808+0000","last_change":"2023-10-26T05:54:34.235808+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235808+0000","last_undegraded":"2023-10-26T05:54:34.235808+0000","last_fullsized":"2023-10-26T05:54:34.235808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217369+0000","last_change":"2023-10-26T05:54:34.217369+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217369+0000","last_undegraded":"2023-10-26T05:54:34.217369+0000","last_fullsized":"2023-10-26T05:54:34.217369+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222328+0000","last_change":"2023-10-26T05:54:34.222328+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222328+0000","last_undegraded":"2023-10-26T05:54:34.222328+0000","last_fullsized":"2023-10-26T05:54:34.222328+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219374+0000","last_change":"2023-10-26T05:54:34.219374+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219374+0000","last_undegraded":"2023-10-26T05:54:34.219374+0000","last_fullsized":"2023-10-26T05:54:34.219374+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223861+0000","last_change":"2023-10-26T05:54:34.223861+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223861+0000","last_undegraded":"2023-10-26T05:54:34.223861+0000","last_fullsized":"2023-10-26T05:54:34.223861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235033+0000","last_change":"2023-10-26T05:54:34.235033+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235033+0000","last_undegraded":"2023-10-26T05:54:34.235033+0000","last_fullsized":"2023-10-26T05:54:34.235033+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235409+0000","last_change":"2023-10-26T05:54:34.235409+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235409+0000","last_undegraded":"2023-10-26T05:54:34.235409+0000","last_fullsized":"2023-10-26T05:54:34.235409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222253+0000","last_change":"2023-10-26T05:54:34.222253+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222253+0000","last_undegraded":"2023-10-26T05:54:34.222253+0000","last_fullsized":"2023-10-26T05:54:34.222253+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224745+0000","last_change":"2023-10-26T05:54:34.224745+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224745+0000","last_undegraded":"2023-10-26T05:54:34.224745+0000","last_fullsized":"2023-10-26T05:54:34.224745+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234523+0000","last_change":"2023-10-26T05:54:34.234523+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234523+0000","last_undegraded":"2023-10-26T05:54:34.234523+0000","last_fullsized":"2023-10-26T05:54:34.234523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216288+0000","last_change":"2023-10-26T05:54:34.216288+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216288+0000","last_undegraded":"2023-10-26T05:54:34.216288+0000","last_fullsized":"2023-10-26T05:54:34.216288+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223176+0000","last_change":"2023-10-26T05:54:34.223176+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223176+0000","last_undegraded":"2023-10-26T05:54:34.223176+0000","last_fullsized":"2023-10-26T05:54:34.223176+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218575+0000","last_change":"2023-10-26T05:54:34.218575+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218575+0000","last_undegraded":"2023-10-26T05:54:34.218575+0000","last_fullsized":"2023-10-26T05:54:34.218575+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221557+0000","last_change":"2023-10-26T05:54:34.221557+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221557+0000","last_undegraded":"2023-10-26T05:54:34.221557+0000","last_fullsized":"2023-10-26T05:54:34.221557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223087+0000","last_change":"2023-10-26T05:54:34.223087+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223087+0000","last_undegraded":"2023-10-26T05:54:34.223087+0000","last_fullsized":"2023-10-26T05:54:34.223087+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223894+0000","last_change":"2023-10-26T05:54:34.223894+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223894+0000","last_undegraded":"2023-10-26T05:54:34.223894+0000","last_fullsized":"2023-10-26T05:54:34.223894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225892+0000","last_change":"2023-10-26T05:54:34.225892+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225892+0000","last_undegraded":"2023-10-26T05:54:34.225892+0000","last_fullsized":"2023-10-26T05:54:34.225892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221998+0000","last_change":"2023-10-26T05:54:34.221998+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221998+0000","last_undegraded":"2023-10-26T05:54:34.221998+0000","last_fullsized":"2023-10-26T05:54:34.221998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222347+0000","last_change":"2023-10-26T05:54:34.222347+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222347+0000","last_undegraded":"2023-10-26T05:54:34.222347+0000","last_fullsized":"2023-10-26T05:54:34.222347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225197+0000","last_change":"2023-10-26T05:54:34.225197+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225197+0000","last_undegraded":"2023-10-26T05:54:34.225197+0000","last_fullsized":"2023-10-26T05:54:34.225197+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234208+0000","last_change":"2023-10-26T05:54:34.234208+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234208+0000","last_undegraded":"2023-10-26T05:54:34.234208+0000","last_fullsized":"2023-10-26T05:54:34.234208+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225474+0000","last_change":"2023-10-26T05:54:34.225474+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225474+0000","last_undegraded":"2023-10-26T05:54:34.225474+0000","last_fullsized":"2023-10-26T05:54:34.225474+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221861+0000","last_change":"2023-10-26T05:54:34.221861+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221861+0000","last_undegraded":"2023-10-26T05:54:34.221861+0000","last_fullsized":"2023-10-26T05:54:34.221861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221231+0000","last_change":"2023-10-26T05:54:34.221231+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221231+0000","last_undegraded":"2023-10-26T05:54:34.221231+0000","last_fullsized":"2023-10-26T05:54:34.221231+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217314+0000","last_change":"2023-10-26T05:54:34.217314+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217314+0000","last_undegraded":"2023-10-26T05:54:34.217314+0000","last_fullsized":"2023-10-26T05:54:34.217314+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.242658+0000","last_change":"2023-10-26T05:54:34.242658+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.242658+0000","last_undegraded":"2023-10-26T05:54:34.242658+0000","last_fullsized":"2023-10-26T05:54:34.242658+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216909+0000","last_change":"2023-10-26T05:54:34.216909+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216909+0000","last_undegraded":"2023-10-26T05:54:34.216909+0000","last_fullsized":"2023-10-26T05:54:34.216909+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217829+0000","last_change":"2023-10-26T05:54:34.217829+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217829+0000","last_undegraded":"2023-10-26T05:54:34.217829+0000","last_fullsized":"2023-10-26T05:54:34.217829+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221485+0000","last_change":"2023-10-26T05:54:34.221485+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221485+0000","last_undegraded":"2023-10-26T05:54:34.221485+0000","last_fullsized":"2023-10-26T05:54:34.221485+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222189+0000","last_change":"2023-10-26T05:54:34.222189+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222189+0000","last_undegraded":"2023-10-26T05:54:34.222189+0000","last_fullsized":"2023-10-26T05:54:34.222189+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221314+0000","last_change":"2023-10-26T05:54:34.221314+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221314+0000","last_undegraded":"2023-10-26T05:54:34.221314+0000","last_fullsized":"2023-10-26T05:54:34.221314+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.239407+0000","last_change":"2023-10-26T05:54:34.238405+0000","last_active":"2023-10-26T05:54:34.239407+0000","last_peered":"2023-10-26T05:54:34.239407+0000","last_clean":"2023-10-26T05:54:34.239407+0000","last_became_active":"2023-10-26T05:54:34.234974+0000","last_became_peered":"2023-10-26T05:54:34.234974+0000","last_unstale":"2023-10-26T05:54:34.239407+0000","last_undegraded":"2023-10-26T05:54:34.239407+0000","last_fullsized":"2023-10-26T05:54:34.239407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:49.174594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.233445+0000","last_change":"2023-10-26T05:54:34.233445+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.233445+0000","last_undegraded":"2023-10-26T05:54:34.233445+0000","last_fullsized":"2023-10-26T05:54:34.233445+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238557+0000","last_change":"2023-10-26T05:54:34.238394+0000","last_active":"2023-10-26T05:54:34.238557+0000","last_peered":"2023-10-26T05:54:34.238557+0000","last_clean":"2023-10-26T05:54:34.238557+0000","last_became_active":"2023-10-26T05:54:34.237301+0000","last_became_peered":"2023-10-26T05:54:34.237301+0000","last_unstale":"2023-10-26T05:54:34.238557+0000","last_undegraded":"2023-10-26T05:54:34.238557+0000","last_fullsized":"2023-10-26T05:54:34.238557+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:49:47.037386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220728+0000","last_change":"2023-10-26T05:54:34.220728+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220728+0000","last_undegraded":"2023-10-26T05:54:34.220728+0000","last_fullsized":"2023-10-26T05:54:34.220728+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228507+0000","last_change":"2023-10-26T05:54:34.228217+0000","last_active":"2023-10-26T05:54:34.228507+0000","last_peered":"2023-10-26T05:54:34.228507+0000","last_clean":"2023-10-26T05:54:34.228507+0000","last_became_active":"2023-10-26T05:54:34.227446+0000","last_became_peered":"2023-10-26T05:54:34.227446+0000","last_unstale":"2023-10-26T05:54:34.228507+0000","last_undegraded":"2023-10-26T05:54:34.228507+0000","last_fullsized":"2023-10-26T05:54:34.228507+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:56.388499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221549+0000","last_change":"2023-10-26T05:54:34.221549+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221549+0000","last_undegraded":"2023-10-26T05:54:34.221549+0000","last_fullsized":"2023-10-26T05:54:34.221549+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.224414+0000","last_change":"2023-10-26T05:54:34.219413+0000","last_active":"2023-10-26T05:54:34.224414+0000","last_peered":"2023-10-26T05:54:34.224414+0000","last_clean":"2023-10-26T05:54:34.224414+0000","last_became_active":"2023-10-26T05:54:34.219109+0000","last_became_peered":"2023-10-26T05:54:34.219109+0000","last_unstale":"2023-10-26T05:54:34.224414+0000","last_undegraded":"2023-10-26T05:54:34.224414+0000","last_fullsized":"2023-10-26T05:54:34.224414+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:04:02.823811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223568+0000","last_change":"2023-10-26T05:54:34.223568+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223568+0000","last_undegraded":"2023-10-26T05:54:34.223568+0000","last_fullsized":"2023-10-26T05:54:34.223568+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.225568+0000","last_change":"2023-10-26T05:54:34.225466+0000","last_active":"2023-10-26T05:54:34.225568+0000","last_peered":"2023-10-26T05:54:34.225568+0000","last_clean":"2023-10-26T05:54:34.225568+0000","last_became_active":"2023-10-26T05:54:34.225226+0000","last_became_peered":"2023-10-26T05:54:34.225226+0000","last_unstale":"2023-10-26T05:54:34.225568+0000","last_undegraded":"2023-10-26T05:54:34.225568+0000","last_fullsized":"2023-10-26T05:54:34.225568+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:09:54.209739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221407+0000","last_change":"2023-10-26T05:54:34.221407+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221407+0000","last_undegraded":"2023-10-26T05:54:34.221407+0000","last_fullsized":"2023-10-26T05:54:34.221407+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.231347+0000","last_change":"2023-10-26T05:54:34.231290+0000","last_active":"2023-10-26T05:54:34.231347+0000","last_peered":"2023-10-26T05:54:34.231347+0000","last_clean":"2023-10-26T05:54:34.231347+0000","last_became_active":"2023-10-26T05:54:34.230708+0000","last_became_peered":"2023-10-26T05:54:34.230708+0000","last_unstale":"2023-10-26T05:54:34.231347+0000","last_undegraded":"2023-10-26T05:54:34.231347+0000","last_fullsized":"2023-10-26T05:54:34.231347+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:45:26.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.236168+0000","last_change":"2023-10-26T05:54:34.236168+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.236168+0000","last_undegraded":"2023-10-26T05:54:34.236168+0000","last_fullsized":"2023-10-26T05:54:34.236168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.229222+0000","last_change":"2023-10-26T05:54:34.229066+0000","last_active":"2023-10-26T05:54:34.229222+0000","last_peered":"2023-10-26T05:54:34.229222+0000","last_clean":"2023-10-26T05:54:34.229222+0000","last_became_active":"2023-10-26T05:54:34.228464+0000","last_became_peered":"2023-10-26T05:54:34.228464+0000","last_unstale":"2023-10-26T05:54:34.229222+0000","last_undegraded":"2023-10-26T05:54:34.229222+0000","last_fullsized":"2023-10-26T05:54:34.229222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:39.551427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222740+0000","last_change":"2023-10-26T05:54:34.222740+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222740+0000","last_undegraded":"2023-10-26T05:54:34.222740+0000","last_fullsized":"2023-10-26T05:54:34.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237238+0000","last_change":"2023-10-26T05:54:34.237160+0000","last_active":"2023-10-26T05:54:34.237238+0000","last_peered":"2023-10-26T05:54:34.237238+0000","last_clean":"2023-10-26T05:54:34.237238+0000","last_became_active":"2023-10-26T05:54:34.236869+0000","last_became_peered":"2023-10-26T05:54:34.236869+0000","last_unstale":"2023-10-26T05:54:34.237238+0000","last_undegraded":"2023-10-26T05:54:34.237238+0000","last_fullsized":"2023-10-26T05:54:34.237238+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:15.255164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217012+0000","last_change":"2023-10-26T05:54:34.217012+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217012+0000","last_undegraded":"2023-10-26T05:54:34.217012+0000","last_fullsized":"2023-10-26T05:54:34.217012+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228303+0000","last_change":"2023-10-26T05:54:34.227954+0000","last_active":"2023-10-26T05:54:34.228303+0000","last_peered":"2023-10-26T05:54:34.228303+0000","last_clean":"2023-10-26T05:54:34.228303+0000","last_became_active":"2023-10-26T05:54:34.227315+0000","last_became_peered":"2023-10-26T05:54:34.227315+0000","last_unstale":"2023-10-26T05:54:34.228303+0000","last_undegraded":"2023-10-26T05:54:34.228303+0000","last_fullsized":"2023-10-26T05:54:34.228303+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:36:29.297709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.244373+0000","last_change":"2023-10-26T05:54:34.244373+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.244373+0000","last_undegraded":"2023-10-26T05:54:34.244373+0000","last_fullsized":"2023-10-26T05:54:34.244373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228819+0000","last_change":"2023-10-26T05:54:34.228604+0000","last_active":"2023-10-26T05:54:34.228819+0000","last_peered":"2023-10-26T05:54:34.228819+0000","last_clean":"2023-10-26T05:54:34.228819+0000","last_became_active":"2023-10-26T05:54:34.228363+0000","last_became_peered":"2023-10-26T05:54:34.228363+0000","last_unstale":"2023-10-26T05:54:34.228819+0000","last_undegraded":"2023-10-26T05:54:34.228819+0000","last_fullsized":"2023-10-26T05:54:34.228819+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:08:08.211694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224664+0000","last_change":"2023-10-26T05:54:34.224664+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224664+0000","last_undegraded":"2023-10-26T05:54:34.224664+0000","last_fullsized":"2023-10-26T05:54:34.224664+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.241631+0000","last_change":"2023-10-26T05:54:34.241288+0000","last_active":"2023-10-26T05:54:34.241631+0000","last_peered":"2023-10-26T05:54:34.241631+0000","last_clean":"2023-10-26T05:54:34.241631+0000","last_became_active":"2023-10-26T05:54:34.240585+0000","last_became_peered":"2023-10-26T05:54:34.240585+0000","last_unstale":"2023-10-26T05:54:34.241631+0000","last_undegraded":"2023-10-26T05:54:34.241631+0000","last_fullsized":"2023-10-26T05:54:34.241631+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:17:51.315680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222024+0000","last_change":"2023-10-26T05:54:34.222024+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222024+0000","last_undegraded":"2023-10-26T05:54:34.222024+0000","last_fullsized":"2023-10-26T05:54:34.222024+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.246204+0000","last_change":"2023-10-26T05:54:34.245988+0000","last_active":"2023-10-26T05:54:34.246204+0000","last_peered":"2023-10-26T05:54:34.246204+0000","last_clean":"2023-10-26T05:54:34.246204+0000","last_became_active":"2023-10-26T05:54:34.245363+0000","last_became_peered":"2023-10-26T05:54:34.245363+0000","last_unstale":"2023-10-26T05:54:34.246204+0000","last_undegraded":"2023-10-26T05:54:34.246204+0000","last_fullsized":"2023-10-26T05:54:34.246204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:10:53.581294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216915+0000","last_change":"2023-10-26T05:54:34.216915+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216915+0000","last_undegraded":"2023-10-26T05:54:34.216915+0000","last_fullsized":"2023-10-26T05:54:34.216915+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226894+0000","last_change":"2023-10-26T05:54:34.226642+0000","last_active":"2023-10-26T05:54:34.226894+0000","last_peered":"2023-10-26T05:54:34.226894+0000","last_clean":"2023-10-26T05:54:34.226894+0000","last_became_active":"2023-10-26T05:54:34.226063+0000","last_became_peered":"2023-10-26T05:54:34.226063+0000","last_unstale":"2023-10-26T05:54:34.226894+0000","last_undegraded":"2023-10-26T05:54:34.226894+0000","last_fullsized":"2023-10-26T05:54:34.226894+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:29.737511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224031+0000","last_change":"2023-10-26T05:54:34.224031+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224031+0000","last_undegraded":"2023-10-26T05:54:34.224031+0000","last_fullsized":"2023-10-26T05:54:34.224031+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.230277+0000","last_change":"2023-10-26T05:54:34.230187+0000","last_active":"2023-10-26T05:54:34.230277+0000","last_peered":"2023-10-26T05:54:34.230277+0000","last_clean":"2023-10-26T05:54:34.230277+0000","last_became_active":"2023-10-26T05:54:34.229848+0000","last_became_peered":"2023-10-26T05:54:34.229848+0000","last_unstale":"2023-10-26T05:54:34.230277+0000","last_undegraded":"2023-10-26T05:54:34.230277+0000","last_fullsized":"2023-10-26T05:54:34.230277+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:50.756897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.233720+0000","last_change":"2023-10-26T05:54:34.233720+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.233720+0000","last_undegraded":"2023-10-26T05:54:34.233720+0000","last_fullsized":"2023-10-26T05:54:34.233720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.242840+0000","last_change":"2023-10-26T05:54:34.242272+0000","last_active":"2023-10-26T05:54:34.242840+0000","last_peered":"2023-10-26T05:54:34.242840+0000","last_clean":"2023-10-26T05:54:34.242840+0000","last_became_active":"2023-10-26T05:54:34.240219+0000","last_became_peered":"2023-10-26T05:54:34.240219+0000","last_unstale":"2023-10-26T05:54:34.242840+0000","last_undegraded":"2023-10-26T05:54:34.242840+0000","last_fullsized":"2023-10-26T05:54:34.242840+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:23.700023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234009+0000","last_change":"2023-10-26T05:54:34.234009+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234009+0000","last_undegraded":"2023-10-26T05:54:34.234009+0000","last_fullsized":"2023-10-26T05:54:34.234009+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238519+0000","last_change":"2023-10-26T05:54:34.237939+0000","last_active":"2023-10-26T05:54:34.238519+0000","last_peered":"2023-10-26T05:54:34.238519+0000","last_clean":"2023-10-26T05:54:34.238519+0000","last_became_active":"2023-10-26T05:54:34.236275+0000","last_became_peered":"2023-10-26T05:54:34.236275+0000","last_unstale":"2023-10-26T05:54:34.238519+0000","last_undegraded":"2023-10-26T05:54:34.238519+0000","last_fullsized":"2023-10-26T05:54:34.238519+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:12:15.727069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.242318+0000","last_change":"2023-10-26T05:54:34.242318+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.242318+0000","last_undegraded":"2023-10-26T05:54:34.242318+0000","last_fullsized":"2023-10-26T05:54:34.242318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238966+0000","last_change":"2023-10-26T05:54:34.238890+0000","last_active":"2023-10-26T05:54:34.238966+0000","last_peered":"2023-10-26T05:54:34.238966+0000","last_clean":"2023-10-26T05:54:34.238966+0000","last_became_active":"2023-10-26T05:54:34.238622+0000","last_became_peered":"2023-10-26T05:54:34.238622+0000","last_unstale":"2023-10-26T05:54:34.238966+0000","last_undegraded":"2023-10-26T05:54:34.238966+0000","last_fullsized":"2023-10-26T05:54:34.238966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:26:48.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222516+0000","last_change":"2023-10-26T05:54:34.222516+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222516+0000","last_undegraded":"2023-10-26T05:54:34.222516+0000","last_fullsized":"2023-10-26T05:54:34.222516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.240565+0000","last_change":"2023-10-26T05:54:34.239412+0000","last_active":"2023-10-26T05:54:34.240565+0000","last_peered":"2023-10-26T05:54:34.240565+0000","last_clean":"2023-10-26T05:54:34.240565+0000","last_became_active":"2023-10-26T05:54:34.238306+0000","last_became_peered":"2023-10-26T05:54:34.238306+0000","last_unstale":"2023-10-26T05:54:34.240565+0000","last_undegraded":"2023-10-26T05:54:34.240565+0000","last_fullsized":"2023-10-26T05:54:34.240565+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:09.950751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.226710+0000","last_change":"2023-10-26T05:54:34.226710+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.226710+0000","last_undegraded":"2023-10-26T05:54:34.226710+0000","last_fullsized":"2023-10-26T05:54:34.226710+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237616+0000","last_change":"2023-10-26T05:54:34.237449+0000","last_active":"2023-10-26T05:54:34.237616+0000","last_peered":"2023-10-26T05:54:34.237616+0000","last_clean":"2023-10-26T05:54:34.237616+0000","last_became_active":"2023-10-26T05:54:34.236646+0000","last_became_peered":"2023-10-26T05:54:34.236646+0000","last_unstale":"2023-10-26T05:54:34.237616+0000","last_undegraded":"2023-10-26T05:54:34.237616+0000","last_fullsized":"2023-10-26T05:54:34.237616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:09.282179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216456+0000","last_change":"2023-10-26T05:54:34.216456+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216456+0000","last_undegraded":"2023-10-26T05:54:34.216456+0000","last_fullsized":"2023-10-26T05:54:34.216456+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.230169+0000","last_change":"2023-10-26T05:54:34.229691+0000","last_active":"2023-10-26T05:54:34.230169+0000","last_peered":"2023-10-26T05:54:34.230169+0000","last_clean":"2023-10-26T05:54:34.230169+0000","last_became_active":"2023-10-26T05:54:34.229093+0000","last_became_peered":"2023-10-26T05:54:34.229093+0000","last_unstale":"2023-10-26T05:54:34.230169+0000","last_undegraded":"2023-10-26T05:54:34.230169+0000","last_fullsized":"2023-10-26T05:54:34.230169+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:21.281518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219762+0000","last_change":"2023-10-26T05:54:34.219762+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219762+0000","last_undegraded":"2023-10-26T05:54:34.219762+0000","last_fullsized":"2023-10-26T05:54:34.219762+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.231966+0000","last_change":"2023-10-26T05:54:34.231906+0000","last_active":"2023-10-26T05:54:34.231966+0000","last_peered":"2023-10-26T05:54:34.231966+0000","last_clean":"2023-10-26T05:54:34.231966+0000","last_became_active":"2023-10-26T05:54:34.231677+0000","last_became_peered":"2023-10-26T05:54:34.231677+0000","last_unstale":"2023-10-26T05:54:34.231966+0000","last_undegraded":"2023-10-26T05:54:34.231966+0000","last_fullsized":"2023-10-26T05:54:34.231966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:31.782805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.226387+0000","last_change":"2023-10-26T05:54:34.226387+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.226387+0000","last_undegraded":"2023-10-26T05:54:34.226387+0000","last_fullsized":"2023-10-26T05:54:34.226387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.236138+0000","last_change":"2023-10-26T05:54:34.236055+0000","last_active":"2023-10-26T05:54:34.236138+0000","last_peered":"2023-10-26T05:54:34.236138+0000","last_clean":"2023-10-26T05:54:34.236138+0000","last_became_active":"2023-10-26T05:54:34.235715+0000","last_became_peered":"2023-10-26T05:54:34.235715+0000","last_unstale":"2023-10-26T05:54:34.236138+0000","last_undegraded":"2023-10-26T05:54:34.236138+0000","last_fullsized":"2023-10-26T05:54:34.236138+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:51:26.940485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220755+0000","last_change":"2023-10-26T05:54:34.220755+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220755+0000","last_undegraded":"2023-10-26T05:54:34.220755+0000","last_fullsized":"2023-10-26T05:54:34.220755+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.235237+0000","last_change":"2023-10-26T05:54:34.235155+0000","last_active":"2023-10-26T05:54:34.235237+0000","last_peered":"2023-10-26T05:54:34.235237+0000","last_clean":"2023-10-26T05:54:34.235237+0000","last_became_active":"2023-10-26T05:54:34.234905+0000","last_became_peered":"2023-10-26T05:54:34.234905+0000","last_unstale":"2023-10-26T05:54:34.235237+0000","last_undegraded":"2023-10-26T05:54:34.235237+0000","last_fullsized":"2023-10-26T05:54:34.235237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:33.288741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218889+0000","last_change":"2023-10-26T05:54:34.218889+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218889+0000","last_undegraded":"2023-10-26T05:54:34.218889+0000","last_fullsized":"2023-10-26T05:54:34.218889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226868+0000","last_change":"2023-10-26T05:54:34.226691+0000","last_active":"2023-10-26T05:54:34.226868+0000","last_peered":"2023-10-26T05:54:34.226868+0000","last_clean":"2023-10-26T05:54:34.226868+0000","last_became_active":"2023-10-26T05:54:34.225872+0000","last_became_peered":"2023-10-26T05:54:34.225872+0000","last_unstale":"2023-10-26T05:54:34.226868+0000","last_undegraded":"2023-10-26T05:54:34.226868+0000","last_fullsized":"2023-10-26T05:54:34.226868+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:37.373754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216671+0000","last_change":"2023-10-26T05:54:34.216671+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216671+0000","last_undegraded":"2023-10-26T05:54:34.216671+0000","last_fullsized":"2023-10-26T05:54:34.216671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.239361+0000","last_change":"2023-10-26T05:54:34.235620+0000","last_active":"2023-10-26T05:54:34.239361+0000","last_peered":"2023-10-26T05:54:34.239361+0000","last_clean":"2023-10-26T05:54:34.239361+0000","last_became_active":"2023-10-26T05:54:34.231181+0000","last_became_peered":"2023-10-26T05:54:34.231181+0000","last_unstale":"2023-10-26T05:54:34.239361+0000","last_undegraded":"2023-10-26T05:54:34.239361+0000","last_fullsized":"2023-10-26T05:54:34.239361+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:12.750236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221409+0000","last_change":"2023-10-26T05:54:34.221409+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221409+0000","last_undegraded":"2023-10-26T05:54:34.221409+0000","last_fullsized":"2023-10-26T05:54:34.221409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.234344+0000","last_change":"2023-10-26T05:54:34.234275+0000","last_active":"2023-10-26T05:54:34.234344+0000","last_peered":"2023-10-26T05:54:34.234344+0000","last_clean":"2023-10-26T05:54:34.234344+0000","last_became_active":"2023-10-26T05:54:34.234042+0000","last_became_peered":"2023-10-26T05:54:34.234042+0000","last_unstale":"2023-10-26T05:54:34.234344+0000","last_undegraded":"2023-10-26T05:54:34.234344+0000","last_fullsized":"2023-10-26T05:54:34.234344+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:50:58.640410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218977+0000","last_change":"2023-10-26T05:54:34.218977+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218977+0000","last_undegraded":"2023-10-26T05:54:34.218977+0000","last_fullsized":"2023-10-26T05:54:34.218977+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228533+0000","last_change":"2023-10-26T05:54:34.228407+0000","last_active":"2023-10-26T05:54:34.228533+0000","last_peered":"2023-10-26T05:54:34.228533+0000","last_clean":"2023-10-26T05:54:34.228533+0000","last_became_active":"2023-10-26T05:54:34.227889+0000","last_became_peered":"2023-10-26T05:54:34.227889+0000","last_unstale":"2023-10-26T05:54:34.228533+0000","last_undegraded":"2023-10-26T05:54:34.228533+0000","last_fullsized":"2023-10-26T05:54:34.228533+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:42:05.162676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219774+0000","last_change":"2023-10-26T05:54:34.219774+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219774+0000","last_undegraded":"2023-10-26T05:54:34.219774+0000","last_fullsized":"2023-10-26T05:54:34.219774+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.227621+0000","last_change":"2023-10-26T05:54:34.227337+0000","last_active":"2023-10-26T05:54:34.227621+0000","last_peered":"2023-10-26T05:54:34.227621+0000","last_clean":"2023-10-26T05:54:34.227621+0000","last_became_active":"2023-10-26T05:54:34.227083+0000","last_became_peered":"2023-10-26T05:54:34.227083+0000","last_unstale":"2023-10-26T05:54:34.227621+0000","last_undegraded":"2023-10-26T05:54:34.227621+0000","last_fullsized":"2023-10-26T05:54:34.227621+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:21.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.243298+0000","last_change":"2023-10-26T05:54:34.243298+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.243298+0000","last_undegraded":"2023-10-26T05:54:34.243298+0000","last_fullsized":"2023-10-26T05:54:34.243298+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237935+0000","last_change":"2023-10-26T05:54:34.236735+0000","last_active":"2023-10-26T05:54:34.237935+0000","last_peered":"2023-10-26T05:54:34.237935+0000","last_clean":"2023-10-26T05:54:34.237935+0000","last_became_active":"2023-10-26T05:54:34.233447+0000","last_became_peered":"2023-10-26T05:54:34.233447+0000","last_unstale":"2023-10-26T05:54:34.237935+0000","last_undegraded":"2023-10-26T05:54:34.237935+0000","last_fullsized":"2023-10-26T05:54:34.237935+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:14.747480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220883+0000","last_change":"2023-10-26T05:54:34.220883+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220883+0000","last_undegraded":"2023-10-26T05:54:34.220883+0000","last_fullsized":"2023-10-26T05:54:34.220883+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.234328+0000","last_change":"2023-10-26T05:54:34.234238+0000","last_active":"2023-10-26T05:54:34.234328+0000","last_peered":"2023-10-26T05:54:34.234328+0000","last_clean":"2023-10-26T05:54:34.234328+0000","last_became_active":"2023-10-26T05:54:34.234006+0000","last_became_peered":"2023-10-26T05:54:34.234006+0000","last_unstale":"2023-10-26T05:54:34.234328+0000","last_undegraded":"2023-10-26T05:54:34.234328+0000","last_fullsized":"2023-10-26T05:54:34.234328+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:45.551886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223205+0000","last_change":"2023-10-26T05:54:34.223205+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223205+0000","last_undegraded":"2023-10-26T05:54:34.223205+0000","last_fullsized":"2023-10-26T05:54:34.223205+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226797+0000","last_change":"2023-10-26T05:54:34.226096+0000","last_active":"2023-10-26T05:54:34.226797+0000","last_peered":"2023-10-26T05:54:34.226797+0000","last_clean":"2023-10-26T05:54:34.226797+0000","last_became_active":"2023-10-26T05:54:34.224494+0000","last_became_peered":"2023-10-26T05:54:34.224494+0000","last_unstale":"2023-10-26T05:54:34.226797+0000","last_undegraded":"2023-10-26T05:54:34.226797+0000","last_fullsized":"2023-10-26T05:54:34.226797+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:09.983718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219971+0000","last_change":"2023-10-26T05:54:34.219971+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219971+0000","last_undegraded":"2023-10-26T05:54:34.219971+0000","last_fullsized":"2023-10-26T05:54:34.219971+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.243333+0000","last_change":"2023-10-26T05:54:34.241834+0000","last_active":"2023-10-26T05:54:34.243333+0000","last_peered":"2023-10-26T05:54:34.243333+0000","last_clean":"2023-10-26T05:54:34.243333+0000","last_became_active":"2023-10-26T05:54:34.232265+0000","last_became_peered":"2023-10-26T05:54:34.232265+0000","last_unstale":"2023-10-26T05:54:34.243333+0000","last_undegraded":"2023-10-26T05:54:34.243333+0000","last_fullsized":"2023-10-26T05:54:34.243333+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:19.460844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-26T05:54:33.207194+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:54:33.207194+0000","last_peered":"2023-10-26T05:54:33.207194+0000","last_clean":"2023-10-26T05:54:33.207194+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:54:33.207194+0000","last_undegraded":"2023-10-26T05:54:33.207194+0000","last_fullsized":"2023-10-26T05:54:33.207194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1409,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193902,"statfs":{"total":200000000,"available":198556311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.39600000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.169,"5min":1.286,"15min":1.3100000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94799999999999995,"15min":1.018},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.236,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.93100000000000005},"min":{"1min":0.67100000000000004,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.2849999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69199999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.214,"15min":1.4330000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3129999999999999,"5min":1.375,"15min":1.375},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.1679999999999999,"5min":1.639,"15min":1.639},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96399999999999997,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3149999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.877}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0169999999999999,"15min":1.1000000000000001},"min":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.58699999999999997},"max":{"1min":1.266,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0720000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.53800000000000003},"max":{"1min":1.3029999999999999,"5min":1.702,"15min":1.702},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.081,"15min":1.1100000000000001},"min":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.59599999999999997},"max":{"1min":1.3640000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.0609999999999999,"15min":1.0509999999999999},"min":{"1min":0.66700000000000004,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.6180000000000001},"last":1.0740000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.298,"5min":1.298,"15min":1.3},"last":1.0660000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.62,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.405},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.82099999999999995},"min":{"1min":0.54800000000000004,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1990000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.435,"15min":0.435},"max":{"1min":1.3,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.621}]},{"osd":3,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97999999999999998,"15min":0.90400000000000003},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2250000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.056,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.3700000000000001,"5min":1.399,"15min":1.399},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.85699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.002,"5min":0.97799999999999998,"15min":0.97899999999999998},"min":{"1min":0.64500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.351,"5min":1.375,"15min":1.375},"last":1.143}]},{"osd":5,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0469999999999999},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.76800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.25,"5min":1.512,"15min":1.512},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1060000000000001},"min":{"1min":0.96799999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4550000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.92100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":0.94999999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.381,"5min":1.579,"15min":1.579},"last":0.879},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.53300000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.089,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.75},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.464,"15min":1.464},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.2110000000000001,"5min":1.319,"15min":1.5089999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.92700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2190000000000001,"5min":1.274,"15min":1.3160000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.59299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.333,"5min":1.448,"15min":1.548},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.238,"5min":1.4139999999999999,"15min":1.587},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94099999999999995,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2769999999999999,"5min":1.286,"15min":2.1909999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.091,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.536,"15min":1.536},"last":1.117}]},{"osd":6,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.96899999999999997,"15min":1.018},"min":{"1min":0.63500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0169999999999999,"15min":1.032},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.4199999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.442,"15min":0.442},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.155}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.85999999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.248,"5min":1.2569999999999999,"15min":1.347},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.53100000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.143,"5min":1.502,"15min":1.502},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.878,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.25,"15min":1.375},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.96199999999999997,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":1.3440000000000001,"15min":1.4390000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90700000000000003,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.107,"5min":1.373,"15min":1.373},"last":0.877}]},{"osd":3,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.397,"5min":1.397,"15min":2.0190000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.71099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.232,"5min":1.3300000000000001,"15min":1.4359999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.88900000000000001},"min":{"1min":0.65600000000000003,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3440000000000001,"15min":1.3859999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.323,"5min":1.4179999999999999,"15min":1.673},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0289999999999999,"15min":1.004},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.42099999999999999},"max":{"1min":1.762,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.73799999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3540000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.68600000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.7509999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.137,"15min":1.119},"min":{"1min":0.74299999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"max":{"1min":1.484,"5min":1.728,"15min":1.728},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.29599999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.76800000000000002,"5min":1.385,"15min":1.385},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.94699999999999995,"5min":1.115,"15min":1.2689999999999999},"last":0.309}]},{"osd":1,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80100000000000005,"5min":1.349,"15min":1.349},"last":0.373},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65900000000000003,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.99299999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":1.1619999999999999,"5min":1.236,"15min":1.4199999999999999},"last":0.498}]},{"osd":4,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":0.83499999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.40000000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.073,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.95599999999999996},"min":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55800000000000005},"max":{"1min":1.113,"5min":1.413,"15min":1.478},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.91200000000000003},"min":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.59799999999999998},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93300000000000005,"15min":0.97999999999999998},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.165,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.254,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":0.93200000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.232,"5min":2.004,"15min":2.004},"last":0.81999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.54400000000000004},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.88300000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71799999999999997,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1120000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.67600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.198,"5min":1.226,"15min":1.2609999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68999999999999995,"15min":0.60199999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.157,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.74399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.149,"5min":1.333,"15min":2.8809999999999998},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.089,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.83899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1060000000000001,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.749},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.54400000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1459999999999999,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.91900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.476,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.052},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.99299999999999999,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.653,"15min":1.653},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.002,"15min":0.92700000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.077,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.419,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.135}]},{"osd":7,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.97799999999999998,"15min":0.91000000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.173,"5min":1.581,"15min":1.581},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0900000000000001,"15min":1.0449999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.367,"5min":1.823,"15min":15.352},"last":1.2130000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.157,"15min":1.157},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83599999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27700000000000002},"max":{"1min":1.087,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.378}]},{"osd":3,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.89500000000000002,"5min":1.298,"15min":1.298},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.64500000000000002,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1659999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.27,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.621,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3620000000000001,"5min":1.492,"15min":1.492},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.92400000000000004,"15min":0.92900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3029999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96699999999999997},"min":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.468,"15min":1.837},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.012,"15min":1.022},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.2330000000000001,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.046,"15min":1.0840000000000001},"min":{"1min":0.63200000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4099999999999999,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193908,"statfs":{"total":200000000,"available":198562586,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78100000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.69899999999999995,"15min":0.62},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85899999999999999,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.093,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.437},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1140000000000001,"15min":1.177},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.626},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.99299999999999999,"5min":1.409,"15min":1.409},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80500000000000005,"15min":0.70999999999999996},"min":{"1min":0.33400000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.2629999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.495}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.92100000000000004},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2769999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1399999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.90900000000000003},"min":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.123,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.004,"15min":0.996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4399999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":0.98999999999999999},"min":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.78300000000000003}]}]}],"pool_statfs":[]}} 2023-10-26T05:54:38.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:54:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- 172.21.15.71:0/1683628207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 msgr2=0x7fa950162840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1683628207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950162840 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa948009bc0 tx=0x7fa94801cb40 comp rx=0 tx=0).stop 2023-10-26T05:54:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- 172.21.15.71:0/1683628207 shutdown_connections 2023-10-26T05:54:38.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1683628207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950162840 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- 172.21.15.71:0/1683628207 >> 172.21.15.71:0/1683628207 conn(0x7fa950069770 msgr2=0x7fa950069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:38.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- 172.21.15.71:0/1683628207 shutdown_connections 2023-10-26T05:54:38.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- 172.21.15.71:0/1683628207 wait complete. 2023-10-26T05:54:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 Processor -- start 2023-10-26T05:54:38.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- start start 2023-10-26T05:54:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9500ac740 0x7fa950134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa95013d420 0x7fa950158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950148dd0 con 0x7fa95015bbc0 2023-10-26T05:54:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950134680 con 0x7fa9500ac740 2023-10-26T05:54:38.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa9587aa640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa950134b10 con 0x7fa95013d420 2023-10-26T05:54:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39846/0 (socket says 172.21.15.71:39846) 2023-10-26T05:54:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 -- 172.21.15.71:0/1159781044 learned_addr learned my addr 172.21.15.71:0/1159781044 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa955d45640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa95013d420 0x7fa950158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 -- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa95013d420 msgr2=0x7fa950158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa95013d420 0x7fa950158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956546640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9500ac740 0x7fa950134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 -- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9500ac740 msgr2=0x7fa950134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9500ac740 0x7fa950134140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 -- 172.21.15.71:0/1159781044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948009820 con 0x7fa95015bbc0 2023-10-26T05:54:38.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956546640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9500ac740 0x7fa950134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:38.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.803+0000 7fa956d47640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950159530 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa948009b10 tx=0x7fa9480a5b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:38.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948003b80 con 0x7fa95015bbc0 2023-10-26T05:54:38.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa948003dd0 con 0x7fa95015bbc0 2023-10-26T05:54:38.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95014b2e0 con 0x7fa95015bbc0 2023-10-26T05:54:38.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948004290 con 0x7fa95015bbc0 2023-10-26T05:54:38.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950148710 con 0x7fa95015bbc0 2023-10-26T05:54:38.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa950066b10 con 0x7fa95015bbc0 2023-10-26T05:54:38.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa9480043f0 con 0x7fa95015bbc0 2023-10-26T05:54:38.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa946ffd640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9340664a0 0x7fa934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:38.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa94811f330 con 0x7fa95015bbc0 2023-10-26T05:54:38.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa956546640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9340664a0 0x7fa934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:38.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.807+0000 7fa956546640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9340664a0 0x7fa934068960 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa95015ac50 tx=0x7fa94c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:38.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.811+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa9480ec840 con 0x7fa95015bbc0 2023-10-26T05:54:38.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.959+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa950148f10 con 0x7fa9340664a0 2023-10-26T05:54:38.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.983+0000 7fa946ffd640 1 -- 172.21.15.71:0/1159781044 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278822 (secure 0 0 0) 0x7fa950148f10 con 0x7fa9340664a0 2023-10-26T05:54:38.993 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:38.994 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:54:38.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9340664a0 msgr2=0x7fa934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9340664a0 0x7fa934068960 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa95015ac50 tx=0x7fa94c009040 comp rx=0 tx=0).stop 2023-10-26T05:54:38.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 msgr2=0x7fa950159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:38.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950159530 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa948009b10 tx=0x7fa9480a5b00 comp rx=0 tx=0).stop 2023-10-26T05:54:38.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 shutdown_connections 2023-10-26T05:54:38.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa9340664a0 0x7fa934068960 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa95015bbc0 0x7fa950159530 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa95013d420 0x7fa950158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 --2- 172.21.15.71:0/1159781044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9500ac740 0x7fa950134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:38.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 >> 172.21.15.71:0/1159781044 conn(0x7fa950069770 msgr2=0x7fa9500cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:38.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 shutdown_connections 2023-10-26T05:54:38.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:38.987+0000 7fa9587aa640 1 -- 172.21.15.71:0/1159781044 wait complete. 2023-10-26T05:54:39.011 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2023-10-26T05:54:37.556776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7776,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554723,"statfs":{"total":1600000000,"available":1592036621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.353828"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218733+0000","last_change":"2023-10-26T05:54:34.218733+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218733+0000","last_undegraded":"2023-10-26T05:54:34.218733+0000","last_fullsized":"2023-10-26T05:54:34.218733+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224954+0000","last_change":"2023-10-26T05:54:34.224954+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224954+0000","last_undegraded":"2023-10-26T05:54:34.224954+0000","last_fullsized":"2023-10-26T05:54:34.224954+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235808+0000","last_change":"2023-10-26T05:54:34.235808+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235808+0000","last_undegraded":"2023-10-26T05:54:34.235808+0000","last_fullsized":"2023-10-26T05:54:34.235808+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217369+0000","last_change":"2023-10-26T05:54:34.217369+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217369+0000","last_undegraded":"2023-10-26T05:54:34.217369+0000","last_fullsized":"2023-10-26T05:54:34.217369+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222328+0000","last_change":"2023-10-26T05:54:34.222328+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222328+0000","last_undegraded":"2023-10-26T05:54:34.222328+0000","last_fullsized":"2023-10-26T05:54:34.222328+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219374+0000","last_change":"2023-10-26T05:54:34.219374+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219374+0000","last_undegraded":"2023-10-26T05:54:34.219374+0000","last_fullsized":"2023-10-26T05:54:34.219374+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223861+0000","last_change":"2023-10-26T05:54:34.223861+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223861+0000","last_undegraded":"2023-10-26T05:54:34.223861+0000","last_fullsized":"2023-10-26T05:54:34.223861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235033+0000","last_change":"2023-10-26T05:54:34.235033+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235033+0000","last_undegraded":"2023-10-26T05:54:34.235033+0000","last_fullsized":"2023-10-26T05:54:34.235033+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.235409+0000","last_change":"2023-10-26T05:54:34.235409+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.235409+0000","last_undegraded":"2023-10-26T05:54:34.235409+0000","last_fullsized":"2023-10-26T05:54:34.235409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222253+0000","last_change":"2023-10-26T05:54:34.222253+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222253+0000","last_undegraded":"2023-10-26T05:54:34.222253+0000","last_fullsized":"2023-10-26T05:54:34.222253+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224745+0000","last_change":"2023-10-26T05:54:34.224745+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224745+0000","last_undegraded":"2023-10-26T05:54:34.224745+0000","last_fullsized":"2023-10-26T05:54:34.224745+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234523+0000","last_change":"2023-10-26T05:54:34.234523+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234523+0000","last_undegraded":"2023-10-26T05:54:34.234523+0000","last_fullsized":"2023-10-26T05:54:34.234523+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216288+0000","last_change":"2023-10-26T05:54:34.216288+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216288+0000","last_undegraded":"2023-10-26T05:54:34.216288+0000","last_fullsized":"2023-10-26T05:54:34.216288+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223176+0000","last_change":"2023-10-26T05:54:34.223176+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223176+0000","last_undegraded":"2023-10-26T05:54:34.223176+0000","last_fullsized":"2023-10-26T05:54:34.223176+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218575+0000","last_change":"2023-10-26T05:54:34.218575+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218575+0000","last_undegraded":"2023-10-26T05:54:34.218575+0000","last_fullsized":"2023-10-26T05:54:34.218575+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221557+0000","last_change":"2023-10-26T05:54:34.221557+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221557+0000","last_undegraded":"2023-10-26T05:54:34.221557+0000","last_fullsized":"2023-10-26T05:54:34.221557+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223087+0000","last_change":"2023-10-26T05:54:34.223087+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223087+0000","last_undegraded":"2023-10-26T05:54:34.223087+0000","last_fullsized":"2023-10-26T05:54:34.223087+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223894+0000","last_change":"2023-10-26T05:54:34.223894+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223894+0000","last_undegraded":"2023-10-26T05:54:34.223894+0000","last_fullsized":"2023-10-26T05:54:34.223894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225892+0000","last_change":"2023-10-26T05:54:34.225892+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225892+0000","last_undegraded":"2023-10-26T05:54:34.225892+0000","last_fullsized":"2023-10-26T05:54:34.225892+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221998+0000","last_change":"2023-10-26T05:54:34.221998+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221998+0000","last_undegraded":"2023-10-26T05:54:34.221998+0000","last_fullsized":"2023-10-26T05:54:34.221998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222347+0000","last_change":"2023-10-26T05:54:34.222347+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222347+0000","last_undegraded":"2023-10-26T05:54:34.222347+0000","last_fullsized":"2023-10-26T05:54:34.222347+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225197+0000","last_change":"2023-10-26T05:54:34.225197+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225197+0000","last_undegraded":"2023-10-26T05:54:34.225197+0000","last_fullsized":"2023-10-26T05:54:34.225197+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234208+0000","last_change":"2023-10-26T05:54:34.234208+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234208+0000","last_undegraded":"2023-10-26T05:54:34.234208+0000","last_fullsized":"2023-10-26T05:54:34.234208+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.225474+0000","last_change":"2023-10-26T05:54:34.225474+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.225474+0000","last_undegraded":"2023-10-26T05:54:34.225474+0000","last_fullsized":"2023-10-26T05:54:34.225474+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221861+0000","last_change":"2023-10-26T05:54:34.221861+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221861+0000","last_undegraded":"2023-10-26T05:54:34.221861+0000","last_fullsized":"2023-10-26T05:54:34.221861+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221231+0000","last_change":"2023-10-26T05:54:34.221231+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221231+0000","last_undegraded":"2023-10-26T05:54:34.221231+0000","last_fullsized":"2023-10-26T05:54:34.221231+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217314+0000","last_change":"2023-10-26T05:54:34.217314+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217314+0000","last_undegraded":"2023-10-26T05:54:34.217314+0000","last_fullsized":"2023-10-26T05:54:34.217314+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.242658+0000","last_change":"2023-10-26T05:54:34.242658+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.242658+0000","last_undegraded":"2023-10-26T05:54:34.242658+0000","last_fullsized":"2023-10-26T05:54:34.242658+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216909+0000","last_change":"2023-10-26T05:54:34.216909+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216909+0000","last_undegraded":"2023-10-26T05:54:34.216909+0000","last_fullsized":"2023-10-26T05:54:34.216909+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217829+0000","last_change":"2023-10-26T05:54:34.217829+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217829+0000","last_undegraded":"2023-10-26T05:54:34.217829+0000","last_fullsized":"2023-10-26T05:54:34.217829+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221485+0000","last_change":"2023-10-26T05:54:34.221485+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221485+0000","last_undegraded":"2023-10-26T05:54:34.221485+0000","last_fullsized":"2023-10-26T05:54:34.221485+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222189+0000","last_change":"2023-10-26T05:54:34.222189+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222189+0000","last_undegraded":"2023-10-26T05:54:34.222189+0000","last_fullsized":"2023-10-26T05:54:34.222189+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221314+0000","last_change":"2023-10-26T05:54:34.221314+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221314+0000","last_undegraded":"2023-10-26T05:54:34.221314+0000","last_fullsized":"2023-10-26T05:54:34.221314+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.239407+0000","last_change":"2023-10-26T05:54:34.238405+0000","last_active":"2023-10-26T05:54:34.239407+0000","last_peered":"2023-10-26T05:54:34.239407+0000","last_clean":"2023-10-26T05:54:34.239407+0000","last_became_active":"2023-10-26T05:54:34.234974+0000","last_became_peered":"2023-10-26T05:54:34.234974+0000","last_unstale":"2023-10-26T05:54:34.239407+0000","last_undegraded":"2023-10-26T05:54:34.239407+0000","last_fullsized":"2023-10-26T05:54:34.239407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:49.174594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.233445+0000","last_change":"2023-10-26T05:54:34.233445+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.233445+0000","last_undegraded":"2023-10-26T05:54:34.233445+0000","last_fullsized":"2023-10-26T05:54:34.233445+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238557+0000","last_change":"2023-10-26T05:54:34.238394+0000","last_active":"2023-10-26T05:54:34.238557+0000","last_peered":"2023-10-26T05:54:34.238557+0000","last_clean":"2023-10-26T05:54:34.238557+0000","last_became_active":"2023-10-26T05:54:34.237301+0000","last_became_peered":"2023-10-26T05:54:34.237301+0000","last_unstale":"2023-10-26T05:54:34.238557+0000","last_undegraded":"2023-10-26T05:54:34.238557+0000","last_fullsized":"2023-10-26T05:54:34.238557+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:49:47.037386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220728+0000","last_change":"2023-10-26T05:54:34.220728+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220728+0000","last_undegraded":"2023-10-26T05:54:34.220728+0000","last_fullsized":"2023-10-26T05:54:34.220728+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228507+0000","last_change":"2023-10-26T05:54:34.228217+0000","last_active":"2023-10-26T05:54:34.228507+0000","last_peered":"2023-10-26T05:54:34.228507+0000","last_clean":"2023-10-26T05:54:34.228507+0000","last_became_active":"2023-10-26T05:54:34.227446+0000","last_became_peered":"2023-10-26T05:54:34.227446+0000","last_unstale":"2023-10-26T05:54:34.228507+0000","last_undegraded":"2023-10-26T05:54:34.228507+0000","last_fullsized":"2023-10-26T05:54:34.228507+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:56.388499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221549+0000","last_change":"2023-10-26T05:54:34.221549+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221549+0000","last_undegraded":"2023-10-26T05:54:34.221549+0000","last_fullsized":"2023-10-26T05:54:34.221549+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.224414+0000","last_change":"2023-10-26T05:54:34.219413+0000","last_active":"2023-10-26T05:54:34.224414+0000","last_peered":"2023-10-26T05:54:34.224414+0000","last_clean":"2023-10-26T05:54:34.224414+0000","last_became_active":"2023-10-26T05:54:34.219109+0000","last_became_peered":"2023-10-26T05:54:34.219109+0000","last_unstale":"2023-10-26T05:54:34.224414+0000","last_undegraded":"2023-10-26T05:54:34.224414+0000","last_fullsized":"2023-10-26T05:54:34.224414+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:04:02.823811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223568+0000","last_change":"2023-10-26T05:54:34.223568+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223568+0000","last_undegraded":"2023-10-26T05:54:34.223568+0000","last_fullsized":"2023-10-26T05:54:34.223568+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.225568+0000","last_change":"2023-10-26T05:54:34.225466+0000","last_active":"2023-10-26T05:54:34.225568+0000","last_peered":"2023-10-26T05:54:34.225568+0000","last_clean":"2023-10-26T05:54:34.225568+0000","last_became_active":"2023-10-26T05:54:34.225226+0000","last_became_peered":"2023-10-26T05:54:34.225226+0000","last_unstale":"2023-10-26T05:54:34.225568+0000","last_undegraded":"2023-10-26T05:54:34.225568+0000","last_fullsized":"2023-10-26T05:54:34.225568+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:09:54.209739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221407+0000","last_change":"2023-10-26T05:54:34.221407+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221407+0000","last_undegraded":"2023-10-26T05:54:34.221407+0000","last_fullsized":"2023-10-26T05:54:34.221407+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.231347+0000","last_change":"2023-10-26T05:54:34.231290+0000","last_active":"2023-10-26T05:54:34.231347+0000","last_peered":"2023-10-26T05:54:34.231347+0000","last_clean":"2023-10-26T05:54:34.231347+0000","last_became_active":"2023-10-26T05:54:34.230708+0000","last_became_peered":"2023-10-26T05:54:34.230708+0000","last_unstale":"2023-10-26T05:54:34.231347+0000","last_undegraded":"2023-10-26T05:54:34.231347+0000","last_fullsized":"2023-10-26T05:54:34.231347+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:45:26.636962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.236168+0000","last_change":"2023-10-26T05:54:34.236168+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.236168+0000","last_undegraded":"2023-10-26T05:54:34.236168+0000","last_fullsized":"2023-10-26T05:54:34.236168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.229222+0000","last_change":"2023-10-26T05:54:34.229066+0000","last_active":"2023-10-26T05:54:34.229222+0000","last_peered":"2023-10-26T05:54:34.229222+0000","last_clean":"2023-10-26T05:54:34.229222+0000","last_became_active":"2023-10-26T05:54:34.228464+0000","last_became_peered":"2023-10-26T05:54:34.228464+0000","last_unstale":"2023-10-26T05:54:34.229222+0000","last_undegraded":"2023-10-26T05:54:34.229222+0000","last_fullsized":"2023-10-26T05:54:34.229222+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:39.551427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222740+0000","last_change":"2023-10-26T05:54:34.222740+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222740+0000","last_undegraded":"2023-10-26T05:54:34.222740+0000","last_fullsized":"2023-10-26T05:54:34.222740+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237238+0000","last_change":"2023-10-26T05:54:34.237160+0000","last_active":"2023-10-26T05:54:34.237238+0000","last_peered":"2023-10-26T05:54:34.237238+0000","last_clean":"2023-10-26T05:54:34.237238+0000","last_became_active":"2023-10-26T05:54:34.236869+0000","last_became_peered":"2023-10-26T05:54:34.236869+0000","last_unstale":"2023-10-26T05:54:34.237238+0000","last_undegraded":"2023-10-26T05:54:34.237238+0000","last_fullsized":"2023-10-26T05:54:34.237238+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:53:15.255164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.217012+0000","last_change":"2023-10-26T05:54:34.217012+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.217012+0000","last_undegraded":"2023-10-26T05:54:34.217012+0000","last_fullsized":"2023-10-26T05:54:34.217012+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228303+0000","last_change":"2023-10-26T05:54:34.227954+0000","last_active":"2023-10-26T05:54:34.228303+0000","last_peered":"2023-10-26T05:54:34.228303+0000","last_clean":"2023-10-26T05:54:34.228303+0000","last_became_active":"2023-10-26T05:54:34.227315+0000","last_became_peered":"2023-10-26T05:54:34.227315+0000","last_unstale":"2023-10-26T05:54:34.228303+0000","last_undegraded":"2023-10-26T05:54:34.228303+0000","last_fullsized":"2023-10-26T05:54:34.228303+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:36:29.297709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.244373+0000","last_change":"2023-10-26T05:54:34.244373+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.244373+0000","last_undegraded":"2023-10-26T05:54:34.244373+0000","last_fullsized":"2023-10-26T05:54:34.244373+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228819+0000","last_change":"2023-10-26T05:54:34.228604+0000","last_active":"2023-10-26T05:54:34.228819+0000","last_peered":"2023-10-26T05:54:34.228819+0000","last_clean":"2023-10-26T05:54:34.228819+0000","last_became_active":"2023-10-26T05:54:34.228363+0000","last_became_peered":"2023-10-26T05:54:34.228363+0000","last_unstale":"2023-10-26T05:54:34.228819+0000","last_undegraded":"2023-10-26T05:54:34.228819+0000","last_fullsized":"2023-10-26T05:54:34.228819+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:08:08.211694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224664+0000","last_change":"2023-10-26T05:54:34.224664+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224664+0000","last_undegraded":"2023-10-26T05:54:34.224664+0000","last_fullsized":"2023-10-26T05:54:34.224664+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.241631+0000","last_change":"2023-10-26T05:54:34.241288+0000","last_active":"2023-10-26T05:54:34.241631+0000","last_peered":"2023-10-26T05:54:34.241631+0000","last_clean":"2023-10-26T05:54:34.241631+0000","last_became_active":"2023-10-26T05:54:34.240585+0000","last_became_peered":"2023-10-26T05:54:34.240585+0000","last_unstale":"2023-10-26T05:54:34.241631+0000","last_undegraded":"2023-10-26T05:54:34.241631+0000","last_fullsized":"2023-10-26T05:54:34.241631+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:17:51.315680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222024+0000","last_change":"2023-10-26T05:54:34.222024+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222024+0000","last_undegraded":"2023-10-26T05:54:34.222024+0000","last_fullsized":"2023-10-26T05:54:34.222024+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.246204+0000","last_change":"2023-10-26T05:54:34.245988+0000","last_active":"2023-10-26T05:54:34.246204+0000","last_peered":"2023-10-26T05:54:34.246204+0000","last_clean":"2023-10-26T05:54:34.246204+0000","last_became_active":"2023-10-26T05:54:34.245363+0000","last_became_peered":"2023-10-26T05:54:34.245363+0000","last_unstale":"2023-10-26T05:54:34.246204+0000","last_undegraded":"2023-10-26T05:54:34.246204+0000","last_fullsized":"2023-10-26T05:54:34.246204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:10:53.581294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216915+0000","last_change":"2023-10-26T05:54:34.216915+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216915+0000","last_undegraded":"2023-10-26T05:54:34.216915+0000","last_fullsized":"2023-10-26T05:54:34.216915+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226894+0000","last_change":"2023-10-26T05:54:34.226642+0000","last_active":"2023-10-26T05:54:34.226894+0000","last_peered":"2023-10-26T05:54:34.226894+0000","last_clean":"2023-10-26T05:54:34.226894+0000","last_became_active":"2023-10-26T05:54:34.226063+0000","last_became_peered":"2023-10-26T05:54:34.226063+0000","last_unstale":"2023-10-26T05:54:34.226894+0000","last_undegraded":"2023-10-26T05:54:34.226894+0000","last_fullsized":"2023-10-26T05:54:34.226894+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:02:29.737511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.224031+0000","last_change":"2023-10-26T05:54:34.224031+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.224031+0000","last_undegraded":"2023-10-26T05:54:34.224031+0000","last_fullsized":"2023-10-26T05:54:34.224031+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.230277+0000","last_change":"2023-10-26T05:54:34.230187+0000","last_active":"2023-10-26T05:54:34.230277+0000","last_peered":"2023-10-26T05:54:34.230277+0000","last_clean":"2023-10-26T05:54:34.230277+0000","last_became_active":"2023-10-26T05:54:34.229848+0000","last_became_peered":"2023-10-26T05:54:34.229848+0000","last_unstale":"2023-10-26T05:54:34.230277+0000","last_undegraded":"2023-10-26T05:54:34.230277+0000","last_fullsized":"2023-10-26T05:54:34.230277+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:50.756897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.233720+0000","last_change":"2023-10-26T05:54:34.233720+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.233720+0000","last_undegraded":"2023-10-26T05:54:34.233720+0000","last_fullsized":"2023-10-26T05:54:34.233720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.242840+0000","last_change":"2023-10-26T05:54:34.242272+0000","last_active":"2023-10-26T05:54:34.242840+0000","last_peered":"2023-10-26T05:54:34.242840+0000","last_clean":"2023-10-26T05:54:34.242840+0000","last_became_active":"2023-10-26T05:54:34.240219+0000","last_became_peered":"2023-10-26T05:54:34.240219+0000","last_unstale":"2023-10-26T05:54:34.242840+0000","last_undegraded":"2023-10-26T05:54:34.242840+0000","last_fullsized":"2023-10-26T05:54:34.242840+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:05:23.700023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.234009+0000","last_change":"2023-10-26T05:54:34.234009+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.234009+0000","last_undegraded":"2023-10-26T05:54:34.234009+0000","last_fullsized":"2023-10-26T05:54:34.234009+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238519+0000","last_change":"2023-10-26T05:54:34.237939+0000","last_active":"2023-10-26T05:54:34.238519+0000","last_peered":"2023-10-26T05:54:34.238519+0000","last_clean":"2023-10-26T05:54:34.238519+0000","last_became_active":"2023-10-26T05:54:34.236275+0000","last_became_peered":"2023-10-26T05:54:34.236275+0000","last_unstale":"2023-10-26T05:54:34.238519+0000","last_undegraded":"2023-10-26T05:54:34.238519+0000","last_fullsized":"2023-10-26T05:54:34.238519+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:12:15.727069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.242318+0000","last_change":"2023-10-26T05:54:34.242318+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.242318+0000","last_undegraded":"2023-10-26T05:54:34.242318+0000","last_fullsized":"2023-10-26T05:54:34.242318+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.238966+0000","last_change":"2023-10-26T05:54:34.238890+0000","last_active":"2023-10-26T05:54:34.238966+0000","last_peered":"2023-10-26T05:54:34.238966+0000","last_clean":"2023-10-26T05:54:34.238966+0000","last_became_active":"2023-10-26T05:54:34.238622+0000","last_became_peered":"2023-10-26T05:54:34.238622+0000","last_unstale":"2023-10-26T05:54:34.238966+0000","last_undegraded":"2023-10-26T05:54:34.238966+0000","last_fullsized":"2023-10-26T05:54:34.238966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:26:48.748239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.222516+0000","last_change":"2023-10-26T05:54:34.222516+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.222516+0000","last_undegraded":"2023-10-26T05:54:34.222516+0000","last_fullsized":"2023-10-26T05:54:34.222516+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.240565+0000","last_change":"2023-10-26T05:54:34.239412+0000","last_active":"2023-10-26T05:54:34.240565+0000","last_peered":"2023-10-26T05:54:34.240565+0000","last_clean":"2023-10-26T05:54:34.240565+0000","last_became_active":"2023-10-26T05:54:34.238306+0000","last_became_peered":"2023-10-26T05:54:34.238306+0000","last_unstale":"2023-10-26T05:54:34.240565+0000","last_undegraded":"2023-10-26T05:54:34.240565+0000","last_fullsized":"2023-10-26T05:54:34.240565+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:09.950751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.226710+0000","last_change":"2023-10-26T05:54:34.226710+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.226710+0000","last_undegraded":"2023-10-26T05:54:34.226710+0000","last_fullsized":"2023-10-26T05:54:34.226710+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237616+0000","last_change":"2023-10-26T05:54:34.237449+0000","last_active":"2023-10-26T05:54:34.237616+0000","last_peered":"2023-10-26T05:54:34.237616+0000","last_clean":"2023-10-26T05:54:34.237616+0000","last_became_active":"2023-10-26T05:54:34.236646+0000","last_became_peered":"2023-10-26T05:54:34.236646+0000","last_unstale":"2023-10-26T05:54:34.237616+0000","last_undegraded":"2023-10-26T05:54:34.237616+0000","last_fullsized":"2023-10-26T05:54:34.237616+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:26:09.282179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216456+0000","last_change":"2023-10-26T05:54:34.216456+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216456+0000","last_undegraded":"2023-10-26T05:54:34.216456+0000","last_fullsized":"2023-10-26T05:54:34.216456+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.230169+0000","last_change":"2023-10-26T05:54:34.229691+0000","last_active":"2023-10-26T05:54:34.230169+0000","last_peered":"2023-10-26T05:54:34.230169+0000","last_clean":"2023-10-26T05:54:34.230169+0000","last_became_active":"2023-10-26T05:54:34.229093+0000","last_became_peered":"2023-10-26T05:54:34.229093+0000","last_unstale":"2023-10-26T05:54:34.230169+0000","last_undegraded":"2023-10-26T05:54:34.230169+0000","last_fullsized":"2023-10-26T05:54:34.230169+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:21.281518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219762+0000","last_change":"2023-10-26T05:54:34.219762+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219762+0000","last_undegraded":"2023-10-26T05:54:34.219762+0000","last_fullsized":"2023-10-26T05:54:34.219762+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.231966+0000","last_change":"2023-10-26T05:54:34.231906+0000","last_active":"2023-10-26T05:54:34.231966+0000","last_peered":"2023-10-26T05:54:34.231966+0000","last_clean":"2023-10-26T05:54:34.231966+0000","last_became_active":"2023-10-26T05:54:34.231677+0000","last_became_peered":"2023-10-26T05:54:34.231677+0000","last_unstale":"2023-10-26T05:54:34.231966+0000","last_undegraded":"2023-10-26T05:54:34.231966+0000","last_fullsized":"2023-10-26T05:54:34.231966+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:29:31.782805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.226387+0000","last_change":"2023-10-26T05:54:34.226387+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.226387+0000","last_undegraded":"2023-10-26T05:54:34.226387+0000","last_fullsized":"2023-10-26T05:54:34.226387+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.236138+0000","last_change":"2023-10-26T05:54:34.236055+0000","last_active":"2023-10-26T05:54:34.236138+0000","last_peered":"2023-10-26T05:54:34.236138+0000","last_clean":"2023-10-26T05:54:34.236138+0000","last_became_active":"2023-10-26T05:54:34.235715+0000","last_became_peered":"2023-10-26T05:54:34.235715+0000","last_unstale":"2023-10-26T05:54:34.236138+0000","last_undegraded":"2023-10-26T05:54:34.236138+0000","last_fullsized":"2023-10-26T05:54:34.236138+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:51:26.940485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220755+0000","last_change":"2023-10-26T05:54:34.220755+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220755+0000","last_undegraded":"2023-10-26T05:54:34.220755+0000","last_fullsized":"2023-10-26T05:54:34.220755+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.235237+0000","last_change":"2023-10-26T05:54:34.235155+0000","last_active":"2023-10-26T05:54:34.235237+0000","last_peered":"2023-10-26T05:54:34.235237+0000","last_clean":"2023-10-26T05:54:34.235237+0000","last_became_active":"2023-10-26T05:54:34.234905+0000","last_became_peered":"2023-10-26T05:54:34.234905+0000","last_unstale":"2023-10-26T05:54:34.235237+0000","last_undegraded":"2023-10-26T05:54:34.235237+0000","last_fullsized":"2023-10-26T05:54:34.235237+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:33.288741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218889+0000","last_change":"2023-10-26T05:54:34.218889+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218889+0000","last_undegraded":"2023-10-26T05:54:34.218889+0000","last_fullsized":"2023-10-26T05:54:34.218889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226868+0000","last_change":"2023-10-26T05:54:34.226691+0000","last_active":"2023-10-26T05:54:34.226868+0000","last_peered":"2023-10-26T05:54:34.226868+0000","last_clean":"2023-10-26T05:54:34.226868+0000","last_became_active":"2023-10-26T05:54:34.225872+0000","last_became_peered":"2023-10-26T05:54:34.225872+0000","last_unstale":"2023-10-26T05:54:34.226868+0000","last_undegraded":"2023-10-26T05:54:34.226868+0000","last_fullsized":"2023-10-26T05:54:34.226868+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:37.373754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.216671+0000","last_change":"2023-10-26T05:54:34.216671+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.216671+0000","last_undegraded":"2023-10-26T05:54:34.216671+0000","last_fullsized":"2023-10-26T05:54:34.216671+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.239361+0000","last_change":"2023-10-26T05:54:34.235620+0000","last_active":"2023-10-26T05:54:34.239361+0000","last_peered":"2023-10-26T05:54:34.239361+0000","last_clean":"2023-10-26T05:54:34.239361+0000","last_became_active":"2023-10-26T05:54:34.231181+0000","last_became_peered":"2023-10-26T05:54:34.231181+0000","last_unstale":"2023-10-26T05:54:34.239361+0000","last_undegraded":"2023-10-26T05:54:34.239361+0000","last_fullsized":"2023-10-26T05:54:34.239361+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:12.750236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.221409+0000","last_change":"2023-10-26T05:54:34.221409+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.221409+0000","last_undegraded":"2023-10-26T05:54:34.221409+0000","last_fullsized":"2023-10-26T05:54:34.221409+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.234344+0000","last_change":"2023-10-26T05:54:34.234275+0000","last_active":"2023-10-26T05:54:34.234344+0000","last_peered":"2023-10-26T05:54:34.234344+0000","last_clean":"2023-10-26T05:54:34.234344+0000","last_became_active":"2023-10-26T05:54:34.234042+0000","last_became_peered":"2023-10-26T05:54:34.234042+0000","last_unstale":"2023-10-26T05:54:34.234344+0000","last_undegraded":"2023-10-26T05:54:34.234344+0000","last_fullsized":"2023-10-26T05:54:34.234344+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:50:58.640410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.218977+0000","last_change":"2023-10-26T05:54:34.218977+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.218977+0000","last_undegraded":"2023-10-26T05:54:34.218977+0000","last_fullsized":"2023-10-26T05:54:34.218977+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.228533+0000","last_change":"2023-10-26T05:54:34.228407+0000","last_active":"2023-10-26T05:54:34.228533+0000","last_peered":"2023-10-26T05:54:34.228533+0000","last_clean":"2023-10-26T05:54:34.228533+0000","last_became_active":"2023-10-26T05:54:34.227889+0000","last_became_peered":"2023-10-26T05:54:34.227889+0000","last_unstale":"2023-10-26T05:54:34.228533+0000","last_undegraded":"2023-10-26T05:54:34.228533+0000","last_fullsized":"2023-10-26T05:54:34.228533+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:42:05.162676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219774+0000","last_change":"2023-10-26T05:54:34.219774+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219774+0000","last_undegraded":"2023-10-26T05:54:34.219774+0000","last_fullsized":"2023-10-26T05:54:34.219774+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.227621+0000","last_change":"2023-10-26T05:54:34.227337+0000","last_active":"2023-10-26T05:54:34.227621+0000","last_peered":"2023-10-26T05:54:34.227621+0000","last_clean":"2023-10-26T05:54:34.227621+0000","last_became_active":"2023-10-26T05:54:34.227083+0000","last_became_peered":"2023-10-26T05:54:34.227083+0000","last_unstale":"2023-10-26T05:54:34.227621+0000","last_undegraded":"2023-10-26T05:54:34.227621+0000","last_fullsized":"2023-10-26T05:54:34.227621+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:06:21.622602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.243298+0000","last_change":"2023-10-26T05:54:34.243298+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.243298+0000","last_undegraded":"2023-10-26T05:54:34.243298+0000","last_fullsized":"2023-10-26T05:54:34.243298+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.237935+0000","last_change":"2023-10-26T05:54:34.236735+0000","last_active":"2023-10-26T05:54:34.237935+0000","last_peered":"2023-10-26T05:54:34.237935+0000","last_clean":"2023-10-26T05:54:34.237935+0000","last_became_active":"2023-10-26T05:54:34.233447+0000","last_became_peered":"2023-10-26T05:54:34.233447+0000","last_unstale":"2023-10-26T05:54:34.237935+0000","last_undegraded":"2023-10-26T05:54:34.237935+0000","last_fullsized":"2023-10-26T05:54:34.237935+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:14.747480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.220883+0000","last_change":"2023-10-26T05:54:34.220883+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.220883+0000","last_undegraded":"2023-10-26T05:54:34.220883+0000","last_fullsized":"2023-10-26T05:54:34.220883+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.234328+0000","last_change":"2023-10-26T05:54:34.234238+0000","last_active":"2023-10-26T05:54:34.234328+0000","last_peered":"2023-10-26T05:54:34.234328+0000","last_clean":"2023-10-26T05:54:34.234328+0000","last_became_active":"2023-10-26T05:54:34.234006+0000","last_became_peered":"2023-10-26T05:54:34.234006+0000","last_unstale":"2023-10-26T05:54:34.234328+0000","last_undegraded":"2023-10-26T05:54:34.234328+0000","last_fullsized":"2023-10-26T05:54:34.234328+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:16:45.551886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.223205+0000","last_change":"2023-10-26T05:54:34.223205+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.223205+0000","last_undegraded":"2023-10-26T05:54:34.223205+0000","last_fullsized":"2023-10-26T05:54:34.223205+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.226797+0000","last_change":"2023-10-26T05:54:34.226096+0000","last_active":"2023-10-26T05:54:34.226797+0000","last_peered":"2023-10-26T05:54:34.226797+0000","last_clean":"2023-10-26T05:54:34.226797+0000","last_became_active":"2023-10-26T05:54:34.224494+0000","last_became_peered":"2023-10-26T05:54:34.224494+0000","last_unstale":"2023-10-26T05:54:34.226797+0000","last_undegraded":"2023-10-26T05:54:34.226797+0000","last_fullsized":"2023-10-26T05:54:34.226797+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:09.983718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":153,"state":"creating+peering","last_fresh":"2023-10-26T05:54:34.219971+0000","last_change":"2023-10-26T05:54:34.219971+0000","last_active":"2023-10-26T05:54:34.195118+0000","last_peered":"2023-10-26T05:54:34.195118+0000","last_clean":"2023-10-26T05:54:34.195118+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:54:34.219971+0000","last_undegraded":"2023-10-26T05:54:34.219971+0000","last_fullsized":"2023-10-26T05:54:34.219971+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-26T05:54:34.243333+0000","last_change":"2023-10-26T05:54:34.241834+0000","last_active":"2023-10-26T05:54:34.243333+0000","last_peered":"2023-10-26T05:54:34.243333+0000","last_clean":"2023-10-26T05:54:34.243333+0000","last_became_active":"2023-10-26T05:54:34.232265+0000","last_became_peered":"2023-10-26T05:54:34.232265+0000","last_unstale":"2023-10-26T05:54:34.243333+0000","last_undegraded":"2023-10-26T05:54:34.243333+0000","last_fullsized":"2023-10-26T05:54:34.243333+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:19.460844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-26T05:54:33.207194+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:54:33.207194+0000","last_peered":"2023-10-26T05:54:33.207194+0000","last_clean":"2023-10-26T05:54:33.207194+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:54:33.207194+0000","last_undegraded":"2023-10-26T05:54:33.207194+0000","last_fullsized":"2023-10-26T05:54:33.207194+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574939,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1409,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193902,"statfs":{"total":200000000,"available":198556311,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.39600000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.347},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.169,"5min":1.286,"15min":1.3100000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94799999999999995,"15min":1.018},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.236,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.93100000000000005},"min":{"1min":0.67100000000000004,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.2849999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":1.0229999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69199999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.214,"15min":1.4330000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3129999999999999,"5min":1.375,"15min":1.375},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.1679999999999999,"5min":1.639,"15min":1.639},"last":1.046},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96399999999999997,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3149999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.877}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0169999999999999,"15min":1.1000000000000001},"min":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.58699999999999997},"max":{"1min":1.266,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0720000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.53800000000000003},"max":{"1min":1.3029999999999999,"5min":1.702,"15min":1.702},"last":0.94899999999999995}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.081,"15min":1.1100000000000001},"min":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.59599999999999997},"max":{"1min":1.3640000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.0609999999999999,"15min":1.0509999999999999},"min":{"1min":0.66700000000000004,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.6180000000000001},"last":1.0740000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.298,"5min":1.298,"15min":1.3},"last":1.0660000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.62,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.405},"last":0.58599999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.82099999999999995},"min":{"1min":0.54800000000000004,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1990000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.435,"15min":0.435},"max":{"1min":1.3,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.621}]},{"osd":3,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97999999999999998,"15min":0.90400000000000003},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2250000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.056,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.3700000000000001,"5min":1.399,"15min":1.399},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.85699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.002,"5min":0.97799999999999998,"15min":0.97899999999999998},"min":{"1min":0.64500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.351,"5min":1.375,"15min":1.375},"last":1.143}]},{"osd":5,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0469999999999999},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.76800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.25,"5min":1.512,"15min":1.512},"last":0.77300000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1060000000000001},"min":{"1min":0.96799999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4550000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.92100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574939,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":0.94999999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.381,"5min":1.579,"15min":1.579},"last":0.879},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.83899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.53300000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.089,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.75},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.464,"15min":1.464},"last":0.61399999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.2110000000000001,"5min":1.319,"15min":1.5089999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.92700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2190000000000001,"5min":1.274,"15min":1.3160000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.59299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.333,"5min":1.448,"15min":1.548},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.238,"5min":1.4139999999999999,"15min":1.587},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94099999999999995,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2769999999999999,"5min":1.286,"15min":2.1909999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.091,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.536,"15min":1.536},"last":1.117}]},{"osd":6,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.96899999999999997,"15min":1.018},"min":{"1min":0.63500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0169999999999999,"15min":1.032},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.4199999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1899999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.442,"15min":0.442},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.008},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.155}]}]},{"osd":4,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.85999999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.248,"5min":1.2569999999999999,"15min":1.347},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.53100000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.143,"5min":1.502,"15min":1.502},"last":0.80100000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.878,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.25,"15min":1.375},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.96199999999999997,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":1.3440000000000001,"15min":1.4390000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90700000000000003,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.107,"5min":1.373,"15min":1.373},"last":0.877}]},{"osd":3,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.397,"5min":1.397,"15min":2.0190000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.71099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.232,"5min":1.3300000000000001,"15min":1.4359999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.88900000000000001},"min":{"1min":0.65600000000000003,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3440000000000001,"15min":1.3859999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.323,"5min":1.4179999999999999,"15min":1.673},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0289999999999999,"15min":1.004},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.42099999999999999},"max":{"1min":1.762,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.73799999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3540000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.68600000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.7509999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.137,"15min":1.119},"min":{"1min":0.74299999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"max":{"1min":1.484,"5min":1.728,"15min":1.728},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.29599999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.76800000000000002,"5min":1.385,"15min":1.385},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.94699999999999995,"5min":1.115,"15min":1.2689999999999999},"last":0.309}]},{"osd":1,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80100000000000005,"5min":1.349,"15min":1.349},"last":0.373},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65900000000000003,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.99299999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":1.1619999999999999,"5min":1.236,"15min":1.4199999999999999},"last":0.498}]},{"osd":4,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":0.83499999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.40000000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.073,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.95599999999999996},"min":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55800000000000005},"max":{"1min":1.113,"5min":1.413,"15min":1.478},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.91200000000000003},"min":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.59799999999999998},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93300000000000005,"15min":0.97999999999999998},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.165,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.254,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":0.93200000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.232,"5min":2.004,"15min":2.004},"last":0.81999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.54400000000000004},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.88300000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71799999999999997,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1120000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.67600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.198,"5min":1.226,"15min":1.2609999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68999999999999995,"15min":0.60199999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.157,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.74399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.149,"5min":1.333,"15min":2.8809999999999998},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.089,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.83899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1060000000000001,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.749},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.54400000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1459999999999999,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.91900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.476,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.052},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.99299999999999999,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.653,"15min":1.653},"last":0.91500000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.002,"15min":0.92700000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.077,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.419,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.135}]},{"osd":7,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.97799999999999998,"15min":0.91000000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.173,"5min":1.581,"15min":1.581},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0900000000000001,"15min":1.0449999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.367,"5min":1.823,"15min":15.352},"last":1.2130000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152954,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.157,"15min":1.157},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83599999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27700000000000002},"max":{"1min":1.087,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.378}]},{"osd":3,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.89500000000000002,"5min":1.298,"15min":1.298},"last":0.39500000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.64500000000000002,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1659999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.27,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.621,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3620000000000001,"5min":1.492,"15min":1.492},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.92400000000000004,"15min":0.92900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3029999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96699999999999997},"min":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.468,"15min":1.837},"last":0.69599999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.012,"15min":1.022},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.2330000000000001,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.046,"15min":1.0840000000000001},"min":{"1min":0.63200000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4099999999999999,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":0.63900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574939,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193908,"statfs":{"total":200000000,"available":198562586,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78100000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.69899999999999995,"15min":0.62},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85899999999999999,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.53600000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.093,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.437},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1140000000000001,"15min":1.177},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.626},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.99299999999999999,"5min":1.409,"15min":1.409},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80500000000000005,"15min":0.70999999999999996},"min":{"1min":0.33400000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.2629999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.495}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.92100000000000004},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2769999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1399999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.90900000000000003},"min":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.123,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.004,"15min":0.996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4399999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":0.98999999999999999},"min":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.78300000000000003}]}]}],"pool_statfs":[]}} 2023-10-26T05:54:39.047 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3f is not active+clean 2023-10-26T05:54:39.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.218733+0000', 'last_change': '2023-10-26T05:54:34.218733+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.218733+0000', 'last_undegraded': '2023-10-26T05:54:34.218733+0000', 'last_fullsized': '2023-10-26T05:54:34.218733+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.048 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2023-10-26T05:54:39.048 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.224954+0000', 'last_change': '2023-10-26T05:54:34.224954+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.224954+0000', 'last_undegraded': '2023-10-26T05:54:34.224954+0000', 'last_fullsized': '2023-10-26T05:54:34.224954+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.049 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2023-10-26T05:54:39.049 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.235808+0000', 'last_change': '2023-10-26T05:54:34.235808+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.235808+0000', 'last_undegraded': '2023-10-26T05:54:34.235808+0000', 'last_fullsized': '2023-10-26T05:54:34.235808+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3c is not active+clean 2023-10-26T05:54:39.050 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.217369+0000', 'last_change': '2023-10-26T05:54:34.217369+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.217369+0000', 'last_undegraded': '2023-10-26T05:54:34.217369+0000', 'last_fullsized': '2023-10-26T05:54:34.217369+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.050 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-10-26T05:54:39.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222328+0000', 'last_change': '2023-10-26T05:54:34.222328+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222328+0000', 'last_undegraded': '2023-10-26T05:54:34.222328+0000', 'last_fullsized': '2023-10-26T05:54:34.222328+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-10-26T05:54:39.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.219374+0000', 'last_change': '2023-10-26T05:54:34.219374+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.219374+0000', 'last_undegraded': '2023-10-26T05:54:34.219374+0000', 'last_fullsized': '2023-10-26T05:54:34.219374+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-10-26T05:54:39.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.223861+0000', 'last_change': '2023-10-26T05:54:34.223861+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.223861+0000', 'last_undegraded': '2023-10-26T05:54:34.223861+0000', 'last_fullsized': '2023-10-26T05:54:34.223861+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2023-10-26T05:54:39.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.235033+0000', 'last_change': '2023-10-26T05:54:34.235033+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.235033+0000', 'last_undegraded': '2023-10-26T05:54:34.235033+0000', 'last_fullsized': '2023-10-26T05:54:34.235033+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2023-10-26T05:54:39.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.235409+0000', 'last_change': '2023-10-26T05:54:34.235409+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.235409+0000', 'last_undegraded': '2023-10-26T05:54:34.235409+0000', 'last_fullsized': '2023-10-26T05:54:34.235409+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2023-10-26T05:54:39.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222253+0000', 'last_change': '2023-10-26T05:54:34.222253+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222253+0000', 'last_undegraded': '2023-10-26T05:54:34.222253+0000', 'last_fullsized': '2023-10-26T05:54:34.222253+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-10-26T05:54:39.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.224745+0000', 'last_change': '2023-10-26T05:54:34.224745+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.224745+0000', 'last_undegraded': '2023-10-26T05:54:34.224745+0000', 'last_fullsized': '2023-10-26T05:54:34.224745+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:54:39.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2023-10-26T05:54:39.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.234523+0000', 'last_change': '2023-10-26T05:54:34.234523+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.234523+0000', 'last_undegraded': '2023-10-26T05:54:34.234523+0000', 'last_fullsized': '2023-10-26T05:54:34.234523+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.33 is not active+clean 2023-10-26T05:54:39.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.216288+0000', 'last_change': '2023-10-26T05:54:34.216288+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.216288+0000', 'last_undegraded': '2023-10-26T05:54:34.216288+0000', 'last_fullsized': '2023-10-26T05:54:34.216288+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2023-10-26T05:54:39.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.223176+0000', 'last_change': '2023-10-26T05:54:34.223176+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.223176+0000', 'last_undegraded': '2023-10-26T05:54:34.223176+0000', 'last_fullsized': '2023-10-26T05:54:34.223176+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.31 is not active+clean 2023-10-26T05:54:39.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.218575+0000', 'last_change': '2023-10-26T05:54:34.218575+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.218575+0000', 'last_undegraded': '2023-10-26T05:54:34.218575+0000', 'last_fullsized': '2023-10-26T05:54:34.218575+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2023-10-26T05:54:39.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221557+0000', 'last_change': '2023-10-26T05:54:34.221557+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221557+0000', 'last_undegraded': '2023-10-26T05:54:34.221557+0000', 'last_fullsized': '2023-10-26T05:54:34.221557+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-10-26T05:54:39.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.223087+0000', 'last_change': '2023-10-26T05:54:34.223087+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.223087+0000', 'last_undegraded': '2023-10-26T05:54:34.223087+0000', 'last_fullsized': '2023-10-26T05:54:34.223087+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2023-10-26T05:54:39.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.223894+0000', 'last_change': '2023-10-26T05:54:34.223894+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.223894+0000', 'last_undegraded': '2023-10-26T05:54:34.223894+0000', 'last_fullsized': '2023-10-26T05:54:34.223894+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-10-26T05:54:39.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.225892+0000', 'last_change': '2023-10-26T05:54:34.225892+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.225892+0000', 'last_undegraded': '2023-10-26T05:54:34.225892+0000', 'last_fullsized': '2023-10-26T05:54:34.225892+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:54:39.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2023-10-26T05:54:39.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221998+0000', 'last_change': '2023-10-26T05:54:34.221998+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221998+0000', 'last_undegraded': '2023-10-26T05:54:34.221998+0000', 'last_fullsized': '2023-10-26T05:54:34.221998+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-10-26T05:54:39.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222347+0000', 'last_change': '2023-10-26T05:54:34.222347+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222347+0000', 'last_undegraded': '2023-10-26T05:54:34.222347+0000', 'last_fullsized': '2023-10-26T05:54:34.222347+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-10-26T05:54:39.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.225197+0000', 'last_change': '2023-10-26T05:54:34.225197+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.225197+0000', 'last_undegraded': '2023-10-26T05:54:34.225197+0000', 'last_fullsized': '2023-10-26T05:54:34.225197+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:54:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2023-10-26T05:54:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.234208+0000', 'last_change': '2023-10-26T05:54:34.234208+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.234208+0000', 'last_undegraded': '2023-10-26T05:54:34.234208+0000', 'last_fullsized': '2023-10-26T05:54:34.234208+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-10-26T05:54:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.225474+0000', 'last_change': '2023-10-26T05:54:34.225474+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.225474+0000', 'last_undegraded': '2023-10-26T05:54:34.225474+0000', 'last_fullsized': '2023-10-26T05:54:34.225474+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:54:39.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2023-10-26T05:54:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221861+0000', 'last_change': '2023-10-26T05:54:34.221861+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221861+0000', 'last_undegraded': '2023-10-26T05:54:34.221861+0000', 'last_fullsized': '2023-10-26T05:54:34.221861+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2023-10-26T05:54:39.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221231+0000', 'last_change': '2023-10-26T05:54:34.221231+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221231+0000', 'last_undegraded': '2023-10-26T05:54:34.221231+0000', 'last_fullsized': '2023-10-26T05:54:34.221231+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.25 is not active+clean 2023-10-26T05:54:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.217314+0000', 'last_change': '2023-10-26T05:54:34.217314+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.217314+0000', 'last_undegraded': '2023-10-26T05:54:34.217314+0000', 'last_fullsized': '2023-10-26T05:54:34.217314+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2023-10-26T05:54:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.242658+0000', 'last_change': '2023-10-26T05:54:34.242658+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.242658+0000', 'last_undegraded': '2023-10-26T05:54:34.242658+0000', 'last_fullsized': '2023-10-26T05:54:34.242658+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:54:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.23 is not active+clean 2023-10-26T05:54:39.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.216909+0000', 'last_change': '2023-10-26T05:54:34.216909+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.216909+0000', 'last_undegraded': '2023-10-26T05:54:34.216909+0000', 'last_fullsized': '2023-10-26T05:54:34.216909+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.22 is not active+clean 2023-10-26T05:54:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.217829+0000', 'last_change': '2023-10-26T05:54:34.217829+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.217829+0000', 'last_undegraded': '2023-10-26T05:54:34.217829+0000', 'last_fullsized': '2023-10-26T05:54:34.217829+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-10-26T05:54:39.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221485+0000', 'last_change': '2023-10-26T05:54:34.221485+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221485+0000', 'last_undegraded': '2023-10-26T05:54:34.221485+0000', 'last_fullsized': '2023-10-26T05:54:34.221485+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-10-26T05:54:39.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222189+0000', 'last_change': '2023-10-26T05:54:34.222189+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222189+0000', 'last_undegraded': '2023-10-26T05:54:34.222189+0000', 'last_fullsized': '2023-10-26T05:54:34.222189+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-10-26T05:54:39.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221314+0000', 'last_change': '2023-10-26T05:54:34.221314+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221314+0000', 'last_undegraded': '2023-10-26T05:54:34.221314+0000', 'last_fullsized': '2023-10-26T05:54:34.221314+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-10-26T05:54:39.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.233445+0000', 'last_change': '2023-10-26T05:54:34.233445+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.233445+0000', 'last_undegraded': '2023-10-26T05:54:34.233445+0000', 'last_fullsized': '2023-10-26T05:54:34.233445+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-10-26T05:54:39.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.220728+0000', 'last_change': '2023-10-26T05:54:34.220728+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.220728+0000', 'last_undegraded': '2023-10-26T05:54:34.220728+0000', 'last_fullsized': '2023-10-26T05:54:34.220728+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-10-26T05:54:39.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221549+0000', 'last_change': '2023-10-26T05:54:34.221549+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221549+0000', 'last_undegraded': '2023-10-26T05:54:34.221549+0000', 'last_fullsized': '2023-10-26T05:54:34.221549+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-10-26T05:54:39.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.223568+0000', 'last_change': '2023-10-26T05:54:34.223568+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.223568+0000', 'last_undegraded': '2023-10-26T05:54:34.223568+0000', 'last_fullsized': '2023-10-26T05:54:34.223568+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-10-26T05:54:39.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221407+0000', 'last_change': '2023-10-26T05:54:34.221407+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221407+0000', 'last_undegraded': '2023-10-26T05:54:34.221407+0000', 'last_fullsized': '2023-10-26T05:54:34.221407+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-10-26T05:54:39.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.236168+0000', 'last_change': '2023-10-26T05:54:34.236168+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.236168+0000', 'last_undegraded': '2023-10-26T05:54:34.236168+0000', 'last_fullsized': '2023-10-26T05:54:34.236168+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-10-26T05:54:39.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222740+0000', 'last_change': '2023-10-26T05:54:34.222740+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222740+0000', 'last_undegraded': '2023-10-26T05:54:34.222740+0000', 'last_fullsized': '2023-10-26T05:54:34.222740+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2023-10-26T05:54:39.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.217012+0000', 'last_change': '2023-10-26T05:54:34.217012+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.217012+0000', 'last_undegraded': '2023-10-26T05:54:34.217012+0000', 'last_fullsized': '2023-10-26T05:54:34.217012+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-10-26T05:54:39.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.244373+0000', 'last_change': '2023-10-26T05:54:34.244373+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.244373+0000', 'last_undegraded': '2023-10-26T05:54:34.244373+0000', 'last_fullsized': '2023-10-26T05:54:34.244373+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:54:39.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-10-26T05:54:39.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.224664+0000', 'last_change': '2023-10-26T05:54:34.224664+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.224664+0000', 'last_undegraded': '2023-10-26T05:54:34.224664+0000', 'last_fullsized': '2023-10-26T05:54:34.224664+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-10-26T05:54:39.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222024+0000', 'last_change': '2023-10-26T05:54:34.222024+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222024+0000', 'last_undegraded': '2023-10-26T05:54:34.222024+0000', 'last_fullsized': '2023-10-26T05:54:34.222024+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2023-10-26T05:54:39.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.216915+0000', 'last_change': '2023-10-26T05:54:34.216915+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.216915+0000', 'last_undegraded': '2023-10-26T05:54:34.216915+0000', 'last_fullsized': '2023-10-26T05:54:34.216915+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-10-26T05:54:39.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.224031+0000', 'last_change': '2023-10-26T05:54:34.224031+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.224031+0000', 'last_undegraded': '2023-10-26T05:54:34.224031+0000', 'last_fullsized': '2023-10-26T05:54:34.224031+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-10-26T05:54:39.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.233720+0000', 'last_change': '2023-10-26T05:54:34.233720+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.233720+0000', 'last_undegraded': '2023-10-26T05:54:34.233720+0000', 'last_fullsized': '2023-10-26T05:54:34.233720+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-10-26T05:54:39.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.234009+0000', 'last_change': '2023-10-26T05:54:34.234009+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.234009+0000', 'last_undegraded': '2023-10-26T05:54:34.234009+0000', 'last_fullsized': '2023-10-26T05:54:34.234009+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:54:39.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-10-26T05:54:39.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.242318+0000', 'last_change': '2023-10-26T05:54:34.242318+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.242318+0000', 'last_undegraded': '2023-10-26T05:54:34.242318+0000', 'last_fullsized': '2023-10-26T05:54:34.242318+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:54:39.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-10-26T05:54:39.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.222516+0000', 'last_change': '2023-10-26T05:54:34.222516+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.222516+0000', 'last_undegraded': '2023-10-26T05:54:34.222516+0000', 'last_fullsized': '2023-10-26T05:54:34.222516+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-10-26T05:54:39.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.226710+0000', 'last_change': '2023-10-26T05:54:34.226710+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.226710+0000', 'last_undegraded': '2023-10-26T05:54:34.226710+0000', 'last_fullsized': '2023-10-26T05:54:34.226710+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2023-10-26T05:54:39.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.216456+0000', 'last_change': '2023-10-26T05:54:34.216456+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.216456+0000', 'last_undegraded': '2023-10-26T05:54:34.216456+0000', 'last_fullsized': '2023-10-26T05:54:34.216456+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-10-26T05:54:39.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.219762+0000', 'last_change': '2023-10-26T05:54:34.219762+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.219762+0000', 'last_undegraded': '2023-10-26T05:54:34.219762+0000', 'last_fullsized': '2023-10-26T05:54:34.219762+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-10-26T05:54:39.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.226387+0000', 'last_change': '2023-10-26T05:54:34.226387+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.226387+0000', 'last_undegraded': '2023-10-26T05:54:34.226387+0000', 'last_fullsized': '2023-10-26T05:54:34.226387+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-10-26T05:54:39.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.220755+0000', 'last_change': '2023-10-26T05:54:34.220755+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.220755+0000', 'last_undegraded': '2023-10-26T05:54:34.220755+0000', 'last_fullsized': '2023-10-26T05:54:34.220755+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-10-26T05:54:39.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.218889+0000', 'last_change': '2023-10-26T05:54:34.218889+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.218889+0000', 'last_undegraded': '2023-10-26T05:54:34.218889+0000', 'last_fullsized': '2023-10-26T05:54:34.218889+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2023-10-26T05:54:39.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.216671+0000', 'last_change': '2023-10-26T05:54:34.216671+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.216671+0000', 'last_undegraded': '2023-10-26T05:54:34.216671+0000', 'last_fullsized': '2023-10-26T05:54:34.216671+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:54:39.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-10-26T05:54:39.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.221409+0000', 'last_change': '2023-10-26T05:54:34.221409+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.221409+0000', 'last_undegraded': '2023-10-26T05:54:34.221409+0000', 'last_fullsized': '2023-10-26T05:54:34.221409+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-10-26T05:54:39.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.218977+0000', 'last_change': '2023-10-26T05:54:34.218977+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.218977+0000', 'last_undegraded': '2023-10-26T05:54:34.218977+0000', 'last_fullsized': '2023-10-26T05:54:34.218977+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:54:39.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-10-26T05:54:39.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.219774+0000', 'last_change': '2023-10-26T05:54:34.219774+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.219774+0000', 'last_undegraded': '2023-10-26T05:54:34.219774+0000', 'last_fullsized': '2023-10-26T05:54:34.219774+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-10-26T05:54:39.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.243298+0000', 'last_change': '2023-10-26T05:54:34.243298+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.243298+0000', 'last_undegraded': '2023-10-26T05:54:34.243298+0000', 'last_fullsized': '2023-10-26T05:54:34.243298+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:54:39.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-10-26T05:54:39.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.220883+0000', 'last_change': '2023-10-26T05:54:34.220883+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.220883+0000', 'last_undegraded': '2023-10-26T05:54:34.220883+0000', 'last_fullsized': '2023-10-26T05:54:34.220883+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:54:39.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-10-26T05:54:39.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.223205+0000', 'last_change': '2023-10-26T05:54:34.223205+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.223205+0000', 'last_undegraded': '2023-10-26T05:54:34.223205+0000', 'last_fullsized': '2023-10-26T05:54:34.223205+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:54:39.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-10-26T05:54:39.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 153, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:54:34.219971+0000', 'last_change': '2023-10-26T05:54:34.219971+0000', 'last_active': '2023-10-26T05:54:34.195118+0000', 'last_peered': '2023-10-26T05:54:34.195118+0000', 'last_clean': '2023-10-26T05:54:34.195118+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:54:34.219971+0000', 'last_undegraded': '2023-10-26T05:54:34.219971+0000', 'last_fullsized': '2023-10-26T05:54:34.219971+0000', 'mapping_epoch': 153, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 153, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'last_clean_scrub_stamp': '2023-10-26T05:54:34.195118+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:54:39.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T05:54:39.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 -- 172.21.15.71:0/3256592854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140d0850 msgr2=0x7f221415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 --2- 172.21.15.71:0/3256592854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140d0850 0x7f221415b3a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f221408a3b0 tx=0x7f220801c7b0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 -- 172.21.15.71:0/3256592854 shutdown_connections 2023-10-26T05:54:39.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 --2- 172.21.15.71:0/3256592854 >> v2:172.21.15.86:3300/0 conn(0x7f22140cd880 0x7f221415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 --2- 172.21.15.71:0/3256592854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140d0850 0x7f221415b3a0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 -- 172.21.15.71:0/3256592854 >> 172.21.15.71:0/3256592854 conn(0x7f22140cdd80 msgr2=0x7f22140ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:39.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 -- 172.21.15.71:0/3256592854 shutdown_connections 2023-10-26T05:54:39.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 -- 172.21.15.71:0/3256592854 wait complete. 2023-10-26T05:54:39.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 Processor -- start 2023-10-26T05:54:39.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.211+0000 7f2218adb640 1 -- start start 2023-10-26T05:54:39.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 0x7f221415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22140d0470 0x7f221414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22140d0850 0x7f2214153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221415d980 con 0x7f22140cd880 2023-10-26T05:54:39.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214150060 con 0x7f22140d0470 2023-10-26T05:54:39.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214158b80 con 0x7f22140d0850 2023-10-26T05:54:39.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 0x7f221415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 0x7f221415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39864/0 (socket says 172.21.15.71:39864) 2023-10-26T05:54:39.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 -- 172.21.15.71:0/844077593 learned_addr learned my addr 172.21.15.71:0/844077593 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:39.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2212d9d640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22140d0850 0x7f2214153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2211d9b640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22140d0470 0x7f221414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 -- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22140d0850 msgr2=0x7f2214153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22140d0850 0x7f2214153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 -- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22140d0470 msgr2=0x7f221414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22140d0470 0x7f221414fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 -- 172.21.15.71:0/844077593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208009560 con 0x7f22140cd880 2023-10-26T05:54:39.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2212d9d640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22140d0850 0x7f2214153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:39.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f221259c640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 0x7f221415d440 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f220400c450 tx=0x7f220400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:39.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204012830 con 0x7f22140cd880 2023-10-26T05:54:39.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2204012aa0 con 0x7f22140cd880 2023-10-26T05:54:39.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2214158df0 con 0x7f22140cd880 2023-10-26T05:54:39.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204011380 con 0x7f22140cd880 2023-10-26T05:54:39.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22141593b0 con 0x7f22140cd880 2023-10-26T05:54:39.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2214066b10 con 0x7f22140cd880 2023-10-26T05:54:39.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f22040114e0 con 0x7f22140cd880 2023-10-26T05:54:39.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2202ffd640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21f40664a0 0x7f21f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2204083f10 con 0x7f22140cd880 2023-10-26T05:54:39.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.215+0000 7f2211d9b640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21f40664a0 0x7f21f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.219+0000 7f2211d9b640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21f40664a0 0x7f21f4068960 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f22080af000 tx=0x7f220801aff0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:39.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.219+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2204051390 con 0x7f22140cd880 2023-10-26T05:54:39.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.423+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f22140b62a0 con 0x7f22140cd880 2023-10-26T05:54:39.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.423+0000 7f2202ffd640 1 -- 172.21.15.71:0/844077593 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1447 (secure 0 0 0) 0x7f22040549b0 con 0x7f22140cd880 2023-10-26T05:54:39.433 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:39.433 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":457,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":155,"num_osds":8,"num_up_osds":8,"osd_up_since":1698299229,"num_in_osds":8,"osd_in_since":1698299226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":7963379,"bytes_avail":1592036621,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":58,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":8517}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-10-26T05:53:29.534833+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-26T05:54:39.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21f40664a0 msgr2=0x7f21f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21f40664a0 0x7f21f4068960 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f22080af000 tx=0x7f220801aff0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 msgr2=0x7f221415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 0x7f221415d440 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f220400c450 tx=0x7f220400c910 comp rx=0 tx=0).stop 2023-10-26T05:54:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 shutdown_connections 2023-10-26T05:54:39.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22140d0850 0x7f2214153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22140d0470 0x7f221414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f21f40664a0 0x7f21f4068960 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 --2- 172.21.15.71:0/844077593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22140cd880 0x7f221415d440 secure :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f220400c450 tx=0x7f220400c910 comp rx=0 tx=0).stop 2023-10-26T05:54:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 >> 172.21.15.71:0/844077593 conn(0x7f22140cdd80 msgr2=0x7f221415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:39.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 shutdown_connections 2023-10-26T05:54:39.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.427+0000 7f2218adb640 1 -- 172.21.15.71:0/844077593 wait complete. 2023-10-26T05:54:39.450 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T05:54:39.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:54:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/583101722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 msgr2=0x7f6668070d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- 172.21.15.71:0/583101722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f6668070d70 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f666806f570 tx=0x7f666001cca0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/583101722 shutdown_connections 2023-10-26T05:54:39.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- 172.21.15.71:0/583101722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f6668070d70 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- 172.21.15.71:0/583101722 >> v2:172.21.15.86:3300/0 conn(0x7f66680afa90 0x7f666806a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- 172.21.15.71:0/583101722 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6668155140 0x7f66680ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/583101722 >> 172.21.15.71:0/583101722 conn(0x7f66680aef00 msgr2=0x7f66680af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/583101722 shutdown_connections 2023-10-26T05:54:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/583101722 wait complete. 2023-10-26T05:54:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 Processor -- start 2023-10-26T05:54:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- start start 2023-10-26T05:54:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f66681566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66680afa90 0x7f66681509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668155140 0x7f6668159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666806c120 con 0x7f666806a690 2023-10-26T05:54:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66681598b0 con 0x7f6668155140 2023-10-26T05:54:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6668159b60 con 0x7f66680afa90 2023-10-26T05:54:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f66681566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f66681566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39874/0 (socket says 172.21.15.71:39874) 2023-10-26T05:54:39.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 -- 172.21.15.71:0/28084904 learned_addr learned my addr 172.21.15.71:0/28084904 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:39.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666d39b640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668155140 0x7f6668159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 -- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66680afa90 msgr2=0x7f66681509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f6667fff640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66680afa90 0x7f66681509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66680afa90 0x7f66681509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 -- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668155140 msgr2=0x7f6668159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668155140 0x7f6668159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 -- 172.21.15.71:0/28084904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600098e0 con 0x7f666806a690 2023-10-26T05:54:39.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666d39b640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668155140 0x7f6668159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:39.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666cb9a640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f66681566b0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b410 tx=0x7f665c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:39.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c011830 con 0x7f666806a690 2023-10-26T05:54:39.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f665c011a80 con 0x7f666806a690 2023-10-26T05:54:39.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6668159e40 con 0x7f666806a690 2023-10-26T05:54:39.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c011e90 con 0x7f666806a690 2023-10-26T05:54:39.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666815a400 con 0x7f666806a690 2023-10-26T05:54:39.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.555+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6668066d00 con 0x7f666806a690 2023-10-26T05:54:39.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.559+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f665c01a400 con 0x7f666806a690 2023-10-26T05:54:39.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.559+0000 7f66657fa640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6654066470 0x7f6654068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:39.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.559+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f665c0842b0 con 0x7f666806a690 2023-10-26T05:54:39.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.559+0000 7f6667fff640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6654066470 0x7f6654068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:39.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.559+0000 7f6667fff640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6654066470 0x7f6654068930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f66680b32d0 tx=0x7f6658008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:39.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.563+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f665c051720 con 0x7f666806a690 2023-10-26T05:54:39.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.699+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f666806c260 con 0x7f6654066470 2023-10-26T05:54:39.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.723+0000 7f66657fa640 1 -- 172.21.15.71:0/28084904 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284085 (secure 0 0 0) 0x7f666806c260 con 0x7f6654066470 2023-10-26T05:54:39.735 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:39.737 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:54:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6654066470 msgr2=0x7f6654068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6654066470 0x7f6654068930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f66680b32d0 tx=0x7f6658008040 comp rx=0 tx=0).stop 2023-10-26T05:54:39.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 msgr2=0x7f66681566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:39.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f66681566b0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b410 tx=0x7f665c00b8d0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 shutdown_connections 2023-10-26T05:54:39.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6668155140 0x7f6668159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66680afa90 0x7f66681509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6654066470 0x7f6654068930 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 --2- 172.21.15.71:0/28084904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f666806a690 0x7f66681566b0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:39.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 >> 172.21.15.71:0/28084904 conn(0x7f66680aef00 msgr2=0x7f6668164240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:39.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 shutdown_connections 2023-10-26T05:54:39.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:39.727+0000 7f666edfe640 1 -- 172.21.15.71:0/28084904 wait complete. 2023-10-26T05:54:39.751 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2023-10-26T05:54:39.557347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7926,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554573,"statfs":{"total":1600000000,"available":1591883276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001664"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667522+0000","last_change":"2023-10-26T05:54:35.235085+0000","last_active":"2023-10-26T05:54:37.667522+0000","last_peered":"2023-10-26T05:54:37.667522+0000","last_clean":"2023-10-26T05:54:37.667522+0000","last_became_active":"2023-10-26T05:54:35.234844+0000","last_became_peered":"2023-10-26T05:54:35.234844+0000","last_unstale":"2023-10-26T05:54:37.667522+0000","last_undegraded":"2023-10-26T05:54:37.667522+0000","last_fullsized":"2023-10-26T05:54:37.667522+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:45.644368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217879+0000","last_change":"2023-10-26T05:54:35.246520+0000","last_active":"2023-10-26T05:54:36.217879+0000","last_peered":"2023-10-26T05:54:36.217879+0000","last_clean":"2023-10-26T05:54:36.217879+0000","last_became_active":"2023-10-26T05:54:35.246384+0000","last_became_peered":"2023-10-26T05:54:35.246384+0000","last_unstale":"2023-10-26T05:54:36.217879+0000","last_undegraded":"2023-10-26T05:54:36.217879+0000","last_fullsized":"2023-10-26T05:54:36.217879+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:31:51.083174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672228+0000","last_change":"2023-10-26T05:54:35.268032+0000","last_active":"2023-10-26T05:54:37.672228+0000","last_peered":"2023-10-26T05:54:37.672228+0000","last_clean":"2023-10-26T05:54:37.672228+0000","last_became_active":"2023-10-26T05:54:35.263610+0000","last_became_peered":"2023-10-26T05:54:35.263610+0000","last_unstale":"2023-10-26T05:54:37.672228+0000","last_undegraded":"2023-10-26T05:54:37.672228+0000","last_fullsized":"2023-10-26T05:54:37.672228+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:39.249088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666720+0000","last_change":"2023-10-26T05:54:35.244388+0000","last_active":"2023-10-26T05:54:37.666720+0000","last_peered":"2023-10-26T05:54:37.666720+0000","last_clean":"2023-10-26T05:54:37.666720+0000","last_became_active":"2023-10-26T05:54:35.244136+0000","last_became_peered":"2023-10-26T05:54:35.244136+0000","last_unstale":"2023-10-26T05:54:37.666720+0000","last_undegraded":"2023-10-26T05:54:37.666720+0000","last_fullsized":"2023-10-26T05:54:37.666720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:14:18.669585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673746+0000","last_change":"2023-10-26T05:54:35.264038+0000","last_active":"2023-10-26T05:54:37.673746+0000","last_peered":"2023-10-26T05:54:37.673746+0000","last_clean":"2023-10-26T05:54:37.673746+0000","last_became_active":"2023-10-26T05:54:35.263494+0000","last_became_peered":"2023-10-26T05:54:35.263494+0000","last_unstale":"2023-10-26T05:54:37.673746+0000","last_undegraded":"2023-10-26T05:54:37.673746+0000","last_fullsized":"2023-10-26T05:54:37.673746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:54:52.578426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225856+0000","last_change":"2023-10-26T05:54:35.241648+0000","last_active":"2023-10-26T05:54:36.225856+0000","last_peered":"2023-10-26T05:54:36.225856+0000","last_clean":"2023-10-26T05:54:36.225856+0000","last_became_active":"2023-10-26T05:54:35.238344+0000","last_became_peered":"2023-10-26T05:54:35.238344+0000","last_unstale":"2023-10-26T05:54:36.225856+0000","last_undegraded":"2023-10-26T05:54:36.225856+0000","last_fullsized":"2023-10-26T05:54:36.225856+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:21:05.364628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225885+0000","last_change":"2023-10-26T05:54:35.252109+0000","last_active":"2023-10-26T05:54:36.225885+0000","last_peered":"2023-10-26T05:54:36.225885+0000","last_clean":"2023-10-26T05:54:36.225885+0000","last_became_active":"2023-10-26T05:54:35.250289+0000","last_became_peered":"2023-10-26T05:54:35.250289+0000","last_unstale":"2023-10-26T05:54:36.225885+0000","last_undegraded":"2023-10-26T05:54:36.225885+0000","last_fullsized":"2023-10-26T05:54:36.225885+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:26.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672600+0000","last_change":"2023-10-26T05:54:35.267644+0000","last_active":"2023-10-26T05:54:37.672600+0000","last_peered":"2023-10-26T05:54:37.672600+0000","last_clean":"2023-10-26T05:54:37.672600+0000","last_became_active":"2023-10-26T05:54:35.261742+0000","last_became_peered":"2023-10-26T05:54:35.261742+0000","last_unstale":"2023-10-26T05:54:37.672600+0000","last_undegraded":"2023-10-26T05:54:37.672600+0000","last_fullsized":"2023-10-26T05:54:37.672600+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:14:00.205146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672291+0000","last_change":"2023-10-26T05:54:35.266334+0000","last_active":"2023-10-26T05:54:37.672291+0000","last_peered":"2023-10-26T05:54:37.672291+0000","last_clean":"2023-10-26T05:54:37.672291+0000","last_became_active":"2023-10-26T05:54:35.265828+0000","last_became_peered":"2023-10-26T05:54:35.265828+0000","last_unstale":"2023-10-26T05:54:37.672291+0000","last_undegraded":"2023-10-26T05:54:37.672291+0000","last_fullsized":"2023-10-26T05:54:37.672291+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:56:47.066685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218036+0000","last_change":"2023-10-26T05:54:35.226995+0000","last_active":"2023-10-26T05:54:36.218036+0000","last_peered":"2023-10-26T05:54:36.218036+0000","last_clean":"2023-10-26T05:54:36.218036+0000","last_became_active":"2023-10-26T05:54:35.226383+0000","last_became_peered":"2023-10-26T05:54:35.226383+0000","last_unstale":"2023-10-26T05:54:36.218036+0000","last_undegraded":"2023-10-26T05:54:36.218036+0000","last_fullsized":"2023-10-26T05:54:36.218036+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:09:55.305666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.366853+0000","last_change":"2023-10-26T05:54:35.222284+0000","last_active":"2023-10-26T05:54:37.366853+0000","last_peered":"2023-10-26T05:54:37.366853+0000","last_clean":"2023-10-26T05:54:37.366853+0000","last_became_active":"2023-10-26T05:54:35.222007+0000","last_became_peered":"2023-10-26T05:54:35.222007+0000","last_unstale":"2023-10-26T05:54:37.366853+0000","last_undegraded":"2023-10-26T05:54:37.366853+0000","last_fullsized":"2023-10-26T05:54:37.366853+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:49.970014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673053+0000","last_change":"2023-10-26T05:54:35.266246+0000","last_active":"2023-10-26T05:54:37.673053+0000","last_peered":"2023-10-26T05:54:37.673053+0000","last_clean":"2023-10-26T05:54:37.673053+0000","last_became_active":"2023-10-26T05:54:35.265665+0000","last_became_peered":"2023-10-26T05:54:35.265665+0000","last_unstale":"2023-10-26T05:54:37.673053+0000","last_undegraded":"2023-10-26T05:54:37.673053+0000","last_fullsized":"2023-10-26T05:54:37.673053+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:03.500716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666922+0000","last_change":"2023-10-26T05:54:35.233067+0000","last_active":"2023-10-26T05:54:37.666922+0000","last_peered":"2023-10-26T05:54:37.666922+0000","last_clean":"2023-10-26T05:54:37.666922+0000","last_became_active":"2023-10-26T05:54:35.232829+0000","last_became_peered":"2023-10-26T05:54:35.232829+0000","last_unstale":"2023-10-26T05:54:37.666922+0000","last_undegraded":"2023-10-26T05:54:37.666922+0000","last_fullsized":"2023-10-26T05:54:37.666922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:07:41.415849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217615+0000","last_change":"2023-10-26T05:54:35.231235+0000","last_active":"2023-10-26T05:54:36.217615+0000","last_peered":"2023-10-26T05:54:36.217615+0000","last_clean":"2023-10-26T05:54:36.217615+0000","last_became_active":"2023-10-26T05:54:35.231053+0000","last_became_peered":"2023-10-26T05:54:35.231053+0000","last_unstale":"2023-10-26T05:54:36.217615+0000","last_undegraded":"2023-10-26T05:54:36.217615+0000","last_fullsized":"2023-10-26T05:54:36.217615+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:07.494883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667641+0000","last_change":"2023-10-26T05:54:35.244131+0000","last_active":"2023-10-26T05:54:37.667641+0000","last_peered":"2023-10-26T05:54:37.667641+0000","last_clean":"2023-10-26T05:54:37.667641+0000","last_became_active":"2023-10-26T05:54:35.243825+0000","last_became_peered":"2023-10-26T05:54:35.243825+0000","last_unstale":"2023-10-26T05:54:37.667641+0000","last_undegraded":"2023-10-26T05:54:37.667641+0000","last_fullsized":"2023-10-26T05:54:37.667641+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:24:17.574638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666860+0000","last_change":"2023-10-26T05:54:35.230464+0000","last_active":"2023-10-26T05:54:37.666860+0000","last_peered":"2023-10-26T05:54:37.666860+0000","last_clean":"2023-10-26T05:54:37.666860+0000","last_became_active":"2023-10-26T05:54:35.230052+0000","last_became_peered":"2023-10-26T05:54:35.230052+0000","last_unstale":"2023-10-26T05:54:37.666860+0000","last_undegraded":"2023-10-26T05:54:37.666860+0000","last_fullsized":"2023-10-26T05:54:37.666860+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:18:17.385537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226602+0000","last_change":"2023-10-26T05:54:35.229697+0000","last_active":"2023-10-26T05:54:36.226602+0000","last_peered":"2023-10-26T05:54:36.226602+0000","last_clean":"2023-10-26T05:54:36.226602+0000","last_became_active":"2023-10-26T05:54:35.229055+0000","last_became_peered":"2023-10-26T05:54:35.229055+0000","last_unstale":"2023-10-26T05:54:36.226602+0000","last_undegraded":"2023-10-26T05:54:36.226602+0000","last_fullsized":"2023-10-26T05:54:36.226602+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:23:18.236195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667541+0000","last_change":"2023-10-26T05:54:35.232314+0000","last_active":"2023-10-26T05:54:37.667541+0000","last_peered":"2023-10-26T05:54:37.667541+0000","last_clean":"2023-10-26T05:54:37.667541+0000","last_became_active":"2023-10-26T05:54:35.231935+0000","last_became_peered":"2023-10-26T05:54:35.231935+0000","last_unstale":"2023-10-26T05:54:37.667541+0000","last_undegraded":"2023-10-26T05:54:37.667541+0000","last_fullsized":"2023-10-26T05:54:37.667541+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:50:14.667192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.366791+0000","last_change":"2023-10-26T05:54:35.219561+0000","last_active":"2023-10-26T05:54:37.366791+0000","last_peered":"2023-10-26T05:54:37.366791+0000","last_clean":"2023-10-26T05:54:37.366791+0000","last_became_active":"2023-10-26T05:54:35.219330+0000","last_became_peered":"2023-10-26T05:54:35.219330+0000","last_unstale":"2023-10-26T05:54:37.366791+0000","last_undegraded":"2023-10-26T05:54:37.366791+0000","last_fullsized":"2023-10-26T05:54:37.366791+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:47.426860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667384+0000","last_change":"2023-10-26T05:54:35.234316+0000","last_active":"2023-10-26T05:54:37.667384+0000","last_peered":"2023-10-26T05:54:37.667384+0000","last_clean":"2023-10-26T05:54:37.667384+0000","last_became_active":"2023-10-26T05:54:35.234087+0000","last_became_peered":"2023-10-26T05:54:35.234087+0000","last_unstale":"2023-10-26T05:54:37.667384+0000","last_undegraded":"2023-10-26T05:54:37.667384+0000","last_fullsized":"2023-10-26T05:54:37.667384+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:15.559404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672477+0000","last_change":"2023-10-26T05:54:35.243047+0000","last_active":"2023-10-26T05:54:37.672477+0000","last_peered":"2023-10-26T05:54:37.672477+0000","last_clean":"2023-10-26T05:54:37.672477+0000","last_became_active":"2023-10-26T05:54:35.242375+0000","last_became_peered":"2023-10-26T05:54:35.242375+0000","last_unstale":"2023-10-26T05:54:37.672477+0000","last_undegraded":"2023-10-26T05:54:37.672477+0000","last_fullsized":"2023-10-26T05:54:37.672477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:04.885992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.367071+0000","last_change":"2023-10-26T05:54:35.232535+0000","last_active":"2023-10-26T05:54:37.367071+0000","last_peered":"2023-10-26T05:54:37.367071+0000","last_clean":"2023-10-26T05:54:37.367071+0000","last_became_active":"2023-10-26T05:54:35.232261+0000","last_became_peered":"2023-10-26T05:54:35.232261+0000","last_unstale":"2023-10-26T05:54:37.367071+0000","last_undegraded":"2023-10-26T05:54:37.367071+0000","last_fullsized":"2023-10-26T05:54:37.367071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:18:31.467937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673616+0000","last_change":"2023-10-26T05:54:35.264544+0000","last_active":"2023-10-26T05:54:37.673616+0000","last_peered":"2023-10-26T05:54:37.673616+0000","last_clean":"2023-10-26T05:54:37.673616+0000","last_became_active":"2023-10-26T05:54:35.258879+0000","last_became_peered":"2023-10-26T05:54:35.258879+0000","last_unstale":"2023-10-26T05:54:37.673616+0000","last_undegraded":"2023-10-26T05:54:37.673616+0000","last_fullsized":"2023-10-26T05:54:37.673616+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:20.465609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.366972+0000","last_change":"2023-10-26T05:54:35.236930+0000","last_active":"2023-10-26T05:54:37.366972+0000","last_peered":"2023-10-26T05:54:37.366972+0000","last_clean":"2023-10-26T05:54:37.366972+0000","last_became_active":"2023-10-26T05:54:35.235990+0000","last_became_peered":"2023-10-26T05:54:35.235990+0000","last_unstale":"2023-10-26T05:54:37.366972+0000","last_undegraded":"2023-10-26T05:54:37.366972+0000","last_fullsized":"2023-10-26T05:54:37.366972+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:41.140815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218371+0000","last_change":"2023-10-26T05:54:35.243569+0000","last_active":"2023-10-26T05:54:36.218371+0000","last_peered":"2023-10-26T05:54:36.218371+0000","last_clean":"2023-10-26T05:54:36.218371+0000","last_became_active":"2023-10-26T05:54:35.242900+0000","last_became_peered":"2023-10-26T05:54:35.242900+0000","last_unstale":"2023-10-26T05:54:36.218371+0000","last_undegraded":"2023-10-26T05:54:36.218371+0000","last_fullsized":"2023-10-26T05:54:36.218371+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:16.325052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667022+0000","last_change":"2023-10-26T05:54:35.230295+0000","last_active":"2023-10-26T05:54:37.667022+0000","last_peered":"2023-10-26T05:54:37.667022+0000","last_clean":"2023-10-26T05:54:37.667022+0000","last_became_active":"2023-10-26T05:54:35.229723+0000","last_became_peered":"2023-10-26T05:54:35.229723+0000","last_unstale":"2023-10-26T05:54:37.667022+0000","last_undegraded":"2023-10-26T05:54:37.667022+0000","last_fullsized":"2023-10-26T05:54:37.667022+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:10.955516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666667+0000","last_change":"2023-10-26T05:54:35.245335+0000","last_active":"2023-10-26T05:54:37.666667+0000","last_peered":"2023-10-26T05:54:37.666667+0000","last_clean":"2023-10-26T05:54:37.666667+0000","last_became_active":"2023-10-26T05:54:35.245113+0000","last_became_peered":"2023-10-26T05:54:35.245113+0000","last_unstale":"2023-10-26T05:54:37.666667+0000","last_undegraded":"2023-10-26T05:54:37.666667+0000","last_fullsized":"2023-10-26T05:54:37.666667+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:42.045883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.224678+0000","last_change":"2023-10-26T05:54:35.238317+0000","last_active":"2023-10-26T05:54:36.224678+0000","last_peered":"2023-10-26T05:54:36.224678+0000","last_clean":"2023-10-26T05:54:36.224678+0000","last_became_active":"2023-10-26T05:54:35.237222+0000","last_became_peered":"2023-10-26T05:54:35.237222+0000","last_unstale":"2023-10-26T05:54:36.224678+0000","last_undegraded":"2023-10-26T05:54:36.224678+0000","last_fullsized":"2023-10-26T05:54:36.224678+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:50.874423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666998+0000","last_change":"2023-10-26T05:54:35.239954+0000","last_active":"2023-10-26T05:54:37.666998+0000","last_peered":"2023-10-26T05:54:37.666998+0000","last_clean":"2023-10-26T05:54:37.666998+0000","last_became_active":"2023-10-26T05:54:35.239727+0000","last_became_peered":"2023-10-26T05:54:35.239727+0000","last_unstale":"2023-10-26T05:54:37.666998+0000","last_undegraded":"2023-10-26T05:54:37.666998+0000","last_fullsized":"2023-10-26T05:54:37.666998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:09.453871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667827+0000","last_change":"2023-10-26T05:54:35.243983+0000","last_active":"2023-10-26T05:54:37.667827+0000","last_peered":"2023-10-26T05:54:37.667827+0000","last_clean":"2023-10-26T05:54:37.667827+0000","last_became_active":"2023-10-26T05:54:35.243572+0000","last_became_peered":"2023-10-26T05:54:35.243572+0000","last_unstale":"2023-10-26T05:54:37.667827+0000","last_undegraded":"2023-10-26T05:54:37.667827+0000","last_fullsized":"2023-10-26T05:54:37.667827+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:19.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672788+0000","last_change":"2023-10-26T05:54:35.256626+0000","last_active":"2023-10-26T05:54:37.672788+0000","last_peered":"2023-10-26T05:54:37.672788+0000","last_clean":"2023-10-26T05:54:37.672788+0000","last_became_active":"2023-10-26T05:54:35.255035+0000","last_became_peered":"2023-10-26T05:54:35.255035+0000","last_unstale":"2023-10-26T05:54:37.672788+0000","last_undegraded":"2023-10-26T05:54:37.672788+0000","last_fullsized":"2023-10-26T05:54:37.672788+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:56:49.755593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226289+0000","last_change":"2023-10-26T05:54:35.247423+0000","last_active":"2023-10-26T05:54:36.226289+0000","last_peered":"2023-10-26T05:54:36.226289+0000","last_clean":"2023-10-26T05:54:36.226289+0000","last_became_active":"2023-10-26T05:54:35.245949+0000","last_became_peered":"2023-10-26T05:54:35.245949+0000","last_unstale":"2023-10-26T05:54:36.226289+0000","last_undegraded":"2023-10-26T05:54:36.226289+0000","last_fullsized":"2023-10-26T05:54:36.226289+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:25:54.809563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666742+0000","last_change":"2023-10-26T05:54:35.256672+0000","last_active":"2023-10-26T05:54:37.666742+0000","last_peered":"2023-10-26T05:54:37.666742+0000","last_clean":"2023-10-26T05:54:37.666742+0000","last_became_active":"2023-10-26T05:54:35.256263+0000","last_became_peered":"2023-10-26T05:54:35.256263+0000","last_unstale":"2023-10-26T05:54:37.666742+0000","last_undegraded":"2023-10-26T05:54:37.666742+0000","last_fullsized":"2023-10-26T05:54:37.666742+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:18:40.543626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"154'4","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.258894+0000","last_change":"2023-10-26T05:54:34.238405+0000","last_active":"2023-10-26T05:54:35.258894+0000","last_peered":"2023-10-26T05:54:35.258894+0000","last_clean":"2023-10-26T05:54:35.258894+0000","last_became_active":"2023-10-26T05:54:34.234974+0000","last_became_peered":"2023-10-26T05:54:34.234974+0000","last_unstale":"2023-10-26T05:54:35.258894+0000","last_undegraded":"2023-10-26T05:54:35.258894+0000","last_fullsized":"2023-10-26T05:54:35.258894+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:14.554639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672922+0000","last_change":"2023-10-26T05:54:35.267032+0000","last_active":"2023-10-26T05:54:37.672922+0000","last_peered":"2023-10-26T05:54:37.672922+0000","last_clean":"2023-10-26T05:54:37.672922+0000","last_became_active":"2023-10-26T05:54:35.260307+0000","last_became_peered":"2023-10-26T05:54:35.260307+0000","last_unstale":"2023-10-26T05:54:37.672922+0000","last_undegraded":"2023-10-26T05:54:37.672922+0000","last_fullsized":"2023-10-26T05:54:37.672922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:03.455094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.264825+0000","last_change":"2023-10-26T05:54:34.238394+0000","last_active":"2023-10-26T05:54:35.264825+0000","last_peered":"2023-10-26T05:54:35.264825+0000","last_clean":"2023-10-26T05:54:35.264825+0000","last_became_active":"2023-10-26T05:54:34.237301+0000","last_became_peered":"2023-10-26T05:54:34.237301+0000","last_unstale":"2023-10-26T05:54:35.264825+0000","last_undegraded":"2023-10-26T05:54:35.264825+0000","last_fullsized":"2023-10-26T05:54:35.264825+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:43.524228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673063+0000","last_change":"2023-10-26T05:54:35.256264+0000","last_active":"2023-10-26T05:54:37.673063+0000","last_peered":"2023-10-26T05:54:37.673063+0000","last_clean":"2023-10-26T05:54:37.673063+0000","last_became_active":"2023-10-26T05:54:35.253718+0000","last_became_peered":"2023-10-26T05:54:35.253718+0000","last_unstale":"2023-10-26T05:54:37.673063+0000","last_undegraded":"2023-10-26T05:54:37.673063+0000","last_fullsized":"2023-10-26T05:54:37.673063+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:26.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.241279+0000","last_change":"2023-10-26T05:54:34.228217+0000","last_active":"2023-10-26T05:54:35.241279+0000","last_peered":"2023-10-26T05:54:35.241279+0000","last_clean":"2023-10-26T05:54:35.241279+0000","last_became_active":"2023-10-26T05:54:34.227446+0000","last_became_peered":"2023-10-26T05:54:34.227446+0000","last_unstale":"2023-10-26T05:54:35.241279+0000","last_undegraded":"2023-10-26T05:54:35.241279+0000","last_fullsized":"2023-10-26T05:54:35.241279+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:31:57.251535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673897+0000","last_change":"2023-10-26T05:54:35.252615+0000","last_active":"2023-10-26T05:54:37.673897+0000","last_peered":"2023-10-26T05:54:37.673897+0000","last_clean":"2023-10-26T05:54:37.673897+0000","last_became_active":"2023-10-26T05:54:35.251779+0000","last_became_peered":"2023-10-26T05:54:35.251779+0000","last_unstale":"2023-10-26T05:54:37.673897+0000","last_undegraded":"2023-10-26T05:54:37.673897+0000","last_fullsized":"2023-10-26T05:54:37.673897+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:29:24.069539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.240969+0000","last_change":"2023-10-26T05:54:34.219413+0000","last_active":"2023-10-26T05:54:35.240969+0000","last_peered":"2023-10-26T05:54:35.240969+0000","last_clean":"2023-10-26T05:54:35.240969+0000","last_became_active":"2023-10-26T05:54:34.219109+0000","last_became_peered":"2023-10-26T05:54:34.219109+0000","last_unstale":"2023-10-26T05:54:35.240969+0000","last_undegraded":"2023-10-26T05:54:35.240969+0000","last_fullsized":"2023-10-26T05:54:35.240969+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:17:51.315680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667714+0000","last_change":"2023-10-26T05:54:35.232151+0000","last_active":"2023-10-26T05:54:37.667714+0000","last_peered":"2023-10-26T05:54:37.667714+0000","last_clean":"2023-10-26T05:54:37.667714+0000","last_became_active":"2023-10-26T05:54:35.231597+0000","last_became_peered":"2023-10-26T05:54:35.231597+0000","last_unstale":"2023-10-26T05:54:37.667714+0000","last_undegraded":"2023-10-26T05:54:37.667714+0000","last_fullsized":"2023-10-26T05:54:37.667714+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:12.959619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.240597+0000","last_change":"2023-10-26T05:54:34.225466+0000","last_active":"2023-10-26T05:54:35.240597+0000","last_peered":"2023-10-26T05:54:35.240597+0000","last_clean":"2023-10-26T05:54:35.240597+0000","last_became_active":"2023-10-26T05:54:34.225226+0000","last_became_peered":"2023-10-26T05:54:34.225226+0000","last_unstale":"2023-10-26T05:54:35.240597+0000","last_undegraded":"2023-10-26T05:54:35.240597+0000","last_fullsized":"2023-10-26T05:54:35.240597+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:43.524228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226931+0000","last_change":"2023-10-26T05:54:35.246746+0000","last_active":"2023-10-26T05:54:36.226931+0000","last_peered":"2023-10-26T05:54:36.226931+0000","last_clean":"2023-10-26T05:54:36.226931+0000","last_became_active":"2023-10-26T05:54:35.241062+0000","last_became_peered":"2023-10-26T05:54:35.241062+0000","last_unstale":"2023-10-26T05:54:36.226931+0000","last_undegraded":"2023-10-26T05:54:36.226931+0000","last_fullsized":"2023-10-26T05:54:36.226931+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:54:52.578426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.237011+0000","last_change":"2023-10-26T05:54:34.231290+0000","last_active":"2023-10-26T05:54:35.237011+0000","last_peered":"2023-10-26T05:54:35.237011+0000","last_clean":"2023-10-26T05:54:35.237011+0000","last_became_active":"2023-10-26T05:54:34.230708+0000","last_became_peered":"2023-10-26T05:54:34.230708+0000","last_unstale":"2023-10-26T05:54:35.237011+0000","last_undegraded":"2023-10-26T05:54:35.237011+0000","last_fullsized":"2023-10-26T05:54:35.237011+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:49.174594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673554+0000","last_change":"2023-10-26T05:54:35.250576+0000","last_active":"2023-10-26T05:54:37.673554+0000","last_peered":"2023-10-26T05:54:37.673554+0000","last_clean":"2023-10-26T05:54:37.673554+0000","last_became_active":"2023-10-26T05:54:35.247435+0000","last_became_peered":"2023-10-26T05:54:35.247435+0000","last_unstale":"2023-10-26T05:54:37.673554+0000","last_undegraded":"2023-10-26T05:54:37.673554+0000","last_fullsized":"2023-10-26T05:54:37.673554+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:15.752245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.223618+0000","last_change":"2023-10-26T05:54:34.229066+0000","last_active":"2023-10-26T05:54:35.223618+0000","last_peered":"2023-10-26T05:54:35.223618+0000","last_clean":"2023-10-26T05:54:35.223618+0000","last_became_active":"2023-10-26T05:54:34.228464+0000","last_became_peered":"2023-10-26T05:54:34.228464+0000","last_unstale":"2023-10-26T05:54:35.223618+0000","last_undegraded":"2023-10-26T05:54:35.223618+0000","last_fullsized":"2023-10-26T05:54:35.223618+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:02:38.842997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217997+0000","last_change":"2023-10-26T05:54:35.229966+0000","last_active":"2023-10-26T05:54:36.217997+0000","last_peered":"2023-10-26T05:54:36.217997+0000","last_clean":"2023-10-26T05:54:36.217997+0000","last_became_active":"2023-10-26T05:54:35.229248+0000","last_became_peered":"2023-10-26T05:54:35.229248+0000","last_unstale":"2023-10-26T05:54:36.217997+0000","last_undegraded":"2023-10-26T05:54:36.217997+0000","last_fullsized":"2023-10-26T05:54:36.217997+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:28.679833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.221724+0000","last_change":"2023-10-26T05:54:34.237160+0000","last_active":"2023-10-26T05:54:35.221724+0000","last_peered":"2023-10-26T05:54:35.221724+0000","last_clean":"2023-10-26T05:54:35.221724+0000","last_became_active":"2023-10-26T05:54:34.236869+0000","last_became_peered":"2023-10-26T05:54:34.236869+0000","last_unstale":"2023-10-26T05:54:35.221724+0000","last_undegraded":"2023-10-26T05:54:35.221724+0000","last_fullsized":"2023-10-26T05:54:35.221724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:57.286656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666955+0000","last_change":"2023-10-26T05:54:35.231619+0000","last_active":"2023-10-26T05:54:37.666955+0000","last_peered":"2023-10-26T05:54:37.666955+0000","last_clean":"2023-10-26T05:54:37.666955+0000","last_became_active":"2023-10-26T05:54:35.230748+0000","last_became_peered":"2023-10-26T05:54:35.230748+0000","last_unstale":"2023-10-26T05:54:37.666955+0000","last_undegraded":"2023-10-26T05:54:37.666955+0000","last_fullsized":"2023-10-26T05:54:37.666955+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:04:06.063251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.227848+0000","last_change":"2023-10-26T05:54:34.227954+0000","last_active":"2023-10-26T05:54:35.227848+0000","last_peered":"2023-10-26T05:54:35.227848+0000","last_clean":"2023-10-26T05:54:35.227848+0000","last_became_active":"2023-10-26T05:54:34.227315+0000","last_became_peered":"2023-10-26T05:54:34.227315+0000","last_unstale":"2023-10-26T05:54:35.227848+0000","last_undegraded":"2023-10-26T05:54:35.227848+0000","last_fullsized":"2023-10-26T05:54:35.227848+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:57.286656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226148+0000","last_change":"2023-10-26T05:54:35.242187+0000","last_active":"2023-10-26T05:54:36.226148+0000","last_peered":"2023-10-26T05:54:36.226148+0000","last_clean":"2023-10-26T05:54:36.226148+0000","last_became_active":"2023-10-26T05:54:35.240648+0000","last_became_peered":"2023-10-26T05:54:35.240648+0000","last_unstale":"2023-10-26T05:54:36.226148+0000","last_undegraded":"2023-10-26T05:54:36.226148+0000","last_fullsized":"2023-10-26T05:54:36.226148+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:58.785279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.226332+0000","last_change":"2023-10-26T05:54:34.228604+0000","last_active":"2023-10-26T05:54:35.226332+0000","last_peered":"2023-10-26T05:54:35.226332+0000","last_clean":"2023-10-26T05:54:35.226332+0000","last_became_active":"2023-10-26T05:54:34.228363+0000","last_became_peered":"2023-10-26T05:54:34.228363+0000","last_unstale":"2023-10-26T05:54:35.226332+0000","last_undegraded":"2023-10-26T05:54:35.226332+0000","last_fullsized":"2023-10-26T05:54:35.226332+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:19.460844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672955+0000","last_change":"2023-10-26T05:54:35.239171+0000","last_active":"2023-10-26T05:54:37.672955+0000","last_peered":"2023-10-26T05:54:37.672955+0000","last_clean":"2023-10-26T05:54:37.672955+0000","last_became_active":"2023-10-26T05:54:35.238560+0000","last_became_peered":"2023-10-26T05:54:35.238560+0000","last_unstale":"2023-10-26T05:54:37.672955+0000","last_undegraded":"2023-10-26T05:54:37.672955+0000","last_fullsized":"2023-10-26T05:54:37.672955+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:21:05.364628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"154'3","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.262449+0000","last_change":"2023-10-26T05:54:34.241288+0000","last_active":"2023-10-26T05:54:35.262449+0000","last_peered":"2023-10-26T05:54:35.262449+0000","last_clean":"2023-10-26T05:54:35.262449+0000","last_became_active":"2023-10-26T05:54:34.240585+0000","last_became_peered":"2023-10-26T05:54:34.240585+0000","last_unstale":"2023-10-26T05:54:35.262449+0000","last_undegraded":"2023-10-26T05:54:35.262449+0000","last_fullsized":"2023-10-26T05:54:35.262449+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:06.490118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672355+0000","last_change":"2023-10-26T05:54:35.268054+0000","last_active":"2023-10-26T05:54:37.672355+0000","last_peered":"2023-10-26T05:54:37.672355+0000","last_clean":"2023-10-26T05:54:37.672355+0000","last_became_active":"2023-10-26T05:54:35.267577+0000","last_became_peered":"2023-10-26T05:54:35.267577+0000","last_unstale":"2023-10-26T05:54:37.672355+0000","last_undegraded":"2023-10-26T05:54:37.672355+0000","last_fullsized":"2023-10-26T05:54:37.672355+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:50.801746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"154'6","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.268644+0000","last_change":"2023-10-26T05:54:34.245988+0000","last_active":"2023-10-26T05:54:35.268644+0000","last_peered":"2023-10-26T05:54:35.268644+0000","last_clean":"2023-10-26T05:54:35.268644+0000","last_became_active":"2023-10-26T05:54:34.245363+0000","last_became_peered":"2023-10-26T05:54:34.245363+0000","last_unstale":"2023-10-26T05:54:35.268644+0000","last_undegraded":"2023-10-26T05:54:35.268644+0000","last_fullsized":"2023-10-26T05:54:35.268644+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:18.171476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.668063+0000","last_change":"2023-10-26T05:54:35.229031+0000","last_active":"2023-10-26T05:54:37.668063+0000","last_peered":"2023-10-26T05:54:37.668063+0000","last_clean":"2023-10-26T05:54:37.668063+0000","last_became_active":"2023-10-26T05:54:35.228680+0000","last_became_peered":"2023-10-26T05:54:35.228680+0000","last_unstale":"2023-10-26T05:54:37.668063+0000","last_undegraded":"2023-10-26T05:54:37.668063+0000","last_fullsized":"2023-10-26T05:54:37.668063+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:38.417610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.227985+0000","last_change":"2023-10-26T05:54:34.226642+0000","last_active":"2023-10-26T05:54:35.227985+0000","last_peered":"2023-10-26T05:54:35.227985+0000","last_clean":"2023-10-26T05:54:35.227985+0000","last_became_active":"2023-10-26T05:54:34.226063+0000","last_became_peered":"2023-10-26T05:54:34.226063+0000","last_unstale":"2023-10-26T05:54:35.227985+0000","last_undegraded":"2023-10-26T05:54:35.227985+0000","last_fullsized":"2023-10-26T05:54:35.227985+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:35.772656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217893+0000","last_change":"2023-10-26T05:54:35.254873+0000","last_active":"2023-10-26T05:54:36.217893+0000","last_peered":"2023-10-26T05:54:36.217893+0000","last_clean":"2023-10-26T05:54:36.217893+0000","last_became_active":"2023-10-26T05:54:35.254624+0000","last_became_peered":"2023-10-26T05:54:35.254624+0000","last_unstale":"2023-10-26T05:54:36.217893+0000","last_undegraded":"2023-10-26T05:54:36.217893+0000","last_fullsized":"2023-10-26T05:54:36.217893+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:50.874423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.231640+0000","last_change":"2023-10-26T05:54:34.230187+0000","last_active":"2023-10-26T05:54:35.231640+0000","last_peered":"2023-10-26T05:54:35.231640+0000","last_clean":"2023-10-26T05:54:35.231640+0000","last_became_active":"2023-10-26T05:54:34.229848+0000","last_became_peered":"2023-10-26T05:54:34.229848+0000","last_unstale":"2023-10-26T05:54:35.231640+0000","last_undegraded":"2023-10-26T05:54:35.231640+0000","last_fullsized":"2023-10-26T05:54:35.231640+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:29.154810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673304+0000","last_change":"2023-10-26T05:54:35.263246+0000","last_active":"2023-10-26T05:54:37.673304+0000","last_peered":"2023-10-26T05:54:37.673304+0000","last_clean":"2023-10-26T05:54:37.673304+0000","last_became_active":"2023-10-26T05:54:35.262386+0000","last_became_peered":"2023-10-26T05:54:35.262386+0000","last_unstale":"2023-10-26T05:54:37.673304+0000","last_undegraded":"2023-10-26T05:54:37.673304+0000","last_fullsized":"2023-10-26T05:54:37.673304+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:31:47.069483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"154'8","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.276777+0000","last_change":"2023-10-26T05:54:34.242272+0000","last_active":"2023-10-26T05:54:35.276777+0000","last_peered":"2023-10-26T05:54:35.276777+0000","last_clean":"2023-10-26T05:54:35.276777+0000","last_became_active":"2023-10-26T05:54:34.240219+0000","last_became_peered":"2023-10-26T05:54:34.240219+0000","last_unstale":"2023-10-26T05:54:35.276777+0000","last_undegraded":"2023-10-26T05:54:35.276777+0000","last_fullsized":"2023-10-26T05:54:35.276777+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:13:54.872759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673743+0000","last_change":"2023-10-26T05:54:35.250174+0000","last_active":"2023-10-26T05:54:37.673743+0000","last_peered":"2023-10-26T05:54:37.673743+0000","last_clean":"2023-10-26T05:54:37.673743+0000","last_became_active":"2023-10-26T05:54:35.246225+0000","last_became_peered":"2023-10-26T05:54:35.246225+0000","last_unstale":"2023-10-26T05:54:37.673743+0000","last_undegraded":"2023-10-26T05:54:37.673743+0000","last_fullsized":"2023-10-26T05:54:37.673743+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:50.179358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.263990+0000","last_change":"2023-10-26T05:54:34.237939+0000","last_active":"2023-10-26T05:54:35.263990+0000","last_peered":"2023-10-26T05:54:35.263990+0000","last_clean":"2023-10-26T05:54:35.263990+0000","last_became_active":"2023-10-26T05:54:34.236275+0000","last_became_peered":"2023-10-26T05:54:34.236275+0000","last_unstale":"2023-10-26T05:54:35.263990+0000","last_undegraded":"2023-10-26T05:54:35.263990+0000","last_fullsized":"2023-10-26T05:54:35.263990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:38.629159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226056+0000","last_change":"2023-10-26T05:54:35.229868+0000","last_active":"2023-10-26T05:54:36.226056+0000","last_peered":"2023-10-26T05:54:36.226056+0000","last_clean":"2023-10-26T05:54:36.226056+0000","last_became_active":"2023-10-26T05:54:35.229266+0000","last_became_peered":"2023-10-26T05:54:35.229266+0000","last_unstale":"2023-10-26T05:54:36.226056+0000","last_undegraded":"2023-10-26T05:54:36.226056+0000","last_fullsized":"2023-10-26T05:54:36.226056+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:09:55.305666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.245782+0000","last_change":"2023-10-26T05:54:34.238890+0000","last_active":"2023-10-26T05:54:35.245782+0000","last_peered":"2023-10-26T05:54:35.245782+0000","last_clean":"2023-10-26T05:54:35.245782+0000","last_became_active":"2023-10-26T05:54:34.238622+0000","last_became_peered":"2023-10-26T05:54:34.238622+0000","last_unstale":"2023-10-26T05:54:35.245782+0000","last_undegraded":"2023-10-26T05:54:35.245782+0000","last_fullsized":"2023-10-26T05:54:35.245782+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:43.095652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667367+0000","last_change":"2023-10-26T05:54:35.237989+0000","last_active":"2023-10-26T05:54:37.667367+0000","last_peered":"2023-10-26T05:54:37.667367+0000","last_clean":"2023-10-26T05:54:37.667367+0000","last_became_active":"2023-10-26T05:54:35.237809+0000","last_became_peered":"2023-10-26T05:54:35.237809+0000","last_unstale":"2023-10-26T05:54:37.667367+0000","last_undegraded":"2023-10-26T05:54:37.667367+0000","last_fullsized":"2023-10-26T05:54:37.667367+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:40.556192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.254521+0000","last_change":"2023-10-26T05:54:34.239412+0000","last_active":"2023-10-26T05:54:35.254521+0000","last_peered":"2023-10-26T05:54:35.254521+0000","last_clean":"2023-10-26T05:54:35.254521+0000","last_became_active":"2023-10-26T05:54:34.238306+0000","last_became_peered":"2023-10-26T05:54:34.238306+0000","last_unstale":"2023-10-26T05:54:35.254521+0000","last_undegraded":"2023-10-26T05:54:35.254521+0000","last_fullsized":"2023-10-26T05:54:35.254521+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:37.435449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218143+0000","last_change":"2023-10-26T05:54:35.233029+0000","last_active":"2023-10-26T05:54:36.218143+0000","last_peered":"2023-10-26T05:54:36.218143+0000","last_clean":"2023-10-26T05:54:36.218143+0000","last_became_active":"2023-10-26T05:54:35.232401+0000","last_became_peered":"2023-10-26T05:54:35.232401+0000","last_unstale":"2023-10-26T05:54:36.218143+0000","last_undegraded":"2023-10-26T05:54:36.218143+0000","last_fullsized":"2023-10-26T05:54:36.218143+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:58.785279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.228260+0000","last_change":"2023-10-26T05:54:34.237449+0000","last_active":"2023-10-26T05:54:35.228260+0000","last_peered":"2023-10-26T05:54:35.228260+0000","last_clean":"2023-10-26T05:54:35.228260+0000","last_became_active":"2023-10-26T05:54:34.236646+0000","last_became_peered":"2023-10-26T05:54:34.236646+0000","last_unstale":"2023-10-26T05:54:35.228260+0000","last_undegraded":"2023-10-26T05:54:35.228260+0000","last_fullsized":"2023-10-26T05:54:35.228260+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:29.154810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667163+0000","last_change":"2023-10-26T05:54:35.236485+0000","last_active":"2023-10-26T05:54:37.667163+0000","last_peered":"2023-10-26T05:54:37.667163+0000","last_clean":"2023-10-26T05:54:37.667163+0000","last_became_active":"2023-10-26T05:54:35.236273+0000","last_became_peered":"2023-10-26T05:54:35.236273+0000","last_unstale":"2023-10-26T05:54:37.667163+0000","last_undegraded":"2023-10-26T05:54:37.667163+0000","last_fullsized":"2023-10-26T05:54:37.667163+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:47:47.463495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.263089+0000","last_change":"2023-10-26T05:54:34.229691+0000","last_active":"2023-10-26T05:54:35.263089+0000","last_peered":"2023-10-26T05:54:35.263089+0000","last_clean":"2023-10-26T05:54:35.263089+0000","last_became_active":"2023-10-26T05:54:34.229093+0000","last_became_peered":"2023-10-26T05:54:34.229093+0000","last_unstale":"2023-10-26T05:54:35.263089+0000","last_undegraded":"2023-10-26T05:54:35.263089+0000","last_fullsized":"2023-10-26T05:54:35.263089+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:42.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.224894+0000","last_change":"2023-10-26T05:54:35.258639+0000","last_active":"2023-10-26T05:54:36.224894+0000","last_peered":"2023-10-26T05:54:36.224894+0000","last_clean":"2023-10-26T05:54:36.224894+0000","last_became_active":"2023-10-26T05:54:35.258010+0000","last_became_peered":"2023-10-26T05:54:35.258010+0000","last_unstale":"2023-10-26T05:54:36.224894+0000","last_undegraded":"2023-10-26T05:54:36.224894+0000","last_fullsized":"2023-10-26T05:54:36.224894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:04.885992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.225537+0000","last_change":"2023-10-26T05:54:34.231906+0000","last_active":"2023-10-26T05:54:35.225537+0000","last_peered":"2023-10-26T05:54:35.225537+0000","last_clean":"2023-10-26T05:54:35.225537+0000","last_became_active":"2023-10-26T05:54:34.231677+0000","last_became_peered":"2023-10-26T05:54:34.231677+0000","last_unstale":"2023-10-26T05:54:35.225537+0000","last_undegraded":"2023-10-26T05:54:35.225537+0000","last_fullsized":"2023-10-26T05:54:35.225537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:02.450329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217795+0000","last_change":"2023-10-26T05:54:35.222747+0000","last_active":"2023-10-26T05:54:36.217795+0000","last_peered":"2023-10-26T05:54:36.217795+0000","last_clean":"2023-10-26T05:54:36.217795+0000","last_became_active":"2023-10-26T05:54:35.222457+0000","last_became_peered":"2023-10-26T05:54:35.222457+0000","last_unstale":"2023-10-26T05:54:36.217795+0000","last_undegraded":"2023-10-26T05:54:36.217795+0000","last_fullsized":"2023-10-26T05:54:36.217795+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:43.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.224734+0000","last_change":"2023-10-26T05:54:34.236055+0000","last_active":"2023-10-26T05:54:35.224734+0000","last_peered":"2023-10-26T05:54:35.224734+0000","last_clean":"2023-10-26T05:54:35.224734+0000","last_became_active":"2023-10-26T05:54:34.235715+0000","last_became_peered":"2023-10-26T05:54:34.235715+0000","last_unstale":"2023-10-26T05:54:35.224734+0000","last_undegraded":"2023-10-26T05:54:35.224734+0000","last_fullsized":"2023-10-26T05:54:35.224734+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:13:54.872759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673591+0000","last_change":"2023-10-26T05:54:35.248973+0000","last_active":"2023-10-26T05:54:37.673591+0000","last_peered":"2023-10-26T05:54:37.673591+0000","last_clean":"2023-10-26T05:54:37.673591+0000","last_became_active":"2023-10-26T05:54:35.248189+0000","last_became_peered":"2023-10-26T05:54:35.248189+0000","last_unstale":"2023-10-26T05:54:37.673591+0000","last_undegraded":"2023-10-26T05:54:37.673591+0000","last_fullsized":"2023-10-26T05:54:37.673591+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:23:18.236195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.215772+0000","last_change":"2023-10-26T05:54:34.235155+0000","last_active":"2023-10-26T05:54:35.215772+0000","last_peered":"2023-10-26T05:54:35.215772+0000","last_clean":"2023-10-26T05:54:35.215772+0000","last_became_active":"2023-10-26T05:54:34.234905+0000","last_became_peered":"2023-10-26T05:54:34.234905+0000","last_unstale":"2023-10-26T05:54:35.215772+0000","last_undegraded":"2023-10-26T05:54:35.215772+0000","last_fullsized":"2023-10-26T05:54:35.215772+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:03.961130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225235+0000","last_change":"2023-10-26T05:54:35.233645+0000","last_active":"2023-10-26T05:54:36.225235+0000","last_peered":"2023-10-26T05:54:36.225235+0000","last_clean":"2023-10-26T05:54:36.225235+0000","last_became_active":"2023-10-26T05:54:35.230637+0000","last_became_peered":"2023-10-26T05:54:35.230637+0000","last_unstale":"2023-10-26T05:54:36.225235+0000","last_undegraded":"2023-10-26T05:54:36.225235+0000","last_fullsized":"2023-10-26T05:54:36.225235+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:56:49.755593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.257283+0000","last_change":"2023-10-26T05:54:34.226691+0000","last_active":"2023-10-26T05:54:35.257283+0000","last_peered":"2023-10-26T05:54:35.257283+0000","last_clean":"2023-10-26T05:54:35.257283+0000","last_became_active":"2023-10-26T05:54:34.225872+0000","last_became_peered":"2023-10-26T05:54:34.225872+0000","last_unstale":"2023-10-26T05:54:35.257283+0000","last_undegraded":"2023-10-26T05:54:35.257283+0000","last_fullsized":"2023-10-26T05:54:35.257283+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:07.434465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667123+0000","last_change":"2023-10-26T05:54:35.243113+0000","last_active":"2023-10-26T05:54:37.667123+0000","last_peered":"2023-10-26T05:54:37.667123+0000","last_clean":"2023-10-26T05:54:37.667123+0000","last_became_active":"2023-10-26T05:54:35.242839+0000","last_became_peered":"2023-10-26T05:54:35.242839+0000","last_unstale":"2023-10-26T05:54:37.667123+0000","last_undegraded":"2023-10-26T05:54:37.667123+0000","last_fullsized":"2023-10-26T05:54:37.667123+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:32:57.161029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.234133+0000","last_change":"2023-10-26T05:54:34.235620+0000","last_active":"2023-10-26T05:54:35.234133+0000","last_peered":"2023-10-26T05:54:35.234133+0000","last_clean":"2023-10-26T05:54:35.234133+0000","last_became_active":"2023-10-26T05:54:34.231181+0000","last_became_peered":"2023-10-26T05:54:34.231181+0000","last_unstale":"2023-10-26T05:54:35.234133+0000","last_undegraded":"2023-10-26T05:54:35.234133+0000","last_fullsized":"2023-10-26T05:54:35.234133+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:11.954854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218643+0000","last_change":"2023-10-26T05:54:35.243730+0000","last_active":"2023-10-26T05:54:36.218643+0000","last_peered":"2023-10-26T05:54:36.218643+0000","last_clean":"2023-10-26T05:54:36.218643+0000","last_became_active":"2023-10-26T05:54:35.243240+0000","last_became_peered":"2023-10-26T05:54:35.243240+0000","last_unstale":"2023-10-26T05:54:36.218643+0000","last_undegraded":"2023-10-26T05:54:36.218643+0000","last_fullsized":"2023-10-26T05:54:36.218643+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:21:24.235337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.221160+0000","last_change":"2023-10-26T05:54:34.234275+0000","last_active":"2023-10-26T05:54:35.221160+0000","last_peered":"2023-10-26T05:54:35.221160+0000","last_clean":"2023-10-26T05:54:35.221160+0000","last_became_active":"2023-10-26T05:54:34.234042+0000","last_became_peered":"2023-10-26T05:54:34.234042+0000","last_unstale":"2023-10-26T05:54:35.221160+0000","last_undegraded":"2023-10-26T05:54:35.221160+0000","last_fullsized":"2023-10-26T05:54:35.221160+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:21.281518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.224676+0000","last_change":"2023-10-26T05:54:35.241333+0000","last_active":"2023-10-26T05:54:36.224676+0000","last_peered":"2023-10-26T05:54:36.224676+0000","last_clean":"2023-10-26T05:54:36.224676+0000","last_became_active":"2023-10-26T05:54:35.240402+0000","last_became_peered":"2023-10-26T05:54:35.240402+0000","last_unstale":"2023-10-26T05:54:36.224676+0000","last_undegraded":"2023-10-26T05:54:36.224676+0000","last_fullsized":"2023-10-26T05:54:36.224676+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:50.801746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.215608+0000","last_change":"2023-10-26T05:54:34.228407+0000","last_active":"2023-10-26T05:54:35.215608+0000","last_peered":"2023-10-26T05:54:35.215608+0000","last_clean":"2023-10-26T05:54:35.215608+0000","last_became_active":"2023-10-26T05:54:34.227889+0000","last_became_peered":"2023-10-26T05:54:34.227889+0000","last_unstale":"2023-10-26T05:54:35.215608+0000","last_undegraded":"2023-10-26T05:54:35.215608+0000","last_fullsized":"2023-10-26T05:54:35.215608+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:46:25.901115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672240+0000","last_change":"2023-10-26T05:54:35.249718+0000","last_active":"2023-10-26T05:54:37.672240+0000","last_peered":"2023-10-26T05:54:37.672240+0000","last_clean":"2023-10-26T05:54:37.672240+0000","last_became_active":"2023-10-26T05:54:35.248964+0000","last_became_peered":"2023-10-26T05:54:35.248964+0000","last_unstale":"2023-10-26T05:54:37.672240+0000","last_undegraded":"2023-10-26T05:54:37.672240+0000","last_fullsized":"2023-10-26T05:54:37.672240+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:55.060596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.221386+0000","last_change":"2023-10-26T05:54:34.227337+0000","last_active":"2023-10-26T05:54:35.221386+0000","last_peered":"2023-10-26T05:54:35.221386+0000","last_clean":"2023-10-26T05:54:35.221386+0000","last_became_active":"2023-10-26T05:54:34.227083+0000","last_became_peered":"2023-10-26T05:54:34.227083+0000","last_unstale":"2023-10-26T05:54:35.221386+0000","last_undegraded":"2023-10-26T05:54:35.221386+0000","last_fullsized":"2023-10-26T05:54:35.221386+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:05:18.895959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225613+0000","last_change":"2023-10-26T05:54:35.242707+0000","last_active":"2023-10-26T05:54:36.225613+0000","last_peered":"2023-10-26T05:54:36.225613+0000","last_clean":"2023-10-26T05:54:36.225613+0000","last_became_active":"2023-10-26T05:54:35.241479+0000","last_became_peered":"2023-10-26T05:54:35.241479+0000","last_unstale":"2023-10-26T05:54:36.225613+0000","last_undegraded":"2023-10-26T05:54:36.225613+0000","last_fullsized":"2023-10-26T05:54:36.225613+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:28.679833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.234976+0000","last_change":"2023-10-26T05:54:34.236735+0000","last_active":"2023-10-26T05:54:35.234976+0000","last_peered":"2023-10-26T05:54:35.234976+0000","last_clean":"2023-10-26T05:54:35.234976+0000","last_became_active":"2023-10-26T05:54:34.233447+0000","last_became_peered":"2023-10-26T05:54:34.233447+0000","last_unstale":"2023-10-26T05:54:35.234976+0000","last_undegraded":"2023-10-26T05:54:35.234976+0000","last_fullsized":"2023-10-26T05:54:35.234976+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:34.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666909+0000","last_change":"2023-10-26T05:54:35.239344+0000","last_active":"2023-10-26T05:54:37.666909+0000","last_peered":"2023-10-26T05:54:37.666909+0000","last_clean":"2023-10-26T05:54:37.666909+0000","last_became_active":"2023-10-26T05:54:35.239200+0000","last_became_peered":"2023-10-26T05:54:35.239200+0000","last_unstale":"2023-10-26T05:54:37.666909+0000","last_undegraded":"2023-10-26T05:54:37.666909+0000","last_fullsized":"2023-10-26T05:54:37.666909+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:17:41.937810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.227546+0000","last_change":"2023-10-26T05:54:34.234238+0000","last_active":"2023-10-26T05:54:35.227546+0000","last_peered":"2023-10-26T05:54:35.227546+0000","last_clean":"2023-10-26T05:54:35.227546+0000","last_became_active":"2023-10-26T05:54:34.234006+0000","last_became_peered":"2023-10-26T05:54:34.234006+0000","last_unstale":"2023-10-26T05:54:35.227546+0000","last_undegraded":"2023-10-26T05:54:35.227546+0000","last_fullsized":"2023-10-26T05:54:35.227546+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:12.750236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218250+0000","last_change":"2023-10-26T05:54:35.237714+0000","last_active":"2023-10-26T05:54:36.218250+0000","last_peered":"2023-10-26T05:54:36.218250+0000","last_clean":"2023-10-26T05:54:36.218250+0000","last_became_active":"2023-10-26T05:54:35.237486+0000","last_became_peered":"2023-10-26T05:54:35.237486+0000","last_unstale":"2023-10-26T05:54:36.218250+0000","last_undegraded":"2023-10-26T05:54:36.218250+0000","last_fullsized":"2023-10-26T05:54:36.218250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:05.899408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.216019+0000","last_change":"2023-10-26T05:54:34.226096+0000","last_active":"2023-10-26T05:54:35.216019+0000","last_peered":"2023-10-26T05:54:35.216019+0000","last_clean":"2023-10-26T05:54:35.216019+0000","last_became_active":"2023-10-26T05:54:34.224494+0000","last_became_peered":"2023-10-26T05:54:34.224494+0000","last_unstale":"2023-10-26T05:54:35.216019+0000","last_undegraded":"2023-10-26T05:54:35.216019+0000","last_fullsized":"2023-10-26T05:54:35.216019+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:56.591989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673231+0000","last_change":"2023-10-26T05:54:35.256899+0000","last_active":"2023-10-26T05:54:37.673231+0000","last_peered":"2023-10-26T05:54:37.673231+0000","last_clean":"2023-10-26T05:54:37.673231+0000","last_became_active":"2023-10-26T05:54:35.255793+0000","last_became_peered":"2023-10-26T05:54:35.255793+0000","last_unstale":"2023-10-26T05:54:37.673231+0000","last_undegraded":"2023-10-26T05:54:37.673231+0000","last_fullsized":"2023-10-26T05:54:37.673231+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:25:54.809563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.258813+0000","last_change":"2023-10-26T05:54:34.241834+0000","last_active":"2023-10-26T05:54:35.258813+0000","last_peered":"2023-10-26T05:54:35.258813+0000","last_clean":"2023-10-26T05:54:35.258813+0000","last_became_active":"2023-10-26T05:54:34.232265+0000","last_became_peered":"2023-10-26T05:54:34.232265+0000","last_unstale":"2023-10-26T05:54:35.258813+0000","last_undegraded":"2023-10-26T05:54:35.258813+0000","last_fullsized":"2023-10-26T05:54:35.258813+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:25:10.434638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.233346+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:54:35.233346+0000","last_peered":"2023-10-26T05:54:35.233346+0000","last_clean":"2023-10-26T05:54:35.233346+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:54:35.233346+0000","last_undegraded":"2023-10-26T05:54:35.233346+0000","last_fullsized":"2023-10-26T05:54:35.233346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1424,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193887,"statfs":{"total":200000000,"available":198540986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.39600000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.347},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.169,"5min":1.286,"15min":1.3100000000000001},"last":0.93600000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94799999999999995,"15min":1.018},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.236,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.998}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.93100000000000005},"min":{"1min":0.67100000000000004,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.2849999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69199999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.214,"15min":1.4330000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3129999999999999,"5min":1.375,"15min":1.375},"last":1.03}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.1679999999999999,"5min":1.639,"15min":1.639},"last":1.109},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96399999999999997,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3149999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.96999999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0169999999999999,"15min":1.1000000000000001},"min":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.58699999999999997},"max":{"1min":1.266,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":1.147},{"interface":"front","average":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0720000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.53800000000000003},"max":{"1min":1.3029999999999999,"5min":1.702,"15min":1.702},"last":1.262}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.081,"15min":1.1100000000000001},"min":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.59599999999999997},"max":{"1min":1.3640000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0609999999999999,"15min":1.0509999999999999},"min":{"1min":0.66700000000000004,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.6180000000000001},"last":1.0840000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":847,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194464,"statfs":{"total":200000000,"available":199131992,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.034},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.298,"5min":1.298,"15min":1.3},"last":0.5}]},{"osd":1,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.62,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.405},"last":0.91600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.82099999999999995},"min":{"1min":0.54800000000000004,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1990000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.435,"15min":0.435},"max":{"1min":1.3,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.004}]},{"osd":3,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97999999999999998,"15min":0.90400000000000003},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2250000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.113},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.056,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.3700000000000001,"5min":1.399,"15min":1.399},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.85699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.212},{"interface":"front","average":{"1min":1.002,"5min":0.97799999999999998,"15min":0.97899999999999998},"min":{"1min":0.64500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.351,"5min":1.375,"15min":1.375},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0469999999999999},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.554},"last":1.167},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.76800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.25,"5min":1.512,"15min":1.512},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1060000000000001},"min":{"1min":0.96799999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4550000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133492,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":0.94999999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.381,"5min":1.579,"15min":1.579},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.329}]},{"osd":1,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.53300000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.089,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.464,"15min":1.464},"last":0.85999999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.2110000000000001,"5min":1.319,"15min":1.5089999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.92700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2190000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.59299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.333,"5min":1.448,"15min":1.548},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.238,"5min":1.4139999999999999,"15min":1.587},"last":0.76000000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94099999999999995,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2769999999999999,"5min":1.286,"15min":2.1909999999999998},"last":1.282},{"interface":"front","average":{"1min":1.091,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.536,"15min":1.536},"last":1.4370000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.96899999999999997,"15min":1.018},"min":{"1min":0.63500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0169999999999999,"15min":1.032},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.4199999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.442,"15min":0.442},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.238}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.85999999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.248,"5min":1.2569999999999999,"15min":1.347},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.53100000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.143,"5min":1.502,"15min":1.502},"last":0.874}]},{"osd":1,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.878,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.25,"15min":1.375},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.96199999999999997,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":1.3440000000000001,"15min":1.4390000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90700000000000003,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.107,"5min":1.373,"15min":1.373},"last":1.022}]},{"osd":3,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.397,"5min":1.397,"15min":2.0190000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.71099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.232,"5min":1.3300000000000001,"15min":1.4359999999999999},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.88900000000000001},"min":{"1min":0.65600000000000003,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3440000000000001,"15min":1.3859999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.323,"5min":1.4179999999999999,"15min":1.673},"last":1.113}]},{"osd":6,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0289999999999999,"15min":1.004},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.42099999999999999},"max":{"1min":1.762,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.73799999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3540000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.089}]},{"osd":7,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.68600000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.7509999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.137,"15min":1.119},"min":{"1min":0.74299999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"max":{"1min":1.484,"5min":1.728,"15min":1.728},"last":1.204}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133444,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.29599999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.76800000000000002,"5min":1.385,"15min":1.385},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.94699999999999995,"5min":1.115,"15min":1.2689999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80100000000000005,"5min":1.349,"15min":1.349},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65900000000000003,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.99299999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.497},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":1.1619999999999999,"5min":1.236,"15min":1.4199999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":0.83499999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.40000000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.073,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.95599999999999996},"min":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55800000000000005},"max":{"1min":1.113,"5min":1.413,"15min":1.478},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.91200000000000003},"min":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.59799999999999998},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93300000000000005,"15min":0.97999999999999998},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.165,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.254,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":0.93200000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.232,"5min":2.004,"15min":2.004},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194465,"statfs":{"total":200000000,"available":199132988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.54400000000000004},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.88300000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71799999999999997,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1120000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.67600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.198,"5min":1.226,"15min":1.2609999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68999999999999995,"15min":0.60199999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.157,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.74399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.149,"5min":1.333,"15min":2.8809999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.089,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.83899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1060000000000001,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.54400000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1459999999999999,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.91900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.476,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.99299999999999999,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.653,"15min":1.653},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.002,"15min":0.92700000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.02,"5min":1.077,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.419,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.97799999999999998,"15min":0.91000000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.173,"5min":1.581,"15min":1.581},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.0900000000000001,"15min":1.0449999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.367,"5min":1.823,"15min":15.352},"last":0.89800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.157,"15min":1.157},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83599999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27700000000000002},"max":{"1min":1.087,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.89500000000000002,"5min":1.298,"15min":1.298},"last":0.505}]},{"osd":4,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.64500000000000002,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1659999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.27,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.621,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3620000000000001,"5min":1.492,"15min":1.492},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.92400000000000004,"15min":0.92900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3029999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96699999999999997},"min":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.468,"15min":1.837},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.012,"15min":1.022},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.2330000000000001,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.046,"15min":1.0840000000000001},"min":{"1min":0.63200000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4099999999999999,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":0.879}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193889,"statfs":{"total":200000000,"available":198543236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78100000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.69899999999999995,"15min":0.62},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85899999999999999,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.093,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1140000000000001,"15min":1.177},"last":0.436}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.626},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.99299999999999999,"5min":1.409,"15min":1.409},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80500000000000005,"15min":0.70999999999999996},"min":{"1min":0.33400000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.2629999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.92100000000000004},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2769999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.92600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1399999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.90900000000000003},"min":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.123,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.873}]},{"osd":7,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.004,"15min":0.996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4399999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":0.98999999999999999},"min":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.83199999999999996}]}]}],"pool_statfs":[]}} 2023-10-26T05:54:42.760 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:54:42.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- 172.21.15.71:0/731495859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23700039b0 msgr2=0x7f2370003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:42.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 --2- 172.21.15.71:0/731495859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23700039b0 0x7f2370003dd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f23800016b0 tx=0x7f238001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:54:42.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2389c5f640 1 -- 172.21.15.71:0/731495859 reap_dead start 2023-10-26T05:54:42.903 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- 172.21.15.71:0/731495859 shutdown_connections 2023-10-26T05:54:42.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- 172.21.15.71:0/731495859 >> 172.21.15.71:0/731495859 conn(0x7f23840b0c80 msgr2=0x7f23840b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:42.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- 172.21.15.71:0/731495859 shutdown_connections 2023-10-26T05:54:42.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- 172.21.15.71:0/731495859 wait complete. 2023-10-26T05:54:42.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 Processor -- start 2023-10-26T05:54:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- start start 2023-10-26T05:54:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23840c6200 0x7f23840c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 0x7f23840c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23840c7060 0x7f2384162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:42.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23840c89a0 con 0x7f23840c84c0 2023-10-26T05:54:42.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23840c8c80 con 0x7f23840c6200 2023-10-26T05:54:42.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23840c8f60 con 0x7f23840c7060 2023-10-26T05:54:42.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 0x7f23840c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:42.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 0x7f23840c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56012/0 (socket says 172.21.15.71:56012) 2023-10-26T05:54:42.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 -- 172.21.15.71:0/1396292342 learned_addr learned my addr 172.21.15.71:0/1396292342 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:42.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2389c5f640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23840c7060 0x7f2384162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238945e640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23840c6200 0x7f23840c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 -- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23840c7060 msgr2=0x7f2384162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23840c7060 0x7f2384162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 -- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23840c6200 msgr2=0x7f23840c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:42.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23840c6200 0x7f23840c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 -- 172.21.15.71:0/1396292342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840c9240 con 0x7f23840c84c0 2023-10-26T05:54:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238945e640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23840c6200 0x7f23840c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2388c5d640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 0x7f23840c6b20 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2374011ab0 tx=0x7f2374011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:42.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237400fe70 con 0x7f23840c84c0 2023-10-26T05:54:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2374004050 con 0x7f23840c84c0 2023-10-26T05:54:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23841622f0 con 0x7f23840c84c0 2023-10-26T05:54:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.895+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374004440 con 0x7f23840c84c0 2023-10-26T05:54:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2384150930 con 0x7f23840c84c0 2023-10-26T05:54:42.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2384066b10 con 0x7f23840c84c0 2023-10-26T05:54:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2374003600 con 0x7f23840c84c0 2023-10-26T05:54:42.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f2379ffb640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2368066470 0x7f2368068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2374084ba0 con 0x7f23840c84c0 2023-10-26T05:54:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f238945e640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2368066470 0x7f2368068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.899+0000 7f238945e640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2368066470 0x7f2368068930 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f23800aa000 tx=0x7f2380019540 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:42.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:42.903+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23740520d0 con 0x7f23840c84c0 2023-10-26T05:54:43.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.047+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23840721e0 con 0x7f2368066470 2023-10-26T05:54:43.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.071+0000 7f2379ffb640 1 -- 172.21.15.71:0/1396292342 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284085 (secure 0 0 0) 0x7f23840721e0 con 0x7f2368066470 2023-10-26T05:54:43.081 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:54:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2368066470 msgr2=0x7f2368068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2368066470 0x7f2368068930 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f23800aa000 tx=0x7f2380019540 comp rx=0 tx=0).stop 2023-10-26T05:54:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 msgr2=0x7f23840c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 0x7f23840c6b20 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2374011ab0 tx=0x7f2374011f70 comp rx=0 tx=0).stop 2023-10-26T05:54:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 shutdown_connections 2023-10-26T05:54:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23840c7060 0x7f2384162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2368066470 0x7f2368068930 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23840c84c0 0x7f23840c6b20 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 --2- 172.21.15.71:0/1396292342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23840c6200 0x7f23840c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 >> 172.21.15.71:0/1396292342 conn(0x7f23840b0c80 msgr2=0x7f23840b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:43.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 shutdown_connections 2023-10-26T05:54:43.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.075+0000 7f238b6c2640 1 -- 172.21.15.71:0/1396292342 wait complete. 2023-10-26T05:54:43.100 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2023-10-26T05:54:41.558271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7926,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554573,"statfs":{"total":1600000000,"available":1591883276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.341450"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667522+0000","last_change":"2023-10-26T05:54:35.235085+0000","last_active":"2023-10-26T05:54:37.667522+0000","last_peered":"2023-10-26T05:54:37.667522+0000","last_clean":"2023-10-26T05:54:37.667522+0000","last_became_active":"2023-10-26T05:54:35.234844+0000","last_became_peered":"2023-10-26T05:54:35.234844+0000","last_unstale":"2023-10-26T05:54:37.667522+0000","last_undegraded":"2023-10-26T05:54:37.667522+0000","last_fullsized":"2023-10-26T05:54:37.667522+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:45.644368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217879+0000","last_change":"2023-10-26T05:54:35.246520+0000","last_active":"2023-10-26T05:54:36.217879+0000","last_peered":"2023-10-26T05:54:36.217879+0000","last_clean":"2023-10-26T05:54:36.217879+0000","last_became_active":"2023-10-26T05:54:35.246384+0000","last_became_peered":"2023-10-26T05:54:35.246384+0000","last_unstale":"2023-10-26T05:54:36.217879+0000","last_undegraded":"2023-10-26T05:54:36.217879+0000","last_fullsized":"2023-10-26T05:54:36.217879+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:31:51.083174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672228+0000","last_change":"2023-10-26T05:54:35.268032+0000","last_active":"2023-10-26T05:54:37.672228+0000","last_peered":"2023-10-26T05:54:37.672228+0000","last_clean":"2023-10-26T05:54:37.672228+0000","last_became_active":"2023-10-26T05:54:35.263610+0000","last_became_peered":"2023-10-26T05:54:35.263610+0000","last_unstale":"2023-10-26T05:54:37.672228+0000","last_undegraded":"2023-10-26T05:54:37.672228+0000","last_fullsized":"2023-10-26T05:54:37.672228+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:39.249088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666720+0000","last_change":"2023-10-26T05:54:35.244388+0000","last_active":"2023-10-26T05:54:37.666720+0000","last_peered":"2023-10-26T05:54:37.666720+0000","last_clean":"2023-10-26T05:54:37.666720+0000","last_became_active":"2023-10-26T05:54:35.244136+0000","last_became_peered":"2023-10-26T05:54:35.244136+0000","last_unstale":"2023-10-26T05:54:37.666720+0000","last_undegraded":"2023-10-26T05:54:37.666720+0000","last_fullsized":"2023-10-26T05:54:37.666720+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:14:18.669585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673746+0000","last_change":"2023-10-26T05:54:35.264038+0000","last_active":"2023-10-26T05:54:37.673746+0000","last_peered":"2023-10-26T05:54:37.673746+0000","last_clean":"2023-10-26T05:54:37.673746+0000","last_became_active":"2023-10-26T05:54:35.263494+0000","last_became_peered":"2023-10-26T05:54:35.263494+0000","last_unstale":"2023-10-26T05:54:37.673746+0000","last_undegraded":"2023-10-26T05:54:37.673746+0000","last_fullsized":"2023-10-26T05:54:37.673746+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:54:52.578426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225856+0000","last_change":"2023-10-26T05:54:35.241648+0000","last_active":"2023-10-26T05:54:36.225856+0000","last_peered":"2023-10-26T05:54:36.225856+0000","last_clean":"2023-10-26T05:54:36.225856+0000","last_became_active":"2023-10-26T05:54:35.238344+0000","last_became_peered":"2023-10-26T05:54:35.238344+0000","last_unstale":"2023-10-26T05:54:36.225856+0000","last_undegraded":"2023-10-26T05:54:36.225856+0000","last_fullsized":"2023-10-26T05:54:36.225856+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:21:05.364628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225885+0000","last_change":"2023-10-26T05:54:35.252109+0000","last_active":"2023-10-26T05:54:36.225885+0000","last_peered":"2023-10-26T05:54:36.225885+0000","last_clean":"2023-10-26T05:54:36.225885+0000","last_became_active":"2023-10-26T05:54:35.250289+0000","last_became_peered":"2023-10-26T05:54:35.250289+0000","last_unstale":"2023-10-26T05:54:36.225885+0000","last_undegraded":"2023-10-26T05:54:36.225885+0000","last_fullsized":"2023-10-26T05:54:36.225885+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:26.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672600+0000","last_change":"2023-10-26T05:54:35.267644+0000","last_active":"2023-10-26T05:54:37.672600+0000","last_peered":"2023-10-26T05:54:37.672600+0000","last_clean":"2023-10-26T05:54:37.672600+0000","last_became_active":"2023-10-26T05:54:35.261742+0000","last_became_peered":"2023-10-26T05:54:35.261742+0000","last_unstale":"2023-10-26T05:54:37.672600+0000","last_undegraded":"2023-10-26T05:54:37.672600+0000","last_fullsized":"2023-10-26T05:54:37.672600+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:14:00.205146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672291+0000","last_change":"2023-10-26T05:54:35.266334+0000","last_active":"2023-10-26T05:54:37.672291+0000","last_peered":"2023-10-26T05:54:37.672291+0000","last_clean":"2023-10-26T05:54:37.672291+0000","last_became_active":"2023-10-26T05:54:35.265828+0000","last_became_peered":"2023-10-26T05:54:35.265828+0000","last_unstale":"2023-10-26T05:54:37.672291+0000","last_undegraded":"2023-10-26T05:54:37.672291+0000","last_fullsized":"2023-10-26T05:54:37.672291+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:56:47.066685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218036+0000","last_change":"2023-10-26T05:54:35.226995+0000","last_active":"2023-10-26T05:54:36.218036+0000","last_peered":"2023-10-26T05:54:36.218036+0000","last_clean":"2023-10-26T05:54:36.218036+0000","last_became_active":"2023-10-26T05:54:35.226383+0000","last_became_peered":"2023-10-26T05:54:35.226383+0000","last_unstale":"2023-10-26T05:54:36.218036+0000","last_undegraded":"2023-10-26T05:54:36.218036+0000","last_fullsized":"2023-10-26T05:54:36.218036+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:09:55.305666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.366853+0000","last_change":"2023-10-26T05:54:35.222284+0000","last_active":"2023-10-26T05:54:37.366853+0000","last_peered":"2023-10-26T05:54:37.366853+0000","last_clean":"2023-10-26T05:54:37.366853+0000","last_became_active":"2023-10-26T05:54:35.222007+0000","last_became_peered":"2023-10-26T05:54:35.222007+0000","last_unstale":"2023-10-26T05:54:37.366853+0000","last_undegraded":"2023-10-26T05:54:37.366853+0000","last_fullsized":"2023-10-26T05:54:37.366853+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:48:49.970014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673053+0000","last_change":"2023-10-26T05:54:35.266246+0000","last_active":"2023-10-26T05:54:37.673053+0000","last_peered":"2023-10-26T05:54:37.673053+0000","last_clean":"2023-10-26T05:54:37.673053+0000","last_became_active":"2023-10-26T05:54:35.265665+0000","last_became_peered":"2023-10-26T05:54:35.265665+0000","last_unstale":"2023-10-26T05:54:37.673053+0000","last_undegraded":"2023-10-26T05:54:37.673053+0000","last_fullsized":"2023-10-26T05:54:37.673053+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:01:03.500716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666922+0000","last_change":"2023-10-26T05:54:35.233067+0000","last_active":"2023-10-26T05:54:37.666922+0000","last_peered":"2023-10-26T05:54:37.666922+0000","last_clean":"2023-10-26T05:54:37.666922+0000","last_became_active":"2023-10-26T05:54:35.232829+0000","last_became_peered":"2023-10-26T05:54:35.232829+0000","last_unstale":"2023-10-26T05:54:37.666922+0000","last_undegraded":"2023-10-26T05:54:37.666922+0000","last_fullsized":"2023-10-26T05:54:37.666922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:07:41.415849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217615+0000","last_change":"2023-10-26T05:54:35.231235+0000","last_active":"2023-10-26T05:54:36.217615+0000","last_peered":"2023-10-26T05:54:36.217615+0000","last_clean":"2023-10-26T05:54:36.217615+0000","last_became_active":"2023-10-26T05:54:35.231053+0000","last_became_peered":"2023-10-26T05:54:35.231053+0000","last_unstale":"2023-10-26T05:54:36.217615+0000","last_undegraded":"2023-10-26T05:54:36.217615+0000","last_fullsized":"2023-10-26T05:54:36.217615+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:07.494883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667641+0000","last_change":"2023-10-26T05:54:35.244131+0000","last_active":"2023-10-26T05:54:37.667641+0000","last_peered":"2023-10-26T05:54:37.667641+0000","last_clean":"2023-10-26T05:54:37.667641+0000","last_became_active":"2023-10-26T05:54:35.243825+0000","last_became_peered":"2023-10-26T05:54:35.243825+0000","last_unstale":"2023-10-26T05:54:37.667641+0000","last_undegraded":"2023-10-26T05:54:37.667641+0000","last_fullsized":"2023-10-26T05:54:37.667641+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:24:17.574638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666860+0000","last_change":"2023-10-26T05:54:35.230464+0000","last_active":"2023-10-26T05:54:37.666860+0000","last_peered":"2023-10-26T05:54:37.666860+0000","last_clean":"2023-10-26T05:54:37.666860+0000","last_became_active":"2023-10-26T05:54:35.230052+0000","last_became_peered":"2023-10-26T05:54:35.230052+0000","last_unstale":"2023-10-26T05:54:37.666860+0000","last_undegraded":"2023-10-26T05:54:37.666860+0000","last_fullsized":"2023-10-26T05:54:37.666860+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:18:17.385537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226602+0000","last_change":"2023-10-26T05:54:35.229697+0000","last_active":"2023-10-26T05:54:36.226602+0000","last_peered":"2023-10-26T05:54:36.226602+0000","last_clean":"2023-10-26T05:54:36.226602+0000","last_became_active":"2023-10-26T05:54:35.229055+0000","last_became_peered":"2023-10-26T05:54:35.229055+0000","last_unstale":"2023-10-26T05:54:36.226602+0000","last_undegraded":"2023-10-26T05:54:36.226602+0000","last_fullsized":"2023-10-26T05:54:36.226602+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:23:18.236195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667541+0000","last_change":"2023-10-26T05:54:35.232314+0000","last_active":"2023-10-26T05:54:37.667541+0000","last_peered":"2023-10-26T05:54:37.667541+0000","last_clean":"2023-10-26T05:54:37.667541+0000","last_became_active":"2023-10-26T05:54:35.231935+0000","last_became_peered":"2023-10-26T05:54:35.231935+0000","last_unstale":"2023-10-26T05:54:37.667541+0000","last_undegraded":"2023-10-26T05:54:37.667541+0000","last_fullsized":"2023-10-26T05:54:37.667541+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:50:14.667192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.366791+0000","last_change":"2023-10-26T05:54:35.219561+0000","last_active":"2023-10-26T05:54:37.366791+0000","last_peered":"2023-10-26T05:54:37.366791+0000","last_clean":"2023-10-26T05:54:37.366791+0000","last_became_active":"2023-10-26T05:54:35.219330+0000","last_became_peered":"2023-10-26T05:54:35.219330+0000","last_unstale":"2023-10-26T05:54:37.366791+0000","last_undegraded":"2023-10-26T05:54:37.366791+0000","last_fullsized":"2023-10-26T05:54:37.366791+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:47.426860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667384+0000","last_change":"2023-10-26T05:54:35.234316+0000","last_active":"2023-10-26T05:54:37.667384+0000","last_peered":"2023-10-26T05:54:37.667384+0000","last_clean":"2023-10-26T05:54:37.667384+0000","last_became_active":"2023-10-26T05:54:35.234087+0000","last_became_peered":"2023-10-26T05:54:35.234087+0000","last_unstale":"2023-10-26T05:54:37.667384+0000","last_undegraded":"2023-10-26T05:54:37.667384+0000","last_fullsized":"2023-10-26T05:54:37.667384+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:15.559404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672477+0000","last_change":"2023-10-26T05:54:35.243047+0000","last_active":"2023-10-26T05:54:37.672477+0000","last_peered":"2023-10-26T05:54:37.672477+0000","last_clean":"2023-10-26T05:54:37.672477+0000","last_became_active":"2023-10-26T05:54:35.242375+0000","last_became_peered":"2023-10-26T05:54:35.242375+0000","last_unstale":"2023-10-26T05:54:37.672477+0000","last_undegraded":"2023-10-26T05:54:37.672477+0000","last_fullsized":"2023-10-26T05:54:37.672477+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:04.885992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.367071+0000","last_change":"2023-10-26T05:54:35.232535+0000","last_active":"2023-10-26T05:54:37.367071+0000","last_peered":"2023-10-26T05:54:37.367071+0000","last_clean":"2023-10-26T05:54:37.367071+0000","last_became_active":"2023-10-26T05:54:35.232261+0000","last_became_peered":"2023-10-26T05:54:35.232261+0000","last_unstale":"2023-10-26T05:54:37.367071+0000","last_undegraded":"2023-10-26T05:54:37.367071+0000","last_fullsized":"2023-10-26T05:54:37.367071+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:18:31.467937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673616+0000","last_change":"2023-10-26T05:54:35.264544+0000","last_active":"2023-10-26T05:54:37.673616+0000","last_peered":"2023-10-26T05:54:37.673616+0000","last_clean":"2023-10-26T05:54:37.673616+0000","last_became_active":"2023-10-26T05:54:35.258879+0000","last_became_peered":"2023-10-26T05:54:35.258879+0000","last_unstale":"2023-10-26T05:54:37.673616+0000","last_undegraded":"2023-10-26T05:54:37.673616+0000","last_fullsized":"2023-10-26T05:54:37.673616+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:20.465609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.366972+0000","last_change":"2023-10-26T05:54:35.236930+0000","last_active":"2023-10-26T05:54:37.366972+0000","last_peered":"2023-10-26T05:54:37.366972+0000","last_clean":"2023-10-26T05:54:37.366972+0000","last_became_active":"2023-10-26T05:54:35.235990+0000","last_became_peered":"2023-10-26T05:54:35.235990+0000","last_unstale":"2023-10-26T05:54:37.366972+0000","last_undegraded":"2023-10-26T05:54:37.366972+0000","last_fullsized":"2023-10-26T05:54:37.366972+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:41.140815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218371+0000","last_change":"2023-10-26T05:54:35.243569+0000","last_active":"2023-10-26T05:54:36.218371+0000","last_peered":"2023-10-26T05:54:36.218371+0000","last_clean":"2023-10-26T05:54:36.218371+0000","last_became_active":"2023-10-26T05:54:35.242900+0000","last_became_peered":"2023-10-26T05:54:35.242900+0000","last_unstale":"2023-10-26T05:54:36.218371+0000","last_undegraded":"2023-10-26T05:54:36.218371+0000","last_fullsized":"2023-10-26T05:54:36.218371+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:25:16.325052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667022+0000","last_change":"2023-10-26T05:54:35.230295+0000","last_active":"2023-10-26T05:54:37.667022+0000","last_peered":"2023-10-26T05:54:37.667022+0000","last_clean":"2023-10-26T05:54:37.667022+0000","last_became_active":"2023-10-26T05:54:35.229723+0000","last_became_peered":"2023-10-26T05:54:35.229723+0000","last_unstale":"2023-10-26T05:54:37.667022+0000","last_undegraded":"2023-10-26T05:54:37.667022+0000","last_fullsized":"2023-10-26T05:54:37.667022+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:24:10.955516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666667+0000","last_change":"2023-10-26T05:54:35.245335+0000","last_active":"2023-10-26T05:54:37.666667+0000","last_peered":"2023-10-26T05:54:37.666667+0000","last_clean":"2023-10-26T05:54:37.666667+0000","last_became_active":"2023-10-26T05:54:35.245113+0000","last_became_peered":"2023-10-26T05:54:35.245113+0000","last_unstale":"2023-10-26T05:54:37.666667+0000","last_undegraded":"2023-10-26T05:54:37.666667+0000","last_fullsized":"2023-10-26T05:54:37.666667+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:42.045883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.224678+0000","last_change":"2023-10-26T05:54:35.238317+0000","last_active":"2023-10-26T05:54:36.224678+0000","last_peered":"2023-10-26T05:54:36.224678+0000","last_clean":"2023-10-26T05:54:36.224678+0000","last_became_active":"2023-10-26T05:54:35.237222+0000","last_became_peered":"2023-10-26T05:54:35.237222+0000","last_unstale":"2023-10-26T05:54:36.224678+0000","last_undegraded":"2023-10-26T05:54:36.224678+0000","last_fullsized":"2023-10-26T05:54:36.224678+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:50.874423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666998+0000","last_change":"2023-10-26T05:54:35.239954+0000","last_active":"2023-10-26T05:54:37.666998+0000","last_peered":"2023-10-26T05:54:37.666998+0000","last_clean":"2023-10-26T05:54:37.666998+0000","last_became_active":"2023-10-26T05:54:35.239727+0000","last_became_peered":"2023-10-26T05:54:35.239727+0000","last_unstale":"2023-10-26T05:54:37.666998+0000","last_undegraded":"2023-10-26T05:54:37.666998+0000","last_fullsized":"2023-10-26T05:54:37.666998+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:07:09.453871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667827+0000","last_change":"2023-10-26T05:54:35.243983+0000","last_active":"2023-10-26T05:54:37.667827+0000","last_peered":"2023-10-26T05:54:37.667827+0000","last_clean":"2023-10-26T05:54:37.667827+0000","last_became_active":"2023-10-26T05:54:35.243572+0000","last_became_peered":"2023-10-26T05:54:35.243572+0000","last_unstale":"2023-10-26T05:54:37.667827+0000","last_undegraded":"2023-10-26T05:54:37.667827+0000","last_fullsized":"2023-10-26T05:54:37.667827+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:19:19.447021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672788+0000","last_change":"2023-10-26T05:54:35.256626+0000","last_active":"2023-10-26T05:54:37.672788+0000","last_peered":"2023-10-26T05:54:37.672788+0000","last_clean":"2023-10-26T05:54:37.672788+0000","last_became_active":"2023-10-26T05:54:35.255035+0000","last_became_peered":"2023-10-26T05:54:35.255035+0000","last_unstale":"2023-10-26T05:54:37.672788+0000","last_undegraded":"2023-10-26T05:54:37.672788+0000","last_fullsized":"2023-10-26T05:54:37.672788+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:56:49.755593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226289+0000","last_change":"2023-10-26T05:54:35.247423+0000","last_active":"2023-10-26T05:54:36.226289+0000","last_peered":"2023-10-26T05:54:36.226289+0000","last_clean":"2023-10-26T05:54:36.226289+0000","last_became_active":"2023-10-26T05:54:35.245949+0000","last_became_peered":"2023-10-26T05:54:35.245949+0000","last_unstale":"2023-10-26T05:54:36.226289+0000","last_undegraded":"2023-10-26T05:54:36.226289+0000","last_fullsized":"2023-10-26T05:54:36.226289+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:25:54.809563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666742+0000","last_change":"2023-10-26T05:54:35.256672+0000","last_active":"2023-10-26T05:54:37.666742+0000","last_peered":"2023-10-26T05:54:37.666742+0000","last_clean":"2023-10-26T05:54:37.666742+0000","last_became_active":"2023-10-26T05:54:35.256263+0000","last_became_peered":"2023-10-26T05:54:35.256263+0000","last_unstale":"2023-10-26T05:54:37.666742+0000","last_undegraded":"2023-10-26T05:54:37.666742+0000","last_fullsized":"2023-10-26T05:54:37.666742+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:18:40.543626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"154'4","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.258894+0000","last_change":"2023-10-26T05:54:34.238405+0000","last_active":"2023-10-26T05:54:35.258894+0000","last_peered":"2023-10-26T05:54:35.258894+0000","last_clean":"2023-10-26T05:54:35.258894+0000","last_became_active":"2023-10-26T05:54:34.234974+0000","last_became_peered":"2023-10-26T05:54:34.234974+0000","last_unstale":"2023-10-26T05:54:35.258894+0000","last_undegraded":"2023-10-26T05:54:35.258894+0000","last_fullsized":"2023-10-26T05:54:35.258894+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:51:14.554639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672922+0000","last_change":"2023-10-26T05:54:35.267032+0000","last_active":"2023-10-26T05:54:37.672922+0000","last_peered":"2023-10-26T05:54:37.672922+0000","last_clean":"2023-10-26T05:54:37.672922+0000","last_became_active":"2023-10-26T05:54:35.260307+0000","last_became_peered":"2023-10-26T05:54:35.260307+0000","last_unstale":"2023-10-26T05:54:37.672922+0000","last_undegraded":"2023-10-26T05:54:37.672922+0000","last_fullsized":"2023-10-26T05:54:37.672922+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:03.455094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.264825+0000","last_change":"2023-10-26T05:54:34.238394+0000","last_active":"2023-10-26T05:54:35.264825+0000","last_peered":"2023-10-26T05:54:35.264825+0000","last_clean":"2023-10-26T05:54:35.264825+0000","last_became_active":"2023-10-26T05:54:34.237301+0000","last_became_peered":"2023-10-26T05:54:34.237301+0000","last_unstale":"2023-10-26T05:54:35.264825+0000","last_undegraded":"2023-10-26T05:54:35.264825+0000","last_fullsized":"2023-10-26T05:54:35.264825+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:43.524228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673063+0000","last_change":"2023-10-26T05:54:35.256264+0000","last_active":"2023-10-26T05:54:37.673063+0000","last_peered":"2023-10-26T05:54:37.673063+0000","last_clean":"2023-10-26T05:54:37.673063+0000","last_became_active":"2023-10-26T05:54:35.253718+0000","last_became_peered":"2023-10-26T05:54:35.253718+0000","last_unstale":"2023-10-26T05:54:37.673063+0000","last_undegraded":"2023-10-26T05:54:37.673063+0000","last_fullsized":"2023-10-26T05:54:37.673063+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:35:26.568398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.241279+0000","last_change":"2023-10-26T05:54:34.228217+0000","last_active":"2023-10-26T05:54:35.241279+0000","last_peered":"2023-10-26T05:54:35.241279+0000","last_clean":"2023-10-26T05:54:35.241279+0000","last_became_active":"2023-10-26T05:54:34.227446+0000","last_became_peered":"2023-10-26T05:54:34.227446+0000","last_unstale":"2023-10-26T05:54:35.241279+0000","last_undegraded":"2023-10-26T05:54:35.241279+0000","last_fullsized":"2023-10-26T05:54:35.241279+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:31:57.251535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673897+0000","last_change":"2023-10-26T05:54:35.252615+0000","last_active":"2023-10-26T05:54:37.673897+0000","last_peered":"2023-10-26T05:54:37.673897+0000","last_clean":"2023-10-26T05:54:37.673897+0000","last_became_active":"2023-10-26T05:54:35.251779+0000","last_became_peered":"2023-10-26T05:54:35.251779+0000","last_unstale":"2023-10-26T05:54:37.673897+0000","last_undegraded":"2023-10-26T05:54:37.673897+0000","last_fullsized":"2023-10-26T05:54:37.673897+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:29:24.069539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.240969+0000","last_change":"2023-10-26T05:54:34.219413+0000","last_active":"2023-10-26T05:54:35.240969+0000","last_peered":"2023-10-26T05:54:35.240969+0000","last_clean":"2023-10-26T05:54:35.240969+0000","last_became_active":"2023-10-26T05:54:34.219109+0000","last_became_peered":"2023-10-26T05:54:34.219109+0000","last_unstale":"2023-10-26T05:54:35.240969+0000","last_undegraded":"2023-10-26T05:54:35.240969+0000","last_fullsized":"2023-10-26T05:54:35.240969+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:17:51.315680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667714+0000","last_change":"2023-10-26T05:54:35.232151+0000","last_active":"2023-10-26T05:54:37.667714+0000","last_peered":"2023-10-26T05:54:37.667714+0000","last_clean":"2023-10-26T05:54:37.667714+0000","last_became_active":"2023-10-26T05:54:35.231597+0000","last_became_peered":"2023-10-26T05:54:35.231597+0000","last_unstale":"2023-10-26T05:54:37.667714+0000","last_undegraded":"2023-10-26T05:54:37.667714+0000","last_fullsized":"2023-10-26T05:54:37.667714+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:12.959619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.240597+0000","last_change":"2023-10-26T05:54:34.225466+0000","last_active":"2023-10-26T05:54:35.240597+0000","last_peered":"2023-10-26T05:54:35.240597+0000","last_clean":"2023-10-26T05:54:35.240597+0000","last_became_active":"2023-10-26T05:54:34.225226+0000","last_became_peered":"2023-10-26T05:54:34.225226+0000","last_unstale":"2023-10-26T05:54:35.240597+0000","last_undegraded":"2023-10-26T05:54:35.240597+0000","last_fullsized":"2023-10-26T05:54:35.240597+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:36:43.524228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226931+0000","last_change":"2023-10-26T05:54:35.246746+0000","last_active":"2023-10-26T05:54:36.226931+0000","last_peered":"2023-10-26T05:54:36.226931+0000","last_clean":"2023-10-26T05:54:36.226931+0000","last_became_active":"2023-10-26T05:54:35.241062+0000","last_became_peered":"2023-10-26T05:54:35.241062+0000","last_unstale":"2023-10-26T05:54:36.226931+0000","last_undegraded":"2023-10-26T05:54:36.226931+0000","last_fullsized":"2023-10-26T05:54:36.226931+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:54:52.578426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.237011+0000","last_change":"2023-10-26T05:54:34.231290+0000","last_active":"2023-10-26T05:54:35.237011+0000","last_peered":"2023-10-26T05:54:35.237011+0000","last_clean":"2023-10-26T05:54:35.237011+0000","last_became_active":"2023-10-26T05:54:34.230708+0000","last_became_peered":"2023-10-26T05:54:34.230708+0000","last_unstale":"2023-10-26T05:54:35.237011+0000","last_undegraded":"2023-10-26T05:54:35.237011+0000","last_fullsized":"2023-10-26T05:54:35.237011+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:49.174594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673554+0000","last_change":"2023-10-26T05:54:35.250576+0000","last_active":"2023-10-26T05:54:37.673554+0000","last_peered":"2023-10-26T05:54:37.673554+0000","last_clean":"2023-10-26T05:54:37.673554+0000","last_became_active":"2023-10-26T05:54:35.247435+0000","last_became_peered":"2023-10-26T05:54:35.247435+0000","last_unstale":"2023-10-26T05:54:37.673554+0000","last_undegraded":"2023-10-26T05:54:37.673554+0000","last_fullsized":"2023-10-26T05:54:37.673554+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:07:15.752245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.223618+0000","last_change":"2023-10-26T05:54:34.229066+0000","last_active":"2023-10-26T05:54:35.223618+0000","last_peered":"2023-10-26T05:54:35.223618+0000","last_clean":"2023-10-26T05:54:35.223618+0000","last_became_active":"2023-10-26T05:54:34.228464+0000","last_became_peered":"2023-10-26T05:54:34.228464+0000","last_unstale":"2023-10-26T05:54:35.223618+0000","last_undegraded":"2023-10-26T05:54:35.223618+0000","last_fullsized":"2023-10-26T05:54:35.223618+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:02:38.842997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217997+0000","last_change":"2023-10-26T05:54:35.229966+0000","last_active":"2023-10-26T05:54:36.217997+0000","last_peered":"2023-10-26T05:54:36.217997+0000","last_clean":"2023-10-26T05:54:36.217997+0000","last_became_active":"2023-10-26T05:54:35.229248+0000","last_became_peered":"2023-10-26T05:54:35.229248+0000","last_unstale":"2023-10-26T05:54:36.217997+0000","last_undegraded":"2023-10-26T05:54:36.217997+0000","last_fullsized":"2023-10-26T05:54:36.217997+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:28.679833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.221724+0000","last_change":"2023-10-26T05:54:34.237160+0000","last_active":"2023-10-26T05:54:35.221724+0000","last_peered":"2023-10-26T05:54:35.221724+0000","last_clean":"2023-10-26T05:54:35.221724+0000","last_became_active":"2023-10-26T05:54:34.236869+0000","last_became_peered":"2023-10-26T05:54:34.236869+0000","last_unstale":"2023-10-26T05:54:35.221724+0000","last_undegraded":"2023-10-26T05:54:35.221724+0000","last_fullsized":"2023-10-26T05:54:35.221724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:57.286656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666955+0000","last_change":"2023-10-26T05:54:35.231619+0000","last_active":"2023-10-26T05:54:37.666955+0000","last_peered":"2023-10-26T05:54:37.666955+0000","last_clean":"2023-10-26T05:54:37.666955+0000","last_became_active":"2023-10-26T05:54:35.230748+0000","last_became_peered":"2023-10-26T05:54:35.230748+0000","last_unstale":"2023-10-26T05:54:37.666955+0000","last_undegraded":"2023-10-26T05:54:37.666955+0000","last_fullsized":"2023-10-26T05:54:37.666955+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:04:06.063251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.227848+0000","last_change":"2023-10-26T05:54:34.227954+0000","last_active":"2023-10-26T05:54:35.227848+0000","last_peered":"2023-10-26T05:54:35.227848+0000","last_clean":"2023-10-26T05:54:35.227848+0000","last_became_active":"2023-10-26T05:54:34.227315+0000","last_became_peered":"2023-10-26T05:54:34.227315+0000","last_unstale":"2023-10-26T05:54:35.227848+0000","last_undegraded":"2023-10-26T05:54:35.227848+0000","last_fullsized":"2023-10-26T05:54:35.227848+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:57.286656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226148+0000","last_change":"2023-10-26T05:54:35.242187+0000","last_active":"2023-10-26T05:54:36.226148+0000","last_peered":"2023-10-26T05:54:36.226148+0000","last_clean":"2023-10-26T05:54:36.226148+0000","last_became_active":"2023-10-26T05:54:35.240648+0000","last_became_peered":"2023-10-26T05:54:35.240648+0000","last_unstale":"2023-10-26T05:54:36.226148+0000","last_undegraded":"2023-10-26T05:54:36.226148+0000","last_fullsized":"2023-10-26T05:54:36.226148+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:58.785279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.226332+0000","last_change":"2023-10-26T05:54:34.228604+0000","last_active":"2023-10-26T05:54:35.226332+0000","last_peered":"2023-10-26T05:54:35.226332+0000","last_clean":"2023-10-26T05:54:35.226332+0000","last_became_active":"2023-10-26T05:54:34.228363+0000","last_became_peered":"2023-10-26T05:54:34.228363+0000","last_unstale":"2023-10-26T05:54:35.226332+0000","last_undegraded":"2023-10-26T05:54:35.226332+0000","last_fullsized":"2023-10-26T05:54:35.226332+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:55:19.460844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672955+0000","last_change":"2023-10-26T05:54:35.239171+0000","last_active":"2023-10-26T05:54:37.672955+0000","last_peered":"2023-10-26T05:54:37.672955+0000","last_clean":"2023-10-26T05:54:37.672955+0000","last_became_active":"2023-10-26T05:54:35.238560+0000","last_became_peered":"2023-10-26T05:54:35.238560+0000","last_unstale":"2023-10-26T05:54:37.672955+0000","last_undegraded":"2023-10-26T05:54:37.672955+0000","last_fullsized":"2023-10-26T05:54:37.672955+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:21:05.364628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"154'3","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.262449+0000","last_change":"2023-10-26T05:54:34.241288+0000","last_active":"2023-10-26T05:54:35.262449+0000","last_peered":"2023-10-26T05:54:35.262449+0000","last_clean":"2023-10-26T05:54:35.262449+0000","last_became_active":"2023-10-26T05:54:34.240585+0000","last_became_peered":"2023-10-26T05:54:34.240585+0000","last_unstale":"2023-10-26T05:54:35.262449+0000","last_undegraded":"2023-10-26T05:54:35.262449+0000","last_fullsized":"2023-10-26T05:54:35.262449+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:15:06.490118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672355+0000","last_change":"2023-10-26T05:54:35.268054+0000","last_active":"2023-10-26T05:54:37.672355+0000","last_peered":"2023-10-26T05:54:37.672355+0000","last_clean":"2023-10-26T05:54:37.672355+0000","last_became_active":"2023-10-26T05:54:35.267577+0000","last_became_peered":"2023-10-26T05:54:35.267577+0000","last_unstale":"2023-10-26T05:54:37.672355+0000","last_undegraded":"2023-10-26T05:54:37.672355+0000","last_fullsized":"2023-10-26T05:54:37.672355+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:50.801746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"154'6","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.268644+0000","last_change":"2023-10-26T05:54:34.245988+0000","last_active":"2023-10-26T05:54:35.268644+0000","last_peered":"2023-10-26T05:54:35.268644+0000","last_clean":"2023-10-26T05:54:35.268644+0000","last_became_active":"2023-10-26T05:54:34.245363+0000","last_became_peered":"2023-10-26T05:54:34.245363+0000","last_unstale":"2023-10-26T05:54:35.268644+0000","last_undegraded":"2023-10-26T05:54:35.268644+0000","last_fullsized":"2023-10-26T05:54:35.268644+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:18.171476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.668063+0000","last_change":"2023-10-26T05:54:35.229031+0000","last_active":"2023-10-26T05:54:37.668063+0000","last_peered":"2023-10-26T05:54:37.668063+0000","last_clean":"2023-10-26T05:54:37.668063+0000","last_became_active":"2023-10-26T05:54:35.228680+0000","last_became_peered":"2023-10-26T05:54:35.228680+0000","last_unstale":"2023-10-26T05:54:37.668063+0000","last_undegraded":"2023-10-26T05:54:37.668063+0000","last_fullsized":"2023-10-26T05:54:37.668063+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:38.417610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.227985+0000","last_change":"2023-10-26T05:54:34.226642+0000","last_active":"2023-10-26T05:54:35.227985+0000","last_peered":"2023-10-26T05:54:35.227985+0000","last_clean":"2023-10-26T05:54:35.227985+0000","last_became_active":"2023-10-26T05:54:34.226063+0000","last_became_peered":"2023-10-26T05:54:34.226063+0000","last_unstale":"2023-10-26T05:54:35.227985+0000","last_undegraded":"2023-10-26T05:54:35.227985+0000","last_fullsized":"2023-10-26T05:54:35.227985+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:33:35.772656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217893+0000","last_change":"2023-10-26T05:54:35.254873+0000","last_active":"2023-10-26T05:54:36.217893+0000","last_peered":"2023-10-26T05:54:36.217893+0000","last_clean":"2023-10-26T05:54:36.217893+0000","last_became_active":"2023-10-26T05:54:35.254624+0000","last_became_peered":"2023-10-26T05:54:35.254624+0000","last_unstale":"2023-10-26T05:54:36.217893+0000","last_undegraded":"2023-10-26T05:54:36.217893+0000","last_fullsized":"2023-10-26T05:54:36.217893+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:50.874423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.231640+0000","last_change":"2023-10-26T05:54:34.230187+0000","last_active":"2023-10-26T05:54:35.231640+0000","last_peered":"2023-10-26T05:54:35.231640+0000","last_clean":"2023-10-26T05:54:35.231640+0000","last_became_active":"2023-10-26T05:54:34.229848+0000","last_became_peered":"2023-10-26T05:54:34.229848+0000","last_unstale":"2023-10-26T05:54:35.231640+0000","last_undegraded":"2023-10-26T05:54:35.231640+0000","last_fullsized":"2023-10-26T05:54:35.231640+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:29.154810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673304+0000","last_change":"2023-10-26T05:54:35.263246+0000","last_active":"2023-10-26T05:54:37.673304+0000","last_peered":"2023-10-26T05:54:37.673304+0000","last_clean":"2023-10-26T05:54:37.673304+0000","last_became_active":"2023-10-26T05:54:35.262386+0000","last_became_peered":"2023-10-26T05:54:35.262386+0000","last_unstale":"2023-10-26T05:54:37.673304+0000","last_undegraded":"2023-10-26T05:54:37.673304+0000","last_fullsized":"2023-10-26T05:54:37.673304+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:31:47.069483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"154'8","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.276777+0000","last_change":"2023-10-26T05:54:34.242272+0000","last_active":"2023-10-26T05:54:35.276777+0000","last_peered":"2023-10-26T05:54:35.276777+0000","last_clean":"2023-10-26T05:54:35.276777+0000","last_became_active":"2023-10-26T05:54:34.240219+0000","last_became_peered":"2023-10-26T05:54:34.240219+0000","last_unstale":"2023-10-26T05:54:35.276777+0000","last_undegraded":"2023-10-26T05:54:35.276777+0000","last_fullsized":"2023-10-26T05:54:35.276777+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:13:54.872759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673743+0000","last_change":"2023-10-26T05:54:35.250174+0000","last_active":"2023-10-26T05:54:37.673743+0000","last_peered":"2023-10-26T05:54:37.673743+0000","last_clean":"2023-10-26T05:54:37.673743+0000","last_became_active":"2023-10-26T05:54:35.246225+0000","last_became_peered":"2023-10-26T05:54:35.246225+0000","last_unstale":"2023-10-26T05:54:37.673743+0000","last_undegraded":"2023-10-26T05:54:37.673743+0000","last_fullsized":"2023-10-26T05:54:37.673743+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:48:50.179358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.263990+0000","last_change":"2023-10-26T05:54:34.237939+0000","last_active":"2023-10-26T05:54:35.263990+0000","last_peered":"2023-10-26T05:54:35.263990+0000","last_clean":"2023-10-26T05:54:35.263990+0000","last_became_active":"2023-10-26T05:54:34.236275+0000","last_became_peered":"2023-10-26T05:54:34.236275+0000","last_unstale":"2023-10-26T05:54:35.263990+0000","last_undegraded":"2023-10-26T05:54:35.263990+0000","last_fullsized":"2023-10-26T05:54:35.263990+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:37:38.629159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.226056+0000","last_change":"2023-10-26T05:54:35.229868+0000","last_active":"2023-10-26T05:54:36.226056+0000","last_peered":"2023-10-26T05:54:36.226056+0000","last_clean":"2023-10-26T05:54:36.226056+0000","last_became_active":"2023-10-26T05:54:35.229266+0000","last_became_peered":"2023-10-26T05:54:35.229266+0000","last_unstale":"2023-10-26T05:54:36.226056+0000","last_undegraded":"2023-10-26T05:54:36.226056+0000","last_fullsized":"2023-10-26T05:54:36.226056+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:09:55.305666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.245782+0000","last_change":"2023-10-26T05:54:34.238890+0000","last_active":"2023-10-26T05:54:35.245782+0000","last_peered":"2023-10-26T05:54:35.245782+0000","last_clean":"2023-10-26T05:54:35.245782+0000","last_became_active":"2023-10-26T05:54:34.238622+0000","last_became_peered":"2023-10-26T05:54:34.238622+0000","last_unstale":"2023-10-26T05:54:35.245782+0000","last_undegraded":"2023-10-26T05:54:35.245782+0000","last_fullsized":"2023-10-26T05:54:35.245782+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:30:43.095652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667367+0000","last_change":"2023-10-26T05:54:35.237989+0000","last_active":"2023-10-26T05:54:37.667367+0000","last_peered":"2023-10-26T05:54:37.667367+0000","last_clean":"2023-10-26T05:54:37.667367+0000","last_became_active":"2023-10-26T05:54:35.237809+0000","last_became_peered":"2023-10-26T05:54:35.237809+0000","last_unstale":"2023-10-26T05:54:37.667367+0000","last_undegraded":"2023-10-26T05:54:37.667367+0000","last_fullsized":"2023-10-26T05:54:37.667367+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:40.556192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"154'1","reported_seq":13,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.254521+0000","last_change":"2023-10-26T05:54:34.239412+0000","last_active":"2023-10-26T05:54:35.254521+0000","last_peered":"2023-10-26T05:54:35.254521+0000","last_clean":"2023-10-26T05:54:35.254521+0000","last_became_active":"2023-10-26T05:54:34.238306+0000","last_became_peered":"2023-10-26T05:54:34.238306+0000","last_unstale":"2023-10-26T05:54:35.254521+0000","last_undegraded":"2023-10-26T05:54:35.254521+0000","last_fullsized":"2023-10-26T05:54:35.254521+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:37:37.435449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218143+0000","last_change":"2023-10-26T05:54:35.233029+0000","last_active":"2023-10-26T05:54:36.218143+0000","last_peered":"2023-10-26T05:54:36.218143+0000","last_clean":"2023-10-26T05:54:36.218143+0000","last_became_active":"2023-10-26T05:54:35.232401+0000","last_became_peered":"2023-10-26T05:54:35.232401+0000","last_unstale":"2023-10-26T05:54:36.218143+0000","last_undegraded":"2023-10-26T05:54:36.218143+0000","last_fullsized":"2023-10-26T05:54:36.218143+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:24:58.785279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.228260+0000","last_change":"2023-10-26T05:54:34.237449+0000","last_active":"2023-10-26T05:54:35.228260+0000","last_peered":"2023-10-26T05:54:35.228260+0000","last_clean":"2023-10-26T05:54:35.228260+0000","last_became_active":"2023-10-26T05:54:34.236646+0000","last_became_peered":"2023-10-26T05:54:34.236646+0000","last_unstale":"2023-10-26T05:54:35.228260+0000","last_undegraded":"2023-10-26T05:54:35.228260+0000","last_fullsized":"2023-10-26T05:54:35.228260+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:16:29.154810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667163+0000","last_change":"2023-10-26T05:54:35.236485+0000","last_active":"2023-10-26T05:54:37.667163+0000","last_peered":"2023-10-26T05:54:37.667163+0000","last_clean":"2023-10-26T05:54:37.667163+0000","last_became_active":"2023-10-26T05:54:35.236273+0000","last_became_peered":"2023-10-26T05:54:35.236273+0000","last_unstale":"2023-10-26T05:54:37.667163+0000","last_undegraded":"2023-10-26T05:54:37.667163+0000","last_fullsized":"2023-10-26T05:54:37.667163+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:47:47.463495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.263089+0000","last_change":"2023-10-26T05:54:34.229691+0000","last_active":"2023-10-26T05:54:35.263089+0000","last_peered":"2023-10-26T05:54:35.263089+0000","last_clean":"2023-10-26T05:54:35.263089+0000","last_became_active":"2023-10-26T05:54:34.229093+0000","last_became_peered":"2023-10-26T05:54:34.229093+0000","last_unstale":"2023-10-26T05:54:35.263089+0000","last_undegraded":"2023-10-26T05:54:35.263089+0000","last_fullsized":"2023-10-26T05:54:35.263089+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:42.423166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.224894+0000","last_change":"2023-10-26T05:54:35.258639+0000","last_active":"2023-10-26T05:54:36.224894+0000","last_peered":"2023-10-26T05:54:36.224894+0000","last_clean":"2023-10-26T05:54:36.224894+0000","last_became_active":"2023-10-26T05:54:35.258010+0000","last_became_peered":"2023-10-26T05:54:35.258010+0000","last_unstale":"2023-10-26T05:54:36.224894+0000","last_undegraded":"2023-10-26T05:54:36.224894+0000","last_fullsized":"2023-10-26T05:54:36.224894+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:16:04.885992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.225537+0000","last_change":"2023-10-26T05:54:34.231906+0000","last_active":"2023-10-26T05:54:35.225537+0000","last_peered":"2023-10-26T05:54:35.225537+0000","last_clean":"2023-10-26T05:54:35.225537+0000","last_became_active":"2023-10-26T05:54:34.231677+0000","last_became_peered":"2023-10-26T05:54:34.231677+0000","last_unstale":"2023-10-26T05:54:35.225537+0000","last_undegraded":"2023-10-26T05:54:35.225537+0000","last_fullsized":"2023-10-26T05:54:35.225537+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:58:02.450329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.217795+0000","last_change":"2023-10-26T05:54:35.222747+0000","last_active":"2023-10-26T05:54:36.217795+0000","last_peered":"2023-10-26T05:54:36.217795+0000","last_clean":"2023-10-26T05:54:36.217795+0000","last_became_active":"2023-10-26T05:54:35.222457+0000","last_became_peered":"2023-10-26T05:54:35.222457+0000","last_unstale":"2023-10-26T05:54:36.217795+0000","last_undegraded":"2023-10-26T05:54:36.217795+0000","last_fullsized":"2023-10-26T05:54:36.217795+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:09:43.427931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.224734+0000","last_change":"2023-10-26T05:54:34.236055+0000","last_active":"2023-10-26T05:54:35.224734+0000","last_peered":"2023-10-26T05:54:35.224734+0000","last_clean":"2023-10-26T05:54:35.224734+0000","last_became_active":"2023-10-26T05:54:34.235715+0000","last_became_peered":"2023-10-26T05:54:34.235715+0000","last_unstale":"2023-10-26T05:54:35.224734+0000","last_undegraded":"2023-10-26T05:54:35.224734+0000","last_fullsized":"2023-10-26T05:54:35.224734+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:13:54.872759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673591+0000","last_change":"2023-10-26T05:54:35.248973+0000","last_active":"2023-10-26T05:54:37.673591+0000","last_peered":"2023-10-26T05:54:37.673591+0000","last_clean":"2023-10-26T05:54:37.673591+0000","last_became_active":"2023-10-26T05:54:35.248189+0000","last_became_peered":"2023-10-26T05:54:35.248189+0000","last_unstale":"2023-10-26T05:54:37.673591+0000","last_undegraded":"2023-10-26T05:54:37.673591+0000","last_fullsized":"2023-10-26T05:54:37.673591+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:23:18.236195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.215772+0000","last_change":"2023-10-26T05:54:34.235155+0000","last_active":"2023-10-26T05:54:35.215772+0000","last_peered":"2023-10-26T05:54:35.215772+0000","last_clean":"2023-10-26T05:54:35.215772+0000","last_became_active":"2023-10-26T05:54:34.234905+0000","last_became_peered":"2023-10-26T05:54:34.234905+0000","last_unstale":"2023-10-26T05:54:35.215772+0000","last_undegraded":"2023-10-26T05:54:35.215772+0000","last_fullsized":"2023-10-26T05:54:35.215772+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:43:03.961130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225235+0000","last_change":"2023-10-26T05:54:35.233645+0000","last_active":"2023-10-26T05:54:36.225235+0000","last_peered":"2023-10-26T05:54:36.225235+0000","last_clean":"2023-10-26T05:54:36.225235+0000","last_became_active":"2023-10-26T05:54:35.230637+0000","last_became_peered":"2023-10-26T05:54:35.230637+0000","last_unstale":"2023-10-26T05:54:36.225235+0000","last_undegraded":"2023-10-26T05:54:36.225235+0000","last_fullsized":"2023-10-26T05:54:36.225235+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:56:49.755593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.257283+0000","last_change":"2023-10-26T05:54:34.226691+0000","last_active":"2023-10-26T05:54:35.257283+0000","last_peered":"2023-10-26T05:54:35.257283+0000","last_clean":"2023-10-26T05:54:35.257283+0000","last_became_active":"2023-10-26T05:54:34.225872+0000","last_became_peered":"2023-10-26T05:54:34.225872+0000","last_unstale":"2023-10-26T05:54:35.257283+0000","last_undegraded":"2023-10-26T05:54:35.257283+0000","last_fullsized":"2023-10-26T05:54:35.257283+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:34:07.434465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.667123+0000","last_change":"2023-10-26T05:54:35.243113+0000","last_active":"2023-10-26T05:54:37.667123+0000","last_peered":"2023-10-26T05:54:37.667123+0000","last_clean":"2023-10-26T05:54:37.667123+0000","last_became_active":"2023-10-26T05:54:35.242839+0000","last_became_peered":"2023-10-26T05:54:35.242839+0000","last_unstale":"2023-10-26T05:54:37.667123+0000","last_undegraded":"2023-10-26T05:54:37.667123+0000","last_fullsized":"2023-10-26T05:54:37.667123+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:32:57.161029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.234133+0000","last_change":"2023-10-26T05:54:34.235620+0000","last_active":"2023-10-26T05:54:35.234133+0000","last_peered":"2023-10-26T05:54:35.234133+0000","last_clean":"2023-10-26T05:54:35.234133+0000","last_became_active":"2023-10-26T05:54:34.231181+0000","last_became_peered":"2023-10-26T05:54:34.231181+0000","last_unstale":"2023-10-26T05:54:35.234133+0000","last_undegraded":"2023-10-26T05:54:35.234133+0000","last_fullsized":"2023-10-26T05:54:35.234133+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:16:11.954854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218643+0000","last_change":"2023-10-26T05:54:35.243730+0000","last_active":"2023-10-26T05:54:36.218643+0000","last_peered":"2023-10-26T05:54:36.218643+0000","last_clean":"2023-10-26T05:54:36.218643+0000","last_became_active":"2023-10-26T05:54:35.243240+0000","last_became_peered":"2023-10-26T05:54:35.243240+0000","last_unstale":"2023-10-26T05:54:36.218643+0000","last_undegraded":"2023-10-26T05:54:36.218643+0000","last_fullsized":"2023-10-26T05:54:36.218643+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:21:24.235337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.221160+0000","last_change":"2023-10-26T05:54:34.234275+0000","last_active":"2023-10-26T05:54:35.221160+0000","last_peered":"2023-10-26T05:54:35.221160+0000","last_clean":"2023-10-26T05:54:35.221160+0000","last_became_active":"2023-10-26T05:54:34.234042+0000","last_became_peered":"2023-10-26T05:54:34.234042+0000","last_unstale":"2023-10-26T05:54:35.221160+0000","last_undegraded":"2023-10-26T05:54:35.221160+0000","last_fullsized":"2023-10-26T05:54:35.221160+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:21.281518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.224676+0000","last_change":"2023-10-26T05:54:35.241333+0000","last_active":"2023-10-26T05:54:36.224676+0000","last_peered":"2023-10-26T05:54:36.224676+0000","last_clean":"2023-10-26T05:54:36.224676+0000","last_became_active":"2023-10-26T05:54:35.240402+0000","last_became_peered":"2023-10-26T05:54:35.240402+0000","last_unstale":"2023-10-26T05:54:36.224676+0000","last_undegraded":"2023-10-26T05:54:36.224676+0000","last_fullsized":"2023-10-26T05:54:36.224676+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:59:50.801746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.215608+0000","last_change":"2023-10-26T05:54:34.228407+0000","last_active":"2023-10-26T05:54:35.215608+0000","last_peered":"2023-10-26T05:54:35.215608+0000","last_clean":"2023-10-26T05:54:35.215608+0000","last_became_active":"2023-10-26T05:54:34.227889+0000","last_became_peered":"2023-10-26T05:54:34.227889+0000","last_unstale":"2023-10-26T05:54:35.215608+0000","last_undegraded":"2023-10-26T05:54:35.215608+0000","last_fullsized":"2023-10-26T05:54:35.215608+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:46:25.901115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.672240+0000","last_change":"2023-10-26T05:54:35.249718+0000","last_active":"2023-10-26T05:54:37.672240+0000","last_peered":"2023-10-26T05:54:37.672240+0000","last_clean":"2023-10-26T05:54:37.672240+0000","last_became_active":"2023-10-26T05:54:35.248964+0000","last_became_peered":"2023-10-26T05:54:35.248964+0000","last_unstale":"2023-10-26T05:54:37.672240+0000","last_undegraded":"2023-10-26T05:54:37.672240+0000","last_fullsized":"2023-10-26T05:54:37.672240+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:06:55.060596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.221386+0000","last_change":"2023-10-26T05:54:34.227337+0000","last_active":"2023-10-26T05:54:35.221386+0000","last_peered":"2023-10-26T05:54:35.221386+0000","last_clean":"2023-10-26T05:54:35.221386+0000","last_became_active":"2023-10-26T05:54:34.227083+0000","last_became_peered":"2023-10-26T05:54:34.227083+0000","last_unstale":"2023-10-26T05:54:35.221386+0000","last_undegraded":"2023-10-26T05:54:35.221386+0000","last_fullsized":"2023-10-26T05:54:35.221386+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:05:18.895959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.225613+0000","last_change":"2023-10-26T05:54:35.242707+0000","last_active":"2023-10-26T05:54:36.225613+0000","last_peered":"2023-10-26T05:54:36.225613+0000","last_clean":"2023-10-26T05:54:36.225613+0000","last_became_active":"2023-10-26T05:54:35.241479+0000","last_became_peered":"2023-10-26T05:54:35.241479+0000","last_unstale":"2023-10-26T05:54:36.225613+0000","last_undegraded":"2023-10-26T05:54:36.225613+0000","last_fullsized":"2023-10-26T05:54:36.225613+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:34:28.679833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.234976+0000","last_change":"2023-10-26T05:54:34.236735+0000","last_active":"2023-10-26T05:54:35.234976+0000","last_peered":"2023-10-26T05:54:35.234976+0000","last_clean":"2023-10-26T05:54:35.234976+0000","last_became_active":"2023-10-26T05:54:34.233447+0000","last_became_peered":"2023-10-26T05:54:34.233447+0000","last_unstale":"2023-10-26T05:54:35.234976+0000","last_undegraded":"2023-10-26T05:54:35.234976+0000","last_fullsized":"2023-10-26T05:54:35.234976+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:06:34.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.666909+0000","last_change":"2023-10-26T05:54:35.239344+0000","last_active":"2023-10-26T05:54:37.666909+0000","last_peered":"2023-10-26T05:54:37.666909+0000","last_clean":"2023-10-26T05:54:37.666909+0000","last_became_active":"2023-10-26T05:54:35.239200+0000","last_became_peered":"2023-10-26T05:54:35.239200+0000","last_unstale":"2023-10-26T05:54:37.666909+0000","last_undegraded":"2023-10-26T05:54:37.666909+0000","last_fullsized":"2023-10-26T05:54:37.666909+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:17:41.937810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.227546+0000","last_change":"2023-10-26T05:54:34.234238+0000","last_active":"2023-10-26T05:54:35.227546+0000","last_peered":"2023-10-26T05:54:35.227546+0000","last_clean":"2023-10-26T05:54:35.227546+0000","last_became_active":"2023-10-26T05:54:34.234006+0000","last_became_peered":"2023-10-26T05:54:34.234006+0000","last_unstale":"2023-10-26T05:54:35.227546+0000","last_undegraded":"2023-10-26T05:54:35.227546+0000","last_fullsized":"2023-10-26T05:54:35.227546+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:12.750236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:36.218250+0000","last_change":"2023-10-26T05:54:35.237714+0000","last_active":"2023-10-26T05:54:36.218250+0000","last_peered":"2023-10-26T05:54:36.218250+0000","last_clean":"2023-10-26T05:54:36.218250+0000","last_became_active":"2023-10-26T05:54:35.237486+0000","last_became_peered":"2023-10-26T05:54:35.237486+0000","last_unstale":"2023-10-26T05:54:36.218250+0000","last_undegraded":"2023-10-26T05:54:36.218250+0000","last_fullsized":"2023-10-26T05:54:36.218250+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:43:05.899408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.216019+0000","last_change":"2023-10-26T05:54:34.226096+0000","last_active":"2023-10-26T05:54:35.216019+0000","last_peered":"2023-10-26T05:54:35.216019+0000","last_clean":"2023-10-26T05:54:35.216019+0000","last_became_active":"2023-10-26T05:54:34.224494+0000","last_became_peered":"2023-10-26T05:54:34.224494+0000","last_unstale":"2023-10-26T05:54:35.216019+0000","last_undegraded":"2023-10-26T05:54:35.216019+0000","last_fullsized":"2023-10-26T05:54:35.216019+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:56.591989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":155,"state":"active+clean","last_fresh":"2023-10-26T05:54:37.673231+0000","last_change":"2023-10-26T05:54:35.256899+0000","last_active":"2023-10-26T05:54:37.673231+0000","last_peered":"2023-10-26T05:54:37.673231+0000","last_clean":"2023-10-26T05:54:37.673231+0000","last_became_active":"2023-10-26T05:54:35.255793+0000","last_became_peered":"2023-10-26T05:54:35.255793+0000","last_unstale":"2023-10-26T05:54:37.673231+0000","last_undegraded":"2023-10-26T05:54:37.673231+0000","last_fullsized":"2023-10-26T05:54:37.673231+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:34.195118+0000","last_clean_scrub_stamp":"2023-10-26T05:54:34.195118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:25:54.809563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"154'2","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.258813+0000","last_change":"2023-10-26T05:54:34.241834+0000","last_active":"2023-10-26T05:54:35.258813+0000","last_peered":"2023-10-26T05:54:35.258813+0000","last_clean":"2023-10-26T05:54:35.258813+0000","last_became_active":"2023-10-26T05:54:34.232265+0000","last_became_peered":"2023-10-26T05:54:34.232265+0000","last_unstale":"2023-10-26T05:54:35.258813+0000","last_undegraded":"2023-10-26T05:54:35.258813+0000","last_fullsized":"2023-10-26T05:54:35.258813+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:54:33.190353+0000","last_clean_scrub_stamp":"2023-10-26T05:54:33.190353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:25:10.434638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2023-10-26T05:54:35.233346+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:54:35.233346+0000","last_peered":"2023-10-26T05:54:35.233346+0000","last_clean":"2023-10-26T05:54:35.233346+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:54:35.233346+0000","last_undegraded":"2023-10-26T05:54:35.233346+0000","last_fullsized":"2023-10-26T05:54:35.233346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1424,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193887,"statfs":{"total":200000000,"available":198540986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.39600000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.347},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.84899999999999998,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.169,"5min":1.286,"15min":1.3100000000000001},"last":0.93600000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.94799999999999995,"15min":1.018},"min":{"1min":0.51900000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.98999999999999999,"15min":0.97499999999999998},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.236,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.998}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.61799999999999999,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.92200000000000004,"15min":0.93100000000000005},"min":{"1min":0.67100000000000004,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.2849999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.90300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69199999999999995,"5min":0.51900000000000002,"15min":0.51500000000000001},"max":{"1min":1.2,"5min":1.214,"15min":1.4330000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.95899999999999996,"15min":0.96599999999999997},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3129999999999999,"5min":1.375,"15min":1.375},"last":1.03}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.1679999999999999,"5min":1.639,"15min":1.639},"last":1.109},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.96399999999999997,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3149999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.96999999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.0169999999999999,"15min":1.1000000000000001},"min":{"1min":0.60899999999999999,"5min":0.58999999999999997,"15min":0.58699999999999997},"max":{"1min":1.266,"5min":1.3899999999999999,"15min":1.6679999999999999},"last":1.147},{"interface":"front","average":{"1min":1.002,"5min":1.0129999999999999,"15min":1.0720000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.53800000000000003},"max":{"1min":1.3029999999999999,"5min":1.702,"15min":1.702},"last":1.262}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.081,"15min":1.1100000000000001},"min":{"1min":0.68999999999999995,"5min":0.66200000000000003,"15min":0.59599999999999997},"max":{"1min":1.3640000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0609999999999999,"15min":1.0509999999999999},"min":{"1min":0.66700000000000004,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.5,"5min":1.5,"15min":1.6180000000000001},"last":1.0840000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":847,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194464,"statfs":{"total":200000000,"available":199131992,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.498,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.034},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86399999999999999,"15min":0.85799999999999998},"min":{"1min":0.52000000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.298,"5min":1.298,"15min":1.3},"last":0.5}]},{"osd":1,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.78100000000000003},"min":{"1min":0.55400000000000005,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.0660000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.62,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.405},"last":0.91600000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.89300000000000002,"15min":0.82099999999999995},"min":{"1min":0.54800000000000004,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1990000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.502,"5min":0.435,"15min":0.435},"max":{"1min":1.3,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.004}]},{"osd":3,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":0.97999999999999998,"15min":0.90400000000000003},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2250000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":1.113},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.056,"15min":0.97999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.3700000000000001,"5min":1.399,"15min":1.399},"last":0.96199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.85699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.212},{"interface":"front","average":{"1min":1.002,"5min":0.97799999999999998,"15min":0.97899999999999998},"min":{"1min":0.64500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.351,"5min":1.375,"15min":1.375},"last":0.81299999999999994}]},{"osd":5,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.032,"15min":1.0469999999999999},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.554},"last":1.167},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.95499999999999996,"15min":0.99199999999999999},"min":{"1min":0.76800000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.25,"5min":1.512,"15min":1.512},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1000000000000001,"15min":1.1060000000000001},"min":{"1min":0.96799999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.94999999999999996,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4550000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133492,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.93500000000000005,"15min":0.94999999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.381,"5min":1.579,"15min":1.579},"last":1.3640000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.94799999999999995},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.329}]},{"osd":1,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.53300000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.089,"5min":1.2130000000000001,"15min":2.1150000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.87,"15min":0.85699999999999998},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1459999999999999,"5min":1.464,"15min":1.464},"last":0.85999999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.877,"15min":0.91500000000000004},"min":{"1min":0.58699999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.2110000000000001,"5min":1.319,"15min":1.5089999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.91300000000000003,"15min":0.92700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2190000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.91100000000000003,"15min":0.91900000000000004},"min":{"1min":0.59299999999999997,"5min":0.502,"15min":0.502},"max":{"1min":1.333,"5min":1.448,"15min":1.548},"last":1.1890000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97599999999999998,"15min":0.995},"min":{"1min":0.70399999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.238,"5min":1.4139999999999999,"15min":1.587},"last":0.76000000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.94099999999999995,"15min":0.99399999999999999},"min":{"1min":0.61599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2769999999999999,"5min":1.286,"15min":2.1909999999999998},"last":1.282},{"interface":"front","average":{"1min":1.091,"5min":1.0680000000000001,"15min":1.095},"min":{"1min":0.59099999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.4950000000000001,"5min":1.536,"15min":1.536},"last":1.4370000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.96899999999999997,"15min":1.018},"min":{"1min":0.63500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.0169999999999999,"15min":1.032},"min":{"1min":0.80700000000000005,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.4199999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:54:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0469999999999999,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.442,"15min":0.442},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.125,"15min":1.1599999999999999},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.238}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.85999999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.248,"5min":1.2569999999999999,"15min":1.347},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88400000000000001,"15min":0.88},"min":{"1min":0.53100000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.143,"5min":1.502,"15min":1.502},"last":0.874}]},{"osd":1,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.878,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.171,"5min":1.25,"15min":1.375},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.96199999999999997,"15min":0.90700000000000003},"min":{"1min":0.65100000000000002,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":1.3440000000000001,"15min":1.4390000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.90700000000000003,"15min":0.83099999999999996},"min":{"1min":0.61099999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.306,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.89500000000000002,"15min":0.86799999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.107,"5min":1.373,"15min":1.373},"last":1.022}]},{"osd":3,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.94599999999999995,"15min":0.94399999999999995},"min":{"1min":0.61499999999999999,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.397,"5min":1.397,"15min":2.0190000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.71099999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.232,"5min":1.3300000000000001,"15min":1.4359999999999999},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90600000000000003,"15min":0.88900000000000001},"min":{"1min":0.65600000000000003,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.3340000000000001,"5min":1.3440000000000001,"15min":1.3859999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.99399999999999999,"15min":0.998},"min":{"1min":0.64300000000000002,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.323,"5min":1.4179999999999999,"15min":1.673},"last":1.113}]},{"osd":6,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0289999999999999,"15min":1.004},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.42099999999999999},"max":{"1min":1.762,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.73799999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3540000000000001,"5min":1.4690000000000001,"15min":1.4970000000000001},"last":1.089}]},{"osd":7,"last update":"Thu Oct 26 05:54:29 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.069,"15min":1.0429999999999999},"min":{"1min":0.68600000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.659,"15min":1.7509999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.137,"15min":1.119},"min":{"1min":0.74299999999999999,"5min":0.55500000000000005,"15min":0.53700000000000003},"max":{"1min":1.484,"5min":1.728,"15min":1.728},"last":1.204}]}]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133444,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.59599999999999997,"15min":0.56999999999999995},"min":{"1min":0.29599999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.76800000000000002,"5min":1.385,"15min":1.385},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.58199999999999996},"min":{"1min":0.30399999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.94699999999999995,"5min":1.115,"15min":1.2689999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.30599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.80100000000000005,"5min":1.349,"15min":1.349},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65900000000000003,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.33700000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.35099999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.99299999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.497},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.76900000000000002},"min":{"1min":0.39300000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":1.1619999999999999,"5min":1.236,"15min":1.4199999999999999},"last":0.36699999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.88600000000000001,"15min":0.83499999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.40000000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.57499999999999996,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.073,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.94999999999999996,"15min":0.95599999999999996},"min":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.55800000000000005},"max":{"1min":1.113,"5min":1.413,"15min":1.478},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.94699999999999995,"15min":0.91200000000000003},"min":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.59799999999999998},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.84399999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93300000000000005,"15min":0.97999999999999998},"min":{"1min":0.54100000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.165,"5min":1.6060000000000001,"15min":1.6060000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.0349999999999999,"15min":1.0189999999999999},"min":{"1min":0.53900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.254,"5min":5.6859999999999999,"15min":5.6859999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Oct 26 05:54:21 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.98899999999999999,"15min":0.93200000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.232,"5min":2.004,"15min":2.004},"last":0.629}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194465,"statfs":{"total":200000000,"available":199132988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.54400000000000004},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.88300000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71799999999999997,"15min":0.61199999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1120000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.67600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.198,"5min":1.226,"15min":1.2609999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.68999999999999995,"15min":0.60199999999999998},"min":{"1min":0.30199999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.157,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80900000000000005,"15min":0.74399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.149,"5min":1.333,"15min":2.8809999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71599999999999997,"15min":0.61099999999999999},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.089,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.91700000000000004,"15min":0.83899999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1060000000000001,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.88100000000000001,"15min":0.84599999999999997},"min":{"1min":0.54400000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1459999999999999,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.96999999999999997,"15min":0.91900000000000004},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.476,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.99299999999999999,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.653,"15min":1.653},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":1.002,"15min":0.92700000000000005},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.1100000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.02,"5min":1.077,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.419,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:54:23 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.97799999999999998,"15min":0.91000000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.173,"5min":1.581,"15min":1.581},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.0900000000000001,"15min":1.0449999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.367,"5min":1.823,"15min":15.352},"last":0.89800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":846,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194466,"statfs":{"total":200000000,"available":199133604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.90800000000000003,"5min":1.0049999999999999,"15min":1.0349999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":1.157,"15min":1.157},"last":0.66900000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83599999999999997,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.006},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":0.72299999999999998},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27700000000000002},"max":{"1min":1.087,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0149999999999999,"5min":1.5369999999999999,"15min":1.7949999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.70299999999999996,"15min":0.68200000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.89500000000000002,"5min":1.298,"15min":1.298},"last":0.505}]},{"osd":4,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88300000000000001,"15min":0.86899999999999999},"min":{"1min":0.64500000000000002,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1659999999999999,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.95099999999999996,"15min":0.94699999999999995},"min":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.27,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":0.80100000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95699999999999996,"15min":0.96299999999999997},"min":{"1min":0.621,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.3620000000000001,"5min":1.492,"15min":1.492},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.92400000000000004,"15min":0.92900000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.1439999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.98999999999999999,"15min":1.028},"min":{"1min":0.59299999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3029999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96799999999999997,"15min":0.96699999999999997},"min":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.56899999999999995},"max":{"1min":1.1850000000000001,"5min":1.468,"15min":1.837},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:54:24 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.012,"15min":1.022},"min":{"1min":0.621,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.2330000000000001,"5min":1.4450000000000001,"15min":1.6339999999999999},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.046,"15min":1.0840000000000001},"min":{"1min":0.63200000000000001,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.4099999999999999,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":0.879}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193889,"statfs":{"total":200000000,"available":198543236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78100000000000003,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.69899999999999995,"15min":0.62},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.85899999999999999,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.56100000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.79500000000000004,"15min":0.68700000000000006},"min":{"1min":0.39100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.093,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67700000000000005,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1020000000000001,"5min":1.1140000000000001,"15min":1.177},"last":0.436}]},{"osd":3,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.67000000000000004,"15min":0.626},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":0.99299999999999999,"5min":1.409,"15min":1.409},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80500000000000005,"15min":0.70999999999999996},"min":{"1min":0.33400000000000002,"5min":0.247,"15min":0.247},"max":{"1min":1.2629999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.90100000000000002,"15min":0.85899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.99299999999999999,"15min":0.92100000000000004},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.2769999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.92600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.1399999999999999,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.90900000000000003},"min":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.52400000000000002},"max":{"1min":1.208,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":0.90900000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1299999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.123,"15min":1.004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.873}]},{"osd":7,"last update":"Thu Oct 26 05:54:22 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.004,"15min":0.996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4399999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0329999999999999,"15min":0.98999999999999999},"min":{"1min":0.59699999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.329,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.83199999999999996}]}]}],"pool_statfs":[]}} 2023-10-26T05:54:43.106 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:54:43.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:54:43.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 -- 172.21.15.71:0/4290902694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 msgr2=0x7ff9940cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 --2- 172.21.15.71:0/4290902694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff9940cb820 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7ff984009900 tx=0x7ff98401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 -- 172.21.15.71:0/4290902694 shutdown_connections 2023-10-26T05:54:43.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 --2- 172.21.15.71:0/4290902694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff9940cb820 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 -- 172.21.15.71:0/4290902694 >> 172.21.15.71:0/4290902694 conn(0x7ff9940aec60 msgr2=0x7ff9940af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:43.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 -- 172.21.15.71:0/4290902694 shutdown_connections 2023-10-26T05:54:43.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 -- 172.21.15.71:0/4290902694 wait complete. 2023-10-26T05:54:43.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 Processor -- start 2023-10-26T05:54:43.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.271+0000 7ff99be05640 1 -- start start 2023-10-26T05:54:43.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9940cbd60 0x7ff9941358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99413c220 0x7ff99413ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff99406ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff994136140 con 0x7ff99415d3f0 2023-10-26T05:54:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff994070490 con 0x7ff99413c220 2023-10-26T05:54:43.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940706a0 con 0x7ff9940cbd60 2023-10-26T05:54:43.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff99406ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff99406ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56038/0 (socket says 172.21.15.71:56038) 2023-10-26T05:54:43.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 -- 172.21.15.71:0/1111797201 learned_addr learned my addr 172.21.15.71:0/1111797201 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:43.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff9993a0640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99413c220 0x7ff99413ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff999ba1640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9940cbd60 0x7ff9941358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 -- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9940cbd60 msgr2=0x7ff9941358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9940cbd60 0x7ff9941358a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 -- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99413c220 msgr2=0x7ff99413ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99413c220 0x7ff99413ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 -- 172.21.15.71:0/1111797201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984009560 con 0x7ff99415d3f0 2023-10-26T05:54:43.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff9993a0640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99413c220 0x7ff99413ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:43.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff999ba1640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9940cbd60 0x7ff9941358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:43.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99a3a2640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff99406ff50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c770 tx=0x7ff98c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:43.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c003b80 con 0x7ff99415d3f0 2023-10-26T05:54:43.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff98c003ce0 con 0x7ff99415d3f0 2023-10-26T05:54:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff994139e80 con 0x7ff99415d3f0 2023-10-26T05:54:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00aab0 con 0x7ff99415d3f0 2023-10-26T05:54:43.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99413a440 con 0x7ff99415d3f0 2023-10-26T05:54:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994066b10 con 0x7ff99415d3f0 2023-10-26T05:54:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff98c00ac10 con 0x7ff99415d3f0 2023-10-26T05:54:43.291 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff98a7fc640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9780664a0 0x7ff978068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff98c083e20 con 0x7ff99415d3f0 2023-10-26T05:54:43.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff999ba1640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9780664a0 0x7ff978068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.275+0000 7ff999ba1640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9780664a0 0x7ff978068960 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff994158c90 tx=0x7ff990009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:43.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.279+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff98c0512a0 con 0x7ff99415d3f0 2023-10-26T05:54:43.502 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.495+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff99406b060 con 0x7ff99415d3f0 2023-10-26T05:54:43.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.499+0000 7ff98a7fc640 1 -- 172.21.15.71:0/1111797201 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff98c0548c0 con 0x7ff99415d3f0 2023-10-26T05:54:43.508 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:54:43.508 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:54:43.509 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:54:43.509 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:54:43.509 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:54:43.511 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:54:43.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9780664a0 msgr2=0x7ff978068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9780664a0 0x7ff978068960 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff994158c90 tx=0x7ff990009040 comp rx=0 tx=0).stop 2023-10-26T05:54:43.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 msgr2=0x7ff99406ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff99406ff50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c770 tx=0x7ff98c00cc30 comp rx=0 tx=0).stop 2023-10-26T05:54:43.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 shutdown_connections 2023-10-26T05:54:43.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff9780664a0 0x7ff978068960 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff99415d3f0 0x7ff99406ff50 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff99413c220 0x7ff99413ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 --2- 172.21.15.71:0/1111797201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9940cbd60 0x7ff9941358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 >> 172.21.15.71:0/1111797201 conn(0x7ff9940aec60 msgr2=0x7ff9940b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:43.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 shutdown_connections 2023-10-26T05:54:43.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.503+0000 7ff99be05640 1 -- 172.21.15.71:0/1111797201 wait complete. 2023-10-26T05:54:43.527 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 -- 172.21.15.71:0/36433665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 msgr2=0x7f732015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 --2- 172.21.15.71:0/36433665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015b8e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f73200c4dd0 tx=0x7f731801ccb0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 -- 172.21.15.71:0/36433665 shutdown_connections 2023-10-26T05:54:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 --2- 172.21.15.71:0/36433665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015b8e0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 --2- 172.21.15.71:0/36433665 >> v2:172.21.15.86:3300/0 conn(0x7f73200d0850 0x7f732015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 -- 172.21.15.71:0/36433665 >> 172.21.15.71:0/36433665 conn(0x7f73200cdd80 msgr2=0x7f73200ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 -- 172.21.15.71:0/36433665 shutdown_connections 2023-10-26T05:54:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.643+0000 7f7327de8640 1 -- 172.21.15.71:0/36433665 wait complete. 2023-10-26T05:54:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 Processor -- start 2023-10-26T05:54:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- start start 2023-10-26T05:54:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f73200d0470 0x7f7320159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73200d0850 0x7f7320152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73201599a0 con 0x7f73200cd880 2023-10-26T05:54:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732015d620 con 0x7f73200d0850 2023-10-26T05:54:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732015e1a0 con 0x7f73200d0470 2023-10-26T05:54:43.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56044/0 (socket says 172.21.15.71:56044) 2023-10-26T05:54:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 -- 172.21.15.71:0/3441549561 learned_addr learned my addr 172.21.15.71:0/3441549561 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7326385640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73200d0850 0x7f7320152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 -- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f73200d0470 msgr2=0x7f7320159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f73200d0470 0x7f7320159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 -- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73200d0850 msgr2=0x7f7320152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73200d0850 0x7f7320152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 -- 172.21.15.71:0/3441549561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180098e0 con 0x7f73200cd880 2023-10-26T05:54:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325b84640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015a230 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f731000c500 tx=0x7f731000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:43.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310003600 con 0x7f73200cd880 2023-10-26T05:54:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7310003850 con 0x7f73200cd880 2023-10-26T05:54:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732015d870 con 0x7f73200cd880 2023-10-26T05:54:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7310003c60 con 0x7f73200cd880 2023-10-26T05:54:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732015de30 con 0x7f73200cd880 2023-10-26T05:54:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7320066b10 con 0x7f73200cd880 2023-10-26T05:54:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f731001b3b0 con 0x7f73200cd880 2023-10-26T05:54:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f73167fc640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7304066470 0x7f7304068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:43.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7310083770 con 0x7f73200cd880 2023-10-26T05:54:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.647+0000 7f7325383640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7304066470 0x7f7304068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.651+0000 7f7325383640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7304066470 0x7f7304068930 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f73200ccf20 tx=0x7f731c005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:43.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.651+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7310050cc0 con 0x7f73200cd880 2023-10-26T05:54:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73200721e0 con 0x7f73200cd880 2023-10-26T05:54:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f73167fc640 1 -- 172.21.15.71:0/3441549561 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7310017090 con 0x7f73200cd880 2023-10-26T05:54:43.851 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:43.851 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:54:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:54:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7304066470 msgr2=0x7f7304068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7304066470 0x7f7304068930 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f73200ccf20 tx=0x7f731c005b60 comp rx=0 tx=0).stop 2023-10-26T05:54:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 msgr2=0x7f732015a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015a230 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f731000c500 tx=0x7f731000c9c0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 shutdown_connections 2023-10-26T05:54:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73200d0850 0x7f7320152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f73200d0470 0x7f7320159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7304066470 0x7f7304068930 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 --2- 172.21.15.71:0/3441549561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73200cd880 0x7f732015a230 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 >> 172.21.15.71:0/3441549561 conn(0x7f73200cdd80 msgr2=0x7f73200b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 shutdown_connections 2023-10-26T05:54:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:43.843+0000 7f7327de8640 1 -- 172.21.15.71:0/3441549561 wait complete. 2023-10-26T05:54:44.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:54:34.550038+0000', 'modified': '2023-10-26T05:54:36.609944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8517': {'gid': 8517, 'name': 'c', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/2112769040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6833', 'nonce': 2112769040}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2112769040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:54:44.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:54:44.274 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T05:54:44.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 -- 172.21.15.71:0/1198122627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 msgr2=0x7f8e0815ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 --2- 172.21.15.71:0/1198122627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e0815ef00 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f8e080d2220 tx=0x7f8e0401c870 comp rx=0 tx=0).stop 2023-10-26T05:54:44.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 -- 172.21.15.71:0/1198122627 shutdown_connections 2023-10-26T05:54:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 --2- 172.21.15.71:0/1198122627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 0x7f8e0815f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 --2- 172.21.15.71:0/1198122627 >> v2:172.21.15.86:3300/0 conn(0x7f8e0815f980 0x7f8e080cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 --2- 172.21.15.71:0/1198122627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e0815ef00 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 -- 172.21.15.71:0/1198122627 >> 172.21.15.71:0/1198122627 conn(0x7f8e080b3b90 msgr2=0x7f8e080d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 -- 172.21.15.71:0/1198122627 shutdown_connections 2023-10-26T05:54:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 -- 172.21.15.71:0/1198122627 wait complete. 2023-10-26T05:54:44.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 Processor -- start 2023-10-26T05:54:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.403+0000 7f8e1062a640 1 -- start start 2023-10-26T05:54:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 0x7f8e081498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e08153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0815f980 0x7f8e081543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08153500 con 0x7f8e0815b410 2023-10-26T05:54:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08149e10 con 0x7f8e0815f980 2023-10-26T05:54:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08154e30 con 0x7f8e080b0440 2023-10-26T05:54:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e08153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e08153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56058/0 (socket says 172.21.15.71:56058) 2023-10-26T05:54:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/150028976 learned_addr learned my addr 172.21.15.71:0/150028976 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0815f980 0x7f8e081543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 0x7f8e081498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 msgr2=0x7f8e081498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 0x7f8e081498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0815f980 msgr2=0x7f8e081543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0815f980 0x7f8e081543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/150028976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04009560 con 0x7f8e0815b410 2023-10-26T05:54:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 0x7f8e081498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e08153e90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c9a0 tx=0x7f8df800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8dfeffd640 1 -- 172.21.15.71:0/150028976 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8007bb0 con 0x7f8e0815b410 2023-10-26T05:54:44.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8dfeffd640 1 -- 172.21.15.71:0/150028976 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8df8007e20 con 0x7f8e0815b410 2023-10-26T05:54:44.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e08153710 con 0x7f8e0815b410 2023-10-26T05:54:44.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8dfeffd640 1 -- 172.21.15.71:0/150028976 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8005280 con 0x7f8e0815b410 2023-10-26T05:54:44.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e08154910 con 0x7f8e0815b410 2023-10-26T05:54:44.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8dfeffd640 1 -- 172.21.15.71:0/150028976 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8df80053e0 con 0x7f8e0815b410 2023-10-26T05:54:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8dfeffd640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8dec066470 0x7f8dec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8dfeffd640 1 -- 172.21.15.71:0/150028976 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8df8084760 con 0x7f8e0815b410 2023-10-26T05:54:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8dec066470 0x7f8dec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.407+0000 7f8e13774000 1 Processor -- start 2023-10-26T05:54:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- start start 2023-10-26T05:54:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8dec066470 0x7f8dec068930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8e040af000 tx=0x7f8e0401b0b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998078bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078c2e0 0x559980741a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559980741f80 0x559980742360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559980536cc0 con 0x55998078b9c0 2023-10-26T05:54:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559980599920 con 0x559980741f80 2023-10-26T05:54:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x559980596e50 con 0x55998078c2e0 2023-10-26T05:54:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998078bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998078bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56066/0 (socket says 172.21.15.71:56066) 2023-10-26T05:54:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 -- 172.21.15.71:0/2575003798 learned_addr learned my addr 172.21.15.71:0/2575003798 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559980741f80 0x559980742360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078c2e0 0x559980741a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 -- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078c2e0 msgr2=0x559980741a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078c2e0 0x559980741a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 -- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559980741f80 msgr2=0x559980742360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559980741f80 0x559980742360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 -- 172.21.15.71:0/2575003798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55998028b890 con 0x55998078b9c0 2023-10-26T05:54:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998078bda0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005ed0 tx=0x7f8e040b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68033af8b81de844 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8debfff640 1 -- 172.21.15.71:0/2575003798 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004240 con 0x55998078b9c0 2023-10-26T05:54:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8debfff640 1 -- 172.21.15.71:0/2575003798 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8e040044b0 con 0x55998078b9c0 2023-10-26T05:54:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8debfff640 1 -- 172.21.15.71:0/2575003798 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e040052b0 con 0x55998078b9c0 2023-10-26T05:54:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- 172.21.15.71:0/2575003798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 msgr2=0x55998078bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998078bda0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8e04005ed0 tx=0x7f8e040b5000 comp rx=0 tx=0).stop 2023-10-26T05:54:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- 172.21.15.71:0/2575003798 shutdown_connections 2023-10-26T05:54:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559980741f80 0x559980742360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078c2e0 0x559980741a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- 172.21.15.71:0/2575003798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998078bda0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- 172.21.15.71:0/2575003798 >> 172.21.15.71:0/2575003798 conn(0x5599807392f0 msgr2=0x559980740320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- 172.21.15.71:0/2575003798 shutdown_connections 2023-10-26T05:54:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- 172.21.15.71:0/2575003798 wait complete. 2023-10-26T05:54:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- start start 2023-10-26T05:54:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 Processor -- start 2023-10-26T05:54:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x55998074cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998074d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55998078c2e0 0x55998074d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559980592740 con 0x55998078b9c0 2023-10-26T05:54:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559980551d10 con 0x55998078c2e0 2023-10-26T05:54:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599805519b0 con 0x559980741f80 2023-10-26T05:54:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e13774000 10 client.0 _ll_register_callbacks cb 0x55998072d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:54:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0dbc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998074d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0dbc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998074d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56070/0 (socket says 172.21.15.71:56070) 2023-10-26T05:54:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.411+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/2522022609 learned_addr learned my addr 172.21.15.71:0/2522022609 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55998078c2e0 0x55998074d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x55998074cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 msgr2=0x55998074cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x55998074cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55998078c2e0 msgr2=0x55998074d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55998078c2e0 0x55998074d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/2522022609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0806a710 con 0x55998078b9c0 2023-10-26T05:54:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55998078c2e0 0x55998074d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998074d0d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8df800afd0 tx=0x7f8df8055950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec06c309aff2a05 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:54:44.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 1 -- 172.21.15.71:0/2522022609 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8054f80 con 0x55998078b9c0 2023-10-26T05:54:44.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 1 -- 172.21.15.71:0/2522022609 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8df80828b0 con 0x55998078b9c0 2023-10-26T05:54:44.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559980546370 con 0x55998078b9c0 2023-10-26T05:54:44.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 1 -- 172.21.15.71:0/2522022609 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df805d700 con 0x55998078b9c0 2023-10-26T05:54:44.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 10 client.8682 fetch_fsmap learned FSMap version 58 2023-10-26T05:54:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x559980554460 con 0x55998078b9c0 2023-10-26T05:54:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 1 -- 172.21.15.71:0/2522022609 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x559980546370 con 0x55998078b9c0 2023-10-26T05:54:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 1 -- 172.21.15.71:0/2522022609 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8df8020070 con 0x55998078b9c0 2023-10-26T05:54:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 10 client.8682 fetch_fsmap finished waiting for FSMap version 58 2023-10-26T05:54:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 10 client.8682 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:54:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 20 client.8682 populate_metadata read hostname 'smithi071' 2023-10-26T05:54:44.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998079aa50 0x55998079ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 4 client.8682 mds_command: new command op to 8517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:54:44.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 --> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55998052c2d0 con 0x55998079aa50 2023-10-26T05:54:44.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998079aa50 0x55998079ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998079aa50 0x55998079ce40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee62500bb764cc7b server_cookie=26f966a1639cb8d0 in_seq=0 out_seq=0 2023-10-26T05:54:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 10 client.8682 ms_handle_connect on v2:172.21.15.71:6833/2112769040 2023-10-26T05:54:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 1 -- 172.21.15.71:0/2522022609 <== mds.0 v2:172.21.15.71:6833/2112769040 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55998052c2d0 con 0x55998079aa50 2023-10-26T05:54:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 10 client.8682 handle_command_reply: tid=0 2023-10-26T05:54:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 client.8682 shutdown 2023-10-26T05:54:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 msgr2=0x55998074d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998074d0d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8df800afd0 tx=0x7f8df8055950 comp rx=0 tx=0).stop 2023-10-26T05:54:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 shutdown_connections 2023-10-26T05:54:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998079aa50 0x55998079ce40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55998078c2e0 0x55998074d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8de97fa640 0 client.8682 ms_handle_reset on v2:172.21.15.71:6833/2112769040 2023-10-26T05:54:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x55998074d0d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 --2- 172.21.15.71:0/2522022609 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x55998074cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 >> 172.21.15.71:0/2522022609 conn(0x5599807392f0 msgr2=0x559980740a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 shutdown_connections 2023-10-26T05:54:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 1 -- 172.21.15.71:0/2522022609 wait complete. 2023-10-26T05:54:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.415+0000 7f8e13774000 20 client.8682 trim_cache size 0 max 16384 2023-10-26T05:54:44.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 Processor -- start 2023-10-26T05:54:44.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 -- start start 2023-10-26T05:54:44.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x5599807e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 0x5599807e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078b9c0 0x5599807e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998024c940 con 0x559980741f80 2023-10-26T05:54:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x559980744d90 con 0x5599807392f0 2023-10-26T05:54:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599805406c0 con 0x55998078b9c0 2023-10-26T05:54:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 0x5599807e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 0x5599807e69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56086/0 (socket says 172.21.15.71:56086) 2023-10-26T05:54:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/3747240350 learned_addr learned my addr 172.21.15.71:0/3747240350 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/3747240350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078b9c0 msgr2=0x5599807e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078b9c0 0x5599807e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/3747240350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 msgr2=0x5599807e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x5599807e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 -- 172.21.15.71:0/3747240350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55998053a760 con 0x559980741f80 2023-10-26T05:54:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 0x5599807e69a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8df8053be0 tx=0x7f8df800a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17bfa1d9ed617dc server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8dfcff9640 1 -- 172.21.15.71:0/3747240350 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8055660 con 0x559980741f80 2023-10-26T05:54:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8dfcff9640 1 -- 172.21.15.71:0/3747240350 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8df805d950 con 0x559980741f80 2023-10-26T05:54:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.431+0000 7f8dfcff9640 1 -- 172.21.15.71:0/3747240350 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80599e0 con 0x559980741f80 2023-10-26T05:54:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/3747240350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 msgr2=0x5599807e69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 0x5599807e69a0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8df8053be0 tx=0x7f8df800a520 comp rx=0 tx=0).stop 2023-10-26T05:54:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/3747240350 shutdown_connections 2023-10-26T05:54:44.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55998078b9c0 0x5599807e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559980741f80 0x5599807e69a0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/3747240350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x5599807e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/3747240350 >> 172.21.15.71:0/3747240350 conn(0x559980747910 msgr2=0x55998079dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.454 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/3747240350 shutdown_connections 2023-10-26T05:54:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/3747240350 wait complete. 2023-10-26T05:54:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- start start 2023-10-26T05:54:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 Processor -- start 2023-10-26T05:54:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x55998078a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x5599807a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x5599807a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599805952a0 con 0x55998078b9c0 2023-10-26T05:54:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x55998058e0b0 con 0x5599807392f0 2023-10-26T05:54:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x559980596740 con 0x559980741f80 2023-10-26T05:54:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 10 client.0 _ll_register_callbacks cb 0x559980590ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:54:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x5599807a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x5599807a6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56100/0 (socket says 172.21.15.71:56100) 2023-10-26T05:54:44.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 -- 172.21.15.71:0/2059729966 learned_addr learned my addr 172.21.15.71:0/2059729966 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:44.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x55998078a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x5599807a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 -- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 msgr2=0x5599807a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x5599807a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 -- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 msgr2=0x55998078a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:54:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x55998078a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 -- 172.21.15.71:0/2059729966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559980554460 con 0x55998078b9c0 2023-10-26T05:54:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0dbc5640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x5599807a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:54:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0ebc7640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x5599807a6740 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8e000117d0 tx=0x7f8e00011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69091d44869e3749 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 1 -- 172.21.15.71:0/2059729966 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00011ee0 con 0x55998078b9c0 2023-10-26T05:54:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 1 -- 172.21.15.71:0/2059729966 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e0000fcf0 con 0x55998078b9c0 2023-10-26T05:54:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559980745910 con 0x55998078b9c0 2023-10-26T05:54:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 1 -- 172.21.15.71:0/2059729966 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0000feb0 con 0x55998078b9c0 2023-10-26T05:54:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 10 client.8688 fetch_fsmap learned FSMap version 58 2023-10-26T05:54:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55998028bcc0 con 0x55998078b9c0 2023-10-26T05:54:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 1 -- 172.21.15.71:0/2059729966 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x559980745910 con 0x55998078b9c0 2023-10-26T05:54:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 1 -- 172.21.15.71:0/2059729966 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e00012450 con 0x55998078b9c0 2023-10-26T05:54:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 10 client.8688 fetch_fsmap finished waiting for FSMap version 58 2023-10-26T05:54:44.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 10 client.8688 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:54:44.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 20 client.8688 populate_metadata read hostname 'smithi071' 2023-10-26T05:54:44.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998078c2e0 0x5599807ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 4 client.8688 mds_command: new command op to 8517 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:54:44.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 --> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55998052a6f0 con 0x55998078c2e0 2023-10-26T05:54:44.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998078c2e0 0x5599807ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e0e3c6640 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998078c2e0 0x5599807ea0c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6818ea78d8edce server_cookie=959dd17901d1dcfd in_seq=0 out_seq=0 2023-10-26T05:54:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 10 client.8688 ms_handle_connect on v2:172.21.15.71:6833/2112769040 2023-10-26T05:54:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 1 -- 172.21.15.71:0/2059729966 <== mds.0 v2:172.21.15.71:6833/2112769040 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55998052a6f0 con 0x55998078c2e0 2023-10-26T05:54:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 10 client.8688 handle_command_reply: tid=0 2023-10-26T05:54:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 client.8688 shutdown 2023-10-26T05:54:44.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 msgr2=0x5599807a6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x5599807a6740 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f8e000117d0 tx=0x7f8e00011c90 comp rx=0 tx=0).stop 2023-10-26T05:54:44.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 shutdown_connections 2023-10-26T05:54:44.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:6833/2112769040,v1:172.21.15.71:6835/2112769040] conn(0x55998078c2e0 0x5599807ea0c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55998078b9c0 0x5599807a6740 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x559980741f80 0x5599807a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8de97fa640 0 client.8688 ms_handle_reset on v2:172.21.15.71:6833/2112769040 2023-10-26T05:54:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 --2- 172.21.15.71:0/2059729966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5599807392f0 0x55998078a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 >> 172.21.15.71:0/2059729966 conn(0x559980747910 msgr2=0x55998079e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 shutdown_connections 2023-10-26T05:54:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 1 -- 172.21.15.71:0/2059729966 wait complete. 2023-10-26T05:54:44.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.435+0000 7f8e13774000 20 client.8688 trim_cache size 0 max 16384 2023-10-26T05:54:44.465 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:44.465 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:54:44.466 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:54:44.466 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8517, 2023-10-26T05:54:44.466 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:54:44.466 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:54:44.466 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:54:44.467 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 9.2282143449999996, 2023-10-26T05:54:44.467 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 58, 2023-10-26T05:54:44.467 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 155, 2023-10-26T05:54:44.467 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 155, 2023-10-26T05:54:44.467 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 14.177733707 2023-10-26T05:54:44.467 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:44.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8dec066470 msgr2=0x7f8dec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8dec066470 0x7f8dec068930 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8e040af000 tx=0x7f8e0401b0b0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 msgr2=0x7f8e08153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e08153e90 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f8df800c9a0 tx=0x7f8df800ce60 comp rx=0 tx=0).stop 2023-10-26T05:54:44.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 shutdown_connections 2023-10-26T05:54:44.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e0815f980 0x7f8e081543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8dec066470 0x7f8dec068930 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e0815b410 0x7f8e08153e90 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 --2- 172.21.15.71:0/150028976 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e080b0440 0x7f8e081498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 >> 172.21.15.71:0/150028976 conn(0x7f8e080b3b90 msgr2=0x7f8e080b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 shutdown_connections 2023-10-26T05:54:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.439+0000 7f8e1062a640 1 -- 172.21.15.71:0/150028976 wait complete. 2023-10-26T05:54:44.472 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:54:44.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/3532069783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0470 msgr2=0x7f3cc806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 --2- 172.21.15.71:0/3532069783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0470 0x7f3cc806aa70 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f3cc808a3b0 tx=0x7f3cbc01c720 comp rx=0 tx=0).stop 2023-10-26T05:54:44.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/3532069783 shutdown_connections 2023-10-26T05:54:44.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 --2- 172.21.15.71:0/3532069783 >> v2:172.21.15.86:3300/0 conn(0x7f3cc80cd880 0x7f3cc815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 --2- 172.21.15.71:0/3532069783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0470 0x7f3cc806aa70 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/3532069783 >> 172.21.15.71:0/3532069783 conn(0x7f3cc80cdd80 msgr2=0x7f3cc80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/3532069783 shutdown_connections 2023-10-26T05:54:44.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/3532069783 wait complete. 2023-10-26T05:54:44.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 Processor -- start 2023-10-26T05:54:44.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- start start 2023-10-26T05:54:44.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80cd880 0x7f3cc80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cc80d0470 0x7f3cc814e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 0x7f3cc815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8153130 con 0x7f3cc80d0850 2023-10-26T05:54:44.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8153410 con 0x7f3cc80cd880 2023-10-26T05:54:44.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc81536f0 con 0x7f3cc80d0470 2023-10-26T05:54:44.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 0x7f3cc815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 0x7f3cc815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56110/0 (socket says 172.21.15.71:56110) 2023-10-26T05:54:44.596 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 -- 172.21.15.71:0/2335168460 learned_addr learned my addr 172.21.15.71:0/2335168460 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:44.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd100640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80cd880 0x7f3cc80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccc8ff640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cc80d0470 0x7f3cc814e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 -- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cc80d0470 msgr2=0x7f3cc814e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cc80d0470 0x7f3cc814e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 -- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80cd880 msgr2=0x7f3cc80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80cd880 0x7f3cc80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 -- 172.21.15.71:0/2335168460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc0094a0 con 0x7f3cc80d0850 2023-10-26T05:54:44.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd100640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80cd880 0x7f3cc80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:44.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccc8ff640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cc80d0470 0x7f3cc814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:44.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccd901640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 0x7f3cc815dc90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000ccb0 tx=0x7f3cc000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0004b30 con 0x7f3cc80d0850 2023-10-26T05:54:44.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3cc0004da0 con 0x7f3cc80d0850 2023-10-26T05:54:44.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0005280 con 0x7f3cc80d0850 2023-10-26T05:54:44.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc81539d0 con 0x7f3cc80d0850 2023-10-26T05:54:44.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.583+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc815d770 con 0x7f3cc80d0850 2023-10-26T05:54:44.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.587+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc8066b10 con 0x7f3cc80d0850 2023-10-26T05:54:44.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.587+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3cc00053e0 con 0x7f3cc80d0850 2023-10-26T05:54:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.587+0000 7f3cb9ffb640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3cac066470 0x7f3cac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.587+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cc0084770 con 0x7f3cc80d0850 2023-10-26T05:54:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.587+0000 7f3ccd100640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3cac066470 0x7f3cac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.587+0000 7f3ccd100640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3cac066470 0x7f3cac068930 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0af000 tx=0x7f3cbc01af60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:44.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.591+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3cc0051c00 con 0x7f3cc80d0850 2023-10-26T05:54:44.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.775+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3cc815d990 con 0x7f3cc80d0850 2023-10-26T05:54:44.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.775+0000 7f3cb9ffb640 1 -- 172.21.15.71:0/2335168460 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3cc001d070 con 0x7f3cc80d0850 2023-10-26T05:54:44.784 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:54:44.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3cac066470 msgr2=0x7f3cac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3cac066470 0x7f3cac068930 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0af000 tx=0x7f3cbc01af60 comp rx=0 tx=0).stop 2023-10-26T05:54:44.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 msgr2=0x7f3cc815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:44.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 0x7f3cc815dc90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000ccb0 tx=0x7f3cc000a620 comp rx=0 tx=0).stop 2023-10-26T05:54:44.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 shutdown_connections 2023-10-26T05:54:44.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3cac066470 0x7f3cac068930 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3cc80d0850 0x7f3cc815dc90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3cc80d0470 0x7f3cc814e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 --2- 172.21.15.71:0/2335168460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80cd880 0x7f3cc80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 >> 172.21.15.71:0/2335168460 conn(0x7f3cc80cdd80 msgr2=0x7f3cc80ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 shutdown_connections 2023-10-26T05:54:44.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:44.779+0000 7f3ccf364640 1 -- 172.21.15.71:0/2335168460 wait complete. 2023-10-26T05:54:44.801 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:54:44.802 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:54:44.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:54:44.802 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:54:44.803 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:44.803 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:54:44.803 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:54:44.804 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:54:44.804 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:54:44.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:44.805 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:54:44.811 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:54:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:54:44.811 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:54:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:54:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.812 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:54:44.812 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:54:44.813 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:54:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:44.813 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:54:44.813 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:54:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:44.814 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:54:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:54:44.814 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42092sec preferred_lft 42092sec 2023-10-26T05:54:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:54:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.815 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:54:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:54:44.815 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.816 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:54:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:54:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.816 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:54:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.817 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:54:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:54:44.817 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.818 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:54:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:54:44.818 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:44.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:44.819 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:54:44.863 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:54:44.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:44.864 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:54:44.912 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:54:44.912 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:54:44.913 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:54:44.913 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:54:44.913 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:54:44.913 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:54:44.913 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:54:44.914 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:54:44.914 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:54:44.914 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:54:44.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:44.915 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:54:44.916 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:54:44.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:54:44.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:54:44.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:54:44.917 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:54:44.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:44.987 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:54:45.034 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:54:45.035 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:54:45.035 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:54:45.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:45.036 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:54:45.090 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:54:45.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:54:45.091 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:45.138 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:54:45.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:54:45.139 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:45.185 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:54:45.235 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:54:45.235 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:54:45.236 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:54:45.236 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:54:45.236 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:54:45.236 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:54:45.236 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:54:45.237 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:54:45.237 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:54:45.237 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:54:45.237 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:54:45.237 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:54:45.238 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:54:45.238 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:54:45.238 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:54:45.238 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:54:45.238 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:54:45.239 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:54:45.239 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:54:45.239 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:54:45.239 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:54:45.239 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:54:45.240 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:54:45.240 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:54:45.240 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:54:45.240 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:54:45.240 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:54:45.240 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:54:45.241 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:54:45.241 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:45.241 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:45.241 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:45.242 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:45.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:45.243 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:54:45.290 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:54:45.291 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:54:45.335 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:54:45.342 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:54:45.342 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:54:45.343 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:54:45.343 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:54:45.343 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:54:45.343 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:54:45.343 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:54:45.344 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:54:45.344 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:54:45.344 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:54:45.344 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:54:45.344 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:54:45.345 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:54:45.345 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:54:45.345 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:54:45.345 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:54:45.345 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:54:45.345 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:54:45.346 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:54:45.346 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:54:45.346 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:54:45.346 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:54:45.347 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:54:45.347 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:54:45.348 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:54:45.348 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:54:45.348 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:54:45.348 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:54:45.348 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:54:45.349 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:45.349 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:45.349 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:45.349 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:45.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:45.351 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:54:45.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:54:45.411+0000 7f067a7cc580 -1 init, newargv = 0x55fdfb1c2640 newargc=15 2023-10-26T05:54:45.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[51380]: starting ceph client 2023-10-26T05:54:45.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[51380]: starting fuse 2023-10-26T05:54:46.127 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:54:46.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:54:46.500 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:54:46.509 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:54:46.509 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:54:46.509 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:54:46.510 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:54:46.510 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:54:46.510 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:54:46.510 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:54:46.510 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:54:46.511 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:54:46.511 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:54:46.511 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:54:46.511 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:54:46.511 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:54:46.512 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:54:46.512 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:54:46.512 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:54:46.512 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:54:46.512 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:54:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:54:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:54:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:54:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:54:46.513 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:54:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:54:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:54:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:54:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:54:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:54:46.514 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:54:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:54:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:54:46.515 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:46.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:46.517 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:54:46.562 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:54:46.563 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:54:46.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:54:46.564 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:54:46.564 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:54:46.565 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:54:46.565 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:54:46.565 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:46.566 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:54:46.566 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:54:46.567 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:54:46.567 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:54:46.567 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:46.568 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:54:46.568 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:54:46.568 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:54:46.569 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:54:46.569 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:46.569 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:54:46.570 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:54:46.570 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:54:46.571 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:54:46.571 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:54:46.571 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:54:46.572 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:54:46.572 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:54:46.573 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:46.573 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:54:46.573 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:54:46.687 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.51380.asok 2023-10-26T05:54:46.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:46.699 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51380.asok 2023-10-26T05:54:46.699 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:46.700 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.51380.asok status 2023-10-26T05:54:46.795 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:46.897 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:54:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:54:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:54:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:54:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:54:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:54:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:54:46.899 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "51380", 2023-10-26T05:54:46.899 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:54:46.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:46.899 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:54:46.899 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:54:46.900 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8703, 2023-10-26T05:54:46.900 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:54:46.900 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:54:46.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:54:46.900 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 8703 2023-10-26T05:54:46.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:54:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:54:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1170065942 2023-10-26T05:54:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:54:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:54:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:54:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:54:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1170065942 2023-10-26T05:54:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:54:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.8703 192.168.0.1:0/1170065942", 2023-10-26T05:54:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/1170065942", 2023-10-26T05:54:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:54:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 58, 2023-10-26T05:54:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 155, 2023-10-26T05:54:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:54:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:54:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:54:46.904 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:54:46.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:46.917 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:46.917 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:54:46.918 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:46.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:46.966 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:54:46.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:46.974 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:46.975 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:46.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:54:47.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:47.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:47.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:54:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/2149005152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 msgr2=0x7f0f7815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/2149005152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f7815b3a0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009840 tx=0x7f0f6001c6b0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/2149005152 shutdown_connections 2023-10-26T05:54:47.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/2149005152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f7815b3a0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/2149005152 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f780d0470 0x7f0f7806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/2149005152 >> 172.21.15.71:0/2149005152 conn(0x7f0f780cdd80 msgr2=0x7f0f780ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:47.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/2149005152 shutdown_connections 2023-10-26T05:54:47.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/2149005152 wait complete. 2023-10-26T05:54:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 Processor -- start 2023-10-26T05:54:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- start start 2023-10-26T05:54:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f780cd880 0x7f0f78152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:47.199 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f780d0470 0x7f0f780c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:47.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f780cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:47.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78154800 con 0x7f0f780d0850 2023-10-26T05:54:47.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78153f60 con 0x7f0f780d0470 2023-10-26T05:54:47.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f7dc4e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78154240 con 0x7f0f780cd880 2023-10-26T05:54:47.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f780cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:47.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f780cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56134/0 (socket says 172.21.15.71:56134) 2023-10-26T05:54:47.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 -- 172.21.15.71:0/3741162242 learned_addr learned my addr 172.21.15.71:0/3741162242 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:47.201 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f6effd640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f780d0470 0x7f0f780c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f777fe640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f780cd880 0x7f0f78152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 -- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f780cd880 msgr2=0x7f0f78152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f780cd880 0x7f0f78152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 -- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f780d0470 msgr2=0x7f0f780c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f780d0470 0x7f0f780c3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.187+0000 7f0f77fff640 1 -- 172.21.15.71:0/3741162242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f600094a0 con 0x7f0f780d0850 2023-10-26T05:54:47.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f77fff640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f780cb930 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000cae0 tx=0x7f0f7000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:47.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70003b80 con 0x7f0f780d0850 2023-10-26T05:54:47.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f78154520 con 0x7f0f780d0850 2023-10-26T05:54:47.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0f70003dd0 con 0x7f0f780d0850 2023-10-26T05:54:47.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000aab0 con 0x7f0f780d0850 2023-10-26T05:54:47.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f780c4fe0 con 0x7f0f780d0850 2023-10-26T05:54:47.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f78066b10 con 0x7f0f780d0850 2023-10-26T05:54:47.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.191+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f7000ac10 con 0x7f0f780d0850 2023-10-26T05:54:47.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.195+0000 7f0f74ff9640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f58066470 0x7f0f58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:47.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.195+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f700844a0 con 0x7f0f780d0850 2023-10-26T05:54:47.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.195+0000 7f0f777fe640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f58066470 0x7f0f58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:47.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.195+0000 7f0f777fe640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f58066470 0x7f0f58068930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:47.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.199+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0f70051930 con 0x7f0f780d0850 2023-10-26T05:54:47.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.391+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f780cd040 con 0x7f0f780d0850 2023-10-26T05:54:47.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.391+0000 7f0f74ff9640 1 -- 172.21.15.71:0/3741162242 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f70054f50 con 0x7f0f780d0850 2023-10-26T05:54:47.399 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:54:47.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f58066470 msgr2=0x7f0f58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:47.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f58066470 0x7f0f58068930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2023-10-26T05:54:47.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 msgr2=0x7f0f780cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:47.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f780cb930 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000cae0 tx=0x7f0f7000cbd0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 shutdown_connections 2023-10-26T05:54:47.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f58066470 0x7f0f58068930 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f780d0850 0x7f0f780cb930 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f780d0470 0x7f0f780c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 --2- 172.21.15.71:0/3741162242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f780cd880 0x7f0f78152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:47.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 >> 172.21.15.71:0/3741162242 conn(0x7f0f780cdd80 msgr2=0x7f0f78158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:47.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 shutdown_connections 2023-10-26T05:54:47.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:47.395+0000 7f0f7dc4e640 1 -- 172.21.15.71:0/3741162242 wait complete. 2023-10-26T05:54:47.419 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:54:47.420 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:54:47.420 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:54:47.421 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:54:47.421 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:47.421 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:54:47.422 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:54:47.422 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:54:47.422 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:54:47.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.423 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:54:47.430 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:54:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:54:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:54:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.431 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:54:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.432 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:54:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:47.432 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:54:47.432 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:54:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:54:47.433 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:54:47.433 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:47.433 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:54:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:54:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42083sec preferred_lft 42083sec 2023-10-26T05:54:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:54:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.435 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:54:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:54:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.436 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:54:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:54:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:54:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:54:47.438 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.439 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:54:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:54:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.439 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:54:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:54:47.440 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:54:47.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.441 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:54:47.484 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:54:47.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.486 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:54:47.541 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:54:47.541 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:54:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:54:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:54:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:54:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:54:47.542 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:54:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:54:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:54:47.543 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:54:47.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.544 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:54:47.544 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:54:47.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:54:47.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:54:47.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:54:47.546 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:54:47.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.620 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:54:47.665 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:54:47.665 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:54:47.665 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:54:47.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:54:47.722 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:54:47.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:54:47.723 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:47.769 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:54:47.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:54:47.770 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:47.817 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:54:47.867 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:54:47.867 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:54:47.868 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:54:47.868 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:54:47.868 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:54:47.868 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:54:47.868 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:54:47.869 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:54:47.869 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:54:47.869 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:54:47.869 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:54:47.869 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:54:47.870 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:54:47.870 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:54:47.870 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:54:47.870 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:54:47.870 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:54:47.871 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:54:47.871 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:54:47.871 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:54:47.871 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:54:47.871 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:54:47.872 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:54:47.872 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:54:47.872 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:54:47.872 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:54:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:54:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:54:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:54:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:47.873 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:47.874 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:47.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.875 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:54:47.925 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:54:47.926 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:54:47.974 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:54:47.980 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:54:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:54:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:54:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:54:47.982 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:54:47.982 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:54:47.982 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:54:47.982 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:54:47.982 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:54:47.983 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:54:47.983 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:54:47.983 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:54:47.983 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:54:47.984 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:54:47.984 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:54:47.984 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:54:47.984 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:54:47.984 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:54:47.985 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:54:47.985 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:54:47.985 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:54:47.985 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:54:47.985 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:54:47.986 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:54:47.986 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:54:47.986 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:54:47.986 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:54:47.986 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:54:47.987 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:54:47.987 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:47.987 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:47.987 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:47.987 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:47.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:47.989 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:54:48.059 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:54:48.055+0000 7fa032d90580 -1 init, newargv = 0x55e711ae6390 newargc=15 2023-10-26T05:54:48.059 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35205]: starting ceph client 2023-10-26T05:54:48.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35205]: starting fuse 2023-10-26T05:54:49.138 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:54:49.146 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:54:49.146 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:54:49.146 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:54:49.147 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:54:49.147 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:54:49.147 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:54:49.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:54:49.147 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:54:49.148 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:54:49.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:54:49.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:54:49.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:54:49.148 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:54:49.149 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:54:49.149 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:54:49.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:54:49.149 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:54:49.149 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:54:49.150 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:54:49.150 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:54:49.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:54:49.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:54:49.150 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:54:49.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:54:49.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:54:49.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:54:49.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:54:49.151 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:54:49.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:54:49.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:49.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:54:49.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:49.152 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:54:49.153 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:49.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:54:49.154 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:54:49.201 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:54:49.201 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:54:49.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:54:49.202 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:54:49.202 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:54:49.203 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:54:49.203 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:54:49.204 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:54:49.204 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:54:49.204 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:54:49.205 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:54:49.205 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:54:49.205 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:54:49.206 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:54:49.206 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:54:49.207 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:54:49.207 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:54:49.207 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:54:49.208 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:54:49.208 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:54:49.208 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:54:49.209 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:54:49.209 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:54:49.209 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:54:49.210 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:54:49.210 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:54:49.211 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:54:49.211 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:54:49.211 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:54:49.323 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.35205.asok 2023-10-26T05:54:49.327 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:49.334 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35205.asok 2023-10-26T05:54:49.335 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:49.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.35205.asok status 2023-10-26T05:54:49.421 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:49.524 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:54:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:54:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:54:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:54:49.525 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:54:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:54:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:54:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "35205", 2023-10-26T05:54:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:54:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:54:49.526 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:54:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:54:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6574, 2023-10-26T05:54:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:54:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:54:49.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:54:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 6574 2023-10-26T05:54:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:54:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:54:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:54:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:54:49.528 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3444137167 2023-10-26T05:54:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:54:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:54:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:54:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:54:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:54:49.529 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3444137167 2023-10-26T05:54:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:54:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.6574 192.168.0.1:0/3444137167", 2023-10-26T05:54:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3444137167", 2023-10-26T05:54:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:54:49.530 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 58, 2023-10-26T05:54:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 155, 2023-10-26T05:54:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:54:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:54:49.531 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:54:49.531 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:54:49.538 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:49.545 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:49.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:54:49.546 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:49.588 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:49.589 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:54:49.590 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:49.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:49.596 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:54:49.596 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:54:49.678 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:54:49.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:54:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:54:49.685 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:54:49.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1676331503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 msgr2=0x7f983015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:49.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1676331503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983015b4b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f98300caa50 tx=0x7f981801cb60 comp rx=0 tx=0).stop 2023-10-26T05:54:49.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1676331503 shutdown_connections 2023-10-26T05:54:49.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1676331503 >> v2:172.21.15.86:3300/0 conn(0x7f983015b9f0 0x7f9830070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:49.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1676331503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983015b4b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:49.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1676331503 >> 172.21.15.71:0/1676331503 conn(0x7f9830069770 msgr2=0x7f9830069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:49.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1676331503 shutdown_connections 2023-10-26T05:54:49.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1676331503 wait complete. 2023-10-26T05:54:49.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 Processor -- start 2023-10-26T05:54:49.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- start start 2023-10-26T05:54:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f983013d420 0x7f98300aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f983015b9f0 0x7f98300abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300accb0 con 0x7f98300ac740 2023-10-26T05:54:49.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9830137710 con 0x7f983015b9f0 2023-10-26T05:54:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300acf00 con 0x7f983013d420 2023-10-26T05:54:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:49.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983006a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56160/0 (socket says 172.21.15.71:56160) 2023-10-26T05:54:49.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 -- 172.21.15.71:0/1973083646 learned_addr learned my addr 172.21.15.71:0/1973083646 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:49.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 -- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f983013d420 msgr2=0x7f98300aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:49.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f983013d420 0x7f98300aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:49.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 -- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f983015b9f0 msgr2=0x7f98300abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:49.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f983015b9f0 0x7f98300abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:49.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 -- 172.21.15.71:0/1973083646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818009820 con 0x7f98300ac740 2023-10-26T05:54:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f982e59c640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983006a8f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f982800c4d0 tx=0x7f982800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828005480 con 0x7f98300ac740 2023-10-26T05:54:49.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f98280036f0 con 0x7f98300ac740 2023-10-26T05:54:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98300ad1e0 con 0x7f98300ac740 2023-10-26T05:54:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828003b00 con 0x7f98300ac740 2023-10-26T05:54:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.811+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98300ad7a0 con 0x7f98300ac740 2023-10-26T05:54:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.815+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9830066b10 con 0x7f98300ac740 2023-10-26T05:54:49.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.815+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9828003c60 con 0x7f98300ac740 2023-10-26T05:54:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.815+0000 7f9826ffd640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9810066470 0x7f9810068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:49.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.815+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9828083710 con 0x7f98300ac740 2023-10-26T05:54:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.815+0000 7f982dd9b640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9810066470 0x7f9810068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.815+0000 7f982dd9b640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9810066470 0x7f9810068930 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f98180af000 tx=0x7f9818019900 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:49.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.819+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9828050c50 con 0x7f98300ac740 2023-10-26T05:54:50.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9830135710 con 0x7f98300ac740 2023-10-26T05:54:50.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9826ffd640 1 -- 172.21.15.71:0/1973083646 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f9828017020 con 0x7f98300ac740 2023-10-26T05:54:50.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:54:50.008 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:54:50.008 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:54:50.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9810066470 msgr2=0x7f9810068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9810066470 0x7f9810068930 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f98180af000 tx=0x7f9818019900 comp rx=0 tx=0).stop 2023-10-26T05:54:50.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 msgr2=0x7f983006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983006a8f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f982800c4d0 tx=0x7f982800c990 comp rx=0 tx=0).stop 2023-10-26T05:54:50.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 shutdown_connections 2023-10-26T05:54:50.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f983015b9f0 0x7f98300abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f983013d420 0x7f98300aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9810066470 0x7f9810068930 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 --2- 172.21.15.71:0/1973083646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98300ac740 0x7f983006a8f0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 >> 172.21.15.71:0/1973083646 conn(0x7f9830069770 msgr2=0x7f98300c93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:50.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 shutdown_connections 2023-10-26T05:54:50.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:49.999+0000 7f9834d5a640 1 -- 172.21.15.71:0/1973083646 wait complete. 2023-10-26T05:54:50.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:54:50.144 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":8517,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.914730548,"mdsmap_epoch":58,"osdmap_epoch":155,"osdmap_epoch_barrier":155,"uptime":19.864261622000001} 2023-10-26T05:54:50.144 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 8517, "mdsmap_epoch": 58, "osdmap_epoch": 155, "osdmap_epoch_barrier": 155, "rank_uptime": 14.914730548, "state": "up:active", "uptime": 19.864261622, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:54:50.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T05:54:50.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 -- 172.21.15.71:0/381643419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 msgr2=0x7fab4013ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 --2- 172.21.15.71:0/381643419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab4013ed90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fab4006c780 tx=0x7fab3801cd30 comp rx=0 tx=0).stop 2023-10-26T05:54:50.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 -- 172.21.15.71:0/381643419 shutdown_connections 2023-10-26T05:54:50.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 --2- 172.21.15.71:0/381643419 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab400cbe00 0x7fab4013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 --2- 172.21.15.71:0/381643419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab4013ed90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 --2- 172.21.15.71:0/381643419 >> v2:172.21.15.86:3300/0 conn(0x7fab4013c120 0x7fab400cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 -- 172.21.15.71:0/381643419 >> 172.21.15.71:0/381643419 conn(0x7fab400aec60 msgr2=0x7fab400af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:50.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 -- 172.21.15.71:0/381643419 shutdown_connections 2023-10-26T05:54:50.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 -- 172.21.15.71:0/381643419 wait complete. 2023-10-26T05:54:50.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 Processor -- start 2023-10-26T05:54:50.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.267+0000 7fab46d85640 1 -- start start 2023-10-26T05:54:50.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab400cbe00 0x7fab4014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:50.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab4013c120 0x7fab400adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:50.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab400ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:50.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40139f50 con 0x7fab4015a3f0 2023-10-26T05:54:50.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40135e40 con 0x7fab400cbe00 2023-10-26T05:54:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40136120 con 0x7fab4013c120 2023-10-26T05:54:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab400ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab400ae520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56172/0 (socket says 172.21.15.71:56172) 2023-10-26T05:54:50.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 -- 172.21.15.71:0/2144551387 learned_addr learned my addr 172.21.15.71:0/2144551387 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:54:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab44b21640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab400cbe00 0x7fab4014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3ffff640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab4013c120 0x7fab400adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 -- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab4013c120 msgr2=0x7fab400adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab4013c120 0x7fab400adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 -- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab400cbe00 msgr2=0x7fab4014ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab400cbe00 0x7fab4014ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 -- 172.21.15.71:0/2144551387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380098e0 con 0x7fab4015a3f0 2023-10-26T05:54:50.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab44b21640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab400cbe00 0x7fab4014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:54:50.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab45322640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab400ae520 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fab3801b3e0 tx=0x7fab38003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:50.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380adab0 con 0x7fab4015a3f0 2023-10-26T05:54:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fab380add00 con 0x7fab4015a3f0 2023-10-26T05:54:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab40136400 con 0x7fab4015a3f0 2023-10-26T05:54:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380b6690 con 0x7fab4015a3f0 2023-10-26T05:54:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4014f160 con 0x7fab4015a3f0 2023-10-26T05:54:50.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab40066b10 con 0x7fab4015a3f0 2023-10-26T05:54:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fab380b4040 con 0x7fab4015a3f0 2023-10-26T05:54:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3d7fa640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fab2c0664a0 0x7fab2c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:54:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab3811ef40 con 0x7fab4015a3f0 2023-10-26T05:54:50.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab44b21640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fab2c0664a0 0x7fab2c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:54:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.271+0000 7fab44b21640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fab2c0664a0 0x7fab2c068960 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fab30005fd0 tx=0x7fab30005dc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:54:50.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.275+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fab380ec320 con 0x7fab4015a3f0 2023-10-26T05:54:50.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.431+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fab400721e0 con 0x7fab4015a3f0 2023-10-26T05:54:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab3d7fa640 1 -- 172.21.15.71:0/2144551387 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v156) v1 ==== 181+0+0 (secure 0 0 0) 0x7fab380b2020 con 0x7fab4015a3f0 2023-10-26T05:54:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T05:54:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fab2c0664a0 msgr2=0x7fab2c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fab2c0664a0 0x7fab2c068960 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fab30005fd0 tx=0x7fab30005dc0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 msgr2=0x7fab400ae520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:54:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab400ae520 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fab3801b3e0 tx=0x7fab38003b70 comp rx=0 tx=0).stop 2023-10-26T05:54:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 shutdown_connections 2023-10-26T05:54:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fab2c0664a0 0x7fab2c068960 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab4015a3f0 0x7fab400ae520 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fab4013c120 0x7fab400adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 --2- 172.21.15.71:0/2144551387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab400cbe00 0x7fab4014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:54:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.527+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 >> 172.21.15.71:0/2144551387 conn(0x7fab400aec60 msgr2=0x7fab40137ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:54:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.531+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 shutdown_connections 2023-10-26T05:54:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:54:50.531+0000 7fab46d85640 1 -- 172.21.15.71:0/2144551387 wait complete. 2023-10-26T05:54:50.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:54:50.690 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:54:50.690 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:54:50.691 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T05:54:50.817 INFO:teuthology.orchestra.run.smithi071.stdout:{"mon_tick_interval":"5"} 2023-10-26T05:54:50.818 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T05:54:50.818 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-26T05:54:50.819 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:54:50.819 DEBUG:teuthology.orchestra.run.smithi071:> import time 2023-10-26T05:54:50.819 DEBUG:teuthology.orchestra.run.smithi071:> import datetime 2023-10-26T05:54:50.820 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:54:50.820 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:54:50.820 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.821 DEBUG:teuthology.orchestra.run.smithi071:> # Write some buffered data through before going full, all should be well 2023-10-26T05:54:50.821 DEBUG:teuthology.orchestra.run.smithi071:> print("writing some data through which we expect to succeed") 2023-10-26T05:54:50.821 DEBUG:teuthology.orchestra.run.smithi071:> bytes = 0 2023-10-26T05:54:50.822 DEBUG:teuthology.orchestra.run.smithi071:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T05:54:50.822 DEBUG:teuthology.orchestra.run.smithi071:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T05:54:50.822 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-10-26T05:54:50.823 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:54:50.823 DEBUG:teuthology.orchestra.run.smithi071:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T05:54:50.823 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.823 DEBUG:teuthology.orchestra.run.smithi071:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T05:54:50.824 DEBUG:teuthology.orchestra.run.smithi071:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T05:54:50.824 DEBUG:teuthology.orchestra.run.smithi071:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T05:54:50.825 DEBUG:teuthology.orchestra.run.smithi071:> # from write 2023-10-26T05:54:50.825 DEBUG:teuthology.orchestra.run.smithi071:> full = False 2023-10-26T05:54:50.825 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.825 DEBUG:teuthology.orchestra.run.smithi071:> for n in range(0, int(32 * 0.9)): 2023-10-26T05:54:50.826 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T05:54:50.826 DEBUG:teuthology.orchestra.run.smithi071:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-10-26T05:54:50.827 DEBUG:teuthology.orchestra.run.smithi071:> print("done writing {0} bytes".format(bytes)) 2023-10-26T05:54:50.827 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.827 DEBUG:teuthology.orchestra.run.smithi071:> # OK, now we should sneak in under the full condition 2023-10-26T05:54:50.827 DEBUG:teuthology.orchestra.run.smithi071:> # due to the time it takes the OSDs to report to the 2023-10-26T05:54:50.828 DEBUG:teuthology.orchestra.run.smithi071:> # mons, and get a successful fsync on our full-making data 2023-10-26T05:54:50.828 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:54:50.828 DEBUG:teuthology.orchestra.run.smithi071:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-10-26T05:54:50.829 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.829 DEBUG:teuthology.orchestra.run.smithi071:> # buffered write, add more dirty data to the buffer 2023-10-26T05:54:50.829 DEBUG:teuthology.orchestra.run.smithi071:> print("starting buffered write") 2023-10-26T05:54:50.830 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:54:50.830 DEBUG:teuthology.orchestra.run.smithi071:> for n in range(0, int(32 * 0.2)): 2023-10-26T05:54:50.830 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T05:54:50.831 DEBUG:teuthology.orchestra.run.smithi071:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T05:54:50.831 DEBUG:teuthology.orchestra.run.smithi071:> time.sleep(15.0) 2023-10-26T05:54:50.831 DEBUG:teuthology.orchestra.run.smithi071:> except OSError: 2023-10-26T05:54:50.832 DEBUG:teuthology.orchestra.run.smithi071:> pass; 2023-10-26T05:54:50.832 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.832 DEBUG:teuthology.orchestra.run.smithi071:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-10-26T05:54:50.833 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.833 DEBUG:teuthology.orchestra.run.smithi071:> # Wait long enough for a background flush that should fail 2023-10-26T05:54:50.833 DEBUG:teuthology.orchestra.run.smithi071:> time.sleep(30) 2023-10-26T05:54:50.834 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.834 DEBUG:teuthology.orchestra.run.smithi071:> if True: 2023-10-26T05:54:50.834 DEBUG:teuthology.orchestra.run.smithi071:> # ...and check that the failed background flush is reflected in fclose 2023-10-26T05:54:50.835 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:54:50.835 DEBUG:teuthology.orchestra.run.smithi071:> os.close(f) 2023-10-26T05:54:50.835 DEBUG:teuthology.orchestra.run.smithi071:> except OSError: 2023-10-26T05:54:50.836 DEBUG:teuthology.orchestra.run.smithi071:> print("close() returned an error as expected") 2023-10-26T05:54:50.836 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:54:50.836 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("close() failed to raise error") 2023-10-26T05:54:50.837 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:54:50.837 DEBUG:teuthology.orchestra.run.smithi071:> # The kernel cephfs client does not raise errors on fclose 2023-10-26T05:54:50.837 DEBUG:teuthology.orchestra.run.smithi071:> os.close(f) 2023-10-26T05:54:50.837 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:54:50.838 DEBUG:teuthology.orchestra.run.smithi071:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T05:54:50.838 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:55:16.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:55:16.294 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:55:46.388 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:55:46.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:56:16.551 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:56:16.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:56:40.547 INFO:teuthology.orchestra.run.smithi071.stdout:writing some data through which we expect to succeed 2023-10-26T05:56:40.547 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T05:56:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-10-26T05:56:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-10-26T05:56:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-10-26T05:56:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-10-26T05:56:40.548 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-10-26T05:56:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-10-26T05:56:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-10-26T05:56:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-10-26T05:56:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-10-26T05:56:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-10-26T05:56:40.549 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-10-26T05:56:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-10-26T05:56:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-10-26T05:56:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-10-26T05:56:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-10-26T05:56:40.550 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-10-26T05:56:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-10-26T05:56:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-10-26T05:56:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-10-26T05:56:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-10-26T05:56:40.551 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-10-26T05:56:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-10-26T05:56:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-10-26T05:56:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-10-26T05:56:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-10-26T05:56:40.552 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-10-26T05:56:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-10-26T05:56:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-10-26T05:56:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:done writing 29884416 bytes 2023-10-26T05:56:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:successfully fsync'ed prior to getting full state reported 2023-10-26T05:56:40.553 INFO:teuthology.orchestra.run.smithi071.stdout:starting buffered write 2023-10-26T05:56:40.554 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:56:40.554 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:56:40.554 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:56:40.554 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:56:40.555 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:56:40.555 INFO:teuthology.orchestra.run.smithi071.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-10-26T05:56:40.555 INFO:teuthology.orchestra.run.smithi071.stdout:close() returned an error as expected 2023-10-26T05:56:40.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:56:40.573 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:56:40.574 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:56:40.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:56:40.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:56:40.658 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:56:40.658 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:40.664 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:40.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- 172.21.15.71:0/1110626414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 msgr2=0x7f062c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:40.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 --2- 172.21.15.71:0/1110626414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f062c005190 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f063800c6c0 tx=0x7f063801fb40 comp rx=0 tx=0).stop 2023-10-26T05:56:40.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f06462d7640 1 -- 172.21.15.71:0/1110626414 reap_dead start 2023-10-26T05:56:40.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- 172.21.15.71:0/1110626414 shutdown_connections 2023-10-26T05:56:40.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- 172.21.15.71:0/1110626414 >> 172.21.15.71:0/1110626414 conn(0x7f06400b0c80 msgr2=0x7f06400b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:40.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- 172.21.15.71:0/1110626414 shutdown_connections 2023-10-26T05:56:40.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- 172.21.15.71:0/1110626414 wait complete. 2023-10-26T05:56:40.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 Processor -- start 2023-10-26T05:56:40.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- start start 2023-10-26T05:56:40.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f06401e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:40.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064006cc00 0x7f0640151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:40.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06400c7190 0x7f06400c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:40.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06400c7b50 con 0x7f062c004d10 2023-10-26T05:56:40.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401e5320 con 0x7f06400c7190 2023-10-26T05:56:40.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0647d3a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06401e4710 con 0x7f064006cc00 2023-10-26T05:56:40.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0645ad6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f06401e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:40.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0645ad6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f06401e4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51966/0 (socket says 172.21.15.71:51966) 2023-10-26T05:56:40.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f0645ad6640 1 -- 172.21.15.71:0/117477082 learned_addr learned my addr 172.21.15.71:0/117477082 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:40.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.831+0000 7f06452d5640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064006cc00 0x7f0640151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:40.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06462d7640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06400c7190 0x7f06400c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:40.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0645ad6640 1 -- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064006cc00 msgr2=0x7f0640151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:40.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0645ad6640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064006cc00 0x7f0640151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:40.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0645ad6640 1 -- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06400c7190 msgr2=0x7f06400c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:40.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0645ad6640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06400c7190 0x7f06400c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:40.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0645ad6640 1 -- 172.21.15.71:0/117477082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06401e49f0 con 0x7f062c004d10 2023-10-26T05:56:40.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0645ad6640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f06401e4080 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f063c011a50 tx=0x7f063c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:40.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00fd90 con 0x7f062c004d10 2023-10-26T05:56:40.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f063c012450 con 0x7f062c004d10 2023-10-26T05:56:40.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c012800 con 0x7f062c004d10 2023-10-26T05:56:40.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06401e4cd0 con 0x7f062c004d10 2023-10-26T05:56:40.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064006d080 con 0x7f062c004d10 2023-10-26T05:56:40.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0640066b10 con 0x7f062c004d10 2023-10-26T05:56:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f063c012960 con 0x7f062c004d10 2023-10-26T05:56:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0624066470 0x7f0624068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f063c085610 con 0x7f062c004d10 2023-10-26T05:56:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f062406e060 con 0x7f062c004d10 2023-10-26T05:56:40.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06452d5640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0624066470 0x7f0624068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:40.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.835+0000 7f06452d5640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0624066470 0x7f0624068930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f06400b0a60 tx=0x7f06300091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:40.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.839+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f063c01f060 con 0x7f062c004d10 2023-10-26T05:56:41.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.995+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06400b1fd0 con 0x7f062c004d10 2023-10-26T05:56:41.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f06367fc640 1 -- 172.21.15.71:0/117477082 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16619 (secure 0 0 0) 0x7f063c029070 con 0x7f062c004d10 2023-10-26T05:56:41.006 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:41.007 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":159,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:55:52.244888+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.3a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:41.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0624066470 msgr2=0x7f0624068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0624066470 0x7f0624068930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f06400b0a60 tx=0x7f06300091e0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 msgr2=0x7f06401e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f06401e4080 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f063c011a50 tx=0x7f063c011f10 comp rx=0 tx=0).stop 2023-10-26T05:56:41.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 shutdown_connections 2023-10-26T05:56:41.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06400c7190 0x7f06400c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f064006cc00 0x7f0640151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0624066470 0x7f0624068930 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 --2- 172.21.15.71:0/117477082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c004d10 0x7f06401e4080 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 >> 172.21.15.71:0/117477082 conn(0x7f06400b0c80 msgr2=0x7f06400b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 shutdown_connections 2023-10-26T05:56:41.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:40.999+0000 7f0647d3a640 1 -- 172.21.15.71:0/117477082 wait complete. 2023-10-26T05:56:41.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:54:32.632697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:54:33.544620+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:56:41.026 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:56:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- 172.21.15.71:0/1380114623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd22c004d10 msgr2=0x7fd22c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1380114623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd22c004d10 0x7fd22c005190 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00c590 tx=0x7fd23c01d310 comp rx=0 tx=0).stop 2023-10-26T05:56:41.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd242d9d640 1 -- 172.21.15.71:0/1380114623 reap_dead start 2023-10-26T05:56:41.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- 172.21.15.71:0/1380114623 shutdown_connections 2023-10-26T05:56:41.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- 172.21.15.71:0/1380114623 >> 172.21.15.71:0/1380114623 conn(0x7fd2440cdd80 msgr2=0x7fd2440ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- 172.21.15.71:0/1380114623 shutdown_connections 2023-10-26T05:56:41.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- 172.21.15.71:0/1380114623 wait complete. 2023-10-26T05:56:41.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 Processor -- start 2023-10-26T05:56:41.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- start start 2023-10-26T05:56:41.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd22c004d10 0x7fd244166a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd244156f40 0x7fd2441573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 0x7fd24414b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24406a650 con 0x7fd24414ae50 2023-10-26T05:56:41.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24414b810 con 0x7fd22c004d10 2023-10-26T05:56:41.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd248fb6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441665a0 con 0x7fd244156f40 2023-10-26T05:56:41.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd242d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 0x7fd24414b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd242d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 0x7fd24414b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51978/0 (socket says 172.21.15.71:51978) 2023-10-26T05:56:41.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.143+0000 7fd242d9d640 1 -- 172.21.15.71:0/1861398392 learned_addr learned my addr 172.21.15.71:0/1861398392 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:41.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd24259c640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd22c004d10 0x7fd244166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd241d9b640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd244156f40 0x7fd2441573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd242d9d640 1 -- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd244156f40 msgr2=0x7fd2441573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd242d9d640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd244156f40 0x7fd2441573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd242d9d640 1 -- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd22c004d10 msgr2=0x7fd244166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd242d9d640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd22c004d10 0x7fd244166a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd242d9d640 1 -- 172.21.15.71:0/1861398392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244160050 con 0x7fd24414ae50 2023-10-26T05:56:41.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd242d9d640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 0x7fd24414b2d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd23c01e9f0 tx=0x7fd23c01d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:41.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0b1070 con 0x7fd24414ae50 2023-10-26T05:56:41.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd23c0061c0 con 0x7fd24414ae50 2023-10-26T05:56:41.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2441656b0 con 0x7fd24414ae50 2023-10-26T05:56:41.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c006570 con 0x7fd24414ae50 2023-10-26T05:56:41.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244165c70 con 0x7fd24414ae50 2023-10-26T05:56:41.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd244066b10 con 0x7fd24414ae50 2023-10-26T05:56:41.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd23c0066d0 con 0x7fd24414ae50 2023-10-26T05:56:41.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd224066470 0x7fd224068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd23c0b4040 con 0x7fd24414ae50 2023-10-26T05:56:41.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fd22406e060 con 0x7fd24414ae50 2023-10-26T05:56:41.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.147+0000 7fd24259c640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd224066470 0x7fd224068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.151+0000 7fd24259c640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd224066470 0x7fd224068930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd234006590 tx=0x7fd234006b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:41.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.151+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd23c0eb850 con 0x7fd24414ae50 2023-10-26T05:56:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd244166730 con 0x7fd24414ae50 2023-10-26T05:56:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd232ffd640 1 -- 172.21.15.71:0/1861398392 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd23c006980 con 0x7fd24414ae50 2023-10-26T05:56:41.320 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:56:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd224066470 msgr2=0x7fd224068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd224066470 0x7fd224068930 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd234006590 tx=0x7fd234006b60 comp rx=0 tx=0).stop 2023-10-26T05:56:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 msgr2=0x7fd24414b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 0x7fd24414b2d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd23c01e9f0 tx=0x7fd23c01d580 comp rx=0 tx=0).stop 2023-10-26T05:56:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 shutdown_connections 2023-10-26T05:56:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd224066470 0x7fd224068930 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd24414ae50 0x7fd24414b2d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd244156f40 0x7fd2441573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 --2- 172.21.15.71:0/1861398392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd22c004d10 0x7fd244166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 >> 172.21.15.71:0/1861398392 conn(0x7fd2440cdd80 msgr2=0x7fd244164570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 shutdown_connections 2023-10-26T05:56:41.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.311+0000 7fd248fb6640 1 -- 172.21.15.71:0/1861398392 wait complete. 2023-10-26T05:56:41.339 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:56:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- 172.21.15.71:0/3854277188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 msgr2=0x7fa23c069d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 --2- 172.21.15.71:0/3854277188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c069d30 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa234009c80 tx=0x7fa23401cd50 comp rx=0 tx=0).stop 2023-10-26T05:56:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- 172.21.15.71:0/3854277188 shutdown_connections 2023-10-26T05:56:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 --2- 172.21.15.71:0/3854277188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c069d30 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 --2- 172.21.15.71:0/3854277188 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa23c15d3d0 0x7fa23c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- 172.21.15.71:0/3854277188 >> 172.21.15.71:0/3854277188 conn(0x7fa23c0aec60 msgr2=0x7fa23c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- 172.21.15.71:0/3854277188 shutdown_connections 2023-10-26T05:56:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- 172.21.15.71:0/3854277188 wait complete. 2023-10-26T05:56:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 Processor -- start 2023-10-26T05:56:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- start start 2023-10-26T05:56:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c13c220 0x7fa23c1354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa23c15d3d0 0x7fa23c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c139f90 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c13f640 con 0x7fa23c13c220 2023-10-26T05:56:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa243faa640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c13dc60 con 0x7fa23c15d3d0 2023-10-26T05:56:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.451+0000 7fa241d46640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c13f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52006/0 (socket says 172.21.15.71:52006) 2023-10-26T05:56:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 -- 172.21.15.71:0/811794994 learned_addr learned my addr 172.21.15.71:0/811794994 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:41.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 -- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa23c15d3d0 msgr2=0x7fa23c0b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa23c15d3d0 0x7fa23c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 -- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c13c220 msgr2=0x7fa23c1354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c13c220 0x7fa23c1354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 -- 172.21.15.71:0/811794994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340098e0 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241d46640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c13f100 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fa23800c500 tx=0x7fa23800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:41.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238005220 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa238005470 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238007290 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c13d3c0 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c13d980 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c066b10 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa2380073f0 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa220066470 0x7fa220068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa238083a70 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fa22006e060 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241545640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa220066470 0x7fa220068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.455+0000 7fa241545640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa220066470 0x7fa220068930 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa23c13cb90 tx=0x7fa22c005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.459+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa238050ec0 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.611+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa23c139d60 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.611+0000 7fa2327fc640 1 -- 172.21.15.71:0/811794994 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa2380544e0 con 0x7fa23c0cbcd0 2023-10-26T05:56:41.622 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:56:41.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa220066470 msgr2=0x7fa220068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa220066470 0x7fa220068930 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa23c13cb90 tx=0x7fa22c005be0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 msgr2=0x7fa23c13f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c13f100 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fa23800c500 tx=0x7fa23800c9c0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 shutdown_connections 2023-10-26T05:56:41.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa23c15d3d0 0x7fa23c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa23c13c220 0x7fa23c1354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa220066470 0x7fa220068930 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 --2- 172.21.15.71:0/811794994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa23c0cbcd0 0x7fa23c13f100 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 >> 172.21.15.71:0/811794994 conn(0x7fa23c0aec60 msgr2=0x7fa23c15a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 shutdown_connections 2023-10-26T05:56:41.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.615+0000 7fa243faa640 1 -- 172.21.15.71:0/811794994 wait complete. 2023-10-26T05:56:41.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:56:41.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- 172.21.15.71:0/2708941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0470 msgr2=0x7f96dc06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 --2- 172.21.15.71:0/2708941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0470 0x7f96dc06aa70 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009930 tx=0x7f96cc01c730 comp rx=0 tx=0).stop 2023-10-26T05:56:41.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- 172.21.15.71:0/2708941053 shutdown_connections 2023-10-26T05:56:41.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 --2- 172.21.15.71:0/2708941053 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f96dc0d0850 0x7f96dc15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 --2- 172.21.15.71:0/2708941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0470 0x7f96dc06aa70 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- 172.21.15.71:0/2708941053 >> 172.21.15.71:0/2708941053 conn(0x7f96dc0cdd80 msgr2=0x7f96dc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- 172.21.15.71:0/2708941053 shutdown_connections 2023-10-26T05:56:41.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- 172.21.15.71:0/2708941053 wait complete. 2023-10-26T05:56:41.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 Processor -- start 2023-10-26T05:56:41.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- start start 2023-10-26T05:56:41.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96dc0cd880 0x7f96dc157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f96dc0d0470 0x7f96dc156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 0x7f96dc0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc156de0 con 0x7f96dc0d0850 2023-10-26T05:56:41.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc15dea0 con 0x7f96dc0cd880 2023-10-26T05:56:41.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96e109b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1570a0 con 0x7f96dc0d0470 2023-10-26T05:56:41.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96db59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 0x7f96dc0cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96db59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 0x7f96dc0cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49462/0 (socket says 172.21.15.71:49462) 2023-10-26T05:56:41.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.747+0000 7f96db59e640 1 -- 172.21.15.71:0/1820548681 learned_addr learned my addr 172.21.15.71:0/1820548681 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:41.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96dad9d640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96dc0cd880 0x7f96dc157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96da59c640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f96dc0d0470 0x7f96dc156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96db59e640 1 -- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f96dc0d0470 msgr2=0x7f96dc156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96db59e640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f96dc0d0470 0x7f96dc156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96db59e640 1 -- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96dc0cd880 msgr2=0x7f96dc157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96db59e640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96dc0cd880 0x7f96dc157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96db59e640 1 -- 172.21.15.71:0/1820548681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009560 con 0x7f96dc0d0850 2023-10-26T05:56:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96db59e640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 0x7f96dc0cb930 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c780 tx=0x7f96d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003d90 con 0x7f96dc0d0850 2023-10-26T05:56:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96d400ce90 con 0x7f96dc0d0850 2023-10-26T05:56:41.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc157380 con 0x7f96dc0d0850 2023-10-26T05:56:41.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40052d0 con 0x7f96dc0d0850 2023-10-26T05:56:41.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc1580c0 con 0x7f96dc0d0850 2023-10-26T05:56:41.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc066b10 con 0x7f96dc0d0850 2023-10-26T05:56:41.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96d4005430 con 0x7f96dc0d0850 2023-10-26T05:56:41.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f96c8066470 0x7f96c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:41.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96d40839c0 con 0x7f96dc0d0850 2023-10-26T05:56:41.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f96c806e060 con 0x7f96dc0d0850 2023-10-26T05:56:41.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96dad9d640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f96c8066470 0x7f96c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:41.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.751+0000 7f96dad9d640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f96c8066470 0x7f96c8068930 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0af000 tx=0x7f96cc01af70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:41.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.755+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96d4017070 con 0x7f96dc0d0850 2023-10-26T05:56:41.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.903+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f96dc0721e0 con 0x7f96dc0d0850 2023-10-26T05:56:41.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.903+0000 7f96c37fe640 1 -- 172.21.15.71:0/1820548681 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96d4054880 con 0x7f96dc0d0850 2023-10-26T05:56:41.913 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:56:41.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f96c8066470 msgr2=0x7f96c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f96c8066470 0x7f96c8068930 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0af000 tx=0x7f96cc01af70 comp rx=0 tx=0).stop 2023-10-26T05:56:41.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 msgr2=0x7f96dc0cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:41.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 0x7f96dc0cb930 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c780 tx=0x7f96d400cc40 comp rx=0 tx=0).stop 2023-10-26T05:56:41.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 shutdown_connections 2023-10-26T05:56:41.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f96c8066470 0x7f96c8068930 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96dc0d0850 0x7f96dc0cb930 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f96dc0d0470 0x7f96dc156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 --2- 172.21.15.71:0/1820548681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96dc0cd880 0x7f96dc157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:41.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 >> 172.21.15.71:0/1820548681 conn(0x7f96dc0cdd80 msgr2=0x7f96dc0c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:41.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 shutdown_connections 2023-10-26T05:56:41.916 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:41.907+0000 7f96e109b640 1 -- 172.21.15.71:0/1820548681 wait complete. 2023-10-26T05:56:41.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:56:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- 172.21.15.71:0/1159882761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa62006a000 msgr2=0x7fa62006c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- 172.21.15.71:0/1159882761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa62006a000 0x7fa62006c0a0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa6200716c0 tx=0x7fa61401c870 comp rx=0 tx=0).stop 2023-10-26T05:56:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- 172.21.15.71:0/1159882761 shutdown_connections 2023-10-26T05:56:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- 172.21.15.71:0/1159882761 >> v2:172.21.15.86:3300/0 conn(0x7fa6201596d0 0x7fa620159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- 172.21.15.71:0/1159882761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa62006a000 0x7fa62006c0a0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- 172.21.15.71:0/1159882761 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa62015ccc0 0x7fa62015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- 172.21.15.71:0/1159882761 >> 172.21.15.71:0/1159882761 conn(0x7fa6200afeb0 msgr2=0x7fa6200b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- 172.21.15.71:0/1159882761 shutdown_connections 2023-10-26T05:56:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- 172.21.15.71:0/1159882761 wait complete. 2023-10-26T05:56:42.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 Processor -- start 2023-10-26T05:56:42.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- start start 2023-10-26T05:56:42.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62006a000 0x7fa620153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 0x7fa620146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa62015ccc0 0x7fa62014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201538c0 con 0x7fa6201596d0 2023-10-26T05:56:42.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa620151170 con 0x7fa62006a000 2023-10-26T05:56:42.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa62830f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa620151450 con 0x7fa62015ccc0 2023-10-26T05:56:42.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 0x7fa620146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 0x7fa620146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49486/0 (socket says 172.21.15.71:49486) 2023-10-26T05:56:42.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 -- 172.21.15.71:0/1241813349 learned_addr learned my addr 172.21.15.71:0/1241813349 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:42.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6260ab640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62006a000 0x7fa620153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 -- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa62015ccc0 msgr2=0x7fa62014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa62015ccc0 0x7fa62014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 -- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62006a000 msgr2=0x7fa620153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62006a000 0x7fa620153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.035+0000 7fa6258aa640 1 -- 172.21.15.71:0/1241813349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa614009560 con 0x7fa6201596d0 2023-10-26T05:56:42.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa6258aa640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 0x7fa620146570 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fa614009850 tx=0x7fa6140a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:42.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614005220 con 0x7fa6201596d0 2023-10-26T05:56:42.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa614005380 con 0x7fa6201596d0 2023-10-26T05:56:42.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140046d0 con 0x7fa6201596d0 2023-10-26T05:56:42.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa620151730 con 0x7fa6201596d0 2023-10-26T05:56:42.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa620151cf0 con 0x7fa6201596d0 2023-10-26T05:56:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa620151ee0 con 0x7fa6201596d0 2023-10-26T05:56:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa6140b4040 con 0x7fa6201596d0 2023-10-26T05:56:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa60c066470 0x7fa60c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa61411d320 con 0x7fa6201596d0 2023-10-26T05:56:42.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fa60c06e060 con 0x7fa6201596d0 2023-10-26T05:56:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa6260ab640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa60c066470 0x7fa60c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.039+0000 7fa6260ab640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa60c066470 0x7fa60c068930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa6180058d0 tx=0x7fa618005b70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:42.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.043+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa6140ea7f0 con 0x7fa6201596d0 2023-10-26T05:56:42.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.207+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa620069770 con 0x7fa6201596d0 2023-10-26T05:56:42.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.207+0000 7fa61effd640 1 -- 172.21.15.71:0/1241813349 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v159) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa6140b2030 con 0x7fa6201596d0 2023-10-26T05:56:42.216 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:56:42.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa60c066470 msgr2=0x7fa60c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa60c066470 0x7fa60c068930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa6180058d0 tx=0x7fa618005b70 comp rx=0 tx=0).stop 2023-10-26T05:56:42.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 msgr2=0x7fa620146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 0x7fa620146570 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fa614009850 tx=0x7fa6140a5770 comp rx=0 tx=0).stop 2023-10-26T05:56:42.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 shutdown_connections 2023-10-26T05:56:42.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa62015ccc0 0x7fa62014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa60c066470 0x7fa60c068930 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6201596d0 0x7fa620146570 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 --2- 172.21.15.71:0/1241813349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa62006a000 0x7fa620153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 >> 172.21.15.71:0/1241813349 conn(0x7fa6200afeb0 msgr2=0x7fa6200ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:42.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 shutdown_connections 2023-10-26T05:56:42.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.211+0000 7fa62830f640 1 -- 172.21.15.71:0/1241813349 wait complete. 2023-10-26T05:56:42.234 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:56:42.234 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:56:42.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:56:42.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:56:42.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[51380]: fuse finished with error 0 and tester_r 0 2023-10-26T05:56:42.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:56:42.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:56:42.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:56:42.286 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:56:42.290 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:56:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:56:42.299 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:56:42.299 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:42.306 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:42.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/3004075475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 msgr2=0x7f4f280ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/3004075475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f280ac740 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f4f28155a00 tx=0x7f4f1801db10 comp rx=0 tx=0).stop 2023-10-26T05:56:42.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/3004075475 shutdown_connections 2023-10-26T05:56:42.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/3004075475 >> v2:172.21.15.86:3300/0 conn(0x7f4f2813d800 0x7f4f2815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/3004075475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f280ac740 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/3004075475 >> 172.21.15.71:0/3004075475 conn(0x7f4f28069770 msgr2=0x7f4f28069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:42.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/3004075475 shutdown_connections 2023-10-26T05:56:42.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/3004075475 wait complete. 2023-10-26T05:56:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 Processor -- start 2023-10-26T05:56:42.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- start start 2023-10-26T05:56:42.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f28136250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f2813d800 0x7f4f2814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f2813dc50 0x7f4f2813b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28150d30 con 0x7f4f2813d420 2023-10-26T05:56:42.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28150120 con 0x7f4f2813dc50 2023-10-26T05:56:42.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f2dfb5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28150400 con 0x7f4f2813d800 2023-10-26T05:56:42.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f28136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f277fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f28136250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49512/0 (socket says 172.21.15.71:49512) 2023-10-26T05:56:42.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.451+0000 7f4f277fe640 1 -- 172.21.15.71:0/1448181026 learned_addr learned my addr 172.21.15.71:0/1448181026 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:42.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f277fe640 1 -- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f2813d800 msgr2=0x7f4f2814da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f277fe640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f2813d800 0x7f4f2814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f277fe640 1 -- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f2813dc50 msgr2=0x7f4f2813b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f277fe640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f2813dc50 0x7f4f2813b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f277fe640 1 -- 172.21.15.71:0/1448181026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1800a5b0 con 0x7f4f2813d420 2023-10-26T05:56:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f277fe640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f28136250 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f4f1801b740 tx=0x7f4f18005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18004ae0 con 0x7f4f2813d420 2023-10-26T05:56:42.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4f18004d30 con 0x7f4f2813d420 2023-10-26T05:56:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180b7700 con 0x7f4f2813d420 2023-10-26T05:56:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f281506e0 con 0x7f4f2813d420 2023-10-26T05:56:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f28136790 con 0x7f4f2813d420 2023-10-26T05:56:42.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f28066b10 con 0x7f4f2813d420 2023-10-26T05:56:42.469 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4f180b7860 con 0x7f4f2813d420 2023-10-26T05:56:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4f14066470 0x7f4f14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4f181215b0 con 0x7f4f2813d420 2023-10-26T05:56:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f4f1406e060 con 0x7f4f2813d420 2023-10-26T05:56:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f26ffd640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4f14066470 0x7f4f14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.455+0000 7f4f26ffd640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4f14066470 0x7f4f14068930 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4f2815a870 tx=0x7f4f1c006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:42.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.459+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4f180b3020 con 0x7f4f2813d420 2023-10-26T05:56:42.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.619+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f280721e0 con 0x7f4f2813d420 2023-10-26T05:56:42.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f0ffff640 1 -- 172.21.15.71:0/1448181026 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16619 (secure 0 0 0) 0x7f4f180eeb60 con 0x7f4f2813d420 2023-10-26T05:56:42.631 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:42.631 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":159,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:55:52.244888+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.3a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4f14066470 msgr2=0x7f4f14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4f14066470 0x7f4f14068930 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4f2815a870 tx=0x7f4f1c006ca0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 msgr2=0x7f4f28136250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f28136250 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f4f1801b740 tx=0x7f4f18005740 comp rx=0 tx=0).stop 2023-10-26T05:56:42.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 shutdown_connections 2023-10-26T05:56:42.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f2813dc50 0x7f4f2813b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4f2813d800 0x7f4f2814da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4f14066470 0x7f4f14068930 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 --2- 172.21.15.71:0/1448181026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f2813d420 0x7f4f28136250 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 >> 172.21.15.71:0/1448181026 conn(0x7f4f28069770 msgr2=0x7f4f280b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:42.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 shutdown_connections 2023-10-26T05:56:42.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.623+0000 7f4f2dfb5640 1 -- 172.21.15.71:0/1448181026 wait complete. 2023-10-26T05:56:42.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:54:32.632697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:54:33.544620+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:56:42.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:56:42.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/1736607383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440d0470 msgr2=0x7fd04406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- 172.21.15.71:0/1736607383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440d0470 0x7fd04406aa70 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd0440c4dd0 tx=0x7fd03801c810 comp rx=0 tx=0).stop 2023-10-26T05:56:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/1736607383 shutdown_connections 2023-10-26T05:56:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- 172.21.15.71:0/1736607383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0440cd880 0x7fd04415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- 172.21.15.71:0/1736607383 >> v2:172.21.15.86:3300/0 conn(0x7fd0440d0850 0x7fd04415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- 172.21.15.71:0/1736607383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440d0470 0x7fd04406aa70 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/1736607383 >> 172.21.15.71:0/1736607383 conn(0x7fd0440cdd80 msgr2=0x7fd0440ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:42.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/1736607383 shutdown_connections 2023-10-26T05:56:42.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/1736607383 wait complete. 2023-10-26T05:56:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 Processor -- start 2023-10-26T05:56:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- start start 2023-10-26T05:56:42.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 0x7fd044157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0440d0470 0x7fd0440cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0440d0850 0x7fd0440c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440cb5a0 con 0x7fd0440cd880 2023-10-26T05:56:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044157c80 con 0x7fd0440d0470 2023-10-26T05:56:42.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd044157f60 con 0x7fd0440d0850 2023-10-26T05:56:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 0x7fd044157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 0x7fd044157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49530/0 (socket says 172.21.15.71:49530) 2023-10-26T05:56:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 -- 172.21.15.71:0/2858273556 learned_addr learned my addr 172.21.15.71:0/2858273556 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 -- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0440d0850 msgr2=0x7fd0440c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0440d0850 0x7fd0440c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 -- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0440d0470 msgr2=0x7fd0440cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0440d0470 0x7fd0440cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 -- 172.21.15.71:0/2858273556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038009560 con 0x7fd0440cd880 2023-10-26T05:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd048da2640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 0x7fd044157580 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd0380b0040 tx=0x7fd0380a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:42.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038005440 con 0x7fd0440cd880 2023-10-26T05:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0380046d0 con 0x7fd0440cd880 2023-10-26T05:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04415c1f0 con 0x7fd0440cd880 2023-10-26T05:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038004ae0 con 0x7fd0440cd880 2023-10-26T05:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.763+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04415c7b0 con 0x7fd0440cd880 2023-10-26T05:56:42.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd044066b10 con 0x7fd0440cd880 2023-10-26T05:56:42.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0380b4070 con 0x7fd0440cd880 2023-10-26T05:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd0417fa640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0280664a0 0x7fd028068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd03811e870 con 0x7fd0440cd880 2023-10-26T05:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fd02806e090 con 0x7fd0440cd880 2023-10-26T05:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd043fff640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0280664a0 0x7fd028068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:42.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.767+0000 7fd043fff640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0280664a0 0x7fd028068960 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd04415a640 tx=0x7fd030005b60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:42.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.771+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0380b2060 con 0x7fd0440cd880 2023-10-26T05:56:42.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd04415c9a0 con 0x7fd0440cd880 2023-10-26T05:56:42.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd0417fa640 1 -- 172.21.15.71:0/2858273556 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0380ba030 con 0x7fd0440cd880 2023-10-26T05:56:42.931 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:56:42.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0280664a0 msgr2=0x7fd028068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0280664a0 0x7fd028068960 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd04415a640 tx=0x7fd030005b60 comp rx=0 tx=0).stop 2023-10-26T05:56:42.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 msgr2=0x7fd044157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:42.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 0x7fd044157580 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd0380b0040 tx=0x7fd0380a5870 comp rx=0 tx=0).stop 2023-10-26T05:56:42.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 shutdown_connections 2023-10-26T05:56:42.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0440d0850 0x7fd0440c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0440d0470 0x7fd0440cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0280664a0 0x7fd028068960 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 --2- 172.21.15.71:0/2858273556 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0440cd880 0x7fd044157580 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:42.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 >> 172.21.15.71:0/2858273556 conn(0x7fd0440cdd80 msgr2=0x7fd0440cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:42.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 shutdown_connections 2023-10-26T05:56:42.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:42.923+0000 7fd04b006640 1 -- 172.21.15.71:0/2858273556 wait complete. 2023-10-26T05:56:42.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:56:43.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3943662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea0070750 msgr2=0x7f5ea006a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3943662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea0070750 0x7f5ea006a1b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800acd0 tx=0x7f5e9801dea0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3943662403 shutdown_connections 2023-10-26T05:56:43.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3943662403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea0070750 0x7f5ea006a1b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3943662403 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea00cd000 0x7f5ea00b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3943662403 >> 172.21.15.71:0/3943662403 conn(0x7f5ea00af7d0 msgr2=0x7f5ea00afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:43.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3943662403 shutdown_connections 2023-10-26T05:56:43.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3943662403 wait complete. 2023-10-26T05:56:43.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 Processor -- start 2023-10-26T05:56:43.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.067+0000 7f5ea7b79640 1 -- start start 2023-10-26T05:56:43.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 0x7f5ea00a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea00b0740 0x7f5ea013cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea00cd000 0x7f5ea00a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00ad7b0 con 0x7f5ea0070750 2023-10-26T05:56:43.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b1410 con 0x7f5ea00b0740 2023-10-26T05:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00aa030 con 0x7f5ea00cd000 2023-10-26T05:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 0x7f5ea00a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 0x7f5ea00a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49532/0 (socket says 172.21.15.71:49532) 2023-10-26T05:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 -- 172.21.15.71:0/3393002923 learned_addr learned my addr 172.21.15.71:0/3393002923 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:43.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea6116640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea00cd000 0x7f5ea00a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5114640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea00b0740 0x7f5ea013cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 -- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea00cd000 msgr2=0x7f5ea00a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea00cd000 0x7f5ea00a7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 -- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea00b0740 msgr2=0x7f5ea013cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea00b0740 0x7f5ea013cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 -- 172.21.15.71:0/3393002923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9800a930 con 0x7f5ea0070750 2023-10-26T05:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5915640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 0x7f5ea00a9af0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00e7d0 tx=0x7f5e9c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c009590 con 0x7f5ea0070750 2023-10-26T05:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5e9c00cdb0 con 0x7f5ea0070750 2023-10-26T05:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00eee0 con 0x7f5ea0070750 2023-10-26T05:56:43.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea00b1940 con 0x7f5ea0070750 2023-10-26T05:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea00b1660 con 0x7f5ea0070750 2023-10-26T05:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea0066d00 con 0x7f5ea0070750 2023-10-26T05:56:43.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5e9c010690 con 0x7f5ea0070750 2023-10-26T05:56:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e840664a0 0x7f5e84068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5e9c014070 con 0x7f5ea0070750 2023-10-26T05:56:43.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f5e8406e090 con 0x7f5ea0070750 2023-10-26T05:56:43.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.071+0000 7f5ea5114640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e840664a0 0x7f5e84068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.075+0000 7f5ea5114640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e840664a0 0x7f5e84068960 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00ac4b0 tx=0x7f5e90009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:43.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.075+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5e9c050090 con 0x7f5ea0070750 2023-10-26T05:56:43.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.223+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ea00690a0 con 0x7f5ea0070750 2023-10-26T05:56:43.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.223+0000 7f5e967fc640 1 -- 172.21.15.71:0/3393002923 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5e9c055800 con 0x7f5ea0070750 2023-10-26T05:56:43.234 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:56:43.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e840664a0 msgr2=0x7f5e84068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e840664a0 0x7f5e84068960 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00ac4b0 tx=0x7f5e90009040 comp rx=0 tx=0).stop 2023-10-26T05:56:43.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 msgr2=0x7f5ea00a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 0x7f5ea00a9af0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00e7d0 tx=0x7f5e9c00ec90 comp rx=0 tx=0).stop 2023-10-26T05:56:43.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 shutdown_connections 2023-10-26T05:56:43.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5ea00cd000 0x7f5ea00a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ea00b0740 0x7f5ea013cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5e840664a0 0x7f5e84068960 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 --2- 172.21.15.71:0/3393002923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5ea0070750 0x7f5ea00a9af0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 >> 172.21.15.71:0/3393002923 conn(0x7f5ea00af7d0 msgr2=0x7f5ea006cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:43.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 shutdown_connections 2023-10-26T05:56:43.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.227+0000 7f5ea7b79640 1 -- 172.21.15.71:0/3393002923 wait complete. 2023-10-26T05:56:43.252 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:56:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/2809848780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6d80 msgr2=0x7f6dc00cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/2809848780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6d80 0x7f6dc00cf5b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009c80 tx=0x7f6db801cc80 comp rx=0 tx=0).stop 2023-10-26T05:56:43.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/2809848780 shutdown_connections 2023-10-26T05:56:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/2809848780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6d80 0x7f6dc00cf5b0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/2809848780 >> 172.21.15.71:0/2809848780 conn(0x7f6dc00d12d0 msgr2=0x7f6dc00d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/2809848780 shutdown_connections 2023-10-26T05:56:43.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/2809848780 wait complete. 2023-10-26T05:56:43.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 Processor -- start 2023-10-26T05:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- start start 2023-10-26T05:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 0x7f6dc014e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6dc00b6d80 0x7f6dc014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc015b5c0 0x7f6dc01500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00cb260 con 0x7f6dc00b6910 2023-10-26T05:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc01505f0 con 0x7f6dc015b5c0 2023-10-26T05:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00caf80 con 0x7f6dc00b6d80 2023-10-26T05:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 0x7f6dc014e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 0x7f6dc014e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49558/0 (socket says 172.21.15.71:49558) 2023-10-26T05:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 -- 172.21.15.71:0/1425837169 learned_addr learned my addr 172.21.15.71:0/1425837169 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 -- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6dc00b6d80 msgr2=0x7f6dc014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6dc00b6d80 0x7f6dc014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 -- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc015b5c0 msgr2=0x7f6dc01500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbffff640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc015b5c0 0x7f6dc01500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc015b5c0 0x7f6dc01500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 -- 172.21.15.71:0/1425837169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80098e0 con 0x7f6dc00b6910 2023-10-26T05:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dbf7fe640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 0x7f6dc014e260 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f6db000b3e0 tx=0x7f6db000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0011830 con 0x7f6dc00b6910 2023-10-26T05:56:43.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6db0011aa0 con 0x7f6dc00b6910 2023-10-26T05:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc00cdb70 con 0x7f6dc00b6910 2023-10-26T05:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0010380 con 0x7f6dc00b6910 2023-10-26T05:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.371+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc015e610 con 0x7f6dc00b6910 2023-10-26T05:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc0066b10 con 0x7f6dc00b6910 2023-10-26T05:56:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6db00104e0 con 0x7f6dc00b6910 2023-10-26T05:56:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6da7fff640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dac066470 0x7f6dac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6db0084180 con 0x7f6dc00b6910 2023-10-26T05:56:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f6dac06e060 con 0x7f6dc00b6910 2023-10-26T05:56:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6dbeffd640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dac066470 0x7f6dac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.375+0000 7f6dbeffd640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dac066470 0x7f6dac068930 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6db4000f70 tx=0x7f6db40079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:43.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.379+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6db0051650 con 0x7f6dc00b6910 2023-10-26T05:56:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6dc00721e0 con 0x7f6dc00b6910 2023-10-26T05:56:43.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6da7fff640 1 -- 172.21.15.71:0/1425837169 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6db0015070 con 0x7f6dc00b6910 2023-10-26T05:56:43.543 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:56:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dac066470 msgr2=0x7f6dac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dac066470 0x7f6dac068930 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6db4000f70 tx=0x7f6db40079c0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 msgr2=0x7f6dc014e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 0x7f6dc014e260 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f6db000b3e0 tx=0x7f6db000b8a0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 shutdown_connections 2023-10-26T05:56:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc015b5c0 0x7f6dc01500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6dc00b6d80 0x7f6dc014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6dac066470 0x7f6dac068930 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 --2- 172.21.15.71:0/1425837169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6dc00b6910 0x7f6dc014e260 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 >> 172.21.15.71:0/1425837169 conn(0x7f6dc00d12d0 msgr2=0x7f6dc00b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 shutdown_connections 2023-10-26T05:56:43.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.535+0000 7f6dc5fea640 1 -- 172.21.15.71:0/1425837169 wait complete. 2023-10-26T05:56:43.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:56:43.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2884221620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0cc730 msgr2=0x7f620c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- 172.21.15.71:0/2884221620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0cc730 0x7f620c15b3a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f620c0c9360 tx=0x7f620801dba0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2884221620 shutdown_connections 2023-10-26T05:56:43.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- 172.21.15.71:0/2884221620 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c0b15f0 0x7f620c1586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- 172.21.15.71:0/2884221620 >> v2:172.21.15.86:3300/0 conn(0x7f620c06a000 0x7f620c06a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- 172.21.15.71:0/2884221620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0cc730 0x7f620c15b3a0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2884221620 >> 172.21.15.71:0/2884221620 conn(0x7f620c06b8c0 msgr2=0x7f620c06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2884221620 shutdown_connections 2023-10-26T05:56:43.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2884221620 wait complete. 2023-10-26T05:56:43.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 Processor -- start 2023-10-26T05:56:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- start start 2023-10-26T05:56:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c06a000 0x7f620c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 0x7f620c161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f620c0cc730 0x7f620c14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c162230 con 0x7f620c0b15f0 2023-10-26T05:56:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c1f4820 con 0x7f620c0cc730 2023-10-26T05:56:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c15a470 con 0x7f620c06a000 2023-10-26T05:56:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 0x7f620c161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 0x7f620c161cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49568/0 (socket says 172.21.15.71:49568) 2023-10-26T05:56:43.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 -- 172.21.15.71:0/2518447360 learned_addr learned my addr 172.21.15.71:0/2518447360 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621290d640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f620c0cc730 0x7f620c14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 -- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c06a000 msgr2=0x7f620c159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621210c640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c06a000 0x7f620c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c06a000 0x7f620c159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 -- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f620c0cc730 msgr2=0x7f620c14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f620c0cc730 0x7f620c14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 -- 172.21.15.71:0/2518447360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620800a670 con 0x7f620c0b15f0 2023-10-26T05:56:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621290d640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f620c0cc730 0x7f620c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621210c640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c06a000 0x7f620c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:43.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f621190b640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 0x7f620c161cf0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00d9b0 tx=0x7f61fc00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc009bd0 con 0x7f620c0b15f0 2023-10-26T05:56:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f61fc009e20 con 0x7f620c0b15f0 2023-10-26T05:56:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c15a750 con 0x7f620c0b15f0 2023-10-26T05:56:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc005280 con 0x7f620c0b15f0 2023-10-26T05:56:43.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.679+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c15ad10 con 0x7f620c0b15f0 2023-10-26T05:56:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c066b10 con 0x7f620c0b15f0 2023-10-26T05:56:43.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f61fc0053e0 con 0x7f620c0b15f0 2023-10-26T05:56:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f6202ffd640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61f0066470 0x7f61f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f61fc084180 con 0x7f620c0b15f0 2023-10-26T05:56:43.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f61f006e060 con 0x7f620c0b15f0 2023-10-26T05:56:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f621210c640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61f0066470 0x7f61f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.683+0000 7f621210c640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61f0066470 0x7f61f0068930 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f62080b0000 tx=0x7f620801c3e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:43.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.687+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f61fc017060 con 0x7f620c0b15f0 2023-10-26T05:56:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.839+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f620c06c7d0 con 0x7f620c0b15f0 2023-10-26T05:56:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6202ffd640 1 -- 172.21.15.71:0/2518447360 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v159) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61fc020020 con 0x7f620c0b15f0 2023-10-26T05:56:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:56:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61f0066470 msgr2=0x7f61f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61f0066470 0x7f61f0068930 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f62080b0000 tx=0x7f620801c3e0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 msgr2=0x7f620c161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 0x7f620c161cf0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00d9b0 tx=0x7f61fc00de70 comp rx=0 tx=0).stop 2023-10-26T05:56:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 shutdown_connections 2023-10-26T05:56:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f620c0cc730 0x7f620c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f61f0066470 0x7f61f0068930 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f620c0b15f0 0x7f620c161cf0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 --2- 172.21.15.71:0/2518447360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f620c06a000 0x7f620c159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 >> 172.21.15.71:0/2518447360 conn(0x7f620c06b8c0 msgr2=0x7f620c06bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 shutdown_connections 2023-10-26T05:56:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:43.843+0000 7f6214370640 1 -- 172.21.15.71:0/2518447360 wait complete. 2023-10-26T05:56:43.867 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:56:43.868 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:56:43.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:56:43.924 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:56:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:56:43.924 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35205]: fuse finished with error 0 and tester_r 0 2023-10-26T05:56:43.925 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:56:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:56:43.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:56:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:56:43.933 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:56:43.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:56:43.934 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:56:43.940 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:56:43.940 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:43.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:56:43.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:56:43.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:56:43.950 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:56:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:56:43.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:56:43.996 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:56:43.996 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:44.001 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:56:44.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:56:44.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:56:44.003 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:56:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:56:44.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:56:44.094 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:56:44.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 -- 172.21.15.71:0/977132884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05380b2a60 msgr2=0x7f05380caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 --2- 172.21.15.71:0/977132884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05380b2a60 0x7f05380caad0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f0520009900 tx=0x7f052001c880 comp rx=0 tx=0).stop 2023-10-26T05:56:44.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 -- 172.21.15.71:0/977132884 shutdown_connections 2023-10-26T05:56:44.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 --2- 172.21.15.71:0/977132884 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05380cb010 0x7f0538159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 --2- 172.21.15.71:0/977132884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05380b2a60 0x7f05380caad0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 -- 172.21.15.71:0/977132884 >> 172.21.15.71:0/977132884 conn(0x7f0538069770 msgr2=0x7f0538069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:44.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 -- 172.21.15.71:0/977132884 shutdown_connections 2023-10-26T05:56:44.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.211+0000 7f053ebe8640 1 -- 172.21.15.71:0/977132884 wait complete. 2023-10-26T05:56:44.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 Processor -- start 2023-10-26T05:56:44.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- start start 2023-10-26T05:56:44.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05380b2a60 0x7f053815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05380cb010 0x7f053815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 0x7f0538133150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0538139940 con 0x7f053813c2f0 2023-10-26T05:56:44.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053813c820 con 0x7f05380b2a60 2023-10-26T05:56:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0538138d30 con 0x7f05380cb010 2023-10-26T05:56:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 0x7f0538133150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 0x7f0538133150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49598/0 (socket says 172.21.15.71:49598) 2023-10-26T05:56:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 -- 172.21.15.71:0/184960216 learned_addr learned my addr 172.21.15.71:0/184960216 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:44.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f0537fff640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05380cb010 0x7f053815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053c984640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05380b2a60 0x7f053815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 -- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05380cb010 msgr2=0x7f053815bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05380cb010 0x7f053815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 -- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05380b2a60 msgr2=0x7f053815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05380b2a60 0x7f053815b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 -- 172.21.15.71:0/184960216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520009560 con 0x7f053813c2f0 2023-10-26T05:56:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053d185640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 0x7f0538133150 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f053000b970 tx=0x7f053000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053000cb40 con 0x7f053813c2f0 2023-10-26T05:56:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f053000cdb0 con 0x7f053813c2f0 2023-10-26T05:56:44.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0538139010 con 0x7f053813c2f0 2023-10-26T05:56:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05300146a0 con 0x7f053813c2f0 2023-10-26T05:56:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05381395d0 con 0x7f053813c2f0 2023-10-26T05:56:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.215+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0538066b10 con 0x7f053813c2f0 2023-10-26T05:56:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.219+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f053001d3b0 con 0x7f053813c2f0 2023-10-26T05:56:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.219+0000 7f05357fa640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0518066470 0x7f0518068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.219+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0530084090 con 0x7f053813c2f0 2023-10-26T05:56:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.219+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f051806e060 con 0x7f053813c2f0 2023-10-26T05:56:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.219+0000 7f053c984640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0518066470 0x7f0518068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.219+0000 7f053c984640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0518066470 0x7f0518068930 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0538142cb0 tx=0x7f0528008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.223+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0530019020 con 0x7f053813c2f0 2023-10-26T05:56:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.395+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05380721e0 con 0x7f053813c2f0 2023-10-26T05:56:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.395+0000 7f05357fa640 1 -- 172.21.15.71:0/184960216 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f05300514c0 con 0x7f053813c2f0 2023-10-26T05:56:44.404 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:44.405 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:56:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:56:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0518066470 msgr2=0x7f0518068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0518066470 0x7f0518068930 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0538142cb0 tx=0x7f0528008040 comp rx=0 tx=0).stop 2023-10-26T05:56:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 msgr2=0x7f0538133150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 0x7f0538133150 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f053000b970 tx=0x7f053000be30 comp rx=0 tx=0).stop 2023-10-26T05:56:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 shutdown_connections 2023-10-26T05:56:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0518066470 0x7f0518068930 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f053813c2f0 0x7f0538133150 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05380cb010 0x7f053815bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 --2- 172.21.15.71:0/184960216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05380b2a60 0x7f053815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 >> 172.21.15.71:0/184960216 conn(0x7f0538069770 msgr2=0x7f05380ad990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:44.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 shutdown_connections 2023-10-26T05:56:44.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.399+0000 7f053ebe8640 1 -- 172.21.15.71:0/184960216 wait complete. 2023-10-26T05:56:44.422 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:44.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1117270330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5734070fe0 msgr2=0x7f5734071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- 172.21.15.71:0/1117270330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5734070fe0 0x7f5734071440 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f57340aecd0 tx=0x7f572801c870 comp rx=0 tx=0).stop 2023-10-26T05:56:44.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1117270330 shutdown_connections 2023-10-26T05:56:44.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- 172.21.15.71:0/1117270330 >> v2:172.21.15.86:3300/0 conn(0x7f57340b0040 0x7f57340b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- 172.21.15.71:0/1117270330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5734070fe0 0x7f5734071440 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- 172.21.15.71:0/1117270330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5734140cc0 0x7f573415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1117270330 >> 172.21.15.71:0/1117270330 conn(0x7f5734069770 msgr2=0x7f5734069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1117270330 shutdown_connections 2023-10-26T05:56:44.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1117270330 wait complete. 2023-10-26T05:56:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 Processor -- start 2023-10-26T05:56:44.551 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- start start 2023-10-26T05:56:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5734070fe0 0x7f5734157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 0x7f573406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734140cc0 0x7f57340703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340ac3b0 con 0x7f57340b0040 2023-10-26T05:56:44.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57340acd10 con 0x7f5734140cc0 2023-10-26T05:56:44.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57340abdf0 con 0x7f5734070fe0 2023-10-26T05:56:44.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 0x7f573406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 0x7f573406fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49614/0 (socket says 172.21.15.71:49614) 2023-10-26T05:56:44.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 -- 172.21.15.71:0/1885819639 learned_addr learned my addr 172.21.15.71:0/1885819639 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:44.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f573259c640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5734070fe0 0x7f5734157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5732d9d640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734140cc0 0x7f57340703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 -- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5734070fe0 msgr2=0x7f5734157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5734070fe0 0x7f5734157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 -- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734140cc0 msgr2=0x7f57340703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734140cc0 0x7f57340703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 -- 172.21.15.71:0/1885819639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728009560 con 0x7f57340b0040 2023-10-26T05:56:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5731d9b640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 0x7f573406fe80 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f5728009850 tx=0x7f57280a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728005220 con 0x7f57340b0040 2023-10-26T05:56:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5728005380 con 0x7f57340b0040 2023-10-26T05:56:44.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57340ac0d0 con 0x7f57340b0040 2023-10-26T05:56:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280046d0 con 0x7f57340b0040 2023-10-26T05:56:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.539+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57341580a0 con 0x7f57340b0040 2023-10-26T05:56:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734066b10 con 0x7f57340b0040 2023-10-26T05:56:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57280b4040 con 0x7f57340b0040 2023-10-26T05:56:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f5722ffd640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5714066470 0x7f5714068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f572811e2a0 con 0x7f57340b0040 2023-10-26T05:56:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f571406e060 con 0x7f57340b0040 2023-10-26T05:56:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f573259c640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5714066470 0x7f5714068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.543+0000 7f573259c640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5714066470 0x7f5714068930 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5724006fd0 tx=0x7f5724008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:44.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.547+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f57280eb640 con 0x7f57340b0040 2023-10-26T05:56:44.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.691+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57340acb70 con 0x7f57340b0040 2023-10-26T05:56:44.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.691+0000 7f5722ffd640 1 -- 172.21.15.71:0/1885819639 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16619 (secure 0 0 0) 0x7f57340acb70 con 0x7f57340b0040 2023-10-26T05:56:44.701 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:44.701 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":159,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:55:52.244888+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.3a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:44.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5714066470 msgr2=0x7f5714068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5714066470 0x7f5714068930 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5724006fd0 tx=0x7f5724008040 comp rx=0 tx=0).stop 2023-10-26T05:56:44.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 msgr2=0x7f573406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 0x7f573406fe80 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f5728009850 tx=0x7f57280a5770 comp rx=0 tx=0).stop 2023-10-26T05:56:44.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 shutdown_connections 2023-10-26T05:56:44.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5734140cc0 0x7f57340703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5714066470 0x7f5714068930 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57340b0040 0x7f573406fe80 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 --2- 172.21.15.71:0/1885819639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5734070fe0 0x7f5734157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 >> 172.21.15.71:0/1885819639 conn(0x7f5734069770 msgr2=0x7f57340b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:44.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 shutdown_connections 2023-10-26T05:56:44.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.695+0000 7f57389b7640 1 -- 172.21.15.71:0/1885819639 wait complete. 2023-10-26T05:56:44.719 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:54:32.632697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:54:33.544620+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:56:44.720 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:56:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- 172.21.15.71:0/3660222594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe77c0039b0 msgr2=0x7fe77c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 --2- 172.21.15.71:0/3660222594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe77c0039b0 0x7fe77c003dd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe79408a3b0 tx=0x7fe78401cec0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe793fff640 1 -- 172.21.15.71:0/3660222594 reap_dead start 2023-10-26T05:56:44.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- 172.21.15.71:0/3660222594 shutdown_connections 2023-10-26T05:56:44.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- 172.21.15.71:0/3660222594 >> 172.21.15.71:0/3660222594 conn(0x7fe7940cdd80 msgr2=0x7fe7940ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:44.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- 172.21.15.71:0/3660222594 shutdown_connections 2023-10-26T05:56:44.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- 172.21.15.71:0/3660222594 wait complete. 2023-10-26T05:56:44.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 Processor -- start 2023-10-26T05:56:44.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- start start 2023-10-26T05:56:44.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe77c0039b0 0x7fe794157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 0x7fe794146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe79415a240 0x7fe79415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79415ac00 con 0x7fe794161500 2023-10-26T05:56:44.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe794146ed0 con 0x7fe77c0039b0 2023-10-26T05:56:44.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe794161ca0 con 0x7fe79415a240 2023-10-26T05:56:44.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 0x7fe794146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 0x7fe794146990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49628/0 (socket says 172.21.15.71:49628) 2023-10-26T05:56:44.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 -- 172.21.15.71:0/2529938307 learned_addr learned my addr 172.21.15.71:0/2529938307 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:44.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe7937fe640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe77c0039b0 0x7fe794157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe793fff640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe79415a240 0x7fe79415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 -- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe79415a240 msgr2=0x7fe79415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe79415a240 0x7fe79415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 -- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe77c0039b0 msgr2=0x7fe794157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:44.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe77c0039b0 0x7fe794157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:44.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 -- 172.21.15.71:0/2529938307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794161f80 con 0x7fe794161500 2023-10-26T05:56:44.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe792ffd640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 0x7fe794146990 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe78800e990 tx=0x7fe78800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:44.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78800cbe0 con 0x7fe794161500 2023-10-26T05:56:44.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe78800ce30 con 0x7fe794161500 2023-10-26T05:56:44.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.835+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7940c8570 con 0x7fe794161500 2023-10-26T05:56:44.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788005280 con 0x7fe794161500 2023-10-26T05:56:44.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe794161980 con 0x7fe794161500 2023-10-26T05:56:44.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe794066b10 con 0x7fe794161500 2023-10-26T05:56:44.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7880053e0 con 0x7fe794161500 2023-10-26T05:56:44.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe77bfff640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe780066470 0x7fe780068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:44.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe788084850 con 0x7fe794161500 2023-10-26T05:56:44.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fe78006e060 con 0x7fe794161500 2023-10-26T05:56:44.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe7937fe640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe780066470 0x7fe780068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:44.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.839+0000 7fe7937fe640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe780066470 0x7fe780068930 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe7840ac000 tx=0x7fe7840b0040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:44.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.843+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe78801f070 con 0x7fe794161500 2023-10-26T05:56:45.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.999+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7940c83b0 con 0x7fe794161500 2023-10-26T05:56:45.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:44.999+0000 7fe77bfff640 1 -- 172.21.15.71:0/2529938307 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe788051c90 con 0x7fe794161500 2023-10-26T05:56:45.008 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:56:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe780066470 msgr2=0x7fe780068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe780066470 0x7fe780068930 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe7840ac000 tx=0x7fe7840b0040 comp rx=0 tx=0).stop 2023-10-26T05:56:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 msgr2=0x7fe794146990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 0x7fe794146990 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe78800e990 tx=0x7fe78800ee50 comp rx=0 tx=0).stop 2023-10-26T05:56:45.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 shutdown_connections 2023-10-26T05:56:45.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe79415a240 0x7fe79415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe780066470 0x7fe780068930 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe794161500 0x7fe794146990 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 --2- 172.21.15.71:0/2529938307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe77c0039b0 0x7fe794157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 >> 172.21.15.71:0/2529938307 conn(0x7fe7940cdd80 msgr2=0x7fe794164f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 shutdown_connections 2023-10-26T05:56:45.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.003+0000 7fe799b4d640 1 -- 172.21.15.71:0/2529938307 wait complete. 2023-10-26T05:56:45.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:56:45.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/22448703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b44070680 msgr2=0x7f3b44164010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 --2- 172.21.15.71:0/22448703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b44070680 0x7f3b44164010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009900 tx=0x7f3b3401c880 comp rx=0 tx=0).stop 2023-10-26T05:56:45.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/22448703 shutdown_connections 2023-10-26T05:56:45.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 --2- 172.21.15.71:0/22448703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b44070680 0x7f3b44164010 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/22448703 >> 172.21.15.71:0/22448703 conn(0x7f3b440af7d0 msgr2=0x7f3b440afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/22448703 shutdown_connections 2023-10-26T05:56:45.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/22448703 wait complete. 2023-10-26T05:56:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 Processor -- start 2023-10-26T05:56:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- start start 2023-10-26T05:56:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 0x7f3b44152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b4406ca30 0x7f3b441458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44070680 0x7f3b4414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4414c1e0 con 0x7f3b4406c650 2023-10-26T05:56:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b44152d10 con 0x7f3b44070680 2023-10-26T05:56:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441510f0 con 0x7f3b4406ca30 2023-10-26T05:56:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 0x7f3b44152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 0x7f3b44152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49636/0 (socket says 172.21.15.71:49636) 2023-10-26T05:56:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 -- 172.21.15.71:0/3618829924 learned_addr learned my addr 172.21.15.71:0/3618829924 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 -- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b4406ca30 msgr2=0x7f3b441458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b4406ca30 0x7f3b441458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 -- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44070680 msgr2=0x7f3b4414cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44070680 0x7f3b4414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 -- 172.21.15.71:0/3618829924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b34009560 con 0x7f3b4406c650 2023-10-26T05:56:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4a4fc640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 0x7f3b44152630 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c7c0 tx=0x7f3b3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00aa50 con 0x7f3b4406c650 2023-10-26T05:56:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b3c00aca0 con 0x7f3b4406c650 2023-10-26T05:56:45.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4414bc20 con 0x7f3b4406c650 2023-10-26T05:56:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c012680 con 0x7f3b4406c650 2023-10-26T05:56:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.143+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b44150bd0 con 0x7f3b4406c650 2023-10-26T05:56:45.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b44150a30 con 0x7f3b4406c650 2023-10-26T05:56:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b3c01b400 con 0x7f3b4406c650 2023-10-26T05:56:45.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b42ffd640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b30066470 0x7f3b30068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3b3c084350 con 0x7f3b4406c650 2023-10-26T05:56:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f3b3006e060 con 0x7f3b4406c650 2023-10-26T05:56:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b49cfb640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b30066470 0x7f3b30068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.147+0000 7f3b49cfb640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b30066470 0x7f3b30068930 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3b38007fd0 tx=0x7f3b38007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:45.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.151+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b44150a30 con 0x7f3b4406c650 2023-10-26T05:56:45.308 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.299+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b44150a30 con 0x7f3b4406c650 2023-10-26T05:56:45.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.299+0000 7f3b42ffd640 1 -- 172.21.15.71:0/3618829924 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b3c016020 con 0x7f3b4406c650 2023-10-26T05:56:45.309 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:56:45.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b30066470 msgr2=0x7f3b30068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b30066470 0x7f3b30068930 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3b38007fd0 tx=0x7f3b38007920 comp rx=0 tx=0).stop 2023-10-26T05:56:45.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 msgr2=0x7f3b44152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 0x7f3b44152630 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c7c0 tx=0x7f3b3c00cc80 comp rx=0 tx=0).stop 2023-10-26T05:56:45.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 shutdown_connections 2023-10-26T05:56:45.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b44070680 0x7f3b4414cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b4406ca30 0x7f3b441458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b30066470 0x7f3b30068930 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 --2- 172.21.15.71:0/3618829924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b4406c650 0x7f3b44152630 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 >> 172.21.15.71:0/3618829924 conn(0x7f3b440af7d0 msgr2=0x7f3b440afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 shutdown_connections 2023-10-26T05:56:45.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.303+0000 7f3b4c760640 1 -- 172.21.15.71:0/3618829924 wait complete. 2023-10-26T05:56:45.327 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:56:45.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2094137310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415c720 msgr2=0x7fd2140b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2094137310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415c720 0x7fd2140b0560 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd204009840 tx=0x7fd20401c740 comp rx=0 tx=0).stop 2023-10-26T05:56:45.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2094137310 shutdown_connections 2023-10-26T05:56:45.442 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2094137310 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd2140cede0 0x7fd21415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2094137310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415c720 0x7fd2140b0560 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2094137310 >> 172.21.15.71:0/2094137310 conn(0x7fd2140b0f20 msgr2=0x7fd2140b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2094137310 shutdown_connections 2023-10-26T05:56:45.443 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.431+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2094137310 wait complete. 2023-10-26T05:56:45.445 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 Processor -- start 2023-10-26T05:56:45.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- start start 2023-10-26T05:56:45.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 0x7fd214148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd21415c720 0x7fd21406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.446 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415d390 0x7fd21406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21414ad20 con 0x7fd2140cede0 2023-10-26T05:56:45.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140b4490 con 0x7fd21415d390 2023-10-26T05:56:45.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140b4770 con 0x7fd21415c720 2023-10-26T05:56:45.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 0x7fd214148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 0x7fd214148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49648/0 (socket says 172.21.15.71:49648) 2023-10-26T05:56:45.447 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 -- 172.21.15.71:0/2506160005 learned_addr learned my addr 172.21.15.71:0/2506160005 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:45.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd2137fe640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd21415c720 0x7fd21406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd218d5d640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415d390 0x7fd21406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 -- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd21415c720 msgr2=0x7fd21406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd21415c720 0x7fd21406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 -- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415d390 msgr2=0x7fd21406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.448 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415d390 0x7fd21406e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 -- 172.21.15.71:0/2506160005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009560 con 0x7fd2140cede0 2023-10-26T05:56:45.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd2137fe640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd21415c720 0x7fd21406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:56:45.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd213fff640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 0x7fd214148c40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fd204009790 tx=0x7fd2040a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:45.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040bc070 con 0x7fd2140cede0 2023-10-26T05:56:45.449 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2040094a0 con 0x7fd2140cede0 2023-10-26T05:56:45.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd2040041f0 con 0x7fd2140cede0 2023-10-26T05:56:45.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204003600 con 0x7fd2140cede0 2023-10-26T05:56:45.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2140b4d30 con 0x7fd2140cede0 2023-10-26T05:56:45.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd214066d00 con 0x7fd2140cede0 2023-10-26T05:56:45.450 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.435+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd2040b4040 con 0x7fd2140cede0 2023-10-26T05:56:45.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.439+0000 7fd210ff9640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd200066470 0x7fd200068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.439+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd20411e460 con 0x7fd2140cede0 2023-10-26T05:56:45.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.439+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fd20006e060 con 0x7fd2140cede0 2023-10-26T05:56:45.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.439+0000 7fd2137fe640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd200066470 0x7fd200068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.439+0000 7fd2137fe640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd200066470 0x7fd200068930 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd2141640f0 tx=0x7fd208009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:45.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.443+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd2040b2060 con 0x7fd2140cede0 2023-10-26T05:56:45.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2140b48b0 con 0x7fd2140cede0 2023-10-26T05:56:45.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd210ff9640 1 -- 172.21.15.71:0/2506160005 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd2140b48b0 con 0x7fd2140cede0 2023-10-26T05:56:45.595 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:56:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd200066470 msgr2=0x7fd200068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd200066470 0x7fd200068930 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd2141640f0 tx=0x7fd208009260 comp rx=0 tx=0).stop 2023-10-26T05:56:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 msgr2=0x7fd214148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.597 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 0x7fd214148c40 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fd204009790 tx=0x7fd2040a5620 comp rx=0 tx=0).stop 2023-10-26T05:56:45.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 shutdown_connections 2023-10-26T05:56:45.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd21415d390 0x7fd21406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd21415c720 0x7fd21406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd200066470 0x7fd200068930 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.598 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 --2- 172.21.15.71:0/2506160005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2140cede0 0x7fd214148c40 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 >> 172.21.15.71:0/2506160005 conn(0x7fd2140b0f20 msgr2=0x7fd2140b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 shutdown_connections 2023-10-26T05:56:45.599 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.587+0000 7fd21a7c0640 1 -- 172.21.15.71:0/2506160005 wait complete. 2023-10-26T05:56:45.612 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:56:45.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/4200240515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ce8005190 msgr2=0x7f8ce800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 --2- 172.21.15.71:0/4200240515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ce8005190 0x7f8ce800b990 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400c610 tx=0x7f8cf401fb00 comp rx=0 tx=0).stop 2023-10-26T05:56:45.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/4200240515 shutdown_connections 2023-10-26T05:56:45.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 --2- 172.21.15.71:0/4200240515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ce8005190 0x7f8ce800b990 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/4200240515 >> 172.21.15.71:0/4200240515 conn(0x7f8cfc0ce270 msgr2=0x7f8cfc0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/4200240515 shutdown_connections 2023-10-26T05:56:45.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/4200240515 wait complete. 2023-10-26T05:56:45.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 Processor -- start 2023-10-26T05:56:45.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- start start 2023-10-26T05:56:45.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ce8005190 0x7f8cfc150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 0x7f8cfc14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8cfc150960 0x7f8cfc14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc139720 con 0x7f8cfc13fb70 2023-10-26T05:56:45.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc14f440 con 0x7f8ce8005190 2023-10-26T05:56:45.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc157930 con 0x7f8cfc150960 2023-10-26T05:56:45.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 0x7f8cfc14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 0x7f8cfc14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49664/0 (socket says 172.21.15.71:49664) 2023-10-26T05:56:45.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 -- 172.21.15.71:0/1215354268 learned_addr learned my addr 172.21.15.71:0/1215354268 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:45.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d02026640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ce8005190 0x7f8cfc150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d02827640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8cfc150960 0x7f8cfc14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 -- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8cfc150960 msgr2=0x7f8cfc14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8cfc150960 0x7f8cfc14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 -- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ce8005190 msgr2=0x7f8cfc150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ce8005190 0x7f8cfc150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 -- 172.21.15.71:0/1215354268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc154db0 con 0x7f8cfc13fb70 2023-10-26T05:56:45.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d02827640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8cfc150960 0x7f8cfc14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:45.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d01825640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 0x7f8cfc14f8d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f8cec003e80 tx=0x7f8cec001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:45.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec011840 con 0x7f8cfc13fb70 2023-10-26T05:56:45.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8cec0119a0 con 0x7f8cfc13fb70 2023-10-26T05:56:45.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc157650 con 0x7f8cfc13fb70 2023-10-26T05:56:45.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec011d10 con 0x7f8cfc13fb70 2023-10-26T05:56:45.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.715+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc1549d0 con 0x7f8cfc13fb70 2023-10-26T05:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc066d00 con 0x7f8cfc13fb70 2023-10-26T05:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8cec01a400 con 0x7f8cfc13fb70 2023-10-26T05:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8cf2ffd640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ce0066470 0x7f8ce0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8cec084c70 con 0x7f8cfc13fb70 2023-10-26T05:56:45.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f8ce006e060 con 0x7f8cfc13fb70 2023-10-26T05:56:45.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8d02026640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ce0066470 0x7f8ce0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:45.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.719+0000 7f8d02026640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ce0066470 0x7f8ce0068930 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009a30 tx=0x7f8cf80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:45.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.723+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8cec0520b0 con 0x7f8cfc13fb70 2023-10-26T05:56:45.901 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.891+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cfc1550c0 con 0x7f8cfc13fb70 2023-10-26T05:56:45.902 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8cf2ffd640 1 -- 172.21.15.71:0/1215354268 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f8cec0556d0 con 0x7f8cfc13fb70 2023-10-26T05:56:45.902 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:45.902 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:56:45.903 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:56:45.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ce0066470 msgr2=0x7f8ce0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ce0066470 0x7f8ce0068930 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009a30 tx=0x7f8cf80091e0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 msgr2=0x7f8cfc14f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:45.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 0x7f8cfc14f8d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f8cec003e80 tx=0x7f8cec001640 comp rx=0 tx=0).stop 2023-10-26T05:56:45.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 shutdown_connections 2023-10-26T05:56:45.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8cfc150960 0x7f8cfc14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ce0066470 0x7f8ce0068930 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cfc13fb70 0x7f8cfc14f8d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 --2- 172.21.15.71:0/1215354268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ce8005190 0x7f8cfc150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:45.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 >> 172.21.15.71:0/1215354268 conn(0x7f8cfc0ce270 msgr2=0x7f8cfc153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:45.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 shutdown_connections 2023-10-26T05:56:45.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:45.895+0000 7f8d0428a640 1 -- 172.21.15.71:0/1215354268 wait complete. 2023-10-26T05:56:45.919 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:46.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/328332963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 msgr2=0x7f4008159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 --2- 172.21.15.71:0/328332963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f4008159810 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009900 tx=0x7f3ff801c8c0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/328332963 shutdown_connections 2023-10-26T05:56:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 --2- 172.21.15.71:0/328332963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f4008159810 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 --2- 172.21.15.71:0/328332963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40080b2a60 0x7f40080caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/328332963 >> 172.21.15.71:0/328332963 conn(0x7f4008069770 msgr2=0x7f4008069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/328332963 shutdown_connections 2023-10-26T05:56:46.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/328332963 wait complete. 2023-10-26T05:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 Processor -- start 2023-10-26T05:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- start start 2023-10-26T05:56:46.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40080b2a60 0x7f400815b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f400814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f400813c2f0 0x7f400813ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400815bc90 con 0x7f40080cb010 2023-10-26T05:56:46.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400815bf70 con 0x7f400813c2f0 2023-10-26T05:56:46.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400815c250 con 0x7f40080b2a60 2023-10-26T05:56:46.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f400814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f400814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49668/0 (socket says 172.21.15.71:49668) 2023-10-26T05:56:46.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 -- 172.21.15.71:0/3118701495 learned_addr learned my addr 172.21.15.71:0/3118701495 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:46.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f4007fff640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40080b2a60 0x7f400815b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400ca8d640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f400813c2f0 0x7f400813ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 -- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40080b2a60 msgr2=0x7f400815b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40080b2a60 0x7f400815b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 -- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f400813c2f0 msgr2=0x7f400813ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f400813c2f0 0x7f400813ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 -- 172.21.15.71:0/3118701495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8009560 con 0x7f40080cb010 2023-10-26T05:56:46.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400ca8d640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f400813c2f0 0x7f400813ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:46.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f40077fe640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f400814f660 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000d980 tx=0x7f3ff000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:46.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0009c30 con 0x7f40080cb010 2023-10-26T05:56:46.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ff0005020 con 0x7f40080cb010 2023-10-26T05:56:46.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40081361a0 con 0x7f40080cb010 2023-10-26T05:56:46.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0005430 con 0x7f40080cb010 2023-10-26T05:56:46.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.035+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400815b8d0 con 0x7f40080cb010 2023-10-26T05:56:46.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008066b10 con 0x7f40080cb010 2023-10-26T05:56:46.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3ff00047b0 con 0x7f40080cb010 2023-10-26T05:56:46.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f4004ff9640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3fec066470 0x7f3fec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3ff0084b60 con 0x7f40080cb010 2023-10-26T05:56:46.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f3fec06e060 con 0x7f40080cb010 2023-10-26T05:56:46.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f4007fff640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3fec066470 0x7f3fec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.039+0000 7f4007fff640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3fec066470 0x7f3fec068930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f400815cc20 tx=0x7f4000009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:46.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.043+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3ff0017070 con 0x7f40080cb010 2023-10-26T05:56:46.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.199+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40080721e0 con 0x7f40080cb010 2023-10-26T05:56:46.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.199+0000 7f4004ff9640 1 -- 172.21.15.71:0/3118701495 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16619 (secure 0 0 0) 0x7f3ff0020030 con 0x7f40080cb010 2023-10-26T05:56:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:46.209 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":159,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:55:52.244888+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.3a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3fec066470 msgr2=0x7f3fec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3fec066470 0x7f3fec068930 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f400815cc20 tx=0x7f4000009040 comp rx=0 tx=0).stop 2023-10-26T05:56:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 msgr2=0x7f400814f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f400814f660 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000d980 tx=0x7f3ff000de40 comp rx=0 tx=0).stop 2023-10-26T05:56:46.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 shutdown_connections 2023-10-26T05:56:46.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f400813c2f0 0x7f400813ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3fec066470 0x7f3fec068930 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40080cb010 0x7f400814f660 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 --2- 172.21.15.71:0/3118701495 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40080b2a60 0x7f400815b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 >> 172.21.15.71:0/3118701495 conn(0x7f4008069770 msgr2=0x7f4008071760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:46.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 shutdown_connections 2023-10-26T05:56:46.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.203+0000 7f400e4f0640 1 -- 172.21.15.71:0/3118701495 wait complete. 2023-10-26T05:56:46.227 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:56:46.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:56:46.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/438141529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0b2a60 msgr2=0x7ff5dc0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 --2- 172.21.15.71:0/438141529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0b2a60 0x7ff5dc0caad0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc1573e0 tx=0x7ff5c401c880 comp rx=0 tx=0).stop 2023-10-26T05:56:46.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/438141529 shutdown_connections 2023-10-26T05:56:46.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 --2- 172.21.15.71:0/438141529 >> v2:172.21.15.86:3300/0 conn(0x7ff5dc0cb010 0x7ff5dc159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 --2- 172.21.15.71:0/438141529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0b2a60 0x7ff5dc0caad0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/438141529 >> 172.21.15.71:0/438141529 conn(0x7ff5dc069770 msgr2=0x7ff5dc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:46.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/438141529 shutdown_connections 2023-10-26T05:56:46.348 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/438141529 wait complete. 2023-10-26T05:56:46.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 Processor -- start 2023-10-26T05:56:46.350 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- start start 2023-10-26T05:56:46.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5dc0b2a60 0x7ff5dc152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 0x7ff5dc137aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5dc13c2f0 0x7ff5dc14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc152a80 con 0x7ff5dc0cb010 2023-10-26T05:56:46.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc1514a0 con 0x7ff5dc13c2f0 2023-10-26T05:56:46.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc151780 con 0x7ff5dc0b2a60 2023-10-26T05:56:46.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 0x7ff5dc137aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 0x7ff5dc137aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49688/0 (socket says 172.21.15.71:49688) 2023-10-26T05:56:46.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 -- 172.21.15.71:0/4027414355 learned_addr learned my addr 172.21.15.71:0/4027414355 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:46.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5db59e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5dc13c2f0 0x7ff5dc14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5dad9d640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5dc0b2a60 0x7ff5dc152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 -- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5dc0b2a60 msgr2=0x7ff5dc152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5dc0b2a60 0x7ff5dc152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 -- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5dc13c2f0 msgr2=0x7ff5dc14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5dc13c2f0 0x7ff5dc14fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 -- 172.21.15.71:0/4027414355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c4009560 con 0x7ff5dc0cb010 2023-10-26T05:56:46.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5db59e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5dc13c2f0 0x7ff5dc14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:46.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d259c640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 0x7ff5dc137aa0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4000c10 tx=0x7ff5c40a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:46.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40ba070 con 0x7ff5dc0cb010 2023-10-26T05:56:46.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff5c4004250 con 0x7ff5dc0cb010 2023-10-26T05:56:46.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc151a60 con 0x7ff5dc0cb010 2023-10-26T05:56:46.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4005030 con 0x7ff5dc0cb010 2023-10-26T05:56:46.354 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.339+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc152020 con 0x7ff5dc0cb010 2023-10-26T05:56:46.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc066b10 con 0x7ff5dc0cb010 2023-10-26T05:56:46.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff5c40b4070 con 0x7ff5dc0cb010 2023-10-26T05:56:46.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5d3fff640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff5bc0664a0 0x7ff5bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff5c411db00 con 0x7ff5dc0cb010 2023-10-26T05:56:46.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7ff5bc06e090 con 0x7ff5dc0cb010 2023-10-26T05:56:46.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5dad9d640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff5bc0664a0 0x7ff5bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.343+0000 7ff5dad9d640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff5bc0664a0 0x7ff5bc068960 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc000f70 tx=0x7ff5cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:46.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.347+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff5c40eaf50 con 0x7ff5dc0cb010 2023-10-26T05:56:46.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.523+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5dc152210 con 0x7ff5dc0cb010 2023-10-26T05:56:46.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5d3fff640 1 -- 172.21.15.71:0/4027414355 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7ff5c40b2030 con 0x7ff5dc0cb010 2023-10-26T05:56:46.534 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:46.534 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:56:46.535 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:56:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff5bc0664a0 msgr2=0x7ff5bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff5bc0664a0 0x7ff5bc068960 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc000f70 tx=0x7ff5cc009040 comp rx=0 tx=0).stop 2023-10-26T05:56:46.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 msgr2=0x7ff5dc137aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 0x7ff5dc137aa0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4000c10 tx=0x7ff5c40a5800 comp rx=0 tx=0).stop 2023-10-26T05:56:46.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 shutdown_connections 2023-10-26T05:56:46.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5dc13c2f0 0x7ff5dc14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff5bc0664a0 0x7ff5bc068960 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5dc0cb010 0x7ff5dc137aa0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 --2- 172.21.15.71:0/4027414355 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff5dc0b2a60 0x7ff5dc152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 >> 172.21.15.71:0/4027414355 conn(0x7ff5dc069770 msgr2=0x7ff5dc0accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:46.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 shutdown_connections 2023-10-26T05:56:46.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.527+0000 7ff5e116e640 1 -- 172.21.15.71:0/4027414355 wait complete. 2023-10-26T05:56:46.551 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:56:46.605 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:56:46.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:56:46.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 -- 172.21.15.71:0/985926504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc340062a0 msgr2=0x7fcc3400caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 --2- 172.21.15.71:0/985926504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc340062a0 0x7fcc3400caa0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009e30 tx=0x7fcc4001d190 comp rx=0 tx=0).stop 2023-10-26T05:56:46.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 -- 172.21.15.71:0/985926504 shutdown_connections 2023-10-26T05:56:46.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 --2- 172.21.15.71:0/985926504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc340062a0 0x7fcc3400caa0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 -- 172.21.15.71:0/985926504 >> 172.21.15.71:0/985926504 conn(0x7fcc48069770 msgr2=0x7fcc48069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:46.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 -- 172.21.15.71:0/985926504 shutdown_connections 2023-10-26T05:56:46.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.663+0000 7fcc501b9640 1 -- 172.21.15.71:0/985926504 wait complete. 2023-10-26T05:56:46.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 Processor -- start 2023-10-26T05:56:46.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- start start 2023-10-26T05:56:46.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340062a0 0x7fcc480a8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc480ab5e0 0x7fcc48070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 0x7fcc48130c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc481552f0 con 0x7fcc48070dd0 2023-10-26T05:56:46.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48071280 con 0x7fcc340062a0 2023-10-26T05:56:46.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4812f4d0 con 0x7fcc480ab5e0 2023-10-26T05:56:46.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 0x7fcc48130c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 0x7fcc48130c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49714/0 (socket says 172.21.15.71:49714) 2023-10-26T05:56:46.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 -- 172.21.15.71:0/2489984738 learned_addr learned my addr 172.21.15.71:0/2489984738 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:46.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4d754640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc480ab5e0 0x7fcc48070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4df55640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340062a0 0x7fcc480a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 -- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc480ab5e0 msgr2=0x7fcc48070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc480ab5e0 0x7fcc48070890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 -- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340062a0 msgr2=0x7fcc480a8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340062a0 0x7fcc480a8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 -- 172.21.15.71:0/2489984738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4812e850 con 0x7fcc48070dd0 2023-10-26T05:56:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc4e756640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 0x7fcc48130c60 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcc4001da30 tx=0x7fcc40005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40005410 con 0x7fcc48070dd0 2023-10-26T05:56:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcc40005660 con 0x7fcc48070dd0 2023-10-26T05:56:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4812eb30 con 0x7fcc48070dd0 2023-10-26T05:56:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc400b7470 con 0x7fcc48070dd0 2023-10-26T05:56:46.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4812f0f0 con 0x7fcc48070dd0 2023-10-26T05:56:46.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc48066b10 con 0x7fcc48070dd0 2023-10-26T05:56:46.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcc400b75d0 con 0x7fcc48070dd0 2023-10-26T05:56:46.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc2c066470 0x7fcc2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:46.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcc4011f630 con 0x7fcc48070dd0 2023-10-26T05:56:46.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.667+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fcc2c06e060 con 0x7fcc48070dd0 2023-10-26T05:56:46.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.671+0000 7fcc4df55640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc2c066470 0x7fcc2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:46.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.671+0000 7fcc4df55640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc2c066470 0x7fcc2c068930 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcc440017d0 tx=0x7fcc44001ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:46.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.671+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcc400eca80 con 0x7fcc48070dd0 2023-10-26T05:56:46.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.843+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc4812f610 con 0x7fcc48070dd0 2023-10-26T05:56:46.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc3e7fc640 1 -- 172.21.15.71:0/2489984738 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fcc4812f610 con 0x7fcc48070dd0 2023-10-26T05:56:46.854 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:46.854 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:54:34.550038+0000","modified":"2023-10-26T05:54:36.609944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8517":{"gid":8517,"name":"c","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2112769040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":2112769040},{"type":"v1","addr":"172.21.15.71:6835","nonce":2112769040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-26T05:56:46.855 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 58 2023-10-26T05:56:46.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc2c066470 msgr2=0x7fcc2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc2c066470 0x7fcc2c068930 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcc440017d0 tx=0x7fcc44001ca0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 msgr2=0x7fcc48130c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:46.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 0x7fcc48130c60 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcc4001da30 tx=0x7fcc40005c00 comp rx=0 tx=0).stop 2023-10-26T05:56:46.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 shutdown_connections 2023-10-26T05:56:46.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcc2c066470 0x7fcc2c068930 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc48070dd0 0x7fcc48130c60 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcc480ab5e0 0x7fcc48070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 --2- 172.21.15.71:0/2489984738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340062a0 0x7fcc480a8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:46.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 >> 172.21.15.71:0/2489984738 conn(0x7fcc48069770 msgr2=0x7fcc481591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:46.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 shutdown_connections 2023-10-26T05:56:46.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:46.847+0000 7fcc501b9640 1 -- 172.21.15.71:0/2489984738 wait complete. 2023-10-26T05:56:46.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:47.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/3065508110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b880039b0 msgr2=0x7f9b88003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/3065508110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b880039b0 0x7f9b88003db0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c730 tx=0x7f9b9801c690 comp rx=0 tx=0).stop 2023-10-26T05:56:47.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/3065508110 shutdown_connections 2023-10-26T05:56:47.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/3065508110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b880039b0 0x7f9b88003db0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/3065508110 >> 172.21.15.71:0/3065508110 conn(0x7f9ba0069770 msgr2=0x7f9ba0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:47.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/3065508110 shutdown_connections 2023-10-26T05:56:47.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/3065508110 wait complete. 2023-10-26T05:56:47.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 Processor -- start 2023-10-26T05:56:47.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- start start 2023-10-26T05:56:47.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b880039b0 0x7f9ba0157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 0x7f9ba0136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba0135a90 0x7f9ba01373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba01587d0 con 0x7f9ba0135610 2023-10-26T05:56:47.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0135f80 con 0x7f9b880039b0 2023-10-26T05:56:47.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba0136260 con 0x7f9ba0135a90 2023-10-26T05:56:47.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 0x7f9ba0136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 0x7f9ba0136e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49736/0 (socket says 172.21.15.71:49736) 2023-10-26T05:56:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 -- 172.21.15.71:0/1796109832 learned_addr learned my addr 172.21.15.71:0/1796109832 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9e59c640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b880039b0 0x7f9ba0157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9ed9d640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba0135a90 0x7f9ba01373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 -- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba0135a90 msgr2=0x7f9ba01373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba0135a90 0x7f9ba01373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 -- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b880039b0 msgr2=0x7f9ba0157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b880039b0 0x7f9ba0157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 -- 172.21.15.71:0/1796109832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0136540 con 0x7f9ba0135610 2023-10-26T05:56:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b9dd9b640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 0x7f9ba0136e80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400caa0 tx=0x7f9b9400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:47.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94003d90 con 0x7f9ba0135610 2023-10-26T05:56:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b94005070 con 0x7f9ba0135610 2023-10-26T05:56:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba0158a20 con 0x7f9ba0135610 2023-10-26T05:56:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94005480 con 0x7f9ba0135610 2023-10-26T05:56:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.027+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba01f56a0 con 0x7f9ba0135610 2023-10-26T05:56:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba0066b10 con 0x7f9ba0135610 2023-10-26T05:56:47.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b940055e0 con 0x7f9ba0135610 2023-10-26T05:56:47.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9b8effd640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b80066470 0x7f9b80068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9b94083b80 con 0x7f9ba0135610 2023-10-26T05:56:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f9b8006e060 con 0x7f9ba0135610 2023-10-26T05:56:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9b9e59c640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b80066470 0x7f9b80068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.031+0000 7f9b9e59c640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b80066470 0x7f9b80068930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c700 tx=0x7f9b98019430 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:47.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.035+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b94083e60 con 0x7f9ba0135610 2023-10-26T05:56:47.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.179+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ba00721e0 con 0x7f9ba0135610 2023-10-26T05:56:47.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.179+0000 7f9b8effd640 1 -- 172.21.15.71:0/1796109832 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16619 (secure 0 0 0) 0x7f9b94017070 con 0x7f9ba0135610 2023-10-26T05:56:47.188 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:47.188 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":159,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:55:52.244888+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:54:32.632697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-26T05:54:33.544620+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.c","mappings":[{"from":2,"to":0}]},{"pgid":"19.3a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:47.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.179+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b80066470 msgr2=0x7f9b80068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.179+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b80066470 0x7f9b80068930 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c700 tx=0x7f9b98019430 comp rx=0 tx=0).stop 2023-10-26T05:56:47.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 msgr2=0x7f9ba0136e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 0x7f9ba0136e80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400caa0 tx=0x7f9b9400cea0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 shutdown_connections 2023-10-26T05:56:47.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba0135a90 0x7f9ba01373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b80066470 0x7f9b80068930 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ba0135610 0x7f9ba0136e80 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 --2- 172.21.15.71:0/1796109832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b880039b0 0x7f9ba0157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 >> 172.21.15.71:0/1796109832 conn(0x7f9ba0069770 msgr2=0x7f9ba00ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:47.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 shutdown_connections 2023-10-26T05:56:47.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.183+0000 7f9ba4f4a640 1 -- 172.21.15.71:0/1796109832 wait complete. 2023-10-26T05:56:47.206 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:56:47.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1516107842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6500cced0 msgr2=0x7ff65006a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1516107842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6500cced0 0x7ff65006a7b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff648009c80 tx=0x7ff64801f540 comp rx=0 tx=0).stop 2023-10-26T05:56:47.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1516107842 shutdown_connections 2023-10-26T05:56:47.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1516107842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6500cced0 0x7ff65006a7b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1516107842 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff65015b3b0 0x7ff6500b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1516107842 >> 172.21.15.71:0/1516107842 conn(0x7ff6500d12d0 msgr2=0x7ff6500d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:47.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1516107842 shutdown_connections 2023-10-26T05:56:47.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1516107842 wait complete. 2023-10-26T05:56:47.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 Processor -- start 2023-10-26T05:56:47.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- start start 2023-10-26T05:56:47.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 0x7ff6501f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6500cced0 0x7ff650158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff65015b3b0 0x7ff6501f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6501f2820 con 0x7ff65006a390 2023-10-26T05:56:47.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6500a8090 con 0x7ff65015b3b0 2023-10-26T05:56:47.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6500a8370 con 0x7ff6500cced0 2023-10-26T05:56:47.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 0x7ff6501f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 0x7ff6501f22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49748/0 (socket says 172.21.15.71:49748) 2023-10-26T05:56:47.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 -- 172.21.15.71:0/1939595632 learned_addr learned my addr 172.21.15.71:0/1939595632 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:47.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64ed9d640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff65015b3b0 0x7ff6501f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 -- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6500cced0 msgr2=0x7ff650158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6500cced0 0x7ff650158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 -- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff65015b3b0 msgr2=0x7ff6501f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff65015b3b0 0x7ff6501f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 -- 172.21.15.71:0/1939595632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6480098e0 con 0x7ff65006a390 2023-10-26T05:56:47.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64ed9d640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff65015b3b0 0x7ff6501f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:47.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff64e59c640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 0x7ff6501f22e0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b410 tx=0x7ff63800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:47.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638011830 con 0x7ff65006a390 2023-10-26T05:56:47.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6500a8650 con 0x7ff65006a390 2023-10-26T05:56:47.334 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff638011a80 con 0x7ff65006a390 2023-10-26T05:56:47.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638011e90 con 0x7ff65006a390 2023-10-26T05:56:47.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.319+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6500a8c10 con 0x7ff65006a390 2023-10-26T05:56:47.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.323+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff650066b10 con 0x7ff65006a390 2023-10-26T05:56:47.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.323+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff638024440 con 0x7ff65006a390 2023-10-26T05:56:47.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.323+0000 7ff646ffd640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff630066470 0x7ff630068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.323+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff638084780 con 0x7ff65006a390 2023-10-26T05:56:47.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.323+0000 7ff647fff640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff630066470 0x7ff630068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.323+0000 7ff647fff640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff630066470 0x7ff630068930 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff640007fd0 tx=0x7ff6400079a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:47.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.327+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff638051ca0 con 0x7ff65006a390 2023-10-26T05:56:47.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.499+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff65004db60 con 0x7ff65006a390 2023-10-26T05:56:47.508 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:56:47.499+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:56:47.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff646ffd640 1 -- 172.21.15.71:0/1939595632 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff638082e60 con 0x7ff65006a390 2023-10-26T05:56:47.511 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:56:47.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff630066470 msgr2=0x7ff630068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff630066470 0x7ff630068930 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff640007fd0 tx=0x7ff6400079a0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 msgr2=0x7ff6501f22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 0x7ff6501f22e0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7ff63800b410 tx=0x7ff63800b8d0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 shutdown_connections 2023-10-26T05:56:47.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff65015b3b0 0x7ff6501f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff6500cced0 0x7ff650158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff630066470 0x7ff630068930 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 --2- 172.21.15.71:0/1939595632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff65006a390 0x7ff6501f22e0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 >> 172.21.15.71:0/1939595632 conn(0x7ff6500d12d0 msgr2=0x7ff65015a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:47.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 shutdown_connections 2023-10-26T05:56:47.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.503+0000 7ff654ccc640 1 -- 172.21.15.71:0/1939595632 wait complete. 2023-10-26T05:56:47.521 INFO:tasks.ceph.mds.c.smithi071.stderr: -9337> 2023-10-26T05:54:37.031+0000 7fea3f163640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:56:47.532 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:56:47.558 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:56:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 -- 172.21.15.71:0/3910936546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c069e60 msgr2=0x7fa28c06a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 --2- 172.21.15.71:0/3910936546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c069e60 0x7fa28c06a2b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0d1f00 tx=0x7fa28401cca0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 -- 172.21.15.71:0/3910936546 shutdown_connections 2023-10-26T05:56:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 --2- 172.21.15.71:0/3910936546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c069e60 0x7fa28c06a2b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 --2- 172.21.15.71:0/3910936546 >> v2:172.21.15.86:3300/0 conn(0x7fa28c0b6910 0x7fa28c0b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 --2- 172.21.15.71:0/3910936546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa28c15b5c0 0x7fa28c0b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 -- 172.21.15.71:0/3910936546 >> 172.21.15.71:0/3910936546 conn(0x7fa28c0d12d0 msgr2=0x7fa28c0d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:47.662 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 -- 172.21.15.71:0/3910936546 shutdown_connections 2023-10-26T05:56:47.663 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.651+0000 7fa293a50640 1 -- 172.21.15.71:0/3910936546 wait complete. 2023-10-26T05:56:47.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 Processor -- start 2023-10-26T05:56:47.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- start start 2023-10-26T05:56:47.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa28c069e60 0x7fa28c14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 0x7fa28c156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa28c15b5c0 0x7fa28c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c14d2e0 con 0x7fa28c0b6910 2023-10-26T05:56:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c158e70 con 0x7fa28c069e60 2023-10-26T05:56:47.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c14d9d0 con 0x7fa28c15b5c0 2023-10-26T05:56:47.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 0x7fa28c156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 0x7fa28c156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49770/0 (socket says 172.21.15.71:49770) 2023-10-26T05:56:47.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 -- 172.21.15.71:0/199853653 learned_addr learned my addr 172.21.15.71:0/199853653 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:47.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa2917ec640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa28c069e60 0x7fa28c14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa291fed640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa28c15b5c0 0x7fa28c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 -- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa28c15b5c0 msgr2=0x7fa28c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa28c15b5c0 0x7fa28c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 -- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa28c069e60 msgr2=0x7fa28c14ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:47.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa28c069e60 0x7fa28c14ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:47.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 -- 172.21.15.71:0/199853653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840098e0 con 0x7fa28c0b6910 2023-10-26T05:56:47.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa2917ec640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa28c069e60 0x7fa28c14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:47.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa290feb640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 0x7fa28c156730 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c9a0 tx=0x7fa27c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:47.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c007c10 con 0x7fa28c0b6910 2023-10-26T05:56:47.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa27c007e60 con 0x7fa28c0b6910 2023-10-26T05:56:47.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c0cdb70 con 0x7fa28c0b6910 2023-10-26T05:56:47.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c005280 con 0x7fa28c0b6910 2023-10-26T05:56:47.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c14cfd0 con 0x7fa28c0b6910 2023-10-26T05:56:47.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c066b10 con 0x7fa28c0b6910 2023-10-26T05:56:47.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa27c0053e0 con 0x7fa28c0b6910 2023-10-26T05:56:47.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa281ffb640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa270066470 0x7fa270068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:47.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.655+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa27c0844f0 con 0x7fa28c0b6910 2023-10-26T05:56:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.659+0000 7fa2917ec640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa270066470 0x7fa270068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:47.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.659+0000 7fa2917ec640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa270066470 0x7fa270068930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa28800a870 tx=0x7fa288008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:47.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.663+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa27c055400 con 0x7fa28c0b6910 2023-10-26T05:56:47.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:47.831+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa28c0721e0 con 0x7fa28c0b6910 2023-10-26T05:56:48.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.507+0000 7fa281ffb640 1 -- 172.21.15.71:0/199853653 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa27c054f90 con 0x7fa28c0b6910 2023-10-26T05:56:48.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa270066470 msgr2=0x7fa270068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa270066470 0x7fa270068930 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa28800a870 tx=0x7fa288008040 comp rx=0 tx=0).stop 2023-10-26T05:56:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 msgr2=0x7fa28c156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 0x7fa28c156730 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c9a0 tx=0x7fa27c00ce60 comp rx=0 tx=0).stop 2023-10-26T05:56:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 shutdown_connections 2023-10-26T05:56:48.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa28c15b5c0 0x7fa28c156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa270066470 0x7fa270068930 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa28c0b6910 0x7fa28c156730 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 --2- 172.21.15.71:0/199853653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa28c069e60 0x7fa28c14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 >> 172.21.15.71:0/199853653 conn(0x7fa28c0d12d0 msgr2=0x7fa28c1559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 shutdown_connections 2023-10-26T05:56:48.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.511+0000 7fa293a50640 1 -- 172.21.15.71:0/199853653 wait complete. 2023-10-26T05:56:48.541 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:56:48.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- 172.21.15.71:0/932878950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88005190 msgr2=0x7f9b8800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:48.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/932878950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88005190 0x7f9b8800b990 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c590 tx=0x7f9b9401d340 comp rx=0 tx=0).stop 2023-10-26T05:56:48.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- 172.21.15.71:0/932878950 shutdown_connections 2023-10-26T05:56:48.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/932878950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88005190 0x7f9b8800b990 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- 172.21.15.71:0/932878950 >> 172.21.15.71:0/932878950 conn(0x7f9b9c0d13f0 msgr2=0x7f9b9c0d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:48.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- 172.21.15.71:0/932878950 shutdown_connections 2023-10-26T05:56:48.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- 172.21.15.71:0/932878950 wait complete. 2023-10-26T05:56:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 Processor -- start 2023-10-26T05:56:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- start start 2023-10-26T05:56:48.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b88005190 0x7f9b9c149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b9c1e3f40 0x7f9b9c14a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 0x7f9b9c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c15d850 con 0x7f9b9c15ce60 2023-10-26T05:56:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0c6380 con 0x7f9b88005190 2023-10-26T05:56:48.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba3b41640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0c6660 con 0x7f9b9c1e3f40 2023-10-26T05:56:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 0x7f9b9c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 0x7f9b9c15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49786/0 (socket says 172.21.15.71:49786) 2023-10-26T05:56:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 -- 172.21.15.71:0/1682051779 learned_addr learned my addr 172.21.15.71:0/1682051779 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba18dd640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b88005190 0x7f9b9c149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:48.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba10dc640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b9c1e3f40 0x7f9b9c14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:48.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 -- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b9c1e3f40 msgr2=0x7f9b9c14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:48.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b9c1e3f40 0x7f9b9c14a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 -- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b88005190 msgr2=0x7f9b9c149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:48.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b88005190 0x7f9b9c149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:48.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 -- 172.21.15.71:0/1682051779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0c6940 con 0x7f9b9c15ce60 2023-10-26T05:56:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba18dd640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b88005190 0x7f9b9c149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9ba20de640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 0x7f9b9c15d310 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9b9401d170 tx=0x7f9b94001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b940b5070 con 0x7f9b9c15ce60 2023-10-26T05:56:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.711+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9b940018c0 con 0x7f9b9c15ce60 2023-10-26T05:56:48.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94003980 con 0x7f9b9c15ce60 2023-10-26T05:56:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c0c6c20 con 0x7f9b9c15ce60 2023-10-26T05:56:48.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c149cd0 con 0x7f9b9c15ce60 2023-10-26T05:56:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c066b10 con 0x7f9b9c15ce60 2023-10-26T05:56:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b940ab030 con 0x7f9b9c15ce60 2023-10-26T05:56:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9b927fc640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b800664a0 0x7f9b80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9b9411d120 con 0x7f9b9c15ce60 2023-10-26T05:56:48.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9ba18dd640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b800664a0 0x7f9b80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:48.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.715+0000 7f9ba18dd640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b800664a0 0x7f9b80068960 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c15e440 tx=0x7f9b8c006a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:48.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.719+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9b940ea5a0 con 0x7f9b9c15ce60 2023-10-26T05:56:48.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:48.863+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9b9c0721e0 con 0x7f9b9c15ce60 2023-10-26T05:56:49.611 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.603+0000 7f9b927fc640 1 -- 172.21.15.71:0/1682051779 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v162) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9b940edbc0 con 0x7f9b9c15ce60 2023-10-26T05:56:49.611 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:56:49.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b800664a0 msgr2=0x7f9b80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:49.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b800664a0 0x7f9b80068960 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c15e440 tx=0x7f9b8c006a00 comp rx=0 tx=0).stop 2023-10-26T05:56:49.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 msgr2=0x7f9b9c15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:49.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 0x7f9b9c15d310 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9b9401d170 tx=0x7f9b94001330 comp rx=0 tx=0).stop 2023-10-26T05:56:49.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 shutdown_connections 2023-10-26T05:56:49.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9b800664a0 0x7f9b80068960 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c15ce60 0x7f9b9c15d310 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9b9c1e3f40 0x7f9b9c14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 --2- 172.21.15.71:0/1682051779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b88005190 0x7f9b9c149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 >> 172.21.15.71:0/1682051779 conn(0x7f9b9c0d13f0 msgr2=0x7f9b9c0b6610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:49.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 shutdown_connections 2023-10-26T05:56:49.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.607+0000 7f9ba3b41640 1 -- 172.21.15.71:0/1682051779 wait complete. 2023-10-26T05:56:49.638 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:56:49.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/2281981277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49815bf70 msgr2=0x7ff4980ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:49.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 --2- 172.21.15.71:0/2281981277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49815bf70 0x7ff4980ca6d0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ff490009900 tx=0x7ff49001c850 comp rx=0 tx=0).stop 2023-10-26T05:56:49.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/2281981277 shutdown_connections 2023-10-26T05:56:49.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 --2- 172.21.15.71:0/2281981277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff49815bf70 0x7ff4980ca6d0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 --2- 172.21.15.71:0/2281981277 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bb90 0x7ff4980b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/2281981277 >> 172.21.15.71:0/2281981277 conn(0x7ff4980aec60 msgr2=0x7ff4980af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:49.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/2281981277 shutdown_connections 2023-10-26T05:56:49.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/2281981277 wait complete. 2023-10-26T05:56:49.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 Processor -- start 2023-10-26T05:56:49.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- start start 2023-10-26T05:56:49.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 0x7ff4981535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:49.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff49815bb90 0x7ff49815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:49.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bf70 0x7ff4981f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:49.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff498158610 con 0x7ff498070dc0 2023-10-26T05:56:49.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980adc90 con 0x7ff49815bb90 2023-10-26T05:56:49.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980adf50 con 0x7ff49815bf70 2023-10-26T05:56:49.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 0x7ff4981535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:49.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 0x7ff4981535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49812/0 (socket says 172.21.15.71:49812) 2023-10-26T05:56:49.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 -- 172.21.15.71:0/3686238443 learned_addr learned my addr 172.21.15.71:0/3686238443 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:49.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff497fff640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bf70 0x7ff4981f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:49.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 -- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bf70 msgr2=0x7ff4981f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:49.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bf70 0x7ff4981f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 -- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff49815bb90 msgr2=0x7ff49815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:49.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff496ffd640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff49815bb90 0x7ff49815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:49.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff49815bb90 0x7ff49815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:49.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 -- 172.21.15.71:0/3686238443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490009560 con 0x7ff498070dc0 2023-10-26T05:56:49.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff497fff640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bf70 0x7ff4981f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:49.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff4977fe640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 0x7ff4981535b0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff48800ead0 tx=0x7ff48800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:49.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800ce40 con 0x7ff498070dc0 2023-10-26T05:56:49.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff488004c40 con 0x7ff498070dc0 2023-10-26T05:56:49.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4980ae230 con 0x7ff498070dc0 2023-10-26T05:56:49.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488005020 con 0x7ff498070dc0 2023-10-26T05:56:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.767+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49815b4d0 con 0x7ff498070dc0 2023-10-26T05:56:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.771+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff498066b10 con 0x7ff498070dc0 2023-10-26T05:56:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.771+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff488005180 con 0x7ff498070dc0 2023-10-26T05:56:49.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.771+0000 7ff47ffff640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff484066470 0x7ff484068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:49.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.771+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5415+0+0 (secure 0 0 0) 0x7ff488084880 con 0x7ff498070dc0 2023-10-26T05:56:49.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.771+0000 7ff496ffd640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff484066470 0x7ff484068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.771+0000 7ff496ffd640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff484066470 0x7ff484068930 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:49.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.775+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff488051d90 con 0x7ff498070dc0 2023-10-26T05:56:49.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:49.935+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff498158b90 con 0x7ff498070dc0 2023-10-26T05:56:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.615+0000 7ff47ffff640 1 -- 172.21.15.71:0/3686238443 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v163) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff48801f030 con 0x7ff498070dc0 2023-10-26T05:56:50.625 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:56:50.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff484066470 msgr2=0x7ff484068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff484066470 0x7ff484068930 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff48c007fd0 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2023-10-26T05:56:50.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 msgr2=0x7ff4981535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 0x7ff4981535b0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ff48800ead0 tx=0x7ff48800ef90 comp rx=0 tx=0).stop 2023-10-26T05:56:50.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 shutdown_connections 2023-10-26T05:56:50.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff49815bf70 0x7ff4981f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff49815bb90 0x7ff49815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff484066470 0x7ff484068930 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 --2- 172.21.15.71:0/3686238443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff498070dc0 0x7ff4981535b0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 >> 172.21.15.71:0/3686238443 conn(0x7ff4980aec60 msgr2=0x7ff4981571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:50.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 shutdown_connections 2023-10-26T05:56:50.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.619+0000 7ff49de38640 1 -- 172.21.15.71:0/3686238443 wait complete. 2023-10-26T05:56:50.648 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:56:50.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- 172.21.15.71:0/1097851946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 msgr2=0x7f24fc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 --2- 172.21.15.71:0/1097851946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f24fc004c50 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f2504006f10 tx=0x7f250401cf00 comp rx=0 tx=0).stop 2023-10-26T05:56:50.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251970b640 1 -- 172.21.15.71:0/1097851946 reap_dead start 2023-10-26T05:56:50.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- 172.21.15.71:0/1097851946 shutdown_connections 2023-10-26T05:56:50.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- 172.21.15.71:0/1097851946 >> 172.21.15.71:0/1097851946 conn(0x7f251406b8c0 msgr2=0x7f251406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:50.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- 172.21.15.71:0/1097851946 shutdown_connections 2023-10-26T05:56:50.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- 172.21.15.71:0/1097851946 wait complete. 2023-10-26T05:56:50.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 Processor -- start 2023-10-26T05:56:50.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- start start 2023-10-26T05:56:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f2514138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2514138de0 0x7f251413b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f251413cee0 0x7f251413d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251413d8a0 con 0x7f24fc0047d0 2023-10-26T05:56:50.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514139310 con 0x7f251413cee0 2023-10-26T05:56:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f251b16e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25141395f0 con 0x7f2514138de0 2023-10-26T05:56:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f2518f0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f2514138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f2518f0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f2514138610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49830/0 (socket says 172.21.15.71:49830) 2023-10-26T05:56:50.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.767+0000 7f2518f0a640 1 -- 172.21.15.71:0/2341910309 learned_addr learned my addr 172.21.15.71:0/2341910309 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:50.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2518f0a640 1 -- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2514138de0 msgr2=0x7f251413b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2518f0a640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2514138de0 0x7f251413b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2518f0a640 1 -- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f251413cee0 msgr2=0x7f251413d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f251970b640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f251413cee0 0x7f251413d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:50.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2518f0a640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f251413cee0 0x7f251413d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2518f0a640 1 -- 172.21.15.71:0/2341910309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25141398d0 con 0x7f24fc0047d0 2023-10-26T05:56:50.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2518f0a640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f2514138610 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f250800d640 tx=0x7f250800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:50.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080062e0 con 0x7f24fc0047d0 2023-10-26T05:56:50.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2508003620 con 0x7f24fc0047d0 2023-10-26T05:56:50.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2514139bb0 con 0x7f24fc0047d0 2023-10-26T05:56:50.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080039d0 con 0x7f24fc0047d0 2023-10-26T05:56:50.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2514138b50 con 0x7f24fc0047d0 2023-10-26T05:56:50.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514066b10 con 0x7f24fc0047d0 2023-10-26T05:56:50.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2508003b30 con 0x7f24fc0047d0 2023-10-26T05:56:50.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f25117fa640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25000664a0 0x7f2500068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:50.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2508084450 con 0x7f24fc0047d0 2023-10-26T05:56:50.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2513fff640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25000664a0 0x7f2500068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:50.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.771+0000 7f2513fff640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25000664a0 0x7f2500068960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f25040ac000 tx=0x7f24fc005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:50.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.775+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2508051bd0 con 0x7f24fc0047d0 2023-10-26T05:56:50.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.931+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f251406c7d0 con 0x7f24fc0047d0 2023-10-26T05:56:50.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.939+0000 7f25117fa640 1 -- 172.21.15.71:0/2341910309 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v34) v1 ==== 71+0+0 (secure 0 0 0) 0x7f2508014080 con 0x7f24fc0047d0 2023-10-26T05:56:50.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25000664a0 msgr2=0x7f2500068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25000664a0 0x7f2500068960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f25040ac000 tx=0x7f24fc005800 comp rx=0 tx=0).stop 2023-10-26T05:56:50.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 msgr2=0x7f2514138610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:50.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f2514138610 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f250800d640 tx=0x7f250800db00 comp rx=0 tx=0).stop 2023-10-26T05:56:50.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 shutdown_connections 2023-10-26T05:56:50.951 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f251413cee0 0x7f251413d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2514138de0 0x7f251413b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f25000664a0 0x7f2500068960 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 --2- 172.21.15.71:0/2341910309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24fc0047d0 0x7f2514138610 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 >> 172.21.15.71:0/2341910309 conn(0x7f251406b8c0 msgr2=0x7f251413e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:50.952 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 shutdown_connections 2023-10-26T05:56:50.953 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:50.943+0000 7f251b16e640 1 -- 172.21.15.71:0/2341910309 wait complete. 2023-10-26T05:56:50.968 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-10-26T05:56:51.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- 172.21.15.71:0/733820101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013d420 msgr2=0x7fda900ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- 172.21.15.71:0/733820101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013d420 0x7fda900ac740 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fda90155a00 tx=0x7fda8001dbd0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- 172.21.15.71:0/733820101 shutdown_connections 2023-10-26T05:56:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- 172.21.15.71:0/733820101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda9013dc50 0x7fda9015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- 172.21.15.71:0/733820101 >> v2:172.21.15.86:3300/0 conn(0x7fda9013d800 0x7fda9015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- 172.21.15.71:0/733820101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013d420 0x7fda900ac740 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- 172.21.15.71:0/733820101 >> 172.21.15.71:0/733820101 conn(0x7fda90069770 msgr2=0x7fda90069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- 172.21.15.71:0/733820101 shutdown_connections 2023-10-26T05:56:51.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- 172.21.15.71:0/733820101 wait complete. 2023-10-26T05:56:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 Processor -- start 2023-10-26T05:56:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- start start 2023-10-26T05:56:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda9013d420 0x7fda900b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda9013d800 0x7fda901524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 0x7fda901619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90160940 con 0x7fda9013dc50 2023-10-26T05:56:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda90160c20 con 0x7fda9013d800 2023-10-26T05:56:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda94d64640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda90160f00 con 0x7fda9013d420 2023-10-26T05:56:51.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 0x7fda901619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 0x7fda901619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49846/0 (socket says 172.21.15.71:49846) 2023-10-26T05:56:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 -- 172.21.15.71:0/104902280 learned_addr learned my addr 172.21.15.71:0/104902280 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8e59c640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda9013d420 0x7fda900b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8dd9b640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda9013d800 0x7fda901524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 -- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda9013d420 msgr2=0x7fda900b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda9013d420 0x7fda900b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 -- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda9013d800 msgr2=0x7fda901524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda9013d800 0x7fda901524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.083+0000 7fda8ed9d640 1 -- 172.21.15.71:0/104902280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8000a670 con 0x7fda9013dc50 2023-10-26T05:56:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda8ed9d640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 0x7fda901619e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fda8800ca60 tx=0x7fda8800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:51.098 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88007d90 con 0x7fda9013dc50 2023-10-26T05:56:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fda88004b30 con 0x7fda9013dc50 2023-10-26T05:56:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda901611e0 con 0x7fda9013dc50 2023-10-26T05:56:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88005020 con 0x7fda9013dc50 2023-10-26T05:56:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda900b1b10 con 0x7fda9013dc50 2023-10-26T05:56:51.099 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda90066b10 con 0x7fda9013dc50 2023-10-26T05:56:51.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda88005180 con 0x7fda9013dc50 2023-10-26T05:56:51.100 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda7effd640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fda70066470 0x7fda70068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fda88084430 con 0x7fda9013dc50 2023-10-26T05:56:51.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda8e59c640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fda70066470 0x7fda70068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.087+0000 7fda8e59c640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fda70066470 0x7fda70068930 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fda800b0000 tx=0x7fda8001c410 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:51.101 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.091+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fda88051b20 con 0x7fda9013dc50 2023-10-26T05:56:51.312 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.303+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fda900721e0 con 0x7fda9013dc50 2023-10-26T05:56:51.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.619+0000 7fda7effd640 1 -- 172.21.15.71:0/104902280 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v468) v1 ==== 132+0+0 (secure 0 0 0) 0x7fda8801d070 con 0x7fda9013dc50 2023-10-26T05:56:51.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fda70066470 msgr2=0x7fda70068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fda70066470 0x7fda70068930 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fda800b0000 tx=0x7fda8001c410 comp rx=0 tx=0).stop 2023-10-26T05:56:51.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 msgr2=0x7fda901619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 0x7fda901619e0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fda8800ca60 tx=0x7fda8800cf20 comp rx=0 tx=0).stop 2023-10-26T05:56:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 shutdown_connections 2023-10-26T05:56:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fda70066470 0x7fda70068930 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda9013dc50 0x7fda901619e0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda9013d800 0x7fda901524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 --2- 172.21.15.71:0/104902280 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda9013d420 0x7fda900b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 >> 172.21.15.71:0/104902280 conn(0x7fda90069770 msgr2=0x7fda9015a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 shutdown_connections 2023-10-26T05:56:51.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.623+0000 7fda94d64640 1 -- 172.21.15.71:0/104902280 wait complete. 2023-10-26T05:56:51.653 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-26T05:56:51.653 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2023-10-26T05:56:51.654 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T05:56:51.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:51.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 -- 172.21.15.71:0/2732786416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 msgr2=0x7f632415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 --2- 172.21.15.71:0/2732786416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f632415b3a0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f632408a140 tx=0x7f631801c810 comp rx=0 tx=0).stop 2023-10-26T05:56:51.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 -- 172.21.15.71:0/2732786416 shutdown_connections 2023-10-26T05:56:51.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 --2- 172.21.15.71:0/2732786416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240cd880 0x7f632415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 --2- 172.21.15.71:0/2732786416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f632415b3a0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 --2- 172.21.15.71:0/2732786416 >> v2:172.21.15.86:3300/0 conn(0x7f63240d0470 0x7f632406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 -- 172.21.15.71:0/2732786416 >> 172.21.15.71:0/2732786416 conn(0x7f63240cdd80 msgr2=0x7f63240ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:51.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 -- 172.21.15.71:0/2732786416 shutdown_connections 2023-10-26T05:56:51.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.771+0000 7f6329527640 1 -- 172.21.15.71:0/2732786416 wait complete. 2023-10-26T05:56:51.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 Processor -- start 2023-10-26T05:56:51.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- start start 2023-10-26T05:56:51.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240cd880 0x7f6324159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d0470 0x7f6324154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f63240c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6324155f00 con 0x7f63240d0850 2023-10-26T05:56:51.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63240c60e0 con 0x7f63240d0470 2023-10-26T05:56:51.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63240c9290 con 0x7f63240cd880 2023-10-26T05:56:51.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f63240c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f63240c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53808/0 (socket says 172.21.15.71:53808) 2023-10-26T05:56:51.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 -- 172.21.15.71:0/1709046697 learned_addr learned my addr 172.21.15.71:0/1709046697 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:51.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6322ffd640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240cd880 0x7f6324159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63227fc640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d0470 0x7f6324154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 -- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240cd880 msgr2=0x7f6324159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240cd880 0x7f6324159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 -- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d0470 msgr2=0x7f6324154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d0470 0x7f6324154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 -- 172.21.15.71:0/1709046697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6318009560 con 0x7f63240d0850 2023-10-26T05:56:51.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63227fc640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d0470 0x7f6324154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:51.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f63237fe640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f63240c83d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f631c00ca60 tx=0x7f631c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:51.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c00adc0 con 0x7f63240d0850 2023-10-26T05:56:51.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f631c004c40 con 0x7f63240d0850 2023-10-26T05:56:51.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63240c5ab0 con 0x7f63240d0850 2023-10-26T05:56:51.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631c005280 con 0x7f63240d0850 2023-10-26T05:56:51.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63240c5d70 con 0x7f63240d0850 2023-10-26T05:56:51.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.775+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6324066b10 con 0x7f63240d0850 2023-10-26T05:56:51.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.779+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f631c0053e0 con 0x7f63240d0850 2023-10-26T05:56:51.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.779+0000 7f630b7fe640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6310066470 0x7f6310068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:51.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.779+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f631c0843d0 con 0x7f63240d0850 2023-10-26T05:56:51.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.779+0000 7f6322ffd640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6310066470 0x7f6310068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:51.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.779+0000 7f6322ffd640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6310066470 0x7f6310068930 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6314005fd0 tx=0x7f631400a4e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:51.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.783+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f631c051b40 con 0x7f63240d0850 2023-10-26T05:56:51.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.935+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63240c6250 con 0x7f63240d0850 2023-10-26T05:56:51.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f630b7fe640 1 -- 172.21.15.71:0/1709046697 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+12913 (secure 0 0 0) 0x7f631c01d090 con 0x7f63240d0850 2023-10-26T05:56:51.946 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:51.946 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":163,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:56:50.607337+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/2112769040":"2023-10-27T05:56:47.504480+0000","172.21.15.71:6833/2112769040":"2023-10-27T05:56:47.504480+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:51.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6310066470 msgr2=0x7f6310068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6310066470 0x7f6310068930 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6314005fd0 tx=0x7f631400a4e0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 msgr2=0x7f63240c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f63240c83d0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f631c00ca60 tx=0x7f631c00cf20 comp rx=0 tx=0).stop 2023-10-26T05:56:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 shutdown_connections 2023-10-26T05:56:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6310066470 0x7f6310068930 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63240d0850 0x7f63240c83d0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63240d0470 0x7f6324154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 --2- 172.21.15.71:0/1709046697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63240cd880 0x7f6324159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 >> 172.21.15.71:0/1709046697 conn(0x7f63240cdd80 msgr2=0x7f63240b6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 shutdown_connections 2023-10-26T05:56:51.950 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:51.939+0000 7f6329527640 1 -- 172.21.15.71:0/1709046697 wait complete. 2023-10-26T05:56:51.966 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:56:51.966 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:56:52.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 -- 172.21.15.71:0/4282987287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 msgr2=0x7f797c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 --2- 172.21.15.71:0/4282987287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f797c004c50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f798c001ec0 tx=0x7f798c01f6b0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f799589c640 1 -- 172.21.15.71:0/4282987287 reap_dead start 2023-10-26T05:56:52.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 -- 172.21.15.71:0/4282987287 shutdown_connections 2023-10-26T05:56:52.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 -- 172.21.15.71:0/4282987287 >> 172.21.15.71:0/4282987287 conn(0x7f79900caa90 msgr2=0x7f79900caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:52.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 -- 172.21.15.71:0/4282987287 shutdown_connections 2023-10-26T05:56:52.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 -- 172.21.15.71:0/4282987287 wait complete. 2023-10-26T05:56:52.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 Processor -- start 2023-10-26T05:56:52.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.075+0000 7f79972ff640 1 -- start start 2023-10-26T05:56:52.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f7990141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990150440 0x7f79900c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7990154f90 0x7f7990155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7990155960 con 0x7f797c0047d0 2023-10-26T05:56:52.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799006b020 con 0x7f7990150440 2023-10-26T05:56:52.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79901429c0 con 0x7f7990154f90 2023-10-26T05:56:52.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f7990141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f7990141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53832/0 (socket says 172.21.15.71:53832) 2023-10-26T05:56:52.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 -- 172.21.15.71:0/3520171739 learned_addr learned my addr 172.21.15.71:0/3520171739 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:52.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799589c640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7990154f90 0x7f7990155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799489a640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990150440 0x7f79900c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 -- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7990154f90 msgr2=0x7f7990155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7990154f90 0x7f7990155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 -- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990150440 msgr2=0x7f79900c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990150440 0x7f79900c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 -- 172.21.15.71:0/3520171739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900c3490 con 0x7f797c0047d0 2023-10-26T05:56:52.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799489a640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990150440 0x7f79900c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:52.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799509b640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f7990141b80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7988010cc0 tx=0x7f798800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:52.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988011640 con 0x7f797c0047d0 2023-10-26T05:56:52.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7988015070 con 0x7f797c0047d0 2023-10-26T05:56:52.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799006ad40 con 0x7f797c0047d0 2023-10-26T05:56:52.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988011af0 con 0x7f797c0047d0 2023-10-26T05:56:52.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7990142650 con 0x7f797c0047d0 2023-10-26T05:56:52.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7990066d00 con 0x7f797c0047d0 2023-10-26T05:56:52.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7988011c50 con 0x7f797c0047d0 2023-10-26T05:56:52.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f7985ffb640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79740664a0 0x7f7974068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7988084750 con 0x7f797c0047d0 2023-10-26T05:56:52.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.079+0000 7f799489a640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79740664a0 0x7f7974068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.083+0000 7f799489a640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79740664a0 0x7f7974068960 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f798c0b0000 tx=0x7f798c0b2040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:52.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.083+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7988051f60 con 0x7f797c0047d0 2023-10-26T05:56:52.252 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.243+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79900cd740 con 0x7f797c0047d0 2023-10-26T05:56:52.253 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.243+0000 7f7985ffb640 1 -- 172.21.15.71:0/3520171739 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v163) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7988055580 con 0x7f797c0047d0 2023-10-26T05:56:52.253 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:56:52.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79740664a0 msgr2=0x7f7974068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.255 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79740664a0 0x7f7974068960 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f798c0b0000 tx=0x7f798c0b2040 comp rx=0 tx=0).stop 2023-10-26T05:56:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 msgr2=0x7f7990141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f7990141b80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7988010cc0 tx=0x7f798800e720 comp rx=0 tx=0).stop 2023-10-26T05:56:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 shutdown_connections 2023-10-26T05:56:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7990154f90 0x7f7990155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990150440 0x7f79900c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.256 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f79740664a0 0x7f7974068960 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 --2- 172.21.15.71:0/3520171739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f797c0047d0 0x7f7990141b80 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 >> 172.21.15.71:0/3520171739 conn(0x7f79900caa90 msgr2=0x7f79900704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 shutdown_connections 2023-10-26T05:56:52.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.247+0000 7f79972ff640 1 -- 172.21.15.71:0/3520171739 wait complete. 2023-10-26T05:56:52.271 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-26T05:56:52.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/3254425948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 msgr2=0x7fee4c159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 --2- 172.21.15.71:0/3254425948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c159810 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009900 tx=0x7fee3c01c8c0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/3254425948 shutdown_connections 2023-10-26T05:56:52.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 --2- 172.21.15.71:0/3254425948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c159810 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 --2- 172.21.15.71:0/3254425948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee4c13c2f0 0x7fee4c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/3254425948 >> 172.21.15.71:0/3254425948 conn(0x7fee4c069770 msgr2=0x7fee4c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:52.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/3254425948 shutdown_connections 2023-10-26T05:56:52.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/3254425948 wait complete. 2023-10-26T05:56:52.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 Processor -- start 2023-10-26T05:56:52.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- start start 2023-10-26T05:56:52.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee4c0b2890 0x7fee4c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee4c13c2f0 0x7fee4c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15c2e0 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15ad00 con 0x7fee4c13c2f0 2023-10-26T05:56:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15afe0 con 0x7fee4c0b2890 2023-10-26T05:56:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53858/0 (socket says 172.21.15.71:53858) 2023-10-26T05:56:52.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 -- 172.21.15.71:0/949967811 learned_addr learned my addr 172.21.15.71:0/949967811 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:52.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee51136640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee4c0b2890 0x7fee4c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee51937640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee4c13c2f0 0x7fee4c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 -- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee4c0b2890 msgr2=0x7fee4c156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee4c0b2890 0x7fee4c156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 -- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee4c13c2f0 msgr2=0x7fee4c15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee4c13c2f0 0x7fee4c15bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 -- 172.21.15.71:0/949967811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c009560 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee51937640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee4c13c2f0 0x7fee4c15bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:52.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee50935640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c153360 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fee4000c4d0 tx=0x7fee4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:52.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40005480 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fee400036f0 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c15b2c0 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40003b00 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c15b880 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.391+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c066b10 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.395+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fee40003c60 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.395+0000 7fee49ffb640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee38066470 0x7fee38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.395+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fee40083370 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.395+0000 7fee51136640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee38066470 0x7fee38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.395+0000 7fee51136640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee38066470 0x7fee38068930 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fee44005c50 tx=0x7fee44005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:52.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.399+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fee40050c40 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.615+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fee4c0721e0 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.619+0000 7fee49ffb640 1 -- 172.21.15.71:0/949967811 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v469) v1 ==== 134+0+0 (secure 0 0 0) 0x7fee400170c0 con 0x7fee4c0cb1e0 2023-10-26T05:56:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee38066470 msgr2=0x7fee38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee38066470 0x7fee38068930 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fee44005c50 tx=0x7fee44005be0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 msgr2=0x7fee4c153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c153360 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fee4000c4d0 tx=0x7fee4000c990 comp rx=0 tx=0).stop 2023-10-26T05:56:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 shutdown_connections 2023-10-26T05:56:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee4c13c2f0 0x7fee4c15bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fee38066470 0x7fee38068930 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee4c0cb1e0 0x7fee4c153360 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 --2- 172.21.15.71:0/949967811 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fee4c0b2890 0x7fee4c156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 >> 172.21.15.71:0/949967811 conn(0x7fee4c069770 msgr2=0x7fee4c158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:52.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 shutdown_connections 2023-10-26T05:56:52.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.623+0000 7fee5339a640 1 -- 172.21.15.71:0/949967811 wait complete. 2023-10-26T05:56:52.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:56:52.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/3454359613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d800 msgr2=0x7ff1c815b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 --2- 172.21.15.71:0/3454359613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d800 0x7ff1c815b4b0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff1c806db30 tx=0x7ff1bc01c7e0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/3454359613 shutdown_connections 2023-10-26T05:56:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 --2- 172.21.15.71:0/3454359613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d800 0x7ff1c815b4b0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 --2- 172.21.15.71:0/3454359613 >> v2:172.21.15.86:3300/0 conn(0x7ff1c813d420 0x7ff1c80ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/3454359613 >> 172.21.15.71:0/3454359613 conn(0x7ff1c8069770 msgr2=0x7ff1c8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/3454359613 shutdown_connections 2023-10-26T05:56:52.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/3454359613 wait complete. 2023-10-26T05:56:52.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 Processor -- start 2023-10-26T05:56:52.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- start start 2023-10-26T05:56:52.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 0x7ff1c80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1c813d800 0x7ff1c814ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c813dc50 0x7ff1c8156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8133540 con 0x7ff1c813d420 2023-10-26T05:56:52.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c814dda0 con 0x7ff1c813dc50 2023-10-26T05:56:52.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c80b1480 con 0x7ff1c813d800 2023-10-26T05:56:52.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 0x7ff1c80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 0x7ff1c80b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53880/0 (socket says 172.21.15.71:53880) 2023-10-26T05:56:52.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 -- 172.21.15.71:0/1595397907 learned_addr learned my addr 172.21.15.71:0/1595397907 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:52.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 -- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1c813d800 msgr2=0x7ff1c814ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1c813d800 0x7ff1c814ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 -- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c813dc50 msgr2=0x7ff1c8156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cd733640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c813dc50 0x7ff1c8156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c813dc50 0x7ff1c8156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 -- 172.21.15.71:0/1595397907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0094a0 con 0x7ff1c813d420 2023-10-26T05:56:52.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cd733640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c813dc50 0x7ff1c8156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:52.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1ccf32640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 0x7ff1c80b23f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800ddc0 tx=0x7ff1b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:52.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800e650 con 0x7ff1c813d420 2023-10-26T05:56:52.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff1b800e8a0 con 0x7ff1c813d420 2023-10-26T05:56:52.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800ecb0 con 0x7ff1c813d420 2023-10-26T05:56:52.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c80b1760 con 0x7ff1c813d420 2023-10-26T05:56:52.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.763+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c80b1d20 con 0x7ff1c813d420 2023-10-26T05:56:52.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.767+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c8066b10 con 0x7ff1c813d420 2023-10-26T05:56:52.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.767+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1b8016440 con 0x7ff1c813d420 2023-10-26T05:56:52.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.767+0000 7ff1c57fa640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1b40664a0 0x7ff1b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:52.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.767+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff1b8084380 con 0x7ff1c813d420 2023-10-26T05:56:52.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.767+0000 7ff1c7fff640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1b40664a0 0x7ff1b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:52.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.767+0000 7ff1c7fff640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1b40664a0 0x7ff1b4068960 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0af000 tx=0x7ff1bc019580 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:52.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.771+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff1b8051c40 con 0x7ff1c813d420 2023-10-26T05:56:52.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.923+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7ff1c80721e0 con 0x7ff1c813d420 2023-10-26T05:56:52.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.923+0000 7ff1c57fa640 1 -- 172.21.15.71:0/1595397907 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v34) v1 ==== 59+0+1116 (secure 0 0 0) 0x7ff1b801b470 con 0x7ff1c813d420 2023-10-26T05:56:52.934 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:56:52.934 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:56:52.934 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:56:52.934 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:56:52.935 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:56:52.935 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:56:52.935 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:56:52.935 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:56:52.935 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:56:52.935 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:56:52.936 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:56:52.936 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:56:52.936 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:56:52.936 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:56:52.936 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:56:52.937 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:56:52.937 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:56:52.937 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:56:52.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1b40664a0 msgr2=0x7ff1b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1b40664a0 0x7ff1b4068960 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0af000 tx=0x7ff1bc019580 comp rx=0 tx=0).stop 2023-10-26T05:56:52.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 msgr2=0x7ff1c80b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:52.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 0x7ff1c80b23f0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800ddc0 tx=0x7ff1b800a620 comp rx=0 tx=0).stop 2023-10-26T05:56:52.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 shutdown_connections 2023-10-26T05:56:52.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1c813dc50 0x7ff1c8156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1c813d800 0x7ff1c814ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff1b40664a0 0x7ff1b4068960 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 --2- 172.21.15.71:0/1595397907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c813d420 0x7ff1c80b23f0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:52.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 >> 172.21.15.71:0/1595397907 conn(0x7ff1c8069770 msgr2=0x7ff1c8156250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:52.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 shutdown_connections 2023-10-26T05:56:52.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:52.927+0000 7ff1cf196640 1 -- 172.21.15.71:0/1595397907 wait complete. 2023-10-26T05:56:52.951 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:56:53.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/735905639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc200039b0 msgr2=0x7fbc20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/735905639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc200039b0 0x7fbc20003dd0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fbc28001680 tx=0x7fbc2801c760 comp rx=0 tx=0).stop 2023-10-26T05:56:53.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc37fff640 1 -- 172.21.15.71:0/735905639 reap_dead start 2023-10-26T05:56:53.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/735905639 shutdown_connections 2023-10-26T05:56:53.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/735905639 >> 172.21.15.71:0/735905639 conn(0x7fbc380aec60 msgr2=0x7fbc380af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:53.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/735905639 shutdown_connections 2023-10-26T05:56:53.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/735905639 wait complete. 2023-10-26T05:56:53.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 Processor -- start 2023-10-26T05:56:53.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- start start 2023-10-26T05:56:53.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc200039b0 0x7fbc380ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 0x7fbc380ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc38139db0 0x7fbc3813a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380acea0 con 0x7fbc38156bf0 2023-10-26T05:56:53.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3813a770 con 0x7fbc200039b0 2023-10-26T05:56:53.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380ac250 con 0x7fbc38139db0 2023-10-26T05:56:53.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 0x7fbc380ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 0x7fbc380ac960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53890/0 (socket says 172.21.15.71:53890) 2023-10-26T05:56:53.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 -- 172.21.15.71:0/1336230705 learned_addr learned my addr 172.21.15.71:0/1336230705 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc377fe640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc200039b0 0x7fbc380ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc37fff640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc38139db0 0x7fbc3813a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 -- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc38139db0 msgr2=0x7fbc3813a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc38139db0 0x7fbc3813a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 -- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc200039b0 msgr2=0x7fbc380ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc200039b0 0x7fbc380ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 -- 172.21.15.71:0/1336230705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380adb70 con 0x7fbc38156bf0 2023-10-26T05:56:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc36ffd640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 0x7fbc380ac960 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d7b0 tx=0x7fbc2c00dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00de60 con 0x7fbc38156bf0 2023-10-26T05:56:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbc2c0052e0 con 0x7fbc38156bf0 2023-10-26T05:56:53.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc380abdc0 con 0x7fbc38156bf0 2023-10-26T05:56:53.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c00b8a0 con 0x7fbc38156bf0 2023-10-26T05:56:53.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.071+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc380ab490 con 0x7fbc38156bf0 2023-10-26T05:56:53.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.075+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc38066b10 con 0x7fbc38156bf0 2023-10-26T05:56:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.075+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc2c00ba00 con 0x7fbc38156bf0 2023-10-26T05:56:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.075+0000 7fbc1ffff640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc24066470 0x7fbc24068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.075+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbc2c0835f0 con 0x7fbc38156bf0 2023-10-26T05:56:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.075+0000 7fbc377fe640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc24066470 0x7fbc24068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.075+0000 7fbc377fe640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc24066470 0x7fbc24068930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbc200057b0 tx=0x7fbc28019500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:53.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.079+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbc2c050e50 con 0x7fbc38156bf0 2023-10-26T05:56:53.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.219+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbc380abf50 con 0x7fbc38156bf0 2023-10-26T05:56:53.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.223+0000 7fbc1ffff640 1 -- 172.21.15.71:0/1336230705 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v35)=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc2c00bcb0 con 0x7fbc38156bf0 2023-10-26T05:56:53.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc24066470 msgr2=0x7fbc24068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc24066470 0x7fbc24068930 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbc200057b0 tx=0x7fbc28019500 comp rx=0 tx=0).stop 2023-10-26T05:56:53.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 msgr2=0x7fbc380ac960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 0x7fbc380ac960 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00d7b0 tx=0x7fbc2c00dc70 comp rx=0 tx=0).stop 2023-10-26T05:56:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 shutdown_connections 2023-10-26T05:56:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbc38139db0 0x7fbc3813a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbc24066470 0x7fbc24068930 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc38156bf0 0x7fbc380ac960 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 --2- 172.21.15.71:0/1336230705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc200039b0 0x7fbc380ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 >> 172.21.15.71:0/1336230705 conn(0x7fbc380aec60 msgr2=0x7fbc380b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:53.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 shutdown_connections 2023-10-26T05:56:53.238 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.227+0000 7fbc3dd64640 1 -- 172.21.15.71:0/1336230705 wait complete. 2023-10-26T05:56:53.251 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:56:53.252 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:56:53.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:56:53.353 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:56:53.353 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:56:53.354 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:53.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:56:53.360 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:56:53.360 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:53.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:56:53.366 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:56:53.366 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:53.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:56:53.406 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:56:53.406 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:56:53.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:56:53.411 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:56:53.412 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:56:53.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/456495566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 msgr2=0x7f4ee0164190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 --2- 172.21.15.71:0/456495566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee0164190 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009bc0 tx=0x7f4ed801cb40 comp rx=0 tx=0).stop 2023-10-26T05:56:53.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/456495566 shutdown_connections 2023-10-26T05:56:53.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 --2- 172.21.15.71:0/456495566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee0164190 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/456495566 >> 172.21.15.71:0/456495566 conn(0x7f4ee0069770 msgr2=0x7f4ee0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:53.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/456495566 shutdown_connections 2023-10-26T05:56:53.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/456495566 wait complete. 2023-10-26T05:56:53.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 Processor -- start 2023-10-26T05:56:53.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- start start 2023-10-26T05:56:53.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee00b2a60 0x7f4ee013dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee01373c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee013c2f0 0x7f4ee013ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0142960 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee013fa70 con 0x7f4ee00b2a60 2023-10-26T05:56:53.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0137900 con 0x7f4ee013c2f0 2023-10-26T05:56:53.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee01373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee01373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53908/0 (socket says 172.21.15.71:53908) 2023-10-26T05:56:53.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 -- 172.21.15.71:0/918234288 learned_addr learned my addr 172.21.15.71:0/918234288 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:53.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee6406640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee00b2a60 0x7f4ee013dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee6c07640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee013c2f0 0x7f4ee013ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 -- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee013c2f0 msgr2=0x7f4ee013ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee013c2f0 0x7f4ee013ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 -- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee00b2a60 msgr2=0x7f4ee013dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee00b2a60 0x7f4ee013dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 -- 172.21.15.71:0/918234288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8009820 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee5c05640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee01373c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000c480 tx=0x7f4ed000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:53.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0012830 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4ed0012aa0 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee013f270 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0011380 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.567+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee013ffa0 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.571+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee0066b10 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.571+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4ed00114e0 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.571+0000 7f4ed6ffd640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ec40664a0 0x7f4ec4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.571+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ed0083c20 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.571+0000 7f4ee6406640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ec40664a0 0x7f4ec4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.571+0000 7f4ee6406640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ec40664a0 0x7f4ec4068960 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006ad60 tx=0x7f4edc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:53.587 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.575+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4ed0051430 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.747+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ee00721e0 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ed6ffd640 1 -- 172.21.15.71:0/918234288 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f4ed0054a50 con 0x7f4ee00cb1e0 2023-10-26T05:56:53.758 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:53.759 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3267374593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3267374593},{"type":"v1","addr":"172.21.15.86:6837","nonce":3267374593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6484,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3715929246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3715929246},{"type":"v1","addr":"172.21.15.86:6836","nonce":3715929246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6834/3064913504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3064913504},{"type":"v1","addr":"172.21.15.71:6834","nonce":3064913504}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":8868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1450952645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6833","nonce":1450952645},{"type":"v1","addr":"172.21.15.71:6835","nonce":1450952645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2023-10-26T05:56:53.759 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 60 2023-10-26T05:56:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ec40664a0 msgr2=0x7f4ec4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ec40664a0 0x7f4ec4068960 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006ad60 tx=0x7f4edc009040 comp rx=0 tx=0).stop 2023-10-26T05:56:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 msgr2=0x7f4ee01373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee01373c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000c480 tx=0x7f4ed000c940 comp rx=0 tx=0).stop 2023-10-26T05:56:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 shutdown_connections 2023-10-26T05:56:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4ee013c2f0 0x7f4ee013ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4ec40664a0 0x7f4ec4068960 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee00cb1e0 0x7f4ee01373c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 --2- 172.21.15.71:0/918234288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee00b2a60 0x7f4ee013dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 >> 172.21.15.71:0/918234288 conn(0x7f4ee0069770 msgr2=0x7f4ee00ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 shutdown_connections 2023-10-26T05:56:53.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.751+0000 7f4ee866a640 1 -- 172.21.15.71:0/918234288 wait complete. 2023-10-26T05:56:53.775 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:56:53.776 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:56:53.776 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:56:53.776 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:56:53.777 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:56:53.777 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:56:53.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:56:53.777 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:56:53.778 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:56:53.778 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:56:53.778 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:56:53.778 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:56:53.781 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:56:53.781 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:56:53.782 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:56:53.782 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:56:53.787 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:56:53.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:56:53.788 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:56:53.788 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:56:53.799 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:56:53.799 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:56:53.801 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:56:53.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:56:53.831 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:56:53.832 DEBUG:teuthology.parallel:result is None 2023-10-26T05:56:53.832 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:56:53.833 DEBUG:teuthology.parallel:result is None 2023-10-26T05:56:53.844 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:56:53.844 DEBUG:teuthology.parallel:result is None 2023-10-26T05:56:53.845 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:56:53.846 DEBUG:teuthology.parallel:result is None 2023-10-26T05:56:53.846 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:56:53.887 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:56:53.897 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:56:53.916 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:56:53.916 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:56:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1751442980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3740039b0 msgr2=0x7fc374003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 --2- 172.21.15.71:0/1751442980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3740039b0 0x7fc374003dd0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fc384009b30 tx=0x7fc38401d1c0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 -- 172.21.15.71:0/1751442980 reap_dead start 2023-10-26T05:56:53.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1751442980 shutdown_connections 2023-10-26T05:56:53.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1751442980 >> 172.21.15.71:0/1751442980 conn(0x7fc3880b4510 msgr2=0x7fc3880b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:53.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1751442980 shutdown_connections 2023-10-26T05:56:53.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1751442980 wait complete. 2023-10-26T05:56:53.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 Processor -- start 2023-10-26T05:56:53.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- start start 2023-10-26T05:56:53.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3740039b0 0x7fc388148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc38814c860 0x7fc38814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 0x7fc38815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815bed0 con 0x7fc38815b510 2023-10-26T05:56:53.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38814e380 con 0x7fc38814c860 2023-10-26T05:56:53.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38814d2e0 con 0x7fc3740039b0 2023-10-26T05:56:53.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 0x7fc38815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 0x7fc38815b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53964/0 (socket says 172.21.15.71:53964) 2023-10-26T05:56:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 -- 172.21.15.71:0/1975452778 learned_addr learned my addr 172.21.15.71:0/1975452778 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38e232640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3740039b0 0x7fc388148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38da31640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc38814c860 0x7fc38814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 -- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3740039b0 msgr2=0x7fc388148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3740039b0 0x7fc388148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 -- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc38814c860 msgr2=0x7fc38814cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc38814c860 0x7fc38814cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 -- 172.21.15.71:0/1975452778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38814d5c0 con 0x7fc38815b510 2023-10-26T05:56:53.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc38ea33640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 0x7fc38815b990 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fc3880cd420 tx=0x7fc3800019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011870 con 0x7fc38815b510 2023-10-26T05:56:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc380011ae0 con 0x7fc38815b510 2023-10-26T05:56:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38814d8a0 con 0x7fc38815b510 2023-10-26T05:56:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800103c0 con 0x7fc38815b510 2023-10-26T05:56:53.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.955+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38814de60 con 0x7fc38815b510 2023-10-26T05:56:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.959+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc388066b10 con 0x7fc38815b510 2023-10-26T05:56:53.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.959+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc380010520 con 0x7fc38815b510 2023-10-26T05:56:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.959+0000 7fc37effd640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc36c066470 0x7fc36c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.959+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc380084750 con 0x7fc38815b510 2023-10-26T05:56:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.959+0000 7fc38e232640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc36c066470 0x7fc36c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.959+0000 7fc38e232640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc36c066470 0x7fc36c068930 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc3840ae000 tx=0x7fc384019f60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:53.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:53.963+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc380051f90 con 0x7fc38815b510 2023-10-26T05:56:54.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:54.131+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc38806a7b0 con 0x7fc38815b510 2023-10-26T05:56:55.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.259+0000 7fc37effd640 1 -- 172.21.15.71:0/1975452778 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v168) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc380015080 con 0x7fc38815b510 2023-10-26T05:56:55.270 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:56:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc36c066470 msgr2=0x7fc36c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc36c066470 0x7fc36c068930 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc3840ae000 tx=0x7fc384019f60 comp rx=0 tx=0).stop 2023-10-26T05:56:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 msgr2=0x7fc38815b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 0x7fc38815b990 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fc3880cd420 tx=0x7fc3800019c0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 shutdown_connections 2023-10-26T05:56:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc36c066470 0x7fc36c068930 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc38815b510 0x7fc38815b990 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc38814c860 0x7fc38814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 --2- 172.21.15.71:0/1975452778 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc3740039b0 0x7fc388148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 >> 172.21.15.71:0/1975452778 conn(0x7fc3880b4510 msgr2=0x7fc3880b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:55.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 shutdown_connections 2023-10-26T05:56:55.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.263+0000 7fc390496640 1 -- 172.21.15.71:0/1975452778 wait complete. 2023-10-26T05:56:55.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:56:55.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/4293450443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b0158250 msgr2=0x7fe3b006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/4293450443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b0158250 0x7fe3b006b110 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009bc0 tx=0x7fe3a001cb60 comp rx=0 tx=0).stop 2023-10-26T05:56:55.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/4293450443 shutdown_connections 2023-10-26T05:56:55.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/4293450443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00ad750 0x7fe3b00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/4293450443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b0158250 0x7fe3b006b110 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/4293450443 >> 172.21.15.71:0/4293450443 conn(0x7fe3b0071650 msgr2=0x7fe3b00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:55.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/4293450443 shutdown_connections 2023-10-26T05:56:55.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/4293450443 wait complete. 2023-10-26T05:56:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 Processor -- start 2023-10-26T05:56:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- start start 2023-10-26T05:56:55.470 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 0x7fe3b014c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00cd740 0x7fe3b014ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b0158250 0x7fe3b0140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b014c6b0 con 0x7fe3b00ad750 2023-10-26T05:56:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b014d350 con 0x7fe3b0158250 2023-10-26T05:56:55.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b0155c00 con 0x7fe3b00cd740 2023-10-26T05:56:55.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 0x7fe3b014c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 0x7fe3b014c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53970/0 (socket says 172.21.15.71:53970) 2023-10-26T05:56:55.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 -- 172.21.15.71:0/2146978131 learned_addr learned my addr 172.21.15.71:0/2146978131 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:55.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3affff640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b0158250 0x7fe3b0140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3aeffd640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00cd740 0x7fe3b014ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 -- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00cd740 msgr2=0x7fe3b014ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00cd740 0x7fe3b014ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 -- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b0158250 msgr2=0x7fe3b0140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b0158250 0x7fe3b0140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 -- 172.21.15.71:0/2146978131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0009820 con 0x7fe3b00ad750 2023-10-26T05:56:55.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3aeffd640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00cd740 0x7fe3b014ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3af7fe640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 0x7fe3b014c170 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe3a001b210 tx=0x7fe3a00a5ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0004c50 con 0x7fe3b00ad750 2023-10-26T05:56:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe3a0004000 con 0x7fe3b00ad750 2023-10-26T05:56:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b01408c0 con 0x7fe3b00ad750 2023-10-26T05:56:55.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0004ea0 con 0x7fe3b00ad750 2023-10-26T05:56:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.459+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b01555b0 con 0x7fe3b00ad750 2023-10-26T05:56:55.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.463+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0066b10 con 0x7fe3b00ad750 2023-10-26T05:56:55.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.463+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe3a00b4040 con 0x7fe3b00ad750 2023-10-26T05:56:55.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.463+0000 7fe3a7fff640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3900664a0 0x7fe390068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.463+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe3a011ddc0 con 0x7fe3b00ad750 2023-10-26T05:56:55.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.463+0000 7fe3aeffd640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3900664a0 0x7fe390068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.463+0000 7fe3aeffd640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3900664a0 0x7fe390068960 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe398005fd0 tx=0x7fe398005d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:55.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.467+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe3a00eb590 con 0x7fe3b00ad750 2023-10-26T05:56:55.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe3b00721e0 con 0x7fe3b00ad750 2023-10-26T05:56:55.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3a7fff640 1 -- 172.21.15.71:0/2146978131 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe3a00b2020 con 0x7fe3b00ad750 2023-10-26T05:56:55.688 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:55.688 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:56:55.688 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:56:55.689 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.689 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:56:55.689 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:56:55.689 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:56:55.690 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:56:55.690 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.690 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.690 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.691 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:56:55.691 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:56:55.691 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.691 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.691 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:56:55.691 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:56:55.692 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.692 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.692 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:56:55.693 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:56:55.693 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.693 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.693 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:56:55.693 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:56:55.693 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.694 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.694 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.694 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:56:55.694 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:56:55.695 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:56:55.695 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:56:55.695 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.695 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.696 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:56:55.696 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:56:55.696 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.696 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.697 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.697 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.697 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.697 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.697 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:56:55.698 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:56:55.698 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.698 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.698 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.698 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:56:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:56:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.700 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.700 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.700 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.700 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.700 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.701 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:56:55.701 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:56:55.701 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.701 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.701 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.702 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.702 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.702 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:56:55.703 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:56:55.703 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.703 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.703 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.703 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.703 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.704 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.704 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:56:55.704 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:56:55.704 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.704 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.705 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.705 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.705 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.705 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.705 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:56:55.706 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:56:55.706 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.706 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.706 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.706 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.707 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.707 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:56:55.707 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:56:55.707 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.708 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:56:55.708 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:56:55.708 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.708 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.708 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.709 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:56:55.709 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:56:55.709 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.709 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.709 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:56:55.709 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:56:55.710 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.710 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.710 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:56:55.710 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:56:55.711 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.711 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.711 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:56:55.711 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:56:55.711 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:56:55.712 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.712 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.712 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.712 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:56:55.712 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:56:55.712 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.713 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.713 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:56:55.713 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:56:55.713 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:56:55.713 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.714 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.714 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:56:55.714 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:56:55.714 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.714 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:56:55.715 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:56:55.715 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:56:55.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:56:55.715 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.715 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.716 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:56:55.716 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:56:55.716 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:56:55.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:56:55.716 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:56:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:56:55.717 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:56:55.718 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.718 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:56:55.718 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:56:55.718 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.718 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:56:55.719 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:55.719 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:56:55.719 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:56:55.719 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:56:55.719 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:56:55.720 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:56:55.720 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:56:55.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:55.720 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:56:55.720 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:56:55.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3900664a0 msgr2=0x7fe390068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3900664a0 0x7fe390068960 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe398005fd0 tx=0x7fe398005d40 comp rx=0 tx=0).stop 2023-10-26T05:56:55.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 msgr2=0x7fe3b014c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 0x7fe3b014c170 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe3a001b210 tx=0x7fe3a00a5ad0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 shutdown_connections 2023-10-26T05:56:55.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3b0158250 0x7fe3b0140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe3b00cd740 0x7fe3b014ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3900664a0 0x7fe390068960 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 --2- 172.21.15.71:0/2146978131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b00ad750 0x7fe3b014c170 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 >> 172.21.15.71:0/2146978131 conn(0x7fe3b0071650 msgr2=0x7fe3b015a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:55.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 shutdown_connections 2023-10-26T05:56:55.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.679+0000 7fe3b60b5640 1 -- 172.21.15.71:0/2146978131 wait complete. 2023-10-26T05:56:55.725 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:55.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 -- 172.21.15.71:0/2893102069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c003f50 msgr2=0x7fc79c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/2893102069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c003f50 0x7fc79c0047d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc79800baf0 tx=0x7fc79801c890 comp rx=0 tx=0).stop 2023-10-26T05:56:55.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7af59e640 1 -- 172.21.15.71:0/2893102069 reap_dead start 2023-10-26T05:56:55.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 -- 172.21.15.71:0/2893102069 shutdown_connections 2023-10-26T05:56:55.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 -- 172.21.15.71:0/2893102069 >> 172.21.15.71:0/2893102069 conn(0x7fc7b0069770 msgr2=0x7fc7b0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:55.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 -- 172.21.15.71:0/2893102069 shutdown_connections 2023-10-26T05:56:55.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 -- 172.21.15.71:0/2893102069 wait complete. 2023-10-26T05:56:55.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 Processor -- start 2023-10-26T05:56:55.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.843+0000 7fc7b50c7640 1 -- start start 2023-10-26T05:56:55.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b0132140 0x7fc7b0136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b014ce90 0x7fc7b0137370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 0x7fc7b0149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0132f90 con 0x7fc7b01378b0 2023-10-26T05:56:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0135ae0 con 0x7fc7b014ce90 2023-10-26T05:56:55.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b01ed7c0 con 0x7fc7b0132140 2023-10-26T05:56:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 0x7fc7b0149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 0x7fc7b0149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53998/0 (socket says 172.21.15.71:53998) 2023-10-26T05:56:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 -- 172.21.15.71:0/4278048064 learned_addr learned my addr 172.21.15.71:0/4278048064 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7aed9d640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b0132140 0x7fc7b0136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a659c640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b014ce90 0x7fc7b0137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 -- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b0132140 msgr2=0x7fc7b0136e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b0132140 0x7fc7b0136e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 -- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b014ce90 msgr2=0x7fc7b0137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b014ce90 0x7fc7b0137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:55.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 -- 172.21.15.71:0/4278048064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b01329d0 con 0x7fc7b01378b0 2023-10-26T05:56:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7aed9d640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b0132140 0x7fc7b0136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:56:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7af59e640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 0x7fc7b0149360 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0073060 tx=0x7fc7a8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a801a030 con 0x7fc7b01378b0 2023-10-26T05:56:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc7a8011ed0 con 0x7fc7b01378b0 2023-10-26T05:56:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0132cb0 con 0x7fc7b01378b0 2023-10-26T05:56:55.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a800fe50 con 0x7fc7b01378b0 2023-10-26T05:56:55.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b0135800 con 0x7fc7b01378b0 2023-10-26T05:56:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0066b10 con 0x7fc7b01378b0 2023-10-26T05:56:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc7a801d070 con 0x7fc7b01378b0 2023-10-26T05:56:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a7fff640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc790066470 0x7fc790068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7a801b070 con 0x7fc7b01378b0 2023-10-26T05:56:55.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.847+0000 7fc7aed9d640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc790066470 0x7fc790068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.851+0000 7fc7aed9d640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc790066470 0x7fc790068930 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000fdf0 tx=0x7fc7a0011040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:55.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:55.851+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc7a8054a80 con 0x7fc7b01378b0 2023-10-26T05:56:56.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.007+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7b0135380 con 0x7fc7b01378b0 2023-10-26T05:56:56.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7a7fff640 1 -- 172.21.15.71:0/4278048064 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+12784 (secure 0 0 0) 0x7fc7b0135380 con 0x7fc7b01378b0 2023-10-26T05:56:56.019 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:56.019 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":168,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:56:55.262618+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":153,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:56:56.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc790066470 msgr2=0x7fc790068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc790066470 0x7fc790068930 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000fdf0 tx=0x7fc7a0011040 comp rx=0 tx=0).stop 2023-10-26T05:56:56.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 msgr2=0x7fc7b0149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 0x7fc7b0149360 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0073060 tx=0x7fc7a8011ce0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 shutdown_connections 2023-10-26T05:56:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc790066470 0x7fc790068930 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7b01378b0 0x7fc7b0149360 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7b014ce90 0x7fc7b0137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.022 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 --2- 172.21.15.71:0/4278048064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc7b0132140 0x7fc7b0136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 >> 172.21.15.71:0/4278048064 conn(0x7fc7b0069770 msgr2=0x7fc7b013bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:56.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 shutdown_connections 2023-10-26T05:56:56.023 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.011+0000 7fc7b50c7640 1 -- 172.21.15.71:0/4278048064 wait complete. 2023-10-26T05:56:56.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:56:56.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:56:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/283882959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 msgr2=0x7f54fc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 --2- 172.21.15.71:0/283882959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f54fc004c50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f55080016b0 tx=0x7f550801c7a0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5512d9d640 1 -- 172.21.15.71:0/283882959 reap_dead start 2023-10-26T05:56:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/283882959 shutdown_connections 2023-10-26T05:56:56.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/283882959 >> 172.21.15.71:0/283882959 conn(0x7f55140aec60 msgr2=0x7f55140af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/283882959 shutdown_connections 2023-10-26T05:56:56.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/283882959 wait complete. 2023-10-26T05:56:56.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 Processor -- start 2023-10-26T05:56:56.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- start start 2023-10-26T05:56:56.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f551413a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f551414b4b0 0x7f5514149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5514147920 0x7f55140cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551414b020 con 0x7f54fc0047d0 2023-10-26T05:56:56.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551414a410 con 0x7f5514147920 2023-10-26T05:56:56.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551414a6f0 con 0x7f551414b4b0 2023-10-26T05:56:56.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f551413a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f551413a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54004/0 (socket says 172.21.15.71:54004) 2023-10-26T05:56:56.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 -- 172.21.15.71:0/4177433558 learned_addr learned my addr 172.21.15.71:0/4177433558 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:56.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5512d9d640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5514147920 0x7f55140cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5511d9b640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f551414b4b0 0x7f5514149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 -- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f551414b4b0 msgr2=0x7f5514149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f551414b4b0 0x7f5514149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 -- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5514147920 msgr2=0x7f55140cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5514147920 0x7f55140cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 -- 172.21.15.71:0/4177433558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551414a9d0 con 0x7f54fc0047d0 2023-10-26T05:56:56.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5511d9b640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f551414b4b0 0x7f5514149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:56.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f551259c640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f551413a5e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f550400f5a0 tx=0x7f550400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:56.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55040118a0 con 0x7f54fc0047d0 2023-10-26T05:56:56.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5504011af0 con 0x7f54fc0047d0 2023-10-26T05:56:56.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551414acb0 con 0x7f54fc0047d0 2023-10-26T05:56:56.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504012470 con 0x7f54fc0047d0 2023-10-26T05:56:56.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.159+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5514149fc0 con 0x7f54fc0047d0 2023-10-26T05:56:56.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.163+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5514066b10 con 0x7f54fc0047d0 2023-10-26T05:56:56.175 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.163+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55040125d0 con 0x7f54fc0047d0 2023-10-26T05:56:56.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.163+0000 7f5502ffd640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54f4066470 0x7f54f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.163+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55040872e0 con 0x7f54fc0047d0 2023-10-26T05:56:56.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.163+0000 7f5511d9b640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54f4066470 0x7f54f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.163+0000 7f5511d9b640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54f4066470 0x7f54f4068930 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f55080a9000 tx=0x7f5508019540 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:56.176 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.167+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5504054ab0 con 0x7f54fc0047d0 2023-10-26T05:56:56.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.319+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55140721e0 con 0x7f54fc0047d0 2023-10-26T05:56:56.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.319+0000 7f5502ffd640 1 -- 172.21.15.71:0/4177433558 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v168) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55040580d0 con 0x7f54fc0047d0 2023-10-26T05:56:56.329 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:56:56.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54f4066470 msgr2=0x7f54f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54f4066470 0x7f54f4068930 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f55080a9000 tx=0x7f5508019540 comp rx=0 tx=0).stop 2023-10-26T05:56:56.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 msgr2=0x7f551413a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f551413a5e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f550400f5a0 tx=0x7f550400df90 comp rx=0 tx=0).stop 2023-10-26T05:56:56.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 shutdown_connections 2023-10-26T05:56:56.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5514147920 0x7f55140cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f551414b4b0 0x7f5514149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f54f4066470 0x7f54f4068930 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 --2- 172.21.15.71:0/4177433558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54fc0047d0 0x7f551413a5e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 >> 172.21.15.71:0/4177433558 conn(0x7f55140aec60 msgr2=0x7f55140b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:56.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 shutdown_connections 2023-10-26T05:56:56.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.323+0000 7f55189d8640 1 -- 172.21.15.71:0/4177433558 wait complete. 2023-10-26T05:56:56.347 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:56:56.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/184276293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1710005190 msgr2=0x7f171000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 --2- 172.21.15.71:0/184276293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1710005190 0x7f171000b990 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1720009e30 tx=0x7f172001d190 comp rx=0 tx=0).stop 2023-10-26T05:56:56.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/184276293 shutdown_connections 2023-10-26T05:56:56.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 --2- 172.21.15.71:0/184276293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1710005190 0x7f171000b990 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/184276293 >> 172.21.15.71:0/184276293 conn(0x7f1728069770 msgr2=0x7f1728069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:56.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/184276293 shutdown_connections 2023-10-26T05:56:56.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/184276293 wait complete. 2023-10-26T05:56:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 Processor -- start 2023-10-26T05:56:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- start start 2023-10-26T05:56:56.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1710005190 0x7f172813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 0x7f17281531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17280acb70 0x7f17280ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280ad540 con 0x7f172812a940 2023-10-26T05:56:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728153720 con 0x7f17280acb70 2023-10-26T05:56:56.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1728140c60 con 0x7f1710005190 2023-10-26T05:56:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 0x7f17281531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 0x7f17281531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54018/0 (socket says 172.21.15.71:54018) 2023-10-26T05:56:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 -- 172.21.15.71:0/1886197757 learned_addr learned my addr 172.21.15.71:0/1886197757 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1726d9d640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17280acb70 0x7f17280ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172659c640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1710005190 0x7f172813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 -- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1710005190 msgr2=0x7f172813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1710005190 0x7f172813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 -- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17280acb70 msgr2=0x7f17280ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17280acb70 0x7f17280ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 -- 172.21.15.71:0/1886197757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728140050 con 0x7f172812a940 2023-10-26T05:56:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1725d9b640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 0x7f17281531e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f171c00d560 tx=0x7f171c00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:56.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c0062e0 con 0x7f172812a940 2023-10-26T05:56:56.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f171c006530 con 0x7f172812a940 2023-10-26T05:56:56.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1728140330 con 0x7f172812a940 2023-10-26T05:56:56.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c003840 con 0x7f172812a940 2023-10-26T05:56:56.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.451+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17281408f0 con 0x7f172812a940 2023-10-26T05:56:56.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.455+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728066b10 con 0x7f172812a940 2023-10-26T05:56:56.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.455+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f171c0039a0 con 0x7f172812a940 2023-10-26T05:56:56.467 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.455+0000 7f1716ffd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1708066470 0x7f1708068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.455+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f171c084380 con 0x7f172812a940 2023-10-26T05:56:56.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.455+0000 7f172659c640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1708066470 0x7f1708068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.455+0000 7f172659c640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1708066470 0x7f1708068930 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f17180017b0 tx=0x7f1718001c80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:56.468 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.459+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f171c051ac0 con 0x7f172812a940 2023-10-26T05:56:56.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.611+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f17280721e0 con 0x7f172812a940 2023-10-26T05:56:56.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.611+0000 7f1716ffd640 1 -- 172.21.15.71:0/1886197757 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v168) v1 ==== 84+0+65 (secure 0 0 0) 0x7f171c014020 con 0x7f172812a940 2023-10-26T05:56:56.622 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:56:56.622 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:56:56.622 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:56:56.622 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:56:56.623 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:56:56.623 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:56:56.623 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:56:56.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1708066470 msgr2=0x7f1708068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1708066470 0x7f1708068930 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f17180017b0 tx=0x7f1718001c80 comp rx=0 tx=0).stop 2023-10-26T05:56:56.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 msgr2=0x7f17281531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 0x7f17281531e0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f171c00d560 tx=0x7f171c00da20 comp rx=0 tx=0).stop 2023-10-26T05:56:56.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 shutdown_connections 2023-10-26T05:56:56.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17280acb70 0x7f17280ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1708066470 0x7f1708068930 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f172812a940 0x7f17281531e0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 --2- 172.21.15.71:0/1886197757 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1710005190 0x7f172813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 >> 172.21.15.71:0/1886197757 conn(0x7f1728069770 msgr2=0x7f1728158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 shutdown_connections 2023-10-26T05:56:56.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.615+0000 7f172c9dd640 1 -- 172.21.15.71:0/1886197757 wait complete. 2023-10-26T05:56:56.642 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:56:56.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:56:56.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 -- 172.21.15.71:0/4113139785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b3c0039b0 msgr2=0x7f3b3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 --2- 172.21.15.71:0/4113139785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b3c0039b0 0x7f3b3c003dd0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f3b500710c0 tx=0x7f3b4001cc10 comp rx=0 tx=0).stop 2023-10-26T05:56:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b4f7fe640 1 -- 172.21.15.71:0/4113139785 reap_dead start 2023-10-26T05:56:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 -- 172.21.15.71:0/4113139785 shutdown_connections 2023-10-26T05:56:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 -- 172.21.15.71:0/4113139785 >> 172.21.15.71:0/4113139785 conn(0x7f3b50070cb0 msgr2=0x7f3b5006a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:56.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 -- 172.21.15.71:0/4113139785 shutdown_connections 2023-10-26T05:56:56.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 -- 172.21.15.71:0/4113139785 wait complete. 2023-10-26T05:56:56.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 Processor -- start 2023-10-26T05:56:56.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.751+0000 7f3b55758640 1 -- start start 2023-10-26T05:56:56.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c0039b0 0x7f3b50145be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b50146120 0x7f3b5006c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 0x7f3b5006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5006dbb0 con 0x7f3b5006cc40 2023-10-26T05:56:56.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5006d630 con 0x7f3b3c0039b0 2023-10-26T05:56:56.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5014e3d0 con 0x7f3b50146120 2023-10-26T05:56:56.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 0x7f3b5006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 0x7f3b5006d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54022/0 (socket says 172.21.15.71:54022) 2023-10-26T05:56:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 -- 172.21.15.71:0/1321052494 learned_addr learned my addr 172.21.15.71:0/1321052494 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4effd640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c0039b0 0x7f3b50145be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4e7fc640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b50146120 0x7f3b5006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 -- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b50146120 msgr2=0x7f3b5006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b50146120 0x7f3b5006c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 -- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c0039b0 msgr2=0x7f3b50145be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c0039b0 0x7f3b50145be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 -- 172.21.15.71:0/1321052494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5014e6b0 con 0x7f3b5006cc40 2023-10-26T05:56:56.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4f7fe640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 0x7f3b5006d0f0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3b500cd870 tx=0x7f3b4800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4800ca10 con 0x7f3b5006cc40 2023-10-26T05:56:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3b4800cc60 con 0x7f3b5006cc40 2023-10-26T05:56:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5014e990 con 0x7f3b5006cc40 2023-10-26T05:56:56.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48011460 con 0x7f3b5006cc40 2023-10-26T05:56:56.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5014ef50 con 0x7f3b5006cc40 2023-10-26T05:56:56.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b50153750 con 0x7f3b5006cc40 2023-10-26T05:56:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3b480117d0 con 0x7f3b5006cc40 2023-10-26T05:56:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b477fe640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b300664a0 0x7f3b30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:56.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3b48084460 con 0x7f3b5006cc40 2023-10-26T05:56:56.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.755+0000 7f3b4effd640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b300664a0 0x7f3b30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:56.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.759+0000 7f3b4effd640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b300664a0 0x7f3b30068960 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b0000 tx=0x7f3b400a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:56.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.759+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3b50153750 con 0x7f3b5006cc40 2023-10-26T05:56:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:56.899+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3b50153750 con 0x7f3b5006cc40 2023-10-26T05:56:57.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.655+0000 7f3b477fe640 1 -- 172.21.15.71:0/1321052494 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v169) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3b50153750 con 0x7f3b5006cc40 2023-10-26T05:56:57.665 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:56:57.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b300664a0 msgr2=0x7f3b30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:57.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b300664a0 0x7f3b30068960 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b0000 tx=0x7f3b400a7040 comp rx=0 tx=0).stop 2023-10-26T05:56:57.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 msgr2=0x7f3b5006d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:57.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 0x7f3b5006d0f0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f3b500cd870 tx=0x7f3b4800ecf0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 shutdown_connections 2023-10-26T05:56:57.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3b300664a0 0x7f3b30068960 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b5006cc40 0x7f3b5006d0f0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3b50146120 0x7f3b5006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 --2- 172.21.15.71:0/1321052494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3b3c0039b0 0x7f3b50145be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 >> 172.21.15.71:0/1321052494 conn(0x7f3b50070cb0 msgr2=0x7f3b5015a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:57.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 shutdown_connections 2023-10-26T05:56:57.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.659+0000 7f3b55758640 1 -- 172.21.15.71:0/1321052494 wait complete. 2023-10-26T05:56:57.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:56:57.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/3013854132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 msgr2=0x7effb40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:57.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 --2- 172.21.15.71:0/3013854132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb40b1ff0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7effb0009900 tx=0x7effb001c7c0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/3013854132 shutdown_connections 2023-10-26T05:56:57.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 --2- 172.21.15.71:0/3013854132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb40b1ff0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/3013854132 >> 172.21.15.71:0/3013854132 conn(0x7effb406f440 msgr2=0x7effb406f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:57.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/3013854132 shutdown_connections 2023-10-26T05:56:57.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/3013854132 wait complete. 2023-10-26T05:56:57.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 Processor -- start 2023-10-26T05:56:57.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- start start 2023-10-26T05:56:57.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb40af990 0x7effb4137d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:57.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7effb40cc220 0x7effb4138460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:57.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb41389a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:57.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb41530c0 con 0x7effb4142b30 2023-10-26T05:56:57.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb41f5510 con 0x7effb40af990 2023-10-26T05:56:57.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb41f57f0 con 0x7effb40cc220 2023-10-26T05:56:57.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb41389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:57.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb41389a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54048/0 (socket says 172.21.15.71:54048) 2023-10-26T05:56:57.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 -- 172.21.15.71:0/2415086681 learned_addr learned my addr 172.21.15.71:0/2415086681 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:57.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 -- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7effb40cc220 msgr2=0x7effb4138460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:57.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7effb40cc220 0x7effb4138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 -- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb40af990 msgr2=0x7effb4137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:57.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb40af990 0x7effb4137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:57.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 -- 172.21.15.71:0/2415086681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0009560 con 0x7effb4142b30 2023-10-26T05:56:57.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effb99c9640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb41389a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7effac00c780 tx=0x7effac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:57.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac003d90 con 0x7effb4142b30 2023-10-26T05:56:57.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7effac00ce90 con 0x7effb4142b30 2023-10-26T05:56:57.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb4143400 con 0x7effb4142b30 2023-10-26T05:56:57.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac0052d0 con 0x7effb4142b30 2023-10-26T05:56:57.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.823+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb41f4840 con 0x7effb4142b30 2023-10-26T05:56:57.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.827+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb4066b10 con 0x7effb4142b30 2023-10-26T05:56:57.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.827+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7effac005430 con 0x7effb4142b30 2023-10-26T05:56:57.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.827+0000 7effa9ffb640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff98066470 0x7eff98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:57.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.827+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5227+0+0 (secure 0 0 0) 0x7effac083660 con 0x7effb4142b30 2023-10-26T05:56:57.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.827+0000 7effb91c8640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff98066470 0x7eff98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:57.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.827+0000 7effb91c8640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff98066470 0x7eff98068930 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7effb00af000 tx=0x7effb0019560 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:57.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.831+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7effac050da0 con 0x7effb4142b30 2023-10-26T05:56:57.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:57.983+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7effb40721e0 con 0x7effb4142b30 2023-10-26T05:56:58.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.667+0000 7effa9ffb640 1 -- 172.21.15.71:0/2415086681 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v170) v1 ==== 170+0+0 (secure 0 0 0) 0x7effac0170d0 con 0x7effb4142b30 2023-10-26T05:56:58.678 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:56:58.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff98066470 msgr2=0x7eff98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:58.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff98066470 0x7eff98068930 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7effb00af000 tx=0x7effb0019560 comp rx=0 tx=0).stop 2023-10-26T05:56:58.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 msgr2=0x7effb41389a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:58.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb41389a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7effac00c780 tx=0x7effac00cc40 comp rx=0 tx=0).stop 2023-10-26T05:56:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 shutdown_connections 2023-10-26T05:56:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7eff98066470 0x7eff98068930 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb4142b30 0x7effb41389a0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7effb40cc220 0x7effb4138460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:58.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 --2- 172.21.15.71:0/2415086681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7effb40af990 0x7effb4137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:58.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 >> 172.21.15.71:0/2415086681 conn(0x7effb406f440 msgr2=0x7effb415bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:58.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 shutdown_connections 2023-10-26T05:56:58.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.671+0000 7effbb42c640 1 -- 172.21.15.71:0/2415086681 wait complete. 2023-10-26T05:56:58.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:56:58.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/2108374744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 msgr2=0x7fa404003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:58.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 --2- 172.21.15.71:0/2108374744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa404003dd0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006ee0 tx=0x7fa40c01ce70 comp rx=0 tx=0).stop 2023-10-26T05:56:58.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41b7fe640 1 -- 172.21.15.71:0/2108374744 reap_dead start 2023-10-26T05:56:58.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/2108374744 shutdown_connections 2023-10-26T05:56:58.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/2108374744 >> 172.21.15.71:0/2108374744 conn(0x7fa41c0cdd80 msgr2=0x7fa41c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:58.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/2108374744 shutdown_connections 2023-10-26T05:56:58.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/2108374744 wait complete. 2023-10-26T05:56:58.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 Processor -- start 2023-10-26T05:56:58.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- start start 2023-10-26T05:56:58.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa41c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:58.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c1510b0 0x7fa41c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c15dcd0 0x7fa41c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c06a650 con 0x7fa4040039b0 2023-10-26T05:56:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c150f50 con 0x7fa41c15dcd0 2023-10-26T05:56:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c15e150 con 0x7fa41c1510b0 2023-10-26T05:56:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa41c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:58.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa41c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54072/0 (socket says 172.21.15.71:54072) 2023-10-26T05:56:58.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 -- 172.21.15.71:0/1033314882 learned_addr learned my addr 172.21.15.71:0/1033314882 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:58.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41a7fc640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c1510b0 0x7fa41c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:58.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41b7fe640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c15dcd0 0x7fa41c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:58.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 -- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c1510b0 msgr2=0x7fa41c15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:58.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c1510b0 0x7fa41c15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:58.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 -- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c15dcd0 msgr2=0x7fa41c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:58.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c15dcd0 0x7fa41c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:58.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 -- 172.21.15.71:0/1033314882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c155690 con 0x7fa4040039b0 2023-10-26T05:56:58.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41b7fe640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c15dcd0 0x7fa41c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:56:58.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41a7fc640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c1510b0 0x7fa41c15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:56:58.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa41affd640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa41c150a10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006ee0 tx=0x7fa40c001490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:58.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0b4070 con 0x7fa4040039b0 2023-10-26T05:56:58.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa40c001680 con 0x7fa4040039b0 2023-10-26T05:56:58.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c0ca010 con 0x7fa4040039b0 2023-10-26T05:56:58.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c006330 con 0x7fa4040039b0 2023-10-26T05:56:58.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.831+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c155040 con 0x7fa4040039b0 2023-10-26T05:56:58.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.835+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c066b10 con 0x7fa4040039b0 2023-10-26T05:56:58.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.835+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa40c0b9420 con 0x7fa4040039b0 2023-10-26T05:56:58.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.835+0000 7fa4037fe640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4080664a0 0x7fa408068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:58.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.835+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa40c11d750 con 0x7fa4040039b0 2023-10-26T05:56:58.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.835+0000 7fa41a7fc640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4080664a0 0x7fa408068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:58.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.835+0000 7fa41a7fc640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4080664a0 0x7fa408068960 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0b6ef0 tx=0x7fa410006ba0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:58.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:58.839+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa40c0eac30 con 0x7fa4040039b0 2023-10-26T05:56:59.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.027+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa41c0b5120 con 0x7fa4040039b0 2023-10-26T05:56:59.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.671+0000 7fa4037fe640 1 -- 172.21.15.71:0/1033314882 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa40c0ee250 con 0x7fa4040039b0 2023-10-26T05:56:59.681 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 20 and data pool 21 2023-10-26T05:56:59.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4080664a0 msgr2=0x7fa408068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:59.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4080664a0 0x7fa408068960 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0b6ef0 tx=0x7fa410006ba0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 msgr2=0x7fa41c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa41c150a10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fa40c006ee0 tx=0x7fa40c001490 comp rx=0 tx=0).stop 2023-10-26T05:56:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 shutdown_connections 2023-10-26T05:56:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c15dcd0 0x7fa41c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c1510b0 0x7fa41c15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa4080664a0 0x7fa408068960 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 --2- 172.21.15.71:0/1033314882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4040039b0 0x7fa41c150a10 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 >> 172.21.15.71:0/1033314882 conn(0x7fa41c0cdd80 msgr2=0x7fa41c0b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:59.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 shutdown_connections 2023-10-26T05:56:59.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.675+0000 7fa4212c8640 1 -- 172.21.15.71:0/1033314882 wait complete. 2023-10-26T05:56:59.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:56:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.823+0000 7f813113c640 1 -- 172.21.15.71:0/1292981802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0d0850 msgr2=0x7f812c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:59.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.823+0000 7f813113c640 1 --2- 172.21.15.71:0/1292981802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0d0850 0x7f812c15b3a0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f812c08a140 tx=0x7f811c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.823+0000 7f813113c640 1 -- 172.21.15.71:0/1292981802 shutdown_connections 2023-10-26T05:56:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.823+0000 7f813113c640 1 --2- 172.21.15.71:0/1292981802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0d0850 0x7f812c15b3a0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.823+0000 7f813113c640 1 --2- 172.21.15.71:0/1292981802 >> v2:172.21.15.86:3300/0 conn(0x7f812c0d0470 0x7f812c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.823+0000 7f813113c640 1 -- 172.21.15.71:0/1292981802 >> 172.21.15.71:0/1292981802 conn(0x7f812c0cdd80 msgr2=0x7f812c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:56:59.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- 172.21.15.71:0/1292981802 shutdown_connections 2023-10-26T05:56:59.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- 172.21.15.71:0/1292981802 wait complete. 2023-10-26T05:56:59.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 Processor -- start 2023-10-26T05:56:59.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- start start 2023-10-26T05:56:59.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 0x7f812c0b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:59.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f812c0d0470 0x7f812c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:59.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f812c0d0850 0x7f812c150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:59.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c15c490 con 0x7f812c0cd880 2023-10-26T05:56:59.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0ca290 con 0x7f812c0d0850 2023-10-26T05:56:59.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0ca570 con 0x7f812c0d0470 2023-10-26T05:56:59.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 0x7f812c0b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:59.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 0x7f812c0b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54090/0 (socket says 172.21.15.71:54090) 2023-10-26T05:56:59.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 -- 172.21.15.71:0/3347696639 learned_addr learned my addr 172.21.15.71:0/3347696639 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:56:59.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812a59c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f812c0d0470 0x7f812c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:59.840 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 -- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f812c0d0470 msgr2=0x7f812c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:59.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f812c0d0470 0x7f812c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812b59e640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f812c0d0850 0x7f812c150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:59.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 -- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f812c0d0850 msgr2=0x7f812c150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:56:59.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f812c0d0850 0x7f812c150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:56:59.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 -- 172.21.15.71:0/3347696639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009560 con 0x7f812c0cd880 2023-10-26T05:56:59.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812b59e640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f812c0d0850 0x7f812c150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:56:59.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f812ad9d640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 0x7f812c0b19f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f812000ba50 tx=0x7f812000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:59.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000cbe0 con 0x7f812c0cd880 2023-10-26T05:56:59.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f812000ce50 con 0x7f812c0cd880 2023-10-26T05:56:59.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0ca850 con 0x7f812c0cd880 2023-10-26T05:56:59.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120014680 con 0x7f812c0cd880 2023-10-26T05:56:59.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c151c30 con 0x7f812c0cd880 2023-10-26T05:56:59.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.827+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c066b10 con 0x7f812c0cd880 2023-10-26T05:56:59.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.831+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f812001d3b0 con 0x7f812c0cd880 2023-10-26T05:56:59.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.831+0000 7f81137fe640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81180664a0 0x7f8118068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:56:59.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.831+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8120084080 con 0x7f812c0cd880 2023-10-26T05:56:59.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.831+0000 7f812a59c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81180664a0 0x7f8118068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:56:59.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.831+0000 7f812a59c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81180664a0 0x7f8118068960 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f811c0af000 tx=0x7f811c01afe0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:56:59.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.835+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8120051500 con 0x7f812c0cd880 2023-10-26T05:56:59.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.991+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f812c0721e0 con 0x7f812c0cd880 2023-10-26T05:57:00.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.991+0000 7f81137fe640 1 -- 172.21.15.71:0/3347696639 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16509 (secure 0 0 0) 0x7f8120019090 con 0x7f812c0cd880 2023-10-26T05:57:00.000 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:00.001 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":171,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:56:59.034600+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:57:00.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81180664a0 msgr2=0x7f8118068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81180664a0 0x7f8118068960 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f811c0af000 tx=0x7f811c01afe0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 msgr2=0x7f812c0b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 0x7f812c0b19f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f812000ba50 tx=0x7f812000bf10 comp rx=0 tx=0).stop 2023-10-26T05:57:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 shutdown_connections 2023-10-26T05:57:00.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f812c0d0850 0x7f812c150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f812c0d0470 0x7f812c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81180664a0 0x7f8118068960 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 --2- 172.21.15.71:0/3347696639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f812c0cd880 0x7f812c0b19f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 >> 172.21.15.71:0/3347696639 conn(0x7f812c0cdd80 msgr2=0x7f812c0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:00.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 shutdown_connections 2023-10-26T05:57:00.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:56:59.995+0000 7f813113c640 1 -- 172.21.15.71:0/3347696639 wait complete. 2023-10-26T05:57:00.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:57:00.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2301833866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5264005190 msgr2=0x7f526400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 --2- 172.21.15.71:0/2301833866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5264005190 0x7f526400b990 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f526c009b60 tx=0x7f526c01d290 comp rx=0 tx=0).stop 2023-10-26T05:57:00.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2301833866 shutdown_connections 2023-10-26T05:57:00.155 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 --2- 172.21.15.71:0/2301833866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5264005190 0x7f526400b990 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2301833866 >> 172.21.15.71:0/2301833866 conn(0x7f527c069770 msgr2=0x7f527c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:00.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2301833866 shutdown_connections 2023-10-26T05:57:00.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2301833866 wait complete. 2023-10-26T05:57:00.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 Processor -- start 2023-10-26T05:57:00.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- start start 2023-10-26T05:57:00.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5264005190 0x7f527c1f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 0x7f527c13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f527c13c4c0 0x7f527c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c14f960 con 0x7f527c1f5f40 2023-10-26T05:57:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c158d80 con 0x7f5264005190 2023-10-26T05:57:00.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c14fb70 con 0x7f527c13c4c0 2023-10-26T05:57:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 0x7f527c13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 0x7f527c13bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54104/0 (socket says 172.21.15.71:54104) 2023-10-26T05:57:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 -- 172.21.15.71:0/2963300723 learned_addr learned my addr 172.21.15.71:0/2963300723 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f528239f640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5264005190 0x7f527c1f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5282ba0640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f527c13c4c0 0x7f527c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 -- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f527c13c4c0 msgr2=0x7f527c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f527c13c4c0 0x7f527c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 -- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5264005190 msgr2=0x7f527c1f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5264005190 0x7f527c1f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 -- 172.21.15.71:0/2963300723 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c14fe50 con 0x7f527c1f5f40 2023-10-26T05:57:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5282ba0640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f527c13c4c0 0x7f527c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:57:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f528239f640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5264005190 0x7f527c1f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:57:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5281b9e640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 0x7f527c13bf80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f527000c700 tx=0x7f527000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270005270 con 0x7f527c1f5f40 2023-10-26T05:57:00.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f52700053d0 con 0x7f527c1f5f40 2023-10-26T05:57:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c150130 con 0x7f527c1f5f40 2023-10-26T05:57:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000a880 con 0x7f527c1f5f40 2023-10-26T05:57:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.147+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c13c950 con 0x7f527c1f5f40 2023-10-26T05:57:00.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.151+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c066b10 con 0x7f527c1f5f40 2023-10-26T05:57:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.151+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f527000a9e0 con 0x7f527c1f5f40 2023-10-26T05:57:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.151+0000 7f527affd640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5268066490 0x7f5268068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.151+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5270083610 con 0x7f527c1f5f40 2023-10-26T05:57:00.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.151+0000 7f528239f640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5268066490 0x7f5268068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.151+0000 7f528239f640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5268066490 0x7f5268068950 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5274001ea0 tx=0x7f5274006ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:00.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.155+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5270050b20 con 0x7f527c1f5f40 2023-10-26T05:57:00.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.311+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f527c0721e0 con 0x7f527c1f5f40 2023-10-26T05:57:00.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.311+0000 7f527affd640 1 -- 172.21.15.71:0/2963300723 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16509 (secure 0 0 0) 0x7f5270017090 con 0x7f527c1f5f40 2023-10-26T05:57:00.321 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:00.322 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":171,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:56:59.034600+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:57:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5268066490 msgr2=0x7f5268068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5268066490 0x7f5268068950 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5274001ea0 tx=0x7f5274006ed0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 msgr2=0x7f527c13bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 0x7f527c13bf80 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f527000c700 tx=0x7f527000cbc0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 shutdown_connections 2023-10-26T05:57:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f527c13c4c0 0x7f527c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5268066490 0x7f5268068950 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f527c1f5f40 0x7f527c13bf80 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 --2- 172.21.15.71:0/2963300723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5264005190 0x7f527c1f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 >> 172.21.15.71:0/2963300723 conn(0x7f527c069770 msgr2=0x7f527c0aaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 shutdown_connections 2023-10-26T05:57:00.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.315+0000 7f5284603640 1 -- 172.21.15.71:0/2963300723 wait complete. 2023-10-26T05:57:00.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:57:00.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/4224837674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415d390 msgr2=0x7f2914159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- 172.21.15.71:0/4224837674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415d390 0x7f2914159810 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2914072e90 tx=0x7f290401c7e0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/4224837674 shutdown_connections 2023-10-26T05:57:00.471 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- 172.21.15.71:0/4224837674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415d390 0x7f2914159810 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- 172.21.15.71:0/4224837674 >> v2:172.21.15.86:3300/0 conn(0x7f29140cede0 0x7f291415ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- 172.21.15.71:0/4224837674 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f291415c720 0x7f29140b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/4224837674 >> 172.21.15.71:0/4224837674 conn(0x7f29140b0f20 msgr2=0x7f29140b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:00.472 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/4224837674 shutdown_connections 2023-10-26T05:57:00.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/4224837674 wait complete. 2023-10-26T05:57:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 Processor -- start 2023-10-26T05:57:00.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- start start 2023-10-26T05:57:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29140cede0 0x7f2914146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 0x7f291414f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291415d390 0x7f2914145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914147560 con 0x7f291415c720 2023-10-26T05:57:00.476 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914147840 con 0x7f291415d390 2023-10-26T05:57:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2914147b20 con 0x7f29140cede0 2023-10-26T05:57:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 0x7f291414f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 0x7f291414f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54122/0 (socket says 172.21.15.71:54122) 2023-10-26T05:57:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 -- 172.21.15.71:0/2553612312 learned_addr learned my addr 172.21.15.71:0/2553612312 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:00.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f2912d9d640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29140cede0 0x7f2914146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291359e640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291415d390 0x7f2914145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 -- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29140cede0 msgr2=0x7f2914146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29140cede0 0x7f2914146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 -- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291415d390 msgr2=0x7f2914145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291415d390 0x7f2914145aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 -- 172.21.15.71:0/2553612312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009560 con 0x7f291415c720 2023-10-26T05:57:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291259c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 0x7f291414f920 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f290800cc30 tx=0x7f290800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290800f650 con 0x7f291415c720 2023-10-26T05:57:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f290800f8a0 con 0x7f291415c720 2023-10-26T05:57:00.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2914147e00 con 0x7f291415c720 2023-10-26T05:57:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290800fcb0 con 0x7f291415c720 2023-10-26T05:57:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.463+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2914147150 con 0x7f291415c720 2023-10-26T05:57:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.467+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2914066d00 con 0x7f291415c720 2023-10-26T05:57:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.467+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2908018400 con 0x7f291415c720 2023-10-26T05:57:00.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.467+0000 7f28fb7fe640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.467+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2908084b60 con 0x7f291415c720 2023-10-26T05:57:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.467+0000 7f2912d9d640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.467+0000 7f2912d9d640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f290c00bb30 tx=0x7f290c00b510 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:00.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.471+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2908051f50 con 0x7f291415c720 2023-10-26T05:57:00.667 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.659+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f29141482b0 con 0x7f291415c720 2023-10-26T05:57:00.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.719+0000 7f28fb7fe640 1 -- 172.21.15.71:0/2553612312 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f290801d020 con 0x7f291415c720 2023-10-26T05:57:00.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 msgr2=0x7f2900068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f290c00bb30 tx=0x7f290c00b510 comp rx=0 tx=0).stop 2023-10-26T05:57:00.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 msgr2=0x7f291414f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 0x7f291414f920 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f290800cc30 tx=0x7f290800a620 comp rx=0 tx=0).stop 2023-10-26T05:57:00.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 shutdown_connections 2023-10-26T05:57:00.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f291415d390 0x7f2914145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2900066470 0x7f2900068930 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f291415c720 0x7f291414f920 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 --2- 172.21.15.71:0/2553612312 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f29140cede0 0x7f2914146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 >> 172.21.15.71:0/2553612312 conn(0x7f29140b0f20 msgr2=0x7f2914157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:00.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 shutdown_connections 2023-10-26T05:57:00.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.723+0000 7f291901c640 1 -- 172.21.15.71:0/2553612312 wait complete. 2023-10-26T05:57:00.749 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:57:00.749 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:57:00.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/176723459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdac0051e0 msgr2=0x7fcdac00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/176723459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdac0051e0 0x7fcdac00b990 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40cdc60 tx=0x7fcdbc01cfc0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 -- 172.21.15.71:0/176723459 reap_dead start 2023-10-26T05:57:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/176723459 shutdown_connections 2023-10-26T05:57:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/176723459 >> 172.21.15.71:0/176723459 conn(0x7fcdc40cdd80 msgr2=0x7fcdc40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/176723459 shutdown_connections 2023-10-26T05:57:00.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/176723459 wait complete. 2023-10-26T05:57:00.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 Processor -- start 2023-10-26T05:57:00.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- start start 2023-10-26T05:57:00.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac0051e0 0x7fcdc40af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc40c4e00 0x7fcdc40c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 0x7fcdc41e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc406a650 con 0x7fcdc41e6160 2023-10-26T05:57:00.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc41e6b30 con 0x7fcdac0051e0 2023-10-26T05:57:00.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40c7c80 con 0x7fcdc40c4e00 2023-10-26T05:57:00.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 0x7fcdc41e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 0x7fcdc41e65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:54130/0 (socket says 172.21.15.71:54130) 2023-10-26T05:57:00.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 -- 172.21.15.71:0/857037390 learned_addr learned my addr 172.21.15.71:0/857037390 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:00.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdbbfff640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc40c4e00 0x7fcdc40c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc8939640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac0051e0 0x7fcdc40af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 -- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc40c4e00 msgr2=0x7fcdc40c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc40c4e00 0x7fcdc40c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 -- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac0051e0 msgr2=0x7fcdc40af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:00.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac0051e0 0x7fcdc40af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:00.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 -- 172.21.15.71:0/857037390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc40c5300 con 0x7fcdc41e6160 2023-10-26T05:57:00.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc8939640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac0051e0 0x7fcdc40af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:57:00.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdc913a640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 0x7fcdc41e65f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc098af0 tx=0x7fcdbc0af900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:00.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc004b10 con 0x7fcdc41e6160 2023-10-26T05:57:00.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcdbc004c70 con 0x7fcdc41e6160 2023-10-26T05:57:00.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40c55e0 con 0x7fcdc41e6160 2023-10-26T05:57:00.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc004e40 con 0x7fcdc41e6160 2023-10-26T05:57:00.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.871+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40c5ba0 con 0x7fcdc41e6160 2023-10-26T05:57:00.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.875+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc4066b10 con 0x7fcdc41e6160 2023-10-26T05:57:00.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.875+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcdc0001c40 con 0x7fcdc41e6160 2023-10-26T05:57:00.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.875+0000 7fcdb97fa640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdb00664a0 0x7fcdb0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:00.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.875+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdbc11e740 con 0x7fcdc41e6160 2023-10-26T05:57:00.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.875+0000 7fcdc8939640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdb00664a0 0x7fcdb0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:00.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.875+0000 7fcdc8939640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdb00664a0 0x7fcdb0068960 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001430 tx=0x7fcdc0000f80 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:00.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:00.879+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcdbc0ebc50 con 0x7fcdc41e6160 2023-10-26T05:57:01.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.067+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcdc40c5d90 con 0x7fcdc41e6160 2023-10-26T05:57:01.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.723+0000 7fcdb97fa640 1 -- 172.21.15.71:0/857037390 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcdbc0ef270 con 0x7fcdc41e6160 2023-10-26T05:57:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdb00664a0 msgr2=0x7fcdb0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:01.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdb00664a0 0x7fcdb0068960 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001430 tx=0x7fcdc0000f80 comp rx=0 tx=0).stop 2023-10-26T05:57:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 msgr2=0x7fcdc41e65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 0x7fcdc41e65f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc098af0 tx=0x7fcdbc0af900 comp rx=0 tx=0).stop 2023-10-26T05:57:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 shutdown_connections 2023-10-26T05:57:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcdb00664a0 0x7fcdb0068960 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcdc41e6160 0x7fcdc41e65f0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdc40c4e00 0x7fcdc40c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 --2- 172.21.15.71:0/857037390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac0051e0 0x7fcdc40af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 >> 172.21.15.71:0/857037390 conn(0x7fcdc40cdd80 msgr2=0x7fcdc415a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:01.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 shutdown_connections 2023-10-26T05:57:01.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.727+0000 7fcdcab9d640 1 -- 172.21.15.71:0/857037390 wait complete. 2023-10-26T05:57:01.757 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:57:01.758 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:57:01.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/2444983775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 msgr2=0x7f5358071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:01.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 --2- 172.21.15.71:0/2444983775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f5358071470 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f5350009c80 tx=0x7f535001ccd0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/2444983775 shutdown_connections 2023-10-26T05:57:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 --2- 172.21.15.71:0/2444983775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f5358071470 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 --2- 172.21.15.71:0/2444983775 >> v2:172.21.15.86:3300/0 conn(0x7f5358140cc0 0x7f535815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/2444983775 >> 172.21.15.71:0/2444983775 conn(0x7f5358069770 msgr2=0x7f5358069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/2444983775 shutdown_connections 2023-10-26T05:57:01.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/2444983775 wait complete. 2023-10-26T05:57:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 Processor -- start 2023-10-26T05:57:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- start start 2023-10-26T05:57:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5358070fe0 0x7f53581f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f53581568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:01.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5358140cc0 0x7f535806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:01.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358158010 con 0x7f53580b2370 2023-10-26T05:57:01.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358138870 con 0x7f5358070fe0 2023-10-26T05:57:01.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5358138b50 con 0x7f5358140cc0 2023-10-26T05:57:01.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f53581568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:01.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f53581568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35720/0 (socket says 172.21.15.71:35720) 2023-10-26T05:57:01.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 -- 172.21.15.71:0/3873389769 learned_addr learned my addr 172.21.15.71:0/3873389769 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:01.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535c8ec640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5358070fe0 0x7f53581f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:01.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535d0ed640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5358140cc0 0x7f535806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:01.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 -- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5358140cc0 msgr2=0x7f535806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:01.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5358140cc0 0x7f535806feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 -- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5358070fe0 msgr2=0x7f53581f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:01.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5358070fe0 0x7f53581f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:01.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 -- 172.21.15.71:0/3873389769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53500098e0 con 0x7f53580b2370 2023-10-26T05:57:01.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535d0ed640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5358140cc0 0x7f535806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:57:01.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f5357fff640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f53581568b0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f534c00c450 tx=0x7f534c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c012830 con 0x7f53580b2370 2023-10-26T05:57:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f534c012aa0 con 0x7f53580b2370 2023-10-26T05:57:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358138e30 con 0x7f53580b2370 2023-10-26T05:57:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c011380 con 0x7f53580b2370 2023-10-26T05:57:01.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53581393f0 con 0x7f53580b2370 2023-10-26T05:57:01.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.879+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5358066b10 con 0x7f53580b2370 2023-10-26T05:57:01.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.883+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f534c0114e0 con 0x7f53580b2370 2023-10-26T05:57:01.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.883+0000 7f53557fa640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5344066470 0x7f5344068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:01.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.883+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f534c083f00 con 0x7f53580b2370 2023-10-26T05:57:01.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.883+0000 7f535c8ec640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5344066470 0x7f5344068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:01.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.883+0000 7f535c8ec640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5344066470 0x7f5344068930 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f53480058d0 tx=0x7f5348005d90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:01.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:01.887+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f534c051410 con 0x7f53580b2370 2023-10-26T05:57:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.067+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53581395e0 con 0x7f53580b2370 2023-10-26T05:57:02.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.067+0000 7f53557fa640 1 -- 172.21.15.71:0/3873389769 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f534c054a30 con 0x7f53580b2370 2023-10-26T05:57:02.076 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:02.076 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:57:02.076 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:57:02.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.067+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5344066470 msgr2=0x7f5344068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.067+0000 7f535eb50640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5344066470 0x7f5344068930 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f53480058d0 tx=0x7f5348005d90 comp rx=0 tx=0).stop 2023-10-26T05:57:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.067+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 msgr2=0x7f53581568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.067+0000 7f535eb50640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f53581568b0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f534c00c450 tx=0x7f534c00c910 comp rx=0 tx=0).stop 2023-10-26T05:57:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 shutdown_connections 2023-10-26T05:57:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5358140cc0 0x7f535806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5344066470 0x7f5344068930 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53580b2370 0x7f53581568b0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 --2- 172.21.15.71:0/3873389769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5358070fe0 0x7f53581f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 >> 172.21.15.71:0/3873389769 conn(0x7f5358069770 msgr2=0x7f535813e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:02.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 shutdown_connections 2023-10-26T05:57:02.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.071+0000 7f535eb50640 1 -- 172.21.15.71:0/3873389769 wait complete. 2023-10-26T05:57:02.093 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:57:02.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.195+0000 7ff7ce511640 1 -- 172.21.15.71:0/4115269711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c815b5c0 msgr2=0x7ff7c80b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.195+0000 7ff7ce511640 1 --2- 172.21.15.71:0/4115269711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c815b5c0 0x7ff7c80b02a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009840 tx=0x7ff7b801c740 comp rx=0 tx=0).stop 2023-10-26T05:57:02.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.195+0000 7ff7ce511640 1 -- 172.21.15.71:0/4115269711 shutdown_connections 2023-10-26T05:57:02.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.195+0000 7ff7ce511640 1 --2- 172.21.15.71:0/4115269711 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7c80b6d80 0x7ff7c806a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.195+0000 7ff7ce511640 1 --2- 172.21.15.71:0/4115269711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c815b5c0 0x7ff7c80b02a0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.195+0000 7ff7ce511640 1 -- 172.21.15.71:0/4115269711 >> 172.21.15.71:0/4115269711 conn(0x7ff7c80d12d0 msgr2=0x7ff7c80d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- 172.21.15.71:0/4115269711 shutdown_connections 2023-10-26T05:57:02.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- 172.21.15.71:0/4115269711 wait complete. 2023-10-26T05:57:02.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 Processor -- start 2023-10-26T05:57:02.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- start start 2023-10-26T05:57:02.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 0x7ff7c8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7c80b6d80 0x7ff7c815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c815b5c0 0x7ff7c8156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c815ea00 con 0x7ff7c80b6910 2023-10-26T05:57:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c8141a30 con 0x7ff7c815b5c0 2023-10-26T05:57:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c8141e70 con 0x7ff7c80b6d80 2023-10-26T05:57:02.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 0x7ff7c8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 0x7ff7c8157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35748/0 (socket says 172.21.15.71:35748) 2023-10-26T05:57:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 -- 172.21.15.71:0/590571771 learned_addr learned my addr 172.21.15.71:0/590571771 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 -- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7c80b6d80 msgr2=0x7ff7c815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7c80b6d80 0x7ff7c815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 -- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c815b5c0 msgr2=0x7ff7c8156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c815b5c0 0x7ff7c8156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 -- 172.21.15.71:0/590571771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80094a0 con 0x7ff7c80b6910 2023-10-26T05:57:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c7fff640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 0x7ff7c8157610 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009790 tx=0x7ff7b80a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80042f0 con 0x7ff7c80b6910 2023-10-26T05:57:02.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff7b8005070 con 0x7ff7c80b6910 2023-10-26T05:57:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8005460 con 0x7ff7c80b6910 2023-10-26T05:57:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c80cdb70 con 0x7ff7c80b6910 2023-10-26T05:57:02.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c815e440 con 0x7ff7c80b6910 2023-10-26T05:57:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.199+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c8066b10 con 0x7ff7c80b6910 2023-10-26T05:57:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.203+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7b80b4040 con 0x7ff7c80b6910 2023-10-26T05:57:02.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.203+0000 7ff7c4ff9640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7b4066470 0x7ff7b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.203+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7b811d8f0 con 0x7ff7c80b6910 2023-10-26T05:57:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.203+0000 7ff7c77fe640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7b4066470 0x7ff7b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.203+0000 7ff7c77fe640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7b4066470 0x7ff7b4068930 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc007fd0 tx=0x7ff7bc0011a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:02.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.207+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff7b80eae00 con 0x7ff7c80b6910 2023-10-26T05:57:02.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.351+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7c80721e0 con 0x7ff7c80b6910 2023-10-26T05:57:02.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.351+0000 7ff7c4ff9640 1 -- 172.21.15.71:0/590571771 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16496 (secure 0 0 0) 0x7ff7b80b2020 con 0x7ff7c80b6910 2023-10-26T05:57:02.361 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:02.361 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":172,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:57:00.676422+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:57:02.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7b4066470 msgr2=0x7ff7b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7b4066470 0x7ff7b4068930 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc007fd0 tx=0x7ff7bc0011a0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 msgr2=0x7ff7c8157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 0x7ff7c8157610 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009790 tx=0x7ff7b80a5730 comp rx=0 tx=0).stop 2023-10-26T05:57:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 shutdown_connections 2023-10-26T05:57:02.363 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7c815b5c0 0x7ff7c8156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7c80b6d80 0x7ff7c815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff7b4066470 0x7ff7b4068930 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 --2- 172.21.15.71:0/590571771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7c80b6910 0x7ff7c8157610 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 >> 172.21.15.71:0/590571771 conn(0x7ff7c80d12d0 msgr2=0x7ff7c80b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 shutdown_connections 2023-10-26T05:57:02.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.355+0000 7ff7ce511640 1 -- 172.21.15.71:0/590571771 wait complete. 2023-10-26T05:57:02.378 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:57:02.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:57:02.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 -- 172.21.15.71:0/1159420806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980b1360 msgr2=0x7ff3980b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/1159420806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980b1360 0x7ff3980b17b0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7ff390009c80 tx=0x7ff39001ca10 comp rx=0 tx=0).stop 2023-10-26T05:57:02.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 -- 172.21.15.71:0/1159420806 shutdown_connections 2023-10-26T05:57:02.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/1159420806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980b1360 0x7ff3980b17b0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/1159420806 >> v2:172.21.15.86:3300/0 conn(0x7ff398070fe0 0x7ff398071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 -- 172.21.15.71:0/1159420806 >> 172.21.15.71:0/1159420806 conn(0x7ff398069770 msgr2=0x7ff398069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:02.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 -- 172.21.15.71:0/1159420806 shutdown_connections 2023-10-26T05:57:02.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 -- 172.21.15.71:0/1159420806 wait complete. 2023-10-26T05:57:02.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 Processor -- start 2023-10-26T05:57:02.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.491+0000 7ff39d3f0640 1 -- start start 2023-10-26T05:57:02.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff398070fe0 0x7ff398136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3980b1360 0x7ff398157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 0x7ff398136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398162f10 con 0x7ff398140cc0 2023-10-26T05:57:02.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff398135a50 con 0x7ff398070fe0 2023-10-26T05:57:02.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff398135d30 con 0x7ff3980b1360 2023-10-26T05:57:02.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 0x7ff398136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 0x7ff398136e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35758/0 (socket says 172.21.15.71:35758) 2023-10-26T05:57:02.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 -- 172.21.15.71:0/356577784 learned_addr learned my addr 172.21.15.71:0/356577784 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:02.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 -- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3980b1360 msgr2=0x7ff398157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3980b1360 0x7ff398157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 -- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff398070fe0 msgr2=0x7ff398136810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff398070fe0 0x7ff398136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 -- 172.21.15.71:0/356577784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900098e0 con 0x7ff398140cc0 2023-10-26T05:57:02.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff3977fe640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 0x7ff398136e40 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff390009bd0 tx=0x7ff3900a59d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:02.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900bc070 con 0x7ff398140cc0 2023-10-26T05:57:02.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff390004250 con 0x7ff398140cc0 2023-10-26T05:57:02.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff398136010 con 0x7ff398140cc0 2023-10-26T05:57:02.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390004660 con 0x7ff398140cc0 2023-10-26T05:57:02.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff398137380 con 0x7ff398140cc0 2023-10-26T05:57:02.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff398066b10 con 0x7ff398140cc0 2023-10-26T05:57:02.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff3900b4040 con 0x7ff398140cc0 2023-10-26T05:57:02.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff38ffff640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3780664a0 0x7ff378068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff39011e940 con 0x7ff398140cc0 2023-10-26T05:57:02.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.495+0000 7ff396ffd640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3780664a0 0x7ff378068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.499+0000 7ff396ffd640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3780664a0 0x7ff378068960 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff388009650 tx=0x7ff388006bb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:02.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.499+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff3900ebe50 con 0x7ff398140cc0 2023-10-26T05:57:02.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.651+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff398135bc0 con 0x7ff3780664a0 2023-10-26T05:57:02.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff38ffff640 1 -- 172.21.15.71:0/356577784 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278382 (secure 0 0 0) 0x7ff398135bc0 con 0x7ff3780664a0 2023-10-26T05:57:02.683 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:02.685 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:57:02.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3780664a0 msgr2=0x7ff378068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3780664a0 0x7ff378068960 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff388009650 tx=0x7ff388006bb0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 msgr2=0x7ff398136e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 0x7ff398136e40 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7ff390009bd0 tx=0x7ff3900a59d0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 shutdown_connections 2023-10-26T05:57:02.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff3780664a0 0x7ff378068960 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff398140cc0 0x7ff398136e40 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff3980b1360 0x7ff398157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 --2- 172.21.15.71:0/356577784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff398070fe0 0x7ff398136810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 >> 172.21.15.71:0/356577784 conn(0x7ff398069770 msgr2=0x7ff3980b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 shutdown_connections 2023-10-26T05:57:02.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.675+0000 7ff39d3f0640 1 -- 172.21.15.71:0/356577784 wait complete. 2023-10-26T05:57:02.702 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2023-10-26T05:57:01.607623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553911,"statfs":{"total":1600000000,"available":1591205414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001501"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687936+0000","last_change":"2023-10-26T05:56:58.687936+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687936+0000","last_undegraded":"2023-10-26T05:56:58.687936+0000","last_fullsized":"2023-10-26T05:56:58.687936+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689543+0000","last_change":"2023-10-26T05:56:58.689543+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689543+0000","last_undegraded":"2023-10-26T05:56:58.689543+0000","last_fullsized":"2023-10-26T05:56:58.689543+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.701504+0000","last_change":"2023-10-26T05:56:58.701504+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.701504+0000","last_undegraded":"2023-10-26T05:56:58.701504+0000","last_fullsized":"2023-10-26T05:56:58.701504+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687487+0000","last_change":"2023-10-26T05:56:58.687487+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687487+0000","last_undegraded":"2023-10-26T05:56:58.687487+0000","last_fullsized":"2023-10-26T05:56:58.687487+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699011+0000","last_change":"2023-10-26T05:56:58.699011+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699011+0000","last_undegraded":"2023-10-26T05:56:58.699011+0000","last_fullsized":"2023-10-26T05:56:58.699011+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687527+0000","last_change":"2023-10-26T05:56:58.687527+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687527+0000","last_undegraded":"2023-10-26T05:56:58.687527+0000","last_fullsized":"2023-10-26T05:56:58.687527+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684911+0000","last_change":"2023-10-26T05:56:58.684911+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684911+0000","last_undegraded":"2023-10-26T05:56:58.684911+0000","last_fullsized":"2023-10-26T05:56:58.684911+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686493+0000","last_change":"2023-10-26T05:56:58.686493+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686493+0000","last_undegraded":"2023-10-26T05:56:58.686493+0000","last_fullsized":"2023-10-26T05:56:58.686493+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686070+0000","last_change":"2023-10-26T05:56:58.686070+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686070+0000","last_undegraded":"2023-10-26T05:56:58.686070+0000","last_fullsized":"2023-10-26T05:56:58.686070+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697900+0000","last_change":"2023-10-26T05:56:58.697900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697900+0000","last_undegraded":"2023-10-26T05:56:58.697900+0000","last_fullsized":"2023-10-26T05:56:58.697900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687124+0000","last_change":"2023-10-26T05:56:58.687124+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687124+0000","last_undegraded":"2023-10-26T05:56:58.687124+0000","last_fullsized":"2023-10-26T05:56:58.687124+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.700372+0000","last_change":"2023-10-26T05:56:58.700372+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.700372+0000","last_undegraded":"2023-10-26T05:56:58.700372+0000","last_fullsized":"2023-10-26T05:56:58.700372+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695733+0000","last_change":"2023-10-26T05:56:58.695733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695733+0000","last_undegraded":"2023-10-26T05:56:58.695733+0000","last_fullsized":"2023-10-26T05:56:58.695733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685971+0000","last_change":"2023-10-26T05:56:58.685971+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685971+0000","last_undegraded":"2023-10-26T05:56:58.685971+0000","last_fullsized":"2023-10-26T05:56:58.685971+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697158+0000","last_change":"2023-10-26T05:56:58.697158+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697158+0000","last_undegraded":"2023-10-26T05:56:58.697158+0000","last_fullsized":"2023-10-26T05:56:58.697158+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682093+0000","last_change":"2023-10-26T05:56:58.682093+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682093+0000","last_undegraded":"2023-10-26T05:56:58.682093+0000","last_fullsized":"2023-10-26T05:56:58.682093+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693783+0000","last_change":"2023-10-26T05:56:58.693783+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693783+0000","last_undegraded":"2023-10-26T05:56:58.693783+0000","last_fullsized":"2023-10-26T05:56:58.693783+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.702305+0000","last_change":"2023-10-26T05:56:58.702305+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.702305+0000","last_undegraded":"2023-10-26T05:56:58.702305+0000","last_fullsized":"2023-10-26T05:56:58.702305+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686171+0000","last_change":"2023-10-26T05:56:58.686171+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686171+0000","last_undegraded":"2023-10-26T05:56:58.686171+0000","last_fullsized":"2023-10-26T05:56:58.686171+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684098+0000","last_change":"2023-10-26T05:56:58.684098+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684098+0000","last_undegraded":"2023-10-26T05:56:58.684098+0000","last_fullsized":"2023-10-26T05:56:58.684098+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693775+0000","last_change":"2023-10-26T05:56:58.693775+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693775+0000","last_undegraded":"2023-10-26T05:56:58.693775+0000","last_fullsized":"2023-10-26T05:56:58.693775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.688204+0000","last_change":"2023-10-26T05:56:58.688204+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.688204+0000","last_undegraded":"2023-10-26T05:56:58.688204+0000","last_fullsized":"2023-10-26T05:56:58.688204+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684900+0000","last_change":"2023-10-26T05:56:58.684900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684900+0000","last_undegraded":"2023-10-26T05:56:58.684900+0000","last_fullsized":"2023-10-26T05:56:58.684900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694702+0000","last_change":"2023-10-26T05:56:58.694702+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694702+0000","last_undegraded":"2023-10-26T05:56:58.694702+0000","last_fullsized":"2023-10-26T05:56:58.694702+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683660+0000","last_change":"2023-10-26T05:56:58.683660+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683660+0000","last_undegraded":"2023-10-26T05:56:58.683660+0000","last_fullsized":"2023-10-26T05:56:58.683660+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686703+0000","last_change":"2023-10-26T05:56:58.686703+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686703+0000","last_undegraded":"2023-10-26T05:56:58.686703+0000","last_fullsized":"2023-10-26T05:56:58.686703+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682875+0000","last_change":"2023-10-26T05:56:58.682875+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682875+0000","last_undegraded":"2023-10-26T05:56:58.682875+0000","last_fullsized":"2023-10-26T05:56:58.682875+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685434+0000","last_change":"2023-10-26T05:56:58.685434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685434+0000","last_undegraded":"2023-10-26T05:56:58.685434+0000","last_fullsized":"2023-10-26T05:56:58.685434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685841+0000","last_change":"2023-10-26T05:56:58.685841+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685841+0000","last_undegraded":"2023-10-26T05:56:58.685841+0000","last_fullsized":"2023-10-26T05:56:58.685841+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685847+0000","last_change":"2023-10-26T05:56:58.685847+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685847+0000","last_undegraded":"2023-10-26T05:56:58.685847+0000","last_fullsized":"2023-10-26T05:56:58.685847+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684393+0000","last_change":"2023-10-26T05:56:58.684393+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684393+0000","last_undegraded":"2023-10-26T05:56:58.684393+0000","last_fullsized":"2023-10-26T05:56:58.684393+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682434+0000","last_change":"2023-10-26T05:56:58.682434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682434+0000","last_undegraded":"2023-10-26T05:56:58.682434+0000","last_fullsized":"2023-10-26T05:56:58.682434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693816+0000","last_change":"2023-10-26T05:56:58.693816+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693816+0000","last_undegraded":"2023-10-26T05:56:58.693816+0000","last_fullsized":"2023-10-26T05:56:58.693816+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711649+0000","last_change":"2023-10-26T05:56:58.711496+0000","last_active":"2023-10-26T05:56:58.711649+0000","last_peered":"2023-10-26T05:56:58.711649+0000","last_clean":"2023-10-26T05:56:58.711649+0000","last_became_active":"2023-10-26T05:56:58.710397+0000","last_became_peered":"2023-10-26T05:56:58.710397+0000","last_unstale":"2023-10-26T05:56:58.711649+0000","last_undegraded":"2023-10-26T05:56:58.711649+0000","last_fullsized":"2023-10-26T05:56:58.711649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:45:28.308338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683676+0000","last_change":"2023-10-26T05:56:58.683676+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683676+0000","last_undegraded":"2023-10-26T05:56:58.683676+0000","last_fullsized":"2023-10-26T05:56:58.683676+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699941+0000","last_change":"2023-10-26T05:56:58.699902+0000","last_active":"2023-10-26T05:56:58.699941+0000","last_peered":"2023-10-26T05:56:58.699941+0000","last_clean":"2023-10-26T05:56:58.699941+0000","last_became_active":"2023-10-26T05:56:58.699481+0000","last_became_peered":"2023-10-26T05:56:58.699481+0000","last_unstale":"2023-10-26T05:56:58.699941+0000","last_undegraded":"2023-10-26T05:56:58.699941+0000","last_fullsized":"2023-10-26T05:56:58.699941+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:15.048906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686261+0000","last_change":"2023-10-26T05:56:58.686261+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686261+0000","last_undegraded":"2023-10-26T05:56:58.686261+0000","last_fullsized":"2023-10-26T05:56:58.686261+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692322+0000","last_change":"2023-10-26T05:56:58.691994+0000","last_active":"2023-10-26T05:56:58.692322+0000","last_peered":"2023-10-26T05:56:58.692322+0000","last_clean":"2023-10-26T05:56:58.692322+0000","last_became_active":"2023-10-26T05:56:58.691381+0000","last_became_peered":"2023-10-26T05:56:58.691381+0000","last_unstale":"2023-10-26T05:56:58.692322+0000","last_undegraded":"2023-10-26T05:56:58.692322+0000","last_fullsized":"2023-10-26T05:56:58.692322+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:13.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.696248+0000","last_change":"2023-10-26T05:56:58.696248+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.696248+0000","last_undegraded":"2023-10-26T05:56:58.696248+0000","last_fullsized":"2023-10-26T05:56:58.696248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687988+0000","last_change":"2023-10-26T05:56:58.687804+0000","last_active":"2023-10-26T05:56:58.687988+0000","last_peered":"2023-10-26T05:56:58.687988+0000","last_clean":"2023-10-26T05:56:58.687988+0000","last_became_active":"2023-10-26T05:56:58.687167+0000","last_became_peered":"2023-10-26T05:56:58.687167+0000","last_unstale":"2023-10-26T05:56:58.687988+0000","last_undegraded":"2023-10-26T05:56:58.687988+0000","last_fullsized":"2023-10-26T05:56:58.687988+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:47.280630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683215+0000","last_change":"2023-10-26T05:56:58.683215+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683215+0000","last_undegraded":"2023-10-26T05:56:58.683215+0000","last_fullsized":"2023-10-26T05:56:58.683215+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.709083+0000","last_change":"2023-10-26T05:56:58.708982+0000","last_active":"2023-10-26T05:56:58.709083+0000","last_peered":"2023-10-26T05:56:58.709083+0000","last_clean":"2023-10-26T05:56:58.709083+0000","last_became_active":"2023-10-26T05:56:58.708308+0000","last_became_peered":"2023-10-26T05:56:58.708308+0000","last_unstale":"2023-10-26T05:56:58.709083+0000","last_undegraded":"2023-10-26T05:56:58.709083+0000","last_fullsized":"2023-10-26T05:56:58.709083+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:39.204476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685163+0000","last_change":"2023-10-26T05:56:58.685163+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685163+0000","last_undegraded":"2023-10-26T05:56:58.685163+0000","last_fullsized":"2023-10-26T05:56:58.685163+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.707771+0000","last_change":"2023-10-26T05:56:58.707555+0000","last_active":"2023-10-26T05:56:58.707771+0000","last_peered":"2023-10-26T05:56:58.707771+0000","last_clean":"2023-10-26T05:56:58.707771+0000","last_became_active":"2023-10-26T05:56:58.706918+0000","last_became_peered":"2023-10-26T05:56:58.706918+0000","last_unstale":"2023-10-26T05:56:58.707771+0000","last_undegraded":"2023-10-26T05:56:58.707771+0000","last_fullsized":"2023-10-26T05:56:58.707771+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:12:04.008423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686926+0000","last_change":"2023-10-26T05:56:58.686926+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686926+0000","last_undegraded":"2023-10-26T05:56:58.686926+0000","last_fullsized":"2023-10-26T05:56:58.686926+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.702597+0000","last_change":"2023-10-26T05:56:58.702478+0000","last_active":"2023-10-26T05:56:58.702597+0000","last_peered":"2023-10-26T05:56:58.702597+0000","last_clean":"2023-10-26T05:56:58.702597+0000","last_became_active":"2023-10-26T05:56:58.702066+0000","last_became_peered":"2023-10-26T05:56:58.702066+0000","last_unstale":"2023-10-26T05:56:58.702597+0000","last_undegraded":"2023-10-26T05:56:58.702597+0000","last_fullsized":"2023-10-26T05:56:58.702597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:10.521714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.698412+0000","last_change":"2023-10-26T05:56:58.698412+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.698412+0000","last_undegraded":"2023-10-26T05:56:58.698412+0000","last_fullsized":"2023-10-26T05:56:58.698412+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.705203+0000","last_change":"2023-10-26T05:56:58.705010+0000","last_active":"2023-10-26T05:56:58.705203+0000","last_peered":"2023-10-26T05:56:58.705203+0000","last_clean":"2023-10-26T05:56:58.705203+0000","last_became_active":"2023-10-26T05:56:58.704379+0000","last_became_peered":"2023-10-26T05:56:58.704379+0000","last_unstale":"2023-10-26T05:56:58.705203+0000","last_undegraded":"2023-10-26T05:56:58.705203+0000","last_fullsized":"2023-10-26T05:56:58.705203+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:22.102825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691840+0000","last_change":"2023-10-26T05:56:58.691840+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691840+0000","last_undegraded":"2023-10-26T05:56:58.691840+0000","last_fullsized":"2023-10-26T05:56:58.691840+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692523+0000","last_change":"2023-10-26T05:56:58.692438+0000","last_active":"2023-10-26T05:56:58.692523+0000","last_peered":"2023-10-26T05:56:58.692523+0000","last_clean":"2023-10-26T05:56:58.692523+0000","last_became_active":"2023-10-26T05:56:58.692192+0000","last_became_peered":"2023-10-26T05:56:58.692192+0000","last_unstale":"2023-10-26T05:56:58.692523+0000","last_undegraded":"2023-10-26T05:56:58.692523+0000","last_fullsized":"2023-10-26T05:56:58.692523+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:20:10.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692914+0000","last_change":"2023-10-26T05:56:58.692914+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692914+0000","last_undegraded":"2023-10-26T05:56:58.692914+0000","last_fullsized":"2023-10-26T05:56:58.692914+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.715857+0000","last_change":"2023-10-26T05:56:58.715723+0000","last_active":"2023-10-26T05:56:58.715857+0000","last_peered":"2023-10-26T05:56:58.715857+0000","last_clean":"2023-10-26T05:56:58.715857+0000","last_became_active":"2023-10-26T05:56:58.714758+0000","last_became_peered":"2023-10-26T05:56:58.714758+0000","last_unstale":"2023-10-26T05:56:58.715857+0000","last_undegraded":"2023-10-26T05:56:58.715857+0000","last_fullsized":"2023-10-26T05:56:58.715857+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:50.434850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682474+0000","last_change":"2023-10-26T05:56:58.682474+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682474+0000","last_undegraded":"2023-10-26T05:56:58.682474+0000","last_fullsized":"2023-10-26T05:56:58.682474+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.691946+0000","last_change":"2023-10-26T05:56:58.691788+0000","last_active":"2023-10-26T05:56:58.691946+0000","last_peered":"2023-10-26T05:56:58.691946+0000","last_clean":"2023-10-26T05:56:58.691946+0000","last_became_active":"2023-10-26T05:56:58.691194+0000","last_became_peered":"2023-10-26T05:56:58.691194+0000","last_unstale":"2023-10-26T05:56:58.691946+0000","last_undegraded":"2023-10-26T05:56:58.691946+0000","last_fullsized":"2023-10-26T05:56:58.691946+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:05:11.942603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689507+0000","last_change":"2023-10-26T05:56:58.689507+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689507+0000","last_undegraded":"2023-10-26T05:56:58.689507+0000","last_fullsized":"2023-10-26T05:56:58.689507+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.700529+0000","last_change":"2023-10-26T05:56:58.700296+0000","last_active":"2023-10-26T05:56:58.700529+0000","last_peered":"2023-10-26T05:56:58.700529+0000","last_clean":"2023-10-26T05:56:58.700529+0000","last_became_active":"2023-10-26T05:56:58.699651+0000","last_became_peered":"2023-10-26T05:56:58.699651+0000","last_unstale":"2023-10-26T05:56:58.700529+0000","last_undegraded":"2023-10-26T05:56:58.700529+0000","last_fullsized":"2023-10-26T05:56:58.700529+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:42:59.754770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687731+0000","last_change":"2023-10-26T05:56:58.687731+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687731+0000","last_undegraded":"2023-10-26T05:56:58.687731+0000","last_fullsized":"2023-10-26T05:56:58.687731+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708870+0000","last_change":"2023-10-26T05:56:58.708596+0000","last_active":"2023-10-26T05:56:58.708870+0000","last_peered":"2023-10-26T05:56:58.708870+0000","last_clean":"2023-10-26T05:56:58.708870+0000","last_became_active":"2023-10-26T05:56:58.707912+0000","last_became_peered":"2023-10-26T05:56:58.707912+0000","last_unstale":"2023-10-26T05:56:58.708870+0000","last_undegraded":"2023-10-26T05:56:58.708870+0000","last_fullsized":"2023-10-26T05:56:58.708870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:40:53.740361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694993+0000","last_change":"2023-10-26T05:56:58.694993+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694993+0000","last_undegraded":"2023-10-26T05:56:58.694993+0000","last_fullsized":"2023-10-26T05:56:58.694993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.684540+0000","last_change":"2023-10-26T05:56:58.684439+0000","last_active":"2023-10-26T05:56:58.684540+0000","last_peered":"2023-10-26T05:56:58.684540+0000","last_clean":"2023-10-26T05:56:58.684540+0000","last_became_active":"2023-10-26T05:56:58.684093+0000","last_became_peered":"2023-10-26T05:56:58.684093+0000","last_unstale":"2023-10-26T05:56:58.684540+0000","last_undegraded":"2023-10-26T05:56:58.684540+0000","last_fullsized":"2023-10-26T05:56:58.684540+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:28.905001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699789+0000","last_change":"2023-10-26T05:56:58.699789+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699789+0000","last_undegraded":"2023-10-26T05:56:58.699789+0000","last_fullsized":"2023-10-26T05:56:58.699789+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703649+0000","last_change":"2023-10-26T05:56:58.703574+0000","last_active":"2023-10-26T05:56:58.703649+0000","last_peered":"2023-10-26T05:56:58.703649+0000","last_clean":"2023-10-26T05:56:58.703649+0000","last_became_active":"2023-10-26T05:56:58.703256+0000","last_became_peered":"2023-10-26T05:56:58.703256+0000","last_unstale":"2023-10-26T05:56:58.703649+0000","last_undegraded":"2023-10-26T05:56:58.703649+0000","last_fullsized":"2023-10-26T05:56:58.703649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:26.907325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695608+0000","last_change":"2023-10-26T05:56:58.695608+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695608+0000","last_undegraded":"2023-10-26T05:56:58.695608+0000","last_fullsized":"2023-10-26T05:56:58.695608+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708221+0000","last_change":"2023-10-26T05:56:58.707861+0000","last_active":"2023-10-26T05:56:58.708221+0000","last_peered":"2023-10-26T05:56:58.708221+0000","last_clean":"2023-10-26T05:56:58.708221+0000","last_became_active":"2023-10-26T05:56:58.707069+0000","last_became_peered":"2023-10-26T05:56:58.707069+0000","last_unstale":"2023-10-26T05:56:58.708221+0000","last_undegraded":"2023-10-26T05:56:58.708221+0000","last_fullsized":"2023-10-26T05:56:58.708221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:39.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681733+0000","last_change":"2023-10-26T05:56:58.681733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681733+0000","last_undegraded":"2023-10-26T05:56:58.681733+0000","last_fullsized":"2023-10-26T05:56:58.681733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699326+0000","last_change":"2023-10-26T05:56:58.699246+0000","last_active":"2023-10-26T05:56:58.699326+0000","last_peered":"2023-10-26T05:56:58.699326+0000","last_clean":"2023-10-26T05:56:58.699326+0000","last_became_active":"2023-10-26T05:56:58.698939+0000","last_became_peered":"2023-10-26T05:56:58.698939+0000","last_unstale":"2023-10-26T05:56:58.699326+0000","last_undegraded":"2023-10-26T05:56:58.699326+0000","last_fullsized":"2023-10-26T05:56:58.699326+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:17.788046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.690498+0000","last_change":"2023-10-26T05:56:58.690498+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.690498+0000","last_undegraded":"2023-10-26T05:56:58.690498+0000","last_fullsized":"2023-10-26T05:56:58.690498+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.696872+0000","last_change":"2023-10-26T05:56:58.696783+0000","last_active":"2023-10-26T05:56:58.696872+0000","last_peered":"2023-10-26T05:56:58.696872+0000","last_clean":"2023-10-26T05:56:58.696872+0000","last_became_active":"2023-10-26T05:56:58.696463+0000","last_became_peered":"2023-10-26T05:56:58.696463+0000","last_unstale":"2023-10-26T05:56:58.696872+0000","last_undegraded":"2023-10-26T05:56:58.696872+0000","last_fullsized":"2023-10-26T05:56:58.696872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:03:26.952948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685813+0000","last_change":"2023-10-26T05:56:58.685813+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685813+0000","last_undegraded":"2023-10-26T05:56:58.685813+0000","last_fullsized":"2023-10-26T05:56:58.685813+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.694434+0000","last_change":"2023-10-26T05:56:58.694349+0000","last_active":"2023-10-26T05:56:58.694434+0000","last_peered":"2023-10-26T05:56:58.694434+0000","last_clean":"2023-10-26T05:56:58.694434+0000","last_became_active":"2023-10-26T05:56:58.694033+0000","last_became_peered":"2023-10-26T05:56:58.694033+0000","last_unstale":"2023-10-26T05:56:58.694434+0000","last_undegraded":"2023-10-26T05:56:58.694434+0000","last_fullsized":"2023-10-26T05:56:58.694434+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:30:52.321543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684950+0000","last_change":"2023-10-26T05:56:58.684950+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684950+0000","last_undegraded":"2023-10-26T05:56:58.684950+0000","last_fullsized":"2023-10-26T05:56:58.684950+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703746+0000","last_change":"2023-10-26T05:56:58.703649+0000","last_active":"2023-10-26T05:56:58.703746+0000","last_peered":"2023-10-26T05:56:58.703746+0000","last_clean":"2023-10-26T05:56:58.703746+0000","last_became_active":"2023-10-26T05:56:58.703345+0000","last_became_peered":"2023-10-26T05:56:58.703345+0000","last_unstale":"2023-10-26T05:56:58.703746+0000","last_undegraded":"2023-10-26T05:56:58.703746+0000","last_fullsized":"2023-10-26T05:56:58.703746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:04.454516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685015+0000","last_change":"2023-10-26T05:56:58.685015+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685015+0000","last_undegraded":"2023-10-26T05:56:58.685015+0000","last_fullsized":"2023-10-26T05:56:58.685015+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713126+0000","last_change":"2023-10-26T05:56:58.712925+0000","last_active":"2023-10-26T05:56:58.713126+0000","last_peered":"2023-10-26T05:56:58.713126+0000","last_clean":"2023-10-26T05:56:58.713126+0000","last_became_active":"2023-10-26T05:56:58.712384+0000","last_became_peered":"2023-10-26T05:56:58.712384+0000","last_unstale":"2023-10-26T05:56:58.713126+0000","last_undegraded":"2023-10-26T05:56:58.713126+0000","last_fullsized":"2023-10-26T05:56:58.713126+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:18.155446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685000+0000","last_change":"2023-10-26T05:56:58.685000+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685000+0000","last_undegraded":"2023-10-26T05:56:58.685000+0000","last_fullsized":"2023-10-26T05:56:58.685000+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713004+0000","last_change":"2023-10-26T05:56:58.712807+0000","last_active":"2023-10-26T05:56:58.713004+0000","last_peered":"2023-10-26T05:56:58.713004+0000","last_clean":"2023-10-26T05:56:58.713004+0000","last_became_active":"2023-10-26T05:56:58.707471+0000","last_became_peered":"2023-10-26T05:56:58.707471+0000","last_unstale":"2023-10-26T05:56:58.713004+0000","last_undegraded":"2023-10-26T05:56:58.713004+0000","last_fullsized":"2023-10-26T05:56:58.713004+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:13:10.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684550+0000","last_change":"2023-10-26T05:56:58.684550+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684550+0000","last_undegraded":"2023-10-26T05:56:58.684550+0000","last_fullsized":"2023-10-26T05:56:58.684550+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708819+0000","last_change":"2023-10-26T05:56:58.707193+0000","last_active":"2023-10-26T05:56:58.708819+0000","last_peered":"2023-10-26T05:56:58.708819+0000","last_clean":"2023-10-26T05:56:58.708819+0000","last_became_active":"2023-10-26T05:56:58.705895+0000","last_became_peered":"2023-10-26T05:56:58.705895+0000","last_unstale":"2023-10-26T05:56:58.708819+0000","last_undegraded":"2023-10-26T05:56:58.708819+0000","last_fullsized":"2023-10-26T05:56:58.708819+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:33:43.161653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694164+0000","last_change":"2023-10-26T05:56:58.694164+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694164+0000","last_undegraded":"2023-10-26T05:56:58.694164+0000","last_fullsized":"2023-10-26T05:56:58.694164+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.717688+0000","last_change":"2023-10-26T05:56:58.717538+0000","last_active":"2023-10-26T05:56:58.717688+0000","last_peered":"2023-10-26T05:56:58.717688+0000","last_clean":"2023-10-26T05:56:58.717688+0000","last_became_active":"2023-10-26T05:56:58.716243+0000","last_became_peered":"2023-10-26T05:56:58.716243+0000","last_unstale":"2023-10-26T05:56:58.717688+0000","last_undegraded":"2023-10-26T05:56:58.717688+0000","last_fullsized":"2023-10-26T05:56:58.717688+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:55:25.695338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689729+0000","last_change":"2023-10-26T05:56:58.689729+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689729+0000","last_undegraded":"2023-10-26T05:56:58.689729+0000","last_fullsized":"2023-10-26T05:56:58.689729+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.697371+0000","last_change":"2023-10-26T05:56:58.697256+0000","last_active":"2023-10-26T05:56:58.697371+0000","last_peered":"2023-10-26T05:56:58.697371+0000","last_clean":"2023-10-26T05:56:58.697371+0000","last_became_active":"2023-10-26T05:56:58.696984+0000","last_became_peered":"2023-10-26T05:56:58.696984+0000","last_unstale":"2023-10-26T05:56:58.697371+0000","last_undegraded":"2023-10-26T05:56:58.697371+0000","last_fullsized":"2023-10-26T05:56:58.697371+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:47.521770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687331+0000","last_change":"2023-10-26T05:56:58.687331+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687331+0000","last_undegraded":"2023-10-26T05:56:58.687331+0000","last_fullsized":"2023-10-26T05:56:58.687331+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687335+0000","last_change":"2023-10-26T05:56:58.687247+0000","last_active":"2023-10-26T05:56:58.687335+0000","last_peered":"2023-10-26T05:56:58.687335+0000","last_clean":"2023-10-26T05:56:58.687335+0000","last_became_active":"2023-10-26T05:56:58.686900+0000","last_became_peered":"2023-10-26T05:56:58.686900+0000","last_unstale":"2023-10-26T05:56:58.687335+0000","last_undegraded":"2023-10-26T05:56:58.687335+0000","last_fullsized":"2023-10-26T05:56:58.687335+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:57:16.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685469+0000","last_change":"2023-10-26T05:56:58.685469+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685469+0000","last_undegraded":"2023-10-26T05:56:58.685469+0000","last_fullsized":"2023-10-26T05:56:58.685469+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.688086+0000","last_change":"2023-10-26T05:56:58.687941+0000","last_active":"2023-10-26T05:56:58.688086+0000","last_peered":"2023-10-26T05:56:58.688086+0000","last_clean":"2023-10-26T05:56:58.688086+0000","last_became_active":"2023-10-26T05:56:58.687118+0000","last_became_peered":"2023-10-26T05:56:58.687118+0000","last_unstale":"2023-10-26T05:56:58.688086+0000","last_undegraded":"2023-10-26T05:56:58.688086+0000","last_fullsized":"2023-10-26T05:56:58.688086+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:10.994025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681960+0000","last_change":"2023-10-26T05:56:58.681960+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681960+0000","last_undegraded":"2023-10-26T05:56:58.681960+0000","last_fullsized":"2023-10-26T05:56:58.681960+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692825+0000","last_change":"2023-10-26T05:56:58.692632+0000","last_active":"2023-10-26T05:56:58.692825+0000","last_peered":"2023-10-26T05:56:58.692825+0000","last_clean":"2023-10-26T05:56:58.692825+0000","last_became_active":"2023-10-26T05:56:58.690733+0000","last_became_peered":"2023-10-26T05:56:58.690733+0000","last_unstale":"2023-10-26T05:56:58.692825+0000","last_undegraded":"2023-10-26T05:56:58.692825+0000","last_fullsized":"2023-10-26T05:56:58.692825+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:59.579324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691809+0000","last_change":"2023-10-26T05:56:58.691809+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691809+0000","last_undegraded":"2023-10-26T05:56:58.691809+0000","last_fullsized":"2023-10-26T05:56:58.691809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.718038+0000","last_change":"2023-10-26T05:56:58.717853+0000","last_active":"2023-10-26T05:56:58.718038+0000","last_peered":"2023-10-26T05:56:58.718038+0000","last_clean":"2023-10-26T05:56:58.718038+0000","last_became_active":"2023-10-26T05:56:58.716922+0000","last_became_peered":"2023-10-26T05:56:58.716922+0000","last_unstale":"2023-10-26T05:56:58.718038+0000","last_undegraded":"2023-10-26T05:56:58.718038+0000","last_fullsized":"2023-10-26T05:56:58.718038+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:59:48.203254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692948+0000","last_change":"2023-10-26T05:56:58.692948+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692948+0000","last_undegraded":"2023-10-26T05:56:58.692948+0000","last_fullsized":"2023-10-26T05:56:58.692948+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711684+0000","last_change":"2023-10-26T05:56:58.709784+0000","last_active":"2023-10-26T05:56:58.711684+0000","last_peered":"2023-10-26T05:56:58.711684+0000","last_clean":"2023-10-26T05:56:58.711684+0000","last_became_active":"2023-10-26T05:56:58.707275+0000","last_became_peered":"2023-10-26T05:56:58.707275+0000","last_unstale":"2023-10-26T05:56:58.711684+0000","last_undegraded":"2023-10-26T05:56:58.711684+0000","last_fullsized":"2023-10-26T05:56:58.711684+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:25:35.872395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693336+0000","last_change":"2023-10-26T05:56:58.693336+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693336+0000","last_undegraded":"2023-10-26T05:56:58.693336+0000","last_fullsized":"2023-10-26T05:56:58.693336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699845+0000","last_change":"2023-10-26T05:56:58.699782+0000","last_active":"2023-10-26T05:56:58.699845+0000","last_peered":"2023-10-26T05:56:58.699845+0000","last_clean":"2023-10-26T05:56:58.699845+0000","last_became_active":"2023-10-26T05:56:58.699054+0000","last_became_peered":"2023-10-26T05:56:58.699054+0000","last_unstale":"2023-10-26T05:56:58.699845+0000","last_undegraded":"2023-10-26T05:56:58.699845+0000","last_fullsized":"2023-10-26T05:56:58.699845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:55.048724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685430+0000","last_change":"2023-10-26T05:56:58.685430+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685430+0000","last_undegraded":"2023-10-26T05:56:58.685430+0000","last_fullsized":"2023-10-26T05:56:58.685430+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711741+0000","last_change":"2023-10-26T05:56:58.710168+0000","last_active":"2023-10-26T05:56:58.711741+0000","last_peered":"2023-10-26T05:56:58.711741+0000","last_clean":"2023-10-26T05:56:58.711741+0000","last_became_active":"2023-10-26T05:56:58.708495+0000","last_became_peered":"2023-10-26T05:56:58.708495+0000","last_unstale":"2023-10-26T05:56:58.711741+0000","last_undegraded":"2023-10-26T05:56:58.711741+0000","last_fullsized":"2023-10-26T05:56:58.711741+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:24.511118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-26T05:56:57.680722+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:56:57.680722+0000","last_peered":"2023-10-26T05:56:57.680722+0000","last_clean":"2023-10-26T05:56:57.680722+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:56:57.680722+0000","last_undegraded":"2023-10-26T05:56:57.680722+0000","last_fullsized":"2023-10-26T05:56:57.680722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.59099999999999997,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3100000000000001},"last":1.1899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":1.0029999999999999},"min":{"1min":0.64700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.232,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.032,"15min":0.98799999999999999},"min":{"1min":0.97999999999999998,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.139}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.68000000000000005,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1899999999999999,"5min":1.448,"15min":1.448},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93799999999999994,"15min":0.93400000000000005},"min":{"1min":0.63900000000000001,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1990000000000001,"5min":1.3700000000000001,"15min":1.4590000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93200000000000005,"15min":0.89800000000000002},"min":{"1min":0.76000000000000001,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":1.004},{"interface":"front","average":{"1min":1.002,"5min":0.96699999999999997,"15min":0.97199999999999998},"min":{"1min":0.77300000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3089999999999999,"5min":1.3129999999999999,"15min":1.375},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.133,"5min":1.639,"15min":1.639},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.93300000000000005,"15min":0.93999999999999995},"min":{"1min":0.65700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.24,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.337,"5min":1.482,"15min":1.6679999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0369999999999999,"15min":1.0640000000000001},"min":{"1min":0.72299999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.702,"15min":1.702},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.115,"15min":1.1060000000000001},"min":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.3420000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.5,"15min":1.6180000000000001},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.234,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.087},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.69999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.86599999999999999},"min":{"1min":0.64500000000000002,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.385,"5min":1.3899999999999999,"15min":1.405},"last":0.95499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.91000000000000003,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2450000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.2809999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97999999999999998,"15min":0.92500000000000004},"min":{"1min":0.76900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.3080000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0620000000000001,"15min":1.0009999999999999},"min":{"1min":0.69099999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.337,"5min":1.3759999999999999,"15min":1.399},"last":1.212}]},{"osd":4,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.503,"15min":0.502},"max":{"1min":1.2569999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.99299999999999999,"15min":0.98099999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.111}]},{"osd":5,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.05},"min":{"1min":0.89200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.236,"5min":1.5509999999999999,"15min":1.554},"last":1.028},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97199999999999998,"15min":0.98299999999999998},"min":{"1min":0.67200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.452,"5min":1.512,"15min":1.512},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1220000000000001,"15min":1.1120000000000001},"min":{"1min":0.78400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.321,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94499999999999995,"15min":0.95599999999999996},"min":{"1min":0.63400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.579},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.83599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":1.591,"15min":1.591},"last":1.3400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84699999999999998,"15min":0.878},"min":{"1min":0.625,"5min":0.53300000000000003,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.206,"15min":2.1150000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86799999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2430000000000001,"5min":1.272,"15min":1.464},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.70599999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.2170000000000001,"5min":1.569,"15min":1.569},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0640000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90900000000000003,"15min":0.91800000000000004},"min":{"1min":0.69799999999999995,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.363,"5min":1.448,"15min":1.548},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.78200000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.276,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.97599999999999998},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.286,"15min":2.1909999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0760000000000001,"15min":1.091},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.536,"15min":1.536},"last":1.194}]},{"osd":6,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.68000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.268,"5min":1.744,"15min":1.744},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.04,"15min":1.032},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":1.4279999999999999,"15min":1.4950000000000001},"last":1.161}]},{"osd":7,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.071},"min":{"1min":0.64100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.3109999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3360000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.92000000000000004,"15min":0.871},"min":{"1min":0.66200000000000003,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1919999999999999,"5min":1.2569999999999999,"15min":1.347},"last":1.075},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.70799999999999996,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.159,"5min":1.502,"15min":1.502},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.90400000000000003,"15min":0.84499999999999997},"min":{"1min":0.749,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.21,"5min":1.25,"15min":1.375},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.96099999999999997,"15min":0.91700000000000004},"min":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":1.198}]},{"osd":2,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.143},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90700000000000003,"15min":0.879},"min":{"1min":0.72899999999999998,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.143,"5min":1.2170000000000001,"15min":1.373},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":0.95199999999999996},"min":{"1min":0.79700000000000004,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.177,"5min":1.397,"15min":2.0190000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.091,"5min":0.995,"15min":0.96199999999999997},"min":{"1min":0.79100000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.28,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.2430000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0009999999999999,"15min":1},"min":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.2929999999999999,"5min":1.4179999999999999,"15min":1.673},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0429999999999999,"15min":1.014},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.42099999999999999},"max":{"1min":1.337,"5min":1.762,"15min":1.7969999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.054,"15min":1.042},"min":{"1min":0.76500000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.4970000000000001},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.103,"15min":1.0589999999999999},"min":{"1min":0.77300000000000002,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.3280000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.879,"5min":0.69599999999999995,"15min":0.53700000000000003},"max":{"1min":1.341,"5min":1.484,"15min":1.728},"last":1.2709999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.90100000000000002,"5min":1.385,"15min":1.385},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.52500000000000002,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.2689999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":1.091,"5min":1.2,"15min":1.349},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.38,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.02,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.018}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.752,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1479999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81999999999999995,"15min":0.78700000000000003},"min":{"1min":0.623,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.1850000000000001,"15min":1.4199999999999999},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.84399999999999997},"min":{"1min":0.70799999999999996,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.1120000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90200000000000002,"15min":0.91900000000000004},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.2789999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.72399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3,"5min":1.413,"15min":1.478},"last":1.454},{"interface":"front","average":{"1min":1.014,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.54,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.64200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.349,"15min":1.6060000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.091,"5min":0.96699999999999997,"15min":1.014},"min":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":1.359,"15min":5.6859999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.98599999999999999,"15min":0.94299999999999995},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.53400000000000003},"max":{"1min":1.472,"5min":2.016,"15min":2.016},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.008,"15min":1.032},"min":{"1min":0.68799999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.327,"5min":2.004,"15min":2.004},"last":1.2370000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71999999999999997,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.379,"15min":1.5249999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.69599999999999995},"min":{"1min":0.61699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.091,"5min":1.226,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.621},"min":{"1min":0.57799999999999996,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.90700000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.82199999999999995,"15min":0.76500000000000001},"min":{"1min":0.71599999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.24,"15min":2.8809999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94899999999999995,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93000000000000005,"15min":0.85999999999999999},"min":{"1min":0.69099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.151,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88500000000000001,"15min":0.85899999999999999},"min":{"1min":0.78200000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.997,"15min":0.93899999999999995},"min":{"1min":0.84999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.999,"15min":0.93100000000000005},"min":{"1min":0.80000000000000004,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.329,"5min":1.4790000000000001,"15min":1.653},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.008,"15min":0.94699999999999995},"min":{"1min":0.73199999999999998,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.3320000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.089,"15min":1.016},"min":{"1min":0.76500000000000001,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.3300000000000001,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.177}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":0.91900000000000004},"min":{"1min":0.84299999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.073,"5min":1.581,"15min":1.581},"last":1.119},{"interface":"front","average":{"1min":1.133,"5min":1.109,"15min":1.069},"min":{"1min":0.91200000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.462,"5min":1.823,"15min":15.352},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.157},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55600000000000005,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78300000000000003,"15min":0.73999999999999999},"min":{"1min":0.63700000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.073,"5min":1.6080000000000001,"15min":1.8169999999999999},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.754,"15min":0.76100000000000001},"min":{"1min":0.65600000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.147,"15min":1.7949999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.0760000000000001,"5min":1.298,"15min":1.298},"last":0.997}]},{"osd":4,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.874},"min":{"1min":0.73899999999999999,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1020000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.74099999999999999,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.345,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":1.2}]},{"osd":5,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.96299999999999997,"15min":0.96399999999999997},"min":{"1min":0.749,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.319,"5min":1.381,"15min":1.492},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.84899999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.2509999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0249999999999999},"min":{"1min":0.89200000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89700000000000002,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.2669999999999999,"5min":1.468,"15min":1.837},"last":1.155}]},{"osd":7,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.046,"15min":1.0309999999999999},"min":{"1min":0.82499999999999996,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.397,"5min":1.4179999999999999,"15min":1.6339999999999999},"last":1.294},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0640000000000001,"15min":1.0840000000000001},"min":{"1min":0.93100000000000005,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4890000000000001,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":1.0600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95599999999999996,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.39100000000000001,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.1020000000000001,"15min":1.177},"last":0.502}]},{"osd":3,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0249999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.72899999999999998},"min":{"1min":0.58999999999999997,"5min":0.32100000000000001,"15min":0.247},"max":{"1min":1.026,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91400000000000003,"15min":0.873},"min":{"1min":0.64200000000000002,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.319,"5min":1.395,"15min":1.395},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":0.93500000000000005},"min":{"1min":0.73699999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":1.1559999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.92200000000000004},"min":{"1min":0.61299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":2.7480000000000002},"last":1.073},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96299999999999997,"15min":0.92100000000000004},"min":{"1min":0.87,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.91900000000000004},"min":{"1min":0.77500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1639999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.131,"15min":1.0169999999999999},"min":{"1min":0.871,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.2829999999999999,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0109999999999999,"15min":0.998},"min":{"1min":0.74199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1539999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0329999999999999,"15min":0.999},"min":{"1min":0.93500000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.417,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":1.141}]}]}],"pool_statfs":[]}} 2023-10-26T05:57:02.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:57:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa5a383640 1 -- 172.21.15.71:0/1865912425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 msgr2=0x7ffa3c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa5a383640 1 --2- 172.21.15.71:0/1865912425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa3c004c50 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009b30 tx=0x7ffa4801d190 comp rx=0 tx=0).stop 2023-10-26T05:57:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa58920640 1 -- 172.21.15.71:0/1865912425 reap_dead start 2023-10-26T05:57:02.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa5a383640 1 -- 172.21.15.71:0/1865912425 shutdown_connections 2023-10-26T05:57:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa5a383640 1 -- 172.21.15.71:0/1865912425 >> 172.21.15.71:0/1865912425 conn(0x7ffa54069770 msgr2=0x7ffa54069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa5a383640 1 -- 172.21.15.71:0/1865912425 shutdown_connections 2023-10-26T05:57:02.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.851+0000 7ffa5a383640 1 -- 172.21.15.71:0/1865912425 wait complete. 2023-10-26T05:57:02.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 Processor -- start 2023-10-26T05:57:02.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- start start 2023-10-26T05:57:02.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa5413f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5413a1d0 0x7ffa5413c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa5413cf00 0x7ffa54160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5413d3b0 con 0x7ffa3c0047d0 2023-10-26T05:57:02.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54154fc0 con 0x7ffa5413a1d0 2023-10-26T05:57:02.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5413f770 con 0x7ffa5413cf00 2023-10-26T05:57:02.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa5413f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa5413f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35776/0 (socket says 172.21.15.71:35776) 2023-10-26T05:57:02.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 -- 172.21.15.71:0/3983060165 learned_addr learned my addr 172.21.15.71:0/3983060165 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:02.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa537fe640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5413a1d0 0x7ffa5413c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa58920640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa5413cf00 0x7ffa54160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 -- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa5413cf00 msgr2=0x7ffa54160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa5413cf00 0x7ffa54160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 -- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5413a1d0 msgr2=0x7ffa5413c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:02.868 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5413a1d0 0x7ffa5413c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:02.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 -- 172.21.15.71:0/3983060165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa541543b0 con 0x7ffa3c0047d0 2023-10-26T05:57:02.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa537fe640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5413a1d0 0x7ffa5413c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:57:02.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa53fff640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa5413f230 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400e9c0 tx=0x7ffa4400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:02.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400cc60 con 0x7ffa3c0047d0 2023-10-26T05:57:02.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ffa4400ceb0 con 0x7ffa3c0047d0 2023-10-26T05:57:02.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa54154690 con 0x7ffa3c0047d0 2023-10-26T05:57:02.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44011690 con 0x7ffa3c0047d0 2023-10-26T05:57:02.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa54154c50 con 0x7ffa3c0047d0 2023-10-26T05:57:02.870 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.855+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa54066b10 con 0x7ffa3c0047d0 2023-10-26T05:57:02.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.859+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffa4401a400 con 0x7ffa3c0047d0 2023-10-26T05:57:02.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.859+0000 7ffa50ff9640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffa400664a0 0x7ffa40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:02.871 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.859+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa44084d70 con 0x7ffa3c0047d0 2023-10-26T05:57:02.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.859+0000 7ffa537fe640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffa400664a0 0x7ffa40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:02.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.859+0000 7ffa537fe640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffa400664a0 0x7ffa40068960 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffa480ae000 tx=0x7ffa48019f30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:02.872 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.863+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ffa44055ea0 con 0x7ffa3c0047d0 2023-10-26T05:57:03.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:02.999+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa541662d0 con 0x7ffa400664a0 2023-10-26T05:57:03.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa50ff9640 1 -- 172.21.15.71:0/3983060165 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278382 (secure 0 0 0) 0x7ffa541662d0 con 0x7ffa400664a0 2023-10-26T05:57:03.032 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:03.034 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:57:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffa400664a0 msgr2=0x7ffa40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffa400664a0 0x7ffa40068960 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffa480ae000 tx=0x7ffa48019f30 comp rx=0 tx=0).stop 2023-10-26T05:57:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 msgr2=0x7ffa5413f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa5413f230 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400e9c0 tx=0x7ffa4400ee80 comp rx=0 tx=0).stop 2023-10-26T05:57:03.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 shutdown_connections 2023-10-26T05:57:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffa5413cf00 0x7ffa54160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa5413a1d0 0x7ffa5413c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ffa400664a0 0x7ffa40068960 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 --2- 172.21.15.71:0/3983060165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffa3c0047d0 0x7ffa5413f230 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 >> 172.21.15.71:0/3983060165 conn(0x7ffa54069770 msgr2=0x7ffa5413d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:03.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 shutdown_connections 2023-10-26T05:57:03.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.023+0000 7ffa5a383640 1 -- 172.21.15.71:0/3983060165 wait complete. 2023-10-26T05:57:03.049 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2023-10-26T05:57:01.607623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553911,"statfs":{"total":1600000000,"available":1591205414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001501"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687936+0000","last_change":"2023-10-26T05:56:58.687936+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687936+0000","last_undegraded":"2023-10-26T05:56:58.687936+0000","last_fullsized":"2023-10-26T05:56:58.687936+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689543+0000","last_change":"2023-10-26T05:56:58.689543+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689543+0000","last_undegraded":"2023-10-26T05:56:58.689543+0000","last_fullsized":"2023-10-26T05:56:58.689543+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.701504+0000","last_change":"2023-10-26T05:56:58.701504+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.701504+0000","last_undegraded":"2023-10-26T05:56:58.701504+0000","last_fullsized":"2023-10-26T05:56:58.701504+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687487+0000","last_change":"2023-10-26T05:56:58.687487+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687487+0000","last_undegraded":"2023-10-26T05:56:58.687487+0000","last_fullsized":"2023-10-26T05:56:58.687487+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699011+0000","last_change":"2023-10-26T05:56:58.699011+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699011+0000","last_undegraded":"2023-10-26T05:56:58.699011+0000","last_fullsized":"2023-10-26T05:56:58.699011+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687527+0000","last_change":"2023-10-26T05:56:58.687527+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687527+0000","last_undegraded":"2023-10-26T05:56:58.687527+0000","last_fullsized":"2023-10-26T05:56:58.687527+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684911+0000","last_change":"2023-10-26T05:56:58.684911+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684911+0000","last_undegraded":"2023-10-26T05:56:58.684911+0000","last_fullsized":"2023-10-26T05:56:58.684911+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686493+0000","last_change":"2023-10-26T05:56:58.686493+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686493+0000","last_undegraded":"2023-10-26T05:56:58.686493+0000","last_fullsized":"2023-10-26T05:56:58.686493+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686070+0000","last_change":"2023-10-26T05:56:58.686070+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686070+0000","last_undegraded":"2023-10-26T05:56:58.686070+0000","last_fullsized":"2023-10-26T05:56:58.686070+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697900+0000","last_change":"2023-10-26T05:56:58.697900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697900+0000","last_undegraded":"2023-10-26T05:56:58.697900+0000","last_fullsized":"2023-10-26T05:56:58.697900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687124+0000","last_change":"2023-10-26T05:56:58.687124+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687124+0000","last_undegraded":"2023-10-26T05:56:58.687124+0000","last_fullsized":"2023-10-26T05:56:58.687124+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.700372+0000","last_change":"2023-10-26T05:56:58.700372+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.700372+0000","last_undegraded":"2023-10-26T05:56:58.700372+0000","last_fullsized":"2023-10-26T05:56:58.700372+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695733+0000","last_change":"2023-10-26T05:56:58.695733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695733+0000","last_undegraded":"2023-10-26T05:56:58.695733+0000","last_fullsized":"2023-10-26T05:56:58.695733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685971+0000","last_change":"2023-10-26T05:56:58.685971+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685971+0000","last_undegraded":"2023-10-26T05:56:58.685971+0000","last_fullsized":"2023-10-26T05:56:58.685971+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697158+0000","last_change":"2023-10-26T05:56:58.697158+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697158+0000","last_undegraded":"2023-10-26T05:56:58.697158+0000","last_fullsized":"2023-10-26T05:56:58.697158+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682093+0000","last_change":"2023-10-26T05:56:58.682093+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682093+0000","last_undegraded":"2023-10-26T05:56:58.682093+0000","last_fullsized":"2023-10-26T05:56:58.682093+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693783+0000","last_change":"2023-10-26T05:56:58.693783+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693783+0000","last_undegraded":"2023-10-26T05:56:58.693783+0000","last_fullsized":"2023-10-26T05:56:58.693783+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.702305+0000","last_change":"2023-10-26T05:56:58.702305+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.702305+0000","last_undegraded":"2023-10-26T05:56:58.702305+0000","last_fullsized":"2023-10-26T05:56:58.702305+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686171+0000","last_change":"2023-10-26T05:56:58.686171+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686171+0000","last_undegraded":"2023-10-26T05:56:58.686171+0000","last_fullsized":"2023-10-26T05:56:58.686171+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684098+0000","last_change":"2023-10-26T05:56:58.684098+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684098+0000","last_undegraded":"2023-10-26T05:56:58.684098+0000","last_fullsized":"2023-10-26T05:56:58.684098+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693775+0000","last_change":"2023-10-26T05:56:58.693775+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693775+0000","last_undegraded":"2023-10-26T05:56:58.693775+0000","last_fullsized":"2023-10-26T05:56:58.693775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.688204+0000","last_change":"2023-10-26T05:56:58.688204+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.688204+0000","last_undegraded":"2023-10-26T05:56:58.688204+0000","last_fullsized":"2023-10-26T05:56:58.688204+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684900+0000","last_change":"2023-10-26T05:56:58.684900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684900+0000","last_undegraded":"2023-10-26T05:56:58.684900+0000","last_fullsized":"2023-10-26T05:56:58.684900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694702+0000","last_change":"2023-10-26T05:56:58.694702+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694702+0000","last_undegraded":"2023-10-26T05:56:58.694702+0000","last_fullsized":"2023-10-26T05:56:58.694702+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683660+0000","last_change":"2023-10-26T05:56:58.683660+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683660+0000","last_undegraded":"2023-10-26T05:56:58.683660+0000","last_fullsized":"2023-10-26T05:56:58.683660+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686703+0000","last_change":"2023-10-26T05:56:58.686703+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686703+0000","last_undegraded":"2023-10-26T05:56:58.686703+0000","last_fullsized":"2023-10-26T05:56:58.686703+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682875+0000","last_change":"2023-10-26T05:56:58.682875+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682875+0000","last_undegraded":"2023-10-26T05:56:58.682875+0000","last_fullsized":"2023-10-26T05:56:58.682875+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685434+0000","last_change":"2023-10-26T05:56:58.685434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685434+0000","last_undegraded":"2023-10-26T05:56:58.685434+0000","last_fullsized":"2023-10-26T05:56:58.685434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685841+0000","last_change":"2023-10-26T05:56:58.685841+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685841+0000","last_undegraded":"2023-10-26T05:56:58.685841+0000","last_fullsized":"2023-10-26T05:56:58.685841+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685847+0000","last_change":"2023-10-26T05:56:58.685847+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685847+0000","last_undegraded":"2023-10-26T05:56:58.685847+0000","last_fullsized":"2023-10-26T05:56:58.685847+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684393+0000","last_change":"2023-10-26T05:56:58.684393+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684393+0000","last_undegraded":"2023-10-26T05:56:58.684393+0000","last_fullsized":"2023-10-26T05:56:58.684393+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682434+0000","last_change":"2023-10-26T05:56:58.682434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682434+0000","last_undegraded":"2023-10-26T05:56:58.682434+0000","last_fullsized":"2023-10-26T05:56:58.682434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693816+0000","last_change":"2023-10-26T05:56:58.693816+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693816+0000","last_undegraded":"2023-10-26T05:56:58.693816+0000","last_fullsized":"2023-10-26T05:56:58.693816+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711649+0000","last_change":"2023-10-26T05:56:58.711496+0000","last_active":"2023-10-26T05:56:58.711649+0000","last_peered":"2023-10-26T05:56:58.711649+0000","last_clean":"2023-10-26T05:56:58.711649+0000","last_became_active":"2023-10-26T05:56:58.710397+0000","last_became_peered":"2023-10-26T05:56:58.710397+0000","last_unstale":"2023-10-26T05:56:58.711649+0000","last_undegraded":"2023-10-26T05:56:58.711649+0000","last_fullsized":"2023-10-26T05:56:58.711649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:45:28.308338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683676+0000","last_change":"2023-10-26T05:56:58.683676+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683676+0000","last_undegraded":"2023-10-26T05:56:58.683676+0000","last_fullsized":"2023-10-26T05:56:58.683676+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699941+0000","last_change":"2023-10-26T05:56:58.699902+0000","last_active":"2023-10-26T05:56:58.699941+0000","last_peered":"2023-10-26T05:56:58.699941+0000","last_clean":"2023-10-26T05:56:58.699941+0000","last_became_active":"2023-10-26T05:56:58.699481+0000","last_became_peered":"2023-10-26T05:56:58.699481+0000","last_unstale":"2023-10-26T05:56:58.699941+0000","last_undegraded":"2023-10-26T05:56:58.699941+0000","last_fullsized":"2023-10-26T05:56:58.699941+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:15.048906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686261+0000","last_change":"2023-10-26T05:56:58.686261+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686261+0000","last_undegraded":"2023-10-26T05:56:58.686261+0000","last_fullsized":"2023-10-26T05:56:58.686261+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692322+0000","last_change":"2023-10-26T05:56:58.691994+0000","last_active":"2023-10-26T05:56:58.692322+0000","last_peered":"2023-10-26T05:56:58.692322+0000","last_clean":"2023-10-26T05:56:58.692322+0000","last_became_active":"2023-10-26T05:56:58.691381+0000","last_became_peered":"2023-10-26T05:56:58.691381+0000","last_unstale":"2023-10-26T05:56:58.692322+0000","last_undegraded":"2023-10-26T05:56:58.692322+0000","last_fullsized":"2023-10-26T05:56:58.692322+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:13.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.696248+0000","last_change":"2023-10-26T05:56:58.696248+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.696248+0000","last_undegraded":"2023-10-26T05:56:58.696248+0000","last_fullsized":"2023-10-26T05:56:58.696248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687988+0000","last_change":"2023-10-26T05:56:58.687804+0000","last_active":"2023-10-26T05:56:58.687988+0000","last_peered":"2023-10-26T05:56:58.687988+0000","last_clean":"2023-10-26T05:56:58.687988+0000","last_became_active":"2023-10-26T05:56:58.687167+0000","last_became_peered":"2023-10-26T05:56:58.687167+0000","last_unstale":"2023-10-26T05:56:58.687988+0000","last_undegraded":"2023-10-26T05:56:58.687988+0000","last_fullsized":"2023-10-26T05:56:58.687988+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:47.280630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683215+0000","last_change":"2023-10-26T05:56:58.683215+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683215+0000","last_undegraded":"2023-10-26T05:56:58.683215+0000","last_fullsized":"2023-10-26T05:56:58.683215+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.709083+0000","last_change":"2023-10-26T05:56:58.708982+0000","last_active":"2023-10-26T05:56:58.709083+0000","last_peered":"2023-10-26T05:56:58.709083+0000","last_clean":"2023-10-26T05:56:58.709083+0000","last_became_active":"2023-10-26T05:56:58.708308+0000","last_became_peered":"2023-10-26T05:56:58.708308+0000","last_unstale":"2023-10-26T05:56:58.709083+0000","last_undegraded":"2023-10-26T05:56:58.709083+0000","last_fullsized":"2023-10-26T05:56:58.709083+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:39.204476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685163+0000","last_change":"2023-10-26T05:56:58.685163+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685163+0000","last_undegraded":"2023-10-26T05:56:58.685163+0000","last_fullsized":"2023-10-26T05:56:58.685163+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.707771+0000","last_change":"2023-10-26T05:56:58.707555+0000","last_active":"2023-10-26T05:56:58.707771+0000","last_peered":"2023-10-26T05:56:58.707771+0000","last_clean":"2023-10-26T05:56:58.707771+0000","last_became_active":"2023-10-26T05:56:58.706918+0000","last_became_peered":"2023-10-26T05:56:58.706918+0000","last_unstale":"2023-10-26T05:56:58.707771+0000","last_undegraded":"2023-10-26T05:56:58.707771+0000","last_fullsized":"2023-10-26T05:56:58.707771+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:12:04.008423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686926+0000","last_change":"2023-10-26T05:56:58.686926+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686926+0000","last_undegraded":"2023-10-26T05:56:58.686926+0000","last_fullsized":"2023-10-26T05:56:58.686926+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.702597+0000","last_change":"2023-10-26T05:56:58.702478+0000","last_active":"2023-10-26T05:56:58.702597+0000","last_peered":"2023-10-26T05:56:58.702597+0000","last_clean":"2023-10-26T05:56:58.702597+0000","last_became_active":"2023-10-26T05:56:58.702066+0000","last_became_peered":"2023-10-26T05:56:58.702066+0000","last_unstale":"2023-10-26T05:56:58.702597+0000","last_undegraded":"2023-10-26T05:56:58.702597+0000","last_fullsized":"2023-10-26T05:56:58.702597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:10.521714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.698412+0000","last_change":"2023-10-26T05:56:58.698412+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.698412+0000","last_undegraded":"2023-10-26T05:56:58.698412+0000","last_fullsized":"2023-10-26T05:56:58.698412+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.705203+0000","last_change":"2023-10-26T05:56:58.705010+0000","last_active":"2023-10-26T05:56:58.705203+0000","last_peered":"2023-10-26T05:56:58.705203+0000","last_clean":"2023-10-26T05:56:58.705203+0000","last_became_active":"2023-10-26T05:56:58.704379+0000","last_became_peered":"2023-10-26T05:56:58.704379+0000","last_unstale":"2023-10-26T05:56:58.705203+0000","last_undegraded":"2023-10-26T05:56:58.705203+0000","last_fullsized":"2023-10-26T05:56:58.705203+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:22.102825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691840+0000","last_change":"2023-10-26T05:56:58.691840+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691840+0000","last_undegraded":"2023-10-26T05:56:58.691840+0000","last_fullsized":"2023-10-26T05:56:58.691840+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692523+0000","last_change":"2023-10-26T05:56:58.692438+0000","last_active":"2023-10-26T05:56:58.692523+0000","last_peered":"2023-10-26T05:56:58.692523+0000","last_clean":"2023-10-26T05:56:58.692523+0000","last_became_active":"2023-10-26T05:56:58.692192+0000","last_became_peered":"2023-10-26T05:56:58.692192+0000","last_unstale":"2023-10-26T05:56:58.692523+0000","last_undegraded":"2023-10-26T05:56:58.692523+0000","last_fullsized":"2023-10-26T05:56:58.692523+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:20:10.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692914+0000","last_change":"2023-10-26T05:56:58.692914+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692914+0000","last_undegraded":"2023-10-26T05:56:58.692914+0000","last_fullsized":"2023-10-26T05:56:58.692914+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.715857+0000","last_change":"2023-10-26T05:56:58.715723+0000","last_active":"2023-10-26T05:56:58.715857+0000","last_peered":"2023-10-26T05:56:58.715857+0000","last_clean":"2023-10-26T05:56:58.715857+0000","last_became_active":"2023-10-26T05:56:58.714758+0000","last_became_peered":"2023-10-26T05:56:58.714758+0000","last_unstale":"2023-10-26T05:56:58.715857+0000","last_undegraded":"2023-10-26T05:56:58.715857+0000","last_fullsized":"2023-10-26T05:56:58.715857+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:50.434850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682474+0000","last_change":"2023-10-26T05:56:58.682474+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682474+0000","last_undegraded":"2023-10-26T05:56:58.682474+0000","last_fullsized":"2023-10-26T05:56:58.682474+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.691946+0000","last_change":"2023-10-26T05:56:58.691788+0000","last_active":"2023-10-26T05:56:58.691946+0000","last_peered":"2023-10-26T05:56:58.691946+0000","last_clean":"2023-10-26T05:56:58.691946+0000","last_became_active":"2023-10-26T05:56:58.691194+0000","last_became_peered":"2023-10-26T05:56:58.691194+0000","last_unstale":"2023-10-26T05:56:58.691946+0000","last_undegraded":"2023-10-26T05:56:58.691946+0000","last_fullsized":"2023-10-26T05:56:58.691946+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:05:11.942603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689507+0000","last_change":"2023-10-26T05:56:58.689507+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689507+0000","last_undegraded":"2023-10-26T05:56:58.689507+0000","last_fullsized":"2023-10-26T05:56:58.689507+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.700529+0000","last_change":"2023-10-26T05:56:58.700296+0000","last_active":"2023-10-26T05:56:58.700529+0000","last_peered":"2023-10-26T05:56:58.700529+0000","last_clean":"2023-10-26T05:56:58.700529+0000","last_became_active":"2023-10-26T05:56:58.699651+0000","last_became_peered":"2023-10-26T05:56:58.699651+0000","last_unstale":"2023-10-26T05:56:58.700529+0000","last_undegraded":"2023-10-26T05:56:58.700529+0000","last_fullsized":"2023-10-26T05:56:58.700529+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:42:59.754770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687731+0000","last_change":"2023-10-26T05:56:58.687731+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687731+0000","last_undegraded":"2023-10-26T05:56:58.687731+0000","last_fullsized":"2023-10-26T05:56:58.687731+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708870+0000","last_change":"2023-10-26T05:56:58.708596+0000","last_active":"2023-10-26T05:56:58.708870+0000","last_peered":"2023-10-26T05:56:58.708870+0000","last_clean":"2023-10-26T05:56:58.708870+0000","last_became_active":"2023-10-26T05:56:58.707912+0000","last_became_peered":"2023-10-26T05:56:58.707912+0000","last_unstale":"2023-10-26T05:56:58.708870+0000","last_undegraded":"2023-10-26T05:56:58.708870+0000","last_fullsized":"2023-10-26T05:56:58.708870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:40:53.740361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694993+0000","last_change":"2023-10-26T05:56:58.694993+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694993+0000","last_undegraded":"2023-10-26T05:56:58.694993+0000","last_fullsized":"2023-10-26T05:56:58.694993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.684540+0000","last_change":"2023-10-26T05:56:58.684439+0000","last_active":"2023-10-26T05:56:58.684540+0000","last_peered":"2023-10-26T05:56:58.684540+0000","last_clean":"2023-10-26T05:56:58.684540+0000","last_became_active":"2023-10-26T05:56:58.684093+0000","last_became_peered":"2023-10-26T05:56:58.684093+0000","last_unstale":"2023-10-26T05:56:58.684540+0000","last_undegraded":"2023-10-26T05:56:58.684540+0000","last_fullsized":"2023-10-26T05:56:58.684540+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:28.905001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699789+0000","last_change":"2023-10-26T05:56:58.699789+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699789+0000","last_undegraded":"2023-10-26T05:56:58.699789+0000","last_fullsized":"2023-10-26T05:56:58.699789+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703649+0000","last_change":"2023-10-26T05:56:58.703574+0000","last_active":"2023-10-26T05:56:58.703649+0000","last_peered":"2023-10-26T05:56:58.703649+0000","last_clean":"2023-10-26T05:56:58.703649+0000","last_became_active":"2023-10-26T05:56:58.703256+0000","last_became_peered":"2023-10-26T05:56:58.703256+0000","last_unstale":"2023-10-26T05:56:58.703649+0000","last_undegraded":"2023-10-26T05:56:58.703649+0000","last_fullsized":"2023-10-26T05:56:58.703649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:26.907325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695608+0000","last_change":"2023-10-26T05:56:58.695608+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695608+0000","last_undegraded":"2023-10-26T05:56:58.695608+0000","last_fullsized":"2023-10-26T05:56:58.695608+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708221+0000","last_change":"2023-10-26T05:56:58.707861+0000","last_active":"2023-10-26T05:56:58.708221+0000","last_peered":"2023-10-26T05:56:58.708221+0000","last_clean":"2023-10-26T05:56:58.708221+0000","last_became_active":"2023-10-26T05:56:58.707069+0000","last_became_peered":"2023-10-26T05:56:58.707069+0000","last_unstale":"2023-10-26T05:56:58.708221+0000","last_undegraded":"2023-10-26T05:56:58.708221+0000","last_fullsized":"2023-10-26T05:56:58.708221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:39.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681733+0000","last_change":"2023-10-26T05:56:58.681733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681733+0000","last_undegraded":"2023-10-26T05:56:58.681733+0000","last_fullsized":"2023-10-26T05:56:58.681733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699326+0000","last_change":"2023-10-26T05:56:58.699246+0000","last_active":"2023-10-26T05:56:58.699326+0000","last_peered":"2023-10-26T05:56:58.699326+0000","last_clean":"2023-10-26T05:56:58.699326+0000","last_became_active":"2023-10-26T05:56:58.698939+0000","last_became_peered":"2023-10-26T05:56:58.698939+0000","last_unstale":"2023-10-26T05:56:58.699326+0000","last_undegraded":"2023-10-26T05:56:58.699326+0000","last_fullsized":"2023-10-26T05:56:58.699326+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:17.788046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.690498+0000","last_change":"2023-10-26T05:56:58.690498+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.690498+0000","last_undegraded":"2023-10-26T05:56:58.690498+0000","last_fullsized":"2023-10-26T05:56:58.690498+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.696872+0000","last_change":"2023-10-26T05:56:58.696783+0000","last_active":"2023-10-26T05:56:58.696872+0000","last_peered":"2023-10-26T05:56:58.696872+0000","last_clean":"2023-10-26T05:56:58.696872+0000","last_became_active":"2023-10-26T05:56:58.696463+0000","last_became_peered":"2023-10-26T05:56:58.696463+0000","last_unstale":"2023-10-26T05:56:58.696872+0000","last_undegraded":"2023-10-26T05:56:58.696872+0000","last_fullsized":"2023-10-26T05:56:58.696872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:03:26.952948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685813+0000","last_change":"2023-10-26T05:56:58.685813+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685813+0000","last_undegraded":"2023-10-26T05:56:58.685813+0000","last_fullsized":"2023-10-26T05:56:58.685813+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.694434+0000","last_change":"2023-10-26T05:56:58.694349+0000","last_active":"2023-10-26T05:56:58.694434+0000","last_peered":"2023-10-26T05:56:58.694434+0000","last_clean":"2023-10-26T05:56:58.694434+0000","last_became_active":"2023-10-26T05:56:58.694033+0000","last_became_peered":"2023-10-26T05:56:58.694033+0000","last_unstale":"2023-10-26T05:56:58.694434+0000","last_undegraded":"2023-10-26T05:56:58.694434+0000","last_fullsized":"2023-10-26T05:56:58.694434+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:30:52.321543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684950+0000","last_change":"2023-10-26T05:56:58.684950+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684950+0000","last_undegraded":"2023-10-26T05:56:58.684950+0000","last_fullsized":"2023-10-26T05:56:58.684950+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703746+0000","last_change":"2023-10-26T05:56:58.703649+0000","last_active":"2023-10-26T05:56:58.703746+0000","last_peered":"2023-10-26T05:56:58.703746+0000","last_clean":"2023-10-26T05:56:58.703746+0000","last_became_active":"2023-10-26T05:56:58.703345+0000","last_became_peered":"2023-10-26T05:56:58.703345+0000","last_unstale":"2023-10-26T05:56:58.703746+0000","last_undegraded":"2023-10-26T05:56:58.703746+0000","last_fullsized":"2023-10-26T05:56:58.703746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:04.454516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685015+0000","last_change":"2023-10-26T05:56:58.685015+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685015+0000","last_undegraded":"2023-10-26T05:56:58.685015+0000","last_fullsized":"2023-10-26T05:56:58.685015+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713126+0000","last_change":"2023-10-26T05:56:58.712925+0000","last_active":"2023-10-26T05:56:58.713126+0000","last_peered":"2023-10-26T05:56:58.713126+0000","last_clean":"2023-10-26T05:56:58.713126+0000","last_became_active":"2023-10-26T05:56:58.712384+0000","last_became_peered":"2023-10-26T05:56:58.712384+0000","last_unstale":"2023-10-26T05:56:58.713126+0000","last_undegraded":"2023-10-26T05:56:58.713126+0000","last_fullsized":"2023-10-26T05:56:58.713126+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:18.155446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685000+0000","last_change":"2023-10-26T05:56:58.685000+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685000+0000","last_undegraded":"2023-10-26T05:56:58.685000+0000","last_fullsized":"2023-10-26T05:56:58.685000+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713004+0000","last_change":"2023-10-26T05:56:58.712807+0000","last_active":"2023-10-26T05:56:58.713004+0000","last_peered":"2023-10-26T05:56:58.713004+0000","last_clean":"2023-10-26T05:56:58.713004+0000","last_became_active":"2023-10-26T05:56:58.707471+0000","last_became_peered":"2023-10-26T05:56:58.707471+0000","last_unstale":"2023-10-26T05:56:58.713004+0000","last_undegraded":"2023-10-26T05:56:58.713004+0000","last_fullsized":"2023-10-26T05:56:58.713004+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:13:10.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684550+0000","last_change":"2023-10-26T05:56:58.684550+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684550+0000","last_undegraded":"2023-10-26T05:56:58.684550+0000","last_fullsized":"2023-10-26T05:56:58.684550+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708819+0000","last_change":"2023-10-26T05:56:58.707193+0000","last_active":"2023-10-26T05:56:58.708819+0000","last_peered":"2023-10-26T05:56:58.708819+0000","last_clean":"2023-10-26T05:56:58.708819+0000","last_became_active":"2023-10-26T05:56:58.705895+0000","last_became_peered":"2023-10-26T05:56:58.705895+0000","last_unstale":"2023-10-26T05:56:58.708819+0000","last_undegraded":"2023-10-26T05:56:58.708819+0000","last_fullsized":"2023-10-26T05:56:58.708819+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:33:43.161653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694164+0000","last_change":"2023-10-26T05:56:58.694164+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694164+0000","last_undegraded":"2023-10-26T05:56:58.694164+0000","last_fullsized":"2023-10-26T05:56:58.694164+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.717688+0000","last_change":"2023-10-26T05:56:58.717538+0000","last_active":"2023-10-26T05:56:58.717688+0000","last_peered":"2023-10-26T05:56:58.717688+0000","last_clean":"2023-10-26T05:56:58.717688+0000","last_became_active":"2023-10-26T05:56:58.716243+0000","last_became_peered":"2023-10-26T05:56:58.716243+0000","last_unstale":"2023-10-26T05:56:58.717688+0000","last_undegraded":"2023-10-26T05:56:58.717688+0000","last_fullsized":"2023-10-26T05:56:58.717688+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:55:25.695338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689729+0000","last_change":"2023-10-26T05:56:58.689729+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689729+0000","last_undegraded":"2023-10-26T05:56:58.689729+0000","last_fullsized":"2023-10-26T05:56:58.689729+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.697371+0000","last_change":"2023-10-26T05:56:58.697256+0000","last_active":"2023-10-26T05:56:58.697371+0000","last_peered":"2023-10-26T05:56:58.697371+0000","last_clean":"2023-10-26T05:56:58.697371+0000","last_became_active":"2023-10-26T05:56:58.696984+0000","last_became_peered":"2023-10-26T05:56:58.696984+0000","last_unstale":"2023-10-26T05:56:58.697371+0000","last_undegraded":"2023-10-26T05:56:58.697371+0000","last_fullsized":"2023-10-26T05:56:58.697371+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:47.521770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687331+0000","last_change":"2023-10-26T05:56:58.687331+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687331+0000","last_undegraded":"2023-10-26T05:56:58.687331+0000","last_fullsized":"2023-10-26T05:56:58.687331+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687335+0000","last_change":"2023-10-26T05:56:58.687247+0000","last_active":"2023-10-26T05:56:58.687335+0000","last_peered":"2023-10-26T05:56:58.687335+0000","last_clean":"2023-10-26T05:56:58.687335+0000","last_became_active":"2023-10-26T05:56:58.686900+0000","last_became_peered":"2023-10-26T05:56:58.686900+0000","last_unstale":"2023-10-26T05:56:58.687335+0000","last_undegraded":"2023-10-26T05:56:58.687335+0000","last_fullsized":"2023-10-26T05:56:58.687335+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:57:16.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685469+0000","last_change":"2023-10-26T05:56:58.685469+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685469+0000","last_undegraded":"2023-10-26T05:56:58.685469+0000","last_fullsized":"2023-10-26T05:56:58.685469+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.688086+0000","last_change":"2023-10-26T05:56:58.687941+0000","last_active":"2023-10-26T05:56:58.688086+0000","last_peered":"2023-10-26T05:56:58.688086+0000","last_clean":"2023-10-26T05:56:58.688086+0000","last_became_active":"2023-10-26T05:56:58.687118+0000","last_became_peered":"2023-10-26T05:56:58.687118+0000","last_unstale":"2023-10-26T05:56:58.688086+0000","last_undegraded":"2023-10-26T05:56:58.688086+0000","last_fullsized":"2023-10-26T05:56:58.688086+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:10.994025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681960+0000","last_change":"2023-10-26T05:56:58.681960+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681960+0000","last_undegraded":"2023-10-26T05:56:58.681960+0000","last_fullsized":"2023-10-26T05:56:58.681960+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692825+0000","last_change":"2023-10-26T05:56:58.692632+0000","last_active":"2023-10-26T05:56:58.692825+0000","last_peered":"2023-10-26T05:56:58.692825+0000","last_clean":"2023-10-26T05:56:58.692825+0000","last_became_active":"2023-10-26T05:56:58.690733+0000","last_became_peered":"2023-10-26T05:56:58.690733+0000","last_unstale":"2023-10-26T05:56:58.692825+0000","last_undegraded":"2023-10-26T05:56:58.692825+0000","last_fullsized":"2023-10-26T05:56:58.692825+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:59.579324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691809+0000","last_change":"2023-10-26T05:56:58.691809+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691809+0000","last_undegraded":"2023-10-26T05:56:58.691809+0000","last_fullsized":"2023-10-26T05:56:58.691809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.718038+0000","last_change":"2023-10-26T05:56:58.717853+0000","last_active":"2023-10-26T05:56:58.718038+0000","last_peered":"2023-10-26T05:56:58.718038+0000","last_clean":"2023-10-26T05:56:58.718038+0000","last_became_active":"2023-10-26T05:56:58.716922+0000","last_became_peered":"2023-10-26T05:56:58.716922+0000","last_unstale":"2023-10-26T05:56:58.718038+0000","last_undegraded":"2023-10-26T05:56:58.718038+0000","last_fullsized":"2023-10-26T05:56:58.718038+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:59:48.203254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692948+0000","last_change":"2023-10-26T05:56:58.692948+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692948+0000","last_undegraded":"2023-10-26T05:56:58.692948+0000","last_fullsized":"2023-10-26T05:56:58.692948+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711684+0000","last_change":"2023-10-26T05:56:58.709784+0000","last_active":"2023-10-26T05:56:58.711684+0000","last_peered":"2023-10-26T05:56:58.711684+0000","last_clean":"2023-10-26T05:56:58.711684+0000","last_became_active":"2023-10-26T05:56:58.707275+0000","last_became_peered":"2023-10-26T05:56:58.707275+0000","last_unstale":"2023-10-26T05:56:58.711684+0000","last_undegraded":"2023-10-26T05:56:58.711684+0000","last_fullsized":"2023-10-26T05:56:58.711684+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:25:35.872395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693336+0000","last_change":"2023-10-26T05:56:58.693336+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693336+0000","last_undegraded":"2023-10-26T05:56:58.693336+0000","last_fullsized":"2023-10-26T05:56:58.693336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699845+0000","last_change":"2023-10-26T05:56:58.699782+0000","last_active":"2023-10-26T05:56:58.699845+0000","last_peered":"2023-10-26T05:56:58.699845+0000","last_clean":"2023-10-26T05:56:58.699845+0000","last_became_active":"2023-10-26T05:56:58.699054+0000","last_became_peered":"2023-10-26T05:56:58.699054+0000","last_unstale":"2023-10-26T05:56:58.699845+0000","last_undegraded":"2023-10-26T05:56:58.699845+0000","last_fullsized":"2023-10-26T05:56:58.699845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:55.048724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685430+0000","last_change":"2023-10-26T05:56:58.685430+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685430+0000","last_undegraded":"2023-10-26T05:56:58.685430+0000","last_fullsized":"2023-10-26T05:56:58.685430+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711741+0000","last_change":"2023-10-26T05:56:58.710168+0000","last_active":"2023-10-26T05:56:58.711741+0000","last_peered":"2023-10-26T05:56:58.711741+0000","last_clean":"2023-10-26T05:56:58.711741+0000","last_became_active":"2023-10-26T05:56:58.708495+0000","last_became_peered":"2023-10-26T05:56:58.708495+0000","last_unstale":"2023-10-26T05:56:58.711741+0000","last_undegraded":"2023-10-26T05:56:58.711741+0000","last_fullsized":"2023-10-26T05:56:58.711741+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:24.511118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-26T05:56:57.680722+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:56:57.680722+0000","last_peered":"2023-10-26T05:56:57.680722+0000","last_clean":"2023-10-26T05:56:57.680722+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:56:57.680722+0000","last_undegraded":"2023-10-26T05:56:57.680722+0000","last_fullsized":"2023-10-26T05:56:57.680722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.59099999999999997,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3100000000000001},"last":1.1899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":1.0029999999999999},"min":{"1min":0.64700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.232,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.032,"15min":0.98799999999999999},"min":{"1min":0.97999999999999998,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.139}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.68000000000000005,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1899999999999999,"5min":1.448,"15min":1.448},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93799999999999994,"15min":0.93400000000000005},"min":{"1min":0.63900000000000001,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1990000000000001,"5min":1.3700000000000001,"15min":1.4590000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93200000000000005,"15min":0.89800000000000002},"min":{"1min":0.76000000000000001,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":1.004},{"interface":"front","average":{"1min":1.002,"5min":0.96699999999999997,"15min":0.97199999999999998},"min":{"1min":0.77300000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3089999999999999,"5min":1.3129999999999999,"15min":1.375},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.133,"5min":1.639,"15min":1.639},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.93300000000000005,"15min":0.93999999999999995},"min":{"1min":0.65700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.24,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.337,"5min":1.482,"15min":1.6679999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0369999999999999,"15min":1.0640000000000001},"min":{"1min":0.72299999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.702,"15min":1.702},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.115,"15min":1.1060000000000001},"min":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.3420000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.5,"15min":1.6180000000000001},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.234,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.087},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.69999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.86599999999999999},"min":{"1min":0.64500000000000002,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.385,"5min":1.3899999999999999,"15min":1.405},"last":0.95499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.91000000000000003,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2450000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.2809999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97999999999999998,"15min":0.92500000000000004},"min":{"1min":0.76900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.3080000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0620000000000001,"15min":1.0009999999999999},"min":{"1min":0.69099999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.337,"5min":1.3759999999999999,"15min":1.399},"last":1.212}]},{"osd":4,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.503,"15min":0.502},"max":{"1min":1.2569999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.99299999999999999,"15min":0.98099999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.111}]},{"osd":5,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.05},"min":{"1min":0.89200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.236,"5min":1.5509999999999999,"15min":1.554},"last":1.028},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97199999999999998,"15min":0.98299999999999998},"min":{"1min":0.67200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.452,"5min":1.512,"15min":1.512},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1220000000000001,"15min":1.1120000000000001},"min":{"1min":0.78400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.321,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94499999999999995,"15min":0.95599999999999996},"min":{"1min":0.63400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.579},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.83599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":1.591,"15min":1.591},"last":1.3400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84699999999999998,"15min":0.878},"min":{"1min":0.625,"5min":0.53300000000000003,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.206,"15min":2.1150000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86799999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2430000000000001,"5min":1.272,"15min":1.464},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.70599999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.2170000000000001,"5min":1.569,"15min":1.569},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0640000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90900000000000003,"15min":0.91800000000000004},"min":{"1min":0.69799999999999995,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.363,"5min":1.448,"15min":1.548},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.78200000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.276,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.97599999999999998},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.286,"15min":2.1909999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0760000000000001,"15min":1.091},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.536,"15min":1.536},"last":1.194}]},{"osd":6,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.68000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.268,"5min":1.744,"15min":1.744},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.04,"15min":1.032},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":1.4279999999999999,"15min":1.4950000000000001},"last":1.161}]},{"osd":7,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.071},"min":{"1min":0.64100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.3109999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3360000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.92000000000000004,"15min":0.871},"min":{"1min":0.66200000000000003,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1919999999999999,"5min":1.2569999999999999,"15min":1.347},"last":1.075},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.70799999999999996,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.159,"5min":1.502,"15min":1.502},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.90400000000000003,"15min":0.84499999999999997},"min":{"1min":0.749,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.21,"5min":1.25,"15min":1.375},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.96099999999999997,"15min":0.91700000000000004},"min":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":1.198}]},{"osd":2,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.143},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90700000000000003,"15min":0.879},"min":{"1min":0.72899999999999998,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.143,"5min":1.2170000000000001,"15min":1.373},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":0.95199999999999996},"min":{"1min":0.79700000000000004,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.177,"5min":1.397,"15min":2.0190000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.091,"5min":0.995,"15min":0.96199999999999997},"min":{"1min":0.79100000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.28,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.2430000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0009999999999999,"15min":1},"min":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.2929999999999999,"5min":1.4179999999999999,"15min":1.673},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0429999999999999,"15min":1.014},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.42099999999999999},"max":{"1min":1.337,"5min":1.762,"15min":1.7969999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.054,"15min":1.042},"min":{"1min":0.76500000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.4970000000000001},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.103,"15min":1.0589999999999999},"min":{"1min":0.77300000000000002,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.3280000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.879,"5min":0.69599999999999995,"15min":0.53700000000000003},"max":{"1min":1.341,"5min":1.484,"15min":1.728},"last":1.2709999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.90100000000000002,"5min":1.385,"15min":1.385},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.52500000000000002,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.2689999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":1.091,"5min":1.2,"15min":1.349},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.38,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.02,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.018}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.752,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1479999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81999999999999995,"15min":0.78700000000000003},"min":{"1min":0.623,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.1850000000000001,"15min":1.4199999999999999},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.84399999999999997},"min":{"1min":0.70799999999999996,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.1120000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90200000000000002,"15min":0.91900000000000004},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.2789999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.72399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3,"5min":1.413,"15min":1.478},"last":1.454},{"interface":"front","average":{"1min":1.014,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.54,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.64200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.349,"15min":1.6060000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.091,"5min":0.96699999999999997,"15min":1.014},"min":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":1.359,"15min":5.6859999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.98599999999999999,"15min":0.94299999999999995},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.53400000000000003},"max":{"1min":1.472,"5min":2.016,"15min":2.016},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.008,"15min":1.032},"min":{"1min":0.68799999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.327,"5min":2.004,"15min":2.004},"last":1.2370000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71999999999999997,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.379,"15min":1.5249999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.69599999999999995},"min":{"1min":0.61699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.091,"5min":1.226,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.621},"min":{"1min":0.57799999999999996,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.90700000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.82199999999999995,"15min":0.76500000000000001},"min":{"1min":0.71599999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.24,"15min":2.8809999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94899999999999995,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93000000000000005,"15min":0.85999999999999999},"min":{"1min":0.69099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.151,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88500000000000001,"15min":0.85899999999999999},"min":{"1min":0.78200000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.997,"15min":0.93899999999999995},"min":{"1min":0.84999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.999,"15min":0.93100000000000005},"min":{"1min":0.80000000000000004,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.329,"5min":1.4790000000000001,"15min":1.653},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.008,"15min":0.94699999999999995},"min":{"1min":0.73199999999999998,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.3320000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.089,"15min":1.016},"min":{"1min":0.76500000000000001,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.3300000000000001,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.177}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":0.91900000000000004},"min":{"1min":0.84299999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.073,"5min":1.581,"15min":1.581},"last":1.119},{"interface":"front","average":{"1min":1.133,"5min":1.109,"15min":1.069},"min":{"1min":0.91200000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.462,"5min":1.823,"15min":15.352},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.157},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55600000000000005,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78300000000000003,"15min":0.73999999999999999},"min":{"1min":0.63700000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.073,"5min":1.6080000000000001,"15min":1.8169999999999999},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.754,"15min":0.76100000000000001},"min":{"1min":0.65600000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.147,"15min":1.7949999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.0760000000000001,"5min":1.298,"15min":1.298},"last":0.997}]},{"osd":4,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.874},"min":{"1min":0.73899999999999999,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1020000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.74099999999999999,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.345,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":1.2}]},{"osd":5,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.96299999999999997,"15min":0.96399999999999997},"min":{"1min":0.749,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.319,"5min":1.381,"15min":1.492},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.84899999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.2509999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0249999999999999},"min":{"1min":0.89200000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89700000000000002,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.2669999999999999,"5min":1.468,"15min":1.837},"last":1.155}]},{"osd":7,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.046,"15min":1.0309999999999999},"min":{"1min":0.82499999999999996,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.397,"5min":1.4179999999999999,"15min":1.6339999999999999},"last":1.294},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0640000000000001,"15min":1.0840000000000001},"min":{"1min":0.93100000000000005,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4890000000000001,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":1.0600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95599999999999996,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.39100000000000001,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.1020000000000001,"15min":1.177},"last":0.502}]},{"osd":3,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0249999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.72899999999999998},"min":{"1min":0.58999999999999997,"5min":0.32100000000000001,"15min":0.247},"max":{"1min":1.026,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91400000000000003,"15min":0.873},"min":{"1min":0.64200000000000002,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.319,"5min":1.395,"15min":1.395},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":0.93500000000000005},"min":{"1min":0.73699999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":1.1559999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.92200000000000004},"min":{"1min":0.61299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":2.7480000000000002},"last":1.073},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96299999999999997,"15min":0.92100000000000004},"min":{"1min":0.87,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.91900000000000004},"min":{"1min":0.77500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1639999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.131,"15min":1.0169999999999999},"min":{"1min":0.871,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.2829999999999999,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0109999999999999,"15min":0.998},"min":{"1min":0.74199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1539999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0329999999999999,"15min":0.999},"min":{"1min":0.93500000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.417,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":1.141}]}]}],"pool_statfs":[]}} 2023-10-26T05:57:03.054 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:57:03.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 -- 172.21.15.71:0/344508098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d4004d10 msgr2=0x7f47d4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 --2- 172.21.15.71:0/344508098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d4004d10 0x7f47d4005190 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f47e400c5c0 tx=0x7f47e401d340 comp rx=0 tx=0).stop 2023-10-26T05:57:03.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47ead9d640 1 -- 172.21.15.71:0/344508098 reap_dead start 2023-10-26T05:57:03.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 -- 172.21.15.71:0/344508098 shutdown_connections 2023-10-26T05:57:03.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 -- 172.21.15.71:0/344508098 >> 172.21.15.71:0/344508098 conn(0x7f47ec069770 msgr2=0x7f47ec069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:03.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 -- 172.21.15.71:0/344508098 shutdown_connections 2023-10-26T05:57:03.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 -- 172.21.15.71:0/344508098 wait complete. 2023-10-26T05:57:03.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 Processor -- start 2023-10-26T05:57:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.203+0000 7f47f0f87640 1 -- start start 2023-10-26T05:57:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec1376e0 0x7f47ec06a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 0x7f47ec13a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec13fa70 0x7f47ec13aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec13ffa0 con 0x7f47ec137ac0 2023-10-26T05:57:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec140280 con 0x7f47ec13fa70 2023-10-26T05:57:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ec140560 con 0x7f47ec1376e0 2023-10-26T05:57:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 0x7f47ec13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 0x7f47ec13a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35800/0 (socket says 172.21.15.71:35800) 2023-10-26T05:57:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 -- 172.21.15.71:0/1261947693 learned_addr learned my addr 172.21.15.71:0/1261947693 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:03.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47ead9d640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec13fa70 0x7f47ec13aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47ea59c640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec1376e0 0x7f47ec06a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 -- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec1376e0 msgr2=0x7f47ec06a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec1376e0 0x7f47ec06a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 -- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec13fa70 msgr2=0x7f47ec13aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec13fa70 0x7f47ec13aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 -- 172.21.15.71:0/1261947693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec140840 con 0x7f47ec137ac0 2023-10-26T05:57:03.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47ead9d640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec13fa70 0x7f47ec13aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:57:03.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47e9d9b640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 0x7f47ec13a990 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f47e0011a50 tx=0x7f47e0011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:03.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e000fd90 con 0x7f47ec137ac0 2023-10-26T05:57:03.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f47e0012450 con 0x7f47ec137ac0 2023-10-26T05:57:03.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ec140b20 con 0x7f47ec137ac0 2023-10-26T05:57:03.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0012800 con 0x7f47ec137ac0 2023-10-26T05:57:03.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ec13e680 con 0x7f47ec137ac0 2023-10-26T05:57:03.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ec066b10 con 0x7f47ec137ac0 2023-10-26T05:57:03.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f47e0012960 con 0x7f47ec137ac0 2023-10-26T05:57:03.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47daffd640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47cc0664a0 0x7f47cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47e0084d00 con 0x7f47ec137ac0 2023-10-26T05:57:03.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.207+0000 7f47ea59c640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47cc0664a0 0x7f47cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.211+0000 7f47ea59c640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47cc0664a0 0x7f47cc068960 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f47dc001690 tx=0x7f47dc0069b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:03.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.211+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f47e00522c0 con 0x7f47ec137ac0 2023-10-26T05:57:03.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.351+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47ec1403f0 con 0x7f47cc0664a0 2023-10-26T05:57:03.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.375+0000 7f47daffd640 1 -- 172.21.15.71:0/1261947693 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278382 (secure 0 0 0) 0x7f47ec1403f0 con 0x7f47cc0664a0 2023-10-26T05:57:03.385 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:03.387 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:57:03.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47cc0664a0 msgr2=0x7f47cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47cc0664a0 0x7f47cc068960 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f47dc001690 tx=0x7f47dc0069b0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 msgr2=0x7f47ec13a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 0x7f47ec13a990 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f47e0011a50 tx=0x7f47e0011f10 comp rx=0 tx=0).stop 2023-10-26T05:57:03.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 shutdown_connections 2023-10-26T05:57:03.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec13fa70 0x7f47ec13aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f47cc0664a0 0x7f47cc068960 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47ec137ac0 0x7f47ec13a990 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 --2- 172.21.15.71:0/1261947693 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f47ec1376e0 0x7f47ec06a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 >> 172.21.15.71:0/1261947693 conn(0x7f47ec069770 msgr2=0x7f47ec158920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 shutdown_connections 2023-10-26T05:57:03.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.379+0000 7f47f0f87640 1 -- 172.21.15.71:0/1261947693 wait complete. 2023-10-26T05:57:03.401 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2023-10-26T05:57:01.607623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553911,"statfs":{"total":1600000000,"available":1591205414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001501"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687936+0000","last_change":"2023-10-26T05:56:58.687936+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687936+0000","last_undegraded":"2023-10-26T05:56:58.687936+0000","last_fullsized":"2023-10-26T05:56:58.687936+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689543+0000","last_change":"2023-10-26T05:56:58.689543+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689543+0000","last_undegraded":"2023-10-26T05:56:58.689543+0000","last_fullsized":"2023-10-26T05:56:58.689543+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.701504+0000","last_change":"2023-10-26T05:56:58.701504+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.701504+0000","last_undegraded":"2023-10-26T05:56:58.701504+0000","last_fullsized":"2023-10-26T05:56:58.701504+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687487+0000","last_change":"2023-10-26T05:56:58.687487+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687487+0000","last_undegraded":"2023-10-26T05:56:58.687487+0000","last_fullsized":"2023-10-26T05:56:58.687487+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699011+0000","last_change":"2023-10-26T05:56:58.699011+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699011+0000","last_undegraded":"2023-10-26T05:56:58.699011+0000","last_fullsized":"2023-10-26T05:56:58.699011+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687527+0000","last_change":"2023-10-26T05:56:58.687527+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687527+0000","last_undegraded":"2023-10-26T05:56:58.687527+0000","last_fullsized":"2023-10-26T05:56:58.687527+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684911+0000","last_change":"2023-10-26T05:56:58.684911+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684911+0000","last_undegraded":"2023-10-26T05:56:58.684911+0000","last_fullsized":"2023-10-26T05:56:58.684911+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686493+0000","last_change":"2023-10-26T05:56:58.686493+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686493+0000","last_undegraded":"2023-10-26T05:56:58.686493+0000","last_fullsized":"2023-10-26T05:56:58.686493+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686070+0000","last_change":"2023-10-26T05:56:58.686070+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686070+0000","last_undegraded":"2023-10-26T05:56:58.686070+0000","last_fullsized":"2023-10-26T05:56:58.686070+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697900+0000","last_change":"2023-10-26T05:56:58.697900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697900+0000","last_undegraded":"2023-10-26T05:56:58.697900+0000","last_fullsized":"2023-10-26T05:56:58.697900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687124+0000","last_change":"2023-10-26T05:56:58.687124+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687124+0000","last_undegraded":"2023-10-26T05:56:58.687124+0000","last_fullsized":"2023-10-26T05:56:58.687124+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.700372+0000","last_change":"2023-10-26T05:56:58.700372+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.700372+0000","last_undegraded":"2023-10-26T05:56:58.700372+0000","last_fullsized":"2023-10-26T05:56:58.700372+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695733+0000","last_change":"2023-10-26T05:56:58.695733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695733+0000","last_undegraded":"2023-10-26T05:56:58.695733+0000","last_fullsized":"2023-10-26T05:56:58.695733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685971+0000","last_change":"2023-10-26T05:56:58.685971+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685971+0000","last_undegraded":"2023-10-26T05:56:58.685971+0000","last_fullsized":"2023-10-26T05:56:58.685971+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697158+0000","last_change":"2023-10-26T05:56:58.697158+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697158+0000","last_undegraded":"2023-10-26T05:56:58.697158+0000","last_fullsized":"2023-10-26T05:56:58.697158+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682093+0000","last_change":"2023-10-26T05:56:58.682093+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682093+0000","last_undegraded":"2023-10-26T05:56:58.682093+0000","last_fullsized":"2023-10-26T05:56:58.682093+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693783+0000","last_change":"2023-10-26T05:56:58.693783+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693783+0000","last_undegraded":"2023-10-26T05:56:58.693783+0000","last_fullsized":"2023-10-26T05:56:58.693783+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.702305+0000","last_change":"2023-10-26T05:56:58.702305+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.702305+0000","last_undegraded":"2023-10-26T05:56:58.702305+0000","last_fullsized":"2023-10-26T05:56:58.702305+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686171+0000","last_change":"2023-10-26T05:56:58.686171+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686171+0000","last_undegraded":"2023-10-26T05:56:58.686171+0000","last_fullsized":"2023-10-26T05:56:58.686171+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684098+0000","last_change":"2023-10-26T05:56:58.684098+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684098+0000","last_undegraded":"2023-10-26T05:56:58.684098+0000","last_fullsized":"2023-10-26T05:56:58.684098+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693775+0000","last_change":"2023-10-26T05:56:58.693775+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693775+0000","last_undegraded":"2023-10-26T05:56:58.693775+0000","last_fullsized":"2023-10-26T05:56:58.693775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.688204+0000","last_change":"2023-10-26T05:56:58.688204+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.688204+0000","last_undegraded":"2023-10-26T05:56:58.688204+0000","last_fullsized":"2023-10-26T05:56:58.688204+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684900+0000","last_change":"2023-10-26T05:56:58.684900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684900+0000","last_undegraded":"2023-10-26T05:56:58.684900+0000","last_fullsized":"2023-10-26T05:56:58.684900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694702+0000","last_change":"2023-10-26T05:56:58.694702+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694702+0000","last_undegraded":"2023-10-26T05:56:58.694702+0000","last_fullsized":"2023-10-26T05:56:58.694702+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683660+0000","last_change":"2023-10-26T05:56:58.683660+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683660+0000","last_undegraded":"2023-10-26T05:56:58.683660+0000","last_fullsized":"2023-10-26T05:56:58.683660+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686703+0000","last_change":"2023-10-26T05:56:58.686703+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686703+0000","last_undegraded":"2023-10-26T05:56:58.686703+0000","last_fullsized":"2023-10-26T05:56:58.686703+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682875+0000","last_change":"2023-10-26T05:56:58.682875+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682875+0000","last_undegraded":"2023-10-26T05:56:58.682875+0000","last_fullsized":"2023-10-26T05:56:58.682875+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685434+0000","last_change":"2023-10-26T05:56:58.685434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685434+0000","last_undegraded":"2023-10-26T05:56:58.685434+0000","last_fullsized":"2023-10-26T05:56:58.685434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685841+0000","last_change":"2023-10-26T05:56:58.685841+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685841+0000","last_undegraded":"2023-10-26T05:56:58.685841+0000","last_fullsized":"2023-10-26T05:56:58.685841+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685847+0000","last_change":"2023-10-26T05:56:58.685847+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685847+0000","last_undegraded":"2023-10-26T05:56:58.685847+0000","last_fullsized":"2023-10-26T05:56:58.685847+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684393+0000","last_change":"2023-10-26T05:56:58.684393+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684393+0000","last_undegraded":"2023-10-26T05:56:58.684393+0000","last_fullsized":"2023-10-26T05:56:58.684393+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682434+0000","last_change":"2023-10-26T05:56:58.682434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682434+0000","last_undegraded":"2023-10-26T05:56:58.682434+0000","last_fullsized":"2023-10-26T05:56:58.682434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693816+0000","last_change":"2023-10-26T05:56:58.693816+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693816+0000","last_undegraded":"2023-10-26T05:56:58.693816+0000","last_fullsized":"2023-10-26T05:56:58.693816+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711649+0000","last_change":"2023-10-26T05:56:58.711496+0000","last_active":"2023-10-26T05:56:58.711649+0000","last_peered":"2023-10-26T05:56:58.711649+0000","last_clean":"2023-10-26T05:56:58.711649+0000","last_became_active":"2023-10-26T05:56:58.710397+0000","last_became_peered":"2023-10-26T05:56:58.710397+0000","last_unstale":"2023-10-26T05:56:58.711649+0000","last_undegraded":"2023-10-26T05:56:58.711649+0000","last_fullsized":"2023-10-26T05:56:58.711649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:45:28.308338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683676+0000","last_change":"2023-10-26T05:56:58.683676+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683676+0000","last_undegraded":"2023-10-26T05:56:58.683676+0000","last_fullsized":"2023-10-26T05:56:58.683676+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699941+0000","last_change":"2023-10-26T05:56:58.699902+0000","last_active":"2023-10-26T05:56:58.699941+0000","last_peered":"2023-10-26T05:56:58.699941+0000","last_clean":"2023-10-26T05:56:58.699941+0000","last_became_active":"2023-10-26T05:56:58.699481+0000","last_became_peered":"2023-10-26T05:56:58.699481+0000","last_unstale":"2023-10-26T05:56:58.699941+0000","last_undegraded":"2023-10-26T05:56:58.699941+0000","last_fullsized":"2023-10-26T05:56:58.699941+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:15.048906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686261+0000","last_change":"2023-10-26T05:56:58.686261+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686261+0000","last_undegraded":"2023-10-26T05:56:58.686261+0000","last_fullsized":"2023-10-26T05:56:58.686261+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692322+0000","last_change":"2023-10-26T05:56:58.691994+0000","last_active":"2023-10-26T05:56:58.692322+0000","last_peered":"2023-10-26T05:56:58.692322+0000","last_clean":"2023-10-26T05:56:58.692322+0000","last_became_active":"2023-10-26T05:56:58.691381+0000","last_became_peered":"2023-10-26T05:56:58.691381+0000","last_unstale":"2023-10-26T05:56:58.692322+0000","last_undegraded":"2023-10-26T05:56:58.692322+0000","last_fullsized":"2023-10-26T05:56:58.692322+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:13.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.696248+0000","last_change":"2023-10-26T05:56:58.696248+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.696248+0000","last_undegraded":"2023-10-26T05:56:58.696248+0000","last_fullsized":"2023-10-26T05:56:58.696248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687988+0000","last_change":"2023-10-26T05:56:58.687804+0000","last_active":"2023-10-26T05:56:58.687988+0000","last_peered":"2023-10-26T05:56:58.687988+0000","last_clean":"2023-10-26T05:56:58.687988+0000","last_became_active":"2023-10-26T05:56:58.687167+0000","last_became_peered":"2023-10-26T05:56:58.687167+0000","last_unstale":"2023-10-26T05:56:58.687988+0000","last_undegraded":"2023-10-26T05:56:58.687988+0000","last_fullsized":"2023-10-26T05:56:58.687988+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:47.280630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683215+0000","last_change":"2023-10-26T05:56:58.683215+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683215+0000","last_undegraded":"2023-10-26T05:56:58.683215+0000","last_fullsized":"2023-10-26T05:56:58.683215+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.709083+0000","last_change":"2023-10-26T05:56:58.708982+0000","last_active":"2023-10-26T05:56:58.709083+0000","last_peered":"2023-10-26T05:56:58.709083+0000","last_clean":"2023-10-26T05:56:58.709083+0000","last_became_active":"2023-10-26T05:56:58.708308+0000","last_became_peered":"2023-10-26T05:56:58.708308+0000","last_unstale":"2023-10-26T05:56:58.709083+0000","last_undegraded":"2023-10-26T05:56:58.709083+0000","last_fullsized":"2023-10-26T05:56:58.709083+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:39.204476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685163+0000","last_change":"2023-10-26T05:56:58.685163+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685163+0000","last_undegraded":"2023-10-26T05:56:58.685163+0000","last_fullsized":"2023-10-26T05:56:58.685163+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.707771+0000","last_change":"2023-10-26T05:56:58.707555+0000","last_active":"2023-10-26T05:56:58.707771+0000","last_peered":"2023-10-26T05:56:58.707771+0000","last_clean":"2023-10-26T05:56:58.707771+0000","last_became_active":"2023-10-26T05:56:58.706918+0000","last_became_peered":"2023-10-26T05:56:58.706918+0000","last_unstale":"2023-10-26T05:56:58.707771+0000","last_undegraded":"2023-10-26T05:56:58.707771+0000","last_fullsized":"2023-10-26T05:56:58.707771+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:12:04.008423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686926+0000","last_change":"2023-10-26T05:56:58.686926+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686926+0000","last_undegraded":"2023-10-26T05:56:58.686926+0000","last_fullsized":"2023-10-26T05:56:58.686926+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.702597+0000","last_change":"2023-10-26T05:56:58.702478+0000","last_active":"2023-10-26T05:56:58.702597+0000","last_peered":"2023-10-26T05:56:58.702597+0000","last_clean":"2023-10-26T05:56:58.702597+0000","last_became_active":"2023-10-26T05:56:58.702066+0000","last_became_peered":"2023-10-26T05:56:58.702066+0000","last_unstale":"2023-10-26T05:56:58.702597+0000","last_undegraded":"2023-10-26T05:56:58.702597+0000","last_fullsized":"2023-10-26T05:56:58.702597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:10.521714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.698412+0000","last_change":"2023-10-26T05:56:58.698412+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.698412+0000","last_undegraded":"2023-10-26T05:56:58.698412+0000","last_fullsized":"2023-10-26T05:56:58.698412+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.705203+0000","last_change":"2023-10-26T05:56:58.705010+0000","last_active":"2023-10-26T05:56:58.705203+0000","last_peered":"2023-10-26T05:56:58.705203+0000","last_clean":"2023-10-26T05:56:58.705203+0000","last_became_active":"2023-10-26T05:56:58.704379+0000","last_became_peered":"2023-10-26T05:56:58.704379+0000","last_unstale":"2023-10-26T05:56:58.705203+0000","last_undegraded":"2023-10-26T05:56:58.705203+0000","last_fullsized":"2023-10-26T05:56:58.705203+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:22.102825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691840+0000","last_change":"2023-10-26T05:56:58.691840+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691840+0000","last_undegraded":"2023-10-26T05:56:58.691840+0000","last_fullsized":"2023-10-26T05:56:58.691840+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692523+0000","last_change":"2023-10-26T05:56:58.692438+0000","last_active":"2023-10-26T05:56:58.692523+0000","last_peered":"2023-10-26T05:56:58.692523+0000","last_clean":"2023-10-26T05:56:58.692523+0000","last_became_active":"2023-10-26T05:56:58.692192+0000","last_became_peered":"2023-10-26T05:56:58.692192+0000","last_unstale":"2023-10-26T05:56:58.692523+0000","last_undegraded":"2023-10-26T05:56:58.692523+0000","last_fullsized":"2023-10-26T05:56:58.692523+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:20:10.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692914+0000","last_change":"2023-10-26T05:56:58.692914+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692914+0000","last_undegraded":"2023-10-26T05:56:58.692914+0000","last_fullsized":"2023-10-26T05:56:58.692914+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.715857+0000","last_change":"2023-10-26T05:56:58.715723+0000","last_active":"2023-10-26T05:56:58.715857+0000","last_peered":"2023-10-26T05:56:58.715857+0000","last_clean":"2023-10-26T05:56:58.715857+0000","last_became_active":"2023-10-26T05:56:58.714758+0000","last_became_peered":"2023-10-26T05:56:58.714758+0000","last_unstale":"2023-10-26T05:56:58.715857+0000","last_undegraded":"2023-10-26T05:56:58.715857+0000","last_fullsized":"2023-10-26T05:56:58.715857+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:50.434850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682474+0000","last_change":"2023-10-26T05:56:58.682474+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682474+0000","last_undegraded":"2023-10-26T05:56:58.682474+0000","last_fullsized":"2023-10-26T05:56:58.682474+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.691946+0000","last_change":"2023-10-26T05:56:58.691788+0000","last_active":"2023-10-26T05:56:58.691946+0000","last_peered":"2023-10-26T05:56:58.691946+0000","last_clean":"2023-10-26T05:56:58.691946+0000","last_became_active":"2023-10-26T05:56:58.691194+0000","last_became_peered":"2023-10-26T05:56:58.691194+0000","last_unstale":"2023-10-26T05:56:58.691946+0000","last_undegraded":"2023-10-26T05:56:58.691946+0000","last_fullsized":"2023-10-26T05:56:58.691946+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:05:11.942603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689507+0000","last_change":"2023-10-26T05:56:58.689507+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689507+0000","last_undegraded":"2023-10-26T05:56:58.689507+0000","last_fullsized":"2023-10-26T05:56:58.689507+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.700529+0000","last_change":"2023-10-26T05:56:58.700296+0000","last_active":"2023-10-26T05:56:58.700529+0000","last_peered":"2023-10-26T05:56:58.700529+0000","last_clean":"2023-10-26T05:56:58.700529+0000","last_became_active":"2023-10-26T05:56:58.699651+0000","last_became_peered":"2023-10-26T05:56:58.699651+0000","last_unstale":"2023-10-26T05:56:58.700529+0000","last_undegraded":"2023-10-26T05:56:58.700529+0000","last_fullsized":"2023-10-26T05:56:58.700529+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:42:59.754770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687731+0000","last_change":"2023-10-26T05:56:58.687731+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687731+0000","last_undegraded":"2023-10-26T05:56:58.687731+0000","last_fullsized":"2023-10-26T05:56:58.687731+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708870+0000","last_change":"2023-10-26T05:56:58.708596+0000","last_active":"2023-10-26T05:56:58.708870+0000","last_peered":"2023-10-26T05:56:58.708870+0000","last_clean":"2023-10-26T05:56:58.708870+0000","last_became_active":"2023-10-26T05:56:58.707912+0000","last_became_peered":"2023-10-26T05:56:58.707912+0000","last_unstale":"2023-10-26T05:56:58.708870+0000","last_undegraded":"2023-10-26T05:56:58.708870+0000","last_fullsized":"2023-10-26T05:56:58.708870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:40:53.740361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694993+0000","last_change":"2023-10-26T05:56:58.694993+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694993+0000","last_undegraded":"2023-10-26T05:56:58.694993+0000","last_fullsized":"2023-10-26T05:56:58.694993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.684540+0000","last_change":"2023-10-26T05:56:58.684439+0000","last_active":"2023-10-26T05:56:58.684540+0000","last_peered":"2023-10-26T05:56:58.684540+0000","last_clean":"2023-10-26T05:56:58.684540+0000","last_became_active":"2023-10-26T05:56:58.684093+0000","last_became_peered":"2023-10-26T05:56:58.684093+0000","last_unstale":"2023-10-26T05:56:58.684540+0000","last_undegraded":"2023-10-26T05:56:58.684540+0000","last_fullsized":"2023-10-26T05:56:58.684540+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:28.905001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699789+0000","last_change":"2023-10-26T05:56:58.699789+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699789+0000","last_undegraded":"2023-10-26T05:56:58.699789+0000","last_fullsized":"2023-10-26T05:56:58.699789+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703649+0000","last_change":"2023-10-26T05:56:58.703574+0000","last_active":"2023-10-26T05:56:58.703649+0000","last_peered":"2023-10-26T05:56:58.703649+0000","last_clean":"2023-10-26T05:56:58.703649+0000","last_became_active":"2023-10-26T05:56:58.703256+0000","last_became_peered":"2023-10-26T05:56:58.703256+0000","last_unstale":"2023-10-26T05:56:58.703649+0000","last_undegraded":"2023-10-26T05:56:58.703649+0000","last_fullsized":"2023-10-26T05:56:58.703649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:26.907325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695608+0000","last_change":"2023-10-26T05:56:58.695608+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695608+0000","last_undegraded":"2023-10-26T05:56:58.695608+0000","last_fullsized":"2023-10-26T05:56:58.695608+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708221+0000","last_change":"2023-10-26T05:56:58.707861+0000","last_active":"2023-10-26T05:56:58.708221+0000","last_peered":"2023-10-26T05:56:58.708221+0000","last_clean":"2023-10-26T05:56:58.708221+0000","last_became_active":"2023-10-26T05:56:58.707069+0000","last_became_peered":"2023-10-26T05:56:58.707069+0000","last_unstale":"2023-10-26T05:56:58.708221+0000","last_undegraded":"2023-10-26T05:56:58.708221+0000","last_fullsized":"2023-10-26T05:56:58.708221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:39.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681733+0000","last_change":"2023-10-26T05:56:58.681733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681733+0000","last_undegraded":"2023-10-26T05:56:58.681733+0000","last_fullsized":"2023-10-26T05:56:58.681733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699326+0000","last_change":"2023-10-26T05:56:58.699246+0000","last_active":"2023-10-26T05:56:58.699326+0000","last_peered":"2023-10-26T05:56:58.699326+0000","last_clean":"2023-10-26T05:56:58.699326+0000","last_became_active":"2023-10-26T05:56:58.698939+0000","last_became_peered":"2023-10-26T05:56:58.698939+0000","last_unstale":"2023-10-26T05:56:58.699326+0000","last_undegraded":"2023-10-26T05:56:58.699326+0000","last_fullsized":"2023-10-26T05:56:58.699326+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:17.788046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.690498+0000","last_change":"2023-10-26T05:56:58.690498+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.690498+0000","last_undegraded":"2023-10-26T05:56:58.690498+0000","last_fullsized":"2023-10-26T05:56:58.690498+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.696872+0000","last_change":"2023-10-26T05:56:58.696783+0000","last_active":"2023-10-26T05:56:58.696872+0000","last_peered":"2023-10-26T05:56:58.696872+0000","last_clean":"2023-10-26T05:56:58.696872+0000","last_became_active":"2023-10-26T05:56:58.696463+0000","last_became_peered":"2023-10-26T05:56:58.696463+0000","last_unstale":"2023-10-26T05:56:58.696872+0000","last_undegraded":"2023-10-26T05:56:58.696872+0000","last_fullsized":"2023-10-26T05:56:58.696872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:03:26.952948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685813+0000","last_change":"2023-10-26T05:56:58.685813+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685813+0000","last_undegraded":"2023-10-26T05:56:58.685813+0000","last_fullsized":"2023-10-26T05:56:58.685813+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.694434+0000","last_change":"2023-10-26T05:56:58.694349+0000","last_active":"2023-10-26T05:56:58.694434+0000","last_peered":"2023-10-26T05:56:58.694434+0000","last_clean":"2023-10-26T05:56:58.694434+0000","last_became_active":"2023-10-26T05:56:58.694033+0000","last_became_peered":"2023-10-26T05:56:58.694033+0000","last_unstale":"2023-10-26T05:56:58.694434+0000","last_undegraded":"2023-10-26T05:56:58.694434+0000","last_fullsized":"2023-10-26T05:56:58.694434+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:30:52.321543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684950+0000","last_change":"2023-10-26T05:56:58.684950+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684950+0000","last_undegraded":"2023-10-26T05:56:58.684950+0000","last_fullsized":"2023-10-26T05:56:58.684950+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703746+0000","last_change":"2023-10-26T05:56:58.703649+0000","last_active":"2023-10-26T05:56:58.703746+0000","last_peered":"2023-10-26T05:56:58.703746+0000","last_clean":"2023-10-26T05:56:58.703746+0000","last_became_active":"2023-10-26T05:56:58.703345+0000","last_became_peered":"2023-10-26T05:56:58.703345+0000","last_unstale":"2023-10-26T05:56:58.703746+0000","last_undegraded":"2023-10-26T05:56:58.703746+0000","last_fullsized":"2023-10-26T05:56:58.703746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:04.454516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685015+0000","last_change":"2023-10-26T05:56:58.685015+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685015+0000","last_undegraded":"2023-10-26T05:56:58.685015+0000","last_fullsized":"2023-10-26T05:56:58.685015+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713126+0000","last_change":"2023-10-26T05:56:58.712925+0000","last_active":"2023-10-26T05:56:58.713126+0000","last_peered":"2023-10-26T05:56:58.713126+0000","last_clean":"2023-10-26T05:56:58.713126+0000","last_became_active":"2023-10-26T05:56:58.712384+0000","last_became_peered":"2023-10-26T05:56:58.712384+0000","last_unstale":"2023-10-26T05:56:58.713126+0000","last_undegraded":"2023-10-26T05:56:58.713126+0000","last_fullsized":"2023-10-26T05:56:58.713126+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:18.155446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685000+0000","last_change":"2023-10-26T05:56:58.685000+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685000+0000","last_undegraded":"2023-10-26T05:56:58.685000+0000","last_fullsized":"2023-10-26T05:56:58.685000+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713004+0000","last_change":"2023-10-26T05:56:58.712807+0000","last_active":"2023-10-26T05:56:58.713004+0000","last_peered":"2023-10-26T05:56:58.713004+0000","last_clean":"2023-10-26T05:56:58.713004+0000","last_became_active":"2023-10-26T05:56:58.707471+0000","last_became_peered":"2023-10-26T05:56:58.707471+0000","last_unstale":"2023-10-26T05:56:58.713004+0000","last_undegraded":"2023-10-26T05:56:58.713004+0000","last_fullsized":"2023-10-26T05:56:58.713004+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:13:10.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684550+0000","last_change":"2023-10-26T05:56:58.684550+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684550+0000","last_undegraded":"2023-10-26T05:56:58.684550+0000","last_fullsized":"2023-10-26T05:56:58.684550+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708819+0000","last_change":"2023-10-26T05:56:58.707193+0000","last_active":"2023-10-26T05:56:58.708819+0000","last_peered":"2023-10-26T05:56:58.708819+0000","last_clean":"2023-10-26T05:56:58.708819+0000","last_became_active":"2023-10-26T05:56:58.705895+0000","last_became_peered":"2023-10-26T05:56:58.705895+0000","last_unstale":"2023-10-26T05:56:58.708819+0000","last_undegraded":"2023-10-26T05:56:58.708819+0000","last_fullsized":"2023-10-26T05:56:58.708819+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:33:43.161653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694164+0000","last_change":"2023-10-26T05:56:58.694164+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694164+0000","last_undegraded":"2023-10-26T05:56:58.694164+0000","last_fullsized":"2023-10-26T05:56:58.694164+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.717688+0000","last_change":"2023-10-26T05:56:58.717538+0000","last_active":"2023-10-26T05:56:58.717688+0000","last_peered":"2023-10-26T05:56:58.717688+0000","last_clean":"2023-10-26T05:56:58.717688+0000","last_became_active":"2023-10-26T05:56:58.716243+0000","last_became_peered":"2023-10-26T05:56:58.716243+0000","last_unstale":"2023-10-26T05:56:58.717688+0000","last_undegraded":"2023-10-26T05:56:58.717688+0000","last_fullsized":"2023-10-26T05:56:58.717688+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:55:25.695338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689729+0000","last_change":"2023-10-26T05:56:58.689729+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689729+0000","last_undegraded":"2023-10-26T05:56:58.689729+0000","last_fullsized":"2023-10-26T05:56:58.689729+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.697371+0000","last_change":"2023-10-26T05:56:58.697256+0000","last_active":"2023-10-26T05:56:58.697371+0000","last_peered":"2023-10-26T05:56:58.697371+0000","last_clean":"2023-10-26T05:56:58.697371+0000","last_became_active":"2023-10-26T05:56:58.696984+0000","last_became_peered":"2023-10-26T05:56:58.696984+0000","last_unstale":"2023-10-26T05:56:58.697371+0000","last_undegraded":"2023-10-26T05:56:58.697371+0000","last_fullsized":"2023-10-26T05:56:58.697371+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:47.521770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687331+0000","last_change":"2023-10-26T05:56:58.687331+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687331+0000","last_undegraded":"2023-10-26T05:56:58.687331+0000","last_fullsized":"2023-10-26T05:56:58.687331+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687335+0000","last_change":"2023-10-26T05:56:58.687247+0000","last_active":"2023-10-26T05:56:58.687335+0000","last_peered":"2023-10-26T05:56:58.687335+0000","last_clean":"2023-10-26T05:56:58.687335+0000","last_became_active":"2023-10-26T05:56:58.686900+0000","last_became_peered":"2023-10-26T05:56:58.686900+0000","last_unstale":"2023-10-26T05:56:58.687335+0000","last_undegraded":"2023-10-26T05:56:58.687335+0000","last_fullsized":"2023-10-26T05:56:58.687335+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:57:16.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685469+0000","last_change":"2023-10-26T05:56:58.685469+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685469+0000","last_undegraded":"2023-10-26T05:56:58.685469+0000","last_fullsized":"2023-10-26T05:56:58.685469+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.688086+0000","last_change":"2023-10-26T05:56:58.687941+0000","last_active":"2023-10-26T05:56:58.688086+0000","last_peered":"2023-10-26T05:56:58.688086+0000","last_clean":"2023-10-26T05:56:58.688086+0000","last_became_active":"2023-10-26T05:56:58.687118+0000","last_became_peered":"2023-10-26T05:56:58.687118+0000","last_unstale":"2023-10-26T05:56:58.688086+0000","last_undegraded":"2023-10-26T05:56:58.688086+0000","last_fullsized":"2023-10-26T05:56:58.688086+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:10.994025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681960+0000","last_change":"2023-10-26T05:56:58.681960+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681960+0000","last_undegraded":"2023-10-26T05:56:58.681960+0000","last_fullsized":"2023-10-26T05:56:58.681960+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692825+0000","last_change":"2023-10-26T05:56:58.692632+0000","last_active":"2023-10-26T05:56:58.692825+0000","last_peered":"2023-10-26T05:56:58.692825+0000","last_clean":"2023-10-26T05:56:58.692825+0000","last_became_active":"2023-10-26T05:56:58.690733+0000","last_became_peered":"2023-10-26T05:56:58.690733+0000","last_unstale":"2023-10-26T05:56:58.692825+0000","last_undegraded":"2023-10-26T05:56:58.692825+0000","last_fullsized":"2023-10-26T05:56:58.692825+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:59.579324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691809+0000","last_change":"2023-10-26T05:56:58.691809+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691809+0000","last_undegraded":"2023-10-26T05:56:58.691809+0000","last_fullsized":"2023-10-26T05:56:58.691809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.718038+0000","last_change":"2023-10-26T05:56:58.717853+0000","last_active":"2023-10-26T05:56:58.718038+0000","last_peered":"2023-10-26T05:56:58.718038+0000","last_clean":"2023-10-26T05:56:58.718038+0000","last_became_active":"2023-10-26T05:56:58.716922+0000","last_became_peered":"2023-10-26T05:56:58.716922+0000","last_unstale":"2023-10-26T05:56:58.718038+0000","last_undegraded":"2023-10-26T05:56:58.718038+0000","last_fullsized":"2023-10-26T05:56:58.718038+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:59:48.203254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692948+0000","last_change":"2023-10-26T05:56:58.692948+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692948+0000","last_undegraded":"2023-10-26T05:56:58.692948+0000","last_fullsized":"2023-10-26T05:56:58.692948+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711684+0000","last_change":"2023-10-26T05:56:58.709784+0000","last_active":"2023-10-26T05:56:58.711684+0000","last_peered":"2023-10-26T05:56:58.711684+0000","last_clean":"2023-10-26T05:56:58.711684+0000","last_became_active":"2023-10-26T05:56:58.707275+0000","last_became_peered":"2023-10-26T05:56:58.707275+0000","last_unstale":"2023-10-26T05:56:58.711684+0000","last_undegraded":"2023-10-26T05:56:58.711684+0000","last_fullsized":"2023-10-26T05:56:58.711684+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:25:35.872395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693336+0000","last_change":"2023-10-26T05:56:58.693336+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693336+0000","last_undegraded":"2023-10-26T05:56:58.693336+0000","last_fullsized":"2023-10-26T05:56:58.693336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699845+0000","last_change":"2023-10-26T05:56:58.699782+0000","last_active":"2023-10-26T05:56:58.699845+0000","last_peered":"2023-10-26T05:56:58.699845+0000","last_clean":"2023-10-26T05:56:58.699845+0000","last_became_active":"2023-10-26T05:56:58.699054+0000","last_became_peered":"2023-10-26T05:56:58.699054+0000","last_unstale":"2023-10-26T05:56:58.699845+0000","last_undegraded":"2023-10-26T05:56:58.699845+0000","last_fullsized":"2023-10-26T05:56:58.699845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:55.048724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685430+0000","last_change":"2023-10-26T05:56:58.685430+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685430+0000","last_undegraded":"2023-10-26T05:56:58.685430+0000","last_fullsized":"2023-10-26T05:56:58.685430+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711741+0000","last_change":"2023-10-26T05:56:58.710168+0000","last_active":"2023-10-26T05:56:58.711741+0000","last_peered":"2023-10-26T05:56:58.711741+0000","last_clean":"2023-10-26T05:56:58.711741+0000","last_became_active":"2023-10-26T05:56:58.708495+0000","last_became_peered":"2023-10-26T05:56:58.708495+0000","last_unstale":"2023-10-26T05:56:58.711741+0000","last_undegraded":"2023-10-26T05:56:58.711741+0000","last_fullsized":"2023-10-26T05:56:58.711741+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:24.511118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-26T05:56:57.680722+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:56:57.680722+0000","last_peered":"2023-10-26T05:56:57.680722+0000","last_clean":"2023-10-26T05:56:57.680722+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:56:57.680722+0000","last_undegraded":"2023-10-26T05:56:57.680722+0000","last_fullsized":"2023-10-26T05:56:57.680722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.59099999999999997,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3100000000000001},"last":1.1899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":1.0029999999999999},"min":{"1min":0.64700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.232,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.032,"15min":0.98799999999999999},"min":{"1min":0.97999999999999998,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.139}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.68000000000000005,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1899999999999999,"5min":1.448,"15min":1.448},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93799999999999994,"15min":0.93400000000000005},"min":{"1min":0.63900000000000001,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1990000000000001,"5min":1.3700000000000001,"15min":1.4590000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93200000000000005,"15min":0.89800000000000002},"min":{"1min":0.76000000000000001,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":1.004},{"interface":"front","average":{"1min":1.002,"5min":0.96699999999999997,"15min":0.97199999999999998},"min":{"1min":0.77300000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3089999999999999,"5min":1.3129999999999999,"15min":1.375},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.133,"5min":1.639,"15min":1.639},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.93300000000000005,"15min":0.93999999999999995},"min":{"1min":0.65700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.24,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.337,"5min":1.482,"15min":1.6679999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0369999999999999,"15min":1.0640000000000001},"min":{"1min":0.72299999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.702,"15min":1.702},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.115,"15min":1.1060000000000001},"min":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.3420000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.5,"15min":1.6180000000000001},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.234,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.087},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.69999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.86599999999999999},"min":{"1min":0.64500000000000002,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.385,"5min":1.3899999999999999,"15min":1.405},"last":0.95499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.91000000000000003,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2450000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.2809999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97999999999999998,"15min":0.92500000000000004},"min":{"1min":0.76900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.3080000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0620000000000001,"15min":1.0009999999999999},"min":{"1min":0.69099999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.337,"5min":1.3759999999999999,"15min":1.399},"last":1.212}]},{"osd":4,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.503,"15min":0.502},"max":{"1min":1.2569999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.99299999999999999,"15min":0.98099999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.111}]},{"osd":5,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.05},"min":{"1min":0.89200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.236,"5min":1.5509999999999999,"15min":1.554},"last":1.028},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97199999999999998,"15min":0.98299999999999998},"min":{"1min":0.67200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.452,"5min":1.512,"15min":1.512},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1220000000000001,"15min":1.1120000000000001},"min":{"1min":0.78400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.321,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94499999999999995,"15min":0.95599999999999996},"min":{"1min":0.63400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.579},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.83599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":1.591,"15min":1.591},"last":1.3400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84699999999999998,"15min":0.878},"min":{"1min":0.625,"5min":0.53300000000000003,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.206,"15min":2.1150000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86799999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2430000000000001,"5min":1.272,"15min":1.464},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.70599999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.2170000000000001,"5min":1.569,"15min":1.569},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0640000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90900000000000003,"15min":0.91800000000000004},"min":{"1min":0.69799999999999995,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.363,"5min":1.448,"15min":1.548},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.78200000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.276,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.97599999999999998},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.286,"15min":2.1909999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0760000000000001,"15min":1.091},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.536,"15min":1.536},"last":1.194}]},{"osd":6,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.68000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.268,"5min":1.744,"15min":1.744},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.04,"15min":1.032},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":1.4279999999999999,"15min":1.4950000000000001},"last":1.161}]},{"osd":7,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.071},"min":{"1min":0.64100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.3109999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3360000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.92000000000000004,"15min":0.871},"min":{"1min":0.66200000000000003,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1919999999999999,"5min":1.2569999999999999,"15min":1.347},"last":1.075},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.70799999999999996,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.159,"5min":1.502,"15min":1.502},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.90400000000000003,"15min":0.84499999999999997},"min":{"1min":0.749,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.21,"5min":1.25,"15min":1.375},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.96099999999999997,"15min":0.91700000000000004},"min":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":1.198}]},{"osd":2,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.143},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90700000000000003,"15min":0.879},"min":{"1min":0.72899999999999998,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.143,"5min":1.2170000000000001,"15min":1.373},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":0.95199999999999996},"min":{"1min":0.79700000000000004,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.177,"5min":1.397,"15min":2.0190000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.091,"5min":0.995,"15min":0.96199999999999997},"min":{"1min":0.79100000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.28,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.2430000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0009999999999999,"15min":1},"min":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.2929999999999999,"5min":1.4179999999999999,"15min":1.673},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0429999999999999,"15min":1.014},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.42099999999999999},"max":{"1min":1.337,"5min":1.762,"15min":1.7969999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.054,"15min":1.042},"min":{"1min":0.76500000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.4970000000000001},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.103,"15min":1.0589999999999999},"min":{"1min":0.77300000000000002,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.3280000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.879,"5min":0.69599999999999995,"15min":0.53700000000000003},"max":{"1min":1.341,"5min":1.484,"15min":1.728},"last":1.2709999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.90100000000000002,"5min":1.385,"15min":1.385},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.52500000000000002,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.2689999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":1.091,"5min":1.2,"15min":1.349},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.38,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.02,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.018}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.752,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1479999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81999999999999995,"15min":0.78700000000000003},"min":{"1min":0.623,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.1850000000000001,"15min":1.4199999999999999},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.84399999999999997},"min":{"1min":0.70799999999999996,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.1120000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90200000000000002,"15min":0.91900000000000004},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.2789999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.72399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3,"5min":1.413,"15min":1.478},"last":1.454},{"interface":"front","average":{"1min":1.014,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.54,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.64200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.349,"15min":1.6060000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.091,"5min":0.96699999999999997,"15min":1.014},"min":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":1.359,"15min":5.6859999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.98599999999999999,"15min":0.94299999999999995},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.53400000000000003},"max":{"1min":1.472,"5min":2.016,"15min":2.016},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.008,"15min":1.032},"min":{"1min":0.68799999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.327,"5min":2.004,"15min":2.004},"last":1.2370000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71999999999999997,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.379,"15min":1.5249999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.69599999999999995},"min":{"1min":0.61699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.091,"5min":1.226,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.621},"min":{"1min":0.57799999999999996,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.90700000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.82199999999999995,"15min":0.76500000000000001},"min":{"1min":0.71599999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.24,"15min":2.8809999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94899999999999995,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93000000000000005,"15min":0.85999999999999999},"min":{"1min":0.69099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.151,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88500000000000001,"15min":0.85899999999999999},"min":{"1min":0.78200000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.997,"15min":0.93899999999999995},"min":{"1min":0.84999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.999,"15min":0.93100000000000005},"min":{"1min":0.80000000000000004,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.329,"5min":1.4790000000000001,"15min":1.653},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.008,"15min":0.94699999999999995},"min":{"1min":0.73199999999999998,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.3320000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.089,"15min":1.016},"min":{"1min":0.76500000000000001,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.3300000000000001,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.177}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":0.91900000000000004},"min":{"1min":0.84299999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.073,"5min":1.581,"15min":1.581},"last":1.119},{"interface":"front","average":{"1min":1.133,"5min":1.109,"15min":1.069},"min":{"1min":0.91200000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.462,"5min":1.823,"15min":15.352},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.157},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55600000000000005,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78300000000000003,"15min":0.73999999999999999},"min":{"1min":0.63700000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.073,"5min":1.6080000000000001,"15min":1.8169999999999999},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.754,"15min":0.76100000000000001},"min":{"1min":0.65600000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.147,"15min":1.7949999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.0760000000000001,"5min":1.298,"15min":1.298},"last":0.997}]},{"osd":4,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.874},"min":{"1min":0.73899999999999999,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1020000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.74099999999999999,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.345,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":1.2}]},{"osd":5,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.96299999999999997,"15min":0.96399999999999997},"min":{"1min":0.749,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.319,"5min":1.381,"15min":1.492},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.84899999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.2509999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0249999999999999},"min":{"1min":0.89200000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89700000000000002,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.2669999999999999,"5min":1.468,"15min":1.837},"last":1.155}]},{"osd":7,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.046,"15min":1.0309999999999999},"min":{"1min":0.82499999999999996,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.397,"5min":1.4179999999999999,"15min":1.6339999999999999},"last":1.294},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0640000000000001,"15min":1.0840000000000001},"min":{"1min":0.93100000000000005,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4890000000000001,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":1.0600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95599999999999996,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.39100000000000001,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.1020000000000001,"15min":1.177},"last":0.502}]},{"osd":3,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0249999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.72899999999999998},"min":{"1min":0.58999999999999997,"5min":0.32100000000000001,"15min":0.247},"max":{"1min":1.026,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91400000000000003,"15min":0.873},"min":{"1min":0.64200000000000002,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.319,"5min":1.395,"15min":1.395},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":0.93500000000000005},"min":{"1min":0.73699999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":1.1559999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.92200000000000004},"min":{"1min":0.61299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":2.7480000000000002},"last":1.073},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96299999999999997,"15min":0.92100000000000004},"min":{"1min":0.87,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.91900000000000004},"min":{"1min":0.77500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1639999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.131,"15min":1.0169999999999999},"min":{"1min":0.871,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.2829999999999999,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0109999999999999,"15min":0.998},"min":{"1min":0.74199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1539999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0329999999999999,"15min":0.999},"min":{"1min":0.93500000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.417,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":1.141}]}]}],"pool_statfs":[]}} 2023-10-26T05:57:03.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2023-10-26T05:57:03.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.687936+0000', 'last_change': '2023-10-26T05:56:58.687936+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.687936+0000', 'last_undegraded': '2023-10-26T05:56:58.687936+0000', 'last_fullsized': '2023-10-26T05:56:58.687936+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2023-10-26T05:57:03.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.689543+0000', 'last_change': '2023-10-26T05:56:58.689543+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.689543+0000', 'last_undegraded': '2023-10-26T05:56:58.689543+0000', 'last_fullsized': '2023-10-26T05:56:58.689543+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2023-10-26T05:57:03.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.701504+0000', 'last_change': '2023-10-26T05:56:58.701504+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.701504+0000', 'last_undegraded': '2023-10-26T05:56:58.701504+0000', 'last_fullsized': '2023-10-26T05:56:58.701504+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:57:03.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2023-10-26T05:57:03.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.687487+0000', 'last_change': '2023-10-26T05:56:58.687487+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.687487+0000', 'last_undegraded': '2023-10-26T05:56:58.687487+0000', 'last_fullsized': '2023-10-26T05:56:58.687487+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2023-10-26T05:57:03.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.699011+0000', 'last_change': '2023-10-26T05:56:58.699011+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.699011+0000', 'last_undegraded': '2023-10-26T05:56:58.699011+0000', 'last_fullsized': '2023-10-26T05:56:58.699011+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2023-10-26T05:57:03.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.687527+0000', 'last_change': '2023-10-26T05:56:58.687527+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.687527+0000', 'last_undegraded': '2023-10-26T05:56:58.687527+0000', 'last_fullsized': '2023-10-26T05:56:58.687527+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2023-10-26T05:57:03.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.684911+0000', 'last_change': '2023-10-26T05:56:58.684911+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.684911+0000', 'last_undegraded': '2023-10-26T05:56:58.684911+0000', 'last_fullsized': '2023-10-26T05:56:58.684911+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:57:03.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2023-10-26T05:57:03.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.686493+0000', 'last_change': '2023-10-26T05:56:58.686493+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.686493+0000', 'last_undegraded': '2023-10-26T05:56:58.686493+0000', 'last_fullsized': '2023-10-26T05:56:58.686493+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2023-10-26T05:57:03.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.686070+0000', 'last_change': '2023-10-26T05:56:58.686070+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.686070+0000', 'last_undegraded': '2023-10-26T05:56:58.686070+0000', 'last_fullsized': '2023-10-26T05:56:58.686070+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2023-10-26T05:57:03.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.697900+0000', 'last_change': '2023-10-26T05:56:58.697900+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.697900+0000', 'last_undegraded': '2023-10-26T05:56:58.697900+0000', 'last_fullsized': '2023-10-26T05:56:58.697900+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2023-10-26T05:57:03.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.687124+0000', 'last_change': '2023-10-26T05:56:58.687124+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.687124+0000', 'last_undegraded': '2023-10-26T05:56:58.687124+0000', 'last_fullsized': '2023-10-26T05:56:58.687124+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2023-10-26T05:57:03.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.700372+0000', 'last_change': '2023-10-26T05:56:58.700372+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.700372+0000', 'last_undegraded': '2023-10-26T05:56:58.700372+0000', 'last_fullsized': '2023-10-26T05:56:58.700372+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2023-10-26T05:57:03.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.695733+0000', 'last_change': '2023-10-26T05:56:58.695733+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.695733+0000', 'last_undegraded': '2023-10-26T05:56:58.695733+0000', 'last_fullsized': '2023-10-26T05:56:58.695733+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2023-10-26T05:57:03.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685971+0000', 'last_change': '2023-10-26T05:56:58.685971+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685971+0000', 'last_undegraded': '2023-10-26T05:56:58.685971+0000', 'last_fullsized': '2023-10-26T05:56:58.685971+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2023-10-26T05:57:03.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.697158+0000', 'last_change': '2023-10-26T05:56:58.697158+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.697158+0000', 'last_undegraded': '2023-10-26T05:56:58.697158+0000', 'last_fullsized': '2023-10-26T05:56:58.697158+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2023-10-26T05:57:03.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.682093+0000', 'last_change': '2023-10-26T05:56:58.682093+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.682093+0000', 'last_undegraded': '2023-10-26T05:56:58.682093+0000', 'last_fullsized': '2023-10-26T05:56:58.682093+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2023-10-26T05:57:03.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.693783+0000', 'last_change': '2023-10-26T05:56:58.693783+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.693783+0000', 'last_undegraded': '2023-10-26T05:56:58.693783+0000', 'last_fullsized': '2023-10-26T05:56:58.693783+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2023-10-26T05:57:03.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.702305+0000', 'last_change': '2023-10-26T05:56:58.702305+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.702305+0000', 'last_undegraded': '2023-10-26T05:56:58.702305+0000', 'last_fullsized': '2023-10-26T05:56:58.702305+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:57:03.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2023-10-26T05:57:03.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.686171+0000', 'last_change': '2023-10-26T05:56:58.686171+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.686171+0000', 'last_undegraded': '2023-10-26T05:56:58.686171+0000', 'last_fullsized': '2023-10-26T05:56:58.686171+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2023-10-26T05:57:03.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.684098+0000', 'last_change': '2023-10-26T05:56:58.684098+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.684098+0000', 'last_undegraded': '2023-10-26T05:56:58.684098+0000', 'last_fullsized': '2023-10-26T05:56:58.684098+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2023-10-26T05:57:03.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.693775+0000', 'last_change': '2023-10-26T05:56:58.693775+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.693775+0000', 'last_undegraded': '2023-10-26T05:56:58.693775+0000', 'last_fullsized': '2023-10-26T05:56:58.693775+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2023-10-26T05:57:03.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.688204+0000', 'last_change': '2023-10-26T05:56:58.688204+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.688204+0000', 'last_undegraded': '2023-10-26T05:56:58.688204+0000', 'last_fullsized': '2023-10-26T05:56:58.688204+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.424 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2023-10-26T05:57:03.424 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.684900+0000', 'last_change': '2023-10-26T05:56:58.684900+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.684900+0000', 'last_undegraded': '2023-10-26T05:56:58.684900+0000', 'last_fullsized': '2023-10-26T05:56:58.684900+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.425 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2023-10-26T05:57:03.425 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.694702+0000', 'last_change': '2023-10-26T05:56:58.694702+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.694702+0000', 'last_undegraded': '2023-10-26T05:56:58.694702+0000', 'last_fullsized': '2023-10-26T05:56:58.694702+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2023-10-26T05:57:03.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.683660+0000', 'last_change': '2023-10-26T05:56:58.683660+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.683660+0000', 'last_undegraded': '2023-10-26T05:56:58.683660+0000', 'last_fullsized': '2023-10-26T05:56:58.683660+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2023-10-26T05:57:03.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.686703+0000', 'last_change': '2023-10-26T05:56:58.686703+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.686703+0000', 'last_undegraded': '2023-10-26T05:56:58.686703+0000', 'last_fullsized': '2023-10-26T05:56:58.686703+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2023-10-26T05:57:03.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.682875+0000', 'last_change': '2023-10-26T05:56:58.682875+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.682875+0000', 'last_undegraded': '2023-10-26T05:56:58.682875+0000', 'last_fullsized': '2023-10-26T05:56:58.682875+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2023-10-26T05:57:03.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685434+0000', 'last_change': '2023-10-26T05:56:58.685434+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685434+0000', 'last_undegraded': '2023-10-26T05:56:58.685434+0000', 'last_fullsized': '2023-10-26T05:56:58.685434+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:57:03.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2023-10-26T05:57:03.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685841+0000', 'last_change': '2023-10-26T05:56:58.685841+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685841+0000', 'last_undegraded': '2023-10-26T05:56:58.685841+0000', 'last_fullsized': '2023-10-26T05:56:58.685841+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2023-10-26T05:57:03.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685847+0000', 'last_change': '2023-10-26T05:56:58.685847+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685847+0000', 'last_undegraded': '2023-10-26T05:56:58.685847+0000', 'last_fullsized': '2023-10-26T05:56:58.685847+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2023-10-26T05:57:03.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.684393+0000', 'last_change': '2023-10-26T05:56:58.684393+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.684393+0000', 'last_undegraded': '2023-10-26T05:56:58.684393+0000', 'last_fullsized': '2023-10-26T05:56:58.684393+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2023-10-26T05:57:03.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.682434+0000', 'last_change': '2023-10-26T05:56:58.682434+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.682434+0000', 'last_undegraded': '2023-10-26T05:56:58.682434+0000', 'last_fullsized': '2023-10-26T05:56:58.682434+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2023-10-26T05:57:03.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.693816+0000', 'last_change': '2023-10-26T05:56:58.693816+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.693816+0000', 'last_undegraded': '2023-10-26T05:56:58.693816+0000', 'last_fullsized': '2023-10-26T05:56:58.693816+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2023-10-26T05:57:03.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.683676+0000', 'last_change': '2023-10-26T05:56:58.683676+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.683676+0000', 'last_undegraded': '2023-10-26T05:56:58.683676+0000', 'last_fullsized': '2023-10-26T05:56:58.683676+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2023-10-26T05:57:03.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.686261+0000', 'last_change': '2023-10-26T05:56:58.686261+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.686261+0000', 'last_undegraded': '2023-10-26T05:56:58.686261+0000', 'last_fullsized': '2023-10-26T05:56:58.686261+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2023-10-26T05:57:03.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.696248+0000', 'last_change': '2023-10-26T05:56:58.696248+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.696248+0000', 'last_undegraded': '2023-10-26T05:56:58.696248+0000', 'last_fullsized': '2023-10-26T05:56:58.696248+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2023-10-26T05:57:03.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.683215+0000', 'last_change': '2023-10-26T05:56:58.683215+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.683215+0000', 'last_undegraded': '2023-10-26T05:56:58.683215+0000', 'last_fullsized': '2023-10-26T05:56:58.683215+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2023-10-26T05:57:03.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685163+0000', 'last_change': '2023-10-26T05:56:58.685163+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685163+0000', 'last_undegraded': '2023-10-26T05:56:58.685163+0000', 'last_fullsized': '2023-10-26T05:56:58.685163+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2023-10-26T05:57:03.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.686926+0000', 'last_change': '2023-10-26T05:56:58.686926+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.686926+0000', 'last_undegraded': '2023-10-26T05:56:58.686926+0000', 'last_fullsized': '2023-10-26T05:56:58.686926+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2023-10-26T05:57:03.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.698412+0000', 'last_change': '2023-10-26T05:56:58.698412+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.698412+0000', 'last_undegraded': '2023-10-26T05:56:58.698412+0000', 'last_fullsized': '2023-10-26T05:56:58.698412+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2023-10-26T05:57:03.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.691840+0000', 'last_change': '2023-10-26T05:56:58.691840+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.691840+0000', 'last_undegraded': '2023-10-26T05:56:58.691840+0000', 'last_fullsized': '2023-10-26T05:56:58.691840+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2023-10-26T05:57:03.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.692914+0000', 'last_change': '2023-10-26T05:56:58.692914+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.692914+0000', 'last_undegraded': '2023-10-26T05:56:58.692914+0000', 'last_fullsized': '2023-10-26T05:56:58.692914+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2023-10-26T05:57:03.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.682474+0000', 'last_change': '2023-10-26T05:56:58.682474+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.682474+0000', 'last_undegraded': '2023-10-26T05:56:58.682474+0000', 'last_fullsized': '2023-10-26T05:56:58.682474+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2023-10-26T05:57:03.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.689507+0000', 'last_change': '2023-10-26T05:56:58.689507+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.689507+0000', 'last_undegraded': '2023-10-26T05:56:58.689507+0000', 'last_fullsized': '2023-10-26T05:56:58.689507+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:57:03.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2023-10-26T05:57:03.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.687731+0000', 'last_change': '2023-10-26T05:56:58.687731+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.687731+0000', 'last_undegraded': '2023-10-26T05:56:58.687731+0000', 'last_fullsized': '2023-10-26T05:56:58.687731+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2023-10-26T05:57:03.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.694993+0000', 'last_change': '2023-10-26T05:56:58.694993+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.694993+0000', 'last_undegraded': '2023-10-26T05:56:58.694993+0000', 'last_fullsized': '2023-10-26T05:56:58.694993+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2023-10-26T05:57:03.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.699789+0000', 'last_change': '2023-10-26T05:56:58.699789+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.699789+0000', 'last_undegraded': '2023-10-26T05:56:58.699789+0000', 'last_fullsized': '2023-10-26T05:56:58.699789+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:57:03.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2023-10-26T05:57:03.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.695608+0000', 'last_change': '2023-10-26T05:56:58.695608+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.695608+0000', 'last_undegraded': '2023-10-26T05:56:58.695608+0000', 'last_fullsized': '2023-10-26T05:56:58.695608+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2023-10-26T05:57:03.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.681733+0000', 'last_change': '2023-10-26T05:56:58.681733+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.681733+0000', 'last_undegraded': '2023-10-26T05:56:58.681733+0000', 'last_fullsized': '2023-10-26T05:56:58.681733+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2023-10-26T05:57:03.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.690498+0000', 'last_change': '2023-10-26T05:56:58.690498+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.690498+0000', 'last_undegraded': '2023-10-26T05:56:58.690498+0000', 'last_fullsized': '2023-10-26T05:56:58.690498+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:57:03.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2023-10-26T05:57:03.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685813+0000', 'last_change': '2023-10-26T05:56:58.685813+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685813+0000', 'last_undegraded': '2023-10-26T05:56:58.685813+0000', 'last_fullsized': '2023-10-26T05:56:58.685813+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2023-10-26T05:57:03.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.684950+0000', 'last_change': '2023-10-26T05:56:58.684950+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.684950+0000', 'last_undegraded': '2023-10-26T05:56:58.684950+0000', 'last_fullsized': '2023-10-26T05:56:58.684950+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2023-10-26T05:57:03.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685015+0000', 'last_change': '2023-10-26T05:56:58.685015+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685015+0000', 'last_undegraded': '2023-10-26T05:56:58.685015+0000', 'last_fullsized': '2023-10-26T05:56:58.685015+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:57:03.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2023-10-26T05:57:03.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685000+0000', 'last_change': '2023-10-26T05:56:58.685000+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685000+0000', 'last_undegraded': '2023-10-26T05:56:58.685000+0000', 'last_fullsized': '2023-10-26T05:56:58.685000+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-26T05:57:03.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2023-10-26T05:57:03.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.684550+0000', 'last_change': '2023-10-26T05:56:58.684550+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.684550+0000', 'last_undegraded': '2023-10-26T05:56:58.684550+0000', 'last_fullsized': '2023-10-26T05:56:58.684550+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:57:03.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2023-10-26T05:57:03.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.694164+0000', 'last_change': '2023-10-26T05:56:58.694164+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.694164+0000', 'last_undegraded': '2023-10-26T05:56:58.694164+0000', 'last_fullsized': '2023-10-26T05:56:58.694164+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2023-10-26T05:57:03.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.689729+0000', 'last_change': '2023-10-26T05:56:58.689729+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.689729+0000', 'last_undegraded': '2023-10-26T05:56:58.689729+0000', 'last_fullsized': '2023-10-26T05:56:58.689729+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-26T05:57:03.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2023-10-26T05:57:03.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.687331+0000', 'last_change': '2023-10-26T05:56:58.687331+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.687331+0000', 'last_undegraded': '2023-10-26T05:56:58.687331+0000', 'last_fullsized': '2023-10-26T05:56:58.687331+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-26T05:57:03.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2023-10-26T05:57:03.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685469+0000', 'last_change': '2023-10-26T05:56:58.685469+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685469+0000', 'last_undegraded': '2023-10-26T05:56:58.685469+0000', 'last_fullsized': '2023-10-26T05:56:58.685469+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-26T05:57:03.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2023-10-26T05:57:03.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.681960+0000', 'last_change': '2023-10-26T05:56:58.681960+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.681960+0000', 'last_undegraded': '2023-10-26T05:56:58.681960+0000', 'last_fullsized': '2023-10-26T05:56:58.681960+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-26T05:57:03.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2023-10-26T05:57:03.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.691809+0000', 'last_change': '2023-10-26T05:56:58.691809+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.691809+0000', 'last_undegraded': '2023-10-26T05:56:58.691809+0000', 'last_fullsized': '2023-10-26T05:56:58.691809+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2023-10-26T05:57:03.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.692948+0000', 'last_change': '2023-10-26T05:56:58.692948+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.692948+0000', 'last_undegraded': '2023-10-26T05:56:58.692948+0000', 'last_fullsized': '2023-10-26T05:56:58.692948+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-26T05:57:03.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2023-10-26T05:57:03.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.693336+0000', 'last_change': '2023-10-26T05:56:58.693336+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.693336+0000', 'last_undegraded': '2023-10-26T05:56:58.693336+0000', 'last_fullsized': '2023-10-26T05:56:58.693336+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-26T05:57:03.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2023-10-26T05:57:03.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 170, 'state': 'creating+peering', 'last_fresh': '2023-10-26T05:56:58.685430+0000', 'last_change': '2023-10-26T05:56:58.685430+0000', 'last_active': '2023-10-26T05:56:58.661153+0000', 'last_peered': '2023-10-26T05:56:58.661153+0000', 'last_clean': '2023-10-26T05:56:58.661153+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-26T05:56:58.685430+0000', 'last_undegraded': '2023-10-26T05:56:58.685430+0000', 'last_fullsized': '2023-10-26T05:56:58.685430+0000', 'mapping_epoch': 170, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 170, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'last_clean_scrub_stamp': '2023-10-26T05:56:58.661153+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-26T05:57:03.458 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-26T05:57:03.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 -- 172.21.15.71:0/1432202530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06c650 msgr2=0x7fdc6c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 --2- 172.21.15.71:0/1432202530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06c650 0x7fdc6c0cebb0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009bc0 tx=0x7fdc6401cb10 comp rx=0 tx=0).stop 2023-10-26T05:57:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 -- 172.21.15.71:0/1432202530 shutdown_connections 2023-10-26T05:57:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 --2- 172.21.15.71:0/1432202530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06c650 0x7fdc6c0cebb0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 -- 172.21.15.71:0/1432202530 >> 172.21.15.71:0/1432202530 conn(0x7fdc6c0b0e00 msgr2=0x7fdc6c0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 -- 172.21.15.71:0/1432202530 shutdown_connections 2023-10-26T05:57:03.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.575+0000 7fdc7341c640 1 -- 172.21.15.71:0/1432202530 wait complete. 2023-10-26T05:57:03.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 Processor -- start 2023-10-26T05:57:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- start start 2023-10-26T05:57:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 0x7fdc6c157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc6c06c650 0x7fdc6c147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc6c158dc0 0x7fdc6c0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0c90e0 con 0x7fdc6c06b820 2023-10-26T05:57:03.589 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0c84d0 con 0x7fdc6c158dc0 2023-10-26T05:57:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0c87b0 con 0x7fdc6c06c650 2023-10-26T05:57:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 0x7fdc6c157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 0x7fdc6c157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35820/0 (socket says 172.21.15.71:35820) 2023-10-26T05:57:03.590 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 -- 172.21.15.71:0/2858968293 learned_addr learned my addr 172.21.15.71:0/2858968293 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc709b7640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc6c06c650 0x7fdc6c147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc719b9640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc6c158dc0 0x7fdc6c0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 -- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc6c06c650 msgr2=0x7fdc6c147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc6c06c650 0x7fdc6c147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 -- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc6c158dc0 msgr2=0x7fdc6c0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc6c158dc0 0x7fdc6c0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 -- 172.21.15.71:0/2858968293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc64009820 con 0x7fdc6c06b820 2023-10-26T05:57:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc719b9640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc6c158dc0 0x7fdc6c0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:57:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc711b8640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 0x7fdc6c157570 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00c370 tx=0x7fdc5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004230 con 0x7fdc6c06b820 2023-10-26T05:57:03.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fdc5c0044a0 con 0x7fdc6c06b820 2023-10-26T05:57:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c0c8a90 con 0x7fdc6c06b820 2023-10-26T05:57:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c005240 con 0x7fdc6c06b820 2023-10-26T05:57:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c148530 con 0x7fdc6c06b820 2023-10-26T05:57:03.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c148720 con 0x7fdc6c06b820 2023-10-26T05:57:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc5c0053a0 con 0x7fdc6c06b820 2023-10-26T05:57:03.594 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.579+0000 7fdc61ffb640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdc50066470 0x7fdc50068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.583+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc5c0832e0 con 0x7fdc6c06b820 2023-10-26T05:57:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.583+0000 7fdc709b7640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdc50066470 0x7fdc50068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.583+0000 7fdc709b7640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdc50066470 0x7fdc50068930 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c06ee40 tx=0x7fdc68007e40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:03.595 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.587+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fdc6c148720 con 0x7fdc6c06b820 2023-10-26T05:57:03.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fdc6c148720 con 0x7fdc6c06b820 2023-10-26T05:57:03.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc61ffb640 1 -- 172.21.15.71:0/2858968293 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1031 (secure 0 0 0) 0x7fdc6c148720 con 0x7fdc6c06b820 2023-10-26T05:57:03.799 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:03.799 INFO:teuthology.orchestra.run.smithi071.stdout:{"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":601,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":172,"num_osds":8,"num_up_osds":8,"osd_up_since":1698299229,"num_in_osds":8,"osd_in_since":1698299226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":8794586,"bytes_avail":1591205414,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":64,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":8958}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-10-26T05:55:43.580012+0000","services":{}},"progress_events":{}} 2023-10-26T05:57:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdc50066470 msgr2=0x7fdc50068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdc50066470 0x7fdc50068930 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c06ee40 tx=0x7fdc68007e40 comp rx=0 tx=0).stop 2023-10-26T05:57:03.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 msgr2=0x7fdc6c157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 0x7fdc6c157570 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00c370 tx=0x7fdc5c00c830 comp rx=0 tx=0).stop 2023-10-26T05:57:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 shutdown_connections 2023-10-26T05:57:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc6c158dc0 0x7fdc6c0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc6c06c650 0x7fdc6c147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdc50066470 0x7fdc50068930 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 --2- 172.21.15.71:0/2858968293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc6c06b820 0x7fdc6c157570 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 >> 172.21.15.71:0/2858968293 conn(0x7fdc6c0b0e00 msgr2=0x7fdc6c159470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:03.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 shutdown_connections 2023-10-26T05:57:03.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.791+0000 7fdc7341c640 1 -- 172.21.15.71:0/2858968293 wait complete. 2023-10-26T05:57:03.816 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-26T05:57:03.817 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:57:03.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- 172.21.15.71:0/497211509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0b2a60 msgr2=0x7f6b8c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 --2- 172.21.15.71:0/497211509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0b2a60 0x7f6b8c0caad0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009840 tx=0x7f6b7401c800 comp rx=0 tx=0).stop 2023-10-26T05:57:03.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- 172.21.15.71:0/497211509 shutdown_connections 2023-10-26T05:57:03.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 --2- 172.21.15.71:0/497211509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0b2a60 0x7f6b8c0caad0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- 172.21.15.71:0/497211509 >> 172.21.15.71:0/497211509 conn(0x7f6b8c069770 msgr2=0x7f6b8c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:03.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- 172.21.15.71:0/497211509 shutdown_connections 2023-10-26T05:57:03.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- 172.21.15.71:0/497211509 wait complete. 2023-10-26T05:57:03.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 Processor -- start 2023-10-26T05:57:03.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- start start 2023-10-26T05:57:03.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b8c0b2a60 0x7f6b8c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0cb010 0x7f6b8c13af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 0x7f6b8c134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c13a1c0 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c0addd0 con 0x7f6b8c0cb010 2023-10-26T05:57:03.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b912e8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c13a470 con 0x7f6b8c0b2a60 2023-10-26T05:57:03.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 0x7f6b8c134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 0x7f6b8c134a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35824/0 (socket says 172.21.15.71:35824) 2023-10-26T05:57:03.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 -- 172.21.15.71:0/264019874 learned_addr learned my addr 172.21.15.71:0/264019874 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:03.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8affd640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b8c0b2a60 0x7f6b8c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b827fc640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0cb010 0x7f6b8c13af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 -- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b8c0b2a60 msgr2=0x7f6b8c13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b8c0b2a60 0x7f6b8c13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 -- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0cb010 msgr2=0x7f6b8c13af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:03.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0cb010 0x7f6b8c13af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:03.945 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 -- 172.21.15.71:0/264019874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740094a0 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.931+0000 7f6b8b7fe640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 0x7f6b8c134a70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400cbc0 tx=0x7f6b8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8400aa50 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6b8400aca0 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c13a750 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.946 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84012680 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c141230 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.947 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b8c066b10 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6b8401a3b0 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b83fff640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b6c0664a0 0x7f6b6c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:03.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b840840c0 con 0x7f6b8c13c2f0 2023-10-26T05:57:03.948 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b8affd640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b6c0664a0 0x7f6b6c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:03.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.935+0000 7f6b8affd640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b6c0664a0 0x7f6b6c068960 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0ca4a0 tx=0x7f6b7c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:03.949 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:03.939+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6b84051680 con 0x7f6b8c13c2f0 2023-10-26T05:57:04.094 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.087+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b8c13a5e0 con 0x7f6b6c0664a0 2023-10-26T05:57:04.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.107+0000 7f6b83fff640 1 -- 172.21.15.71:0/264019874 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278382 (secure 0 0 0) 0x7f6b8c13a5e0 con 0x7f6b6c0664a0 2023-10-26T05:57:04.117 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:57:04.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b6c0664a0 msgr2=0x7f6b6c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b6c0664a0 0x7f6b6c068960 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0ca4a0 tx=0x7f6b7c009040 comp rx=0 tx=0).stop 2023-10-26T05:57:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 msgr2=0x7f6b8c134a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 0x7f6b8c134a70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400cbc0 tx=0x7f6b8400ccb0 comp rx=0 tx=0).stop 2023-10-26T05:57:04.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 shutdown_connections 2023-10-26T05:57:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6b6c0664a0 0x7f6b6c068960 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8c13c2f0 0x7f6b8c134a70 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c0cb010 0x7f6b8c13af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 --2- 172.21.15.71:0/264019874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6b8c0b2a60 0x7f6b8c13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 >> 172.21.15.71:0/264019874 conn(0x7f6b8c069770 msgr2=0x7f6b8c0ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:04.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 shutdown_connections 2023-10-26T05:57:04.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:04.111+0000 7f6b912e8640 1 -- 172.21.15.71:0/264019874 wait complete. 2023-10-26T05:57:04.134 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2023-10-26T05:57:03.608165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8588,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553911,"statfs":{"total":1600000000,"available":1591205414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.934289"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687936+0000","last_change":"2023-10-26T05:56:58.687936+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687936+0000","last_undegraded":"2023-10-26T05:56:58.687936+0000","last_fullsized":"2023-10-26T05:56:58.687936+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689543+0000","last_change":"2023-10-26T05:56:58.689543+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689543+0000","last_undegraded":"2023-10-26T05:56:58.689543+0000","last_fullsized":"2023-10-26T05:56:58.689543+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.701504+0000","last_change":"2023-10-26T05:56:58.701504+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.701504+0000","last_undegraded":"2023-10-26T05:56:58.701504+0000","last_fullsized":"2023-10-26T05:56:58.701504+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687487+0000","last_change":"2023-10-26T05:56:58.687487+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687487+0000","last_undegraded":"2023-10-26T05:56:58.687487+0000","last_fullsized":"2023-10-26T05:56:58.687487+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699011+0000","last_change":"2023-10-26T05:56:58.699011+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699011+0000","last_undegraded":"2023-10-26T05:56:58.699011+0000","last_fullsized":"2023-10-26T05:56:58.699011+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687527+0000","last_change":"2023-10-26T05:56:58.687527+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687527+0000","last_undegraded":"2023-10-26T05:56:58.687527+0000","last_fullsized":"2023-10-26T05:56:58.687527+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684911+0000","last_change":"2023-10-26T05:56:58.684911+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684911+0000","last_undegraded":"2023-10-26T05:56:58.684911+0000","last_fullsized":"2023-10-26T05:56:58.684911+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686493+0000","last_change":"2023-10-26T05:56:58.686493+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686493+0000","last_undegraded":"2023-10-26T05:56:58.686493+0000","last_fullsized":"2023-10-26T05:56:58.686493+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686070+0000","last_change":"2023-10-26T05:56:58.686070+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686070+0000","last_undegraded":"2023-10-26T05:56:58.686070+0000","last_fullsized":"2023-10-26T05:56:58.686070+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697900+0000","last_change":"2023-10-26T05:56:58.697900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697900+0000","last_undegraded":"2023-10-26T05:56:58.697900+0000","last_fullsized":"2023-10-26T05:56:58.697900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687124+0000","last_change":"2023-10-26T05:56:58.687124+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687124+0000","last_undegraded":"2023-10-26T05:56:58.687124+0000","last_fullsized":"2023-10-26T05:56:58.687124+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.700372+0000","last_change":"2023-10-26T05:56:58.700372+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.700372+0000","last_undegraded":"2023-10-26T05:56:58.700372+0000","last_fullsized":"2023-10-26T05:56:58.700372+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695733+0000","last_change":"2023-10-26T05:56:58.695733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695733+0000","last_undegraded":"2023-10-26T05:56:58.695733+0000","last_fullsized":"2023-10-26T05:56:58.695733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685971+0000","last_change":"2023-10-26T05:56:58.685971+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685971+0000","last_undegraded":"2023-10-26T05:56:58.685971+0000","last_fullsized":"2023-10-26T05:56:58.685971+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.697158+0000","last_change":"2023-10-26T05:56:58.697158+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.697158+0000","last_undegraded":"2023-10-26T05:56:58.697158+0000","last_fullsized":"2023-10-26T05:56:58.697158+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682093+0000","last_change":"2023-10-26T05:56:58.682093+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682093+0000","last_undegraded":"2023-10-26T05:56:58.682093+0000","last_fullsized":"2023-10-26T05:56:58.682093+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693783+0000","last_change":"2023-10-26T05:56:58.693783+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693783+0000","last_undegraded":"2023-10-26T05:56:58.693783+0000","last_fullsized":"2023-10-26T05:56:58.693783+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.702305+0000","last_change":"2023-10-26T05:56:58.702305+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.702305+0000","last_undegraded":"2023-10-26T05:56:58.702305+0000","last_fullsized":"2023-10-26T05:56:58.702305+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686171+0000","last_change":"2023-10-26T05:56:58.686171+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686171+0000","last_undegraded":"2023-10-26T05:56:58.686171+0000","last_fullsized":"2023-10-26T05:56:58.686171+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684098+0000","last_change":"2023-10-26T05:56:58.684098+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684098+0000","last_undegraded":"2023-10-26T05:56:58.684098+0000","last_fullsized":"2023-10-26T05:56:58.684098+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693775+0000","last_change":"2023-10-26T05:56:58.693775+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693775+0000","last_undegraded":"2023-10-26T05:56:58.693775+0000","last_fullsized":"2023-10-26T05:56:58.693775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.688204+0000","last_change":"2023-10-26T05:56:58.688204+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.688204+0000","last_undegraded":"2023-10-26T05:56:58.688204+0000","last_fullsized":"2023-10-26T05:56:58.688204+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684900+0000","last_change":"2023-10-26T05:56:58.684900+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684900+0000","last_undegraded":"2023-10-26T05:56:58.684900+0000","last_fullsized":"2023-10-26T05:56:58.684900+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694702+0000","last_change":"2023-10-26T05:56:58.694702+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694702+0000","last_undegraded":"2023-10-26T05:56:58.694702+0000","last_fullsized":"2023-10-26T05:56:58.694702+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683660+0000","last_change":"2023-10-26T05:56:58.683660+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683660+0000","last_undegraded":"2023-10-26T05:56:58.683660+0000","last_fullsized":"2023-10-26T05:56:58.683660+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686703+0000","last_change":"2023-10-26T05:56:58.686703+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686703+0000","last_undegraded":"2023-10-26T05:56:58.686703+0000","last_fullsized":"2023-10-26T05:56:58.686703+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682875+0000","last_change":"2023-10-26T05:56:58.682875+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682875+0000","last_undegraded":"2023-10-26T05:56:58.682875+0000","last_fullsized":"2023-10-26T05:56:58.682875+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685434+0000","last_change":"2023-10-26T05:56:58.685434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685434+0000","last_undegraded":"2023-10-26T05:56:58.685434+0000","last_fullsized":"2023-10-26T05:56:58.685434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685841+0000","last_change":"2023-10-26T05:56:58.685841+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685841+0000","last_undegraded":"2023-10-26T05:56:58.685841+0000","last_fullsized":"2023-10-26T05:56:58.685841+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685847+0000","last_change":"2023-10-26T05:56:58.685847+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685847+0000","last_undegraded":"2023-10-26T05:56:58.685847+0000","last_fullsized":"2023-10-26T05:56:58.685847+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684393+0000","last_change":"2023-10-26T05:56:58.684393+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684393+0000","last_undegraded":"2023-10-26T05:56:58.684393+0000","last_fullsized":"2023-10-26T05:56:58.684393+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682434+0000","last_change":"2023-10-26T05:56:58.682434+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682434+0000","last_undegraded":"2023-10-26T05:56:58.682434+0000","last_fullsized":"2023-10-26T05:56:58.682434+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693816+0000","last_change":"2023-10-26T05:56:58.693816+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693816+0000","last_undegraded":"2023-10-26T05:56:58.693816+0000","last_fullsized":"2023-10-26T05:56:58.693816+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711649+0000","last_change":"2023-10-26T05:56:58.711496+0000","last_active":"2023-10-26T05:56:58.711649+0000","last_peered":"2023-10-26T05:56:58.711649+0000","last_clean":"2023-10-26T05:56:58.711649+0000","last_became_active":"2023-10-26T05:56:58.710397+0000","last_became_peered":"2023-10-26T05:56:58.710397+0000","last_unstale":"2023-10-26T05:56:58.711649+0000","last_undegraded":"2023-10-26T05:56:58.711649+0000","last_fullsized":"2023-10-26T05:56:58.711649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:45:28.308338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683676+0000","last_change":"2023-10-26T05:56:58.683676+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683676+0000","last_undegraded":"2023-10-26T05:56:58.683676+0000","last_fullsized":"2023-10-26T05:56:58.683676+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699941+0000","last_change":"2023-10-26T05:56:58.699902+0000","last_active":"2023-10-26T05:56:58.699941+0000","last_peered":"2023-10-26T05:56:58.699941+0000","last_clean":"2023-10-26T05:56:58.699941+0000","last_became_active":"2023-10-26T05:56:58.699481+0000","last_became_peered":"2023-10-26T05:56:58.699481+0000","last_unstale":"2023-10-26T05:56:58.699941+0000","last_undegraded":"2023-10-26T05:56:58.699941+0000","last_fullsized":"2023-10-26T05:56:58.699941+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:15.048906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686261+0000","last_change":"2023-10-26T05:56:58.686261+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686261+0000","last_undegraded":"2023-10-26T05:56:58.686261+0000","last_fullsized":"2023-10-26T05:56:58.686261+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692322+0000","last_change":"2023-10-26T05:56:58.691994+0000","last_active":"2023-10-26T05:56:58.692322+0000","last_peered":"2023-10-26T05:56:58.692322+0000","last_clean":"2023-10-26T05:56:58.692322+0000","last_became_active":"2023-10-26T05:56:58.691381+0000","last_became_peered":"2023-10-26T05:56:58.691381+0000","last_unstale":"2023-10-26T05:56:58.692322+0000","last_undegraded":"2023-10-26T05:56:58.692322+0000","last_fullsized":"2023-10-26T05:56:58.692322+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:05:13.177638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.696248+0000","last_change":"2023-10-26T05:56:58.696248+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.696248+0000","last_undegraded":"2023-10-26T05:56:58.696248+0000","last_fullsized":"2023-10-26T05:56:58.696248+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687988+0000","last_change":"2023-10-26T05:56:58.687804+0000","last_active":"2023-10-26T05:56:58.687988+0000","last_peered":"2023-10-26T05:56:58.687988+0000","last_clean":"2023-10-26T05:56:58.687988+0000","last_became_active":"2023-10-26T05:56:58.687167+0000","last_became_peered":"2023-10-26T05:56:58.687167+0000","last_unstale":"2023-10-26T05:56:58.687988+0000","last_undegraded":"2023-10-26T05:56:58.687988+0000","last_fullsized":"2023-10-26T05:56:58.687988+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:47.280630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.683215+0000","last_change":"2023-10-26T05:56:58.683215+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.683215+0000","last_undegraded":"2023-10-26T05:56:58.683215+0000","last_fullsized":"2023-10-26T05:56:58.683215+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.709083+0000","last_change":"2023-10-26T05:56:58.708982+0000","last_active":"2023-10-26T05:56:58.709083+0000","last_peered":"2023-10-26T05:56:58.709083+0000","last_clean":"2023-10-26T05:56:58.709083+0000","last_became_active":"2023-10-26T05:56:58.708308+0000","last_became_peered":"2023-10-26T05:56:58.708308+0000","last_unstale":"2023-10-26T05:56:58.709083+0000","last_undegraded":"2023-10-26T05:56:58.709083+0000","last_fullsized":"2023-10-26T05:56:58.709083+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:09:39.204476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685163+0000","last_change":"2023-10-26T05:56:58.685163+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685163+0000","last_undegraded":"2023-10-26T05:56:58.685163+0000","last_fullsized":"2023-10-26T05:56:58.685163+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.707771+0000","last_change":"2023-10-26T05:56:58.707555+0000","last_active":"2023-10-26T05:56:58.707771+0000","last_peered":"2023-10-26T05:56:58.707771+0000","last_clean":"2023-10-26T05:56:58.707771+0000","last_became_active":"2023-10-26T05:56:58.706918+0000","last_became_peered":"2023-10-26T05:56:58.706918+0000","last_unstale":"2023-10-26T05:56:58.707771+0000","last_undegraded":"2023-10-26T05:56:58.707771+0000","last_fullsized":"2023-10-26T05:56:58.707771+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:12:04.008423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.686926+0000","last_change":"2023-10-26T05:56:58.686926+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.686926+0000","last_undegraded":"2023-10-26T05:56:58.686926+0000","last_fullsized":"2023-10-26T05:56:58.686926+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.702597+0000","last_change":"2023-10-26T05:56:58.702478+0000","last_active":"2023-10-26T05:56:58.702597+0000","last_peered":"2023-10-26T05:56:58.702597+0000","last_clean":"2023-10-26T05:56:58.702597+0000","last_became_active":"2023-10-26T05:56:58.702066+0000","last_became_peered":"2023-10-26T05:56:58.702066+0000","last_unstale":"2023-10-26T05:56:58.702597+0000","last_undegraded":"2023-10-26T05:56:58.702597+0000","last_fullsized":"2023-10-26T05:56:58.702597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:34:10.521714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.698412+0000","last_change":"2023-10-26T05:56:58.698412+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.698412+0000","last_undegraded":"2023-10-26T05:56:58.698412+0000","last_fullsized":"2023-10-26T05:56:58.698412+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.705203+0000","last_change":"2023-10-26T05:56:58.705010+0000","last_active":"2023-10-26T05:56:58.705203+0000","last_peered":"2023-10-26T05:56:58.705203+0000","last_clean":"2023-10-26T05:56:58.705203+0000","last_became_active":"2023-10-26T05:56:58.704379+0000","last_became_peered":"2023-10-26T05:56:58.704379+0000","last_unstale":"2023-10-26T05:56:58.705203+0000","last_undegraded":"2023-10-26T05:56:58.705203+0000","last_fullsized":"2023-10-26T05:56:58.705203+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:22.102825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691840+0000","last_change":"2023-10-26T05:56:58.691840+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691840+0000","last_undegraded":"2023-10-26T05:56:58.691840+0000","last_fullsized":"2023-10-26T05:56:58.691840+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692523+0000","last_change":"2023-10-26T05:56:58.692438+0000","last_active":"2023-10-26T05:56:58.692523+0000","last_peered":"2023-10-26T05:56:58.692523+0000","last_clean":"2023-10-26T05:56:58.692523+0000","last_became_active":"2023-10-26T05:56:58.692192+0000","last_became_peered":"2023-10-26T05:56:58.692192+0000","last_unstale":"2023-10-26T05:56:58.692523+0000","last_undegraded":"2023-10-26T05:56:58.692523+0000","last_fullsized":"2023-10-26T05:56:58.692523+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:20:10.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692914+0000","last_change":"2023-10-26T05:56:58.692914+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692914+0000","last_undegraded":"2023-10-26T05:56:58.692914+0000","last_fullsized":"2023-10-26T05:56:58.692914+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.715857+0000","last_change":"2023-10-26T05:56:58.715723+0000","last_active":"2023-10-26T05:56:58.715857+0000","last_peered":"2023-10-26T05:56:58.715857+0000","last_clean":"2023-10-26T05:56:58.715857+0000","last_became_active":"2023-10-26T05:56:58.714758+0000","last_became_peered":"2023-10-26T05:56:58.714758+0000","last_unstale":"2023-10-26T05:56:58.715857+0000","last_undegraded":"2023-10-26T05:56:58.715857+0000","last_fullsized":"2023-10-26T05:56:58.715857+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:50.434850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.682474+0000","last_change":"2023-10-26T05:56:58.682474+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.682474+0000","last_undegraded":"2023-10-26T05:56:58.682474+0000","last_fullsized":"2023-10-26T05:56:58.682474+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.691946+0000","last_change":"2023-10-26T05:56:58.691788+0000","last_active":"2023-10-26T05:56:58.691946+0000","last_peered":"2023-10-26T05:56:58.691946+0000","last_clean":"2023-10-26T05:56:58.691946+0000","last_became_active":"2023-10-26T05:56:58.691194+0000","last_became_peered":"2023-10-26T05:56:58.691194+0000","last_unstale":"2023-10-26T05:56:58.691946+0000","last_undegraded":"2023-10-26T05:56:58.691946+0000","last_fullsized":"2023-10-26T05:56:58.691946+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:05:11.942603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689507+0000","last_change":"2023-10-26T05:56:58.689507+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689507+0000","last_undegraded":"2023-10-26T05:56:58.689507+0000","last_fullsized":"2023-10-26T05:56:58.689507+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.700529+0000","last_change":"2023-10-26T05:56:58.700296+0000","last_active":"2023-10-26T05:56:58.700529+0000","last_peered":"2023-10-26T05:56:58.700529+0000","last_clean":"2023-10-26T05:56:58.700529+0000","last_became_active":"2023-10-26T05:56:58.699651+0000","last_became_peered":"2023-10-26T05:56:58.699651+0000","last_unstale":"2023-10-26T05:56:58.700529+0000","last_undegraded":"2023-10-26T05:56:58.700529+0000","last_fullsized":"2023-10-26T05:56:58.700529+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:42:59.754770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687731+0000","last_change":"2023-10-26T05:56:58.687731+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687731+0000","last_undegraded":"2023-10-26T05:56:58.687731+0000","last_fullsized":"2023-10-26T05:56:58.687731+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708870+0000","last_change":"2023-10-26T05:56:58.708596+0000","last_active":"2023-10-26T05:56:58.708870+0000","last_peered":"2023-10-26T05:56:58.708870+0000","last_clean":"2023-10-26T05:56:58.708870+0000","last_became_active":"2023-10-26T05:56:58.707912+0000","last_became_peered":"2023-10-26T05:56:58.707912+0000","last_unstale":"2023-10-26T05:56:58.708870+0000","last_undegraded":"2023-10-26T05:56:58.708870+0000","last_fullsized":"2023-10-26T05:56:58.708870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:40:53.740361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694993+0000","last_change":"2023-10-26T05:56:58.694993+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694993+0000","last_undegraded":"2023-10-26T05:56:58.694993+0000","last_fullsized":"2023-10-26T05:56:58.694993+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.684540+0000","last_change":"2023-10-26T05:56:58.684439+0000","last_active":"2023-10-26T05:56:58.684540+0000","last_peered":"2023-10-26T05:56:58.684540+0000","last_clean":"2023-10-26T05:56:58.684540+0000","last_became_active":"2023-10-26T05:56:58.684093+0000","last_became_peered":"2023-10-26T05:56:58.684093+0000","last_unstale":"2023-10-26T05:56:58.684540+0000","last_undegraded":"2023-10-26T05:56:58.684540+0000","last_fullsized":"2023-10-26T05:56:58.684540+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:34:28.905001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.699789+0000","last_change":"2023-10-26T05:56:58.699789+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.699789+0000","last_undegraded":"2023-10-26T05:56:58.699789+0000","last_fullsized":"2023-10-26T05:56:58.699789+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703649+0000","last_change":"2023-10-26T05:56:58.703574+0000","last_active":"2023-10-26T05:56:58.703649+0000","last_peered":"2023-10-26T05:56:58.703649+0000","last_clean":"2023-10-26T05:56:58.703649+0000","last_became_active":"2023-10-26T05:56:58.703256+0000","last_became_peered":"2023-10-26T05:56:58.703256+0000","last_unstale":"2023-10-26T05:56:58.703649+0000","last_undegraded":"2023-10-26T05:56:58.703649+0000","last_fullsized":"2023-10-26T05:56:58.703649+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:00:26.907325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.695608+0000","last_change":"2023-10-26T05:56:58.695608+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.695608+0000","last_undegraded":"2023-10-26T05:56:58.695608+0000","last_fullsized":"2023-10-26T05:56:58.695608+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708221+0000","last_change":"2023-10-26T05:56:58.707861+0000","last_active":"2023-10-26T05:56:58.708221+0000","last_peered":"2023-10-26T05:56:58.708221+0000","last_clean":"2023-10-26T05:56:58.708221+0000","last_became_active":"2023-10-26T05:56:58.707069+0000","last_became_peered":"2023-10-26T05:56:58.707069+0000","last_unstale":"2023-10-26T05:56:58.708221+0000","last_undegraded":"2023-10-26T05:56:58.708221+0000","last_fullsized":"2023-10-26T05:56:58.708221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:53:39.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681733+0000","last_change":"2023-10-26T05:56:58.681733+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681733+0000","last_undegraded":"2023-10-26T05:56:58.681733+0000","last_fullsized":"2023-10-26T05:56:58.681733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699326+0000","last_change":"2023-10-26T05:56:58.699246+0000","last_active":"2023-10-26T05:56:58.699326+0000","last_peered":"2023-10-26T05:56:58.699326+0000","last_clean":"2023-10-26T05:56:58.699326+0000","last_became_active":"2023-10-26T05:56:58.698939+0000","last_became_peered":"2023-10-26T05:56:58.698939+0000","last_unstale":"2023-10-26T05:56:58.699326+0000","last_undegraded":"2023-10-26T05:56:58.699326+0000","last_fullsized":"2023-10-26T05:56:58.699326+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:42:17.788046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.690498+0000","last_change":"2023-10-26T05:56:58.690498+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.690498+0000","last_undegraded":"2023-10-26T05:56:58.690498+0000","last_fullsized":"2023-10-26T05:56:58.690498+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.696872+0000","last_change":"2023-10-26T05:56:58.696783+0000","last_active":"2023-10-26T05:56:58.696872+0000","last_peered":"2023-10-26T05:56:58.696872+0000","last_clean":"2023-10-26T05:56:58.696872+0000","last_became_active":"2023-10-26T05:56:58.696463+0000","last_became_peered":"2023-10-26T05:56:58.696463+0000","last_unstale":"2023-10-26T05:56:58.696872+0000","last_undegraded":"2023-10-26T05:56:58.696872+0000","last_fullsized":"2023-10-26T05:56:58.696872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:03:26.952948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685813+0000","last_change":"2023-10-26T05:56:58.685813+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685813+0000","last_undegraded":"2023-10-26T05:56:58.685813+0000","last_fullsized":"2023-10-26T05:56:58.685813+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.694434+0000","last_change":"2023-10-26T05:56:58.694349+0000","last_active":"2023-10-26T05:56:58.694434+0000","last_peered":"2023-10-26T05:56:58.694434+0000","last_clean":"2023-10-26T05:56:58.694434+0000","last_became_active":"2023-10-26T05:56:58.694033+0000","last_became_peered":"2023-10-26T05:56:58.694033+0000","last_unstale":"2023-10-26T05:56:58.694434+0000","last_undegraded":"2023-10-26T05:56:58.694434+0000","last_fullsized":"2023-10-26T05:56:58.694434+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:30:52.321543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684950+0000","last_change":"2023-10-26T05:56:58.684950+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684950+0000","last_undegraded":"2023-10-26T05:56:58.684950+0000","last_fullsized":"2023-10-26T05:56:58.684950+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.703746+0000","last_change":"2023-10-26T05:56:58.703649+0000","last_active":"2023-10-26T05:56:58.703746+0000","last_peered":"2023-10-26T05:56:58.703746+0000","last_clean":"2023-10-26T05:56:58.703746+0000","last_became_active":"2023-10-26T05:56:58.703345+0000","last_became_peered":"2023-10-26T05:56:58.703345+0000","last_unstale":"2023-10-26T05:56:58.703746+0000","last_undegraded":"2023-10-26T05:56:58.703746+0000","last_fullsized":"2023-10-26T05:56:58.703746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:04.454516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685015+0000","last_change":"2023-10-26T05:56:58.685015+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685015+0000","last_undegraded":"2023-10-26T05:56:58.685015+0000","last_fullsized":"2023-10-26T05:56:58.685015+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713126+0000","last_change":"2023-10-26T05:56:58.712925+0000","last_active":"2023-10-26T05:56:58.713126+0000","last_peered":"2023-10-26T05:56:58.713126+0000","last_clean":"2023-10-26T05:56:58.713126+0000","last_became_active":"2023-10-26T05:56:58.712384+0000","last_became_peered":"2023-10-26T05:56:58.712384+0000","last_unstale":"2023-10-26T05:56:58.713126+0000","last_undegraded":"2023-10-26T05:56:58.713126+0000","last_fullsized":"2023-10-26T05:56:58.713126+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:18.155446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685000+0000","last_change":"2023-10-26T05:56:58.685000+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685000+0000","last_undegraded":"2023-10-26T05:56:58.685000+0000","last_fullsized":"2023-10-26T05:56:58.685000+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.713004+0000","last_change":"2023-10-26T05:56:58.712807+0000","last_active":"2023-10-26T05:56:58.713004+0000","last_peered":"2023-10-26T05:56:58.713004+0000","last_clean":"2023-10-26T05:56:58.713004+0000","last_became_active":"2023-10-26T05:56:58.707471+0000","last_became_peered":"2023-10-26T05:56:58.707471+0000","last_unstale":"2023-10-26T05:56:58.713004+0000","last_undegraded":"2023-10-26T05:56:58.713004+0000","last_fullsized":"2023-10-26T05:56:58.713004+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:13:10.254070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.684550+0000","last_change":"2023-10-26T05:56:58.684550+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.684550+0000","last_undegraded":"2023-10-26T05:56:58.684550+0000","last_fullsized":"2023-10-26T05:56:58.684550+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.708819+0000","last_change":"2023-10-26T05:56:58.707193+0000","last_active":"2023-10-26T05:56:58.708819+0000","last_peered":"2023-10-26T05:56:58.708819+0000","last_clean":"2023-10-26T05:56:58.708819+0000","last_became_active":"2023-10-26T05:56:58.705895+0000","last_became_peered":"2023-10-26T05:56:58.705895+0000","last_unstale":"2023-10-26T05:56:58.708819+0000","last_undegraded":"2023-10-26T05:56:58.708819+0000","last_fullsized":"2023-10-26T05:56:58.708819+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:33:43.161653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.694164+0000","last_change":"2023-10-26T05:56:58.694164+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.694164+0000","last_undegraded":"2023-10-26T05:56:58.694164+0000","last_fullsized":"2023-10-26T05:56:58.694164+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.717688+0000","last_change":"2023-10-26T05:56:58.717538+0000","last_active":"2023-10-26T05:56:58.717688+0000","last_peered":"2023-10-26T05:56:58.717688+0000","last_clean":"2023-10-26T05:56:58.717688+0000","last_became_active":"2023-10-26T05:56:58.716243+0000","last_became_peered":"2023-10-26T05:56:58.716243+0000","last_unstale":"2023-10-26T05:56:58.717688+0000","last_undegraded":"2023-10-26T05:56:58.717688+0000","last_fullsized":"2023-10-26T05:56:58.717688+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:55:25.695338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.689729+0000","last_change":"2023-10-26T05:56:58.689729+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.689729+0000","last_undegraded":"2023-10-26T05:56:58.689729+0000","last_fullsized":"2023-10-26T05:56:58.689729+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.697371+0000","last_change":"2023-10-26T05:56:58.697256+0000","last_active":"2023-10-26T05:56:58.697371+0000","last_peered":"2023-10-26T05:56:58.697371+0000","last_clean":"2023-10-26T05:56:58.697371+0000","last_became_active":"2023-10-26T05:56:58.696984+0000","last_became_peered":"2023-10-26T05:56:58.696984+0000","last_unstale":"2023-10-26T05:56:58.697371+0000","last_undegraded":"2023-10-26T05:56:58.697371+0000","last_fullsized":"2023-10-26T05:56:58.697371+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:31:47.521770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.687331+0000","last_change":"2023-10-26T05:56:58.687331+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.687331+0000","last_undegraded":"2023-10-26T05:56:58.687331+0000","last_fullsized":"2023-10-26T05:56:58.687331+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.687335+0000","last_change":"2023-10-26T05:56:58.687247+0000","last_active":"2023-10-26T05:56:58.687335+0000","last_peered":"2023-10-26T05:56:58.687335+0000","last_clean":"2023-10-26T05:56:58.687335+0000","last_became_active":"2023-10-26T05:56:58.686900+0000","last_became_peered":"2023-10-26T05:56:58.686900+0000","last_unstale":"2023-10-26T05:56:58.687335+0000","last_undegraded":"2023-10-26T05:56:58.687335+0000","last_fullsized":"2023-10-26T05:56:58.687335+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:57:16.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685469+0000","last_change":"2023-10-26T05:56:58.685469+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685469+0000","last_undegraded":"2023-10-26T05:56:58.685469+0000","last_fullsized":"2023-10-26T05:56:58.685469+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.688086+0000","last_change":"2023-10-26T05:56:58.687941+0000","last_active":"2023-10-26T05:56:58.688086+0000","last_peered":"2023-10-26T05:56:58.688086+0000","last_clean":"2023-10-26T05:56:58.688086+0000","last_became_active":"2023-10-26T05:56:58.687118+0000","last_became_peered":"2023-10-26T05:56:58.687118+0000","last_unstale":"2023-10-26T05:56:58.688086+0000","last_undegraded":"2023-10-26T05:56:58.688086+0000","last_fullsized":"2023-10-26T05:56:58.688086+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:32:10.994025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.681960+0000","last_change":"2023-10-26T05:56:58.681960+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.681960+0000","last_undegraded":"2023-10-26T05:56:58.681960+0000","last_fullsized":"2023-10-26T05:56:58.681960+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.692825+0000","last_change":"2023-10-26T05:56:58.692632+0000","last_active":"2023-10-26T05:56:58.692825+0000","last_peered":"2023-10-26T05:56:58.692825+0000","last_clean":"2023-10-26T05:56:58.692825+0000","last_became_active":"2023-10-26T05:56:58.690733+0000","last_became_peered":"2023-10-26T05:56:58.690733+0000","last_unstale":"2023-10-26T05:56:58.692825+0000","last_undegraded":"2023-10-26T05:56:58.692825+0000","last_fullsized":"2023-10-26T05:56:58.692825+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:39:59.579324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.691809+0000","last_change":"2023-10-26T05:56:58.691809+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.691809+0000","last_undegraded":"2023-10-26T05:56:58.691809+0000","last_fullsized":"2023-10-26T05:56:58.691809+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.718038+0000","last_change":"2023-10-26T05:56:58.717853+0000","last_active":"2023-10-26T05:56:58.718038+0000","last_peered":"2023-10-26T05:56:58.718038+0000","last_clean":"2023-10-26T05:56:58.718038+0000","last_became_active":"2023-10-26T05:56:58.716922+0000","last_became_peered":"2023-10-26T05:56:58.716922+0000","last_unstale":"2023-10-26T05:56:58.718038+0000","last_undegraded":"2023-10-26T05:56:58.718038+0000","last_fullsized":"2023-10-26T05:56:58.718038+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:59:48.203254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.692948+0000","last_change":"2023-10-26T05:56:58.692948+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.692948+0000","last_undegraded":"2023-10-26T05:56:58.692948+0000","last_fullsized":"2023-10-26T05:56:58.692948+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711684+0000","last_change":"2023-10-26T05:56:58.709784+0000","last_active":"2023-10-26T05:56:58.711684+0000","last_peered":"2023-10-26T05:56:58.711684+0000","last_clean":"2023-10-26T05:56:58.711684+0000","last_became_active":"2023-10-26T05:56:58.707275+0000","last_became_peered":"2023-10-26T05:56:58.707275+0000","last_unstale":"2023-10-26T05:56:58.711684+0000","last_undegraded":"2023-10-26T05:56:58.711684+0000","last_fullsized":"2023-10-26T05:56:58.711684+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:25:35.872395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.693336+0000","last_change":"2023-10-26T05:56:58.693336+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.693336+0000","last_undegraded":"2023-10-26T05:56:58.693336+0000","last_fullsized":"2023-10-26T05:56:58.693336+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.699845+0000","last_change":"2023-10-26T05:56:58.699782+0000","last_active":"2023-10-26T05:56:58.699845+0000","last_peered":"2023-10-26T05:56:58.699845+0000","last_clean":"2023-10-26T05:56:58.699845+0000","last_became_active":"2023-10-26T05:56:58.699054+0000","last_became_peered":"2023-10-26T05:56:58.699054+0000","last_unstale":"2023-10-26T05:56:58.699845+0000","last_undegraded":"2023-10-26T05:56:58.699845+0000","last_fullsized":"2023-10-26T05:56:58.699845+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:06:55.048724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":170,"state":"creating+peering","last_fresh":"2023-10-26T05:56:58.685430+0000","last_change":"2023-10-26T05:56:58.685430+0000","last_active":"2023-10-26T05:56:58.661153+0000","last_peered":"2023-10-26T05:56:58.661153+0000","last_clean":"2023-10-26T05:56:58.661153+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-26T05:56:58.685430+0000","last_undegraded":"2023-10-26T05:56:58.685430+0000","last_fullsized":"2023-10-26T05:56:58.685430+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-26T05:56:58.711741+0000","last_change":"2023-10-26T05:56:58.710168+0000","last_active":"2023-10-26T05:56:58.711741+0000","last_peered":"2023-10-26T05:56:58.711741+0000","last_clean":"2023-10-26T05:56:58.711741+0000","last_became_active":"2023-10-26T05:56:58.708495+0000","last_became_peered":"2023-10-26T05:56:58.708495+0000","last_unstale":"2023-10-26T05:56:58.711741+0000","last_undegraded":"2023-10-26T05:56:58.711741+0000","last_fullsized":"2023-10-26T05:56:58.711741+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:29:24.511118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":304,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-26T05:56:57.680722+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:56:57.680722+0000","last_peered":"2023-10-26T05:56:57.680722+0000","last_clean":"2023-10-26T05:56:57.680722+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:56:57.680722+0000","last_undegraded":"2023-10-26T05:56:57.680722+0000","last_fullsized":"2023-10-26T05:56:57.680722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.59099999999999997,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3100000000000001},"last":1.1899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":1.0029999999999999},"min":{"1min":0.64700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.232,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.032,"15min":0.98799999999999999},"min":{"1min":0.97999999999999998,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.139}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.68000000000000005,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1899999999999999,"5min":1.448,"15min":1.448},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93799999999999994,"15min":0.93400000000000005},"min":{"1min":0.63900000000000001,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1990000000000001,"5min":1.3700000000000001,"15min":1.4590000000000001},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93200000000000005,"15min":0.89800000000000002},"min":{"1min":0.76000000000000001,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":1.004},{"interface":"front","average":{"1min":1.002,"5min":0.96699999999999997,"15min":0.97199999999999998},"min":{"1min":0.77300000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3089999999999999,"5min":1.3129999999999999,"15min":1.375},"last":0.97699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.133,"5min":1.639,"15min":1.639},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.93300000000000005,"15min":0.93999999999999995},"min":{"1min":0.65700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.24,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.337,"5min":1.482,"15min":1.6679999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0369999999999999,"15min":1.0640000000000001},"min":{"1min":0.72299999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.702,"15min":1.702},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.115,"15min":1.1060000000000001},"min":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.3420000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.5,"15min":1.6180000000000001},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.234,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.087},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.69999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.86599999999999999},"min":{"1min":0.64500000000000002,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.385,"5min":1.3899999999999999,"15min":1.405},"last":0.95499999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.91000000000000003,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2450000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.2809999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0529999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97999999999999998,"15min":0.92500000000000004},"min":{"1min":0.76900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.3080000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.0620000000000001,"15min":1.0009999999999999},"min":{"1min":0.69099999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.337,"5min":1.3759999999999999,"15min":1.399},"last":1.212}]},{"osd":4,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.503,"15min":0.502},"max":{"1min":1.2569999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.99299999999999999,"15min":0.98099999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.111}]},{"osd":5,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.05},"min":{"1min":0.89200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.236,"5min":1.5509999999999999,"15min":1.554},"last":1.028},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97199999999999998,"15min":0.98299999999999998},"min":{"1min":0.67200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.452,"5min":1.512,"15min":1.512},"last":0.81999999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1220000000000001,"15min":1.1120000000000001},"min":{"1min":0.78400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.321,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94499999999999995,"15min":0.95599999999999996},"min":{"1min":0.63400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.579},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.83599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":1.591,"15min":1.591},"last":1.3400000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84699999999999998,"15min":0.878},"min":{"1min":0.625,"5min":0.53300000000000003,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.206,"15min":2.1150000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86799999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2430000000000001,"5min":1.272,"15min":1.464},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.70599999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.2170000000000001,"5min":1.569,"15min":1.569},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0640000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90900000000000003,"15min":0.91800000000000004},"min":{"1min":0.69799999999999995,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.363,"5min":1.448,"15min":1.548},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.78200000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.276,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2410000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.97599999999999998},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.286,"15min":2.1909999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0760000000000001,"15min":1.091},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.536,"15min":1.536},"last":1.194}]},{"osd":6,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.68000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.268,"5min":1.744,"15min":1.744},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.04,"15min":1.032},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":1.4279999999999999,"15min":1.4950000000000001},"last":1.161}]},{"osd":7,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.071},"min":{"1min":0.64100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.3109999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.278},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3360000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.92000000000000004,"15min":0.871},"min":{"1min":0.66200000000000003,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1919999999999999,"5min":1.2569999999999999,"15min":1.347},"last":1.075},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.70799999999999996,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.159,"5min":1.502,"15min":1.502},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.90400000000000003,"15min":0.84499999999999997},"min":{"1min":0.749,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.21,"5min":1.25,"15min":1.375},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.96099999999999997,"15min":0.91700000000000004},"min":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":1.198}]},{"osd":2,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.143},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90700000000000003,"15min":0.879},"min":{"1min":0.72899999999999998,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.143,"5min":1.2170000000000001,"15min":1.373},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":0.95199999999999996},"min":{"1min":0.79700000000000004,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.177,"5min":1.397,"15min":2.0190000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.091,"5min":0.995,"15min":0.96199999999999997},"min":{"1min":0.79100000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.28,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.2210000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.2430000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0009999999999999,"15min":1},"min":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.2929999999999999,"5min":1.4179999999999999,"15min":1.673},"last":0.99399999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0429999999999999,"15min":1.014},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.42099999999999999},"max":{"1min":1.337,"5min":1.762,"15min":1.7969999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.054,"15min":1.042},"min":{"1min":0.76500000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.4970000000000001},"last":1.1759999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.103,"15min":1.0589999999999999},"min":{"1min":0.77300000000000002,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.3280000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.879,"5min":0.69599999999999995,"15min":0.53700000000000003},"max":{"1min":1.341,"5min":1.484,"15min":1.728},"last":1.2709999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.90100000000000002,"5min":1.385,"15min":1.385},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.52500000000000002,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.2689999999999999},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":1.091,"5min":1.2,"15min":1.349},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.38,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.02,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":1.018}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.752,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1479999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81999999999999995,"15min":0.78700000000000003},"min":{"1min":0.623,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.1850000000000001,"15min":1.4199999999999999},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.84399999999999997},"min":{"1min":0.70799999999999996,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.1120000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90200000000000002,"15min":0.91900000000000004},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.2789999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.72399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3,"5min":1.413,"15min":1.478},"last":1.454},{"interface":"front","average":{"1min":1.014,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.54,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.64200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.349,"15min":1.6060000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.091,"5min":0.96699999999999997,"15min":1.014},"min":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":1.359,"15min":5.6859999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.98599999999999999,"15min":0.94299999999999995},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.53400000000000003},"max":{"1min":1.472,"5min":2.016,"15min":2.016},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.008,"15min":1.032},"min":{"1min":0.68799999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.327,"5min":2.004,"15min":2.004},"last":1.2370000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":924,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194387,"statfs":{"total":200000000,"available":199052975,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71999999999999997,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.379,"15min":1.5249999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.69599999999999995},"min":{"1min":0.61699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.091,"5min":1.226,"15min":1.2609999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.621},"min":{"1min":0.57799999999999996,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.90700000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.82199999999999995,"15min":0.76500000000000001},"min":{"1min":0.71599999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.24,"15min":2.8809999999999998},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94899999999999995,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.88100000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93000000000000005,"15min":0.85999999999999999},"min":{"1min":0.69099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.151,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88500000000000001,"15min":0.85899999999999999},"min":{"1min":0.78200000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":1.0269999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.997,"15min":0.93899999999999995},"min":{"1min":0.84999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.999,"15min":0.93100000000000005},"min":{"1min":0.80000000000000004,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.329,"5min":1.4790000000000001,"15min":1.653},"last":1.0569999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.008,"15min":0.94699999999999995},"min":{"1min":0.73199999999999998,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.3320000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.089,"15min":1.016},"min":{"1min":0.76500000000000001,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.3300000000000001,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.177}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":0.91900000000000004},"min":{"1min":0.84299999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.073,"5min":1.581,"15min":1.581},"last":1.119},{"interface":"front","average":{"1min":1.133,"5min":1.109,"15min":1.069},"min":{"1min":0.91200000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.462,"5min":1.823,"15min":15.352},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194381,"statfs":{"total":200000000,"available":199046700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.157},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55600000000000005,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78300000000000003,"15min":0.73999999999999999},"min":{"1min":0.63700000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.073,"5min":1.6080000000000001,"15min":1.8169999999999999},"last":0.92700000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.754,"15min":0.76100000000000001},"min":{"1min":0.65600000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.147,"15min":1.7949999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.0760000000000001,"5min":1.298,"15min":1.298},"last":0.997}]},{"osd":4,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.874},"min":{"1min":0.73899999999999999,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1020000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.74099999999999999,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.345,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":1.2}]},{"osd":5,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.96299999999999997,"15min":0.96399999999999997},"min":{"1min":0.749,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.319,"5min":1.381,"15min":1.492},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.84899999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.2509999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.96999999999999997}]},{"osd":6,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0249999999999999},"min":{"1min":0.89200000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89700000000000002,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.2669999999999999,"5min":1.468,"15min":1.837},"last":1.155}]},{"osd":7,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.046,"15min":1.0309999999999999},"min":{"1min":0.82499999999999996,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.397,"5min":1.4179999999999999,"15min":1.6339999999999999},"last":1.294},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0640000000000001,"15min":1.0840000000000001},"min":{"1min":0.93100000000000005,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4890000000000001,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":1.0600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1507,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193805,"statfs":{"total":200000000,"available":198456332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95599999999999996,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.39100000000000001,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.1020000000000001,"15min":1.177},"last":0.502}]},{"osd":3,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0249999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.72899999999999998},"min":{"1min":0.58999999999999997,"5min":0.32100000000000001,"15min":0.247},"max":{"1min":1.026,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91400000000000003,"15min":0.873},"min":{"1min":0.64200000000000002,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.319,"5min":1.395,"15min":1.395},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":0.93500000000000005},"min":{"1min":0.73699999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":1.1559999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.95099999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.92200000000000004},"min":{"1min":0.61299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":2.7480000000000002},"last":1.073},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96299999999999997,"15min":0.92100000000000004},"min":{"1min":0.87,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.91900000000000004},"min":{"1min":0.77500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1639999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.131,"15min":1.0169999999999999},"min":{"1min":0.871,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.2829999999999999,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0109999999999999,"15min":0.998},"min":{"1min":0.74199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1539999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0329999999999999,"15min":0.999},"min":{"1min":0.93500000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.417,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":1.141}]}]}],"pool_statfs":[]}} 2023-10-26T05:57:04.650 INFO:tasks.ceph.mds.a.smithi071.stderr:2023-10-26T05:57:04.643+0000 7f8d197f6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:57:07.140 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:57:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 -- 172.21.15.71:0/375806916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7c005190 msgr2=0x7fbe7c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 --2- 172.21.15.71:0/375806916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7c005190 0x7fbe7c00b990 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c001da0 tx=0x7fbe8c01cb20 comp rx=0 tx=0).stop 2023-10-26T05:57:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 -- 172.21.15.71:0/375806916 shutdown_connections 2023-10-26T05:57:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 --2- 172.21.15.71:0/375806916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe7c005190 0x7fbe7c00b990 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 -- 172.21.15.71:0/375806916 >> 172.21.15.71:0/375806916 conn(0x7fbe94069770 msgr2=0x7fbe94069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:07.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 -- 172.21.15.71:0/375806916 shutdown_connections 2023-10-26T05:57:07.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 -- 172.21.15.71:0/375806916 wait complete. 2023-10-26T05:57:07.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 Processor -- start 2023-10-26T05:57:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.267+0000 7fbe9a902640 1 -- start start 2023-10-26T05:57:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe9a902640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c005190 0x7fbe9413ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe9a902640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 0x7fbe94136f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe9a902640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe94137470 0x7fbe94151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe9a902640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941f46e0 con 0x7fbe9413b380 2023-10-26T05:57:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe9a902640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9413b800 con 0x7fbe7c005190 2023-10-26T05:57:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe9a902640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94152dd0 con 0x7fbe94137470 2023-10-26T05:57:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 0x7fbe94136f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 0x7fbe94136f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35832/0 (socket says 172.21.15.71:35832) 2023-10-26T05:57:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 -- 172.21.15.71:0/1102135765 learned_addr learned my addr 172.21.15.71:0/1102135765 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe93fff640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c005190 0x7fbe9413ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe98e9f640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe94137470 0x7fbe94151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 -- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe94137470 msgr2=0x7fbe94151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe94137470 0x7fbe94151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 -- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c005190 msgr2=0x7fbe9413ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c005190 0x7fbe9413ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 -- 172.21.15.71:0/1102135765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe941530b0 con 0x7fbe9413b380 2023-10-26T05:57:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe937fe640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 0x7fbe94136f30 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbe88001ec0 tx=0x7fbe8800ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88010890 con 0x7fbe9413b380 2023-10-26T05:57:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.271+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbe88010ae0 con 0x7fbe9413b380 2023-10-26T05:57:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011460 con 0x7fbe9413b380 2023-10-26T05:57:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe94153390 con 0x7fbe9413b380 2023-10-26T05:57:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941f2510 con 0x7fbe9413b380 2023-10-26T05:57:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe94066b10 con 0x7fbe9413b380 2023-10-26T05:57:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe880115c0 con 0x7fbe9413b380 2023-10-26T05:57:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe90ff9640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe800664a0 0x7fbe80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe88084570 con 0x7fbe9413b380 2023-10-26T05:57:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe93fff640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe800664a0 0x7fbe80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.275+0000 7fbe93fff640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe800664a0 0x7fbe80068960 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbe9406a870 tx=0x7fbe8400f4d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:07.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.279+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbe880519f0 con 0x7fbe9413b380 2023-10-26T05:57:07.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.427+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe94070b90 con 0x7fbe800664a0 2023-10-26T05:57:07.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe90ff9640 1 -- 172.21.15.71:0/1102135765 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283714 (secure 0 0 0) 0x7fbe84016070 con 0x7fbe800664a0 2023-10-26T05:57:07.458 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:07.460 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:57:07.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe800664a0 msgr2=0x7fbe80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe800664a0 0x7fbe80068960 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbe9406a870 tx=0x7fbe8400f4d0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 msgr2=0x7fbe94136f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 0x7fbe94136f30 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbe88001ec0 tx=0x7fbe8800ea20 comp rx=0 tx=0).stop 2023-10-26T05:57:07.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 shutdown_connections 2023-10-26T05:57:07.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe94137470 0x7fbe94151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbe800664a0 0x7fbe80068960 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe9413b380 0x7fbe94136f30 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 --2- 172.21.15.71:0/1102135765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe7c005190 0x7fbe9413ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 >> 172.21.15.71:0/1102135765 conn(0x7fbe94069770 msgr2=0x7fbe94158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:07.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 shutdown_connections 2023-10-26T05:57:07.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.451+0000 7fbe9a902640 1 -- 172.21.15.71:0/1102135765 wait complete. 2023-10-26T05:57:07.475 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":462,"stamp":"2023-10-26T05:57:05.608688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8707,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553792,"statfs":{"total":1600000000,"available":1591083444,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.922801"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.683010+0000","last_change":"2023-10-26T05:56:59.695586+0000","last_active":"2023-10-26T05:57:00.683010+0000","last_peered":"2023-10-26T05:57:00.683010+0000","last_clean":"2023-10-26T05:57:00.683010+0000","last_became_active":"2023-10-26T05:56:59.695072+0000","last_became_peered":"2023-10-26T05:56:59.695072+0000","last_unstale":"2023-10-26T05:57:00.683010+0000","last_undegraded":"2023-10-26T05:57:00.683010+0000","last_fullsized":"2023-10-26T05:57:00.683010+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:45:29.431930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.683722+0000","last_change":"2023-10-26T05:56:59.689267+0000","last_active":"2023-10-26T05:57:00.683722+0000","last_peered":"2023-10-26T05:57:00.683722+0000","last_clean":"2023-10-26T05:57:00.683722+0000","last_became_active":"2023-10-26T05:56:59.688995+0000","last_became_peered":"2023-10-26T05:56:59.688995+0000","last_unstale":"2023-10-26T05:57:00.683722+0000","last_undegraded":"2023-10-26T05:57:00.683722+0000","last_fullsized":"2023-10-26T05:57:00.683722+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:55:26.709142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.819612+0000","last_change":"2023-10-26T05:56:59.741050+0000","last_active":"2023-10-26T05:57:00.819612+0000","last_peered":"2023-10-26T05:57:00.819612+0000","last_clean":"2023-10-26T05:57:00.819612+0000","last_became_active":"2023-10-26T05:56:59.738991+0000","last_became_peered":"2023-10-26T05:56:59.738991+0000","last_unstale":"2023-10-26T05:57:00.819612+0000","last_undegraded":"2023-10-26T05:57:00.819612+0000","last_fullsized":"2023-10-26T05:57:00.819612+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:11.384472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691367+0000","last_change":"2023-10-26T05:56:59.718163+0000","last_active":"2023-10-26T05:57:00.691367+0000","last_peered":"2023-10-26T05:57:00.691367+0000","last_clean":"2023-10-26T05:57:00.691367+0000","last_became_active":"2023-10-26T05:56:59.717612+0000","last_became_peered":"2023-10-26T05:56:59.717612+0000","last_unstale":"2023-10-26T05:57:00.691367+0000","last_undegraded":"2023-10-26T05:57:00.691367+0000","last_fullsized":"2023-10-26T05:57:00.691367+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:05.468320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898616+0000","last_change":"2023-10-26T05:56:59.727490+0000","last_active":"2023-10-26T05:57:00.898616+0000","last_peered":"2023-10-26T05:57:00.898616+0000","last_clean":"2023-10-26T05:57:00.898616+0000","last_became_active":"2023-10-26T05:56:59.726649+0000","last_became_peered":"2023-10-26T05:56:59.726649+0000","last_unstale":"2023-10-26T05:57:00.898616+0000","last_undegraded":"2023-10-26T05:57:00.898616+0000","last_fullsized":"2023-10-26T05:57:00.898616+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:57:17.044460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684775+0000","last_change":"2023-10-26T05:56:59.714952+0000","last_active":"2023-10-26T05:57:00.684775+0000","last_peered":"2023-10-26T05:57:00.684775+0000","last_clean":"2023-10-26T05:57:00.684775+0000","last_became_active":"2023-10-26T05:56:59.714698+0000","last_became_peered":"2023-10-26T05:56:59.714698+0000","last_unstale":"2023-10-26T05:57:00.684775+0000","last_undegraded":"2023-10-26T05:57:00.684775+0000","last_fullsized":"2023-10-26T05:57:00.684775+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:51:14.436048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.889785+0000","last_change":"2023-10-26T05:56:59.705229+0000","last_active":"2023-10-26T05:57:00.889785+0000","last_peered":"2023-10-26T05:57:00.889785+0000","last_clean":"2023-10-26T05:57:00.889785+0000","last_became_active":"2023-10-26T05:56:59.705018+0000","last_became_peered":"2023-10-26T05:56:59.705018+0000","last_unstale":"2023-10-26T05:57:00.889785+0000","last_undegraded":"2023-10-26T05:57:00.889785+0000","last_fullsized":"2023-10-26T05:57:00.889785+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:02.537459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.682790+0000","last_change":"2023-10-26T05:56:59.708475+0000","last_active":"2023-10-26T05:57:00.682790+0000","last_peered":"2023-10-26T05:57:00.682790+0000","last_clean":"2023-10-26T05:57:00.682790+0000","last_became_active":"2023-10-26T05:56:59.708235+0000","last_became_peered":"2023-10-26T05:56:59.708235+0000","last_unstale":"2023-10-26T05:57:00.682790+0000","last_undegraded":"2023-10-26T05:57:00.682790+0000","last_fullsized":"2023-10-26T05:57:00.682790+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:33:08.566451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.683597+0000","last_change":"2023-10-26T05:56:59.697985+0000","last_active":"2023-10-26T05:57:00.683597+0000","last_peered":"2023-10-26T05:57:00.683597+0000","last_clean":"2023-10-26T05:57:00.683597+0000","last_became_active":"2023-10-26T05:56:59.697588+0000","last_became_peered":"2023-10-26T05:56:59.697588+0000","last_unstale":"2023-10-26T05:57:00.683597+0000","last_undegraded":"2023-10-26T05:57:00.683597+0000","last_fullsized":"2023-10-26T05:57:00.683597+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:51.448654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.899012+0000","last_change":"2023-10-26T05:56:59.733729+0000","last_active":"2023-10-26T05:57:00.899012+0000","last_peered":"2023-10-26T05:57:00.899012+0000","last_clean":"2023-10-26T05:57:00.899012+0000","last_became_active":"2023-10-26T05:56:59.725741+0000","last_became_peered":"2023-10-26T05:56:59.725741+0000","last_unstale":"2023-10-26T05:57:00.899012+0000","last_undegraded":"2023-10-26T05:57:00.899012+0000","last_fullsized":"2023-10-26T05:57:00.899012+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:27:34.270062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.683347+0000","last_change":"2023-10-26T05:56:59.708519+0000","last_active":"2023-10-26T05:57:00.683347+0000","last_peered":"2023-10-26T05:57:00.683347+0000","last_clean":"2023-10-26T05:57:00.683347+0000","last_became_active":"2023-10-26T05:56:59.706691+0000","last_became_peered":"2023-10-26T05:56:59.706691+0000","last_unstale":"2023-10-26T05:57:00.683347+0000","last_undegraded":"2023-10-26T05:57:00.683347+0000","last_fullsized":"2023-10-26T05:57:00.683347+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:25:40.723385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898558+0000","last_change":"2023-10-26T05:56:59.725168+0000","last_active":"2023-10-26T05:57:00.898558+0000","last_peered":"2023-10-26T05:57:00.898558+0000","last_clean":"2023-10-26T05:57:00.898558+0000","last_became_active":"2023-10-26T05:56:59.724537+0000","last_became_peered":"2023-10-26T05:56:59.724537+0000","last_unstale":"2023-10-26T05:57:00.898558+0000","last_undegraded":"2023-10-26T05:57:00.898558+0000","last_fullsized":"2023-10-26T05:57:00.898558+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:44:02.537459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.897413+0000","last_change":"2023-10-26T05:56:59.735429+0000","last_active":"2023-10-26T05:57:00.897413+0000","last_peered":"2023-10-26T05:57:00.897413+0000","last_clean":"2023-10-26T05:57:00.897413+0000","last_became_active":"2023-10-26T05:56:59.734515+0000","last_became_peered":"2023-10-26T05:56:59.734515+0000","last_unstale":"2023-10-26T05:57:00.897413+0000","last_undegraded":"2023-10-26T05:57:00.897413+0000","last_fullsized":"2023-10-26T05:57:00.897413+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:02:15.267781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691701+0000","last_change":"2023-10-26T05:56:59.710568+0000","last_active":"2023-10-26T05:57:00.691701+0000","last_peered":"2023-10-26T05:57:00.691701+0000","last_clean":"2023-10-26T05:57:00.691701+0000","last_became_active":"2023-10-26T05:56:59.709847+0000","last_became_peered":"2023-10-26T05:56:59.709847+0000","last_unstale":"2023-10-26T05:57:00.691701+0000","last_undegraded":"2023-10-26T05:57:00.691701+0000","last_fullsized":"2023-10-26T05:57:00.691701+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:40:00.593128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.899226+0000","last_change":"2023-10-26T05:56:59.734513+0000","last_active":"2023-10-26T05:57:00.899226+0000","last_peered":"2023-10-26T05:57:00.899226+0000","last_clean":"2023-10-26T05:57:00.899226+0000","last_became_active":"2023-10-26T05:56:59.732329+0000","last_became_peered":"2023-10-26T05:56:59.732329+0000","last_unstale":"2023-10-26T05:57:00.899226+0000","last_undegraded":"2023-10-26T05:57:00.899226+0000","last_fullsized":"2023-10-26T05:57:00.899226+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:48:22.990108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.191185+0000","last_change":"2023-10-26T05:56:59.706220+0000","last_active":"2023-10-26T05:57:01.191185+0000","last_peered":"2023-10-26T05:57:01.191185+0000","last_clean":"2023-10-26T05:57:01.191185+0000","last_became_active":"2023-10-26T05:56:59.705894+0000","last_became_peered":"2023-10-26T05:56:59.705894+0000","last_unstale":"2023-10-26T05:57:01.191185+0000","last_undegraded":"2023-10-26T05:57:01.191185+0000","last_fullsized":"2023-10-26T05:57:01.191185+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:04.313796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.899297+0000","last_change":"2023-10-26T05:56:59.733946+0000","last_active":"2023-10-26T05:57:00.899297+0000","last_peered":"2023-10-26T05:57:00.899297+0000","last_clean":"2023-10-26T05:57:00.899297+0000","last_became_active":"2023-10-26T05:56:59.732674+0000","last_became_peered":"2023-10-26T05:56:59.732674+0000","last_unstale":"2023-10-26T05:57:00.899297+0000","last_undegraded":"2023-10-26T05:57:00.899297+0000","last_fullsized":"2023-10-26T05:57:00.899297+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:16:04.931031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.819378+0000","last_change":"2023-10-26T05:56:59.741411+0000","last_active":"2023-10-26T05:57:00.819378+0000","last_peered":"2023-10-26T05:57:00.819378+0000","last_clean":"2023-10-26T05:57:00.819378+0000","last_became_active":"2023-10-26T05:56:59.740286+0000","last_became_peered":"2023-10-26T05:56:59.740286+0000","last_unstale":"2023-10-26T05:57:00.819378+0000","last_undegraded":"2023-10-26T05:57:00.819378+0000","last_fullsized":"2023-10-26T05:57:00.819378+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:23.116628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.682991+0000","last_change":"2023-10-26T05:56:59.706329+0000","last_active":"2023-10-26T05:57:00.682991+0000","last_peered":"2023-10-26T05:57:00.682991+0000","last_clean":"2023-10-26T05:57:00.682991+0000","last_became_active":"2023-10-26T05:56:59.705954+0000","last_became_peered":"2023-10-26T05:56:59.705954+0000","last_unstale":"2023-10-26T05:57:00.682991+0000","last_undegraded":"2023-10-26T05:57:00.682991+0000","last_fullsized":"2023-10-26T05:57:00.682991+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:48:51.371914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.191370+0000","last_change":"2023-10-26T05:56:59.704759+0000","last_active":"2023-10-26T05:57:01.191370+0000","last_peered":"2023-10-26T05:57:01.191370+0000","last_clean":"2023-10-26T05:57:01.191370+0000","last_became_active":"2023-10-26T05:56:59.704393+0000","last_became_peered":"2023-10-26T05:56:59.704393+0000","last_unstale":"2023-10-26T05:57:01.191370+0000","last_undegraded":"2023-10-26T05:57:01.191370+0000","last_fullsized":"2023-10-26T05:57:01.191370+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:44.505290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898356+0000","last_change":"2023-10-26T05:56:59.723229+0000","last_active":"2023-10-26T05:57:00.898356+0000","last_peered":"2023-10-26T05:57:00.898356+0000","last_clean":"2023-10-26T05:57:00.898356+0000","last_became_active":"2023-10-26T05:56:59.722598+0000","last_became_peered":"2023-10-26T05:56:59.722598+0000","last_unstale":"2023-10-26T05:57:00.898356+0000","last_undegraded":"2023-10-26T05:57:00.898356+0000","last_fullsized":"2023-10-26T05:57:00.898356+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:57:36.102080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691356+0000","last_change":"2023-10-26T05:56:59.695867+0000","last_active":"2023-10-26T05:57:00.691356+0000","last_peered":"2023-10-26T05:57:00.691356+0000","last_clean":"2023-10-26T05:57:00.691356+0000","last_became_active":"2023-10-26T05:56:59.695266+0000","last_became_peered":"2023-10-26T05:56:59.695266+0000","last_unstale":"2023-10-26T05:57:00.691356+0000","last_undegraded":"2023-10-26T05:57:00.691356+0000","last_fullsized":"2023-10-26T05:57:00.691356+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:07:33.729359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691096+0000","last_change":"2023-10-26T05:56:59.698557+0000","last_active":"2023-10-26T05:57:00.691096+0000","last_peered":"2023-10-26T05:57:00.691096+0000","last_clean":"2023-10-26T05:57:00.691096+0000","last_became_active":"2023-10-26T05:56:59.696539+0000","last_became_peered":"2023-10-26T05:56:59.696539+0000","last_unstale":"2023-10-26T05:57:00.691096+0000","last_undegraded":"2023-10-26T05:57:00.691096+0000","last_fullsized":"2023-10-26T05:57:00.691096+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:34.731361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898343+0000","last_change":"2023-10-26T05:56:59.739638+0000","last_active":"2023-10-26T05:57:00.898343+0000","last_peered":"2023-10-26T05:57:00.898343+0000","last_clean":"2023-10-26T05:57:00.898343+0000","last_became_active":"2023-10-26T05:56:59.734174+0000","last_became_peered":"2023-10-26T05:56:59.734174+0000","last_unstale":"2023-10-26T05:57:00.898343+0000","last_undegraded":"2023-10-26T05:57:00.898343+0000","last_fullsized":"2023-10-26T05:57:00.898343+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:57:20.519958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.191710+0000","last_change":"2023-10-26T05:56:59.706226+0000","last_active":"2023-10-26T05:57:01.191710+0000","last_peered":"2023-10-26T05:57:01.191710+0000","last_clean":"2023-10-26T05:57:01.191710+0000","last_became_active":"2023-10-26T05:56:59.705968+0000","last_became_peered":"2023-10-26T05:56:59.705968+0000","last_unstale":"2023-10-26T05:57:01.191710+0000","last_undegraded":"2023-10-26T05:57:01.191710+0000","last_fullsized":"2023-10-26T05:57:01.191710+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:40:02.906248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.683666+0000","last_change":"2023-10-26T05:56:59.708763+0000","last_active":"2023-10-26T05:57:00.683666+0000","last_peered":"2023-10-26T05:57:00.683666+0000","last_clean":"2023-10-26T05:57:00.683666+0000","last_became_active":"2023-10-26T05:56:59.707524+0000","last_became_peered":"2023-10-26T05:56:59.707524+0000","last_unstale":"2023-10-26T05:57:00.683666+0000","last_undegraded":"2023-10-26T05:57:00.683666+0000","last_fullsized":"2023-10-26T05:57:00.683666+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:10:22.062788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.191733+0000","last_change":"2023-10-26T05:56:59.714329+0000","last_active":"2023-10-26T05:57:01.191733+0000","last_peered":"2023-10-26T05:57:01.191733+0000","last_clean":"2023-10-26T05:57:01.191733+0000","last_became_active":"2023-10-26T05:56:59.714082+0000","last_became_peered":"2023-10-26T05:56:59.714082+0000","last_unstale":"2023-10-26T05:57:01.191733+0000","last_undegraded":"2023-10-26T05:57:01.191733+0000","last_fullsized":"2023-10-26T05:57:01.191733+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:27:35.905438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.889743+0000","last_change":"2023-10-26T05:56:59.734797+0000","last_active":"2023-10-26T05:57:00.889743+0000","last_peered":"2023-10-26T05:57:00.889743+0000","last_clean":"2023-10-26T05:57:00.889743+0000","last_became_active":"2023-10-26T05:56:59.734529+0000","last_became_peered":"2023-10-26T05:56:59.734529+0000","last_unstale":"2023-10-26T05:57:00.889743+0000","last_undegraded":"2023-10-26T05:57:00.889743+0000","last_fullsized":"2023-10-26T05:57:00.889743+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:57:20.519958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.682767+0000","last_change":"2023-10-26T05:56:59.707465+0000","last_active":"2023-10-26T05:57:00.682767+0000","last_peered":"2023-10-26T05:57:00.682767+0000","last_clean":"2023-10-26T05:57:00.682767+0000","last_became_active":"2023-10-26T05:56:59.707211+0000","last_became_peered":"2023-10-26T05:56:59.707211+0000","last_unstale":"2023-10-26T05:57:00.682767+0000","last_undegraded":"2023-10-26T05:57:00.682767+0000","last_fullsized":"2023-10-26T05:57:00.682767+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:18:14.127629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691945+0000","last_change":"2023-10-26T05:56:59.701420+0000","last_active":"2023-10-26T05:57:00.691945+0000","last_peered":"2023-10-26T05:57:00.691945+0000","last_clean":"2023-10-26T05:57:00.691945+0000","last_became_active":"2023-10-26T05:56:59.700558+0000","last_became_peered":"2023-10-26T05:56:59.700558+0000","last_unstale":"2023-10-26T05:57:00.691945+0000","last_undegraded":"2023-10-26T05:57:00.691945+0000","last_fullsized":"2023-10-26T05:57:00.691945+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:47:14.583088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684155+0000","last_change":"2023-10-26T05:56:59.713314+0000","last_active":"2023-10-26T05:57:00.684155+0000","last_peered":"2023-10-26T05:57:00.684155+0000","last_clean":"2023-10-26T05:57:00.684155+0000","last_became_active":"2023-10-26T05:56:59.712892+0000","last_became_peered":"2023-10-26T05:56:59.712892+0000","last_unstale":"2023-10-26T05:57:00.684155+0000","last_undegraded":"2023-10-26T05:57:00.684155+0000","last_fullsized":"2023-10-26T05:57:00.684155+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:59:49.217057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.192034+0000","last_change":"2023-10-26T05:56:59.704002+0000","last_active":"2023-10-26T05:57:01.192034+0000","last_peered":"2023-10-26T05:57:01.192034+0000","last_clean":"2023-10-26T05:57:01.192034+0000","last_became_active":"2023-10-26T05:56:59.703825+0000","last_became_peered":"2023-10-26T05:56:59.703825+0000","last_unstale":"2023-10-26T05:57:01.192034+0000","last_undegraded":"2023-10-26T05:57:01.192034+0000","last_fullsized":"2023-10-26T05:57:01.192034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:02:15.267781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898678+0000","last_change":"2023-10-26T05:56:59.739197+0000","last_active":"2023-10-26T05:57:00.898678+0000","last_peered":"2023-10-26T05:57:00.898678+0000","last_clean":"2023-10-26T05:57:00.898678+0000","last_became_active":"2023-10-26T05:56:59.723984+0000","last_became_peered":"2023-10-26T05:56:59.723984+0000","last_unstale":"2023-10-26T05:57:00.898678+0000","last_undegraded":"2023-10-26T05:57:00.898678+0000","last_fullsized":"2023-10-26T05:57:00.898678+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:10.153259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"171'4","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.739597+0000","last_change":"2023-10-26T05:56:58.711496+0000","last_active":"2023-10-26T05:56:59.739597+0000","last_peered":"2023-10-26T05:56:59.739597+0000","last_clean":"2023-10-26T05:56:59.739597+0000","last_became_active":"2023-10-26T05:56:58.710397+0000","last_became_peered":"2023-10-26T05:56:58.710397+0000","last_unstale":"2023-10-26T05:56:59.739597+0000","last_undegraded":"2023-10-26T05:56:59.739597+0000","last_fullsized":"2023-10-26T05:56:59.739597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:18:02.347115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684404+0000","last_change":"2023-10-26T05:56:59.689219+0000","last_active":"2023-10-26T05:57:00.684404+0000","last_peered":"2023-10-26T05:57:00.684404+0000","last_clean":"2023-10-26T05:57:00.684404+0000","last_became_active":"2023-10-26T05:56:59.688912+0000","last_became_peered":"2023-10-26T05:56:59.688912+0000","last_unstale":"2023-10-26T05:57:00.684404+0000","last_undegraded":"2023-10-26T05:57:00.684404+0000","last_fullsized":"2023-10-26T05:57:00.684404+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:37.636347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"171'1","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.718614+0000","last_change":"2023-10-26T05:56:58.699902+0000","last_active":"2023-10-26T05:56:59.718614+0000","last_peered":"2023-10-26T05:56:59.718614+0000","last_clean":"2023-10-26T05:56:59.718614+0000","last_became_active":"2023-10-26T05:56:58.699481+0000","last_became_peered":"2023-10-26T05:56:58.699481+0000","last_unstale":"2023-10-26T05:56:59.718614+0000","last_undegraded":"2023-10-26T05:56:59.718614+0000","last_fullsized":"2023-10-26T05:56:59.718614+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:02.941018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.683990+0000","last_change":"2023-10-26T05:56:59.691504+0000","last_active":"2023-10-26T05:57:00.683990+0000","last_peered":"2023-10-26T05:57:00.683990+0000","last_clean":"2023-10-26T05:57:00.683990+0000","last_became_active":"2023-10-26T05:56:59.691074+0000","last_became_peered":"2023-10-26T05:56:59.691074+0000","last_unstale":"2023-10-26T05:57:00.683990+0000","last_undegraded":"2023-10-26T05:57:00.683990+0000","last_fullsized":"2023-10-26T05:57:00.683990+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:47:44.992404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"171'2","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.715885+0000","last_change":"2023-10-26T05:56:58.691994+0000","last_active":"2023-10-26T05:56:59.715885+0000","last_peered":"2023-10-26T05:56:59.715885+0000","last_clean":"2023-10-26T05:56:59.715885+0000","last_became_active":"2023-10-26T05:56:58.691381+0000","last_became_peered":"2023-10-26T05:56:58.691381+0000","last_unstale":"2023-10-26T05:56:59.715885+0000","last_undegraded":"2023-10-26T05:56:59.715885+0000","last_fullsized":"2023-10-26T05:56:59.715885+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:15:50.434850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.899399+0000","last_change":"2023-10-26T05:56:59.724944+0000","last_active":"2023-10-26T05:57:00.899399+0000","last_peered":"2023-10-26T05:57:00.899399+0000","last_clean":"2023-10-26T05:57:00.899399+0000","last_became_active":"2023-10-26T05:56:59.724020+0000","last_became_peered":"2023-10-26T05:56:59.724020+0000","last_unstale":"2023-10-26T05:57:00.899399+0000","last_undegraded":"2023-10-26T05:57:00.899399+0000","last_fullsized":"2023-10-26T05:57:00.899399+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:26:35.483279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"171'2","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.716383+0000","last_change":"2023-10-26T05:56:58.687804+0000","last_active":"2023-10-26T05:56:59.716383+0000","last_peered":"2023-10-26T05:56:59.716383+0000","last_clean":"2023-10-26T05:56:59.716383+0000","last_became_active":"2023-10-26T05:56:58.687167+0000","last_became_peered":"2023-10-26T05:56:58.687167+0000","last_unstale":"2023-10-26T05:56:59.716383+0000","last_undegraded":"2023-10-26T05:56:59.716383+0000","last_fullsized":"2023-10-26T05:56:59.716383+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T05:59:48.203254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684774+0000","last_change":"2023-10-26T05:56:59.734756+0000","last_active":"2023-10-26T05:57:00.684774+0000","last_peered":"2023-10-26T05:57:00.684774+0000","last_clean":"2023-10-26T05:57:00.684774+0000","last_became_active":"2023-10-26T05:56:59.734512+0000","last_became_peered":"2023-10-26T05:56:59.734512+0000","last_unstale":"2023-10-26T05:57:00.684774+0000","last_undegraded":"2023-10-26T05:57:00.684774+0000","last_fullsized":"2023-10-26T05:57:00.684774+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:07:33.729359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"171'1","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.724538+0000","last_change":"2023-10-26T05:56:58.708982+0000","last_active":"2023-10-26T05:56:59.724538+0000","last_peered":"2023-10-26T05:56:59.724538+0000","last_clean":"2023-10-26T05:56:59.724538+0000","last_became_active":"2023-10-26T05:56:58.708308+0000","last_became_peered":"2023-10-26T05:56:58.708308+0000","last_unstale":"2023-10-26T05:56:59.724538+0000","last_undegraded":"2023-10-26T05:56:59.724538+0000","last_fullsized":"2023-10-26T05:56:59.724538+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:18:36.411850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691557+0000","last_change":"2023-10-26T05:56:59.728261+0000","last_active":"2023-10-26T05:57:00.691557+0000","last_peered":"2023-10-26T05:57:00.691557+0000","last_clean":"2023-10-26T05:57:00.691557+0000","last_became_active":"2023-10-26T05:56:59.727262+0000","last_became_peered":"2023-10-26T05:56:59.727262+0000","last_unstale":"2023-10-26T05:57:00.691557+0000","last_undegraded":"2023-10-26T05:57:00.691557+0000","last_fullsized":"2023-10-26T05:57:00.691557+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:05:12.956407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"171'1","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.739124+0000","last_change":"2023-10-26T05:56:58.707555+0000","last_active":"2023-10-26T05:56:59.739124+0000","last_peered":"2023-10-26T05:56:59.739124+0000","last_clean":"2023-10-26T05:56:59.739124+0000","last_became_active":"2023-10-26T05:56:58.706918+0000","last_became_peered":"2023-10-26T05:56:58.706918+0000","last_unstale":"2023-10-26T05:56:59.739124+0000","last_undegraded":"2023-10-26T05:56:59.739124+0000","last_fullsized":"2023-10-26T05:56:59.739124+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:09:18.512828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684869+0000","last_change":"2023-10-26T05:56:59.695227+0000","last_active":"2023-10-26T05:57:00.684869+0000","last_peered":"2023-10-26T05:57:00.684869+0000","last_clean":"2023-10-26T05:57:00.684869+0000","last_became_active":"2023-10-26T05:56:59.695019+0000","last_became_peered":"2023-10-26T05:56:59.695019+0000","last_unstale":"2023-10-26T05:57:00.684869+0000","last_undegraded":"2023-10-26T05:57:00.684869+0000","last_fullsized":"2023-10-26T05:57:00.684869+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:04:05.468320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.695687+0000","last_change":"2023-10-26T05:56:58.702478+0000","last_active":"2023-10-26T05:56:59.695687+0000","last_peered":"2023-10-26T05:56:59.695687+0000","last_clean":"2023-10-26T05:56:59.695687+0000","last_became_active":"2023-10-26T05:56:58.702066+0000","last_became_peered":"2023-10-26T05:56:58.702066+0000","last_unstale":"2023-10-26T05:56:59.695687+0000","last_undegraded":"2023-10-26T05:56:59.695687+0000","last_fullsized":"2023-10-26T05:56:59.695687+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:26:34.407747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.897475+0000","last_change":"2023-10-26T05:56:59.709633+0000","last_active":"2023-10-26T05:57:00.897475+0000","last_peered":"2023-10-26T05:57:00.897475+0000","last_clean":"2023-10-26T05:57:00.897475+0000","last_became_active":"2023-10-26T05:56:59.705320+0000","last_became_peered":"2023-10-26T05:56:59.705320+0000","last_unstale":"2023-10-26T05:57:00.897475+0000","last_undegraded":"2023-10-26T05:57:00.897475+0000","last_fullsized":"2023-10-26T05:57:00.897475+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:18:29.352027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.690780+0000","last_change":"2023-10-26T05:56:58.705010+0000","last_active":"2023-10-26T05:56:59.690780+0000","last_peered":"2023-10-26T05:56:59.690780+0000","last_clean":"2023-10-26T05:56:59.690780+0000","last_became_active":"2023-10-26T05:56:58.704379+0000","last_became_peered":"2023-10-26T05:56:58.704379+0000","last_unstale":"2023-10-26T05:56:59.690780+0000","last_undegraded":"2023-10-26T05:56:59.690780+0000","last_fullsized":"2023-10-26T05:56:59.690780+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:23:36.622543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898174+0000","last_change":"2023-10-26T05:56:59.712012+0000","last_active":"2023-10-26T05:57:00.898174+0000","last_peered":"2023-10-26T05:57:00.898174+0000","last_clean":"2023-10-26T05:57:00.898174+0000","last_became_active":"2023-10-26T05:56:59.710864+0000","last_became_peered":"2023-10-26T05:56:59.710864+0000","last_unstale":"2023-10-26T05:57:00.898174+0000","last_undegraded":"2023-10-26T05:57:00.898174+0000","last_fullsized":"2023-10-26T05:57:00.898174+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:29.830663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.686785+0000","last_change":"2023-10-26T05:56:58.692438+0000","last_active":"2023-10-26T05:56:59.686785+0000","last_peered":"2023-10-26T05:56:59.686785+0000","last_clean":"2023-10-26T05:56:59.686785+0000","last_became_active":"2023-10-26T05:56:58.692192+0000","last_became_peered":"2023-10-26T05:56:58.692192+0000","last_unstale":"2023-10-26T05:56:59.686785+0000","last_undegraded":"2023-10-26T05:56:59.686785+0000","last_fullsized":"2023-10-26T05:56:59.686785+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:28:22.102825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898697+0000","last_change":"2023-10-26T05:56:59.702343+0000","last_active":"2023-10-26T05:57:00.898697+0000","last_peered":"2023-10-26T05:57:00.898697+0000","last_clean":"2023-10-26T05:57:00.898697+0000","last_became_active":"2023-10-26T05:56:59.701723+0000","last_became_peered":"2023-10-26T05:56:59.701723+0000","last_unstale":"2023-10-26T05:57:00.898697+0000","last_undegraded":"2023-10-26T05:57:00.898697+0000","last_fullsized":"2023-10-26T05:57:00.898697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:29:00.557535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.720563+0000","last_change":"2023-10-26T05:56:58.715723+0000","last_active":"2023-10-26T05:56:59.720563+0000","last_peered":"2023-10-26T05:56:59.720563+0000","last_clean":"2023-10-26T05:56:59.720563+0000","last_became_active":"2023-10-26T05:56:58.714758+0000","last_became_peered":"2023-10-26T05:56:58.714758+0000","last_unstale":"2023-10-26T05:56:59.720563+0000","last_undegraded":"2023-10-26T05:56:59.720563+0000","last_fullsized":"2023-10-26T05:56:59.720563+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:02:40.699718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.191272+0000","last_change":"2023-10-26T05:56:59.702308+0000","last_active":"2023-10-26T05:57:01.191272+0000","last_peered":"2023-10-26T05:57:01.191272+0000","last_clean":"2023-10-26T05:57:01.191272+0000","last_became_active":"2023-10-26T05:56:59.701981+0000","last_became_peered":"2023-10-26T05:56:59.701981+0000","last_unstale":"2023-10-26T05:57:01.191272+0000","last_undegraded":"2023-10-26T05:57:01.191272+0000","last_fullsized":"2023-10-26T05:57:01.191272+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:30:42.155306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"171'6","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.725707+0000","last_change":"2023-10-26T05:56:58.691788+0000","last_active":"2023-10-26T05:56:59.725707+0000","last_peered":"2023-10-26T05:56:59.725707+0000","last_clean":"2023-10-26T05:56:59.725707+0000","last_became_active":"2023-10-26T05:56:58.691194+0000","last_became_peered":"2023-10-26T05:56:58.691194+0000","last_unstale":"2023-10-26T05:56:59.725707+0000","last_undegraded":"2023-10-26T05:56:59.725707+0000","last_fullsized":"2023-10-26T05:56:59.725707+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:48.624169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.819745+0000","last_change":"2023-10-26T05:56:59.720576+0000","last_active":"2023-10-26T05:57:00.819745+0000","last_peered":"2023-10-26T05:57:00.819745+0000","last_clean":"2023-10-26T05:57:00.819745+0000","last_became_active":"2023-10-26T05:56:59.719742+0000","last_became_peered":"2023-10-26T05:56:59.719742+0000","last_unstale":"2023-10-26T05:57:00.819745+0000","last_undegraded":"2023-10-26T05:57:00.819745+0000","last_fullsized":"2023-10-26T05:57:00.819745+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:45:29.322142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"171'2","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.715966+0000","last_change":"2023-10-26T05:56:58.700296+0000","last_active":"2023-10-26T05:56:59.715966+0000","last_peered":"2023-10-26T05:56:59.715966+0000","last_clean":"2023-10-26T05:56:59.715966+0000","last_became_active":"2023-10-26T05:56:58.699651+0000","last_became_peered":"2023-10-26T05:56:58.699651+0000","last_unstale":"2023-10-26T05:56:59.715966+0000","last_undegraded":"2023-10-26T05:56:59.715966+0000","last_fullsized":"2023-10-26T05:56:59.715966+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:38:59.274030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684583+0000","last_change":"2023-10-26T05:56:59.694671+0000","last_active":"2023-10-26T05:57:00.684583+0000","last_peered":"2023-10-26T05:57:00.684583+0000","last_clean":"2023-10-26T05:57:00.684583+0000","last_became_active":"2023-10-26T05:56:59.694397+0000","last_became_peered":"2023-10-26T05:56:59.694397+0000","last_unstale":"2023-10-26T05:57:00.684583+0000","last_undegraded":"2023-10-26T05:57:00.684583+0000","last_fullsized":"2023-10-26T05:57:00.684583+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:18:03.360919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.716646+0000","last_change":"2023-10-26T05:56:58.708596+0000","last_active":"2023-10-26T05:56:59.716646+0000","last_peered":"2023-10-26T05:56:59.716646+0000","last_clean":"2023-10-26T05:56:59.716646+0000","last_became_active":"2023-10-26T05:56:58.707912+0000","last_became_peered":"2023-10-26T05:56:58.707912+0000","last_unstale":"2023-10-26T05:56:59.716646+0000","last_undegraded":"2023-10-26T05:56:59.716646+0000","last_fullsized":"2023-10-26T05:56:59.716646+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:39:14.608144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.897755+0000","last_change":"2023-10-26T05:56:59.724001+0000","last_active":"2023-10-26T05:57:00.897755+0000","last_peered":"2023-10-26T05:57:00.897755+0000","last_clean":"2023-10-26T05:57:00.897755+0000","last_became_active":"2023-10-26T05:56:59.718523+0000","last_became_peered":"2023-10-26T05:56:59.718523+0000","last_unstale":"2023-10-26T05:57:00.897755+0000","last_undegraded":"2023-10-26T05:57:00.897755+0000","last_fullsized":"2023-10-26T05:57:00.897755+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:59:14.221628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.680189+0000","last_change":"2023-10-26T05:56:58.684439+0000","last_active":"2023-10-26T05:56:59.680189+0000","last_peered":"2023-10-26T05:56:59.680189+0000","last_clean":"2023-10-26T05:56:59.680189+0000","last_became_active":"2023-10-26T05:56:58.684093+0000","last_became_peered":"2023-10-26T05:56:58.684093+0000","last_unstale":"2023-10-26T05:56:59.680189+0000","last_undegraded":"2023-10-26T05:56:59.680189+0000","last_fullsized":"2023-10-26T05:56:59.680189+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:44:29.078877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.819298+0000","last_change":"2023-10-26T05:56:59.722279+0000","last_active":"2023-10-26T05:57:00.819298+0000","last_peered":"2023-10-26T05:57:00.819298+0000","last_clean":"2023-10-26T05:57:00.819298+0000","last_became_active":"2023-10-26T05:56:59.716651+0000","last_became_peered":"2023-10-26T05:56:59.716651+0000","last_unstale":"2023-10-26T05:57:00.819298+0000","last_undegraded":"2023-10-26T05:57:00.819298+0000","last_fullsized":"2023-10-26T05:57:00.819298+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:03:48.294434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"171'8","reported_seq":20,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.732250+0000","last_change":"2023-10-26T05:56:58.703574+0000","last_active":"2023-10-26T05:56:59.732250+0000","last_peered":"2023-10-26T05:56:59.732250+0000","last_clean":"2023-10-26T05:56:59.732250+0000","last_became_active":"2023-10-26T05:56:58.703256+0000","last_became_peered":"2023-10-26T05:56:58.703256+0000","last_unstale":"2023-10-26T05:56:59.732250+0000","last_undegraded":"2023-10-26T05:56:59.732250+0000","last_fullsized":"2023-10-26T05:56:59.732250+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:04:29.521305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.899584+0000","last_change":"2023-10-26T05:56:59.740507+0000","last_active":"2023-10-26T05:57:00.899584+0000","last_peered":"2023-10-26T05:57:00.899584+0000","last_clean":"2023-10-26T05:57:00.899584+0000","last_became_active":"2023-10-26T05:56:59.733041+0000","last_became_peered":"2023-10-26T05:56:59.733041+0000","last_unstale":"2023-10-26T05:57:00.899584+0000","last_undegraded":"2023-10-26T05:57:00.899584+0000","last_fullsized":"2023-10-26T05:57:00.899584+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:35:43.498185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"171'1","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.716964+0000","last_change":"2023-10-26T05:56:58.707861+0000","last_active":"2023-10-26T05:56:59.716964+0000","last_peered":"2023-10-26T05:56:59.716964+0000","last_clean":"2023-10-26T05:56:59.716964+0000","last_became_active":"2023-10-26T05:56:58.707069+0000","last_became_peered":"2023-10-26T05:56:58.707069+0000","last_unstale":"2023-10-26T05:56:59.716964+0000","last_undegraded":"2023-10-26T05:56:59.716964+0000","last_fullsized":"2023-10-26T05:56:59.716964+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:03.299993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.191795+0000","last_change":"2023-10-26T05:56:59.733463+0000","last_active":"2023-10-26T05:57:01.191795+0000","last_peered":"2023-10-26T05:57:01.191795+0000","last_clean":"2023-10-26T05:57:01.191795+0000","last_became_active":"2023-10-26T05:56:59.733294+0000","last_became_peered":"2023-10-26T05:56:59.733294+0000","last_unstale":"2023-10-26T05:57:01.191795+0000","last_undegraded":"2023-10-26T05:57:01.191795+0000","last_fullsized":"2023-10-26T05:57:01.191795+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:09:19.526631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"171'2","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.710128+0000","last_change":"2023-10-26T05:56:58.699246+0000","last_active":"2023-10-26T05:56:59.710128+0000","last_peered":"2023-10-26T05:56:59.710128+0000","last_clean":"2023-10-26T05:56:59.710128+0000","last_became_active":"2023-10-26T05:56:58.698939+0000","last_became_peered":"2023-10-26T05:56:58.698939+0000","last_unstale":"2023-10-26T05:56:59.710128+0000","last_undegraded":"2023-10-26T05:56:59.710128+0000","last_fullsized":"2023-10-26T05:56:59.710128+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:18.155446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.819503+0000","last_change":"2023-10-26T05:56:59.725396+0000","last_active":"2023-10-26T05:57:00.819503+0000","last_peered":"2023-10-26T05:57:00.819503+0000","last_clean":"2023-10-26T05:57:00.819503+0000","last_became_active":"2023-10-26T05:56:59.721466+0000","last_became_peered":"2023-10-26T05:56:59.721466+0000","last_unstale":"2023-10-26T05:57:00.819503+0000","last_undegraded":"2023-10-26T05:57:00.819503+0000","last_fullsized":"2023-10-26T05:57:00.819503+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:40:30.393756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"171'1","reported_seq":13,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.725104+0000","last_change":"2023-10-26T05:56:58.696783+0000","last_active":"2023-10-26T05:56:59.725104+0000","last_peered":"2023-10-26T05:56:59.725104+0000","last_clean":"2023-10-26T05:56:59.725104+0000","last_became_active":"2023-10-26T05:56:58.696463+0000","last_became_peered":"2023-10-26T05:56:58.696463+0000","last_unstale":"2023-10-26T05:56:59.725104+0000","last_undegraded":"2023-10-26T05:56:59.725104+0000","last_fullsized":"2023-10-26T05:56:59.725104+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:29:37.207232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684472+0000","last_change":"2023-10-26T05:56:59.714844+0000","last_active":"2023-10-26T05:57:00.684472+0000","last_peered":"2023-10-26T05:57:00.684472+0000","last_clean":"2023-10-26T05:57:00.684472+0000","last_became_active":"2023-10-26T05:56:59.714457+0000","last_became_peered":"2023-10-26T05:56:59.714457+0000","last_unstale":"2023-10-26T05:57:00.684472+0000","last_undegraded":"2023-10-26T05:57:00.684472+0000","last_fullsized":"2023-10-26T05:57:00.684472+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:52:11.892895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.687211+0000","last_change":"2023-10-26T05:56:58.694349+0000","last_active":"2023-10-26T05:56:59.687211+0000","last_peered":"2023-10-26T05:56:59.687211+0000","last_clean":"2023-10-26T05:56:59.687211+0000","last_became_active":"2023-10-26T05:56:58.694033+0000","last_became_peered":"2023-10-26T05:56:58.694033+0000","last_unstale":"2023-10-26T05:56:59.687211+0000","last_undegraded":"2023-10-26T05:56:59.687211+0000","last_fullsized":"2023-10-26T05:56:59.687211+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:03:10.370668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.692053+0000","last_change":"2023-10-26T05:56:59.706182+0000","last_active":"2023-10-26T05:57:00.692053+0000","last_peered":"2023-10-26T05:57:00.692053+0000","last_clean":"2023-10-26T05:57:00.692053+0000","last_became_active":"2023-10-26T05:56:59.705254+0000","last_became_peered":"2023-10-26T05:56:59.705254+0000","last_unstale":"2023-10-26T05:57:00.692053+0000","last_undegraded":"2023-10-26T05:57:00.692053+0000","last_fullsized":"2023-10-26T05:57:00.692053+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:03:13.788148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"171'2","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.731904+0000","last_change":"2023-10-26T05:56:58.703649+0000","last_active":"2023-10-26T05:56:59.731904+0000","last_peered":"2023-10-26T05:56:59.731904+0000","last_clean":"2023-10-26T05:56:59.731904+0000","last_became_active":"2023-10-26T05:56:58.703345+0000","last_became_peered":"2023-10-26T05:56:58.703345+0000","last_unstale":"2023-10-26T05:56:59.731904+0000","last_undegraded":"2023-10-26T05:56:59.731904+0000","last_fullsized":"2023-10-26T05:56:59.731904+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:30.970841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.889886+0000","last_change":"2023-10-26T05:56:59.700104+0000","last_active":"2023-10-26T05:57:00.889886+0000","last_peered":"2023-10-26T05:57:00.889886+0000","last_clean":"2023-10-26T05:57:00.889886+0000","last_became_active":"2023-10-26T05:56:59.699605+0000","last_became_peered":"2023-10-26T05:56:59.699605+0000","last_unstale":"2023-10-26T05:57:00.889886+0000","last_undegraded":"2023-10-26T05:57:00.889886+0000","last_fullsized":"2023-10-26T05:57:00.889886+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:04:10.153259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.707090+0000","last_change":"2023-10-26T05:56:58.712925+0000","last_active":"2023-10-26T05:56:59.707090+0000","last_peered":"2023-10-26T05:56:59.707090+0000","last_clean":"2023-10-26T05:56:59.707090+0000","last_became_active":"2023-10-26T05:56:58.712384+0000","last_became_peered":"2023-10-26T05:56:58.712384+0000","last_unstale":"2023-10-26T05:56:59.707090+0000","last_undegraded":"2023-10-26T05:56:59.707090+0000","last_fullsized":"2023-10-26T05:56:59.707090+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:23:30.970841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.691703+0000","last_change":"2023-10-26T05:56:59.699747+0000","last_active":"2023-10-26T05:57:00.691703+0000","last_peered":"2023-10-26T05:57:00.691703+0000","last_clean":"2023-10-26T05:57:00.691703+0000","last_became_active":"2023-10-26T05:56:59.699186+0000","last_became_peered":"2023-10-26T05:56:59.699186+0000","last_unstale":"2023-10-26T05:57:00.691703+0000","last_undegraded":"2023-10-26T05:57:00.691703+0000","last_fullsized":"2023-10-26T05:57:00.691703+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:18:49.637973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.705561+0000","last_change":"2023-10-26T05:56:58.712807+0000","last_active":"2023-10-26T05:56:59.705561+0000","last_peered":"2023-10-26T05:56:59.705561+0000","last_clean":"2023-10-26T05:56:59.705561+0000","last_became_active":"2023-10-26T05:56:58.707471+0000","last_became_peered":"2023-10-26T05:56:58.707471+0000","last_unstale":"2023-10-26T05:56:59.705561+0000","last_undegraded":"2023-10-26T05:56:59.705561+0000","last_fullsized":"2023-10-26T05:56:59.705561+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:28:46.025544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.889793+0000","last_change":"2023-10-26T05:56:59.705319+0000","last_active":"2023-10-26T05:57:00.889793+0000","last_peered":"2023-10-26T05:57:00.889793+0000","last_clean":"2023-10-26T05:57:00.889793+0000","last_became_active":"2023-10-26T05:56:59.704934+0000","last_became_peered":"2023-10-26T05:56:59.704934+0000","last_unstale":"2023-10-26T05:57:00.889793+0000","last_undegraded":"2023-10-26T05:57:00.889793+0000","last_fullsized":"2023-10-26T05:57:00.889793+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:57:36.102080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.716987+0000","last_change":"2023-10-26T05:56:58.707193+0000","last_active":"2023-10-26T05:56:59.716987+0000","last_peered":"2023-10-26T05:56:59.716987+0000","last_clean":"2023-10-26T05:56:59.716987+0000","last_became_active":"2023-10-26T05:56:58.705895+0000","last_became_peered":"2023-10-26T05:56:58.705895+0000","last_unstale":"2023-10-26T05:56:59.716987+0000","last_undegraded":"2023-10-26T05:56:59.716987+0000","last_fullsized":"2023-10-26T05:56:59.716987+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:30:02.941018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898259+0000","last_change":"2023-10-26T05:56:59.723568+0000","last_active":"2023-10-26T05:57:00.898259+0000","last_peered":"2023-10-26T05:57:00.898259+0000","last_clean":"2023-10-26T05:57:00.898259+0000","last_became_active":"2023-10-26T05:56:59.717394+0000","last_became_peered":"2023-10-26T05:56:59.717394+0000","last_unstale":"2023-10-26T05:57:00.898259+0000","last_undegraded":"2023-10-26T05:57:00.898259+0000","last_fullsized":"2023-10-26T05:57:00.898259+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:37:51.034433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"171'2","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.738995+0000","last_change":"2023-10-26T05:56:58.717538+0000","last_active":"2023-10-26T05:56:59.738995+0000","last_peered":"2023-10-26T05:56:59.738995+0000","last_clean":"2023-10-26T05:56:59.738995+0000","last_became_active":"2023-10-26T05:56:58.716243+0000","last_became_peered":"2023-10-26T05:56:58.716243+0000","last_unstale":"2023-10-26T05:56:59.738995+0000","last_undegraded":"2023-10-26T05:56:59.738995+0000","last_fullsized":"2023-10-26T05:56:59.738995+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:30:49.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.819683+0000","last_change":"2023-10-26T05:56:59.722758+0000","last_active":"2023-10-26T05:57:00.819683+0000","last_peered":"2023-10-26T05:57:00.819683+0000","last_clean":"2023-10-26T05:57:00.819683+0000","last_became_active":"2023-10-26T05:56:59.719438+0000","last_became_peered":"2023-10-26T05:56:59.719438+0000","last_unstale":"2023-10-26T05:57:00.819683+0000","last_undegraded":"2023-10-26T05:57:00.819683+0000","last_fullsized":"2023-10-26T05:57:00.819683+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:16:21.093745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"172'1","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.682927+0000","last_change":"2023-10-26T05:56:58.697256+0000","last_active":"2023-10-26T05:57:00.682927+0000","last_peered":"2023-10-26T05:57:00.682927+0000","last_clean":"2023-10-26T05:57:00.682927+0000","last_became_active":"2023-10-26T05:56:58.696984+0000","last_became_peered":"2023-10-26T05:56:58.696984+0000","last_unstale":"2023-10-26T05:57:00.682927+0000","last_undegraded":"2023-10-26T05:57:00.682927+0000","last_fullsized":"2023-10-26T05:57:00.682927+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:52:28.079660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.684727+0000","last_change":"2023-10-26T05:56:59.698382+0000","last_active":"2023-10-26T05:57:00.684727+0000","last_peered":"2023-10-26T05:57:00.684727+0000","last_clean":"2023-10-26T05:57:00.684727+0000","last_became_active":"2023-10-26T05:56:59.698135+0000","last_became_peered":"2023-10-26T05:56:59.698135+0000","last_unstale":"2023-10-26T05:57:00.684727+0000","last_undegraded":"2023-10-26T05:57:00.684727+0000","last_fullsized":"2023-10-26T05:57:00.684727+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:34.731361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.681416+0000","last_change":"2023-10-26T05:56:58.687247+0000","last_active":"2023-10-26T05:56:59.681416+0000","last_peered":"2023-10-26T05:56:59.681416+0000","last_clean":"2023-10-26T05:56:59.681416+0000","last_became_active":"2023-10-26T05:56:58.686900+0000","last_became_peered":"2023-10-26T05:56:58.686900+0000","last_unstale":"2023-10-26T05:56:59.681416+0000","last_undegraded":"2023-10-26T05:56:59.681416+0000","last_fullsized":"2023-10-26T05:56:59.681416+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:07:34.440714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.682861+0000","last_change":"2023-10-26T05:56:59.691182+0000","last_active":"2023-10-26T05:57:00.682861+0000","last_peered":"2023-10-26T05:57:00.682861+0000","last_clean":"2023-10-26T05:57:00.682861+0000","last_became_active":"2023-10-26T05:56:59.690976+0000","last_became_peered":"2023-10-26T05:56:59.690976+0000","last_unstale":"2023-10-26T05:57:00.682861+0000","last_undegraded":"2023-10-26T05:57:00.682861+0000","last_fullsized":"2023-10-26T05:57:00.682861+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:20:24.116757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.680361+0000","last_change":"2023-10-26T05:56:58.687941+0000","last_active":"2023-10-26T05:56:59.680361+0000","last_peered":"2023-10-26T05:56:59.680361+0000","last_clean":"2023-10-26T05:56:59.680361+0000","last_became_active":"2023-10-26T05:56:58.687118+0000","last_became_peered":"2023-10-26T05:56:58.687118+0000","last_unstale":"2023-10-26T05:56:59.680361+0000","last_undegraded":"2023-10-26T05:56:59.680361+0000","last_fullsized":"2023-10-26T05:56:59.680361+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:28:44.889241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:01.192195+0000","last_change":"2023-10-26T05:56:59.703543+0000","last_active":"2023-10-26T05:57:01.192195+0000","last_peered":"2023-10-26T05:57:01.192195+0000","last_clean":"2023-10-26T05:57:01.192195+0000","last_became_active":"2023-10-26T05:56:59.703230+0000","last_became_peered":"2023-10-26T05:56:59.703230+0000","last_unstale":"2023-10-26T05:57:01.192195+0000","last_undegraded":"2023-10-26T05:57:01.192195+0000","last_fullsized":"2023-10-26T05:57:01.192195+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:18:29.352027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.688432+0000","last_change":"2023-10-26T05:56:58.692632+0000","last_active":"2023-10-26T05:56:59.688432+0000","last_peered":"2023-10-26T05:56:59.688432+0000","last_clean":"2023-10-26T05:56:59.688432+0000","last_became_active":"2023-10-26T05:56:58.690733+0000","last_became_peered":"2023-10-26T05:56:58.690733+0000","last_unstale":"2023-10-26T05:56:59.688432+0000","last_undegraded":"2023-10-26T05:56:59.688432+0000","last_fullsized":"2023-10-26T05:56:59.688432+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:02:40.699718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898310+0000","last_change":"2023-10-26T05:56:59.720379+0000","last_active":"2023-10-26T05:57:00.898310+0000","last_peered":"2023-10-26T05:57:00.898310+0000","last_clean":"2023-10-26T05:57:00.898310+0000","last_became_active":"2023-10-26T05:56:59.719724+0000","last_became_peered":"2023-10-26T05:56:59.719724+0000","last_unstale":"2023-10-26T05:57:00.898310+0000","last_undegraded":"2023-10-26T05:57:00.898310+0000","last_fullsized":"2023-10-26T05:57:00.898310+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:28:19.275598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.722096+0000","last_change":"2023-10-26T05:56:58.717853+0000","last_active":"2023-10-26T05:56:59.722096+0000","last_peered":"2023-10-26T05:56:59.722096+0000","last_clean":"2023-10-26T05:56:59.722096+0000","last_became_active":"2023-10-26T05:56:58.716922+0000","last_became_peered":"2023-10-26T05:56:58.716922+0000","last_unstale":"2023-10-26T05:56:59.722096+0000","last_undegraded":"2023-10-26T05:56:59.722096+0000","last_fullsized":"2023-10-26T05:56:59.722096+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:35:45.993366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.899034+0000","last_change":"2023-10-26T05:56:59.740033+0000","last_active":"2023-10-26T05:57:00.899034+0000","last_peered":"2023-10-26T05:57:00.899034+0000","last_clean":"2023-10-26T05:57:00.899034+0000","last_became_active":"2023-10-26T05:56:59.738668+0000","last_became_peered":"2023-10-26T05:56:59.738668+0000","last_unstale":"2023-10-26T05:57:00.899034+0000","last_undegraded":"2023-10-26T05:57:00.899034+0000","last_fullsized":"2023-10-26T05:57:00.899034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:10:22.876577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.732221+0000","last_change":"2023-10-26T05:56:58.709784+0000","last_active":"2023-10-26T05:56:59.732221+0000","last_peered":"2023-10-26T05:56:59.732221+0000","last_clean":"2023-10-26T05:56:59.732221+0000","last_became_active":"2023-10-26T05:56:58.707275+0000","last_became_peered":"2023-10-26T05:56:58.707275+0000","last_unstale":"2023-10-26T05:56:59.732221+0000","last_undegraded":"2023-10-26T05:56:59.732221+0000","last_fullsized":"2023-10-26T05:56:59.732221+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:23:42.859789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.898584+0000","last_change":"2023-10-26T05:56:59.724540+0000","last_active":"2023-10-26T05:57:00.898584+0000","last_peered":"2023-10-26T05:57:00.898584+0000","last_clean":"2023-10-26T05:57:00.898584+0000","last_became_active":"2023-10-26T05:56:59.722184+0000","last_became_peered":"2023-10-26T05:56:59.722184+0000","last_unstale":"2023-10-26T05:57:00.898584+0000","last_undegraded":"2023-10-26T05:57:00.898584+0000","last_fullsized":"2023-10-26T05:57:00.898584+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:25:42.702230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.686031+0000","last_change":"2023-10-26T05:56:58.699782+0000","last_active":"2023-10-26T05:56:59.686031+0000","last_peered":"2023-10-26T05:56:59.686031+0000","last_clean":"2023-10-26T05:56:59.686031+0000","last_became_active":"2023-10-26T05:56:58.699054+0000","last_became_peered":"2023-10-26T05:56:58.699054+0000","last_unstale":"2023-10-26T05:56:59.686031+0000","last_undegraded":"2023-10-26T05:56:59.686031+0000","last_fullsized":"2023-10-26T05:56:59.686031+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:59:54.622923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-26T05:57:00.889717+0000","last_change":"2023-10-26T05:56:59.700516+0000","last_active":"2023-10-26T05:57:00.889717+0000","last_peered":"2023-10-26T05:57:00.889717+0000","last_clean":"2023-10-26T05:57:00.889717+0000","last_became_active":"2023-10-26T05:56:59.700110+0000","last_became_peered":"2023-10-26T05:56:59.700110+0000","last_unstale":"2023-10-26T05:57:00.889717+0000","last_undegraded":"2023-10-26T05:57:00.889717+0000","last_fullsized":"2023-10-26T05:57:00.889717+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:58.661153+0000","last_clean_scrub_stamp":"2023-10-26T05:56:58.661153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:35:47.007170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"171'3","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.744861+0000","last_change":"2023-10-26T05:56:58.710168+0000","last_active":"2023-10-26T05:56:59.744861+0000","last_peered":"2023-10-26T05:56:59.744861+0000","last_clean":"2023-10-26T05:56:59.744861+0000","last_became_active":"2023-10-26T05:56:58.708495+0000","last_became_peered":"2023-10-26T05:56:58.708495+0000","last_unstale":"2023-10-26T05:56:59.744861+0000","last_undegraded":"2023-10-26T05:56:59.744861+0000","last_fullsized":"2023-10-26T05:56:59.744861+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:56:57.647349+0000","last_clean_scrub_stamp":"2023-10-26T05:56:57.647349+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:59:54.622923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":306,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-10-26T05:56:59.707346+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:56:59.707346+0000","last_peered":"2023-10-26T05:56:59.707346+0000","last_clean":"2023-10-26T05:56:59.707346+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:56:59.707346+0000","last_undegraded":"2023-10-26T05:56:59.707346+0000","last_fullsized":"2023-10-26T05:56:59.707346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1520,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193791,"statfs":{"total":200000000,"available":198442605,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.59099999999999997,"5min":0.499,"15min":0.39600000000000002},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84599999999999997,"15min":0.83799999999999997},"min":{"1min":0.66700000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3100000000000001},"last":1.1850000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":1.0029999999999999},"min":{"1min":0.64700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.232,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.014},{"interface":"front","average":{"1min":1.103,"5min":1.032,"15min":0.98799999999999999},"min":{"1min":0.97999999999999998,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.2549999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.121}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.68000000000000005,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1899999999999999,"5min":1.448,"15min":1.448},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.93799999999999994,"15min":0.93400000000000005},"min":{"1min":0.63900000000000001,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1990000000000001,"5min":1.3700000000000001,"15min":1.4590000000000001},"last":1.0900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.93200000000000005,"15min":0.89800000000000002},"min":{"1min":0.76000000000000001,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.002,"5min":0.96699999999999997,"15min":0.97199999999999998},"min":{"1min":0.77300000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3089999999999999,"5min":1.3129999999999999,"15min":1.375},"last":0.624}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.60299999999999998,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.133,"5min":1.639,"15min":1.639},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.93300000000000005,"15min":0.93999999999999995},"min":{"1min":0.65700000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.24,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.034,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.337,"5min":1.482,"15min":1.6679999999999999},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0369999999999999,"15min":1.0640000000000001},"min":{"1min":0.72299999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.504,"5min":1.702,"15min":1.702},"last":0.89800000000000002}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.115,"15min":1.1060000000000001},"min":{"1min":0.77800000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.3420000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.091,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.74399999999999999,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.5,"15min":1.6180000000000001},"last":1.052}]}]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":943,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194368,"statfs":{"total":200000000,"available":199033479,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.234,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.871},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.67800000000000005,"5min":0.376,"15min":0.376},"max":{"1min":1.141,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.81799999999999995,"15min":0.79300000000000004},"min":{"1min":0.69999999999999996,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1120000000000001,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.90300000000000002,"15min":0.86599999999999999},"min":{"1min":0.64500000000000002,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.385,"5min":1.3899999999999999,"15min":1.405},"last":0.68200000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.91000000000000003,"15min":0.84599999999999997},"min":{"1min":0.69699999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2450000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.89200000000000002,"15min":0.85599999999999998},"min":{"1min":0.71699999999999997,"5min":0.435,"15min":0.435},"max":{"1min":1.2809999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":0.97999999999999998,"15min":0.92500000000000004},"min":{"1min":0.76900000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.3080000000000001},"last":1.125},{"interface":"front","average":{"1min":1.095,"5min":1.0620000000000001,"15min":1.0009999999999999},"min":{"1min":0.69099999999999995,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.337,"5min":1.3759999999999999,"15min":1.399},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.64000000000000001,"5min":0.503,"15min":0.502},"max":{"1min":1.2569999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.03},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.99299999999999999,"15min":0.98099999999999998},"min":{"1min":0.621,"5min":0.442,"15min":0.442},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.056,"15min":1.05},"min":{"1min":0.89200000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.236,"5min":1.5509999999999999,"15min":1.554},"last":1.198},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.97199999999999998,"15min":0.98299999999999998},"min":{"1min":0.67200000000000004,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.452,"5min":1.512,"15min":1.512},"last":1.083}]},{"osd":7,"last update":"Thu Oct 26 05:56:33 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1220000000000001,"15min":1.1120000000000001},"min":{"1min":0.78400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1020000000000001,"15min":1.111},"min":{"1min":0.997,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.321,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.169}]}]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":943,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194368,"statfs":{"total":200000000,"available":199033603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94499999999999995,"15min":0.95599999999999996},"min":{"1min":0.63400000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2609999999999999,"5min":1.381,"15min":1.579},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.96899999999999997,"15min":0.96299999999999997},"min":{"1min":0.83599999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.1970000000000001,"5min":1.591,"15min":1.591},"last":1.0600000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84699999999999998,"15min":0.878},"min":{"1min":0.625,"5min":0.53300000000000003,"15min":0.441},"max":{"1min":1.1479999999999999,"5min":1.206,"15min":2.1150000000000002},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86799999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2430000000000001,"5min":1.272,"15min":1.464},"last":0.95799999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.92200000000000004,"15min":0.91900000000000004},"min":{"1min":0.70599999999999996,"5min":0.505,"15min":0.505},"max":{"1min":1.2170000000000001,"5min":1.569,"15min":1.569},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.90300000000000002,"15min":0.92100000000000004},"min":{"1min":0.67900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0640000000000001,"5min":1.274,"15min":1.3160000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.90900000000000003,"15min":0.91800000000000004},"min":{"1min":0.69799999999999995,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.363,"5min":1.448,"15min":1.548},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.78200000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.276,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.395}]},{"osd":4,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.93000000000000005,"15min":0.97599999999999998},"min":{"1min":0.60199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2230000000000001,"5min":1.286,"15min":2.1909999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0760000000000001,"15min":1.091},"min":{"1min":0.58799999999999997,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.407,"5min":1.536,"15min":1.536},"last":1.1140000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.99099999999999999,"15min":1.0089999999999999},"min":{"1min":0.68000000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.268,"5min":1.744,"15min":1.744},"last":1.329},{"interface":"front","average":{"1min":1.002,"5min":1.04,"15min":1.032},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.2110000000000001,"5min":1.4279999999999999,"15min":1.4950000000000001},"last":1.161}]},{"osd":7,"last update":"Thu Oct 26 05:56:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.071},"min":{"1min":0.64100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.3109999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.284},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1499999999999999,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3360000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.444}]}]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":943,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194368,"statfs":{"total":200000000,"available":199033625,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.92000000000000004,"15min":0.871},"min":{"1min":0.66200000000000003,"5min":0.48499999999999999,"15min":0.46400000000000002},"max":{"1min":1.1919999999999999,"5min":1.2569999999999999,"15min":1.347},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.91200000000000003,"15min":0.88700000000000001},"min":{"1min":0.70799999999999996,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.159,"5min":1.502,"15min":1.502},"last":0.90700000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.90400000000000003,"15min":0.84499999999999997},"min":{"1min":0.749,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.21,"5min":1.25,"15min":1.375},"last":1.048},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.96099999999999997,"15min":0.91700000000000004},"min":{"1min":0.68300000000000005,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.88}]},{"osd":2,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92000000000000004,"15min":0.85199999999999998},"min":{"1min":0.67400000000000004,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.206,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90700000000000003,"15min":0.879},"min":{"1min":0.72899999999999998,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":1.143,"5min":1.2170000000000001,"15min":1.373},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.98099999999999998,"15min":0.95199999999999996},"min":{"1min":0.79700000000000004,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.177,"5min":1.397,"15min":2.0190000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.091,"5min":0.995,"15min":0.96199999999999997},"min":{"1min":0.79100000000000004,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.28,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.0880000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.89500000000000002,"15min":0.88900000000000001},"min":{"1min":0.64500000000000002,"5min":0.55900000000000005,"15min":0.44800000000000001},"max":{"1min":1.2430000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.0009999999999999,"15min":1},"min":{"1min":0.69299999999999995,"5min":0.58299999999999996,"15min":0.47799999999999998},"max":{"1min":1.2929999999999999,"5min":1.4179999999999999,"15min":1.673},"last":0.97199999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.0429999999999999,"15min":1.014},"min":{"1min":0.66100000000000003,"5min":0.63200000000000001,"15min":0.42099999999999999},"max":{"1min":1.337,"5min":1.762,"15min":1.7969999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.054,"15min":1.042},"min":{"1min":0.76500000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.4970000000000001},"last":1.1859999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:30 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.103,"15min":1.0589999999999999},"min":{"1min":0.77300000000000002,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.3280000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.008},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.879,"5min":0.69599999999999995,"15min":0.53700000000000003},"max":{"1min":1.341,"5min":1.484,"15min":1.728},"last":1.151}]}]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":943,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194368,"statfs":{"total":200000000,"available":199033535,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.52400000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.90100000000000002,"5min":1.385,"15min":1.385},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.61799999999999999,"15min":0.59399999999999997},"min":{"1min":0.52500000000000002,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":0.88200000000000001,"5min":1.115,"15min":1.2689999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":1.091,"5min":1.2,"15min":1.349},"last":0.629},{"interface":"front","average":{"1min":0.751,"5min":0.66000000000000003,"15min":0.622},"min":{"1min":0.38,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.02,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.752,"15min":0.753},"min":{"1min":0.58199999999999996,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.1479999999999999,"5min":1.3460000000000001,"15min":1.367},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.81999999999999995,"15min":0.78700000000000003},"min":{"1min":0.623,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.173,"5min":1.1850000000000001,"15min":1.4199999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.84399999999999997},"min":{"1min":0.70799999999999996,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.1120000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.90200000000000002,"15min":0.91900000000000004},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.2789999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.93100000000000005,"15min":0.95399999999999996},"min":{"1min":0.72399999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.3,"5min":1.413,"15min":1.478},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.014,"5min":0.94999999999999996,"15min":0.92000000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.54,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.94899999999999995,"15min":0.97999999999999998},"min":{"1min":0.64200000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.282,"5min":1.349,"15min":1.6060000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.091,"5min":0.96699999999999997,"15min":1.014},"min":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.49299999999999999},"max":{"1min":1.359,"5min":1.359,"15min":5.6859999999999999},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.98599999999999999,"15min":0.94299999999999995},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.53400000000000003},"max":{"1min":1.472,"5min":2.016,"15min":2.016},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.127,"5min":1.008,"15min":1.032},"min":{"1min":0.68799999999999994,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.327,"5min":2.004,"15min":2.004},"last":1.0700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":945,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194367,"statfs":{"total":200000000,"available":199031943,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.56499999999999995},"min":{"1min":0.495,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.71999999999999997,"15min":0.63800000000000001},"min":{"1min":0.54400000000000004,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.1619999999999999,"5min":1.379,"15min":1.5249999999999999},"last":0.80900000000000005}]},{"osd":1,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.69599999999999995},"min":{"1min":0.61699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":1.091,"5min":1.226,"15min":1.2609999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67700000000000005,"15min":0.621},"min":{"1min":0.57799999999999996,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":0.90700000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.68100000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.82199999999999995,"15min":0.76500000000000001},"min":{"1min":0.71599999999999997,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.173,"5min":1.24,"15min":2.8809999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.63600000000000001},"min":{"1min":0.50800000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.94899999999999995,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.93000000000000005,"15min":0.85999999999999999},"min":{"1min":0.69099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.151,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88500000000000001,"15min":0.85899999999999999},"min":{"1min":0.78200000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1100000000000001,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":1.0029999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.997,"15min":0.93899999999999995},"min":{"1min":0.84999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.246,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.999,"15min":0.93100000000000005},"min":{"1min":0.80000000000000004,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.329,"5min":1.4790000000000001,"15min":1.653},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.008,"15min":0.94699999999999995},"min":{"1min":0.73199999999999998,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.3320000000000001,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.089,"15min":1.016},"min":{"1min":0.76500000000000001,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.3300000000000001,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:56:26 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":0.91900000000000004},"min":{"1min":0.84299999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.073,"5min":1.581,"15min":1.581},"last":1.117},{"interface":"front","average":{"1min":1.133,"5min":1.109,"15min":1.069},"min":{"1min":0.91200000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.462,"5min":1.823,"15min":15.352},"last":1.085}]}]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":944,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194368,"statfs":{"total":200000000,"available":199033043,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.44400000000000001,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.157},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55600000000000005,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0569999999999999,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.78300000000000003,"15min":0.73999999999999999},"min":{"1min":0.63700000000000001,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.073,"5min":1.6080000000000001,"15min":1.8169999999999999},"last":0.751}]},{"osd":3,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.754,"15min":0.76100000000000001},"min":{"1min":0.65600000000000003,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0740000000000001,"5min":1.147,"15min":1.7949999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.54300000000000004,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.0760000000000001,"5min":1.298,"15min":1.298},"last":0.66100000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.88900000000000001,"15min":0.874},"min":{"1min":0.73899999999999999,"5min":0.54300000000000004,"15min":0.54100000000000004},"max":{"1min":1.1020000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.157},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.95999999999999996,"15min":0.95399999999999996},"min":{"1min":0.74099999999999999,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.345,"5min":1.5069999999999999,"15min":3.7440000000000002},"last":1.0389999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":0.96299999999999997,"15min":0.96399999999999997},"min":{"1min":0.749,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.319,"5min":1.381,"15min":1.492},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.94499999999999995,"15min":0.93700000000000006},"min":{"1min":0.84899999999999998,"5min":0.60699999999999998,"15min":0.51700000000000002},"max":{"1min":1.2509999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.0640000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0249999999999999},"min":{"1min":0.89200000000000002,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":2.7759999999999998},"last":1.018},{"interface":"front","average":{"1min":1.073,"5min":1.006,"15min":0.97799999999999998},"min":{"1min":0.89700000000000002,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.2669999999999999,"5min":1.468,"15min":1.837},"last":1.232}]},{"osd":7,"last update":"Thu Oct 26 05:56:31 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.046,"15min":1.0309999999999999},"min":{"1min":0.82499999999999996,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.397,"5min":1.4179999999999999,"15min":1.6339999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0640000000000001,"15min":1.0840000000000001},"min":{"1min":0.93100000000000005,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4890000000000001,"5min":1.8180000000000001,"15min":5.2430000000000003},"last":0.80800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1521,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193790,"statfs":{"total":200000000,"available":198441611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.55500000000000005},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.23799999999999999},"max":{"1min":0.96599999999999997,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72999999999999998,"15min":0.63900000000000001},"min":{"1min":0.501,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.95599999999999996,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82199999999999995,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.39100000000000001,"15min":0.28399999999999997},"max":{"1min":1.2010000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.63400000000000001},"min":{"1min":0.52800000000000002,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":1.1020000000000001,"15min":1.177},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68200000000000005,"15min":0.63900000000000001},"min":{"1min":0.53500000000000003,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0249999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.441},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82499999999999996,"15min":0.72899999999999998},"min":{"1min":0.58999999999999997,"5min":0.32100000000000001,"15min":0.247},"max":{"1min":1.026,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.91400000000000003,"15min":0.873},"min":{"1min":0.64200000000000002,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.319,"5min":1.395,"15min":1.395},"last":0.622},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0189999999999999,"15min":0.93500000000000005},"min":{"1min":0.73699999999999999,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":1.1559999999999999,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.92100000000000004,"15min":0.92200000000000004},"min":{"1min":0.61299999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":2.7480000000000002},"last":0.71399999999999997},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.96299999999999997,"15min":0.92100000000000004},"min":{"1min":0.87,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.94599999999999995,"15min":0.91900000000000004},"min":{"1min":0.77500000000000002,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1639999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.075,"5min":1.131,"15min":1.0169999999999999},"min":{"1min":0.871,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.2829999999999999,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":0.94899999999999995}]},{"osd":7,"last update":"Thu Oct 26 05:56:29 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0109999999999999,"15min":0.998},"min":{"1min":0.74199999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1539999999999999,"5min":1.669,"15min":1.6990000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0329999999999999,"15min":0.999},"min":{"1min":0.93500000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.417,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":0.80400000000000005}]}]}],"pool_statfs":[]}} 2023-10-26T05:57:07.480 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:57:07.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:57:07.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/1134449884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb40b0fa0 msgr2=0x7f0fb4070dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/1134449884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4070dc0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009840 tx=0x7f0fa401c790 comp rx=0 tx=0).stop 2023-10-26T05:57:07.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/1134449884 shutdown_connections 2023-10-26T05:57:07.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/1134449884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4070dc0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/1134449884 >> 172.21.15.71:0/1134449884 conn(0x7f0fb40aec60 msgr2=0x7f0fb40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:07.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/1134449884 shutdown_connections 2023-10-26T05:57:07.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/1134449884 wait complete. 2023-10-26T05:57:07.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 Processor -- start 2023-10-26T05:57:07.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- start start 2023-10-26T05:57:07.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4134710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 0x7f0fb40afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb415bf70 0x7f0fb40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb4134c50 con 0x7f0fb415bb90 2023-10-26T05:57:07.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415b3f0 con 0x7f0fb40b0fa0 2023-10-26T05:57:07.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415a840 con 0x7f0fb415bf70 2023-10-26T05:57:07.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 0x7f0fb40afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 0x7f0fb40afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35858/0 (socket says 172.21.15.71:35858) 2023-10-26T05:57:07.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 -- 172.21.15.71:0/3441830952 learned_addr learned my addr 172.21.15.71:0/3441830952 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:07.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb9408640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4134710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 -- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb415bf70 msgr2=0x7f0fb40b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb415bf70 0x7f0fb40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 -- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb40b0fa0 msgr2=0x7f0fb4134710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4134710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 -- 172.21.15.71:0/3441830952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40094a0 con 0x7f0fb415bb90 2023-10-26T05:57:07.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb9408640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4134710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:57:07.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fb8c07640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 0x7f0fb40afd50 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4000c10 tx=0x7f0fa40050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:07.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40054b0 con 0x7f0fb415bb90 2023-10-26T05:57:07.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0fa40046d0 con 0x7f0fb415bb90 2023-10-26T05:57:07.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb415aac0 con 0x7f0fb415bb90 2023-10-26T05:57:07.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4004a80 con 0x7f0fb415bb90 2023-10-26T05:57:07.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb415b080 con 0x7f0fb415bb90 2023-10-26T05:57:07.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.631+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb4066b10 con 0x7f0fb415bb90 2023-10-26T05:57:07.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.635+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0fa4004be0 con 0x7f0fb415bb90 2023-10-26T05:57:07.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.635+0000 7f0fa9ffb640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.635+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fa411ee30 con 0x7f0fb415bb90 2023-10-26T05:57:07.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.635+0000 7f0fb9408640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.635+0000 7f0fb9408640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00016f0 tx=0x7f0fb0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:07.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.639+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0fa40ec3d0 con 0x7f0fb415bb90 2023-10-26T05:57:07.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.839+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0fb4066b10 con 0x7f0fb415bb90 2023-10-26T05:57:07.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fa9ffb640 1 -- 172.21.15.71:0/3441830952 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0fa40b2020 con 0x7f0fb415bb90 2023-10-26T05:57:07.852 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:57:07.852 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:57:07.852 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:57:07.852 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:57:07.853 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:57:07.853 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:57:07.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 msgr2=0x7f0f98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00016f0 tx=0x7f0fb0009040 comp rx=0 tx=0).stop 2023-10-26T05:57:07.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 msgr2=0x7f0fb40afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 0x7f0fb40afd50 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4000c10 tx=0x7f0fa40050c0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 shutdown_connections 2023-10-26T05:57:07.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0fb415bf70 0x7f0fb40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0f980664a0 0x7f0f98068960 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb415bb90 0x7f0fb40afd50 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 --2- 172.21.15.71:0/3441830952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fb40b0fa0 0x7f0fb4134710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 >> 172.21.15.71:0/3441830952 conn(0x7f0fb40aec60 msgr2=0x7f0fb413e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 shutdown_connections 2023-10-26T05:57:07.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.843+0000 7f0fbb66c640 1 -- 172.21.15.71:0/3441830952 wait complete. 2023-10-26T05:57:07.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:57:07.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/1431345719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480b2890 msgr2=0x7fc0480caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 --2- 172.21.15.71:0/1431345719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480b2890 0x7fc0480caad0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fc0480cc8d0 tx=0x7fc03001c820 comp rx=0 tx=0).stop 2023-10-26T05:57:07.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/1431345719 shutdown_connections 2023-10-26T05:57:07.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 --2- 172.21.15.71:0/1431345719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480b2890 0x7fc0480caad0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 --2- 172.21.15.71:0/1431345719 >> v2:172.21.15.86:3300/0 conn(0x7fc04813c2f0 0x7fc0480b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/1431345719 >> 172.21.15.71:0/1431345719 conn(0x7fc048069770 msgr2=0x7fc048069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:07.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/1431345719 shutdown_connections 2023-10-26T05:57:07.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/1431345719 wait complete. 2023-10-26T05:57:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 Processor -- start 2023-10-26T05:57:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- start start 2023-10-26T05:57:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0480b2890 0x7fc048139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 0x7fc048130130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc04813c2f0 0x7fc04813a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:07.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04815cbb0 con 0x7fc0480cb010 2023-10-26T05:57:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04815c030 con 0x7fc0480b2890 2023-10-26T05:57:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04815c310 con 0x7fc04813c2f0 2023-10-26T05:57:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 0x7fc048130130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 0x7fc048130130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35882/0 (socket says 172.21.15.71:35882) 2023-10-26T05:57:07.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 -- 172.21.15.71:0/2221837552 learned_addr learned my addr 172.21.15.71:0/2221837552 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc046ffd640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0480b2890 0x7fc048139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc0477fe640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc04813c2f0 0x7fc04813a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 -- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc04813c2f0 msgr2=0x7fc04813a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc04813c2f0 0x7fc04813a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.997 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 -- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0480b2890 msgr2=0x7fc048139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0480b2890 0x7fc048139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 -- 172.21.15.71:0/2221837552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030009560 con 0x7fc0480cb010 2023-10-26T05:57:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc03ffff640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 0x7fc048130130 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc03001d0c0 tx=0x7fc0300a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0300b6070 con 0x7fc0480cb010 2023-10-26T05:57:07.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc030005040 con 0x7fc0480cb010 2023-10-26T05:57:07.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc030005410 con 0x7fc0480cb010 2023-10-26T05:57:07.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04815c5f0 con 0x7fc0480cb010 2023-10-26T05:57:07.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.983+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04813ae50 con 0x7fc0480cb010 2023-10-26T05:57:07.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.987+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc048066b10 con 0x7fc0480cb010 2023-10-26T05:57:07.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.987+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc0300046d0 con 0x7fc0480cb010 2023-10-26T05:57:08.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.987+0000 7fc044ff9640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0280664a0 0x7fc028068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.987+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc03011e080 con 0x7fc0480cb010 2023-10-26T05:57:08.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.987+0000 7fc046ffd640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0280664a0 0x7fc028068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.987+0000 7fc046ffd640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0280664a0 0x7fc028068960 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc038005fd0 tx=0x7fc038005d10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:07.991+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc0300eb590 con 0x7fc0480cb010 2023-10-26T05:57:08.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.171+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0480721e0 con 0x7fc0480cb010 2023-10-26T05:57:08.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc044ff9640 1 -- 172.21.15.71:0/2221837552 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fc0300eebb0 con 0x7fc0480cb010 2023-10-26T05:57:08.182 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:08.183 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:57:08.183 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:57:08.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0280664a0 msgr2=0x7fc028068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0280664a0 0x7fc028068960 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc038005fd0 tx=0x7fc038005d10 comp rx=0 tx=0).stop 2023-10-26T05:57:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 msgr2=0x7fc048130130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 0x7fc048130130 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc03001d0c0 tx=0x7fc0300a6570 comp rx=0 tx=0).stop 2023-10-26T05:57:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 shutdown_connections 2023-10-26T05:57:08.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc04813c2f0 0x7fc04813a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0280664a0 0x7fc028068960 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0480cb010 0x7fc048130130 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 --2- 172.21.15.71:0/2221837552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0480b2890 0x7fc048139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 >> 172.21.15.71:0/2221837552 conn(0x7fc048069770 msgr2=0x7fc0480ad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 shutdown_connections 2023-10-26T05:57:08.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.175+0000 7fc04d91b640 1 -- 172.21.15.71:0/2221837552 wait complete. 2023-10-26T05:57:08.602 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:56:59.035024+0000', 'modified': '2023-10-26T05:57:01.073826+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8958}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8958': {'gid': 8958, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/2999623846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 2999623846}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 2999623846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:57:08.603 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:57:08.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-26T05:57:08.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/720802570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 msgr2=0x7fa1c4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/720802570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1c4003da0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009b60 tx=0x7fa1cc01d250 comp rx=0 tx=0).stop 2023-10-26T05:57:08.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/720802570 shutdown_connections 2023-10-26T05:57:08.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/720802570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1c4003da0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/720802570 >> 172.21.15.71:0/720802570 conn(0x7fa1dc06b8c0 msgr2=0x7fa1dc06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/720802570 shutdown_connections 2023-10-26T05:57:08.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.731+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/720802570 wait complete. 2023-10-26T05:57:08.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 Processor -- start 2023-10-26T05:57:08.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 -- start start 2023-10-26T05:57:08.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1dc13a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1dc139100 0x7fa1dc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc152ab0 0x7fa1dc13b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc13ac60 con 0x7fa1c4003980 2023-10-26T05:57:08.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc139680 con 0x7fa1dc152ab0 2023-10-26T05:57:08.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc139960 con 0x7fa1dc139100 2023-10-26T05:57:08.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1dc13a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1dc13a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35902/0 (socket says 172.21.15.71:35902) 2023-10-26T05:57:08.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 -- 172.21.15.71:0/3457934138 learned_addr learned my addr 172.21.15.71:0/3457934138 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:08.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc152ab0 0x7fa1dc13b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1dc139100 0x7fa1dc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 -- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1dc139100 msgr2=0x7fa1dc158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1dc139100 0x7fa1dc158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 -- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc152ab0 msgr2=0x7fa1dc13b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc152ab0 0x7fa1dc13b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 -- 172.21.15.71:0/3457934138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc139c40 con 0x7fa1c4003980 2023-10-26T05:57:08.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1da59c640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1dc13a720 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc01d080 tx=0x7fa1cc005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1caffd640 1 -- 172.21.15.71:0/3457934138 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004230 con 0x7fa1c4003980 2023-10-26T05:57:08.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1caffd640 1 -- 172.21.15.71:0/3457934138 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1cc004480 con 0x7fa1c4003980 2023-10-26T05:57:08.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1caffd640 1 -- 172.21.15.71:0/3457934138 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0049f0 con 0x7fa1c4003980 2023-10-26T05:57:08.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc139f20 con 0x7fa1c4003980 2023-10-26T05:57:08.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.735+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc158970 con 0x7fa1c4003980 2023-10-26T05:57:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1caffd640 1 -- 172.21.15.71:0/3457934138 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa1cc0b3070 con 0x7fa1c4003980 2023-10-26T05:57:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1caffd640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1bc066470 0x7fa1bc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1caffd640 1 -- 172.21.15.71:0/3457934138 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1cc11df70 con 0x7fa1c4003980 2023-10-26T05:57:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1bc066470 0x7fa1bc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 Processor -- start 2023-10-26T05:57:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 -- start start 2023-10-26T05:57:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1bc066470 0x7fa1bc068930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0000f70 tx=0x7fa1d0000ee0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 0x56318cf0fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf102e0 0x56318cec5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cec6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ccbacc0 con 0x56318cf0f9c0 2023-10-26T05:57:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318cd1d920 con 0x56318cec5f80 2023-10-26T05:57:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318cd1ae50 con 0x56318cf102e0 2023-10-26T05:57:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 0x56318cf0fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 0x56318cf0fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35906/0 (socket says 172.21.15.71:35906) 2023-10-26T05:57:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 -- 172.21.15.71:0/4167696235 learned_addr learned my addr 172.21.15.71:0/4167696235 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf102e0 0x56318cec5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cec6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 -- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf102e0 msgr2=0x56318cec5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf102e0 0x56318cec5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 -- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 msgr2=0x56318cec6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cec6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 -- 172.21.15.71:0/4167696235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318ca0f890 con 0x56318cf0f9c0 2023-10-26T05:57:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cec6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:57:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1da59c640 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 0x56318cf0fda0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0eb660 tx=0x7fa1cc0ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1850d438642b701 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1bbfff640 1 -- 172.21.15.71:0/4167696235 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0f3200 con 0x56318cf0f9c0 2023-10-26T05:57:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1bbfff640 1 -- 172.21.15.71:0/4167696235 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1cc11c5b0 con 0x56318cf0f9c0 2023-10-26T05:57:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.739+0000 7fa1bbfff640 1 -- 172.21.15.71:0/4167696235 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0ed500 con 0x56318cf0f9c0 2023-10-26T05:57:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/4167696235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 msgr2=0x56318cf0fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 0x56318cf0fda0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0eb660 tx=0x7fa1cc0ef040 comp rx=0 tx=0).stop 2023-10-26T05:57:08.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/4167696235 shutdown_connections 2023-10-26T05:57:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cec6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf102e0 0x56318cec5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/4167696235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf0f9c0 0x56318cf0fda0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/4167696235 >> 172.21.15.71:0/4167696235 conn(0x56318cebd2f0 msgr2=0x56318cec4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/4167696235 shutdown_connections 2023-10-26T05:57:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/4167696235 wait complete. 2023-10-26T05:57:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- start start 2023-10-26T05:57:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 Processor -- start 2023-10-26T05:57:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cec5f80 0x56318cecaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf0f9c0 0x56318ced0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318ced10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318cd16740 con 0x56318cf102e0 2023-10-26T05:57:08.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ccd5d10 con 0x56318cf0f9c0 2023-10-26T05:57:08.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318ccd59b0 con 0x56318cec5f80 2023-10-26T05:57:08.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 10 client.0 _ll_register_callbacks cb 0x56318ceb16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:57:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318ced10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318ced10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35908/0 (socket says 172.21.15.71:35908) 2023-10-26T05:57:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 -- 172.21.15.71:0/266550150 learned_addr learned my addr 172.21.15.71:0/266550150 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1da59c640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cec5f80 0x56318cecaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf0f9c0 0x56318ced0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 -- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cec5f80 msgr2=0x56318cecaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cec5f80 0x56318cecaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 -- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf0f9c0 msgr2=0x56318ced0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf0f9c0 0x56318ced0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 -- 172.21.15.71:0/266550150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc158970 con 0x56318cf102e0 2023-10-26T05:57:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318ced10d0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0b0cd0 tx=0x7fa1d4004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268f8f0b9bb2e6ad server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:57:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 1 -- 172.21.15.71:0/266550150 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4012650 con 0x56318cf102e0 2023-10-26T05:57:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 1 -- 172.21.15.71:0/266550150 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1d40127b0 con 0x56318cf102e0 2023-10-26T05:57:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56318ccca370 con 0x56318cf102e0 2023-10-26T05:57:08.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 1 -- 172.21.15.71:0/266550150 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1d4012970 con 0x56318cf102e0 2023-10-26T05:57:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 10 client.9126 fetch_fsmap learned FSMap version 64 2023-10-26T05:57:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x56318ccd8460 con 0x56318cf102e0 2023-10-26T05:57:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 1 -- 172.21.15.71:0/266550150 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x56318ccca370 con 0x56318cf102e0 2023-10-26T05:57:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 1 -- 172.21.15.71:0/266550150 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa1d4012bc0 con 0x56318cf102e0 2023-10-26T05:57:08.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 10 client.9126 fetch_fsmap finished waiting for FSMap version 64 2023-10-26T05:57:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 10 client.9126 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:57:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 20 client.9126 populate_metadata read hostname 'smithi071' 2023-10-26T05:57:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf1df10 0x56318cf20300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 4 client.9126 mds_command: new command op to 8958 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:57:08.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 --> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56318ccb02d0 con 0x56318cf1df10 2023-10-26T05:57:08.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1da59c640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf1df10 0x56318cf20300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1b97fa640 10 client.9126 ms_handle_connect on v2:172.21.15.71:6834/2999623846 2023-10-26T05:57:08.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.743+0000 7fa1da59c640 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf1df10 0x56318cf20300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc1f3f48de85d0e server_cookie=60f6e5b35eeaaa72 in_seq=0 out_seq=0 2023-10-26T05:57:08.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1b97fa640 1 -- 172.21.15.71:0/266550150 <== mds.0 v2:172.21.15.71:6834/2999623846 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56318ccb02d0 con 0x56318cf1df10 2023-10-26T05:57:08.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1b97fa640 10 client.9126 handle_command_reply: tid=0 2023-10-26T05:57:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 client.9126 shutdown 2023-10-26T05:57:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 msgr2=0x56318ced10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318ced10d0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0b0cd0 tx=0x7fa1d4004f90 comp rx=0 tx=0).stop 2023-10-26T05:57:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 shutdown_connections 2023-10-26T05:57:08.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf1df10 0x56318cf20300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318ced10d0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf0f9c0 0x56318ced0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/266550150 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cec5f80 0x56318cecaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1b97fa640 0 client.9126 ms_handle_reset on v2:172.21.15.71:6834/2999623846 2023-10-26T05:57:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 >> 172.21.15.71:0/266550150 conn(0x56318cebd2f0 msgr2=0x56318cec4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 shutdown_connections 2023-10-26T05:57:08.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 1 -- 172.21.15.71:0/266550150 wait complete. 2023-10-26T05:57:08.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.747+0000 7fa1e40f3000 20 client.9126 trim_cache size 0 max 16384 2023-10-26T05:57:08.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 Processor -- start 2023-10-26T05:57:08.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 -- start start 2023-10-26T05:57:08.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cf69ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 0x56318cf6a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318cf6a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318c9d0940 con 0x56318cf102e0 2023-10-26T05:57:08.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318ccb7130 con 0x56318cec5f80 2023-10-26T05:57:08.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318ced0ff0 con 0x56318cf0f9c0 2023-10-26T05:57:08.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318cf6a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1dad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318cf6a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35922/0 (socket says 172.21.15.71:35922) 2023-10-26T05:57:08.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.759+0000 7fa1dad9d640 1 -- 172.21.15.71:0/2563790162 learned_addr learned my addr 172.21.15.71:0/2563790162 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:08.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1dad9d640 1 -- 172.21.15.71:0/2563790162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 msgr2=0x56318cf6a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 0x56318cf6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1dad9d640 1 -- 172.21.15.71:0/2563790162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 msgr2=0x56318cf69ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cf69ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1dad9d640 1 -- 172.21.15.71:0/2563790162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56318ccbe760 con 0x56318cf102e0 2023-10-26T05:57:08.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1dad9d640 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318cf6a540 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fa1d401b460 tx=0x7fa1d4002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d352db4a7c99e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1c8ff9640 1 -- 172.21.15.71:0/2563790162 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4004b40 con 0x56318cf102e0 2023-10-26T05:57:08.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1c8ff9640 1 -- 172.21.15.71:0/2563790162 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa1d4012650 con 0x56318cf102e0 2023-10-26T05:57:08.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1c8ff9640 1 -- 172.21.15.71:0/2563790162 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40129c0 con 0x56318cf102e0 2023-10-26T05:57:08.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/2563790162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 msgr2=0x56318cf6a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318cf6a540 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fa1d401b460 tx=0x7fa1d4002000 comp rx=0 tx=0).stop 2023-10-26T05:57:08.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/2563790162 shutdown_connections 2023-10-26T05:57:08.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cf102e0 0x56318cf6a540 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 0x56318cf6a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/2563790162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cec5f80 0x56318cf69ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/2563790162 >> 172.21.15.71:0/2563790162 conn(0x56318cebd2f0 msgr2=0x56318cf0db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/2563790162 shutdown_connections 2023-10-26T05:57:08.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/2563790162 wait complete. 2023-10-26T05:57:08.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- start start 2023-10-26T05:57:08.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 Processor -- start 2023-10-26T05:57:08.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 0x56318cf21b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 0x56318cf22070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf102e0 0x56318cf6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318cf1b210 con 0x56318cec5f80 2023-10-26T05:57:08.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56318cf1ac70 con 0x56318cf102e0 2023-10-26T05:57:08.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56318cd1a740 con 0x56318cf0f9c0 2023-10-26T05:57:08.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 10 client.0 _ll_register_callbacks cb 0x56318cd14ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:57:08.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 0x56318cf21b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 0x56318cf21b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35924/0 (socket says 172.21.15.71:35924) 2023-10-26T05:57:08.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 -- 172.21.15.71:0/1857217961 learned_addr learned my addr 172.21.15.71:0/1857217961 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:08.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 -- 172.21.15.71:0/1857217961 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 msgr2=0x56318cf22070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 0x56318cf22070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 -- 172.21.15.71:0/1857217961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf102e0 msgr2=0x56318cf6a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:57:08.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf102e0 0x56318cf6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 -- 172.21.15.71:0/1857217961 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56318ccd8460 con 0x56318cec5f80 2023-10-26T05:57:08.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1da59c640 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 0x56318cf21b30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009660 tx=0x7fa1cc0f74b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd89586c31f63689 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:57:08.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1b97fa640 1 -- 172.21.15.71:0/1857217961 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0f3200 con 0x56318cec5f80 2023-10-26T05:57:08.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1b97fa640 1 -- 172.21.15.71:0/1857217961 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1cc009730 con 0x56318cec5f80 2023-10-26T05:57:08.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56318cec86e0 con 0x56318cec5f80 2023-10-26T05:57:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1b97fa640 1 -- 172.21.15.71:0/1857217961 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1cc0ed500 con 0x56318cec5f80 2023-10-26T05:57:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 10 client.9132 fetch_fsmap learned FSMap version 64 2023-10-26T05:57:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x56318ca0fcc0 con 0x56318cec5f80 2023-10-26T05:57:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1b97fa640 1 -- 172.21.15.71:0/1857217961 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x56318cec86e0 con 0x56318cec5f80 2023-10-26T05:57:08.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1b97fa640 1 -- 172.21.15.71:0/1857217961 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa1cc0c0020 con 0x56318cec5f80 2023-10-26T05:57:08.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 10 client.9132 fetch_fsmap finished waiting for FSMap version 64 2023-10-26T05:57:08.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 10 client.9132 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-26T05:57:08.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 20 client.9132 populate_metadata read hostname 'smithi071' 2023-10-26T05:57:08.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf6ab60 0x56318cf6af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 4 client.9132 mds_command: new command op to 8958 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:57:08.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 --> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56318ccae6f0 con 0x56318cf6ab60 2023-10-26T05:57:08.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.763+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf6ab60 0x56318cf6af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1d9d9b640 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf6ab60 0x56318cf6af40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d3cade440bedb1 server_cookie=a1a17909d99f8c3a in_seq=0 out_seq=0 2023-10-26T05:57:08.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1b97fa640 10 client.9132 ms_handle_connect on v2:172.21.15.71:6834/2999623846 2023-10-26T05:57:08.790 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1b97fa640 1 -- 172.21.15.71:0/1857217961 <== mds.0 v2:172.21.15.71:6834/2999623846 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x56318ccae6f0 con 0x56318cf6ab60 2023-10-26T05:57:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1b97fa640 10 client.9132 handle_command_reply: tid=0 2023-10-26T05:57:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 client.9132 shutdown 2023-10-26T05:57:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 msgr2=0x56318cf21b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 0x56318cf21b30 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009660 tx=0x7fa1cc0f74b0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 shutdown_connections 2023-10-26T05:57:08.791 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:6834/2999623846,v1:172.21.15.71:6835/2999623846] conn(0x56318cf6ab60 0x56318cf6af40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56318cf102e0 0x56318cf6a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56318cf0f9c0 0x56318cf22070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1b97fa640 0 client.9132 ms_handle_reset on v2:172.21.15.71:6834/2999623846 2023-10-26T05:57:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 --2- 172.21.15.71:0/1857217961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56318cec5f80 0x56318cf21b30 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.792 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 >> 172.21.15.71:0/1857217961 conn(0x56318cebd2f0 msgr2=0x56318cf0e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 shutdown_connections 2023-10-26T05:57:08.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 1 -- 172.21.15.71:0/1857217961 wait complete. 2023-10-26T05:57:08.793 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e40f3000 20 client.9132 trim_cache size 0 max 16384 2023-10-26T05:57:08.793 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:57:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:57:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:57:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 8958, 2023-10-26T05:57:08.794 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:57:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:57:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:57:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 9.0924335519999993, 2023-10-26T05:57:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 64, 2023-10-26T05:57:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 172, 2023-10-26T05:57:08.795 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 172, 2023-10-26T05:57:08.796 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 14.875842175000001 2023-10-26T05:57:08.796 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:57:08.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1bc066470 msgr2=0x7fa1bc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1bc066470 0x7fa1bc068930 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0000f70 tx=0x7fa1d0000ee0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 msgr2=0x7fa1dc13a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1dc13a720 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc01d080 tx=0x7fa1cc005520 comp rx=0 tx=0).stop 2023-10-26T05:57:08.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 shutdown_connections 2023-10-26T05:57:08.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc152ab0 0x7fa1dc13b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa1dc139100 0x7fa1dc158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa1bc066470 0x7fa1bc068930 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 --2- 172.21.15.71:0/3457934138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa1c4003980 0x7fa1dc13a720 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 >> 172.21.15.71:0/3457934138 conn(0x7fa1dc06b8c0 msgr2=0x7fa1dc140550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 shutdown_connections 2023-10-26T05:57:08.799 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.767+0000 7fa1e0fa9640 1 -- 172.21.15.71:0/3457934138 wait complete. 2023-10-26T05:57:08.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:57:08.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 -- 172.21.15.71:0/2612802181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80b2890 msgr2=0x7fd1b80caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 --2- 172.21.15.71:0/2612802181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80b2890 0x7fd1b80caad0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009900 tx=0x7fd1ac01c880 comp rx=0 tx=0).stop 2023-10-26T05:57:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 -- 172.21.15.71:0/2612802181 shutdown_connections 2023-10-26T05:57:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 --2- 172.21.15.71:0/2612802181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80b2890 0x7fd1b80caad0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 --2- 172.21.15.71:0/2612802181 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1b813c2f0 0x7fd1b80b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 -- 172.21.15.71:0/2612802181 >> 172.21.15.71:0/2612802181 conn(0x7fd1b8069770 msgr2=0x7fd1b8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:08.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 -- 172.21.15.71:0/2612802181 shutdown_connections 2023-10-26T05:57:08.915 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.903+0000 7fd1bedca640 1 -- 172.21.15.71:0/2612802181 wait complete. 2023-10-26T05:57:08.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 Processor -- start 2023-10-26T05:57:08.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- start start 2023-10-26T05:57:08.917 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b80b2890 0x7fd1b813acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 0x7fd1b81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1b813c2f0 0x7fd1b8137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b813f770 con 0x7fd1b80cb010 2023-10-26T05:57:08.918 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b813e1a0 con 0x7fd1b80b2890 2023-10-26T05:57:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b813f1b0 con 0x7fd1b813c2f0 2023-10-26T05:57:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 0x7fd1b81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 0x7fd1b81502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35942/0 (socket says 172.21.15.71:35942) 2023-10-26T05:57:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 -- 172.21.15.71:0/1373494017 learned_addr learned my addr 172.21.15.71:0/1373494017 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bcb66640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b80b2890 0x7fd1b813acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.919 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bd367640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1b813c2f0 0x7fd1b8137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 -- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1b813c2f0 msgr2=0x7fd1b8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1b813c2f0 0x7fd1b8137810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 -- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b80b2890 msgr2=0x7fd1b813acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b80b2890 0x7fd1b813acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 -- 172.21.15.71:0/1373494017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac009560 con 0x7fd1b80cb010 2023-10-26T05:57:08.920 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b7fff640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 0x7fd1b81502d0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac01c6b0 tx=0x7fd1ac0a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0b6070 con 0x7fd1b80cb010 2023-10-26T05:57:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd1ac004020 con 0x7fd1b80cb010 2023-10-26T05:57:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b813f490 con 0x7fd1b80cb010 2023-10-26T05:57:08.921 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac0043f0 con 0x7fd1b80cb010 2023-10-26T05:57:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b8137d50 con 0x7fd1b80cb010 2023-10-26T05:57:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b8066b10 con 0x7fd1b80cb010 2023-10-26T05:57:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd1ac0050c0 con 0x7fd1b80cb010 2023-10-26T05:57:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b57fa640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd1a40664a0 0x7fd1a4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:08.922 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.907+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1ac11e180 con 0x7fd1b80cb010 2023-10-26T05:57:08.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.911+0000 7fd1bcb66640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd1a40664a0 0x7fd1a4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:08.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.911+0000 7fd1bcb66640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd1a40664a0 0x7fd1a4068960 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005fd0 tx=0x7fd1a8005d10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:08.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:08.911+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd1ac0eb690 con 0x7fd1b80cb010 2023-10-26T05:57:09.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1b813dc90 con 0x7fd1b80cb010 2023-10-26T05:57:09.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1b57fa640 1 -- 172.21.15.71:0/1373494017 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd1ac0ac060 con 0x7fd1b80cb010 2023-10-26T05:57:09.107 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:57:09.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd1a40664a0 msgr2=0x7fd1a4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd1a40664a0 0x7fd1a4068960 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005fd0 tx=0x7fd1a8005d10 comp rx=0 tx=0).stop 2023-10-26T05:57:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 msgr2=0x7fd1b81502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 0x7fd1b81502d0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac01c6b0 tx=0x7fd1ac0a6590 comp rx=0 tx=0).stop 2023-10-26T05:57:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 shutdown_connections 2023-10-26T05:57:09.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1b813c2f0 0x7fd1b8137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:09.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd1a40664a0 0x7fd1a4068960 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:09.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1b80cb010 0x7fd1b81502d0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:09.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 --2- 172.21.15.71:0/1373494017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b80b2890 0x7fd1b813acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:09.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 >> 172.21.15.71:0/1373494017 conn(0x7fd1b8069770 msgr2=0x7fd1b80aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:09.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 shutdown_connections 2023-10-26T05:57:09.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:09.099+0000 7fd1bedca640 1 -- 172.21.15.71:0/1373494017 wait complete. 2023-10-26T05:57:09.125 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:57:09.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:57:09.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:57:09.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:57:09.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:09.126 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:57:09.127 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:57:09.127 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:57:09.127 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:57:09.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.128 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:57:09.134 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:57:09.134 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:57:09.135 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:57:09.135 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.135 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:57:09.135 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.135 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:57:09.135 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:09.136 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:57:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:57:09.136 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:09.136 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:57:09.136 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:57:09.136 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:09.137 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:57:09.137 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:09.137 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:57:09.137 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 41948sec preferred_lft 41948sec 2023-10-26T05:57:09.137 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:57:09.138 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.138 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:57:09.138 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:09.138 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:57:09.138 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.139 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:57:09.139 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:09.139 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:57:09.139 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.139 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:57:09.140 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.140 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:57:09.140 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:09.140 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:57:09.140 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.141 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:57:09.141 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:09.141 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:57:09.141 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:09.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.142 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:57:09.186 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:57:09.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.188 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:57:09.240 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:57:09.241 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:57:09.241 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:57:09.241 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:57:09.241 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:57:09.241 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:57:09.242 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:57:09.242 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:57:09.242 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:57:09.242 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:57:09.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.243 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:57:09.243 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:57:09.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:57:09.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:57:09.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:57:09.245 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:57:09.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.310 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:57:09.358 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:57:09.358 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:57:09.359 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:57:09.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.359 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:57:09.414 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:57:09.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:57:09.415 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:09.462 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:57:09.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:57:09.463 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:09.510 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:57:09.559 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:57:09.559 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:57:09.560 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:57:09.560 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:57:09.560 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:57:09.560 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:57:09.560 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:57:09.561 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:57:09.561 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:57:09.561 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:57:09.561 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:57:09.561 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:57:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:57:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:57:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:57:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:57:09.562 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:57:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:57:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:57:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:57:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:57:09.563 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:57:09.564 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:57:09.564 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:57:09.564 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:57:09.564 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:57:09.564 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:57:09.564 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:57:09.565 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:57:09.565 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:09.565 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:09.565 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:09.565 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:09.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.567 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:57:09.614 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:57:09.615 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:57:09.660 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:57:09.666 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:57:09.666 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:57:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:57:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:57:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:57:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:57:09.667 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:57:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:57:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:57:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:57:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:57:09.668 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:57:09.669 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:57:09.669 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:57:09.669 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:57:09.669 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:57:09.669 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:57:09.670 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:57:09.670 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:57:09.670 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:57:09.670 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:57:09.670 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:57:09.671 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:57:09.671 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:57:09.671 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:57:09.671 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:57:09.671 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:57:09.672 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:57:09.672 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:57:09.672 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:09.672 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:09.672 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:09.672 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:09.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:09.674 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:57:09.739 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:57:09.731+0000 7f1b9f904580 -1 init, newargv = 0x5617d7049200 newargc=15 2023-10-26T05:57:09.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[53152]: starting ceph client 2023-10-26T05:57:09.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[53152]: starting fuse 2023-10-26T05:57:10.823 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:57:10.832 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:57:10.832 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:57:10.832 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:57:10.832 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:57:10.832 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:57:10.833 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:57:10.833 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:57:10.833 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:57:10.833 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:57:10.833 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:57:10.834 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:57:10.834 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:57:10.834 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:57:10.834 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:57:10.834 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:57:10.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:57:10.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:57:10.835 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:57:10.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:57:10.835 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:57:10.835 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:57:10.836 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:57:10.836 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:57:10.836 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:57:10.836 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:57:10.836 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:57:10.836 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:57:10.837 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:57:10.837 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:57:10.837 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:10.837 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:57:10.837 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:10.838 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:57:10.838 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:10.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:10.839 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:57:10.886 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:57:10.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:57:10.887 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:57:10.888 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:57:10.888 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:57:10.888 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:57:10.889 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:57:10.889 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:10.890 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:57:10.890 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:57:10.891 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:57:10.891 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:57:10.891 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:10.892 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:57:10.892 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:57:10.892 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:57:10.893 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:57:10.893 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:10.894 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:57:10.894 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:57:10.894 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:57:10.895 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:57:10.895 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:57:10.895 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:57:10.896 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:57:10.896 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:57:10.897 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:10.897 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:57:10.897 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:57:11.012 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.53152.asok 2023-10-26T05:57:11.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:11.023 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53152.asok 2023-10-26T05:57:11.024 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:57:11.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.53152.asok status 2023-10-26T05:57:11.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:57:11.220 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:57:11.220 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:57:11.220 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:57:11.221 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:57:11.221 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:57:11.221 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:57:11.221 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:57:11.221 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "53152", 2023-10-26T05:57:11.222 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:57:11.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:57:11.222 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:57:11.222 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:57:11.222 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 9144, 2023-10-26T05:57:11.223 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:57:11.223 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:57:11.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:57:11.223 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 9144 2023-10-26T05:57:11.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:57:11.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:57:11.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:57:11.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:57:11.224 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2318314113 2023-10-26T05:57:11.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:57:11.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:57:11.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:57:11.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:57:11.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:57:11.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2318314113 2023-10-26T05:57:11.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:57:11.226 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.9144 192.168.0.1:0/2318314113", 2023-10-26T05:57:11.226 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/2318314113", 2023-10-26T05:57:11.226 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:57:11.226 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 64, 2023-10-26T05:57:11.226 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 172, 2023-10-26T05:57:11.227 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:57:11.227 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:57:11.227 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:57:11.227 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:57:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:11.240 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:57:11.241 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:57:11.241 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:11.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:57:11.287 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:57:11.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:11.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:11.294 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:57:11.294 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:57:11.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:57:11.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:11.395 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:57:11.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 -- 172.21.15.71:0/50070804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46dc005190 msgr2=0x7f46dc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:11.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 --2- 172.21.15.71:0/50070804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46dc005190 0x7f46dc00b990 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f46e8001d70 tx=0x7f46e801cae0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 -- 172.21.15.71:0/50070804 shutdown_connections 2023-10-26T05:57:11.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 --2- 172.21.15.71:0/50070804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46dc005190 0x7f46dc00b990 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 -- 172.21.15.71:0/50070804 >> 172.21.15.71:0/50070804 conn(0x7f46f0069770 msgr2=0x7f46f0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:11.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 -- 172.21.15.71:0/50070804 shutdown_connections 2023-10-26T05:57:11.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 -- 172.21.15.71:0/50070804 wait complete. 2023-10-26T05:57:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 Processor -- start 2023-10-26T05:57:11.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.511+0000 7f46f7af2640 1 -- start start 2023-10-26T05:57:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46dc005190 0x7f46f013c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f013c380 0x7f46f013bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 0x7f46f00aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f01e1a30 con 0x7f46f00aa050 2023-10-26T05:57:11.526 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00ab700 con 0x7f46f013c380 2023-10-26T05:57:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f00a9610 con 0x7f46dc005190 2023-10-26T05:57:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 0x7f46f00aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 0x7f46f00aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35964/0 (socket says 172.21.15.71:35964) 2023-10-26T05:57:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 -- 172.21.15.71:0/929712543 learned_addr learned my addr 172.21.15.71:0/929712543 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:11.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f508d640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f013c380 0x7f46f013bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f588e640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46dc005190 0x7f46f013c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 -- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46dc005190 msgr2=0x7f46f013c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46dc005190 0x7f46f013c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 -- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f013c380 msgr2=0x7f46f013bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:11.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f013c380 0x7f46f013bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 -- 172.21.15.71:0/929712543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f01e1220 con 0x7f46f00aa050 2023-10-26T05:57:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f508d640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f013c380 0x7f46f013bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:57:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f608f640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 0x7f46f00aa500 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f46e801d380 tx=0x7f46e8004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e80a9030 con 0x7f46f00aa050 2023-10-26T05:57:11.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f46e80b4070 con 0x7f46f00aa050 2023-10-26T05:57:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f01e2e20 con 0x7f46f00aa050 2023-10-26T05:57:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8006540 con 0x7f46f00aa050 2023-10-26T05:57:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f01e1750 con 0x7f46f00aa050 2023-10-26T05:57:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f0066b10 con 0x7f46f00aa050 2023-10-26T05:57:11.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f46e80ac070 con 0x7f46f00aa050 2023-10-26T05:57:11.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46e67fc640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46d40664a0 0x7f46d4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:11.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f46e80aa070 con 0x7f46f00aa050 2023-10-26T05:57:11.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.515+0000 7f46f588e640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46d40664a0 0x7f46d4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:11.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.519+0000 7f46f588e640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46d40664a0 0x7f46d4068960 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0018f0 tx=0x7f46ec001ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:11.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.519+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f46e80ebb80 con 0x7f46f00aa050 2023-10-26T05:57:11.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.711+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46f00721e0 con 0x7f46f00aa050 2023-10-26T05:57:11.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.711+0000 7f46e67fc640 1 -- 172.21.15.71:0/929712543 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f46e80be070 con 0x7f46f00aa050 2023-10-26T05:57:11.721 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:57:11.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46d40664a0 msgr2=0x7f46d4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:11.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46d40664a0 0x7f46d4068960 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0018f0 tx=0x7f46ec001ca0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 msgr2=0x7f46f00aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:11.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 0x7f46f00aa500 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f46e801d380 tx=0x7f46e8004f50 comp rx=0 tx=0).stop 2023-10-26T05:57:11.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 shutdown_connections 2023-10-26T05:57:11.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f46d40664a0 0x7f46d4068960 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46f00aa050 0x7f46f00aa500 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46f013c380 0x7f46f013bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 --2- 172.21.15.71:0/929712543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f46dc005190 0x7f46f013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:11.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 >> 172.21.15.71:0/929712543 conn(0x7f46f0069770 msgr2=0x7f46f01593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:11.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 shutdown_connections 2023-10-26T05:57:11.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:11.715+0000 7f46f7af2640 1 -- 172.21.15.71:0/929712543 wait complete. 2023-10-26T05:57:11.738 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:57:11.739 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:57:11.739 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:57:11.740 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:57:11.740 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:11.740 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:57:11.741 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:57:11.741 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:57:11.741 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:57:11.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:11.742 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:57:11.749 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:57:11.750 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:57:11.750 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:57:11.750 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.750 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:57:11.750 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.751 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:57:11.751 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:11.751 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:57:11.751 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:57:11.751 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:11.752 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:57:11.752 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:57:11.752 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:11.752 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:57:11.752 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:11.753 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:57:11.753 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41939sec preferred_lft 41939sec 2023-10-26T05:57:11.753 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:57:11.753 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.753 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:57:11.753 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:11.754 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:57:11.754 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.754 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:57:11.754 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:57:11.754 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:57:11.755 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.755 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:57:11.755 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.755 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:57:11.755 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:11.756 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:57:11.756 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.756 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:57:11.756 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:11.756 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:57:11.756 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:57:11.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:11.757 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:57:11.802 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:57:11.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:11.804 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:57:11.855 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:57:11.856 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:57:11.856 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:57:11.856 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:57:11.856 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:57:11.856 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:57:11.857 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:57:11.857 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:57:11.857 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:57:11.858 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:57:11.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:11.859 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:57:11.859 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:57:11.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:57:11.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:57:11.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:57:11.860 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:57:11.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:11.932 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:57:11.980 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:57:11.980 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:57:11.980 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:57:11.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:11.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:57:12.037 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:57:12.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:57:12.038 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:12.084 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:57:12.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:57:12.085 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:12.131 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:57:12.181 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:57:12.181 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:57:12.181 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:57:12.181 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:57:12.182 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:57:12.182 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:57:12.182 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:57:12.182 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:57:12.182 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:57:12.183 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:57:12.183 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:57:12.183 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:57:12.183 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:57:12.183 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:57:12.184 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:57:12.184 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:57:12.184 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:57:12.184 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:57:12.184 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:57:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:57:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:57:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:57:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:57:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:57:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:57:12.186 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:57:12.186 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:57:12.186 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:57:12.186 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:57:12.186 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:12.187 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:12.187 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:12.187 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:12.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:12.188 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:57:12.236 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:57:12.236 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:57:12.281 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:57:12.288 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:57:12.288 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:57:12.288 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:57:12.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:57:12.289 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:57:12.289 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:57:12.289 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:57:12.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:57:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:57:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:57:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:57:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:57:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:57:12.290 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:57:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:57:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:57:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:57:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:57:12.291 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:57:12.292 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:57:12.292 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:57:12.292 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:57:12.292 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:57:12.292 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:57:12.292 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:57:12.293 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:57:12.293 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:57:12.293 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:57:12.293 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:57:12.293 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:12.293 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:12.294 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:12.294 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:12.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:12.295 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:57:12.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:57:12.365+0000 7f82782d5580 -1 init, newargv = 0x5570d247f410 newargc=15 2023-10-26T05:57:12.367 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35369]: starting ceph client 2023-10-26T05:57:12.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35369]: starting fuse 2023-10-26T05:57:13.441 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:57:13.451 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:57:13.451 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:57:13.451 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:57:13.451 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:57:13.452 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:57:13.452 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:57:13.452 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:57:13.452 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:57:13.452 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:57:13.453 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:57:13.453 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:57:13.453 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:57:13.453 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:57:13.453 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:57:13.453 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:57:13.454 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:57:13.454 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:57:13.454 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:57:13.454 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:57:13.454 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:57:13.455 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:57:13.455 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:57:13.455 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:57:13.455 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:57:13.455 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:57:13.456 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:57:13.456 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:57:13.456 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:57:13.456 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:57:13.456 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:13.457 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:57:13.457 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:13.457 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:57:13.457 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:13.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:57:13.461 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:57:13.504 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:57:13.504 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:57:13.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:57:13.505 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:57:13.505 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:57:13.506 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:57:13.506 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:57:13.507 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:57:13.507 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:57:13.507 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:57:13.508 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:57:13.508 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:57:13.508 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:57:13.509 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:57:13.509 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:57:13.510 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:57:13.510 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:57:13.510 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:57:13.511 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:57:13.511 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:57:13.511 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:57:13.512 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:57:13.512 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:57:13.513 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:57:13.513 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:57:13.513 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:57:13.514 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:57:13.514 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:57:13.515 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:57:13.630 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.35369.asok 2023-10-26T05:57:13.634 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:13.642 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35369.asok 2023-10-26T05:57:13.643 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:57:13.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.35369.asok status 2023-10-26T05:57:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:57:13.832 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:57:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:57:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:57:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:57:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:57:13.833 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:57:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:57:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "35369", 2023-10-26T05:57:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:57:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:57:13.834 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:57:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:57:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6215, 2023-10-26T05:57:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:57:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:57:13.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:57:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 6215 2023-10-26T05:57:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:57:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:57:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:57:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:57:13.836 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3239678659 2023-10-26T05:57:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:57:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:57:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:57:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:57:13.837 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:57:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3239678659 2023-10-26T05:57:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:57:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.6215 192.168.0.1:0/3239678659", 2023-10-26T05:57:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3239678659", 2023-10-26T05:57:13.838 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:57:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 64, 2023-10-26T05:57:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 174, 2023-10-26T05:57:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:57:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:57:13.839 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:57:13.839 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:57:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:13.851 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:57:13.852 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:57:13.852 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:13.896 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:57:13.897 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:57:13.898 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:13.904 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:13.904 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:57:13.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:57:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:57:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:57:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:57:14.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:57:14.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb234ba4640 1 -- 172.21.15.71:0/179231627 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200a5e40 con 0x7fb230069770 2023-10-26T05:57:14.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- 172.21.15.71:0/179231627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb230069770 msgr2=0x7fb23006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 --2- 172.21.15.71:0/179231627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb230069770 0x7fb23006a110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb220009900 tx=0x7fb22001c830 comp rx=0 tx=0).stop 2023-10-26T05:57:14.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- 172.21.15.71:0/179231627 shutdown_connections 2023-10-26T05:57:14.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 --2- 172.21.15.71:0/179231627 >> v2:172.21.15.86:3300/0 conn(0x7fb23006a650 0x7fb2300cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 --2- 172.21.15.71:0/179231627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb230069770 0x7fb23006a110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- 172.21.15.71:0/179231627 >> 172.21.15.71:0/179231627 conn(0x7fb2300d12e0 msgr2=0x7fb2300d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:14.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- 172.21.15.71:0/179231627 shutdown_connections 2023-10-26T05:57:14.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- 172.21.15.71:0/179231627 wait complete. 2023-10-26T05:57:14.146 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 Processor -- start 2023-10-26T05:57:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- start start 2023-10-26T05:57:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb23006a650 0x7fb2300b8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23015b3a0 0x7fb2300b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 0x7fb2301598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300b21a0 con 0x7fb2300b1c40 2023-10-26T05:57:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300b2480 con 0x7fb23015b3a0 2023-10-26T05:57:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb237e0a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301529e0 con 0x7fb23006a650 2023-10-26T05:57:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 0x7fb2301598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 0x7fb2301598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34274/0 (socket says 172.21.15.71:34274) 2023-10-26T05:57:14.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 -- 172.21.15.71:0/3149128264 learned_addr learned my addr 172.21.15.71:0/3149128264 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2353a5640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23015b3a0 0x7fb2300b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb235ba6640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb23006a650 0x7fb2300b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 -- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb23006a650 msgr2=0x7fb2300b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb23006a650 0x7fb2300b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 -- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23015b3a0 msgr2=0x7fb2300b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23015b3a0 0x7fb2300b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 -- 172.21.15.71:0/3149128264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009820 con 0x7fb2300b1c40 2023-10-26T05:57:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.135+0000 7fb2363a7640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 0x7fb2301598d0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb22800efd0 tx=0x7fb22800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228009b80 con 0x7fb2300b1c40 2023-10-26T05:57:14.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb220009560 con 0x7fb2300b1c40 2023-10-26T05:57:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb228003b80 con 0x7fb2300b1c40 2023-10-26T05:57:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228004020 con 0x7fb2300b1c40 2023-10-26T05:57:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300b8cd0 con 0x7fb2300b1c40 2023-10-26T05:57:14.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230066b10 con 0x7fb2300b1c40 2023-10-26T05:57:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb228004180 con 0x7fb2300b1c40 2023-10-26T05:57:14.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb2267fc640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb214066470 0x7fb214068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb228014070 con 0x7fb2300b1c40 2023-10-26T05:57:14.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb235ba6640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb214066470 0x7fb214068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.139+0000 7fb235ba6640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb214066470 0x7fb214068930 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0015b0 tx=0x7fb22c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:14.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.143+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb22808a020 con 0x7fb2300b1c40 2023-10-26T05:57:14.342 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb230152f00 con 0x7fb2300b1c40 2023-10-26T05:57:14.343 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb2267fc640 1 -- 172.21.15.71:0/3149128264 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fb230152f00 con 0x7fb2300b1c40 2023-10-26T05:57:14.343 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:57:14.344 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:57:14.344 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:57:14.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb214066470 msgr2=0x7fb214068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb214066470 0x7fb214068930 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0015b0 tx=0x7fb22c009040 comp rx=0 tx=0).stop 2023-10-26T05:57:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 msgr2=0x7fb2301598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 0x7fb2301598d0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb22800efd0 tx=0x7fb22800c6a0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 shutdown_connections 2023-10-26T05:57:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb214066470 0x7fb214068930 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2300b1c40 0x7fb2301598d0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb23015b3a0 0x7fb2300b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 --2- 172.21.15.71:0/3149128264 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb23006a650 0x7fb2300b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 >> 172.21.15.71:0/3149128264 conn(0x7fb2300d12e0 msgr2=0x7fb2300b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 shutdown_connections 2023-10-26T05:57:14.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.335+0000 7fb237e0a640 1 -- 172.21.15.71:0/3149128264 wait complete. 2023-10-26T05:57:14.362 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-10-26T05:57:14.500 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":8958,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.804477809,"mdsmap_epoch":64,"osdmap_epoch":174,"osdmap_epoch_barrier":172,"uptime":20.587896778000001} 2023-10-26T05:57:14.501 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 8958, "mdsmap_epoch": 64, "osdmap_epoch": 174, "osdmap_epoch_barrier": 172, "rank_uptime": 14.804477809, "state": "up:active", "uptime": 20.587896778, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:57:14.502 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T05:57:14.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- 172.21.15.71:0/4084759479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013efd0 msgr2=0x7ff000069d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 --2- 172.21.15.71:0/4084759479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013efd0 0x7ff000069d30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7feff8009c80 tx=0x7feff801cd10 comp rx=0 tx=0).stop 2023-10-26T05:57:14.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- 172.21.15.71:0/4084759479 shutdown_connections 2023-10-26T05:57:14.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 --2- 172.21.15.71:0/4084759479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013efd0 0x7ff000069d30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 --2- 172.21.15.71:0/4084759479 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff00013c190 0x7ff0000cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- 172.21.15.71:0/4084759479 >> 172.21.15.71:0/4084759479 conn(0x7ff0000aec60 msgr2=0x7ff0000af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:14.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- 172.21.15.71:0/4084759479 shutdown_connections 2023-10-26T05:57:14.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- 172.21.15.71:0/4084759479 wait complete. 2023-10-26T05:57:14.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 Processor -- start 2023-10-26T05:57:14.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- start start 2023-10-26T05:57:14.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0000cbd60 0x7ff000135ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 0x7ff00015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff00013efd0 0x7ff000152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00015b710 con 0x7ff00013c190 2023-10-26T05:57:14.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000136020 con 0x7ff0000cbd60 2023-10-26T05:57:14.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff008462640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00015ab00 con 0x7ff00013efd0 2023-10-26T05:57:14.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 0x7ff00015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0061fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0000cbd60 0x7ff000135ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0069ff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff00013efd0 0x7ff000152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0061fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0000cbd60 0x7ff000135ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.71:34028/0 (socket says 172.21.15.71:34028) 2023-10-26T05:57:14.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0061fe640 1 -- 172.21.15.71:0/1948524696 learned_addr learned my addr 172.21.15.71:0/1948524696 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:57:14.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 0x7ff00015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34288/0 (socket says 172.21.15.71:34288) 2023-10-26T05:57:14.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 -- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff00013efd0 msgr2=0x7ff000152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff00013efd0 0x7ff000152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 -- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0000cbd60 msgr2=0x7ff000135ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:14.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0000cbd60 0x7ff000135ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:14.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 -- 172.21.15.71:0/1948524696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80098e0 con 0x7ff00013c190 2023-10-26T05:57:14.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7ff0059fd640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 0x7ff00015a120 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feff000c3e0 tx=0x7feff000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:14.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.623+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0011790 con 0x7ff00013c190 2023-10-26T05:57:14.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7feff00119e0 con 0x7ff00013c190 2023-10-26T05:57:14.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff00015ade0 con 0x7ff00013c190 2023-10-26T05:57:14.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0011e30 con 0x7ff00013c190 2023-10-26T05:57:14.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff00015b3a0 con 0x7ff00013c190 2023-10-26T05:57:14.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff000066b10 con 0x7ff00013c190 2023-10-26T05:57:14.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feff001c460 con 0x7ff00013c190 2023-10-26T05:57:14.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7feff6ffd640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fefe40664a0 0x7fefe4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:57:14.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5738+0+0 (secure 0 0 0) 0x7feff0083de0 con 0x7ff00013c190 2023-10-26T05:57:14.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7ff0061fe640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fefe40664a0 0x7fefe4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:57:14.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.627+0000 7ff0061fe640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fefe40664a0 0x7fefe4068960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7feffc009a50 tx=0x7feffc009210 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:57:14.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.631+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7feff0051380 con 0x7ff00013c190 2023-10-26T05:57:14.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:14.787+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7ff00013ff10 con 0x7ff00013c190 2023-10-26T05:57:15.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.319+0000 7feff6ffd640 1 -- 172.21.15.71:0/1948524696 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7feff00549a0 con 0x7ff00013c190 2023-10-26T05:57:15.328 INFO:teuthology.orchestra.run.smithi071.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T05:57:15.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fefe40664a0 msgr2=0x7fefe4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:15.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fefe40664a0 0x7fefe4068960 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7feffc009a50 tx=0x7feffc009210 comp rx=0 tx=0).stop 2023-10-26T05:57:15.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 msgr2=0x7ff00015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:57:15.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 0x7ff00015a120 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7feff000c3e0 tx=0x7feff000c8a0 comp rx=0 tx=0).stop 2023-10-26T05:57:15.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 shutdown_connections 2023-10-26T05:57:15.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff00013efd0 0x7ff000152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:15.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fefe40664a0 0x7fefe4068960 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:15.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00013c190 0x7ff00015a120 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:15.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 --2- 172.21.15.71:0/1948524696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0000cbd60 0x7ff000135ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:57:15.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 >> 172.21.15.71:0/1948524696 conn(0x7ff0000aec60 msgr2=0x7ff0000b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:57:15.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 shutdown_connections 2023-10-26T05:57:15.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:57:15.323+0000 7ff008462640 1 -- 172.21.15.71:0/1948524696 wait complete. 2023-10-26T05:57:15.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:57:15.491 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:57:15.492 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:57:15.493 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-10-26T05:57:15.624 INFO:teuthology.orchestra.run.smithi071.stdout:{"mon_tick_interval":"5"} 2023-10-26T05:57:15.624 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-10-26T05:57:15.625 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-10-26T05:57:15.625 DEBUG:teuthology.orchestra.run.smithi071:> stdin-killer -- python3 -c ' 2023-10-26T05:57:15.625 DEBUG:teuthology.orchestra.run.smithi071:> import time 2023-10-26T05:57:15.626 DEBUG:teuthology.orchestra.run.smithi071:> import datetime 2023-10-26T05:57:15.626 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:57:15.627 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:57:15.627 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.627 DEBUG:teuthology.orchestra.run.smithi071:> # Write some buffered data through before going full, all should be well 2023-10-26T05:57:15.628 DEBUG:teuthology.orchestra.run.smithi071:> print("writing some data through which we expect to succeed") 2023-10-26T05:57:15.628 DEBUG:teuthology.orchestra.run.smithi071:> bytes = 0 2023-10-26T05:57:15.628 DEBUG:teuthology.orchestra.run.smithi071:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-10-26T05:57:15.629 DEBUG:teuthology.orchestra.run.smithi071:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-10-26T05:57:15.629 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-10-26T05:57:15.630 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:57:15.630 DEBUG:teuthology.orchestra.run.smithi071:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-10-26T05:57:15.630 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.631 DEBUG:teuthology.orchestra.run.smithi071:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-10-26T05:57:15.631 DEBUG:teuthology.orchestra.run.smithi071:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-10-26T05:57:15.631 DEBUG:teuthology.orchestra.run.smithi071:> # buffered IO, the error should always only appear from fsync and not 2023-10-26T05:57:15.632 DEBUG:teuthology.orchestra.run.smithi071:> # from write 2023-10-26T05:57:15.632 DEBUG:teuthology.orchestra.run.smithi071:> full = False 2023-10-26T05:57:15.633 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.633 DEBUG:teuthology.orchestra.run.smithi071:> for n in range(0, int(32 * 1.1)): 2023-10-26T05:57:15.634 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:57:15.634 DEBUG:teuthology.orchestra.run.smithi071:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-10-26T05:57:15.634 DEBUG:teuthology.orchestra.run.smithi071:> print("wrote bytes via buffered write, moving on to fsync") 2023-10-26T05:57:15.635 DEBUG:teuthology.orchestra.run.smithi071:> except OSError as e: 2023-10-26T05:57:15.635 DEBUG:teuthology.orchestra.run.smithi071:> if True: 2023-10-26T05:57:15.635 DEBUG:teuthology.orchestra.run.smithi071:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-10-26T05:57:15.636 DEBUG:teuthology.orchestra.run.smithi071:> raise 2023-10-26T05:57:15.636 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:57:15.637 DEBUG:teuthology.orchestra.run.smithi071:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T05:57:15.637 DEBUG:teuthology.orchestra.run.smithi071:> full = True 2023-10-26T05:57:15.637 DEBUG:teuthology.orchestra.run.smithi071:> break 2023-10-26T05:57:15.638 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.638 DEBUG:teuthology.orchestra.run.smithi071:> try: 2023-10-26T05:57:15.638 DEBUG:teuthology.orchestra.run.smithi071:> os.fsync(f) 2023-10-26T05:57:15.639 DEBUG:teuthology.orchestra.run.smithi071:> print("fsync'"'"'ed successfully") 2023-10-26T05:57:15.639 DEBUG:teuthology.orchestra.run.smithi071:> except OSError as e: 2023-10-26T05:57:15.640 DEBUG:teuthology.orchestra.run.smithi071:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T05:57:15.640 DEBUG:teuthology.orchestra.run.smithi071:> full = True 2023-10-26T05:57:15.640 DEBUG:teuthology.orchestra.run.smithi071:> break 2023-10-26T05:57:15.641 DEBUG:teuthology.orchestra.run.smithi071:> else: 2023-10-26T05:57:15.641 DEBUG:teuthology.orchestra.run.smithi071:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-10-26T05:57:15.641 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.642 DEBUG:teuthology.orchestra.run.smithi071:> if n > 32 * 0.9: 2023-10-26T05:57:15.642 DEBUG:teuthology.orchestra.run.smithi071:> # Be cautious in the last region where we expect to hit 2023-10-26T05:57:15.642 DEBUG:teuthology.orchestra.run.smithi071:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-10-26T05:57:15.643 DEBUG:teuthology.orchestra.run.smithi071:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-10-26T05:57:15.643 DEBUG:teuthology.orchestra.run.smithi071:> time.sleep(15.0) 2023-10-26T05:57:15.643 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.644 DEBUG:teuthology.orchestra.run.smithi071:> if not full: 2023-10-26T05:57:15.644 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-10-26T05:57:15.645 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.645 DEBUG:teuthology.orchestra.run.smithi071:> # close() should not raise an error because we already caught it in 2023-10-26T05:57:15.645 DEBUG:teuthology.orchestra.run.smithi071:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-10-26T05:57:15.646 DEBUG:teuthology.orchestra.run.smithi071:> # since then because all IOs got cancelled on the full flag. 2023-10-26T05:57:15.646 DEBUG:teuthology.orchestra.run.smithi071:> print("calling close") 2023-10-26T05:57:15.646 DEBUG:teuthology.orchestra.run.smithi071:> os.close(f) 2023-10-26T05:57:15.647 DEBUG:teuthology.orchestra.run.smithi071:> print("close() did not raise error") 2023-10-26T05:57:15.647 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:57:15.647 DEBUG:teuthology.orchestra.run.smithi071:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-10-26T05:57:15.647 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:57:16.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:57:16.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:57:46.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:57:46.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:58:04.312 INFO:teuthology.orchestra.run.smithi071.stdout:writing some data through which we expect to succeed 2023-10-26T05:58:04.312 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-10-26T05:58:04.312 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.312 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.313 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 1.00 MB 2023-10-26T05:58:04.313 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.313 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.313 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 2.00 MB 2023-10-26T05:58:04.313 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.314 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.314 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 3.00 MB 2023-10-26T05:58:04.314 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.314 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.315 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 4.00 MB 2023-10-26T05:58:04.315 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.315 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.315 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 5.00 MB 2023-10-26T05:58:04.315 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.316 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.316 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 6.00 MB 2023-10-26T05:58:04.316 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.316 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.316 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 7.00 MB 2023-10-26T05:58:04.316 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.317 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.317 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 8.00 MB 2023-10-26T05:58:04.317 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.317 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.317 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 9.00 MB 2023-10-26T05:58:04.318 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.318 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.318 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 10.00 MB 2023-10-26T05:58:04.318 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.318 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.319 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 11.00 MB 2023-10-26T05:58:04.319 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.319 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.319 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 12.00 MB 2023-10-26T05:58:04.319 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.320 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.320 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 13.00 MB 2023-10-26T05:58:04.320 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.320 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.320 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 14.00 MB 2023-10-26T05:58:04.321 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.321 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.321 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 15.00 MB 2023-10-26T05:58:04.321 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.321 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.321 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 16.00 MB 2023-10-26T05:58:04.322 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.322 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.322 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 17.00 MB 2023-10-26T05:58:04.322 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.322 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.323 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 18.00 MB 2023-10-26T05:58:04.323 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.323 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.323 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 19.00 MB 2023-10-26T05:58:04.323 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.324 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.324 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 20.00 MB 2023-10-26T05:58:04.324 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.324 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.324 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 21.00 MB 2023-10-26T05:58:04.325 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.325 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.325 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 22.00 MB 2023-10-26T05:58:04.325 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.325 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.325 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 23.00 MB 2023-10-26T05:58:04.326 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.326 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.326 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 24.00 MB 2023-10-26T05:58:04.326 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.326 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.327 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 25.00 MB 2023-10-26T05:58:04.327 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.327 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.327 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 26.00 MB 2023-10-26T05:58:04.327 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.328 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.328 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 27.00 MB 2023-10-26T05:58:04.328 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.328 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.328 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 28.00 MB 2023-10-26T05:58:04.328 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.329 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.329 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 29.00 MB 2023-10-26T05:58:04.329 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.329 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.329 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 30.00 MB 2023-10-26T05:58:04.330 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:58:04.330 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.330 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.330 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 31.00 MB 2023-10-26T05:58:04.330 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:58:04.331 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.331 INFO:teuthology.orchestra.run.smithi071.stdout:fsync'ed successfully 2023-10-26T05:58:04.331 INFO:teuthology.orchestra.run.smithi071.stdout:Not full yet after 32.00 MB 2023-10-26T05:58:04.331 INFO:teuthology.orchestra.run.smithi071.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-10-26T05:58:04.331 INFO:teuthology.orchestra.run.smithi071.stdout:wrote bytes via buffered write, moving on to fsync 2023-10-26T05:58:04.332 INFO:teuthology.orchestra.run.smithi071.stdout:Reached fullness after 33.00 MB 2023-10-26T05:58:04.332 INFO:teuthology.orchestra.run.smithi071.stdout:calling close 2023-10-26T05:58:04.332 INFO:teuthology.orchestra.run.smithi071.stdout:close() did not raise error 2023-10-26T05:58:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:04.334 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:58:04.334 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:58:04.384 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:58:04.433 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:58:04.433 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:04.434 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:04.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:04.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/646306117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 msgr2=0x7f9fe415c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- 172.21.15.71:0/646306117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe415c760 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9fe406fa60 tx=0x7f9fe001c870 comp rx=0 tx=0).stop 2023-10-26T05:58:04.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/646306117 shutdown_connections 2023-10-26T05:58:04.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- 172.21.15.71:0/646306117 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe40b0040 0x7f9fe40b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- 172.21.15.71:0/646306117 >> v2:172.21.15.86:3300/0 conn(0x7f9fe4070fe0 0x7f9fe4071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.615 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- 172.21.15.71:0/646306117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe415c760 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/646306117 >> 172.21.15.71:0/646306117 conn(0x7f9fe4069770 msgr2=0x7f9fe4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:04.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/646306117 shutdown_connections 2023-10-26T05:58:04.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/646306117 wait complete. 2023-10-26T05:58:04.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 Processor -- start 2023-10-26T05:58:04.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- start start 2023-10-26T05:58:04.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe4070fe0 0x7f9fe413c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe40b0040 0x7f9fe4134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe4151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe413b300 con 0x7f9fe4140cc0 2023-10-26T05:58:04.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe413bb10 con 0x7f9fe4070fe0 2023-10-26T05:58:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe413b550 con 0x7f9fe40b0040 2023-10-26T05:58:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe4151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe4151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53166/0 (socket says 172.21.15.71:53166) 2023-10-26T05:58:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 -- 172.21.15.71:0/560912428 learned_addr learned my addr 172.21.15.71:0/560912428 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fe9826640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe4070fe0 0x7f9fe413c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fe9025640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe40b0040 0x7f9fe4134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 -- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe40b0040 msgr2=0x7f9fe4134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe40b0040 0x7f9fe4134f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 -- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe4070fe0 msgr2=0x7f9fe413c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe4070fe0 0x7f9fe413c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 -- 172.21.15.71:0/560912428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0009560 con 0x7f9fe4140cc0 2023-10-26T05:58:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fe9826640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe4070fe0 0x7f9fe413c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fea027640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe4151600 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cce0 tx=0x7f9fdc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00f450 con 0x7f9fe4140cc0 2023-10-26T05:58:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9fdc00f6c0 con 0x7f9fe4140cc0 2023-10-26T05:58:04.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe413b830 con 0x7f9fe4140cc0 2023-10-26T05:58:04.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00fad0 con 0x7f9fe4140cc0 2023-10-26T05:58:04.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.607+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe4154e60 con 0x7f9fe4140cc0 2023-10-26T05:58:04.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4066b10 con 0x7f9fe4140cc0 2023-10-26T05:58:04.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9fdc00fc30 con 0x7f9fe4140cc0 2023-10-26T05:58:04.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9fda7fc640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc8066470 0x7f9fc8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9fdc084a30 con 0x7f9fe4140cc0 2023-10-26T05:58:04.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f9fc806e000 con 0x7f9fe4140cc0 2023-10-26T05:58:04.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9fe9826640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc8066470 0x7f9fc8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.611+0000 7f9fe9826640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc8066470 0x7f9fc8068930 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00af000 tx=0x7f9fe001b0b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:04.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.615+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9fdc01d020 con 0x7f9fe4140cc0 2023-10-26T05:58:04.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.771+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fe4155080 con 0x7f9fe4140cc0 2023-10-26T05:58:04.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9fda7fc640 1 -- 172.21.15.71:0/560912428 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16572 (secure 0 0 0) 0x7f9fdc051f20 con 0x7f9fe4140cc0 2023-10-26T05:58:04.783 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:04.783 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":176,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:57:52.290625+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc8066470 msgr2=0x7f9fc8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc8066470 0x7f9fc8068930 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00af000 tx=0x7f9fe001b0b0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 msgr2=0x7f9fe4151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe4151600 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cce0 tx=0x7f9fdc00a620 comp rx=0 tx=0).stop 2023-10-26T05:58:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 shutdown_connections 2023-10-26T05:58:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9fc8066470 0x7f9fc8068930 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9fe4140cc0 0x7f9fe4151600 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9fe40b0040 0x7f9fe4134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 --2- 172.21.15.71:0/560912428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fe4070fe0 0x7f9fe413c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 >> 172.21.15.71:0/560912428 conn(0x7f9fe4069770 msgr2=0x7f9fe415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 shutdown_connections 2023-10-26T05:58:04.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.775+0000 7f9feba8a640 1 -- 172.21.15.71:0/560912428 wait complete. 2023-10-26T05:58:04.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:56:56.944301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:56:58.029095+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:58:04.803 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:58:04.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3406771906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 msgr2=0x7fcd380ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.927 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3406771906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd380ca780 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fcd3806e940 tx=0x7fcd3001f350 comp rx=0 tx=0).stop 2023-10-26T05:58:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3406771906 shutdown_connections 2023-10-26T05:58:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3406771906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd380ca780 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3406771906 >> v2:172.21.15.86:3300/0 conn(0x7fcd38141a40 0x7fcd380c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3406771906 >> 172.21.15.71:0/3406771906 conn(0x7fcd380aec60 msgr2=0x7fcd380af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:04.928 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3406771906 shutdown_connections 2023-10-26T05:58:04.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3406771906 wait complete. 2023-10-26T05:58:04.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 Processor -- start 2023-10-26T05:58:04.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- start start 2023-10-26T05:58:04.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd380b1230 0x7fcd3814b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd381f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd38141a40 0x7fcd3814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3814c860 con 0x7fcd380b2370 2023-10-26T05:58:04.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd38134de0 con 0x7fcd380b1230 2023-10-26T05:58:04.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3f2ab640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3814bc50 con 0x7fcd38141a40 2023-10-26T05:58:04.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3c846640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd381f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3c846640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd381f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53184/0 (socket says 172.21.15.71:53184) 2023-10-26T05:58:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3c846640 1 -- 172.21.15.71:0/3507431093 learned_addr learned my addr 172.21.15.71:0/3507431093 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3d848640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd38141a40 0x7fcd3814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.919+0000 7fcd3d047640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd380b1230 0x7fcd3814b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.933 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3c846640 1 -- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd38141a40 msgr2=0x7fcd3814ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3c846640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd38141a40 0x7fcd3814ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3c846640 1 -- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd380b1230 msgr2=0x7fcd3814b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:04.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3c846640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd380b1230 0x7fcd3814b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:04.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3c846640 1 -- 172.21.15.71:0/3507431093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30009820 con 0x7fcd380b2370 2023-10-26T05:58:04.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3d047640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd380b1230 0x7fcd3814b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3c846640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd381f2400 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c530 tx=0x7fcd2c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c012830 con 0x7fcd380b2370 2023-10-26T05:58:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd2c012a80 con 0x7fcd380b2370 2023-10-26T05:58:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3814bf30 con 0x7fcd380b2370 2023-10-26T05:58:04.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c012e90 con 0x7fcd380b2370 2023-10-26T05:58:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3814c4f0 con 0x7fcd380b2370 2023-10-26T05:58:04.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd38066b10 con 0x7fcd380b2370 2023-10-26T05:58:04.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd2c01a3b0 con 0x7fcd380b2370 2023-10-26T05:58:04.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd1c066470 0x7fcd1c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:04.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcd2c0840a0 con 0x7fcd380b2370 2023-10-26T05:58:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fcd1c06e000 con 0x7fcd380b2370 2023-10-26T05:58:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3d047640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd1c066470 0x7fcd1c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.923+0000 7fcd3d047640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd1c066470 0x7fcd1c068930 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005fd0 tx=0x7fcd34005de0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:04.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:04.927+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd2c016020 con 0x7fcd380b2370 2023-10-26T05:58:05.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd380721e0 con 0x7fcd380b2370 2023-10-26T05:58:05.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd29ffb640 1 -- 172.21.15.71:0/3507431093 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd2c0550e0 con 0x7fcd380b2370 2023-10-26T05:58:05.080 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:58:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd1c066470 msgr2=0x7fcd1c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd1c066470 0x7fcd1c068930 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcd34005fd0 tx=0x7fcd34005de0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 msgr2=0x7fcd381f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd381f2400 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c530 tx=0x7fcd2c00c9f0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 shutdown_connections 2023-10-26T05:58:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd38141a40 0x7fcd3814ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd1c066470 0x7fcd1c068930 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd380b2370 0x7fcd381f2400 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.071+0000 7fcd3f2ab640 1 --2- 172.21.15.71:0/3507431093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd380b1230 0x7fcd3814b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.075+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 >> 172.21.15.71:0/3507431093 conn(0x7fcd380aec60 msgr2=0x7fcd381578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.075+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 shutdown_connections 2023-10-26T05:58:05.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.075+0000 7fcd3f2ab640 1 -- 172.21.15.71:0/3507431093 wait complete. 2023-10-26T05:58:05.097 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:58:05.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 -- 172.21.15.71:0/369746663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3815d3f0 msgr2=0x7efd380cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/369746663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3815d3f0 0x7efd380cb820 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efd28009900 tx=0x7efd2801c7c0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 -- 172.21.15.71:0/369746663 shutdown_connections 2023-10-26T05:58:05.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/369746663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd380cbd60 0x7efd38069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/369746663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3815d3f0 0x7efd380cb820 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 -- 172.21.15.71:0/369746663 >> 172.21.15.71:0/369746663 conn(0x7efd380aec60 msgr2=0x7efd380af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 -- 172.21.15.71:0/369746663 shutdown_connections 2023-10-26T05:58:05.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.215+0000 7efd3fd3e640 1 -- 172.21.15.71:0/369746663 wait complete. 2023-10-26T05:58:05.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 Processor -- start 2023-10-26T05:58:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- start start 2023-10-26T05:58:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd380cbd60 0x7efd380b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 0x7efd3814c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd3815d3f0 0x7efd38133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38137ed0 con 0x7efd3813c220 2023-10-26T05:58:05.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38137350 con 0x7efd380cbd60 2023-10-26T05:58:05.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd38137630 con 0x7efd3815d3f0 2023-10-26T05:58:05.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 0x7efd3814c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 0x7efd3814c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53204/0 (socket says 172.21.15.71:53204) 2023-10-26T05:58:05.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 -- 172.21.15.71:0/3444100316 learned_addr learned my addr 172.21.15.71:0/3444100316 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:05.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3e2db640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd3815d3f0 0x7efd38133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 -- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd3815d3f0 msgr2=0x7efd38133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3dada640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd380cbd60 0x7efd380b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd3815d3f0 0x7efd38133810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 -- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd380cbd60 msgr2=0x7efd380b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd380cbd60 0x7efd380b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 -- 172.21.15.71:0/3444100316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28009560 con 0x7efd3813c220 2023-10-26T05:58:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3d2d9640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 0x7efd3814c720 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efd28009850 tx=0x7efd280a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28003800 con 0x7efd3813c220 2023-10-26T05:58:05.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efd28003960 con 0x7efd3813c220 2023-10-26T05:58:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38137910 con 0x7efd3813c220 2023-10-26T05:58:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28003d30 con 0x7efd3813c220 2023-10-26T05:58:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380ad720 con 0x7efd3813c220 2023-10-26T05:58:05.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.219+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38066b10 con 0x7efd3813c220 2023-10-26T05:58:05.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.223+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd280b4070 con 0x7efd3813c220 2023-10-26T05:58:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.223+0000 7efd2e7fc640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd1c066470 0x7efd1c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.223+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efd2811e0b0 con 0x7efd3813c220 2023-10-26T05:58:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.223+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7efd1c06e000 con 0x7efd3813c220 2023-10-26T05:58:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.223+0000 7efd3dada640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd1c066470 0x7efd1c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.223+0000 7efd3dada640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd1c066470 0x7efd1c068930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd34003d60 tx=0x7efd34009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:05.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.227+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efd280eb540 con 0x7efd3813c220 2023-10-26T05:58:05.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd380721e0 con 0x7efd3813c220 2023-10-26T05:58:05.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd2e7fc640 1 -- 172.21.15.71:0/3444100316 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v176) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd280b2070 con 0x7efd3813c220 2023-10-26T05:58:05.387 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:58:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd1c066470 msgr2=0x7efd1c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd1c066470 0x7efd1c068930 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd34003d60 tx=0x7efd34009040 comp rx=0 tx=0).stop 2023-10-26T05:58:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 msgr2=0x7efd3814c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 0x7efd3814c720 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efd28009850 tx=0x7efd280a56c0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 shutdown_connections 2023-10-26T05:58:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd3815d3f0 0x7efd38133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efd1c066470 0x7efd1c068930 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd3813c220 0x7efd3814c720 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 --2- 172.21.15.71:0/3444100316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd380cbd60 0x7efd380b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 >> 172.21.15.71:0/3444100316 conn(0x7efd380aec60 msgr2=0x7efd380b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 shutdown_connections 2023-10-26T05:58:05.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.379+0000 7efd3fd3e640 1 -- 172.21.15.71:0/3444100316 wait complete. 2023-10-26T05:58:05.405 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:58:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/2818125250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 msgr2=0x7fadbc158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 --2- 172.21.15.71:0/2818125250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc158cd0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0b3630 tx=0x7fadb801c720 comp rx=0 tx=0).stop 2023-10-26T05:58:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/2818125250 shutdown_connections 2023-10-26T05:58:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 --2- 172.21.15.71:0/2818125250 >> v2:172.21.15.86:3300/0 conn(0x7fadbc0cc5f0 0x7fadbc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 --2- 172.21.15.71:0/2818125250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc158cd0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/2818125250 >> 172.21.15.71:0/2818125250 conn(0x7fadbc0ce270 msgr2=0x7fadbc0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/2818125250 shutdown_connections 2023-10-26T05:58:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/2818125250 wait complete. 2023-10-26T05:58:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 Processor -- start 2023-10-26T05:58:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- start start 2023-10-26T05:58:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc0cc5f0 0x7fadbc14bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadbc162670 0x7fadbc162b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc163040 con 0x7fadbc1588f0 2023-10-26T05:58:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc144cf0 con 0x7fadbc0cc5f0 2023-10-26T05:58:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc159ac0 con 0x7fadbc162670 2023-10-26T05:58:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc14d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53212/0 (socket says 172.21.15.71:53212) 2023-10-26T05:58:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 -- 172.21.15.71:0/1850789109 learned_addr learned my addr 172.21.15.71:0/1850789109 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc20f5640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadbc162670 0x7fadbc162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc18f4640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc0cc5f0 0x7fadbc14bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 -- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadbc162670 msgr2=0x7fadbc162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadbc162670 0x7fadbc162b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 -- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc0cc5f0 msgr2=0x7fadbc14bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc0cc5f0 0x7fadbc14bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 -- 172.21.15.71:0/1850789109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80094a0 con 0x7fadbc1588f0 2023-10-26T05:58:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc18f4640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc0cc5f0 0x7fadbc14bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc10f3640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc14d7e0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c530 tx=0x7fadac00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac012830 con 0x7fadbc1588f0 2023-10-26T05:58:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fadac012a80 con 0x7fadbc1588f0 2023-10-26T05:58:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc159da0 con 0x7fadbc1588f0 2023-10-26T05:58:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac012e90 con 0x7fadbc1588f0 2023-10-26T05:58:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.519+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc15a360 con 0x7fadbc1588f0 2023-10-26T05:58:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc06d6c0 con 0x7fadbc1588f0 2023-10-26T05:58:05.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fadac01a3b0 con 0x7fadbc1588f0 2023-10-26T05:58:05.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadb27fc640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fada00664a0 0x7fada0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fadac084000 con 0x7fadbc1588f0 2023-10-26T05:58:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fada006e030 con 0x7fadbc1588f0 2023-10-26T05:58:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadc18f4640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fada00664a0 0x7fada0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.523+0000 7fadc18f4640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fada00664a0 0x7fada0068960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fadb80af000 tx=0x7fadb801af60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.527+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fadac016020 con 0x7fadbc1588f0 2023-10-26T05:58:05.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.667+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadbc066d00 con 0x7fadbc1588f0 2023-10-26T05:58:05.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadb27fc640 1 -- 172.21.15.71:0/1850789109 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadac054fa0 con 0x7fadbc1588f0 2023-10-26T05:58:05.678 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:58:05.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fada00664a0 msgr2=0x7fada0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fada00664a0 0x7fada0068960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fadb80af000 tx=0x7fadb801af60 comp rx=0 tx=0).stop 2023-10-26T05:58:05.680 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 msgr2=0x7fadbc14d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc14d7e0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c530 tx=0x7fadac00c9f0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 shutdown_connections 2023-10-26T05:58:05.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fadbc162670 0x7fadbc162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fada00664a0 0x7fada0068960 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.681 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadbc1588f0 0x7fadbc14d7e0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 --2- 172.21.15.71:0/1850789109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadbc0cc5f0 0x7fadbc14bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 >> 172.21.15.71:0/1850789109 conn(0x7fadbc0ce270 msgr2=0x7fadbc0aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 shutdown_connections 2023-10-26T05:58:05.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.671+0000 7fadc3b58640 1 -- 172.21.15.71:0/1850789109 wait complete. 2023-10-26T05:58:05.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:58:05.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 -- 172.21.15.71:0/624049613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2200052b0 msgr2=0x7fb22000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 --2- 172.21.15.71:0/624049613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2200052b0 0x7fb22000b990 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fb23806b1d0 tx=0x7fb23001d130 comp rx=0 tx=0).stop 2023-10-26T05:58:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 -- 172.21.15.71:0/624049613 shutdown_connections 2023-10-26T05:58:05.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 --2- 172.21.15.71:0/624049613 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb2200052b0 0x7fb22000b990 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 -- 172.21.15.71:0/624049613 >> 172.21.15.71:0/624049613 conn(0x7fb23806b8c0 msgr2=0x7fb23806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 -- 172.21.15.71:0/624049613 shutdown_connections 2023-10-26T05:58:05.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.799+0000 7fb23eb90640 1 -- 172.21.15.71:0/624049613 wait complete. 2023-10-26T05:58:05.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 Processor -- start 2023-10-26T05:58:05.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- start start 2023-10-26T05:58:05.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2200052b0 0x7fb23813b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2381ef3e0 0x7fb2381579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 0x7fb23815cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380a64d0 con 0x7fb238153da0 2023-10-26T05:58:05.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23813bc40 con 0x7fb2381ef3e0 2023-10-26T05:58:05.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2381542f0 con 0x7fb2200052b0 2023-10-26T05:58:05.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 0x7fb23815cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 0x7fb23815cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53220/0 (socket says 172.21.15.71:53220) 2023-10-26T05:58:05.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 -- 172.21.15.71:0/731212986 learned_addr learned my addr 172.21.15.71:0/731212986 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:05.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23c92c640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2200052b0 0x7fb23813b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb237fff640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2381ef3e0 0x7fb2381579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 -- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2200052b0 msgr2=0x7fb23813b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2200052b0 0x7fb23813b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 -- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2381ef3e0 msgr2=0x7fb2381579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2381ef3e0 0x7fb2381579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 -- 172.21.15.71:0/731212986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2381545d0 con 0x7fb238153da0 2023-10-26T05:58:05.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23c92c640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2200052b0 0x7fb23813b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23d12d640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 0x7fb23815cc80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fb23001cba0 tx=0x7fb2300ac950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300aca40 con 0x7fb238153da0 2023-10-26T05:58:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb2300acc90 con 0x7fb238153da0 2023-10-26T05:58:05.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2381548b0 con 0x7fb238153da0 2023-10-26T05:58:05.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2300b7420 con 0x7fb238153da0 2023-10-26T05:58:05.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb238154e70 con 0x7fb238153da0 2023-10-26T05:58:05.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb238066b10 con 0x7fb238153da0 2023-10-26T05:58:05.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.803+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2300b75a0 con 0x7fb238153da0 2023-10-26T05:58:05.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.807+0000 7fb2357fa640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2240664a0 0x7fb224068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:05.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.807+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb23011f270 con 0x7fb238153da0 2023-10-26T05:58:05.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.807+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fb22406e030 con 0x7fb238153da0 2023-10-26T05:58:05.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.807+0000 7fb23c92c640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2240664a0 0x7fb224068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.807+0000 7fb23c92c640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2240664a0 0x7fb224068960 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb228006680 tx=0x7fb228006140 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:05.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.811+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb2300b1060 con 0x7fb238153da0 2023-10-26T05:58:05.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.975+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb23806c7d0 con 0x7fb238153da0 2023-10-26T05:58:05.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.975+0000 7fb2357fa640 1 -- 172.21.15.71:0/731212986 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v176) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb2300c5020 con 0x7fb238153da0 2023-10-26T05:58:05.984 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:58:05.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2240664a0 msgr2=0x7fb224068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2240664a0 0x7fb224068960 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb228006680 tx=0x7fb228006140 comp rx=0 tx=0).stop 2023-10-26T05:58:05.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 msgr2=0x7fb23815cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:05.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 0x7fb23815cc80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fb23001cba0 tx=0x7fb2300ac950 comp rx=0 tx=0).stop 2023-10-26T05:58:05.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 shutdown_connections 2023-10-26T05:58:05.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb2240664a0 0x7fb224068960 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb238153da0 0x7fb23815cc80 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2381ef3e0 0x7fb2381579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 --2- 172.21.15.71:0/731212986 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2200052b0 0x7fb23813b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:05.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 >> 172.21.15.71:0/731212986 conn(0x7fb23806b8c0 msgr2=0x7fb23813e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:05.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 shutdown_connections 2023-10-26T05:58:05.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:05.979+0000 7fb23eb90640 1 -- 172.21.15.71:0/731212986 wait complete. 2023-10-26T05:58:06.003 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:58:06.003 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:06.003 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:06.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:06.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[53152]: fuse finished with error 0 and tester_r 0 2023-10-26T05:58:06.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:06.061 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:58:06.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:06.062 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:58:06.063 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:58:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:06.070 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:58:06.070 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:06.077 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 -- 172.21.15.71:0/2563378194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec140cc0 msgr2=0x7f5cec15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/2563378194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec140cc0 0x7f5cec15c760 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f5cec06fa60 tx=0x7f5cdc01c870 comp rx=0 tx=0).stop 2023-10-26T05:58:06.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 -- 172.21.15.71:0/2563378194 shutdown_connections 2023-10-26T05:58:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/2563378194 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cec0b0040 0x7f5cec0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/2563378194 >> v2:172.21.15.86:3300/0 conn(0x7f5cec070fe0 0x7f5cec071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/2563378194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec140cc0 0x7f5cec15c760 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 -- 172.21.15.71:0/2563378194 >> 172.21.15.71:0/2563378194 conn(0x7f5cec069770 msgr2=0x7f5cec069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:06.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 -- 172.21.15.71:0/2563378194 shutdown_connections 2023-10-26T05:58:06.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.215+0000 7f5cf15d9640 1 -- 172.21.15.71:0/2563378194 wait complete. 2023-10-26T05:58:06.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 Processor -- start 2023-10-26T05:58:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- start start 2023-10-26T05:58:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 0x7f5cec161280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cec0b0040 0x7f5cec158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cec140cc0 0x7f5cec15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec153590 con 0x7f5cec070fe0 2023-10-26T05:58:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15bae0 con 0x7f5cec140cc0 2023-10-26T05:58:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1593c0 con 0x7f5cec0b0040 2023-10-26T05:58:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 0x7f5cec161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.232 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 0x7f5cec161280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53250/0 (socket says 172.21.15.71:53250) 2023-10-26T05:58:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 -- 172.21.15.71:0/1302701984 learned_addr learned my addr 172.21.15.71:0/1302701984 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 -- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cec0b0040 msgr2=0x7f5cec158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cec0b0040 0x7f5cec158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 -- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cec140cc0 msgr2=0x7f5cec15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cec140cc0 0x7f5cec15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 -- 172.21.15.71:0/1302701984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc009560 con 0x7f5cec070fe0 2023-10-26T05:58:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ceaffd640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 0x7f5cec161280 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009850 tx=0x7f5cdc0a58c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0bb070 con 0x7f5cec070fe0 2023-10-26T05:58:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5cdc0042b0 con 0x7f5cec070fe0 2023-10-26T05:58:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec152c60 con 0x7f5cec070fe0 2023-10-26T05:58:06.234 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc003600 con 0x7f5cec070fe0 2023-10-26T05:58:06.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec153220 con 0x7f5cec070fe0 2023-10-26T05:58:06.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec066b10 con 0x7f5cec070fe0 2023-10-26T05:58:06.235 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.219+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5cdc0b4040 con 0x7f5cec070fe0 2023-10-26T05:58:06.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.223+0000 7f5ce37fe640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5ccc0664a0 0x7f5ccc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.223+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5cdc11deb0 con 0x7f5cec070fe0 2023-10-26T05:58:06.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.223+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f5ccc06e030 con 0x7f5cec070fe0 2023-10-26T05:58:06.236 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.223+0000 7f5ce3fff640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5ccc0664a0 0x7f5ccc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.223+0000 7f5ce3fff640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5ccc0664a0 0x7f5ccc068960 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0ae700 tx=0x7f5cd4006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:06.237 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.227+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f5cdc0eb320 con 0x7f5cec070fe0 2023-10-26T05:58:06.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.367+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cec1533b0 con 0x7f5cec070fe0 2023-10-26T05:58:06.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.367+0000 7f5ce37fe640 1 -- 172.21.15.71:0/1302701984 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16572 (secure 0 0 0) 0x7f5cdc0ee940 con 0x7f5cec070fe0 2023-10-26T05:58:06.377 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:06.378 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":176,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:57:52.290625+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:06.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5ccc0664a0 msgr2=0x7f5ccc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5ccc0664a0 0x7f5ccc068960 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0ae700 tx=0x7f5cd4006ca0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 msgr2=0x7f5cec161280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 0x7f5cec161280 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009850 tx=0x7f5cdc0a58c0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 shutdown_connections 2023-10-26T05:58:06.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cec140cc0 0x7f5cec15b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5cec0b0040 0x7f5cec158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5ccc0664a0 0x7f5ccc068960 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 --2- 172.21.15.71:0/1302701984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5cec070fe0 0x7f5cec161280 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 >> 172.21.15.71:0/1302701984 conn(0x7f5cec069770 msgr2=0x7f5cec13c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:06.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 shutdown_connections 2023-10-26T05:58:06.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.371+0000 7f5cf15d9640 1 -- 172.21.15.71:0/1302701984 wait complete. 2023-10-26T05:58:06.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:56:56.944301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:56:58.029095+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:58:06.396 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:58:06.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- 172.21.15.71:0/1518124267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f40047d0 msgr2=0x7f48f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 --2- 172.21.15.71:0/1518124267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f48f40047d0 0x7f48f4004c50 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fb0 tx=0x7f48f001cf90 comp rx=0 tx=0).stop 2023-10-26T05:58:06.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f4906d9d640 1 -- 172.21.15.71:0/1518124267 reap_dead start 2023-10-26T05:58:06.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- 172.21.15.71:0/1518124267 shutdown_connections 2023-10-26T05:58:06.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- 172.21.15.71:0/1518124267 >> 172.21.15.71:0/1518124267 conn(0x7f49080aec60 msgr2=0x7f49080af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:06.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- 172.21.15.71:0/1518124267 shutdown_connections 2023-10-26T05:58:06.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- 172.21.15.71:0/1518124267 wait complete. 2023-10-26T05:58:06.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 Processor -- start 2023-10-26T05:58:06.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- start start 2023-10-26T05:58:06.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f40047d0 0x7f49081341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 0x7f4908134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.528 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f490815a440 0x7f490815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081f5220 con 0x7f4908134a90 2023-10-26T05:58:06.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4908135450 con 0x7f48f40047d0 2023-10-26T05:58:06.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490ca45640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490815c2b0 con 0x7f490815a440 2023-10-26T05:58:06.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 0x7f4908134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.529 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 0x7f4908134f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53266/0 (socket says 172.21.15.71:53266) 2023-10-26T05:58:06.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 -- 172.21.15.71:0/3504301790 learned_addr learned my addr 172.21.15.71:0/3504301790 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:06.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f4906d9d640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f490815a440 0x7f490815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490659c640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f40047d0 0x7f49081341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 -- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f490815a440 msgr2=0x7f490815a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.530 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f490815a440 0x7f490815a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 -- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f40047d0 msgr2=0x7f49081341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f40047d0 0x7f49081341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f48fdd9b640 1 -- 172.21.15.71:0/3504301790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490813a270 con 0x7f4908134a90 2023-10-26T05:58:06.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.515+0000 7f490659c640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f40047d0 0x7f49081341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:06.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48fdd9b640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 0x7f4908134f10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001940 tx=0x7f48f0001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00b3070 con 0x7f4908134a90 2023-10-26T05:58:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48f0001500 con 0x7f4908134a90 2023-10-26T05:58:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4908159680 con 0x7f4908134a90 2023-10-26T05:58:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0003840 con 0x7f4908134a90 2023-10-26T05:58:06.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4908159c40 con 0x7f4908134a90 2023-10-26T05:58:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4908066b10 con 0x7f4908134a90 2023-10-26T05:58:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f48f00ab020 con 0x7f4908134a90 2023-10-26T05:58:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48e8066470 0x7f48e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f48f011e000 con 0x7f4908134a90 2023-10-26T05:58:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f48e806e000 con 0x7f4908134a90 2023-10-26T05:58:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f490659c640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48e8066470 0x7f48e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.519+0000 7f490659c640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48e8066470 0x7f48e8068930 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009aa0 tx=0x7f48f80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:06.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.523+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f48f00eb460 con 0x7f4908134a90 2023-10-26T05:58:06.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.679+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49080721e0 con 0x7f4908134a90 2023-10-26T05:58:06.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.679+0000 7f48ff7fe640 1 -- 172.21.15.71:0/3504301790 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48f00eea80 con 0x7f4908134a90 2023-10-26T05:58:06.689 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:58:06.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48e8066470 msgr2=0x7f48e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48e8066470 0x7f48e8068930 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009aa0 tx=0x7f48f80091e0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 msgr2=0x7f4908134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 0x7f4908134f10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001940 tx=0x7f48f0001d20 comp rx=0 tx=0).stop 2023-10-26T05:58:06.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 shutdown_connections 2023-10-26T05:58:06.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f490815a440 0x7f490815a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f48e8066470 0x7f48e8068930 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4908134a90 0x7f4908134f10 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 --2- 172.21.15.71:0/3504301790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48f40047d0 0x7f49081341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 >> 172.21.15.71:0/3504301790 conn(0x7f49080aec60 msgr2=0x7f49080b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:06.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 shutdown_connections 2023-10-26T05:58:06.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.683+0000 7f490ca45640 1 -- 172.21.15.71:0/3504301790 wait complete. 2023-10-26T05:58:06.708 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:58:06.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 -- 172.21.15.71:0/3225825813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08141980 msgr2=0x7f0d080b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 --2- 172.21.15.71:0/3225825813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08141980 0x7f0d080b1e50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009840 tx=0x7f0d0401c7d0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 -- 172.21.15.71:0/3225825813 shutdown_connections 2023-10-26T05:58:06.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 --2- 172.21.15.71:0/3225825813 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d08070680 0x7f0d080cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 --2- 172.21.15.71:0/3225825813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08141980 0x7f0d080b1e50 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 -- 172.21.15.71:0/3225825813 >> 172.21.15.71:0/3225825813 conn(0x7f0d080aec60 msgr2=0x7f0d080af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:06.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 -- 172.21.15.71:0/3225825813 shutdown_connections 2023-10-26T05:58:06.839 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.827+0000 7f0d107c9640 1 -- 172.21.15.71:0/3225825813 wait complete. 2023-10-26T05:58:06.841 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 Processor -- start 2023-10-26T05:58:06.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- start start 2023-10-26T05:58:06.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 0x7f0d08154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d080b2390 0x7f0d081565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08141980 0x7f0d08140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08156fa0 con 0x7f0d08070680 2023-10-26T05:58:06.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08157280 con 0x7f0d08141980 2023-10-26T05:58:06.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08157560 con 0x7f0d080b2390 2023-10-26T05:58:06.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 0x7f0d08154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 0x7f0d08154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53274/0 (socket says 172.21.15.71:53274) 2023-10-26T05:58:06.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 -- 172.21.15.71:0/2223728189 learned_addr learned my addr 172.21.15.71:0/2223728189 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:06.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0dd64640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d080b2390 0x7f0d081565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 -- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d080b2390 msgr2=0x7f0d081565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d080b2390 0x7f0d081565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 -- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08141980 msgr2=0x7f0d08140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08141980 0x7f0d08140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 -- 172.21.15.71:0/2223728189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040094a0 con 0x7f0d08070680 2023-10-26T05:58:06.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d0e565640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 0x7f0d08154c10 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c10 tx=0x7f0d04005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:06.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040bc070 con 0x7f0d08070680 2023-10-26T05:58:06.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0d040a5800 con 0x7f0d08070680 2023-10-26T05:58:06.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d08157840 con 0x7f0d08070680 2023-10-26T05:58:06.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040048d0 con 0x7f0d08070680 2023-10-26T05:58:06.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d08155150 con 0x7f0d08070680 2023-10-26T05:58:06.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08066b10 con 0x7f0d08070680 2023-10-26T05:58:06.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0d040b4040 con 0x7f0d08070680 2023-10-26T05:58:06.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.831+0000 7f0cfeffd640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0cec0664a0 0x7f0cec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:06.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.835+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0d0411f2b0 con 0x7f0d08070680 2023-10-26T05:58:06.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.835+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f0cec06e030 con 0x7f0d08070680 2023-10-26T05:58:06.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.835+0000 7f0d0dd64640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0cec0664a0 0x7f0cec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:06.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.835+0000 7f0d0dd64640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0cec0664a0 0x7f0cec068960 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8007fd0 tx=0x7f0cf8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:06.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.835+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0d040b2020 con 0x7f0d08070680 2023-10-26T05:58:06.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.975+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d08157d60 con 0x7f0d08070680 2023-10-26T05:58:06.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0cfeffd640 1 -- 172.21.15.71:0/2223728189 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v176) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d08157d60 con 0x7f0d08070680 2023-10-26T05:58:06.987 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:58:06.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0cec0664a0 msgr2=0x7f0cec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0cec0664a0 0x7f0cec068960 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8007fd0 tx=0x7f0cf8009040 comp rx=0 tx=0).stop 2023-10-26T05:58:06.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 msgr2=0x7f0d08154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:06.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 0x7f0d08154c10 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000c10 tx=0x7f0d04005750 comp rx=0 tx=0).stop 2023-10-26T05:58:06.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 shutdown_connections 2023-10-26T05:58:06.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08141980 0x7f0d08140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0d080b2390 0x7f0d081565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0cec0664a0 0x7f0cec068960 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 --2- 172.21.15.71:0/2223728189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d08070680 0x7f0d08154c10 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 >> 172.21.15.71:0/2223728189 conn(0x7f0d080aec60 msgr2=0x7f0d080b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 shutdown_connections 2023-10-26T05:58:06.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:06.979+0000 7f0d107c9640 1 -- 172.21.15.71:0/2223728189 wait complete. 2023-10-26T05:58:07.004 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:58:07.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2190138366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71f4004d10 msgr2=0x7f71f4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2190138366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71f4004d10 0x7f71f4005190 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f720000c590 tx=0x7f720001faa0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 -- 172.21.15.71:0/2190138366 reap_dead start 2023-10-26T05:58:07.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2190138366 shutdown_connections 2023-10-26T05:58:07.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2190138366 >> 172.21.15.71:0/2190138366 conn(0x7f72080aec60 msgr2=0x7f72080af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:07.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2190138366 shutdown_connections 2023-10-26T05:58:07.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2190138366 wait complete. 2023-10-26T05:58:07.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 Processor -- start 2023-10-26T05:58:07.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- start start 2023-10-26T05:58:07.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7208137c00 0x7f7208151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208137fe0 0x7f72081f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 0x7f72081529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72081f3980 con 0x7f72081f34f0 2023-10-26T05:58:07.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720814fdc0 con 0x7f7208137fe0 2023-10-26T05:58:07.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7208150060 con 0x7f7208137c00 2023-10-26T05:58:07.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 0x7f72081529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 0x7f72081529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53296/0 (socket says 172.21.15.71:53296) 2023-10-26T05:58:07.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 -- 172.21.15.71:0/2441861536 learned_addr learned my addr 172.21.15.71:0/2441861536 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:07.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f72077fe640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7208137c00 0x7f7208151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f71feffd640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208137fe0 0x7f72081f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 -- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7208137c00 msgr2=0x7f7208151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7208137c00 0x7f7208151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.133 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 -- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208137fe0 msgr2=0x7f72081f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208137fe0 0x7f72081f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 -- 172.21.15.71:0/2441861536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208150340 con 0x7f72081f34f0 2023-10-26T05:58:07.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7207fff640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 0x7f72081529f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f720001f8d0 tx=0x7f7200001ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:07.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72000b7070 con 0x7f72081f34f0 2023-10-26T05:58:07.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7200001210 con 0x7f72081f34f0 2023-10-26T05:58:07.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7208150620 con 0x7f72081f34f0 2023-10-26T05:58:07.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200003c60 con 0x7f72081f34f0 2023-10-26T05:58:07.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.119+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7208152f30 con 0x7f72081f34f0 2023-10-26T05:58:07.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208066b10 con 0x7f72081f34f0 2023-10-26T05:58:07.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f72000ba070 con 0x7f72081f34f0 2023-10-26T05:58:07.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f7204ff9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7200120f60 con 0x7f72081f34f0 2023-10-26T05:58:07.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f71e806e000 con 0x7f72081f34f0 2023-10-26T05:58:07.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f72077fe640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.123+0000 7f72077fe640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f720815c810 tx=0x7f71f800d390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:07.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.127+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f72000c1020 con 0x7f72081f34f0 2023-10-26T05:58:07.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.279+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72080721e0 con 0x7f72081f34f0 2023-10-26T05:58:07.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f7204ff9640 1 -- 172.21.15.71:0/2441861536 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72000ee3f0 con 0x7f72081f34f0 2023-10-26T05:58:07.290 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:58:07.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 msgr2=0x7f71e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f720815c810 tx=0x7f71f800d390 comp rx=0 tx=0).stop 2023-10-26T05:58:07.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 msgr2=0x7f72081529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 0x7f72081529f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f720001f8d0 tx=0x7f7200001ff0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 shutdown_connections 2023-10-26T05:58:07.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f71e8066470 0x7f71e8068930 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f72081f34f0 0x7f72081529f0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208137fe0 0x7f72081f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 --2- 172.21.15.71:0/2441861536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7208137c00 0x7f7208151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 >> 172.21.15.71:0/2441861536 conn(0x7f72080aec60 msgr2=0x7f72080acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:07.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 shutdown_connections 2023-10-26T05:58:07.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.283+0000 7f720e0e9640 1 -- 172.21.15.71:0/2441861536 wait complete. 2023-10-26T05:58:07.310 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:58:07.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- 172.21.15.71:0/1298157550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 msgr2=0x7fe54c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 --2- 172.21.15.71:0/1298157550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe54c005190 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c590 tx=0x7fe55c01d320 comp rx=0 tx=0).stop 2023-10-26T05:58:07.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe562d9d640 1 -- 172.21.15.71:0/1298157550 reap_dead start 2023-10-26T05:58:07.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- 172.21.15.71:0/1298157550 shutdown_connections 2023-10-26T05:58:07.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- 172.21.15.71:0/1298157550 >> 172.21.15.71:0/1298157550 conn(0x7fe56406b8c0 msgr2=0x7fe56406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:07.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- 172.21.15.71:0/1298157550 shutdown_connections 2023-10-26T05:58:07.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- 172.21.15.71:0/1298157550 wait complete. 2023-10-26T05:58:07.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 Processor -- start 2023-10-26T05:58:07.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- start start 2023-10-26T05:58:07.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe5640af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564155160 0x7fe5641555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5641395f0 0x7fe564139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564139fe0 con 0x7fe54c004d10 2023-10-26T05:58:07.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640afad0 con 0x7fe564155160 2023-10-26T05:58:07.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.419+0000 7fe568883640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640acbd0 con 0x7fe5641395f0 2023-10-26T05:58:07.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe5640af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe5640af590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53318/0 (socket says 172.21.15.71:53318) 2023-10-26T05:58:07.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 -- 172.21.15.71:0/3207449643 learned_addr learned my addr 172.21.15.71:0/3207449643 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:07.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe561d9b640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564155160 0x7fe5641555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe562d9d640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5641395f0 0x7fe564139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 -- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5641395f0 msgr2=0x7fe564139aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5641395f0 0x7fe564139aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 -- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564155160 msgr2=0x7fe5641555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564155160 0x7fe5641555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 -- 172.21.15.71:0/3207449643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5640aceb0 con 0x7fe54c004d10 2023-10-26T05:58:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe56259c640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe5640af590 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe558011a80 tx=0x7fe558011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800fdb0 con 0x7fe54c004d10 2023-10-26T05:58:07.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe558012450 con 0x7fe54c004d10 2023-10-26T05:58:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5640ad190 con 0x7fe54c004d10 2023-10-26T05:58:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558012800 con 0x7fe54c004d10 2023-10-26T05:58:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640ad750 con 0x7fe54c004d10 2023-10-26T05:58:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5640ad910 con 0x7fe54c004d10 2023-10-26T05:58:07.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe558012960 con 0x7fe54c004d10 2023-10-26T05:58:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe544066470 0x7fe544068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe558084c60 con 0x7fe54c004d10 2023-10-26T05:58:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fe54406e000 con 0x7fe54c004d10 2023-10-26T05:58:07.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe561d9b640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe544066470 0x7fe544068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.423+0000 7fe561d9b640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe544066470 0x7fe544068930 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe56413a230 tx=0x7fe554006d40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:07.440 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.427+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe558052200 con 0x7fe54c004d10 2023-10-26T05:58:07.600 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.591+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe564066b10 con 0x7fe54c004d10 2023-10-26T05:58:07.601 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.591+0000 7fe552ffd640 1 -- 172.21.15.71:0/3207449643 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v176) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe558055820 con 0x7fe54c004d10 2023-10-26T05:58:07.601 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:58:07.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe544066470 msgr2=0x7fe544068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe544066470 0x7fe544068930 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe56413a230 tx=0x7fe554006d40 comp rx=0 tx=0).stop 2023-10-26T05:58:07.603 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 msgr2=0x7fe5640af590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe5640af590 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe558011a80 tx=0x7fe558011f40 comp rx=0 tx=0).stop 2023-10-26T05:58:07.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 shutdown_connections 2023-10-26T05:58:07.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe5641395f0 0x7fe564139aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564155160 0x7fe5641555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe544066470 0x7fe544068930 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 --2- 172.21.15.71:0/3207449643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe54c004d10 0x7fe5640af590 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 >> 172.21.15.71:0/3207449643 conn(0x7fe56406b8c0 msgr2=0x7fe56413d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:07.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 shutdown_connections 2023-10-26T05:58:07.605 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.595+0000 7fe568883640 1 -- 172.21.15.71:0/3207449643 wait complete. 2023-10-26T05:58:07.619 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:58:07.619 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:07.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:07.675 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:07.676 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35369]: fuse finished with error 0 and tester_r 0 2023-10-26T05:58:07.676 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:07.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:58:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:07.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:58:07.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:07.685 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:58:07.691 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:07.692 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:07.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:58:07.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:58:07.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:07.699 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:58:07.746 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:58:07.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:58:07.748 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:58:07.748 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:07.754 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:58:07.754 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:58:07.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:07.755 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:58:07.799 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:58:07.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:58:07.801 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:07.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/1048947713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 msgr2=0x7f14c8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 --2- 172.21.15.71:0/1048947713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14c8004c50 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f14d400bb10 tx=0x7f14d401cd60 comp rx=0 tx=0).stop 2023-10-26T05:58:07.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e586e640 1 -- 172.21.15.71:0/1048947713 reap_dead start 2023-10-26T05:58:07.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/1048947713 shutdown_connections 2023-10-26T05:58:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/1048947713 >> 172.21.15.71:0/1048947713 conn(0x7f14e0069770 msgr2=0x7f14e0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/1048947713 shutdown_connections 2023-10-26T05:58:07.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/1048947713 wait complete. 2023-10-26T05:58:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 Processor -- start 2023-10-26T05:58:07.934 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- start start 2023-10-26T05:58:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14e01ef410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e013a710 0x7f14e0142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14e015a790 0x7f14e015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e015b150 con 0x7f14c80047d0 2023-10-26T05:58:07.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0138c90 con 0x7f14e013a710 2023-10-26T05:58:07.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e013ac10 con 0x7f14e015a790 2023-10-26T05:58:07.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14e01ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14e01ef410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53340/0 (socket says 172.21.15.71:53340) 2023-10-26T05:58:07.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 -- 172.21.15.71:0/243688530 learned_addr learned my addr 172.21.15.71:0/243688530 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:07.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e486c640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e013a710 0x7f14e0142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e586e640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14e015a790 0x7f14e015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 -- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14e015a790 msgr2=0x7f14e015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14e015a790 0x7f14e015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 -- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e013a710 msgr2=0x7f14e0142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:07.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e013a710 0x7f14e0142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:07.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 -- 172.21.15.71:0/243688530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e013aef0 con 0x7f14c80047d0 2023-10-26T05:58:07.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e586e640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14e015a790 0x7f14e015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:07.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e506d640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14e01ef410 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f14d800e360 tx=0x7f14d800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:07.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8012910 con 0x7f14c80047d0 2023-10-26T05:58:07.938 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f14d8012b60 con 0x7f14c80047d0 2023-10-26T05:58:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e013b1d0 con 0x7f14c80047d0 2023-10-26T05:58:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d801a400 con 0x7f14c80047d0 2023-10-26T05:58:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.923+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e01ef950 con 0x7f14c80047d0 2023-10-26T05:58:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e0066b10 con 0x7f14c80047d0 2023-10-26T05:58:07.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f14d80233f0 con 0x7f14c80047d0 2023-10-26T05:58:07.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14ddffb640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14cc0664a0 0x7f14cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:07.940 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f14d80875d0 con 0x7f14c80047d0 2023-10-26T05:58:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f14cc06e030 con 0x7f14c80047d0 2023-10-26T05:58:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14e486c640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14cc0664a0 0x7f14cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.927+0000 7f14e486c640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14cc0664a0 0x7f14cc068960 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f14d40b0000 tx=0x7f14d40a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:07.941 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:07.931+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f14d8054ac0 con 0x7f14c80047d0 2023-10-26T05:58:08.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.123+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14e0139120 con 0x7f14c80047d0 2023-10-26T05:58:08.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.123+0000 7f14ddffb640 1 -- 172.21.15.71:0/243688530 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f14e0139120 con 0x7f14c80047d0 2023-10-26T05:58:08.132 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:08.133 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:58:08.133 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:58:08.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.123+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14cc0664a0 msgr2=0x7f14cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.123+0000 7f14e72d1640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14cc0664a0 0x7f14cc068960 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f14d40b0000 tx=0x7f14d40a7040 comp rx=0 tx=0).stop 2023-10-26T05:58:08.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 msgr2=0x7f14e01ef410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14e01ef410 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f14d800e360 tx=0x7f14d800e820 comp rx=0 tx=0).stop 2023-10-26T05:58:08.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 shutdown_connections 2023-10-26T05:58:08.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f14e015a790 0x7f14e015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14e013a710 0x7f14e0142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f14cc0664a0 0x7f14cc068960 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 --2- 172.21.15.71:0/243688530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14c80047d0 0x7f14e01ef410 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 >> 172.21.15.71:0/243688530 conn(0x7f14e0069770 msgr2=0x7f14e0069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:08.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 shutdown_connections 2023-10-26T05:58:08.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.127+0000 7f14e72d1640 1 -- 172.21.15.71:0/243688530 wait complete. 2023-10-26T05:58:08.151 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 -- 172.21.15.71:0/2931235325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03013c2f0 msgr2=0x7ff0300b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 --2- 172.21.15.71:0/2931235325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03013c2f0 0x7ff0300b2350 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7ff028009bc0 tx=0x7ff02801cb80 comp rx=0 tx=0).stop 2023-10-26T05:58:08.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 -- 172.21.15.71:0/2931235325 shutdown_connections 2023-10-26T05:58:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 --2- 172.21.15.71:0/2931235325 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff0300cb1e0 0x7ff030159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 --2- 172.21.15.71:0/2931235325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff03013c2f0 0x7ff0300b2350 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 -- 172.21.15.71:0/2931235325 >> 172.21.15.71:0/2931235325 conn(0x7ff030069770 msgr2=0x7ff030069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 -- 172.21.15.71:0/2931235325 shutdown_connections 2023-10-26T05:58:08.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.263+0000 7ff038359640 1 -- 172.21.15.71:0/2931235325 wait complete. 2023-10-26T05:58:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 Processor -- start 2023-10-26T05:58:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- start start 2023-10-26T05:58:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0300b2890 0x7ff03015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 0x7ff03014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03013c2f0 0x7ff0301323a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0301427a0 con 0x7ff0300cb1e0 2023-10-26T05:58:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff030141b20 con 0x7ff0300b2890 2023-10-26T05:58:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff030141e00 con 0x7ff03013c2f0 2023-10-26T05:58:08.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 0x7ff03014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 0x7ff03014fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53366/0 (socket says 172.21.15.71:53366) 2023-10-26T05:58:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 -- 172.21.15.71:0/3575296771 learned_addr learned my addr 172.21.15.71:0/3575296771 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 -- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03013c2f0 msgr2=0x7ff0301323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0368f6640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03013c2f0 0x7ff0301323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0360f5640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0300b2890 0x7ff03015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03013c2f0 0x7ff0301323a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 -- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0300b2890 msgr2=0x7ff03015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0300b2890 0x7ff03015bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 -- 172.21.15.71:0/3575296771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009820 con 0x7ff0300cb1e0 2023-10-26T05:58:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff0358f4640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 0x7ff03014fdc0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c500 tx=0x7ff02000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:08.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020012830 con 0x7ff0300cb1e0 2023-10-26T05:58:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff020012a80 con 0x7ff0300cb1e0 2023-10-26T05:58:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff020012e90 con 0x7ff0300cb1e0 2023-10-26T05:58:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03015a7f0 con 0x7ff0300cb1e0 2023-10-26T05:58:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0301423c0 con 0x7ff0300cb1e0 2023-10-26T05:58:08.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.267+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff030066b10 con 0x7ff0300cb1e0 2023-10-26T05:58:08.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.271+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff020024430 con 0x7ff0300cb1e0 2023-10-26T05:58:08.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.271+0000 7ff026ffd640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff014066470 0x7ff014068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.271+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff0200843c0 con 0x7ff0300cb1e0 2023-10-26T05:58:08.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.271+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7ff01406e000 con 0x7ff0300cb1e0 2023-10-26T05:58:08.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.271+0000 7ff0360f5640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff014066470 0x7ff014068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.271+0000 7ff0360f5640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff014066470 0x7ff014068930 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff0280af000 tx=0x7ff02801b3c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:08.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.275+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff020055400 con 0x7ff0300cb1e0 2023-10-26T05:58:08.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.415+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0300b2f00 con 0x7ff0300cb1e0 2023-10-26T05:58:08.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff026ffd640 1 -- 172.21.15.71:0/3575296771 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16572 (secure 0 0 0) 0x7ff020016070 con 0x7ff0300cb1e0 2023-10-26T05:58:08.427 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:08.427 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":176,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:57:52.290625+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:08.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff014066470 msgr2=0x7ff014068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff014066470 0x7ff014068930 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff0280af000 tx=0x7ff02801b3c0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 msgr2=0x7ff03014fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 0x7ff03014fdc0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c500 tx=0x7ff02000c9c0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 shutdown_connections 2023-10-26T05:58:08.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff03013c2f0 0x7ff0301323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff014066470 0x7ff014068930 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0300cb1e0 0x7ff03014fdc0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 --2- 172.21.15.71:0/3575296771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0300b2890 0x7ff03015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 >> 172.21.15.71:0/3575296771 conn(0x7ff030069770 msgr2=0x7ff030157430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:08.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 shutdown_connections 2023-10-26T05:58:08.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.419+0000 7ff038359640 1 -- 172.21.15.71:0/3575296771 wait complete. 2023-10-26T05:58:08.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:56:56.944301+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:56:58.029095+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:58:08.445 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:58:08.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- 172.21.15.71:0/2054387705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440047d0 msgr2=0x7fd244004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 --2- 172.21.15.71:0/2054387705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2440047d0 0x7fd244004c50 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fd250001680 tx=0x7fd25001c760 comp rx=0 tx=0).stop 2023-10-26T05:58:08.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd25ad9d640 1 -- 172.21.15.71:0/2054387705 reap_dead start 2023-10-26T05:58:08.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- 172.21.15.71:0/2054387705 shutdown_connections 2023-10-26T05:58:08.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- 172.21.15.71:0/2054387705 >> 172.21.15.71:0/2054387705 conn(0x7fd25c0aec60 msgr2=0x7fd25c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:08.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- 172.21.15.71:0/2054387705 shutdown_connections 2023-10-26T05:58:08.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- 172.21.15.71:0/2054387705 wait complete. 2023-10-26T05:58:08.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 Processor -- start 2023-10-26T05:58:08.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- start start 2023-10-26T05:58:08.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2440047d0 0x7fd25c149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 0x7fd25c14a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd25c0add50 0x7fd25c0ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0cd250 con 0x7fd25c14a510 2023-10-26T05:58:08.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0c1ed0 con 0x7fd2440047d0 2023-10-26T05:58:08.569 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd260d84640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0ae6f0 con 0x7fd25c0add50 2023-10-26T05:58:08.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd259d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 0x7fd25c14a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd259d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 0x7fd25c14a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53372/0 (socket says 172.21.15.71:53372) 2023-10-26T05:58:08.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd259d9b640 1 -- 172.21.15.71:0/3088900387 learned_addr learned my addr 172.21.15.71:0/3088900387 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:08.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd25a59c640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2440047d0 0x7fd25c149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.570 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.555+0000 7fd25ad9d640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd25c0add50 0x7fd25c0ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd259d9b640 1 -- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd25c0add50 msgr2=0x7fd25c0ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd259d9b640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd25c0add50 0x7fd25c0ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd259d9b640 1 -- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2440047d0 msgr2=0x7fd25c149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd259d9b640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2440047d0 0x7fd25c149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.571 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd259d9b640 1 -- 172.21.15.71:0/3088900387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0cd440 con 0x7fd25c14a510 2023-10-26T05:58:08.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd259d9b640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 0x7fd25c14a990 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fd250005040 tx=0x7fd250003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:08.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500a9040 con 0x7fd25c14a510 2023-10-26T05:58:08.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd250003e30 con 0x7fd25c14a510 2023-10-26T05:58:08.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0cd6e0 con 0x7fd25c14a510 2023-10-26T05:58:08.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500a57a0 con 0x7fd25c14a510 2023-10-26T05:58:08.572 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0cdca0 con 0x7fd25c14a510 2023-10-26T05:58:08.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c066b10 con 0x7fd25c14a510 2023-10-26T05:58:08.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd2500b5020 con 0x7fd25c14a510 2023-10-26T05:58:08.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd23c066470 0x7fd23c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd2500aa070 con 0x7fd25c14a510 2023-10-26T05:58:08.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fd23c06e000 con 0x7fd25c14a510 2023-10-26T05:58:08.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd25a59c640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd23c066470 0x7fd23c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.559+0000 7fd25a59c640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd23c066470 0x7fd23c068930 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd25c1579a0 tx=0x7fd24c00a2f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:08.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.563+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd2500bd020 con 0x7fd25c14a510 2023-10-26T05:58:08.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.703+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd25c0c2080 con 0x7fd25c14a510 2023-10-26T05:58:08.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd24affd640 1 -- 172.21.15.71:0/3088900387 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd25c0c2080 con 0x7fd25c14a510 2023-10-26T05:58:08.714 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:58:08.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd23c066470 msgr2=0x7fd23c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd23c066470 0x7fd23c068930 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd25c1579a0 tx=0x7fd24c00a2f0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 msgr2=0x7fd25c14a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 0x7fd25c14a990 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fd250005040 tx=0x7fd250003c40 comp rx=0 tx=0).stop 2023-10-26T05:58:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 shutdown_connections 2023-10-26T05:58:08.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd25c0add50 0x7fd25c0ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd23c066470 0x7fd23c068930 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd25c14a510 0x7fd25c14a990 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 --2- 172.21.15.71:0/3088900387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2440047d0 0x7fd25c149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 >> 172.21.15.71:0/3088900387 conn(0x7fd25c0aec60 msgr2=0x7fd25c159730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:08.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 shutdown_connections 2023-10-26T05:58:08.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.707+0000 7fd260d84640 1 -- 172.21.15.71:0/3088900387 wait complete. 2023-10-26T05:58:08.732 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:58:08.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/4250832428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 msgr2=0x7fce1c1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 --2- 172.21.15.71:0/4250832428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c1589b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fce1c072f30 tx=0x7fce1401cb60 comp rx=0 tx=0).stop 2023-10-26T05:58:08.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/4250832428 shutdown_connections 2023-10-26T05:58:08.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 --2- 172.21.15.71:0/4250832428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c1589b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 --2- 172.21.15.71:0/4250832428 >> v2:172.21.15.86:3300/0 conn(0x7fce1c070e10 0x7fce1c158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/4250832428 >> 172.21.15.71:0/4250832428 conn(0x7fce1c0ca230 msgr2=0x7fce1c0ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:08.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/4250832428 shutdown_connections 2023-10-26T05:58:08.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/4250832428 wait complete. 2023-10-26T05:58:08.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 Processor -- start 2023-10-26T05:58:08.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- start start 2023-10-26T05:58:08.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e10 0x7fce1c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce1c0b1520 0x7fce1c0c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c15f4d0 con 0x7fce1c0b1040 2023-10-26T05:58:08.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c15f7b0 con 0x7fce1c070e10 2023-10-26T05:58:08.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c15fa90 con 0x7fce1c0b1520 2023-10-26T05:58:08.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53394/0 (socket says 172.21.15.71:53394) 2023-10-26T05:58:08.860 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 -- 172.21.15.71:0/2290931536 learned_addr learned my addr 172.21.15.71:0/2290931536 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:08.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 -- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce1c0b1520 msgr2=0x7fce1c0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce229f2640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce1c0b1520 0x7fce1c0c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce221f1640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e10 0x7fce1c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce1c0b1520 0x7fce1c0c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 -- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e10 msgr2=0x7fce1c151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:08.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e10 0x7fce1c151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:08.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 -- 172.21.15.71:0/2290931536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce14009820 con 0x7fce1c0b1040 2023-10-26T05:58:08.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce221f1640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e10 0x7fce1c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:08.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce219f0640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c143390 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b550 tx=0x7fce0c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:08.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c010870 con 0x7fce1c0b1040 2023-10-26T05:58:08.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fce0c010ae0 con 0x7fce1c0b1040 2023-10-26T05:58:08.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c15fd70 con 0x7fce1c0b1040 2023-10-26T05:58:08.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00f3c0 con 0x7fce1c0b1040 2023-10-26T05:58:08.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.847+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1c1438d0 con 0x7fce1c0b1040 2023-10-26T05:58:08.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1c160290 con 0x7fce1c0b1040 2023-10-26T05:58:08.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fce0c00f520 con 0x7fce1c0b1040 2023-10-26T05:58:08.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce12ffd640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce00066470 0x7fce00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:08.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fce0c0840d0 con 0x7fce1c0b1040 2023-10-26T05:58:08.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fce0006e000 con 0x7fce1c0b1040 2023-10-26T05:58:08.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce221f1640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce00066470 0x7fce00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:08.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.851+0000 7fce221f1640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce00066470 0x7fce00068930 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fce180046c0 tx=0x7fce18009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:08.855+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fce0c014060 con 0x7fce1c0b1040 2023-10-26T05:58:09.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.007+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce1c066d00 con 0x7fce1c0b1040 2023-10-26T05:58:09.017 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.007+0000 7fce12ffd640 1 -- 172.21.15.71:0/2290931536 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v176) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce0c01e020 con 0x7fce1c0b1040 2023-10-26T05:58:09.017 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:58:09.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce00066470 msgr2=0x7fce00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce00066470 0x7fce00068930 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fce180046c0 tx=0x7fce18009040 comp rx=0 tx=0).stop 2023-10-26T05:58:09.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 msgr2=0x7fce1c143390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c143390 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00b550 tx=0x7fce0c00ba10 comp rx=0 tx=0).stop 2023-10-26T05:58:09.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 shutdown_connections 2023-10-26T05:58:09.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fce1c0b1520 0x7fce1c0c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fce00066470 0x7fce00068930 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce1c0b1040 0x7fce1c143390 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 --2- 172.21.15.71:0/2290931536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fce1c070e10 0x7fce1c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 >> 172.21.15.71:0/2290931536 conn(0x7fce1c0ca230 msgr2=0x7fce1c069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 shutdown_connections 2023-10-26T05:58:09.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.011+0000 7fce24455640 1 -- 172.21.15.71:0/2290931536 wait complete. 2023-10-26T05:58:09.036 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:58:09.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- 172.21.15.71:0/1983787131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c159fa0 msgr2=0x7fe11c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 --2- 172.21.15.71:0/1983787131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c159fa0 0x7fe11c06ff70 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0ce7c0 tx=0x7fe11001c870 comp rx=0 tx=0).stop 2023-10-26T05:58:09.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- 172.21.15.71:0/1983787131 shutdown_connections 2023-10-26T05:58:09.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 --2- 172.21.15.71:0/1983787131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c159fa0 0x7fe11c06ff70 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 --2- 172.21.15.71:0/1983787131 >> v2:172.21.15.86:3300/0 conn(0x7fe11c06c4b0 0x7fe11c06c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- 172.21.15.71:0/1983787131 >> 172.21.15.71:0/1983787131 conn(0x7fe11c0af7d0 msgr2=0x7fe11c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- 172.21.15.71:0/1983787131 shutdown_connections 2023-10-26T05:58:09.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- 172.21.15.71:0/1983787131 wait complete. 2023-10-26T05:58:09.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 Processor -- start 2023-10-26T05:58:09.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- start start 2023-10-26T05:58:09.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 0x7fe11c136160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe11c159fa0 0x7fe11c13dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe11c15a420 0x7fe11c13e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c1376a0 con 0x7fe11c06c4b0 2023-10-26T05:58:09.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c137980 con 0x7fe11c15a420 2023-10-26T05:58:09.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe122883640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c137c60 con 0x7fe11c159fa0 2023-10-26T05:58:09.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 0x7fe11c136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 0x7fe11c136160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53406/0 (socket says 172.21.15.71:53406) 2023-10-26T05:58:09.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 -- 172.21.15.71:0/2091556827 learned_addr learned my addr 172.21.15.71:0/2091556827 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:09.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe120e20640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe11c15a420 0x7fe11c13e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 -- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe11c159fa0 msgr2=0x7fe11c13dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.158 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11b7fe640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe11c159fa0 0x7fe11c13dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe11c159fa0 0x7fe11c13dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 -- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe11c15a420 msgr2=0x7fe11c13e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe11c15a420 0x7fe11c13e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe11bfff640 1 -- 172.21.15.71:0/2091556827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110009560 con 0x7fe11c06c4b0 2023-10-26T05:58:09.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.143+0000 7fe120e20640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe11c15a420 0x7fe11c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:09.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe11bfff640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 0x7fe11c136160 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b550 tx=0x7fe10c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:09.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c010870 con 0x7fe11c06c4b0 2023-10-26T05:58:09.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe10c010ae0 con 0x7fe11c06c4b0 2023-10-26T05:58:09.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00f3c0 con 0x7fe11c06c4b0 2023-10-26T05:58:09.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c137f40 con 0x7fe11c06c4b0 2023-10-26T05:58:09.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c1369b0 con 0x7fe11c06c4b0 2023-10-26T05:58:09.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c066d00 con 0x7fe11c06c4b0 2023-10-26T05:58:09.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe10c00f520 con 0x7fe11c06c4b0 2023-10-26T05:58:09.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe108066470 0x7fe108068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe10c084100 con 0x7fe11c06c4b0 2023-10-26T05:58:09.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fe10806e000 con 0x7fe11c06c4b0 2023-10-26T05:58:09.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe11b7fe640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe108066470 0x7fe108068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.147+0000 7fe11b7fe640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe108066470 0x7fe108068930 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe1100af000 tx=0x7fe11001b0b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:09.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.151+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe10c014060 con 0x7fe11c06c4b0 2023-10-26T05:58:09.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.291+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe11c06d740 con 0x7fe11c06c4b0 2023-10-26T05:58:09.300 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.291+0000 7fe118ff9640 1 -- 172.21.15.71:0/2091556827 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v176) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe10c01e070 con 0x7fe11c06c4b0 2023-10-26T05:58:09.301 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:58:09.302 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe108066470 msgr2=0x7fe108068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe108066470 0x7fe108068930 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe1100af000 tx=0x7fe11001b0b0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 msgr2=0x7fe11c136160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 0x7fe11c136160 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b550 tx=0x7fe10c00ba10 comp rx=0 tx=0).stop 2023-10-26T05:58:09.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 shutdown_connections 2023-10-26T05:58:09.303 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe11c15a420 0x7fe11c13e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe11c159fa0 0x7fe11c13dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe108066470 0x7fe108068930 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 --2- 172.21.15.71:0/2091556827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c06c4b0 0x7fe11c136160 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 >> 172.21.15.71:0/2091556827 conn(0x7fe11c0af7d0 msgr2=0x7fe11c0aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.304 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 shutdown_connections 2023-10-26T05:58:09.305 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.295+0000 7fe122883640 1 -- 172.21.15.71:0/2091556827 wait complete. 2023-10-26T05:58:09.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:09.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 -- 172.21.15.71:0/4019182312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0d0470 msgr2=0x7fe48c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 --2- 172.21.15.71:0/4019182312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0d0470 0x7fe48c06aa70 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fe48c08a3b0 tx=0x7fe47c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 -- 172.21.15.71:0/4019182312 shutdown_connections 2023-10-26T05:58:09.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 --2- 172.21.15.71:0/4019182312 >> v2:172.21.15.86:3300/0 conn(0x7fe48c0cd880 0x7fe48c15bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 --2- 172.21.15.71:0/4019182312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0d0470 0x7fe48c06aa70 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 -- 172.21.15.71:0/4019182312 >> 172.21.15.71:0/4019182312 conn(0x7fe48c0cdd80 msgr2=0x7fe48c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 -- 172.21.15.71:0/4019182312 shutdown_connections 2023-10-26T05:58:09.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 -- 172.21.15.71:0/4019182312 wait complete. 2023-10-26T05:58:09.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 Processor -- start 2023-10-26T05:58:09.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.419+0000 7fe4908f3640 1 -- start start 2023-10-26T05:58:09.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 0x7fe48c151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe48c0d0470 0x7fe48c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c0d0850 0x7fe48c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c15e0c0 con 0x7fe48c0cd880 2023-10-26T05:58:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c15d820 con 0x7fe48c0d0850 2023-10-26T05:58:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c15db00 con 0x7fe48c0d0470 2023-10-26T05:58:09.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 0x7fe48c151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 0x7fe48c151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53422/0 (socket says 172.21.15.71:53422) 2023-10-26T05:58:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 -- 172.21.15.71:0/3189568841 learned_addr learned my addr 172.21.15.71:0/3189568841 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48ad9d640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c0d0850 0x7fe48c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 -- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe48c0d0470 msgr2=0x7fe48c15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe489d9b640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe48c0d0470 0x7fe48c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe48c0d0470 0x7fe48c15cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 -- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c0d0850 msgr2=0x7fe48c1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c0d0850 0x7fe48c1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 -- 172.21.15.71:0/3189568841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c009560 con 0x7fe48c0cd880 2023-10-26T05:58:09.435 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48ad9d640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c0d0850 0x7fe48c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:09.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe48a59c640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 0x7fe48c151670 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fe47c000c10 tx=0x7fe47c0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:09.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0052f0 con 0x7fe48c0cd880 2023-10-26T05:58:09.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe47c005540 con 0x7fe48c0cd880 2023-10-26T05:58:09.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c15dde0 con 0x7fe48c0cd880 2023-10-26T05:58:09.436 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c004930 con 0x7fe48c0cd880 2023-10-26T05:58:09.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c15c830 con 0x7fe48c0cd880 2023-10-26T05:58:09.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48c066b10 con 0x7fe48c0cd880 2023-10-26T05:58:09.437 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe47c0b4070 con 0x7fe48c0cd880 2023-10-26T05:58:09.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe46c066470 0x7fe46c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe47c11e190 con 0x7fe48c0cd880 2023-10-26T05:58:09.438 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7fe46c06e000 con 0x7fe48c0cd880 2023-10-26T05:58:09.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe489d9b640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe46c066470 0x7fe46c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.423+0000 7fe489d9b640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe46c066470 0x7fe46c068930 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe48c1f8000 tx=0x7fe480008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:09.439 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.427+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe47c0eb680 con 0x7fe48c0cd880 2023-10-26T05:58:09.613 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.603+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe48c0721e0 con 0x7fe48c0cd880 2023-10-26T05:58:09.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe47affd640 1 -- 172.21.15.71:0/3189568841 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fe47c0b2090 con 0x7fe48c0cd880 2023-10-26T05:58:09.614 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:09.615 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:58:09.615 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:58:09.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe46c066470 msgr2=0x7fe46c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe46c066470 0x7fe46c068930 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe48c1f8000 tx=0x7fe480008040 comp rx=0 tx=0).stop 2023-10-26T05:58:09.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 msgr2=0x7fe48c151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 0x7fe48c151670 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fe47c000c10 tx=0x7fe47c0a57d0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 shutdown_connections 2023-10-26T05:58:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe48c0d0850 0x7fe48c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe48c0d0470 0x7fe48c15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe46c066470 0x7fe46c068930 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 --2- 172.21.15.71:0/3189568841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe48c0cd880 0x7fe48c151670 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 >> 172.21.15.71:0/3189568841 conn(0x7fe48c0cdd80 msgr2=0x7fe48c157700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 shutdown_connections 2023-10-26T05:58:09.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.607+0000 7fe4908f3640 1 -- 172.21.15.71:0/3189568841 wait complete. 2023-10-26T05:58:09.631 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:09.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/4197230086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f415b5b0 msgr2=0x7f56f40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/4197230086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f415b5b0 0x7f56f40b4720 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009bc0 tx=0x7f56ec01cb50 comp rx=0 tx=0).stop 2023-10-26T05:58:09.759 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/4197230086 shutdown_connections 2023-10-26T05:58:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/4197230086 >> v2:172.21.15.86:3300/0 conn(0x7f56f40cbc40 0x7f56f406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/4197230086 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f415b5b0 0x7f56f40b4720 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/4197230086 >> 172.21.15.71:0/4197230086 conn(0x7f56f40ce070 msgr2=0x7f56f40ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/4197230086 shutdown_connections 2023-10-26T05:58:09.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/4197230086 wait complete. 2023-10-26T05:58:09.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 Processor -- start 2023-10-26T05:58:09.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- start start 2023-10-26T05:58:09.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f406b0a0 0x7f56f40b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f415b5b0 0x7f56f4153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 0x7f56f4153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40c8760 con 0x7f56f4154c90 2023-10-26T05:58:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f415d580 con 0x7f56f415b5b0 2023-10-26T05:58:09.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40c8a00 con 0x7f56f406b0a0 2023-10-26T05:58:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 0x7f56f4153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 0x7f56f4153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53434/0 (socket says 172.21.15.71:53434) 2023-10-26T05:58:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 -- 172.21.15.71:0/1224908546 learned_addr learned my addr 172.21.15.71:0/1224908546 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:09.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f37fe640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f406b0a0 0x7f56f40b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f2ffd640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f415b5b0 0x7f56f4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 -- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f406b0a0 msgr2=0x7f56f40b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f406b0a0 0x7f56f40b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 -- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f415b5b0 msgr2=0x7f56f4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f415b5b0 0x7f56f4153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 -- 172.21.15.71:0/1224908546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec009820 con 0x7f56f4154c90 2023-10-26T05:58:09.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f3fff640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 0x7f56f4153b50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f56e400c3c0 tx=0x7f56e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:09.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4011790 con 0x7f56f4154c90 2023-10-26T05:58:09.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f56e40119e0 con 0x7f56f4154c90 2023-10-26T05:58:09.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4011e10 con 0x7f56f4154c90 2023-10-26T05:58:09.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f40c8ce0 con 0x7f56f4154c90 2023-10-26T05:58:09.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.751+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f40b2b30 con 0x7f56f4154c90 2023-10-26T05:58:09.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f40c8f00 con 0x7f56f4154c90 2023-10-26T05:58:09.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56e401c460 con 0x7f56f4154c90 2023-10-26T05:58:09.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56ebfff640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f56d4066470 0x7f56d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:09.769 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f56e4084950 con 0x7f56f4154c90 2023-10-26T05:58:09.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f56d406e000 con 0x7f56f4154c90 2023-10-26T05:58:09.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56f37fe640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f56d4066470 0x7f56d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:09.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.755+0000 7f56f37fe640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f56d4066470 0x7f56d4068930 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0af000 tx=0x7f56ec01b390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:09.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.759+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f56e4054e30 con 0x7f56f4154c90 2023-10-26T05:58:09.923 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.915+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56f4066d00 con 0x7f56f4154c90 2023-10-26T05:58:09.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56ebfff640 1 -- 172.21.15.71:0/1224908546 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16572 (secure 0 0 0) 0x7f56e40549c0 con 0x7f56f4154c90 2023-10-26T05:58:09.927 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:09.927 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":176,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:57:52.290625+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:09.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f56d4066470 msgr2=0x7f56d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f56d4066470 0x7f56d4068930 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0af000 tx=0x7f56ec01b390 comp rx=0 tx=0).stop 2023-10-26T05:58:09.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 msgr2=0x7f56f4153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:09.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 0x7f56f4153b50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f56e400c3c0 tx=0x7f56e400c880 comp rx=0 tx=0).stop 2023-10-26T05:58:09.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 shutdown_connections 2023-10-26T05:58:09.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f56d4066470 0x7f56d4068930 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56f4154c90 0x7f56f4153b50 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56f415b5b0 0x7f56f4153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 --2- 172.21.15.71:0/1224908546 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56f406b0a0 0x7f56f40b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:09.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 >> 172.21.15.71:0/1224908546 conn(0x7f56f40ce070 msgr2=0x7f56f40b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:09.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 shutdown_connections 2023-10-26T05:58:09.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:09.919+0000 7f56f9a9c640 1 -- 172.21.15.71:0/1224908546 wait complete. 2023-10-26T05:58:09.945 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:58:09.946 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:10.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 -- 172.21.15.71:0/2469426709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0470 msgr2=0x7f4a1006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 --2- 172.21.15.71:0/2469426709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0470 0x7f4a1006aa70 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009bf0 tx=0x7f4a0801ca30 comp rx=0 tx=0).stop 2023-10-26T05:58:10.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 -- 172.21.15.71:0/2469426709 shutdown_connections 2023-10-26T05:58:10.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 --2- 172.21.15.71:0/2469426709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0850 0x7f4a1015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 --2- 172.21.15.71:0/2469426709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0470 0x7f4a1006aa70 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 -- 172.21.15.71:0/2469426709 >> 172.21.15.71:0/2469426709 conn(0x7f4a100cdd80 msgr2=0x7f4a100ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:10.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 -- 172.21.15.71:0/2469426709 shutdown_connections 2023-10-26T05:58:10.067 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.055+0000 7f4a16c93640 1 -- 172.21.15.71:0/2469426709 wait complete. 2023-10-26T05:58:10.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 Processor -- start 2023-10-26T05:58:10.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- start start 2023-10-26T05:58:10.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a100cd880 0x7f4a10150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0470 0x7f4a10151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 0x7f4a10151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a10152ff0 con 0x7f4a100d0850 2023-10-26T05:58:10.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1015e630 con 0x7f4a100cd880 2023-10-26T05:58:10.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1015dab0 con 0x7f4a100d0470 2023-10-26T05:58:10.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 0x7f4a10151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 0x7f4a10151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53456/0 (socket says 172.21.15.71:53456) 2023-10-26T05:58:10.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 -- 172.21.15.71:0/1245014203 learned_addr learned my addr 172.21.15.71:0/1245014203 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:10.071 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0ffff640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0470 0x7f4a10151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a14a2f640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a100cd880 0x7f4a10150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 -- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0470 msgr2=0x7f4a10151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0470 0x7f4a10151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 -- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a100cd880 msgr2=0x7f4a10150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a100cd880 0x7f4a10150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 -- 172.21.15.71:0/1245014203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a08009820 con 0x7f4a100d0850 2023-10-26T05:58:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0ffff640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0470 0x7f4a10151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a15230640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 0x7f4a10151840 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c400 tx=0x7f4a0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00011790 con 0x7f4a100d0850 2023-10-26T05:58:10.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4a000119e0 con 0x7f4a100d0850 2023-10-26T05:58:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1015dd90 con 0x7f4a100d0850 2023-10-26T05:58:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00011e30 con 0x7f4a100d0850 2023-10-26T05:58:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1015e350 con 0x7f4a100d0850 2023-10-26T05:58:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a10066b10 con 0x7f4a100d0850 2023-10-26T05:58:10.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4a0001c460 con 0x7f4a100d0850 2023-10-26T05:58:10.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.059+0000 7f4a0d7fa640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc066470 0x7f49fc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.063+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4a00083f00 con 0x7f4a100d0850 2023-10-26T05:58:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.063+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f49fc06e000 con 0x7f4a100d0850 2023-10-26T05:58:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.063+0000 7f4a14a2f640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc066470 0x7f49fc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.063+0000 7f4a14a2f640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc066470 0x7f49fc068930 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4a080af000 tx=0x7f4a0801b270 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:10.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.067+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4a000513f0 con 0x7f4a100d0850 2023-10-26T05:58:10.261 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.251+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a10158ce0 con 0x7f4a100d0850 2023-10-26T05:58:10.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.251+0000 7f4a0d7fa640 1 -- 172.21.15.71:0/1245014203 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f4a00054a10 con 0x7f4a100d0850 2023-10-26T05:58:10.262 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:10.262 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:58:10.264 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:58:10.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc066470 msgr2=0x7f49fc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc066470 0x7f49fc068930 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4a080af000 tx=0x7f4a0801b270 comp rx=0 tx=0).stop 2023-10-26T05:58:10.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 msgr2=0x7f4a10151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 0x7f4a10151840 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c400 tx=0x7f4a0000c8c0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 shutdown_connections 2023-10-26T05:58:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49fc066470 0x7f49fc068930 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a100d0850 0x7f4a10151840 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a100d0470 0x7f4a10151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 --2- 172.21.15.71:0/1245014203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a100cd880 0x7f4a10150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 >> 172.21.15.71:0/1245014203 conn(0x7f4a100cdd80 msgr2=0x7f4a100b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 shutdown_connections 2023-10-26T05:58:10.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.255+0000 7f4a16c93640 1 -- 172.21.15.71:0/1245014203 wait complete. 2023-10-26T05:58:10.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:10.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 -- 172.21.15.71:0/3520594915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 msgr2=0x7f0a0c0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 --2- 172.21.15.71:0/3520594915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c0ccba0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009bc0 tx=0x7f0a0401cb50 comp rx=0 tx=0).stop 2023-10-26T05:58:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 -- 172.21.15.71:0/3520594915 shutdown_connections 2023-10-26T05:58:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 --2- 172.21.15.71:0/3520594915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c0ccba0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 --2- 172.21.15.71:0/3520594915 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a0c142d20 0x7f0a0c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 -- 172.21.15.71:0/3520594915 >> 172.21.15.71:0/3520594915 conn(0x7f0a0c0ce270 msgr2=0x7f0a0c0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 -- 172.21.15.71:0/3520594915 shutdown_connections 2023-10-26T05:58:10.406 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 -- 172.21.15.71:0/3520594915 wait complete. 2023-10-26T05:58:10.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 Processor -- start 2023-10-26T05:58:10.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.395+0000 7f0a13a92640 1 -- start start 2023-10-26T05:58:10.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c06a000 0x7f0a0c136ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c1573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a0c142d20 0x7f0a0c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c158ed0 con 0x7f0a0c06ab70 2023-10-26T05:58:10.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c157910 con 0x7f0a0c06a000 2023-10-26T05:58:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c158350 con 0x7f0a0c142d20 2023-10-26T05:58:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c1573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c1573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53466/0 (socket says 172.21.15.71:53466) 2023-10-26T05:58:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 -- 172.21.15.71:0/2257952594 learned_addr learned my addr 172.21.15.71:0/2257952594 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:10.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1182e640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c06a000 0x7f0a0c136ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1202f640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a0c142d20 0x7f0a0c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 -- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a0c142d20 msgr2=0x7f0a0c157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a0c142d20 0x7f0a0c157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 -- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c06a000 msgr2=0x7f0a0c136ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c06a000 0x7f0a0c136ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 -- 172.21.15.71:0/2257952594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009820 con 0x7f0a0c06ab70 2023-10-26T05:58:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1102d640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c1573d0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c430 tx=0x7f09fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004230 con 0x7f0a0c06ab70 2023-10-26T05:58:10.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f09fc0044a0 con 0x7f0a0c06ab70 2023-10-26T05:58:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c158630 con 0x7f0a0c06ab70 2023-10-26T05:58:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc003840 con 0x7f0a0c06ab70 2023-10-26T05:58:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c158bf0 con 0x7f0a0c06ab70 2023-10-26T05:58:10.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c066d00 con 0x7f0a0c06ab70 2023-10-26T05:58:10.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f09fc0039a0 con 0x7f0a0c06ab70 2023-10-26T05:58:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09f0066470 0x7f09f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f09fc0836b0 con 0x7f0a0c06ab70 2023-10-26T05:58:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7f09f006e000 con 0x7f0a0c06ab70 2023-10-26T05:58:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.399+0000 7f0a1182e640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09f0066470 0x7f09f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.403+0000 7f0a1182e640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09f0066470 0x7f09f0068930 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c135e90 tx=0x7f0a080091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:10.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.403+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f09fc017060 con 0x7f0a0c06ab70 2023-10-26T05:58:10.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.595+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a0c13c430 con 0x7f0a0c06ab70 2023-10-26T05:58:10.604 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.595+0000 7f0a027fc640 1 -- 172.21.15.71:0/2257952594 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f09fc054630 con 0x7f0a0c06ab70 2023-10-26T05:58:10.605 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:10.605 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:56:59.035024+0000","modified":"2023-10-26T05:57:01.073826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8958},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8958":{"gid":8958,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/2999623846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":2999623846},{"type":"v1","addr":"172.21.15.71:6835","nonce":2999623846}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-26T05:58:10.606 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 64 2023-10-26T05:58:10.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09f0066470 msgr2=0x7f09f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.607 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09f0066470 0x7f09f0068930 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c135e90 tx=0x7f0a080091e0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 msgr2=0x7f0a0c1573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c1573d0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c430 tx=0x7f09fc00c8f0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 shutdown_connections 2023-10-26T05:58:10.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0a0c142d20 0x7f0a0c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.608 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f09f0066470 0x7f09f0068930 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c06ab70 0x7f0a0c1573d0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 --2- 172.21.15.71:0/2257952594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c06a000 0x7f0a0c136ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 >> 172.21.15.71:0/2257952594 conn(0x7f0a0c0ce270 msgr2=0x7f0a0c15bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 shutdown_connections 2023-10-26T05:58:10.609 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.599+0000 7f0a13a92640 1 -- 172.21.15.71:0/2257952594 wait complete. 2023-10-26T05:58:10.624 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:10.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 -- 172.21.15.71:0/119302088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 msgr2=0x7faa7c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 --2- 172.21.15.71:0/119302088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c15b3a0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7faa64009840 tx=0x7faa6401c700 comp rx=0 tx=0).stop 2023-10-26T05:58:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 -- 172.21.15.71:0/119302088 shutdown_connections 2023-10-26T05:58:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 --2- 172.21.15.71:0/119302088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c15b3a0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 -- 172.21.15.71:0/119302088 >> 172.21.15.71:0/119302088 conn(0x7faa7c0cdd80 msgr2=0x7faa7c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 -- 172.21.15.71:0/119302088 shutdown_connections 2023-10-26T05:58:10.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 -- 172.21.15.71:0/119302088 wait complete. 2023-10-26T05:58:10.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 Processor -- start 2023-10-26T05:58:10.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.731+0000 7faa8274e640 1 -- start start 2023-10-26T05:58:10.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa7c0cd880 0x7faa7c0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa7c0d0470 0x7faa7c0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.746 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c06a650 con 0x7faa7c0d0850 2023-10-26T05:58:10.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c15a5d0 con 0x7faa7c0cd880 2023-10-26T05:58:10.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0c9b30 con 0x7faa7c0d0470 2023-10-26T05:58:10.747 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c1547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53488/0 (socket says 172.21.15.71:53488) 2023-10-26T05:58:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 -- 172.21.15.71:0/2968943777 learned_addr learned my addr 172.21.15.71:0/2968943777 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa737fe640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa7c0d0470 0x7faa7c0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa7bfff640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa7c0cd880 0x7faa7c0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 -- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa7c0d0470 msgr2=0x7faa7c0c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa7c0d0470 0x7faa7c0c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 -- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa7c0cd880 msgr2=0x7faa7c0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa7c0cd880 0x7faa7c0c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 -- 172.21.15.71:0/2968943777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa640094a0 con 0x7faa7c0d0850 2023-10-26T05:58:10.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa7bfff640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa7c0cd880 0x7faa7c0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:10.750 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa80ceb640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c1547d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7faa7400c7e0 tx=0x7faa7400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74003db0 con 0x7faa7c0d0850 2023-10-26T05:58:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faa74005070 con 0x7faa7c0d0850 2023-10-26T05:58:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c1582b0 con 0x7faa7c0d0850 2023-10-26T05:58:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74005480 con 0x7faa7c0d0850 2023-10-26T05:58:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c0c8040 con 0x7faa7c0d0850 2023-10-26T05:58:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c066b10 con 0x7faa7c0d0850 2023-10-26T05:58:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faa740055e0 con 0x7faa7c0d0850 2023-10-26T05:58:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa5c066470 0x7faa5c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7faa74083e00 con 0x7faa7c0d0850 2023-10-26T05:58:10.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7faa5c06e000 con 0x7faa7c0d0850 2023-10-26T05:58:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa7bfff640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa5c066470 0x7faa5c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.735+0000 7faa7bfff640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa5c066470 0x7faa5c068930 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faa7c15e580 tx=0x7faa6c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:10.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.739+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faa740512f0 con 0x7faa7c0d0850 2023-10-26T05:58:10.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.879+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa7c158440 con 0x7faa7c0d0850 2023-10-26T05:58:10.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.883+0000 7faa797fa640 1 -- 172.21.15.71:0/2968943777 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+16572 (secure 0 0 0) 0x7faa74054910 con 0x7faa7c0d0850 2023-10-26T05:58:10.893 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:10.893 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":176,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:57:52.290625+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:56:56.944301+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-26T05:56:58.029095+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:10.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa5c066470 msgr2=0x7faa5c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa5c066470 0x7faa5c068930 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faa7c15e580 tx=0x7faa6c009040 comp rx=0 tx=0).stop 2023-10-26T05:58:10.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 msgr2=0x7faa7c1547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:10.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c1547d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7faa7400c7e0 tx=0x7faa7400cca0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 shutdown_connections 2023-10-26T05:58:10.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faa5c066470 0x7faa5c068930 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c0d0850 0x7faa7c1547d0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faa7c0d0470 0x7faa7c0c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 --2- 172.21.15.71:0/2968943777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa7c0cd880 0x7faa7c0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:10.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 >> 172.21.15.71:0/2968943777 conn(0x7faa7c0cdd80 msgr2=0x7faa7c0b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:10.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 shutdown_connections 2023-10-26T05:58:10.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:10.887+0000 7faa8274e640 1 -- 172.21.15.71:0/2968943777 wait complete. 2023-10-26T05:58:10.911 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:58:11.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- 172.21.15.71:0/750430036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ca960 msgr2=0x7efc5c15a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 --2- 172.21.15.71:0/750430036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ca960 0x7efc5c15a2d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7efc5c071160 tx=0x7efc5801c7a0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- 172.21.15.71:0/750430036 shutdown_connections 2023-10-26T05:58:11.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 --2- 172.21.15.71:0/750430036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ca960 0x7efc5c15a2d0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 --2- 172.21.15.71:0/750430036 >> v2:172.21.15.86:3300/0 conn(0x7efc5c153960 0x7efc5c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- 172.21.15.71:0/750430036 >> 172.21.15.71:0/750430036 conn(0x7efc5c070bc0 msgr2=0x7efc5c0af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:11.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- 172.21.15.71:0/750430036 shutdown_connections 2023-10-26T05:58:11.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- 172.21.15.71:0/750430036 wait complete. 2023-10-26T05:58:11.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 Processor -- start 2023-10-26T05:58:11.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- start start 2023-10-26T05:58:11.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 0x7efc5c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc5c0ca960 0x7efc5c151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5c153960 0x7efc5c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c15b6f0 con 0x7efc5c0ace90 2023-10-26T05:58:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1518a0 con 0x7efc5c0ca960 2023-10-26T05:58:11.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc6341f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c1502c0 con 0x7efc5c153960 2023-10-26T05:58:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc611bb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 0x7efc5c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc611bb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 0x7efc5c15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53514/0 (socket says 172.21.15.71:53514) 2023-10-26T05:58:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc611bb640 1 -- 172.21.15.71:0/3967203593 learned_addr learned my addr 172.21.15.71:0/3967203593 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc611bb640 1 -- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5c153960 msgr2=0x7efc5c14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc619bc640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5c153960 0x7efc5c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc609ba640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc5c0ca960 0x7efc5c151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.023+0000 7efc611bb640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5c153960 0x7efc5c14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc611bb640 1 -- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc5c0ca960 msgr2=0x7efc5c151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc611bb640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc5c0ca960 0x7efc5c151360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc611bb640 1 -- 172.21.15.71:0/3967203593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc58009560 con 0x7efc5c0ace90 2023-10-26T05:58:11.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc611bb640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 0x7efc5c15b1b0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7efc5400ccb0 tx=0x7efc5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:11.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5400f450 con 0x7efc5c0ace90 2023-10-26T05:58:11.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc5400f6a0 con 0x7efc5c0ace90 2023-10-26T05:58:11.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c1505a0 con 0x7efc5c0ace90 2023-10-26T05:58:11.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5400fab0 con 0x7efc5c0ace90 2023-10-26T05:58:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c150b60 con 0x7efc5c0ace90 2023-10-26T05:58:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c071580 con 0x7efc5c0ace90 2023-10-26T05:58:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc5400fc10 con 0x7efc5c0ace90 2023-10-26T05:58:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc40066470 0x7efc40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efc540849c0 con 0x7efc5c0ace90 2023-10-26T05:58:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=177}) v3 -- 0x7efc4006e000 con 0x7efc5c0ace90 2023-10-26T05:58:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc609ba640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc40066470 0x7efc40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.027+0000 7efc609ba640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc40066470 0x7efc40068930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efc5c15b140 tx=0x7efc4c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:11.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.031+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc5401d020 con 0x7efc5c0ace90 2023-10-26T05:58:11.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.207+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efc5c04db60 con 0x7efc5c0ace90 2023-10-26T05:58:11.218 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:58:11.211+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:58:11.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.211+0000 7efc51ffb640 1 -- 172.21.15.71:0/3967203593 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7efc54051eb0 con 0x7efc5c0ace90 2023-10-26T05:58:11.221 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:58:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc40066470 msgr2=0x7efc40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc40066470 0x7efc40068930 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efc5c15b140 tx=0x7efc4c009040 comp rx=0 tx=0).stop 2023-10-26T05:58:11.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 msgr2=0x7efc5c15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 0x7efc5c15b1b0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7efc5400ccb0 tx=0x7efc5400a620 comp rx=0 tx=0).stop 2023-10-26T05:58:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 shutdown_connections 2023-10-26T05:58:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc5c153960 0x7efc5c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc5c0ca960 0x7efc5c151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc40066470 0x7efc40068930 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 --2- 172.21.15.71:0/3967203593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc5c0ace90 0x7efc5c15b1b0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 >> 172.21.15.71:0/3967203593 conn(0x7efc5c070bc0 msgr2=0x7efc5c0af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:11.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 shutdown_connections 2023-10-26T05:58:11.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.215+0000 7efc6341f640 1 -- 172.21.15.71:0/3967203593 wait complete. 2023-10-26T05:58:11.237 INFO:tasks.ceph.mds.a.smithi071.stderr: -8228> 2023-10-26T05:57:04.643+0000 7f8d197f6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:58:11.242 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:58:11.301 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:58:11.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 -- 172.21.15.71:0/1371118795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d80141370 msgr2=0x7f7d80158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 --2- 172.21.15.71:0/1371118795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d80141370 0x7f7d80158680 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7d80071360 tx=0x7f7d7801ccb0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 -- 172.21.15.71:0/1371118795 shutdown_connections 2023-10-26T05:58:11.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 --2- 172.21.15.71:0/1371118795 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d80070550 0x7f7d800709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 --2- 172.21.15.71:0/1371118795 >> v2:172.21.15.86:3300/0 conn(0x7f7d800b0f70 0x7f7d800ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 --2- 172.21.15.71:0/1371118795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d80141370 0x7f7d80158680 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 -- 172.21.15.71:0/1371118795 >> 172.21.15.71:0/1371118795 conn(0x7f7d800afe40 msgr2=0x7f7d800b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:11.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 -- 172.21.15.71:0/1371118795 shutdown_connections 2023-10-26T05:58:11.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 -- 172.21.15.71:0/1371118795 wait complete. 2023-10-26T05:58:11.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 Processor -- start 2023-10-26T05:58:11.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.363+0000 7f7d8708f640 1 -- start start 2023-10-26T05:58:11.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d80070550 0x7f7d80157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 0x7f7d800ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80141370 0x7f7d800ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8006be20 con 0x7f7d800b0f70 2023-10-26T05:58:11.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8006b480 con 0x7f7d80141370 2023-10-26T05:58:11.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8006b760 con 0x7f7d80070550 2023-10-26T05:58:11.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 0x7f7d800ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 0x7f7d800ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53552/0 (socket says 172.21.15.71:53552) 2023-10-26T05:58:11.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 -- 172.21.15.71:0/3684250368 learned_addr learned my addr 172.21.15.71:0/3684250368 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:11.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8562c640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80141370 0x7f7d800ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d84e2b640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d80070550 0x7f7d80157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 -- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d80070550 msgr2=0x7f7d80157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d80070550 0x7f7d80157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 -- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80141370 msgr2=0x7f7d800ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:11.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80141370 0x7f7d800ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:11.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 -- 172.21.15.71:0/3684250368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780098e0 con 0x7f7d800b0f70 2023-10-26T05:58:11.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7ffff640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 0x7f7d800ae110 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c970 tx=0x7f7d7400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:11.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74007bb0 con 0x7f7d800b0f70 2023-10-26T05:58:11.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7d74007e00 con 0x7f7d800b0f70 2023-10-26T05:58:11.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d800b3fa0 con 0x7f7d800b0f70 2023-10-26T05:58:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74005280 con 0x7f7d800b0f70 2023-10-26T05:58:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d80136e40 con 0x7f7d800b0f70 2023-10-26T05:58:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d800ca1c0 con 0x7f7d800b0f70 2023-10-26T05:58:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7d740053e0 con 0x7f7d800b0f70 2023-10-26T05:58:11.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d6c066470 0x7f7d6c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7d740844a0 con 0x7f7d800b0f70 2023-10-26T05:58:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=178}) v3 -- 0x7f7d6c06e110 con 0x7f7d800b0f70 2023-10-26T05:58:11.383 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.367+0000 7f7d84e2b640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d6c066470 0x7f7d6c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:11.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.371+0000 7f7d84e2b640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d6c066470 0x7f7d6c068930 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7d780af000 tx=0x7f7d7801b4f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:11.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.371+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7d7401d020 con 0x7f7d800b0f70 2023-10-26T05:58:11.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:11.547+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7d800cbb50 con 0x7f7d800b0f70 2023-10-26T05:58:12.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.215+0000 7f7d7d7fa640 1 -- 172.21.15.71:0/3684250368 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d800cbb50 con 0x7f7d800b0f70 2023-10-26T05:58:12.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d6c066470 msgr2=0x7f7d6c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:12.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d6c066470 0x7f7d6c068930 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7d780af000 tx=0x7f7d7801b4f0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 msgr2=0x7f7d800ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:12.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 0x7f7d800ae110 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c970 tx=0x7f7d7400ce30 comp rx=0 tx=0).stop 2023-10-26T05:58:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 shutdown_connections 2023-10-26T05:58:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d80141370 0x7f7d800ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7d6c066470 0x7f7d6c068930 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d800b0f70 0x7f7d800ae110 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 --2- 172.21.15.71:0/3684250368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7d80070550 0x7f7d80157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 >> 172.21.15.71:0/3684250368 conn(0x7f7d800afe40 msgr2=0x7f7d800b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:12.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 shutdown_connections 2023-10-26T05:58:12.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.219+0000 7f7d8708f640 1 -- 172.21.15.71:0/3684250368 wait complete. 2023-10-26T05:58:12.247 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:58:12.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 -- 172.21.15.71:0/2039085136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f4069d30 msgr2=0x7f81f406a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 --2- 172.21.15.71:0/2039085136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f4069d30 0x7f81f406a1c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009840 tx=0x7f81ec01c7d0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 -- 172.21.15.71:0/2039085136 shutdown_connections 2023-10-26T05:58:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 --2- 172.21.15.71:0/2039085136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f4069d30 0x7f81f406a1c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 --2- 172.21.15.71:0/2039085136 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81f413e8c0 0x7f81f413ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 -- 172.21.15.71:0/2039085136 >> 172.21.15.71:0/2039085136 conn(0x7f81f40aec60 msgr2=0x7f81f40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 -- 172.21.15.71:0/2039085136 shutdown_connections 2023-10-26T05:58:12.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 -- 172.21.15.71:0/2039085136 wait complete. 2023-10-26T05:58:12.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 Processor -- start 2023-10-26T05:58:12.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.375+0000 7f81f94e6640 1 -- start start 2023-10-26T05:58:12.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81f4069d30 0x7f81f4152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:12.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f413c190 0x7f81f4133930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 0x7f81f414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4151570 con 0x7f81f413e8c0 2023-10-26T05:58:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4151850 con 0x7f81f413c190 2023-10-26T05:58:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f4151b30 con 0x7f81f4069d30 2023-10-26T05:58:12.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 0x7f81f414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 0x7f81f414dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47426/0 (socket says 172.21.15.71:47426) 2023-10-26T05:58:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 -- 172.21.15.71:0/1367412824 learned_addr learned my addr 172.21.15.71:0/1367412824 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f27fc640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f413c190 0x7f81f4133930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f2ffd640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81f4069d30 0x7f81f4152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:12.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 -- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81f4069d30 msgr2=0x7f81f4152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81f4069d30 0x7f81f4152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 -- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f413c190 msgr2=0x7f81f4133930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f413c190 0x7f81f4133930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 -- 172.21.15.71:0/1367412824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec0094a0 con 0x7f81f413e8c0 2023-10-26T05:58:12.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f37fe640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 0x7f81f414dff0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009790 tx=0x7f81ec005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0bb070 con 0x7f81f413e8c0 2023-10-26T05:58:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f81ec0a5880 con 0x7f81f413e8c0 2023-10-26T05:58:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec004290 con 0x7f81f413e8c0 2023-10-26T05:58:12.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f4151e10 con 0x7f81f413e8c0 2023-10-26T05:58:12.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f414e530 con 0x7f81f413e8c0 2023-10-26T05:58:12.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4066b10 con 0x7f81f413e8c0 2023-10-26T05:58:12.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81ec0b4040 con 0x7f81f413e8c0 2023-10-26T05:58:12.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81eb7fe640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81d4066470 0x7f81d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:12.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f81ec11e1e0 con 0x7f81f413e8c0 2023-10-26T05:58:12.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.379+0000 7f81f2ffd640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81d4066470 0x7f81d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:12.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.383+0000 7f81f2ffd640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81d4066470 0x7f81d4068930 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f81e4004b30 tx=0x7f81e4001a60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:12.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.383+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f81ec0eb610 con 0x7f81f413e8c0 2023-10-26T05:58:12.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:12.539+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81f4065c40 con 0x7f81f413e8c0 2023-10-26T05:58:13.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.555+0000 7f81eb7fe640 1 -- 172.21.15.71:0/1367412824 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v179) v1 ==== 181+0+0 (secure 0 0 0) 0x7f81ec0b20d0 con 0x7f81f413e8c0 2023-10-26T05:58:13.563 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:58:13.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81d4066470 msgr2=0x7f81d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:13.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81d4066470 0x7f81d4068930 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f81e4004b30 tx=0x7f81e4001a60 comp rx=0 tx=0).stop 2023-10-26T05:58:13.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 msgr2=0x7f81f414dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:13.573 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 0x7f81f414dff0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009790 tx=0x7f81ec005730 comp rx=0 tx=0).stop 2023-10-26T05:58:13.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 shutdown_connections 2023-10-26T05:58:13.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f81d4066470 0x7f81d4068930 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81f413e8c0 0x7f81f414dff0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81f413c190 0x7f81f4133930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.574 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 --2- 172.21.15.71:0/1367412824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81f4069d30 0x7f81f4152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.563+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 >> 172.21.15.71:0/1367412824 conn(0x7f81f40aec60 msgr2=0x7f81f413a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.567+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 shutdown_connections 2023-10-26T05:58:13.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.567+0000 7f81f94e6640 1 -- 172.21.15.71:0/1367412824 wait complete. 2023-10-26T05:58:13.594 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:58:13.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 -- 172.21.15.71:0/1503129768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 msgr2=0x7f239400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:13.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 --2- 172.21.15.71:0/1503129768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f239400b990 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f239c009ee0 tx=0x7f239c01d5d0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 -- 172.21.15.71:0/1503129768 shutdown_connections 2023-10-26T05:58:13.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 --2- 172.21.15.71:0/1503129768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f239400b990 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 -- 172.21.15.71:0/1503129768 >> 172.21.15.71:0/1503129768 conn(0x7f23ac0caaa0 msgr2=0x7f23ac0caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:13.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 -- 172.21.15.71:0/1503129768 shutdown_connections 2023-10-26T05:58:13.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 -- 172.21.15.71:0/1503129768 wait complete. 2023-10-26T05:58:13.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 Processor -- start 2023-10-26T05:58:13.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 -- start start 2023-10-26T05:58:13.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f23ac1ef350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:13.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.711+0000 7f23b3308640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac1436c0 0x7f23ac0a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:13.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23ac0aa130 0x7f23ac06a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:13.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac1ef890 con 0x7f2394005190 2023-10-26T05:58:13.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0abc70 con 0x7f23ac1436c0 2023-10-26T05:58:13.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0abea0 con 0x7f23ac0aa130 2023-10-26T05:58:13.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f23ac1ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:13.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f23ac1ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47440/0 (socket says 172.21.15.71:47440) 2023-10-26T05:58:13.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 -- 172.21.15.71:0/2446788979 learned_addr learned my addr 172.21.15.71:0/2446788979 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:13.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b18a5640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23ac0aa130 0x7f23ac06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:13.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b08a3640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac1436c0 0x7f23ac0a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:13.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 -- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23ac0aa130 msgr2=0x7f23ac06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:13.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23ac0aa130 0x7f23ac06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 -- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac1436c0 msgr2=0x7f23ac0a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:13.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac1436c0 0x7f23ac0a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:13.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 -- 172.21.15.71:0/2446788979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0ac180 con 0x7f2394005190 2023-10-26T05:58:13.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b10a4640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f23ac1ef350 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f23a4011a40 tx=0x7f23a4011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:13.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a400fce0 con 0x7f2394005190 2023-10-26T05:58:13.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f23a4013450 con 0x7f2394005190 2023-10-26T05:58:13.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac0ac460 con 0x7f2394005190 2023-10-26T05:58:13.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4013860 con 0x7f2394005190 2023-10-26T05:58:13.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac143b40 con 0x7f2394005190 2023-10-26T05:58:13.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac0aa510 con 0x7f2394005190 2023-10-26T05:58:13.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23a40139c0 con 0x7f2394005190 2023-10-26T05:58:13.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23a9ffb640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f23980664a0 0x7f2398068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:13.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f23a4087c80 con 0x7f2394005190 2023-10-26T05:58:13.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b08a3640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f23980664a0 0x7f2398068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:13.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.715+0000 7f23b08a3640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f23980664a0 0x7f2398068960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f23ac149930 tx=0x7f23a0006a70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:13.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.719+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f23a40552d0 con 0x7f2394005190 2023-10-26T05:58:13.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:13.875+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f23ac06bb40 con 0x7f2394005190 2023-10-26T05:58:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.567+0000 7f23a9ffb640 1 -- 172.21.15.71:0/2446788979 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v180) v1 ==== 169+0+0 (secure 0 0 0) 0x7f23a40220d0 con 0x7f2394005190 2023-10-26T05:58:14.575 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:58:14.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.571+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f23980664a0 msgr2=0x7f2398068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.571+0000 7f23b3308640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f23980664a0 0x7f2398068960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f23ac149930 tx=0x7f23a0006a70 comp rx=0 tx=0).stop 2023-10-26T05:58:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.571+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 msgr2=0x7f23ac1ef350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.571+0000 7f23b3308640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f23ac1ef350 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f23a4011a40 tx=0x7f23a4011f00 comp rx=0 tx=0).stop 2023-10-26T05:58:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 shutdown_connections 2023-10-26T05:58:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23ac0aa130 0x7f23ac06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23ac1436c0 0x7f23ac0a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f23980664a0 0x7f2398068960 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 --2- 172.21.15.71:0/2446788979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2394005190 0x7f23ac1ef350 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 >> 172.21.15.71:0/2446788979 conn(0x7f23ac0caaa0 msgr2=0x7f23ac0cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:14.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 shutdown_connections 2023-10-26T05:58:14.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.575+0000 7f23b3308640 1 -- 172.21.15.71:0/2446788979 wait complete. 2023-10-26T05:58:14.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:58:14.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/2192727038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4413b0f0 msgr2=0x7f7e440cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/2192727038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4413b0f0 0x7f7e440cb8c0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009900 tx=0x7f7e3401c850 comp rx=0 tx=0).stop 2023-10-26T05:58:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/2192727038 shutdown_connections 2023-10-26T05:58:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/2192727038 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e441590b0 0x7f7e4413d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/2192727038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e4413b0f0 0x7f7e440cb8c0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/2192727038 >> 172.21.15.71:0/2192727038 conn(0x7f7e4406b8c0 msgr2=0x7f7e4406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:14.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/2192727038 shutdown_connections 2023-10-26T05:58:14.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/2192727038 wait complete. 2023-10-26T05:58:14.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 Processor -- start 2023-10-26T05:58:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.723+0000 7f7e4a4fe640 1 -- start start 2023-10-26T05:58:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e4413b0f0 0x7f7e4414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e4413dd20 0x7f7e4412afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 0x7f7e440b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:14.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e440b22b0 con 0x7f7e441590b0 2023-10-26T05:58:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44159840 con 0x7f7e4413dd20 2023-10-26T05:58:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44159b20 con 0x7f7e4413b0f0 2023-10-26T05:58:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 0x7f7e440b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 0x7f7e440b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47452/0 (socket says 172.21.15.71:47452) 2023-10-26T05:58:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 -- 172.21.15.71:0/1646980166 learned_addr learned my addr 172.21.15.71:0/1646980166 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:14.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 -- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e4413b0f0 msgr2=0x7f7e4414e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e43fff640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e4413b0f0 0x7f7e4414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:14.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e3b7fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e4413dd20 0x7f7e4412afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:14.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e4413b0f0 0x7f7e4414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 -- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e4413dd20 msgr2=0x7f7e4412afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e4413dd20 0x7f7e4412afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 -- 172.21.15.71:0/1646980166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34009560 con 0x7f7e441590b0 2023-10-26T05:58:14.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e48a9b640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 0x7f7e440b1740 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00c780 tx=0x7f7e3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:14.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c004b70 con 0x7f7e441590b0 2023-10-26T05:58:14.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7e3c00ce90 con 0x7f7e441590b0 2023-10-26T05:58:14.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00aab0 con 0x7f7e441590b0 2023-10-26T05:58:14.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e44159e00 con 0x7f7e441590b0 2023-10-26T05:58:14.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4412b4e0 con 0x7f7e441590b0 2023-10-26T05:58:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e44066b10 con 0x7f7e441590b0 2023-10-26T05:58:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7e3c00ac10 con 0x7f7e441590b0 2023-10-26T05:58:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e417fa640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7e240664a0 0x7f7e24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.727+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7e3c0837a0 con 0x7f7e441590b0 2023-10-26T05:58:14.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.731+0000 7f7e43fff640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7e240664a0 0x7f7e24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.731+0000 7f7e43fff640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7e240664a0 0x7f7e24068960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7e340af000 tx=0x7f7e3401b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:14.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.735+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7e3c050fb0 con 0x7f7e441590b0 2023-10-26T05:58:14.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.887+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7f7e4415a320 con 0x7f7e441590b0 2023-10-26T05:58:14.905 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.895+0000 7f7e417fa640 1 -- 172.21.15.71:0/1646980166 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v36) v1 ==== 71+0+0 (secure 0 0 0) 0x7f7e4415a320 con 0x7f7e441590b0 2023-10-26T05:58:14.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7e240664a0 msgr2=0x7f7e24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7e240664a0 0x7f7e24068960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7e340af000 tx=0x7f7e3401b090 comp rx=0 tx=0).stop 2023-10-26T05:58:14.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 msgr2=0x7f7e440b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:14.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 0x7f7e440b1740 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00c780 tx=0x7f7e3c00cc40 comp rx=0 tx=0).stop 2023-10-26T05:58:14.908 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 shutdown_connections 2023-10-26T05:58:14.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7e240664a0 0x7f7e24068960 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e441590b0 0x7f7e440b1740 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e4413dd20 0x7f7e4412afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 --2- 172.21.15.71:0/1646980166 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7e4413b0f0 0x7f7e4414e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:14.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 >> 172.21.15.71:0/1646980166 conn(0x7f7e4406b8c0 msgr2=0x7f7e4413a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:14.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 shutdown_connections 2023-10-26T05:58:14.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:14.899+0000 7f7e4a4fe640 1 -- 172.21.15.71:0/1646980166 wait complete. 2023-10-26T05:58:14.925 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-10-26T05:58:15.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/758583566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0470 msgr2=0x7fe00c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- 172.21.15.71:0/758583566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0470 0x7fe00c06aa70 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0c4dd0 tx=0x7fe00801c810 comp rx=0 tx=0).stop 2023-10-26T05:58:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/758583566 shutdown_connections 2023-10-26T05:58:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- 172.21.15.71:0/758583566 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0cd880 0x7fe00c15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- 172.21.15.71:0/758583566 >> v2:172.21.15.86:3300/0 conn(0x7fe00c0d0850 0x7fe00c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- 172.21.15.71:0/758583566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0470 0x7fe00c06aa70 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/758583566 >> 172.21.15.71:0/758583566 conn(0x7fe00c0cdd80 msgr2=0x7fe00c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/758583566 shutdown_connections 2023-10-26T05:58:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/758583566 wait complete. 2023-10-26T05:58:15.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 Processor -- start 2023-10-26T05:58:15.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- start start 2023-10-26T05:58:15.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c0cd880 0x7fe00c14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0d0470 0x7fe00c14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 0x7fe00c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c154c80 con 0x7fe00c0d0850 2023-10-26T05:58:15.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c154000 con 0x7fe00c0cd880 2023-10-26T05:58:15.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c1542e0 con 0x7fe00c0d0470 2023-10-26T05:58:15.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 0x7fe00c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 0x7fe00c1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47476/0 (socket says 172.21.15.71:47476) 2023-10-26T05:58:15.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 -- 172.21.15.71:0/4033227380 learned_addr learned my addr 172.21.15.71:0/4033227380 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:15.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe011aa8640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0d0470 0x7fe00c14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe0122a9640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c0cd880 0x7fe00c14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 -- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0d0470 msgr2=0x7fe00c14c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0d0470 0x7fe00c14c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 -- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c0cd880 msgr2=0x7fe00c14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c0cd880 0x7fe00c14cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 -- 172.21.15.71:0/4033227380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008009560 con 0x7fe00c0d0850 2023-10-26T05:58:15.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe011aa8640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0d0470 0x7fe00c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:15.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe012aaa640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 0x7fe00c1582c0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe00400ca60 tx=0x7fe00400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:15.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400adc0 con 0x7fe00c0d0850 2023-10-26T05:58:15.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe004004c40 con 0x7fe00c0d0850 2023-10-26T05:58:15.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c1545c0 con 0x7fe00c0d0850 2023-10-26T05:58:15.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004005280 con 0x7fe00c0d0850 2023-10-26T05:58:15.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.047+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c14d440 con 0x7fe00c0d0850 2023-10-26T05:58:15.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.051+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c066b10 con 0x7fe00c0d0850 2023-10-26T05:58:15.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.051+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe0040053e0 con 0x7fe00c0d0850 2023-10-26T05:58:15.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.051+0000 7fe002ffd640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdff0066470 0x7fdff0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.051+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe004084330 con 0x7fe00c0d0850 2023-10-26T05:58:15.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.051+0000 7fe0122a9640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdff0066470 0x7fdff0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.051+0000 7fe0122a9640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdff0066470 0x7fdff0068930 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe0080af000 tx=0x7fe00801b050 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:15.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.055+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe004051b40 con 0x7fe00c0d0850 2023-10-26T05:58:15.272 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.263+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fe00c154dc0 con 0x7fe00c0d0850 2023-10-26T05:58:15.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.567+0000 7fe002ffd640 1 -- 172.21.15.71:0/4033227380 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v525) v1 ==== 131+0+0 (secure 0 0 0) 0x7fe00c154dc0 con 0x7fe00c0d0850 2023-10-26T05:58:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdff0066470 msgr2=0x7fdff0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdff0066470 0x7fdff0068930 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe0080af000 tx=0x7fe00801b050 comp rx=0 tx=0).stop 2023-10-26T05:58:15.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 msgr2=0x7fe00c1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 0x7fe00c1582c0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fe00400ca60 tx=0x7fe00400cf20 comp rx=0 tx=0).stop 2023-10-26T05:58:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 shutdown_connections 2023-10-26T05:58:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fdff0066470 0x7fdff0068930 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe00c0d0850 0x7fe00c1582c0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe00c0d0470 0x7fe00c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 --2- 172.21.15.71:0/4033227380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe00c0cd880 0x7fe00c14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 >> 172.21.15.71:0/4033227380 conn(0x7fe00c0cdd80 msgr2=0x7fe00c15a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 shutdown_connections 2023-10-26T05:58:15.582 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.571+0000 7fe01450d640 1 -- 172.21.15.71:0/4033227380 wait complete. 2023-10-26T05:58:15.602 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-10-26T05:58:15.603 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-10-26T05:58:15.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:15.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- 172.21.15.71:0/1426674498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0850 msgr2=0x7fd51815b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/1426674498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0850 0x7fd51815b3a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009840 tx=0x7fd50c01c6b0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- 172.21.15.71:0/1426674498 shutdown_connections 2023-10-26T05:58:15.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/1426674498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0850 0x7fd51815b3a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/1426674498 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5180d0470 0x7fd51806aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- 172.21.15.71:0/1426674498 >> 172.21.15.71:0/1426674498 conn(0x7fd5180cdd80 msgr2=0x7fd5180ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:15.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- 172.21.15.71:0/1426674498 shutdown_connections 2023-10-26T05:58:15.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- 172.21.15.71:0/1426674498 wait complete. 2023-10-26T05:58:15.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 Processor -- start 2023-10-26T05:58:15.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- start start 2023-10-26T05:58:15.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5180cd880 0x7fd5180c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0470 0x7fd518152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 0x7fd5180ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd518152c50 con 0x7fd5180d0850 2023-10-26T05:58:15.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180caf50 con 0x7fd5180d0470 2023-10-26T05:58:15.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51f0d9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180c6f10 con 0x7fd5180cd880 2023-10-26T05:58:15.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51d676640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 0x7fd5180ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51d676640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 0x7fd5180ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47484/0 (socket says 172.21.15.71:47484) 2023-10-26T05:58:15.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.723+0000 7fd51d676640 1 -- 172.21.15.71:0/2319863904 learned_addr learned my addr 172.21.15.71:0/2319863904 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:15.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51ce75640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5180cd880 0x7fd5180c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd517fff640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0470 0x7fd518152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51d676640 1 -- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5180cd880 msgr2=0x7fd5180c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51d676640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5180cd880 0x7fd5180c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51d676640 1 -- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0470 msgr2=0x7fd518152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51d676640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0470 0x7fd518152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51d676640 1 -- 172.21.15.71:0/2319863904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5080094a0 con 0x7fd5180d0850 2023-10-26T05:58:15.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51d676640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 0x7fd5180ca590 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd51000cae0 tx=0x7fd51000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:15.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510003b80 con 0x7fd5180d0850 2023-10-26T05:58:15.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd510003dd0 con 0x7fd5180d0850 2023-10-26T05:58:15.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c0094a0 con 0x7fd5180d0850 2023-10-26T05:58:15.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51000aab0 con 0x7fd5180d0850 2023-10-26T05:58:15.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5180c74d0 con 0x7fd5180d0850 2023-10-26T05:58:15.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd518066b10 con 0x7fd5180d0850 2023-10-26T05:58:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd51000ac10 con 0x7fd5180d0850 2023-10-26T05:58:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd5157fa640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd504066470 0x7fd504068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd510084330 con 0x7fd5180d0850 2023-10-26T05:58:15.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51ce75640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd504066470 0x7fd504068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.727+0000 7fd51ce75640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd504066470 0x7fd504068930 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd5181f8280 tx=0x7fd5080091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:15.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.731+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd510051990 con 0x7fd5180d0850 2023-10-26T05:58:15.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.883+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5180c76c0 con 0x7fd5180d0850 2023-10-26T05:58:15.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd5157fa640 1 -- 172.21.15.71:0/2319863904 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+12913 (secure 0 0 0) 0x7fd510071290 con 0x7fd5180d0850 2023-10-26T05:58:15.894 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":180,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:14.559692+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/2999623846":"2023-10-27T05:58:11.214455+0000","172.21.15.71:6834/2999623846":"2023-10-27T05:58:11.214455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:15.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd504066470 msgr2=0x7fd504068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd504066470 0x7fd504068930 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd5181f8280 tx=0x7fd5080091e0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.896 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 msgr2=0x7fd5180ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 0x7fd5180ca590 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd51000cae0 tx=0x7fd51000cbd0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 shutdown_connections 2023-10-26T05:58:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd504066470 0x7fd504068930 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5180d0850 0x7fd5180ca590 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.897 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5180d0470 0x7fd518152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 --2- 172.21.15.71:0/2319863904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5180cd880 0x7fd5180c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:15.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 >> 172.21.15.71:0/2319863904 conn(0x7fd5180cdd80 msgr2=0x7fd5180b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:15.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 shutdown_connections 2023-10-26T05:58:15.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:15.887+0000 7fd51f0d9640 1 -- 172.21.15.71:0/2319863904 wait complete. 2023-10-26T05:58:15.911 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:58:15.912 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:58:16.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/1329405059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c15b430 msgr2=0x7f343c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 --2- 172.21.15.71:0/1329405059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c15b430 0x7f343c069980 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f343c0d0ae0 tx=0x7f343401cb30 comp rx=0 tx=0).stop 2023-10-26T05:58:16.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/1329405059 shutdown_connections 2023-10-26T05:58:16.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 --2- 172.21.15.71:0/1329405059 >> v2:172.21.15.86:3300/0 conn(0x7f343c0b4350 0x7f343c06a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 --2- 172.21.15.71:0/1329405059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c15b430 0x7f343c069980 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/1329405059 >> 172.21.15.71:0/1329405059 conn(0x7f343c0d12c0 msgr2=0x7f343c0d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:16.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/1329405059 shutdown_connections 2023-10-26T05:58:16.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/1329405059 wait complete. 2023-10-26T05:58:16.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 Processor -- start 2023-10-26T05:58:16.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- start start 2023-10-26T05:58:16.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f343c06a810 0x7f343c0c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 0x7f343c0c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c15b430 0x7f343c15da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c153b90 con 0x7f343c0b4350 2023-10-26T05:58:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c153e70 con 0x7f343c15b430 2023-10-26T05:58:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c154150 con 0x7f343c06a810 2023-10-26T05:58:16.032 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 0x7f343c0c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 0x7f343c0c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47510/0 (socket says 172.21.15.71:47510) 2023-10-26T05:58:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 -- 172.21.15.71:0/3482667904 learned_addr learned my addr 172.21.15.71:0/3482667904 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343affd640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f343c06a810 0x7f343c0c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343b7fe640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c15b430 0x7f343c15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 -- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f343c06a810 msgr2=0x7f343c0c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f343c06a810 0x7f343c0c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 -- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c15b430 msgr2=0x7f343c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c15b430 0x7f343c15da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 -- 172.21.15.71:0/3482667904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009820 con 0x7f343c0b4350 2023-10-26T05:58:16.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343affd640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f343c06a810 0x7f343c0c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:16.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f343a7fc640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 0x7f343c0c4600 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f342400d980 tx=0x7f342400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:16.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424009c30 con 0x7f343c0b4350 2023-10-26T05:58:16.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3424009e80 con 0x7f343c0b4350 2023-10-26T05:58:16.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c06a0d0 con 0x7f343c0b4350 2023-10-26T05:58:16.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34240106c0 con 0x7f343c0b4350 2023-10-26T05:58:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.019+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c154710 con 0x7f343c0b4350 2023-10-26T05:58:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.023+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c066ae0 con 0x7f343c0b4350 2023-10-26T05:58:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.023+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3424010820 con 0x7f343c0b4350 2023-10-26T05:58:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.023+0000 7f34337fe640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3420066470 0x7f3420068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.023+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3424084110 con 0x7f343c0b4350 2023-10-26T05:58:16.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.023+0000 7f343affd640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3420066470 0x7f3420068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.023+0000 7f343affd640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3420066470 0x7f3420068930 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f34340af000 tx=0x7f343401b370 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:16.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.027+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3424051890 con 0x7f343c0b4350 2023-10-26T05:58:16.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.179+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f343c0721e0 con 0x7f343c0b4350 2023-10-26T05:58:16.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.179+0000 7f34337fe640 1 -- 172.21.15.71:0/3482667904 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3424054eb0 con 0x7f343c0b4350 2023-10-26T05:58:16.188 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:58:16.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.179+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3420066470 msgr2=0x7f3420068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.179+0000 7f34412c6640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3420066470 0x7f3420068930 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f34340af000 tx=0x7f343401b370 comp rx=0 tx=0).stop 2023-10-26T05:58:16.190 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.179+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 msgr2=0x7f343c0c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.179+0000 7f34412c6640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 0x7f343c0c4600 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f342400d980 tx=0x7f342400de40 comp rx=0 tx=0).stop 2023-10-26T05:58:16.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 shutdown_connections 2023-10-26T05:58:16.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f343c15b430 0x7f343c15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3420066470 0x7f3420068930 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.191 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f343c0b4350 0x7f343c0c4600 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 --2- 172.21.15.71:0/3482667904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f343c06a810 0x7f343c0c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 >> 172.21.15.71:0/3482667904 conn(0x7f343c0d12c0 msgr2=0x7f343c0cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:16.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 shutdown_connections 2023-10-26T05:58:16.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.183+0000 7f34412c6640 1 -- 172.21.15.71:0/3482667904 wait complete. 2023-10-26T05:58:16.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-26T05:58:16.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- 172.21.15.71:0/2477245642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 msgr2=0x7f423c15b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- 172.21.15.71:0/2477245642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c15b8e0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f423c08a140 tx=0x7f423401cc40 comp rx=0 tx=0).stop 2023-10-26T05:58:16.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- 172.21.15.71:0/2477245642 shutdown_connections 2023-10-26T05:58:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- 172.21.15.71:0/2477245642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f423c0d0850 0x7f423c15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- 172.21.15.71:0/2477245642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c15b8e0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- 172.21.15.71:0/2477245642 >> v2:172.21.15.86:3300/0 conn(0x7f423c0d0470 0x7f423c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- 172.21.15.71:0/2477245642 >> 172.21.15.71:0/2477245642 conn(0x7f423c0cdd80 msgr2=0x7f423c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:16.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- 172.21.15.71:0/2477245642 shutdown_connections 2023-10-26T05:58:16.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- 172.21.15.71:0/2477245642 wait complete. 2023-10-26T05:58:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 Processor -- start 2023-10-26T05:58:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- start start 2023-10-26T05:58:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f423c0d0470 0x7f423c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.335 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f423c0d0850 0x7f423c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0c5370 con 0x7f423c0cd880 2023-10-26T05:58:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0c7eb0 con 0x7f423c0d0470 2023-10-26T05:58:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f42437ed640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c149c50 con 0x7f423c0d0850 2023-10-26T05:58:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f4241589640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f4241589640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47518/0 (socket says 172.21.15.71:47518) 2023-10-26T05:58:16.336 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f4241589640 1 -- 172.21.15.71:0/2031562489 learned_addr learned my addr 172.21.15.71:0/2031562489 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.323+0000 7f4240d88640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f423c0d0470 0x7f423c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241d8a640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f423c0d0850 0x7f423c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241589640 1 -- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f423c0d0850 msgr2=0x7f423c149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241589640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f423c0d0850 0x7f423c149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241589640 1 -- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f423c0d0470 msgr2=0x7f423c0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.337 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241589640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f423c0d0470 0x7f423c0c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241589640 1 -- 172.21.15.71:0/2031562489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340098e0 con 0x7f423c0cd880 2023-10-26T05:58:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4240d88640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f423c0d0470 0x7f423c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4241589640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c0c4e30 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f423800c940 tx=0x7f423800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238007bb0 con 0x7f423c0cd880 2023-10-26T05:58:16.338 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4238007e20 con 0x7f423c0cd880 2023-10-26T05:58:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c0c5560 con 0x7f423c0cd880 2023-10-26T05:58:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800f6b0 con 0x7f423c0cd880 2023-10-26T05:58:16.339 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c0c5b20 con 0x7f423c0cd880 2023-10-26T05:58:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c066b10 con 0x7f423c0cd880 2023-10-26T05:58:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f423800f810 con 0x7f423c0cd880 2023-10-26T05:58:16.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4231ffb640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4220066470 0x7f4220068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f42380844c0 con 0x7f423c0cd880 2023-10-26T05:58:16.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4240d88640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4220066470 0x7f4220068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.327+0000 7f4240d88640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4220066470 0x7f4220068930 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f422c005fd0 tx=0x7f422c005f20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:16.341 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.331+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4238051c40 con 0x7f423c0cd880 2023-10-26T05:58:16.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.523+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f423c0721e0 con 0x7f423c0cd880 2023-10-26T05:58:16.588 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.579+0000 7f4231ffb640 1 -- 172.21.15.71:0/2031562489 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v526) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4238055260 con 0x7f423c0cd880 2023-10-26T05:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4220066470 msgr2=0x7f4220068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4220066470 0x7f4220068930 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f422c005fd0 tx=0x7f422c005f20 comp rx=0 tx=0).stop 2023-10-26T05:58:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 msgr2=0x7f423c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c0c4e30 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f423800c940 tx=0x7f423800ce00 comp rx=0 tx=0).stop 2023-10-26T05:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 shutdown_connections 2023-10-26T05:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f423c0d0850 0x7f423c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f423c0d0470 0x7f423c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4220066470 0x7f4220068930 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 --2- 172.21.15.71:0/2031562489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f423c0cd880 0x7f423c0c4e30 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 >> 172.21.15.71:0/2031562489 conn(0x7f423c0cdd80 msgr2=0x7f423c15a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 shutdown_connections 2023-10-26T05:58:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.583+0000 7f42437ed640 1 -- 172.21.15.71:0/2031562489 wait complete. 2023-10-26T05:58:16.608 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2023-10-26T05:58:16.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/2534850096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 msgr2=0x7fcd240caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 --2- 172.21.15.71:0/2534850096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd240caad0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009840 tx=0x7fcd0c01c800 comp rx=0 tx=0).stop 2023-10-26T05:58:16.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/2534850096 shutdown_connections 2023-10-26T05:58:16.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 --2- 172.21.15.71:0/2534850096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd240cb010 0x7fcd24159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 --2- 172.21.15.71:0/2534850096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd240caad0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/2534850096 >> 172.21.15.71:0/2534850096 conn(0x7fcd24069770 msgr2=0x7fcd24069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:16.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/2534850096 shutdown_connections 2023-10-26T05:58:16.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/2534850096 wait complete. 2023-10-26T05:58:16.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 Processor -- start 2023-10-26T05:58:16.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- start start 2023-10-26T05:58:16.738 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd2414f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 0x7fcd24158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2413c2f0 0x7fcd24158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd24139190 con 0x7fcd240cb010 2023-10-26T05:58:16.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2413f640 con 0x7fcd240b2a60 2023-10-26T05:58:16.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd24138860 con 0x7fcd2413c2f0 2023-10-26T05:58:16.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 0x7fcd24158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 0x7fcd24158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47520/0 (socket says 172.21.15.71:47520) 2023-10-26T05:58:16.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 -- 172.21.15.71:0/365310755 learned_addr learned my addr 172.21.15.71:0/365310755 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:16.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd2259c640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd2414f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd22d9d640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2413c2f0 0x7fcd24158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 -- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2413c2f0 msgr2=0x7fcd24158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2413c2f0 0x7fcd24158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 -- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 msgr2=0x7fcd2414f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd2414f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 -- 172.21.15.71:0/365310755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0094a0 con 0x7fcd240cb010 2023-10-26T05:58:16.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd2259c640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd2414f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:16.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd21d9b640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 0x7fcd24158600 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c01d0a0 tx=0x7fcd0c0a6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:16.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c004240 con 0x7fcd240cb010 2023-10-26T05:58:16.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcd0c004490 con 0x7fcd240cb010 2023-10-26T05:58:16.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd24138b40 con 0x7fcd240cb010 2023-10-26T05:58:16.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c005280 con 0x7fcd240cb010 2023-10-26T05:58:16.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.727+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2413b4c0 con 0x7fcd240cb010 2023-10-26T05:58:16.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.731+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd24066b10 con 0x7fcd240cb010 2023-10-26T05:58:16.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.731+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd0c0053e0 con 0x7fcd240cb010 2023-10-26T05:58:16.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.731+0000 7fcd1affd640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd04066470 0x7fcd04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:16.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.731+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcd0c0aa070 con 0x7fcd240cb010 2023-10-26T05:58:16.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.731+0000 7fcd2259c640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd04066470 0x7fcd04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:16.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.731+0000 7fcd2259c640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd04066470 0x7fcd04068930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001670 tx=0x7fcd14009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:16.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.735+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcd0c0ebb00 con 0x7fcd240cb010 2023-10-26T05:58:16.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config dump"} v 0) v1 -- 0x7fcd240721e0 con 0x7fcd240cb010 2023-10-26T05:58:16.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd1affd640 1 -- 172.21.15.71:0/365310755 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) v1 ==== 59+0+1116 (secure 0 0 0) 0x7fcd0c0ef120 con 0x7fcd240cb010 2023-10-26T05:58:16.888 INFO:teuthology.orchestra.run.smithi071.stdout:WHO MASK LEVEL OPTION VALUE RO 2023-10-26T05:58:16.888 INFO:teuthology.orchestra.run.smithi071.stdout:mon advanced mon_op_complaint_time 120 2023-10-26T05:58:16.888 INFO:teuthology.orchestra.run.smithi071.stdout:osd advanced osd_op_complaint_time 180.000000 2023-10-26T05:58:16.889 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds 20/20 2023-10-26T05:58:16.889 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_mds_balancer 20/20 2023-10-26T05:58:16.889 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced debug_ms 1/1 2023-10-26T05:58:16.889 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_frag true 2023-10-26T05:58:16.889 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_debug_scatterstat true 2023-10-26T05:58:16.889 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced mds_op_complaint_time 180.000000 2023-10-26T05:58:16.890 INFO:teuthology.orchestra.run.smithi071.stdout:mds dev mds_verify_scatter true 2023-10-26T05:58:16.890 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced osd_op_complaint_time 180.000000 2023-10-26T05:58:16.890 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_mon_op_timeout 900 2023-10-26T05:58:16.890 INFO:teuthology.orchestra.run.smithi071.stdout:mds advanced rados_osd_op_timeout 900 2023-10-26T05:58:16.891 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced client_mount_timeout 600 2023-10-26T05:58:16.891 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_client 20/20 2023-10-26T05:58:16.891 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced debug_ms 1/1 2023-10-26T05:58:16.891 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_mon_op_timeout 900 2023-10-26T05:58:16.891 INFO:teuthology.orchestra.run.smithi071.stdout:client advanced rados_osd_op_timeout 900 2023-10-26T05:58:16.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd04066470 msgr2=0x7fcd04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd04066470 0x7fcd04068930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001670 tx=0x7fcd14009040 comp rx=0 tx=0).stop 2023-10-26T05:58:16.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 msgr2=0x7fcd24158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:16.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 0x7fcd24158600 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c01d0a0 tx=0x7fcd0c0a6550 comp rx=0 tx=0).stop 2023-10-26T05:58:16.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 shutdown_connections 2023-10-26T05:58:16.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd2413c2f0 0x7fcd24158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcd04066470 0x7fcd04068930 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd240cb010 0x7fcd24158600 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 --2- 172.21.15.71:0/365310755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd240b2a60 0x7fcd2414f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 >> 172.21.15.71:0/365310755 conn(0x7fcd24069770 msgr2=0x7fcd240adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:16.894 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.879+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 shutdown_connections 2023-10-26T05:58:16.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:16.883+0000 7fcd28b89640 1 -- 172.21.15.71:0/365310755 wait complete. 2023-10-26T05:58:16.905 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-26T05:58:17.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:58:17.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:58:17.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- 172.21.15.71:0/3989044870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 msgr2=0x7f8d08003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/3989044870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d08003da0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009b60 tx=0x7f8d1001d220 comp rx=0 tx=0).stop 2023-10-26T05:58:17.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1ffff640 1 -- 172.21.15.71:0/3989044870 reap_dead start 2023-10-26T05:58:17.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- 172.21.15.71:0/3989044870 shutdown_connections 2023-10-26T05:58:17.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- 172.21.15.71:0/3989044870 >> 172.21.15.71:0/3989044870 conn(0x7f8d200aec60 msgr2=0x7f8d200af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:17.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- 172.21.15.71:0/3989044870 shutdown_connections 2023-10-26T05:58:17.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- 172.21.15.71:0/3989044870 wait complete. 2023-10-26T05:58:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 Processor -- start 2023-10-26T05:58:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- start start 2023-10-26T05:58:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d201f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d200b2bc0 0x7f8d201f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d200acb90 0x7f8d200ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200ae420 con 0x7f8d08003980 2023-10-26T05:58:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2013b7f0 con 0x7f8d200b2bc0 2023-10-26T05:58:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d25bd5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200ad550 con 0x7f8d200acb90 2023-10-26T05:58:17.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d201f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d201f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47536/0 (socket says 172.21.15.71:47536) 2023-10-26T05:58:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 -- 172.21.15.71:0/2533022612 learned_addr learned my addr 172.21.15.71:0/2533022612 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1effd640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d200b2bc0 0x7f8d201f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1ffff640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d200acb90 0x7f8d200ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.041 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 -- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d200acb90 msgr2=0x7f8d200ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d200acb90 0x7f8d200ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 -- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d200b2bc0 msgr2=0x7f8d201f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d200b2bc0 0x7f8d201f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 -- 172.21.15.71:0/2533022612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d20137d50 con 0x7f8d08003980 2023-10-26T05:58:17.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.027+0000 7f8d1f7fe640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d201f61f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f8d1001d050 tx=0x7f8d10005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004a10 con 0x7f8d08003980 2023-10-26T05:58:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8d10004c60 con 0x7f8d08003980 2023-10-26T05:58:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200ad7a0 con 0x7f8d08003980 2023-10-26T05:58:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d100b6410 con 0x7f8d08003980 2023-10-26T05:58:17.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d200add60 con 0x7f8d08003980 2023-10-26T05:58:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20066b10 con 0x7f8d08003980 2023-10-26T05:58:17.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8d100b3070 con 0x7f8d08003980 2023-10-26T05:58:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d07fff640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066490 0x7f8d0c068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d1011ed30 con 0x7f8d08003980 2023-10-26T05:58:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d1effd640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066490 0x7f8d0c068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.031+0000 7f8d1effd640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066490 0x7f8d0c068950 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8d20159e20 tx=0x7f8d14001690 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:17.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.035+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8d100ec5f0 con 0x7f8d08003980 2023-10-26T05:58:17.194 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.187+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8d200721e0 con 0x7f8d08003980 2023-10-26T05:58:17.200 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.191+0000 7f8d07fff640 1 -- 172.21.15.71:0/2533022612 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8d100b10c0 con 0x7f8d08003980 2023-10-26T05:58:17.202 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066490 msgr2=0x7f8d0c068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066490 0x7f8d0c068950 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8d20159e20 tx=0x7f8d14001690 comp rx=0 tx=0).stop 2023-10-26T05:58:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 msgr2=0x7f8d201f61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d201f61f0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f8d1001d050 tx=0x7f8d10005620 comp rx=0 tx=0).stop 2023-10-26T05:58:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 shutdown_connections 2023-10-26T05:58:17.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8d200acb90 0x7f8d200ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d200b2bc0 0x7f8d201f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8d0c066490 0x7f8d0c068950 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 --2- 172.21.15.71:0/2533022612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d08003980 0x7f8d201f61f0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 >> 172.21.15.71:0/2533022612 conn(0x7f8d200aec60 msgr2=0x7f8d200cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:17.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 shutdown_connections 2023-10-26T05:58:17.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.195+0000 7f8d25bd5640 1 -- 172.21.15.71:0/2533022612 wait complete. 2023-10-26T05:58:17.218 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:58:17.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:58:17.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:58:17.333 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:58:17.333 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:17.334 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:17.339 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:58:17.339 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:58:17.340 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:17.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:58:17.346 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:17.347 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:17.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:58:17.386 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:58:17.386 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:17.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:58:17.392 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-26T05:58:17.392 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:17.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/3732027396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6914004d10 msgr2=0x7f6914005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 --2- 172.21.15.71:0/3732027396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6914004d10 0x7f6914005190 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f691800c640 tx=0x7f691801d380 comp rx=0 tx=0).stop 2023-10-26T05:58:17.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6926d9d640 1 -- 172.21.15.71:0/3732027396 reap_dead start 2023-10-26T05:58:17.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/3732027396 shutdown_connections 2023-10-26T05:58:17.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/3732027396 >> 172.21.15.71:0/3732027396 conn(0x7f6928069770 msgr2=0x7f6928069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:17.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/3732027396 shutdown_connections 2023-10-26T05:58:17.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/3732027396 wait complete. 2023-10-26T05:58:17.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 Processor -- start 2023-10-26T05:58:17.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- start start 2023-10-26T05:58:17.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69280b1cb0 0x7f69280b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 0x7f69280b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69280b0bc0 0x7f69281e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69280abc00 con 0x7f69280ac020 2023-10-26T05:58:17.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69280b2090 con 0x7f69280b0bc0 2023-10-26T05:58:17.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6928137a60 con 0x7f69280b1cb0 2023-10-26T05:58:17.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 0x7f69280b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 0x7f69280b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47550/0 (socket says 172.21.15.71:47550) 2023-10-26T05:58:17.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 -- 172.21.15.71:0/943212696 learned_addr learned my addr 172.21.15.71:0/943212696 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:17.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6926d9d640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69280b0bc0 0x7f69281e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 -- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69280b1cb0 msgr2=0x7f69280b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692659c640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69280b1cb0 0x7f69280b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69280b1cb0 0x7f69280b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 -- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69280b0bc0 msgr2=0x7f69281e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69280b0bc0 0x7f69281e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 -- 172.21.15.71:0/943212696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69280b1070 con 0x7f69280ac020 2023-10-26T05:58:17.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6926d9d640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69280b0bc0 0x7f69281e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:17.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f6925d9b640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 0x7f69280b0680 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f691000e760 tx=0x7f691000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:17.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000cc60 con 0x7f69280ac020 2023-10-26T05:58:17.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f691000ceb0 con 0x7f69280ac020 2023-10-26T05:58:17.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69280ab2a0 con 0x7f69280ac020 2023-10-26T05:58:17.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910005240 con 0x7f69280ac020 2023-10-26T05:58:17.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69280ab540 con 0x7f69280ac020 2023-10-26T05:58:17.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.547+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6928066b10 con 0x7f69280ac020 2023-10-26T05:58:17.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.551+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f69100053a0 con 0x7f69280ac020 2023-10-26T05:58:17.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.551+0000 7f691effd640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f69080664a0 0x7f6908068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.551+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6910084320 con 0x7f69280ac020 2023-10-26T05:58:17.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.551+0000 7f692659c640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f69080664a0 0x7f6908068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.551+0000 7f692659c640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f69080664a0 0x7f6908068960 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6920005d90 tx=0x7f6920005ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:17.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.555+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6910051aa0 con 0x7f69280ac020 2023-10-26T05:58:17.748 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.739+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69280abd40 con 0x7f69280ac020 2023-10-26T05:58:17.749 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.739+0000 7f691effd640 1 -- 172.21.15.71:0/943212696 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3690 (secure 0 0 0) 0x7f69280abd40 con 0x7f69280ac020 2023-10-26T05:58:17.749 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:17.749 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/3218704512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3218704512},{"type":"v1","addr":"172.21.15.86:6836","nonce":3218704512}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":6125,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/334797606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":334797606},{"type":"v1","addr":"172.21.15.86:6837","nonce":334797606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8952,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/980375221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":980375221},{"type":"v1","addr":"172.21.15.71:6833","nonce":980375221}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":9315,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/626118458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":626118458},{"type":"v1","addr":"172.21.15.71:6835","nonce":626118458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[]} 2023-10-26T05:58:17.750 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 66 2023-10-26T05:58:17.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f69080664a0 msgr2=0x7f6908068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f69080664a0 0x7f6908068960 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6920005d90 tx=0x7f6920005ca0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.751 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 msgr2=0x7f69280b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 0x7f69280b0680 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f691000e760 tx=0x7f691000ec20 comp rx=0 tx=0).stop 2023-10-26T05:58:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 shutdown_connections 2023-10-26T05:58:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69280b0bc0 0x7f69281e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f69080664a0 0x7f6908068960 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.752 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69280ac020 0x7f69280b0680 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 --2- 172.21.15.71:0/943212696 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f69280b1cb0 0x7f69280b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 >> 172.21.15.71:0/943212696 conn(0x7f6928069770 msgr2=0x7f69280acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 shutdown_connections 2023-10-26T05:58:17.753 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.743+0000 7f692ce48640 1 -- 172.21.15.71:0/943212696 wait complete. 2023-10-26T05:58:17.768 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-26T05:58:17.768 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-26T05:58:17.769 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:58:17.769 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:58:17.769 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-26T05:58:17.770 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-26T05:58:17.770 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:58:17.770 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:58:17.770 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-26T05:58:17.771 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-26T05:58:17.771 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:58:17.771 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:58:17.772 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-26T05:58:17.772 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-26T05:58:17.772 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:58:17.772 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:58:17.778 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:58:17.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-26T05:58:17.780 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:58:17.780 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-26T05:58:17.791 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:58:17.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-26T05:58:17.792 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:58:17.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-26T05:58:17.822 INFO:tasks.ceph.mds.a:Started 2023-10-26T05:58:17.823 DEBUG:teuthology.parallel:result is None 2023-10-26T05:58:17.824 INFO:tasks.ceph.mds.c:Started 2023-10-26T05:58:17.824 DEBUG:teuthology.parallel:result is None 2023-10-26T05:58:17.837 INFO:tasks.ceph.mds.b:Started 2023-10-26T05:58:17.837 DEBUG:teuthology.parallel:result is None 2023-10-26T05:58:17.838 INFO:tasks.ceph.mds.d:Started 2023-10-26T05:58:17.838 DEBUG:teuthology.parallel:result is None 2023-10-26T05:58:17.839 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-26T05:58:17.886 INFO:tasks.ceph.mds.a.smithi071.stdout:starting mds.a at 2023-10-26T05:58:17.887 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:58:17.909 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-26T05:58:17.910 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-26T05:58:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/3708255866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0b15f0 msgr2=0x7f3bfc1586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/3708255866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0b15f0 0x7f3bfc1586c0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009840 tx=0x7f3bf001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/3708255866 shutdown_connections 2023-10-26T05:58:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/3708255866 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bfc06a000 0x7f3bfc06a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.955 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/3708255866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0b15f0 0x7f3bfc1586c0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/3708255866 >> 172.21.15.71:0/3708255866 conn(0x7f3bfc06b8c0 msgr2=0x7f3bfc06bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/3708255866 shutdown_connections 2023-10-26T05:58:17.956 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/3708255866 wait complete. 2023-10-26T05:58:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 Processor -- start 2023-10-26T05:58:17.958 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- start start 2023-10-26T05:58:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bfc06a000 0x7f3bfc14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bfc0b15f0 0x7f3bfc14b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 0x7f3bfc14dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc14b890 con 0x7f3bfc0cc730 2023-10-26T05:58:17.959 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc154850 con 0x7f3bfc06a000 2023-10-26T05:58:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc154b30 con 0x7f3bfc0b15f0 2023-10-26T05:58:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 0x7f3bfc14dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 0x7f3bfc14dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47588/0 (socket says 172.21.15.71:47588) 2023-10-26T05:58:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 -- 172.21.15.71:0/2980987930 learned_addr learned my addr 172.21.15.71:0/2980987930 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfa59c640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bfc06a000 0x7f3bfc14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.960 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bf9d9b640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bfc0b15f0 0x7f3bfc14b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 -- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bfc0b15f0 msgr2=0x7f3bfc14b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bfc0b15f0 0x7f3bfc14b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 -- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bfc06a000 msgr2=0x7f3bfc14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bfc06a000 0x7f3bfc14a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 -- 172.21.15.71:0/2980987930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00094a0 con 0x7f3bfc0cc730 2023-10-26T05:58:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3bfad9d640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 0x7f3bfc14dfe0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00ea20 tx=0x7f3bec00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec00cde0 con 0x7f3bfc0cc730 2023-10-26T05:58:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc154e10 con 0x7f3bfc0cc730 2023-10-26T05:58:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3bec004c40 con 0x7f3bfc0cc730 2023-10-26T05:58:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec005020 con 0x7f3bfc0cc730 2023-10-26T05:58:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.947+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc14e8d0 con 0x7f3bfc0cc730 2023-10-26T05:58:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.951+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc066b10 con 0x7f3bfc0cc730 2023-10-26T05:58:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.951+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3bec005180 con 0x7f3bfc0cc730 2023-10-26T05:58:17.963 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.951+0000 7f3beaffd640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bdc066470 0x7f3bdc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.951+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3bec084690 con 0x7f3bfc0cc730 2023-10-26T05:58:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.951+0000 7f3bfa59c640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bdc066470 0x7f3bdc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.951+0000 7f3bfa59c640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bdc066470 0x7f3bdc068930 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40046c0 tx=0x7f3bf4007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:17.964 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:17.955+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3bec051db0 con 0x7f3bfc0cc730 2023-10-26T05:58:18.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.115+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3bfc14eac0 con 0x7f3bfc0cc730 2023-10-26T05:58:18.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.611+0000 7f3beaffd640 1 -- 172.21.15.71:0/2980987930 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v182) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3bec01f020 con 0x7f3bfc0cc730 2023-10-26T05:58:18.620 INFO:teuthology.orchestra.run.smithi071.stderr: removed all blocklist entries 2023-10-26T05:58:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bdc066470 msgr2=0x7f3bdc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bdc066470 0x7f3bdc068930 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40046c0 tx=0x7f3bf4007ec0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 msgr2=0x7f3bfc14dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 0x7f3bfc14dfe0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00ea20 tx=0x7f3bec00eee0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 shutdown_connections 2023-10-26T05:58:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3bdc066470 0x7f3bdc068930 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bfc0cc730 0x7f3bfc14dfe0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3bfc0b15f0 0x7f3bfc14b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 --2- 172.21.15.71:0/2980987930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bfc06a000 0x7f3bfc14a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 >> 172.21.15.71:0/2980987930 conn(0x7f3bfc06b8c0 msgr2=0x7f3bfc0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:18.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 shutdown_connections 2023-10-26T05:58:18.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.615+0000 7f3c00ab8640 1 -- 172.21.15.71:0/2980987930 wait complete. 2023-10-26T05:58:18.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-26T05:58:18.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 -- 172.21.15.71:0/3269960995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9950142d20 msgr2=0x7f99500cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:18.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 --2- 172.21.15.71:0/3269960995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9950142d20 0x7f99500cb3d0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f995006dbd0 tx=0x7f994c01c870 comp rx=0 tx=0).stop 2023-10-26T05:58:18.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 -- 172.21.15.71:0/3269960995 shutdown_connections 2023-10-26T05:58:18.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 --2- 172.21.15.71:0/3269960995 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f995006a000 0x7f995006a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 --2- 172.21.15.71:0/3269960995 >> v2:172.21.15.86:3300/0 conn(0x7f995006a9a0 0x7f995006c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 --2- 172.21.15.71:0/3269960995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9950142d20 0x7f99500cb3d0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 -- 172.21.15.71:0/3269960995 >> 172.21.15.71:0/3269960995 conn(0x7f99500ce270 msgr2=0x7f99500ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:18.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 -- 172.21.15.71:0/3269960995 shutdown_connections 2023-10-26T05:58:18.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 -- 172.21.15.71:0/3269960995 wait complete. 2023-10-26T05:58:18.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 Processor -- start 2023-10-26T05:58:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.763+0000 7f99586bc640 1 -- start start 2023-10-26T05:58:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 0x7f9950157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995006a9a0 0x7f9950140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9950142d20 0x7f9950141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:18.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995013bf60 con 0x7f995006a000 2023-10-26T05:58:18.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950158140 con 0x7f995006a9a0 2023-10-26T05:58:18.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995015e1f0 con 0x7f9950142d20 2023-10-26T05:58:18.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 0x7f9950157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:18.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 0x7f9950157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47612/0 (socket says 172.21.15.71:47612) 2023-10-26T05:58:18.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 -- 172.21.15.71:0/1366042729 learned_addr learned my addr 172.21.15.71:0/1366042729 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:18.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 -- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9950142d20 msgr2=0x7f9950141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:18.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9950142d20 0x7f9950141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 -- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995006a9a0 msgr2=0x7f9950140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:18.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995006a9a0 0x7f9950140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:18.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 -- 172.21.15.71:0/1366042729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c009560 con 0x7f995006a000 2023-10-26T05:58:18.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9956458640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 0x7f9950157c00 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f994c0b0040 tx=0x7f994c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:18.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0a58f0 con 0x7f995006a000 2023-10-26T05:58:18.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f994c0038f0 con 0x7f995006a000 2023-10-26T05:58:18.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995015e4d0 con 0x7f995006a000 2023-10-26T05:58:18.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c003d00 con 0x7f995006a000 2023-10-26T05:58:18.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995015e7b0 con 0x7f995006a000 2023-10-26T05:58:18.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995013c0a0 con 0x7f995006a000 2023-10-26T05:58:18.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f994c0b4070 con 0x7f995006a000 2023-10-26T05:58:18.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9946ffd640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99340664a0 0x7f9934068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:18.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f994c11dbe0 con 0x7f995006a000 2023-10-26T05:58:18.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.767+0000 7f9955c57640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99340664a0 0x7f9934068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:18.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.771+0000 7f9955c57640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99340664a0 0x7f9934068960 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9950140700 tx=0x7f9940006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:18.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.771+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f995013c0a0 con 0x7f995006a000 2023-10-26T05:58:18.987 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f995013c0a0 con 0x7f995006a000 2023-10-26T05:58:18.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f9946ffd640 1 -- 172.21.15.71:0/1366042729 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f995013c0a0 con 0x7f995006a000 2023-10-26T05:58:18.988 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:18.988 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:18.989 INFO:teuthology.orchestra.run.smithi071.stdout: "auth_dump": [ 2023-10-26T05:58:18.989 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:18.989 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.a", 2023-10-26T05:58:18.990 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllTYIYJBAAtwVeU5k1nDsTz8TzqRlZyg==", 2023-10-26T05:58:18.990 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:18.990 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:18.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:58:18.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:58:18.992 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:18.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:18.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:18.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:18.993 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.b", 2023-10-26T05:58:18.993 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllWp/4MhAA3P3PxKYmMMpI0pCe4mesKA==", 2023-10-26T05:58:18.994 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:18.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:18.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:58:18.994 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:58:18.995 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:18.995 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:18.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:18.995 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:18.995 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.c", 2023-10-26T05:58:18.996 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllEOquLBAAiyREBi+QshnJ/1otqGEexw==", 2023-10-26T05:58:18.996 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:18.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:18.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:58:18.996 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:58:18.997 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:18.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:18.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:18.997 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:18.997 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mds.d", 2023-10-26T05:58:18.998 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllxdrsOBAADk0WZjh828ulj0do4Mvr+w==", 2023-10-26T05:58:18.998 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:18.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:18.998 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:58:18.999 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:58:18.999 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:18.999 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:18.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:18.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.000 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.0", 2023-10-26T05:58:19.000 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllggcYChAApwJFiU2vpc6dIIsyMLagdg==", 2023-10-26T05:58:19.000 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.000 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.001 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.001 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.1", 2023-10-26T05:58:19.002 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllKlrmEBAAbDSI4jLKn6QAz7F9rkY1Jw==", 2023-10-26T05:58:19.002 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.002 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.003 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.004 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.2", 2023-10-26T05:58:19.004 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/TllLdZbGhAAbS0kgh7Qx4wyT5tI6NPqOA==", 2023-10-26T05:58:19.004 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.004 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.005 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.005 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.005 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.006 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.3", 2023-10-26T05:58:19.006 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBK/Tll+rPLJBAAM7nIKanBvaj6zTLDJka0Lg==", 2023-10-26T05:58:19.006 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.006 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.007 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.007 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.007 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.4", 2023-10-26T05:58:19.008 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/Tllt5/DJBAAZUZ8Mo4ds2xgPG0gFD5WSg==", 2023-10-26T05:58:19.008 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.008 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.008 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.009 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.009 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.5", 2023-10-26T05:58:19.009 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllhAqrKxAA+eHNx1PEIXdatn3q74Da8w==", 2023-10-26T05:58:19.009 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.010 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.010 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.010 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.010 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.011 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.011 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.6", 2023-10-26T05:58:19.011 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBN/TllU8BQNRAAzzdSsMfzYApNoQBPGDuLmA==", 2023-10-26T05:58:19.011 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.011 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.012 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.012 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.012 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.013 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "osd.7", 2023-10-26T05:58:19.013 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBO/TllAq7rAxAAZI32cHTPYlANv6uUKIfIhw==", 2023-10-26T05:58:19.013 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.013 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow profile osd", 2023-10-26T05:58:19.013 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile osd", 2023-10-26T05:58:19.013 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.014 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.014 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.0", 2023-10-26T05:58:19.014 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg==", 2023-10-26T05:58:19.015 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.015 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:19.015 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow r", 2023-10-26T05:58:19.016 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-26T05:58:19.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.017 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.1", 2023-10-26T05:58:19.017 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllMhrYChAA9gmd2xGLytFDxb8oVQBnmQ==", 2023-10-26T05:58:19.017 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.018 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:19.018 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:58:19.018 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:58:19.018 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:58:19.019 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.019 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.2", 2023-10-26T05:58:19.019 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/Tll/cziBxAAZlcDcc55bSYwn7PPBo0LYA==", 2023-10-26T05:58:19.020 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.020 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:19.020 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:58:19.020 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:58:19.020 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:58:19.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.021 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.021 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.3", 2023-10-26T05:58:19.021 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBH/TllfPvsDxAAAZVT1Fujs0CFYWllJbw29Q==", 2023-10-26T05:58:19.021 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow", 2023-10-26T05:58:19.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow r", 2023-10-26T05:58:19.022 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow rw", 2023-10-26T05:58:19.022 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow rwx" 2023-10-26T05:58:19.022 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.023 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "client.admin", 2023-10-26T05:58:19.023 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/TllH0VuDRAAOv4mNpzJAXrp3T5LTy0mXg==", 2023-10-26T05:58:19.023 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.023 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:58:19.024 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": "allow *", 2023-10-26T05:58:19.024 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow *", 2023-10-26T05:58:19.024 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.x", 2023-10-26T05:58:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tll1UJpIRAAFnGldqXAX8tu8lxjrUcfOw==", 2023-10-26T05:58:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:58:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:58:19.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.026 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:19.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "entity": "mgr.y", 2023-10-26T05:58:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "key": "AQBG/Tlli+V/HhAAEQj/tlXCUP8HsA6gGsYTRA==", 2023-10-26T05:58:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": { 2023-10-26T05:58:19.028 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": "allow *", 2023-10-26T05:58:19.028 INFO:teuthology.orchestra.run.smithi071.stdout: "mon": "allow profile mgr", 2023-10-26T05:58:19.029 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": "allow *" 2023-10-26T05:58:19.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.029 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.030 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:19.030 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:19.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99340664a0 msgr2=0x7f9934068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.033 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99340664a0 0x7f9934068960 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9950140700 tx=0x7f9940006ca0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 msgr2=0x7f9950157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 0x7f9950157c00 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f994c0b0040 tx=0x7f994c005790 comp rx=0 tx=0).stop 2023-10-26T05:58:19.034 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 shutdown_connections 2023-10-26T05:58:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9950142d20 0x7f9950141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f995006a9a0 0x7f9950140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.035 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f99340664a0 0x7f9934068960 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 --2- 172.21.15.71:0/1366042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f995006a000 0x7f9950157c00 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 >> 172.21.15.71:0/1366042729 conn(0x7f99500ce270 msgr2=0x7f99500ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.036 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 shutdown_connections 2023-10-26T05:58:19.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:18.979+0000 7f99586bc640 1 -- 172.21.15.71:0/1366042729 wait complete. 2023-10-26T05:58:19.037 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:19.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/1115418481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0d0470 msgr2=0x7faeec06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- 172.21.15.71:0/1115418481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0d0470 0x7faeec06aa70 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7faeec0c4dd0 tx=0x7faee001c810 comp rx=0 tx=0).stop 2023-10-26T05:58:19.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/1115418481 shutdown_connections 2023-10-26T05:58:19.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- 172.21.15.71:0/1115418481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faeec0cd880 0x7faeec15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- 172.21.15.71:0/1115418481 >> v2:172.21.15.86:3300/0 conn(0x7faeec0d0850 0x7faeec15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- 172.21.15.71:0/1115418481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0d0470 0x7faeec06aa70 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/1115418481 >> 172.21.15.71:0/1115418481 conn(0x7faeec0cdd80 msgr2=0x7faeec0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/1115418481 shutdown_connections 2023-10-26T05:58:19.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/1115418481 wait complete. 2023-10-26T05:58:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 Processor -- start 2023-10-26T05:58:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- start start 2023-10-26T05:58:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 0x7faeec0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faeec0d0470 0x7faeec15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faeec0d0850 0x7faeec152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1529e0 con 0x7faeec0cd880 2023-10-26T05:58:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec151dd0 con 0x7faeec0d0850 2023-10-26T05:58:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeec1520b0 con 0x7faeec0d0470 2023-10-26T05:58:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 0x7faeec0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 0x7faeec0c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47632/0 (socket says 172.21.15.71:47632) 2023-10-26T05:58:19.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 -- 172.21.15.71:0/444720036 learned_addr learned my addr 172.21.15.71:0/444720036 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:19.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef168d640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faeec0d0850 0x7faeec152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faeebfff640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faeec0d0470 0x7faeec15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 -- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faeec0d0470 msgr2=0x7faeec15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faeec0d0470 0x7faeec15a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 -- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faeec0d0850 msgr2=0x7faeec152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faeec0d0850 0x7faeec152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 -- 172.21.15.71:0/444720036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0009560 con 0x7faeec0cd880 2023-10-26T05:58:19.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef0e8c640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 0x7faeec0c6670 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7faee00b0040 tx=0x7faee00a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:19.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0005440 con 0x7faeec0cd880 2023-10-26T05:58:19.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faee00046d0 con 0x7faeec0cd880 2023-10-26T05:58:19.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0004ae0 con 0x7faeec0cd880 2023-10-26T05:58:19.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec152390 con 0x7faeec0cd880 2023-10-26T05:58:19.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.155+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec0c6bb0 con 0x7faeec0cd880 2023-10-26T05:58:19.171 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.159+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec066b10 con 0x7faeec0cd880 2023-10-26T05:58:19.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.159+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faee00b4070 con 0x7faeec0cd880 2023-10-26T05:58:19.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.159+0000 7faee97fa640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faed8066470 0x7faed8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.159+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faee011e100 con 0x7faeec0cd880 2023-10-26T05:58:19.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.159+0000 7faeebfff640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faed8066470 0x7faed8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.159+0000 7faeebfff640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faed8066470 0x7faed8068930 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faeec1f8030 tx=0x7faedc005c30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:19.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.163+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faee00eb8e0 con 0x7faeec0cd880 2023-10-26T05:58:19.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faeec0721e0 con 0x7faeec0cd880 2023-10-26T05:58:19.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faee97fa640 1 -- 172.21.15.71:0/444720036 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+12784 (secure 0 0 0) 0x7faee00eef00 con 0x7faeec0cd880 2023-10-26T05:58:19.327 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:19.328 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":184,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:18.630796+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:19.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faed8066470 msgr2=0x7faed8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faed8066470 0x7faed8068930 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7faeec1f8030 tx=0x7faedc005c30 comp rx=0 tx=0).stop 2023-10-26T05:58:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 msgr2=0x7faeec0c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 0x7faeec0c6670 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7faee00b0040 tx=0x7faee00a5870 comp rx=0 tx=0).stop 2023-10-26T05:58:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 shutdown_connections 2023-10-26T05:58:19.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faeec0d0850 0x7faeec152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faeec0d0470 0x7faeec15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faed8066470 0x7faed8068930 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 --2- 172.21.15.71:0/444720036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeec0cd880 0x7faeec0c6670 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 >> 172.21.15.71:0/444720036 conn(0x7faeec0cdd80 msgr2=0x7faeec0b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 shutdown_connections 2023-10-26T05:58:19.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.319+0000 7faef30f0640 1 -- 172.21.15.71:0/444720036 wait complete. 2023-10-26T05:58:19.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:58:19.347 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:58:19.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 -- 172.21.15.71:0/3714319372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a80052b0 msgr2=0x7fa5a800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/3714319372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a80052b0 0x7fa5a800b990 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009e00 tx=0x7fa5b401d150 comp rx=0 tx=0).stop 2023-10-26T05:58:19.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 -- 172.21.15.71:0/3714319372 shutdown_connections 2023-10-26T05:58:19.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/3714319372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5a80052b0 0x7fa5a800b990 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 -- 172.21.15.71:0/3714319372 >> 172.21.15.71:0/3714319372 conn(0x7fa5bc0b0f20 msgr2=0x7fa5bc0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 -- 172.21.15.71:0/3714319372 shutdown_connections 2023-10-26T05:58:19.475 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.463+0000 7fa5c3a10640 1 -- 172.21.15.71:0/3714319372 wait complete. 2023-10-26T05:58:19.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 Processor -- start 2023-10-26T05:58:19.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- start start 2023-10-26T05:58:19.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a80052b0 0x7fa5bc146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 0x7fa5bc152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5bc150dd0 0x7fa5bc153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc1536d0 con 0x7fa5bc150950 2023-10-26T05:58:19.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc0c6d30 con 0x7fa5a80052b0 2023-10-26T05:58:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc1513a0 con 0x7fa5bc150dd0 2023-10-26T05:58:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 0x7fa5bc152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 0x7fa5bc152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47652/0 (socket says 172.21.15.71:47652) 2023-10-26T05:58:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 -- 172.21.15.71:0/1033441393 learned_addr learned my addr 172.21.15.71:0/1033441393 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:19.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c17ac640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a80052b0 0x7fa5bc146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c1fad640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5bc150dd0 0x7fa5bc153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 -- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5bc150dd0 msgr2=0x7fa5bc153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5bc150dd0 0x7fa5bc153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 -- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a80052b0 msgr2=0x7fa5bc146a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a80052b0 0x7fa5bc146a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 -- 172.21.15.71:0/1033441393 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0c6fa0 con 0x7fa5bc150950 2023-10-26T05:58:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c17ac640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a80052b0 0x7fa5bc146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c0fab640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 0x7fa5bc152c50 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d380 tx=0x7fa5ac003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac010860 con 0x7fa5bc150950 2023-10-26T05:58:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa5ac010ab0 con 0x7fa5bc150950 2023-10-26T05:58:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc0c7280 con 0x7fa5bc150950 2023-10-26T05:58:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00f3b0 con 0x7fa5bc150950 2023-10-26T05:58:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc0c8c60 con 0x7fa5bc150950 2023-10-26T05:58:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.467+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc0c74c0 con 0x7fa5bc150950 2023-10-26T05:58:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.471+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa5ac00f510 con 0x7fa5bc150950 2023-10-26T05:58:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.471+0000 7fa5b1ffb640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a00664a0 0x7fa5a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.471+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa5ac0842e0 con 0x7fa5bc150950 2023-10-26T05:58:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.471+0000 7fa5c17ac640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a00664a0 0x7fa5a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.471+0000 7fa5c17ac640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a00664a0 0x7fa5a0068960 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80036b0 tx=0x7fa5b8003600 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.475+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa5bc0c74c0 con 0x7fa5bc150950 2023-10-26T05:58:19.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.627+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5bc0c74c0 con 0x7fa5bc150950 2023-10-26T05:58:19.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.627+0000 7fa5b1ffb640 1 -- 172.21.15.71:0/1033441393 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v184) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5ac014020 con 0x7fa5bc150950 2023-10-26T05:58:19.637 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:58:19.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a00664a0 msgr2=0x7fa5a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a00664a0 0x7fa5a0068960 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80036b0 tx=0x7fa5b8003600 comp rx=0 tx=0).stop 2023-10-26T05:58:19.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 msgr2=0x7fa5bc152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 0x7fa5bc152c50 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d380 tx=0x7fa5ac003f90 comp rx=0 tx=0).stop 2023-10-26T05:58:19.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 shutdown_connections 2023-10-26T05:58:19.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5bc150dd0 0x7fa5bc153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa5a00664a0 0x7fa5a0068960 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5bc150950 0x7fa5bc152c50 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 --2- 172.21.15.71:0/1033441393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5a80052b0 0x7fa5bc146a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 >> 172.21.15.71:0/1033441393 conn(0x7fa5bc0b0f20 msgr2=0x7fa5bc148d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.641 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 shutdown_connections 2023-10-26T05:58:19.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.631+0000 7fa5c3a10640 1 -- 172.21.15.71:0/1033441393 wait complete. 2023-10-26T05:58:19.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-26T05:58:19.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 -- 172.21.15.71:0/2869289764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 msgr2=0x7fbfa415d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.778 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 --2- 172.21.15.71:0/2869289764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa415d480 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbfa415f030 tx=0x7fbf9c01cc90 comp rx=0 tx=0).stop 2023-10-26T05:58:19.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 -- 172.21.15.71:0/2869289764 shutdown_connections 2023-10-26T05:58:19.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 --2- 172.21.15.71:0/2869289764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa415d480 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 --2- 172.21.15.71:0/2869289764 >> v2:172.21.15.86:3300/0 conn(0x7fbfa415fc90 0x7fbfa415f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 -- 172.21.15.71:0/2869289764 >> 172.21.15.71:0/2869289764 conn(0x7fbfa40ce270 msgr2=0x7fbfa40ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 -- 172.21.15.71:0/2869289764 shutdown_connections 2023-10-26T05:58:19.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.767+0000 7fbfac39e640 1 -- 172.21.15.71:0/2869289764 wait complete. 2023-10-26T05:58:19.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 Processor -- start 2023-10-26T05:58:19.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- start start 2023-10-26T05:58:19.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa406d770 0x7fbfa4156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa4159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfa415fc90 0x7fbfa41535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa415a640 con 0x7fbfa415f610 2023-10-26T05:58:19.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4156a50 con 0x7fbfa406d770 2023-10-26T05:58:19.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4156cf0 con 0x7fbfa415fc90 2023-10-26T05:58:19.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa4159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa4159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47676/0 (socket says 172.21.15.71:47676) 2023-10-26T05:58:19.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 -- 172.21.15.71:0/567045244 learned_addr learned my addr 172.21.15.71:0/567045244 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:19.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfaa13a640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa406d770 0x7fbfa4156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfaa93b640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfa415fc90 0x7fbfa41535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 -- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfa415fc90 msgr2=0x7fbfa41535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfa415fc90 0x7fbfa41535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 -- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa406d770 msgr2=0x7fbfa4156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa406d770 0x7fbfa4156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 -- 172.21.15.71:0/567045244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0098e0 con 0x7fbfa415f610 2023-10-26T05:58:19.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfaa93b640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfa415fc90 0x7fbfa41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:19.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfa9939640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa4159ad0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b410 tx=0x7fbf9400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:19.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94011830 con 0x7fbfa415f610 2023-10-26T05:58:19.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbf94011a80 con 0x7fbfa415f610 2023-10-26T05:58:19.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa415e7c0 con 0x7fbfa415f610 2023-10-26T05:58:19.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94011e90 con 0x7fbfa415f610 2023-10-26T05:58:19.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa4157080 con 0x7fbfa415f610 2023-10-26T05:58:19.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.771+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa4066d00 con 0x7fbfa415f610 2023-10-26T05:58:19.787 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.775+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbf9401a400 con 0x7fbfa415f610 2023-10-26T05:58:19.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.775+0000 7fbf9affd640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf880664a0 0x7fbf88068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:19.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.775+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf94083eb0 con 0x7fbfa415f610 2023-10-26T05:58:19.788 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.775+0000 7fbfaa13a640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf880664a0 0x7fbf88068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:19.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.775+0000 7fbfaa13a640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf880664a0 0x7fbf88068960 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4154930 tx=0x7fbfa0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:19.789 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.779+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbf94051680 con 0x7fbfa415f610 2023-10-26T05:58:19.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.923+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbfa40cf0a0 con 0x7fbfa415f610 2023-10-26T05:58:19.932 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.923+0000 7fbf9affd640 1 -- 172.21.15.71:0/567045244 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v184) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbf94015020 con 0x7fbfa415f610 2023-10-26T05:58:19.933 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:19.933 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2023-10-26T05:58:19.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:19.933 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2023-10-26T05:58:19.933 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2023-10-26T05:58:19.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:19.934 INFO:teuthology.orchestra.run.smithi071.stdout:] 2023-10-26T05:58:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf880664a0 msgr2=0x7fbf88068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf880664a0 0x7fbf88068960 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4154930 tx=0x7fbfa0009040 comp rx=0 tx=0).stop 2023-10-26T05:58:19.935 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 msgr2=0x7fbfa4159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:19.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa4159ad0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b410 tx=0x7fbf9400b8d0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 shutdown_connections 2023-10-26T05:58:19.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbfa415fc90 0x7fbfa41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbf880664a0 0x7fbf88068960 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfa415f610 0x7fbfa4159ad0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.936 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 --2- 172.21.15.71:0/567045244 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfa406d770 0x7fbfa4156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:19.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 >> 172.21.15.71:0/567045244 conn(0x7fbfa40ce270 msgr2=0x7fbfa40cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:19.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 shutdown_connections 2023-10-26T05:58:19.937 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:19.927+0000 7fbfac39e640 1 -- 172.21.15.71:0/567045244 wait complete. 2023-10-26T05:58:19.950 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-26T05:58:19.950 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-26T05:58:20.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2124096113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49f0005e20 msgr2=0x7f49f00062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2124096113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49f0005e20 0x7f49f00062a0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f49fc001d70 tx=0x7f49fc01cae0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a08e6b640 1 -- 172.21.15.71:0/2124096113 reap_dead start 2023-10-26T05:58:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2124096113 shutdown_connections 2023-10-26T05:58:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2124096113 >> 172.21.15.71:0/2124096113 conn(0x7f4a040aec60 msgr2=0x7f4a040af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2124096113 shutdown_connections 2023-10-26T05:58:20.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.071+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2124096113 wait complete. 2023-10-26T05:58:20.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 Processor -- start 2023-10-26T05:58:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- start start 2023-10-26T05:58:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a04142680 0x7f4a0415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a04146470 0x7f4a04150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 0x7f4a04156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04146c80 con 0x7f4a04151290 2023-10-26T05:58:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a041508c0 con 0x7f4a04142680 2023-10-26T05:58:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0414ff90 con 0x7f4a04146470 2023-10-26T05:58:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 0x7f4a04156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 0x7f4a04156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47698/0 (socket says 172.21.15.71:47698) 2023-10-26T05:58:20.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 -- 172.21.15.71:0/2798065481 learned_addr learned my addr 172.21.15.71:0/2798065481 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a03fff640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a04142680 0x7f4a0415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f49fb7fe640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a04146470 0x7f4a04150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 -- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a04146470 msgr2=0x7f4a04150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a04146470 0x7f4a04150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 -- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a04142680 msgr2=0x7f4a0415a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a04142680 0x7f4a0415a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 -- 172.21.15.71:0/2798065481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04150270 con 0x7f4a04151290 2023-10-26T05:58:20.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a08e6b640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 0x7f4a04156840 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0aa040 tx=0x7f49fc003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:20.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0a5d00 con 0x7f4a04151290 2023-10-26T05:58:20.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f49fc003e30 con 0x7f4a04151290 2023-10-26T05:58:20.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a04150550 con 0x7f4a04151290 2023-10-26T05:58:20.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0063a0 con 0x7f4a04151290 2023-10-26T05:58:20.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a04142a60 con 0x7f4a04151290 2023-10-26T05:58:20.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.075+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04066b10 con 0x7f4a04151290 2023-10-26T05:58:20.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.079+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f49fc0b5070 con 0x7f4a04151290 2023-10-26T05:58:20.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.079+0000 7f4a017fa640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49e4066470 0x7f49e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.079+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49fc0aa070 con 0x7f4a04151290 2023-10-26T05:58:20.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.079+0000 7f4a03fff640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49e4066470 0x7f49e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.079+0000 7f4a03fff640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49e4066470 0x7f49e4068930 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ab0 tx=0x7f49f40091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:20.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.083+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f49fc0eb320 con 0x7f4a04151290 2023-10-26T05:58:20.233 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.223+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4a040721e0 con 0x7f4a04151290 2023-10-26T05:58:20.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.623+0000 7f4a017fa640 1 -- 172.21.15.71:0/2798065481 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v185) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49fc0ee940 con 0x7f4a04151290 2023-10-26T05:58:20.633 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2023-10-26T05:58:20.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49e4066470 msgr2=0x7f49e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49e4066470 0x7f49e4068930 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009ab0 tx=0x7f49f40091e0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 msgr2=0x7f4a04156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 0x7f4a04156840 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0aa040 tx=0x7f49fc003ac0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 shutdown_connections 2023-10-26T05:58:20.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f49e4066470 0x7f49e4068930 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a04151290 0x7f4a04156840 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4a04146470 0x7f4a04150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 --2- 172.21.15.71:0/2798065481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a04142680 0x7f4a0415a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 >> 172.21.15.71:0/2798065481 conn(0x7f4a040aec60 msgr2=0x7f4a04069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:20.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 shutdown_connections 2023-10-26T05:58:20.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.627+0000 7f4a0a8ce640 1 -- 172.21.15.71:0/2798065481 wait complete. 2023-10-26T05:58:20.661 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-26T05:58:20.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 -- 172.21.15.71:0/2208548534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 msgr2=0x7f716c13eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.796 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 --2- 172.21.15.71:0/2208548534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c13eda0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f715c009840 tx=0x7f715c01c7d0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 -- 172.21.15.71:0/2208548534 shutdown_connections 2023-10-26T05:58:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 --2- 172.21.15.71:0/2208548534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c13eda0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 --2- 172.21.15.71:0/2208548534 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13e9c0 0x7f716c13bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 -- 172.21.15.71:0/2208548534 >> 172.21.15.71:0/2208548534 conn(0x7f716c0aec60 msgr2=0x7f716c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:20.797 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 -- 172.21.15.71:0/2208548534 shutdown_connections 2023-10-26T05:58:20.798 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 -- 172.21.15.71:0/2208548534 wait complete. 2023-10-26T05:58:20.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 Processor -- start 2023-10-26T05:58:20.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 -- start start 2023-10-26T05:58:20.800 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.787+0000 7f71732b9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f716c13e9c0 0x7f716c1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13f2e0 0x7f716c140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c156880 con 0x7f716c1422e0 2023-10-26T05:58:20.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c140ce0 con 0x7f716c13e9c0 2023-10-26T05:58:20.801 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c140fc0 con 0x7f716c13f2e0 2023-10-26T05:58:20.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47718/0 (socket says 172.21.15.71:47718) 2023-10-26T05:58:20.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 -- 172.21.15.71:0/2373458315 learned_addr learned my addr 172.21.15.71:0/2373458315 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:20.802 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7170854640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13f2e0 0x7f716c140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171055640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f716c13e9c0 0x7f716c1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 -- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13f2e0 msgr2=0x7f716c140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13f2e0 0x7f716c140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 -- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f716c13e9c0 msgr2=0x7f716c1399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:20.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f716c13e9c0 0x7f716c1399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:20.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 -- 172.21.15.71:0/2373458315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0094a0 con 0x7f716c1422e0 2023-10-26T05:58:20.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7170854640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13f2e0 0x7f716c140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:20.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171055640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f716c13e9c0 0x7f716c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:20.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7171856640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c155d20 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f716400cbc0 tx=0x7f716400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:20.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400aa50 con 0x7f716c1422e0 2023-10-26T05:58:20.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c1412a0 con 0x7f716c1422e0 2023-10-26T05:58:20.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f716400aca0 con 0x7f716c1422e0 2023-10-26T05:58:20.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164012680 con 0x7f716c1422e0 2023-10-26T05:58:20.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c141860 con 0x7f716c1422e0 2023-10-26T05:58:20.805 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c066b10 con 0x7f716c1422e0 2023-10-26T05:58:20.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f716401b400 con 0x7f716c1422e0 2023-10-26T05:58:20.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7161ffb640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7150066470 0x7f7150068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:20.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.791+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7164084a80 con 0x7f716c1422e0 2023-10-26T05:58:20.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.795+0000 7f7171055640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7150066470 0x7f7150068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:20.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.795+0000 7f7171055640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7150066470 0x7f7150068930 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7168005fd0 tx=0x7f7168005e60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:20.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.795+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7164052040 con 0x7f716c1422e0 2023-10-26T05:58:20.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:20.951+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f716c141a50 con 0x7f716c1422e0 2023-10-26T05:58:21.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.643+0000 7f7161ffb640 1 -- 172.21.15.71:0/2373458315 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v186) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7164055660 con 0x7f716c1422e0 2023-10-26T05:58:21.653 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2023-10-26T05:58:21.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7150066470 msgr2=0x7f7150068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:21.657 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7150066470 0x7f7150068930 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7168005fd0 tx=0x7f7168005e60 comp rx=0 tx=0).stop 2023-10-26T05:58:21.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 msgr2=0x7f716c155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:21.658 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c155d20 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f716400cbc0 tx=0x7f716400ccb0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 shutdown_connections 2023-10-26T05:58:21.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7150066470 0x7f7150068930 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.659 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f716c1422e0 0x7f716c155d20 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f716c13f2e0 0x7f716c140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 --2- 172.21.15.71:0/2373458315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f716c13e9c0 0x7f716c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.660 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 >> 172.21.15.71:0/2373458315 conn(0x7f716c0aec60 msgr2=0x7f716c0b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:21.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 shutdown_connections 2023-10-26T05:58:21.661 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.647+0000 7f71732b9640 1 -- 172.21.15.71:0/2373458315 wait complete. 2023-10-26T05:58:21.688 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-26T05:58:21.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 -- 172.21.15.71:0/635147676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0b0740 msgr2=0x7f894c06a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:21.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 --2- 172.21.15.71:0/635147676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0b0740 0x7f894c06a1b0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f8944009900 tx=0x7f894401c890 comp rx=0 tx=0).stop 2023-10-26T05:58:21.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 -- 172.21.15.71:0/635147676 shutdown_connections 2023-10-26T05:58:21.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 --2- 172.21.15.71:0/635147676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0b0740 0x7f894c06a1b0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 --2- 172.21.15.71:0/635147676 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f894c070750 0x7f894c0b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 -- 172.21.15.71:0/635147676 >> 172.21.15.71:0/635147676 conn(0x7f894c0af7d0 msgr2=0x7f894c0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:21.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 -- 172.21.15.71:0/635147676 shutdown_connections 2023-10-26T05:58:21.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 -- 172.21.15.71:0/635147676 wait complete. 2023-10-26T05:58:21.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 Processor -- start 2023-10-26T05:58:21.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.807+0000 7f8950ffc640 1 -- start start 2023-10-26T05:58:21.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f894c070750 0x7f894c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:21.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c0b0740 0x7f894c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:21.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 0x7f894c14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:21.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c14e580 con 0x7f894c0cd000 2023-10-26T05:58:21.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c152d40 con 0x7f894c0b0740 2023-10-26T05:58:21.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c14c600 con 0x7f894c070750 2023-10-26T05:58:21.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 0x7f894c14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:21.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 0x7f894c14b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46888/0 (socket says 172.21.15.71:46888) 2023-10-26T05:58:21.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 -- 172.21.15.71:0/2649009631 learned_addr learned my addr 172.21.15.71:0/2649009631 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:21.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894a59c640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f894c070750 0x7f894c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:21.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8949d9b640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c0b0740 0x7f894c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:21.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 -- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f894c070750 msgr2=0x7f894c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:21.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f894c070750 0x7f894c0b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 -- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c0b0740 msgr2=0x7f894c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:21.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c0b0740 0x7f894c0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:21.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 -- 172.21.15.71:0/2649009631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944009560 con 0x7f894c0cd000 2023-10-26T05:58:21.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8949d9b640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c0b0740 0x7f894c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:21.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894ad9d640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 0x7f894c14b890 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f89440b0040 tx=0x7f8944005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:21.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944005230 con 0x7f894c0cd000 2023-10-26T05:58:21.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8944005480 con 0x7f894c0cd000 2023-10-26T05:58:21.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c14c8e0 con 0x7f894c0cd000 2023-10-26T05:58:21.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944004930 con 0x7f894c0cd000 2023-10-26T05:58:21.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c14f130 con 0x7f894c0cd000 2023-10-26T05:58:21.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c14cd90 con 0x7f894c0cd000 2023-10-26T05:58:21.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f89440b4070 con 0x7f894c0cd000 2023-10-26T05:58:21.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f893affd640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f892c066470 0x7f892c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:21.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f894411e040 con 0x7f894c0cd000 2023-10-26T05:58:21.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.811+0000 7f894a59c640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f892c066470 0x7f892c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:21.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.815+0000 7f894a59c640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f892c066470 0x7f892c068930 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f893c000f70 tx=0x7f893c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:21.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:21.815+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f89440eb600 con 0x7f894c0cd000 2023-10-26T05:58:22.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.003+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f894c152eb0 con 0x7f894c0cd000 2023-10-26T05:58:22.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f893affd640 1 -- 172.21.15.71:0/2649009631 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v69) v1 ==== 174+0+0 (secure 0 0 0) 0x7f89440eec20 con 0x7f894c0cd000 2023-10-26T05:58:22.699 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 22 and data pool 23 2023-10-26T05:58:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f892c066470 msgr2=0x7f892c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f892c066470 0x7f892c068930 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f893c000f70 tx=0x7f893c009040 comp rx=0 tx=0).stop 2023-10-26T05:58:22.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 msgr2=0x7f894c14b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:22.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 0x7f894c14b890 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f89440b0040 tx=0x7f8944005e90 comp rx=0 tx=0).stop 2023-10-26T05:58:22.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 shutdown_connections 2023-10-26T05:58:22.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f892c066470 0x7f892c068930 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.691+0000 7f8950ffc640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f894c0cd000 0x7f894c14b890 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.695+0000 7f8950ffc640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f894c0b0740 0x7f894c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.695+0000 7f8950ffc640 1 --2- 172.21.15.71:0/2649009631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f894c070750 0x7f894c0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.695+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 >> 172.21.15.71:0/2649009631 conn(0x7f894c0af7d0 msgr2=0x7f894c15a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:22.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.695+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 shutdown_connections 2023-10-26T05:58:22.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.695+0000 7f8950ffc640 1 -- 172.21.15.71:0/2649009631 wait complete. 2023-10-26T05:58:22.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.839+0000 7f7fddb35640 1 -- 172.21.15.71:0/3118625861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8070fe0 msgr2=0x7f7fd8071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.839+0000 7f7fddb35640 1 --2- 172.21.15.71:0/3118625861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8070fe0 0x7f7fd8071470 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009870 tx=0x7f7fc001c790 comp rx=0 tx=0).stop 2023-10-26T05:58:22.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.839+0000 7f7fddb35640 1 -- 172.21.15.71:0/3118625861 shutdown_connections 2023-10-26T05:58:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.839+0000 7f7fddb35640 1 --2- 172.21.15.71:0/3118625861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8070fe0 0x7f7fd8071470 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.839+0000 7f7fddb35640 1 -- 172.21.15.71:0/3118625861 >> 172.21.15.71:0/3118625861 conn(0x7f7fd8069770 msgr2=0x7f7fd8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.839+0000 7f7fddb35640 1 -- 172.21.15.71:0/3118625861 shutdown_connections 2023-10-26T05:58:22.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- 172.21.15.71:0/3118625861 wait complete. 2023-10-26T05:58:22.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 Processor -- start 2023-10-26T05:58:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- start start 2023-10-26T05:58:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd8070fe0 0x7f7fd815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7fd80b1360 0x7f7fd8154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:22.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 0x7f7fd8138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd813c970 con 0x7f7fd8140cc0 2023-10-26T05:58:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8155100 con 0x7f7fd8070fe0 2023-10-26T05:58:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8159380 con 0x7f7fd80b1360 2023-10-26T05:58:22.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 0x7f7fd8138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 0x7f7fd8138550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46896/0 (socket says 172.21.15.71:46896) 2023-10-26T05:58:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 -- 172.21.15.71:0/1064312414 learned_addr learned my addr 172.21.15.71:0/1064312414 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd77fe640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd8070fe0 0x7f7fd815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd6ffd640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7fd80b1360 0x7f7fd8154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:22.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 -- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7fd80b1360 msgr2=0x7f7fd8154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7fd80b1360 0x7f7fd8154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 -- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd8070fe0 msgr2=0x7f7fd815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd8070fe0 0x7f7fd815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 -- 172.21.15.71:0/1064312414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00094a0 con 0x7f7fd8140cc0 2023-10-26T05:58:22.857 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd77fe640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd8070fe0 0x7f7fd815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:22.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fd7fff640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 0x7f7fd8138550 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c3f0 tx=0x7f7fc800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:22.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8011790 con 0x7f7fd8140cc0 2023-10-26T05:58:22.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7fc80119e0 con 0x7f7fd8140cc0 2023-10-26T05:58:22.858 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd8159660 con 0x7f7fd8140cc0 2023-10-26T05:58:22.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8011df0 con 0x7f7fd8140cc0 2023-10-26T05:58:22.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd8159c20 con 0x7f7fd8140cc0 2023-10-26T05:58:22.859 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.843+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd8066b10 con 0x7f7fd8140cc0 2023-10-26T05:58:22.861 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.847+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7fc801c460 con 0x7f7fd8140cc0 2023-10-26T05:58:22.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.847+0000 7f7fcffff640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fb8066470 0x7f7fb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:22.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.847+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fc8084970 con 0x7f7fd8140cc0 2023-10-26T05:58:22.862 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.847+0000 7f7fd77fe640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fb8066470 0x7f7fb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:22.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.847+0000 7f7fd77fe640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fb8066470 0x7f7fb8068930 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0007fd0 tx=0x7f7fd0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:22.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:22.851+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7fc8051f60 con 0x7f7fd8140cc0 2023-10-26T05:58:23.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.003+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fd8159e10 con 0x7f7fd8140cc0 2023-10-26T05:58:23.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.007+0000 7f7fcffff640 1 -- 172.21.15.71:0/1064312414 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+16501 (secure 0 0 0) 0x7f7fc8003810 con 0x7f7fd8140cc0 2023-10-26T05:58:23.017 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:23.017 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":187,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:22.009957+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fb8066470 msgr2=0x7f7fb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fb8066470 0x7f7fb8068930 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0007fd0 tx=0x7f7fd0009040 comp rx=0 tx=0).stop 2023-10-26T05:58:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 msgr2=0x7f7fd8138550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 0x7f7fd8138550 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c3f0 tx=0x7f7fc800c8b0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 shutdown_connections 2023-10-26T05:58:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7fb8066470 0x7f7fb8068930 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd8140cc0 0x7f7fd8138550 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7fd80b1360 0x7f7fd8154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 --2- 172.21.15.71:0/1064312414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fd8070fe0 0x7f7fd815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 >> 172.21.15.71:0/1064312414 conn(0x7f7fd8069770 msgr2=0x7f7fd80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:23.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 shutdown_connections 2023-10-26T05:58:23.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.011+0000 7f7fddb35640 1 -- 172.21.15.71:0/1064312414 wait complete. 2023-10-26T05:58:23.036 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:23.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 -- 172.21.15.71:0/877289421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37480cb1e0 msgr2=0x7f3748159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 --2- 172.21.15.71:0/877289421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37480cb1e0 0x7f3748159810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3740009bc0 tx=0x7f374001cb80 comp rx=0 tx=0).stop 2023-10-26T05:58:23.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 -- 172.21.15.71:0/877289421 shutdown_connections 2023-10-26T05:58:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 --2- 172.21.15.71:0/877289421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37480cb1e0 0x7f3748159810 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 --2- 172.21.15.71:0/877289421 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f374813c2f0 0x7f37480b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 -- 172.21.15.71:0/877289421 >> 172.21.15.71:0/877289421 conn(0x7f3748069770 msgr2=0x7f3748069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 -- 172.21.15.71:0/877289421 shutdown_connections 2023-10-26T05:58:23.157 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 -- 172.21.15.71:0/877289421 wait complete. 2023-10-26T05:58:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 Processor -- start 2023-10-26T05:58:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.147+0000 7f374da98640 1 -- start start 2023-10-26T05:58:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37480b2890 0x7f37480ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 0x7f374813e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.160 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f374813c2f0 0x7f37480acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374815bdd0 con 0x7f37480cb1e0 2023-10-26T05:58:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374813ef30 con 0x7f374813c2f0 2023-10-26T05:58:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374815b5c0 con 0x7f37480b2890 2023-10-26T05:58:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 0x7f374813e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.161 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 0x7f374813e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46904/0 (socket says 172.21.15.71:46904) 2023-10-26T05:58:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 -- 172.21.15.71:0/3988722881 learned_addr learned my addr 172.21.15.71:0/3988722881 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f37477fe640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37480b2890 0x7f37480ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3747fff640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f374813c2f0 0x7f37480acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 -- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37480b2890 msgr2=0x7f37480ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37480b2890 0x7f37480ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 -- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f374813c2f0 msgr2=0x7f37480acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f374813c2f0 0x7f37480acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 -- 172.21.15.71:0/3988722881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3740009820 con 0x7f37480cb1e0 2023-10-26T05:58:23.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3747fff640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f374813c2f0 0x7f37480acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f3746ffd640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 0x7f374813e9f0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f373c00c530 tx=0x7f373c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c005220 con 0x7f37480cb1e0 2023-10-26T05:58:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f373c005470 con 0x7f37480cb1e0 2023-10-26T05:58:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374815b810 con 0x7f37480cb1e0 2023-10-26T05:58:23.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c007290 con 0x7f37480cb1e0 2023-10-26T05:58:23.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3748137430 con 0x7f37480cb1e0 2023-10-26T05:58:23.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3748066b10 con 0x7f37480cb1e0 2023-10-26T05:58:23.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f373c0073f0 con 0x7f37480cb1e0 2023-10-26T05:58:23.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f372ffff640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3734066470 0x7f3734068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f373c0838e0 con 0x7f37480cb1e0 2023-10-26T05:58:23.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.151+0000 7f37477fe640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3734066470 0x7f3734068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.155+0000 7f37477fe640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3734066470 0x7f3734068930 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f37480b1400 tx=0x7f3738009260 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:23.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.159+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f373c050d70 con 0x7f37480cb1e0 2023-10-26T05:58:23.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.311+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37480721e0 con 0x7f37480cb1e0 2023-10-26T05:58:23.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f372ffff640 1 -- 172.21.15.71:0/3988722881 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+16488 (secure 0 0 0) 0x7f373c054390 con 0x7f37480cb1e0 2023-10-26T05:58:23.322 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:23.323 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":188,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:23.304802+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:23.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3734066470 msgr2=0x7f3734068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3734066470 0x7f3734068930 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f37480b1400 tx=0x7f3738009260 comp rx=0 tx=0).stop 2023-10-26T05:58:23.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 msgr2=0x7f374813e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 0x7f374813e9f0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f373c00c530 tx=0x7f373c00c9f0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 shutdown_connections 2023-10-26T05:58:23.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f374813c2f0 0x7f37480acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3734066470 0x7f3734068930 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37480cb1e0 0x7f374813e9f0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 --2- 172.21.15.71:0/3988722881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f37480b2890 0x7f37480ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 >> 172.21.15.71:0/3988722881 conn(0x7f3748069770 msgr2=0x7f37480ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 shutdown_connections 2023-10-26T05:58:23.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.315+0000 7f374da98640 1 -- 172.21.15.71:0/3988722881 wait complete. 2023-10-26T05:58:23.340 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-26T05:58:23.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/32956728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 msgr2=0x7f8e280cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 --2- 172.21.15.71:0/32956728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e280cb820 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009900 tx=0x7f8e1c01c800 comp rx=0 tx=0).stop 2023-10-26T05:58:23.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/32956728 shutdown_connections 2023-10-26T05:58:23.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 --2- 172.21.15.71:0/32956728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e280cb820 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 --2- 172.21.15.71:0/32956728 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e2813c220 0x7f8e2815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/32956728 >> 172.21.15.71:0/32956728 conn(0x7f8e280aec60 msgr2=0x7f8e280af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:23.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/32956728 shutdown_connections 2023-10-26T05:58:23.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/32956728 wait complete. 2023-10-26T05:58:23.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 Processor -- start 2023-10-26T05:58:23.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- start start 2023-10-26T05:58:23.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e280cbd60 0x7f8e28148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e2813c220 0x7f8e280cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e28144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e28149170 con 0x7f8e2815d220 2023-10-26T05:58:23.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2813ac20 con 0x7f8e280cbd60 2023-10-26T05:58:23.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280adae0 con 0x7f8e2813c220 2023-10-26T05:58:23.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e28144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e28144840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46920/0 (socket says 172.21.15.71:46920) 2023-10-26T05:58:23.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 -- 172.21.15.71:0/2793677511 learned_addr learned my addr 172.21.15.71:0/2793677511 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:23.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 -- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e2813c220 msgr2=0x7f8e280cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e2813c220 0x7f8e280cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 -- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e280cbd60 msgr2=0x7f8e28148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e280cbd60 0x7f8e28148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 -- 172.21.15.71:0/2793677511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009560 con 0x7f8e2815d220 2023-10-26T05:58:23.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e2ea8e640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e28144840 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000c710 tx=0x7f8e2000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:23.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20006d60 con 0x7f8e2815d220 2023-10-26T05:58:23.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8e2000ce20 con 0x7f8e2815d220 2023-10-26T05:58:23.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e28148c80 con 0x7f8e2815d220 2023-10-26T05:58:23.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20003be0 con 0x7f8e2815d220 2023-10-26T05:58:23.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.447+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e28149380 con 0x7f8e2815d220 2023-10-26T05:58:23.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.451+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e28066b10 con 0x7f8e2815d220 2023-10-26T05:58:23.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.451+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8e20003d40 con 0x7f8e2815d220 2023-10-26T05:58:23.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.451+0000 7f8e1affd640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e0c066470 0x7f8e0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.451+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e20083c40 con 0x7f8e2815d220 2023-10-26T05:58:23.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.451+0000 7f8e2e28d640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e0c066470 0x7f8e0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.451+0000 7f8e2e28d640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e0c066470 0x7f8e0c068930 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8e28135fc0 tx=0x7f8e24006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:23.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.455+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8e20051150 con 0x7f8e2815d220 2023-10-26T05:58:23.655 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.647+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8e280721e0 con 0x7f8e2815d220 2023-10-26T05:58:23.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.691+0000 7f8e1affd640 1 -- 172.21.15.71:0/2793677511 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8e20054770 con 0x7f8e2815d220 2023-10-26T05:58:23.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e0c066470 msgr2=0x7f8e0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e0c066470 0x7f8e0c068930 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8e28135fc0 tx=0x7f8e24006ca0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 msgr2=0x7f8e28144840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e28144840 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f8e2000c710 tx=0x7f8e2000cbd0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 shutdown_connections 2023-10-26T05:58:23.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8e0c066470 0x7f8e0c068930 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e2815d220 0x7f8e28144840 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8e2813c220 0x7f8e280cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 --2- 172.21.15.71:0/2793677511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e280cbd60 0x7f8e28148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 >> 172.21.15.71:0/2793677511 conn(0x7f8e280aec60 msgr2=0x7f8e280adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:23.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 shutdown_connections 2023-10-26T05:58:23.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.695+0000 7f8e304f1640 1 -- 172.21.15.71:0/2793677511 wait complete. 2023-10-26T05:58:23.722 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-26T05:58:23.723 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-26T05:58:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 -- 172.21.15.71:0/2141806633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c0b1520 msgr2=0x7f7c2c0b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.842 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 --2- 172.21.15.71:0/2141806633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c0b1520 0x7f7c2c0b1900 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009900 tx=0x7f7c1c01c850 comp rx=0 tx=0).stop 2023-10-26T05:58:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 -- 172.21.15.71:0/2141806633 shutdown_connections 2023-10-26T05:58:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 --2- 172.21.15.71:0/2141806633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c0b1520 0x7f7c2c0b1900 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 -- 172.21.15.71:0/2141806633 >> 172.21.15.71:0/2141806633 conn(0x7f7c2c0ca230 msgr2=0x7f7c2c0ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 -- 172.21.15.71:0/2141806633 shutdown_connections 2023-10-26T05:58:23.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.831+0000 7f7c319b3640 1 -- 172.21.15.71:0/2141806633 wait complete. 2023-10-26T05:58:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 Processor -- start 2023-10-26T05:58:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- start start 2023-10-26T05:58:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2c070e10 0x7f7c2c1440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2c0b1520 0x7f7c2c146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 0x7f7c2c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c147160 con 0x7f7c2c158b80 2023-10-26T05:58:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c144800 con 0x7f7c2c070e10 2023-10-26T05:58:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c144ae0 con 0x7f7c2c0b1520 2023-10-26T05:58:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 0x7f7c2c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 0x7f7c2c1458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46942/0 (socket says 172.21.15.71:46942) 2023-10-26T05:58:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 -- 172.21.15.71:0/3213094329 learned_addr learned my addr 172.21.15.71:0/3213094329 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 -- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2c0b1520 msgr2=0x7f7c2c146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2c0b1520 0x7f7c2c146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2affd640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2c070e10 0x7f7c2c1440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 -- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2c070e10 msgr2=0x7f7c2c1440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2c070e10 0x7f7c2c1440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 -- 172.21.15.71:0/3213094329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c009560 con 0x7f7c2c158b80 2023-10-26T05:58:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c2b7fe640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 0x7f7c2c1458a0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c730 tx=0x7f7c2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24003b80 con 0x7f7c2c158b80 2023-10-26T05:58:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7c24003ce0 con 0x7f7c2c158b80 2023-10-26T05:58:23.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c144dc0 con 0x7f7c2c158b80 2023-10-26T05:58:23.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400aa50 con 0x7f7c2c158b80 2023-10-26T05:58:23.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c145380 con 0x7f7c2c158b80 2023-10-26T05:58:23.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.835+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c066d00 con 0x7f7c2c158b80 2023-10-26T05:58:23.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.839+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c2400abb0 con 0x7f7c2c158b80 2023-10-26T05:58:23.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.839+0000 7f7c13fff640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c18066470 0x7f7c18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:23.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.839+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c24083dd0 con 0x7f7c2c158b80 2023-10-26T05:58:23.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.839+0000 7f7c2affd640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c18066470 0x7f7c18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:23.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.839+0000 7f7c2affd640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c18066470 0x7f7c18068930 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0af000 tx=0x7f7c1c01b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:23.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:23.843+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7c24051250 con 0x7f7c2c158b80 2023-10-26T05:58:24.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.031+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7c2c145540 con 0x7f7c2c158b80 2023-10-26T05:58:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.719+0000 7f7c13fff640 1 -- 172.21.15.71:0/3213094329 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7c2400ae60 con 0x7f7c2c158b80 2023-10-26T05:58:24.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c18066470 msgr2=0x7f7c18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:24.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c18066470 0x7f7c18068930 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0af000 tx=0x7f7c1c01b090 comp rx=0 tx=0).stop 2023-10-26T05:58:24.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 msgr2=0x7f7c2c1458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:24.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 0x7f7c2c1458a0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400c730 tx=0x7f7c2400cbf0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 shutdown_connections 2023-10-26T05:58:24.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7c18066470 0x7f7c18068930 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c2c158b80 0x7f7c2c1458a0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c2c0b1520 0x7f7c2c146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 --2- 172.21.15.71:0/3213094329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2c070e10 0x7f7c2c1440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 >> 172.21.15.71:0/3213094329 conn(0x7f7c2c0ca230 msgr2=0x7f7c2c069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:24.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 shutdown_connections 2023-10-26T05:58:24.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.723+0000 7f7c319b3640 1 -- 172.21.15.71:0/3213094329 wait complete. 2023-10-26T05:58:24.751 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-26T05:58:24.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/2894213933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 msgr2=0x7f9f100047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/2894213933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f100047d0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0016b0 tx=0x7f9f1c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2e3eb640 1 -- 172.21.15.71:0/2894213933 reap_dead start 2023-10-26T05:58:24.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/2894213933 shutdown_connections 2023-10-26T05:58:24.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/2894213933 >> 172.21.15.71:0/2894213933 conn(0x7f9f28069770 msgr2=0x7f9f28069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:24.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/2894213933 shutdown_connections 2023-10-26T05:58:24.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.871+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/2894213933 wait complete. 2023-10-26T05:58:24.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 Processor -- start 2023-10-26T05:58:24.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- start start 2023-10-26T05:58:24.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f2813c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:24.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f2813e920 0x7f9f28137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:24.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28154cc0 0x7f9f28155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:24.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2813fdf0 con 0x7f9f10003f30 2023-10-26T05:58:24.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f28155680 con 0x7f9f28154cc0 2023-10-26T05:58:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2813f170 con 0x7f9f2813e920 2023-10-26T05:58:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f2813c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f2813c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46956/0 (socket says 172.21.15.71:46956) 2023-10-26T05:58:24.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 -- 172.21.15.71:0/3939792119 learned_addr learned my addr 172.21.15.71:0/3939792119 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2e3eb640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28154cc0 0x7f9f28155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2d3e9640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f2813e920 0x7f9f28137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 -- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f2813e920 msgr2=0x7f9f28137ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f2813e920 0x7f9f28137ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 -- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28154cc0 msgr2=0x7f9f28155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28154cc0 0x7f9f28155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 -- 172.21.15.71:0/3939792119 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2813f450 con 0x7f9f10003f30 2023-10-26T05:58:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2d3e9640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f2813e920 0x7f9f28137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:24.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2dbea640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f2813c4c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800e620 tx=0x7f9f1800eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18004220 con 0x7f9f10003f30 2023-10-26T05:58:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f18004490 con 0x7f9f10003f30 2023-10-26T05:58:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2813f730 con 0x7f9f10003f30 2023-10-26T05:58:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800c9a0 con 0x7f9f10003f30 2023-10-26T05:58:24.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2813eda0 con 0x7f9f10003f30 2023-10-26T05:58:24.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f28066b10 con 0x7f9f10003f30 2023-10-26T05:58:24.891 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9f1800cb00 con 0x7f9f10003f30 2023-10-26T05:58:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f267fc640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f140664a0 0x7f9f14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.875+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f18084350 con 0x7f9f10003f30 2023-10-26T05:58:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.879+0000 7f9f2d3e9640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f140664a0 0x7f9f14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:24.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.879+0000 7f9f2d3e9640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f140664a0 0x7f9f14068960 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0ab000 tx=0x7f9f10005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:24.893 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:24.883+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9f180517b0 con 0x7f9f10003f30 2023-10-26T05:58:25.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f280721e0 con 0x7f9f10003f30 2023-10-26T05:58:25.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f267fc640 1 -- 172.21.15.71:0/3939792119 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f9f18054dd0 con 0x7f9f10003f30 2023-10-26T05:58:25.079 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:25.079 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:58:25.080 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:58:25.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f140664a0 msgr2=0x7f9f14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f140664a0 0x7f9f14068960 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0ab000 tx=0x7f9f10005800 comp rx=0 tx=0).stop 2023-10-26T05:58:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 msgr2=0x7f9f2813c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f2813c4c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800e620 tx=0x7f9f1800eae0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 shutdown_connections 2023-10-26T05:58:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f28154cc0 0x7f9f28155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f2813e920 0x7f9f28137ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f140664a0 0x7f9f14068960 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 --2- 172.21.15.71:0/3939792119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f10003f30 0x7f9f2813c4c0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 >> 172.21.15.71:0/3939792119 conn(0x7f9f28069770 msgr2=0x7f9f28070860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 shutdown_connections 2023-10-26T05:58:25.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.071+0000 7f9f2fe4e640 1 -- 172.21.15.71:0/3939792119 wait complete. 2023-10-26T05:58:25.097 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:58:25.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- 172.21.15.71:0/1406164171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c070fe0 msgr2=0x7f266c071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 --2- 172.21.15.71:0/1406164171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c070fe0 0x7f266c071440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2660009900 tx=0x7f266001c890 comp rx=0 tx=0).stop 2023-10-26T05:58:25.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- 172.21.15.71:0/1406164171 shutdown_connections 2023-10-26T05:58:25.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 --2- 172.21.15.71:0/1406164171 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c070fe0 0x7f266c071440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 --2- 172.21.15.71:0/1406164171 >> v2:172.21.15.86:3300/0 conn(0x7f266c140cc0 0x7f266c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- 172.21.15.71:0/1406164171 >> 172.21.15.71:0/1406164171 conn(0x7f266c069770 msgr2=0x7f266c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:25.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- 172.21.15.71:0/1406164171 shutdown_connections 2023-10-26T05:58:25.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- 172.21.15.71:0/1406164171 wait complete. 2023-10-26T05:58:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 Processor -- start 2023-10-26T05:58:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- start start 2023-10-26T05:58:25.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 0x7f266c156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f266c0b0040 0x7f266c0ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c140cc0 0x7f266c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.217 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0ade70 con 0x7f266c070fe0 2023-10-26T05:58:25.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0ad260 con 0x7f266c0b0040 2023-10-26T05:58:25.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f267120f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0ad540 con 0x7f266c140cc0 2023-10-26T05:58:25.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f266ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 0x7f266c156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.218 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f266ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 0x7f266c156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46958/0 (socket says 172.21.15.71:46958) 2023-10-26T05:58:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.203+0000 7f266ad9d640 1 -- 172.21.15.71:0/1427427261 learned_addr learned my addr 172.21.15.71:0/1427427261 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266a59c640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f266c0b0040 0x7f266c0ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266b59e640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c140cc0 0x7f266c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266ad9d640 1 -- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c140cc0 msgr2=0x7f266c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266ad9d640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c140cc0 0x7f266c1576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266ad9d640 1 -- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f266c0b0040 msgr2=0x7f266c0ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266ad9d640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f266c0b0040 0x7f266c0ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266ad9d640 1 -- 172.21.15.71:0/1427427261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2660009560 con 0x7f266c070fe0 2023-10-26T05:58:25.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266b59e640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c140cc0 0x7f266c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:25.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266ad9d640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 0x7f266c156d90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e7a0 tx=0x7f265c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:25.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c009590 con 0x7f266c070fe0 2023-10-26T05:58:25.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f265c00cdb0 con 0x7f266c070fe0 2023-10-26T05:58:25.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c0ad820 con 0x7f266c070fe0 2023-10-26T05:58:25.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c010690 con 0x7f266c070fe0 2023-10-26T05:58:25.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c0ae840 con 0x7f266c070fe0 2023-10-26T05:58:25.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c066b10 con 0x7f266c070fe0 2023-10-26T05:58:25.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f265c0107f0 con 0x7f266c070fe0 2023-10-26T05:58:25.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f26537fe640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2658066470 0x7f2658068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f265c014070 con 0x7f266c070fe0 2023-10-26T05:58:25.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266a59c640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2658066470 0x7f2658068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.207+0000 7f266a59c640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2658066470 0x7f2658068930 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f26600af000 tx=0x7f266001b0d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:25.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.211+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f265c050090 con 0x7f266c070fe0 2023-10-26T05:58:25.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.351+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f266c0aea30 con 0x7f266c070fe0 2023-10-26T05:58:25.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.355+0000 7f26537fe640 1 -- 172.21.15.71:0/1427427261 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+16488 (secure 0 0 0) 0x7f265c051d70 con 0x7f266c070fe0 2023-10-26T05:58:25.365 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:25.365 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":188,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:23.304802+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:58:25.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2658066470 msgr2=0x7f2658068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2658066470 0x7f2658068930 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f26600af000 tx=0x7f266001b0d0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 msgr2=0x7f266c156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 0x7f266c156d90 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f265c00e7a0 tx=0x7f265c00ec60 comp rx=0 tx=0).stop 2023-10-26T05:58:25.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 shutdown_connections 2023-10-26T05:58:25.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f266c140cc0 0x7f266c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f266c0b0040 0x7f266c0ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2658066470 0x7f2658068930 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 --2- 172.21.15.71:0/1427427261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f266c070fe0 0x7f266c156d90 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 >> 172.21.15.71:0/1427427261 conn(0x7f266c069770 msgr2=0x7f266c13e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:25.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 shutdown_connections 2023-10-26T05:58:25.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.359+0000 7f267120f640 1 -- 172.21.15.71:0/1427427261 wait complete. 2023-10-26T05:58:25.382 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-26T05:58:25.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:58:25.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/3737732525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc0140cc0 msgr2=0x7f7dc015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 --2- 172.21.15.71:0/3737732525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc0140cc0 0x7f7dc015c760 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009840 tx=0x7f7db001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/3737732525 shutdown_connections 2023-10-26T05:58:25.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 --2- 172.21.15.71:0/3737732525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc0070fe0 0x7f7dc0071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 --2- 172.21.15.71:0/3737732525 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc0140cc0 0x7f7dc015c760 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/3737732525 >> 172.21.15.71:0/3737732525 conn(0x7f7dc0069770 msgr2=0x7f7dc0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:25.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/3737732525 shutdown_connections 2023-10-26T05:58:25.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/3737732525 wait complete. 2023-10-26T05:58:25.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 Processor -- start 2023-10-26T05:58:25.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- start start 2023-10-26T05:58:25.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0070fe0 0x7f7dc006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 0x7f7dc0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc0140cc0 0x7f7dc0137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc015b150 con 0x7f7dc00b0040 2023-10-26T05:58:25.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc015b430 con 0x7f7dc0070fe0 2023-10-26T05:58:25.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc015b710 con 0x7f7dc0140cc0 2023-10-26T05:58:25.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 0x7f7dc0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 0x7f7dc0159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46978/0 (socket says 172.21.15.71:46978) 2023-10-26T05:58:25.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 -- 172.21.15.71:0/1592992491 learned_addr learned my addr 172.21.15.71:0/1592992491 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:25.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbeffd640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0070fe0 0x7f7dc006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 -- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc0140cc0 msgr2=0x7f7dc0137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc0140cc0 0x7f7dc0137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 -- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0070fe0 msgr2=0x7f7dc006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0070fe0 0x7f7dc006a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 -- 172.21.15.71:0/1592992491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00094a0 con 0x7f7dc00b0040 2023-10-26T05:58:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dbe7fc640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 0x7f7dc0159040 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f7db400c970 tx=0x7f7db400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4007bb0 con 0x7f7dc00b0040 2023-10-26T05:58:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f7db4007e00 con 0x7f7dc00b0040 2023-10-26T05:58:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc01412d0 con 0x7f7dc00b0040 2023-10-26T05:58:25.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4005280 con 0x7f7dc00b0040 2023-10-26T05:58:25.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.499+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc015bcd0 con 0x7f7dc00b0040 2023-10-26T05:58:25.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.503+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc0066b10 con 0x7f7dc00b0040 2023-10-26T05:58:25.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.503+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7db40053e0 con 0x7f7dc00b0040 2023-10-26T05:58:25.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.503+0000 7f7da77fe640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7dac0664a0 0x7f7dac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.503+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7db4084450 con 0x7f7dc00b0040 2023-10-26T05:58:25.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.503+0000 7f7dbeffd640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7dac0664a0 0x7f7dac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.503+0000 7f7dbeffd640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7dac0664a0 0x7f7dac068960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7db00af000 tx=0x7f7db001b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:25.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.507+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f7db40518d0 con 0x7f7dc00b0040 2023-10-26T05:58:25.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.659+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7dc015b5a0 con 0x7f7dac0664a0 2023-10-26T05:58:25.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.679+0000 7f7da77fe640 1 -- 172.21.15.71:0/1592992491 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283884 (secure 0 0 0) 0x7f7dc015b5a0 con 0x7f7dac0664a0 2023-10-26T05:58:25.692 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:25.694 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:58:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7dac0664a0 msgr2=0x7f7dac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7dac0664a0 0x7f7dac068960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7db00af000 tx=0x7f7db001b010 comp rx=0 tx=0).stop 2023-10-26T05:58:25.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 msgr2=0x7f7dc0159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 0x7f7dc0159040 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f7db400c970 tx=0x7f7db400ce30 comp rx=0 tx=0).stop 2023-10-26T05:58:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 shutdown_connections 2023-10-26T05:58:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7dc0140cc0 0x7f7dc0137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7dac0664a0 0x7f7dac068960 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dc00b0040 0x7f7dc0159040 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 --2- 172.21.15.71:0/1592992491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7dc0070fe0 0x7f7dc006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 >> 172.21.15.71:0/1592992491 conn(0x7f7dc0069770 msgr2=0x7f7dc01574a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:25.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 shutdown_connections 2023-10-26T05:58:25.697 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.683+0000 7f7dc532a640 1 -- 172.21.15.71:0/1592992491 wait complete. 2023-10-26T05:58:25.710 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2023-10-26T05:58:25.635709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9414,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553085,"statfs":{"total":1600000000,"available":1590359464,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.997241"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.340628+0000","last_change":"2023-10-26T05:58:22.340449+0000","last_active":"2023-10-26T05:58:22.340628+0000","last_peered":"2023-10-26T05:58:22.340628+0000","last_clean":"2023-10-26T05:58:22.340628+0000","last_became_active":"2023-10-26T05:58:22.339601+0000","last_became_peered":"2023-10-26T05:58:22.339601+0000","last_unstale":"2023-10-26T05:58:22.340628+0000","last_undegraded":"2023-10-26T05:58:22.340628+0000","last_fullsized":"2023-10-26T05:58:22.340628+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:48:29.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.356232+0000","last_change":"2023-10-26T05:58:22.356083+0000","last_active":"2023-10-26T05:58:22.356232+0000","last_peered":"2023-10-26T05:58:22.356232+0000","last_clean":"2023-10-26T05:58:22.356232+0000","last_became_active":"2023-10-26T05:58:22.355404+0000","last_became_peered":"2023-10-26T05:58:22.355404+0000","last_unstale":"2023-10-26T05:58:22.356232+0000","last_undegraded":"2023-10-26T05:58:22.356232+0000","last_fullsized":"2023-10-26T05:58:22.356232+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:00.413413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.316128+0000","last_change":"2023-10-26T05:58:22.344429+0000","last_active":"2023-10-26T05:58:24.316128+0000","last_peered":"2023-10-26T05:58:24.316128+0000","last_clean":"2023-10-26T05:58:24.316128+0000","last_became_active":"2023-10-26T05:58:22.343512+0000","last_became_peered":"2023-10-26T05:58:22.343512+0000","last_unstale":"2023-10-26T05:58:24.316128+0000","last_undegraded":"2023-10-26T05:58:24.316128+0000","last_fullsized":"2023-10-26T05:58:24.316128+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:08:56.697268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311108+0000","last_change":"2023-10-26T05:58:22.327278+0000","last_active":"2023-10-26T05:58:23.311108+0000","last_peered":"2023-10-26T05:58:23.311108+0000","last_clean":"2023-10-26T05:58:23.311108+0000","last_became_active":"2023-10-26T05:58:22.325660+0000","last_became_peered":"2023-10-26T05:58:22.325660+0000","last_unstale":"2023-10-26T05:58:23.311108+0000","last_undegraded":"2023-10-26T05:58:23.311108+0000","last_fullsized":"2023-10-26T05:58:23.311108+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:34.729841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354660+0000","last_change":"2023-10-26T05:58:22.349160+0000","last_active":"2023-10-26T05:58:22.354660+0000","last_peered":"2023-10-26T05:58:22.354660+0000","last_clean":"2023-10-26T05:58:22.354660+0000","last_became_active":"2023-10-26T05:58:22.347926+0000","last_became_peered":"2023-10-26T05:58:22.347926+0000","last_unstale":"2023-10-26T05:58:22.354660+0000","last_undegraded":"2023-10-26T05:58:22.354660+0000","last_fullsized":"2023-10-26T05:58:22.354660+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:27.056484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339648+0000","last_change":"2023-10-26T05:58:22.336063+0000","last_active":"2023-10-26T05:58:22.339648+0000","last_peered":"2023-10-26T05:58:22.339648+0000","last_clean":"2023-10-26T05:58:22.339648+0000","last_became_active":"2023-10-26T05:58:22.334975+0000","last_became_peered":"2023-10-26T05:58:22.334975+0000","last_unstale":"2023-10-26T05:58:22.339648+0000","last_undegraded":"2023-10-26T05:58:22.339648+0000","last_fullsized":"2023-10-26T05:58:22.339648+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:05.798695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.335035+0000","last_change":"2023-10-26T05:58:22.334855+0000","last_active":"2023-10-26T05:58:22.335035+0000","last_peered":"2023-10-26T05:58:22.335035+0000","last_clean":"2023-10-26T05:58:22.335035+0000","last_became_active":"2023-10-26T05:58:22.334219+0000","last_became_peered":"2023-10-26T05:58:22.334219+0000","last_unstale":"2023-10-26T05:58:22.335035+0000","last_undegraded":"2023-10-26T05:58:22.335035+0000","last_fullsized":"2023-10-26T05:58:22.335035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:39.737968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346519+0000","last_change":"2023-10-26T05:58:22.345771+0000","last_active":"2023-10-26T05:58:22.346519+0000","last_peered":"2023-10-26T05:58:22.346519+0000","last_clean":"2023-10-26T05:58:22.346519+0000","last_became_active":"2023-10-26T05:58:22.345135+0000","last_became_peered":"2023-10-26T05:58:22.345135+0000","last_unstale":"2023-10-26T05:58:22.346519+0000","last_undegraded":"2023-10-26T05:58:22.346519+0000","last_fullsized":"2023-10-26T05:58:22.346519+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:42:01.516098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354709+0000","last_change":"2023-10-26T05:58:22.349676+0000","last_active":"2023-10-26T05:58:22.354709+0000","last_peered":"2023-10-26T05:58:22.354709+0000","last_clean":"2023-10-26T05:58:22.354709+0000","last_became_active":"2023-10-26T05:58:22.348049+0000","last_became_peered":"2023-10-26T05:58:22.348049+0000","last_unstale":"2023-10-26T05:58:22.354709+0000","last_undegraded":"2023-10-26T05:58:22.354709+0000","last_fullsized":"2023-10-26T05:58:22.354709+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:42:23.374903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354609+0000","last_change":"2023-10-26T05:58:22.348701+0000","last_active":"2023-10-26T05:58:22.354609+0000","last_peered":"2023-10-26T05:58:22.354609+0000","last_clean":"2023-10-26T05:58:22.354609+0000","last_became_active":"2023-10-26T05:58:22.347122+0000","last_became_peered":"2023-10-26T05:58:22.347122+0000","last_unstale":"2023-10-26T05:58:22.354609+0000","last_undegraded":"2023-10-26T05:58:22.354609+0000","last_fullsized":"2023-10-26T05:58:22.354609+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:17.361064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346467+0000","last_change":"2023-10-26T05:58:22.345833+0000","last_active":"2023-10-26T05:58:22.346467+0000","last_peered":"2023-10-26T05:58:22.346467+0000","last_clean":"2023-10-26T05:58:22.346467+0000","last_became_active":"2023-10-26T05:58:22.345068+0000","last_became_peered":"2023-10-26T05:58:22.345068+0000","last_unstale":"2023-10-26T05:58:22.346467+0000","last_undegraded":"2023-10-26T05:58:22.346467+0000","last_fullsized":"2023-10-26T05:58:22.346467+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:28:19.284971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346563+0000","last_change":"2023-10-26T05:58:22.346152+0000","last_active":"2023-10-26T05:58:22.346563+0000","last_peered":"2023-10-26T05:58:22.346563+0000","last_clean":"2023-10-26T05:58:22.346563+0000","last_became_active":"2023-10-26T05:58:22.345448+0000","last_became_peered":"2023-10-26T05:58:22.345448+0000","last_unstale":"2023-10-26T05:58:22.346563+0000","last_undegraded":"2023-10-26T05:58:22.346563+0000","last_fullsized":"2023-10-26T05:58:22.346563+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:36.632867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315282+0000","last_change":"2023-10-26T05:58:22.345674+0000","last_active":"2023-10-26T05:58:24.315282+0000","last_peered":"2023-10-26T05:58:24.315282+0000","last_clean":"2023-10-26T05:58:24.315282+0000","last_became_active":"2023-10-26T05:58:22.345021+0000","last_became_peered":"2023-10-26T05:58:22.345021+0000","last_unstale":"2023-10-26T05:58:24.315282+0000","last_undegraded":"2023-10-26T05:58:24.315282+0000","last_fullsized":"2023-10-26T05:58:24.315282+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:14.416563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315123+0000","last_change":"2023-10-26T05:58:22.334920+0000","last_active":"2023-10-26T05:58:24.315123+0000","last_peered":"2023-10-26T05:58:24.315123+0000","last_clean":"2023-10-26T05:58:24.315123+0000","last_became_active":"2023-10-26T05:58:22.334242+0000","last_became_peered":"2023-10-26T05:58:22.334242+0000","last_unstale":"2023-10-26T05:58:24.315123+0000","last_undegraded":"2023-10-26T05:58:24.315123+0000","last_fullsized":"2023-10-26T05:58:24.315123+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:52.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346082+0000","last_change":"2023-10-26T05:58:22.345914+0000","last_active":"2023-10-26T05:58:22.346082+0000","last_peered":"2023-10-26T05:58:22.346082+0000","last_clean":"2023-10-26T05:58:22.346082+0000","last_became_active":"2023-10-26T05:58:22.345265+0000","last_became_peered":"2023-10-26T05:58:22.345265+0000","last_unstale":"2023-10-26T05:58:22.346082+0000","last_undegraded":"2023-10-26T05:58:22.346082+0000","last_fullsized":"2023-10-26T05:58:22.346082+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:48:36.639748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.333307+0000","last_change":"2023-10-26T05:58:22.333116+0000","last_active":"2023-10-26T05:58:22.333307+0000","last_peered":"2023-10-26T05:58:22.333307+0000","last_clean":"2023-10-26T05:58:22.333307+0000","last_became_active":"2023-10-26T05:58:22.332079+0000","last_became_peered":"2023-10-26T05:58:22.332079+0000","last_unstale":"2023-10-26T05:58:22.333307+0000","last_undegraded":"2023-10-26T05:58:22.333307+0000","last_fullsized":"2023-10-26T05:58:22.333307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:09.897677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.341240+0000","last_change":"2023-10-26T05:58:22.340945+0000","last_active":"2023-10-26T05:58:22.341240+0000","last_peered":"2023-10-26T05:58:22.341240+0000","last_clean":"2023-10-26T05:58:22.341240+0000","last_became_active":"2023-10-26T05:58:22.339985+0000","last_became_peered":"2023-10-26T05:58:22.339985+0000","last_unstale":"2023-10-26T05:58:22.341240+0000","last_undegraded":"2023-10-26T05:58:22.341240+0000","last_fullsized":"2023-10-26T05:58:22.341240+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:19.030437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313106+0000","last_change":"2023-10-26T05:58:22.340074+0000","last_active":"2023-10-26T05:58:23.313106+0000","last_peered":"2023-10-26T05:58:23.313106+0000","last_clean":"2023-10-26T05:58:23.313106+0000","last_became_active":"2023-10-26T05:58:22.339192+0000","last_became_peered":"2023-10-26T05:58:22.339192+0000","last_unstale":"2023-10-26T05:58:23.313106+0000","last_undegraded":"2023-10-26T05:58:23.313106+0000","last_fullsized":"2023-10-26T05:58:23.313106+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:52:56.973845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339496+0000","last_change":"2023-10-26T05:58:22.332034+0000","last_active":"2023-10-26T05:58:22.339496+0000","last_peered":"2023-10-26T05:58:22.339496+0000","last_clean":"2023-10-26T05:58:22.339496+0000","last_became_active":"2023-10-26T05:58:22.331398+0000","last_became_peered":"2023-10-26T05:58:22.331398+0000","last_unstale":"2023-10-26T05:58:22.339496+0000","last_undegraded":"2023-10-26T05:58:22.339496+0000","last_fullsized":"2023-10-26T05:58:22.339496+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:42:17.722074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315145+0000","last_change":"2023-10-26T05:58:22.350736+0000","last_active":"2023-10-26T05:58:24.315145+0000","last_peered":"2023-10-26T05:58:24.315145+0000","last_clean":"2023-10-26T05:58:24.315145+0000","last_became_active":"2023-10-26T05:58:22.342396+0000","last_became_peered":"2023-10-26T05:58:22.342396+0000","last_unstale":"2023-10-26T05:58:24.315145+0000","last_undegraded":"2023-10-26T05:58:24.315145+0000","last_fullsized":"2023-10-26T05:58:24.315145+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:34.235783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339709+0000","last_change":"2023-10-26T05:58:22.337768+0000","last_active":"2023-10-26T05:58:22.339709+0000","last_peered":"2023-10-26T05:58:22.339709+0000","last_clean":"2023-10-26T05:58:22.339709+0000","last_became_active":"2023-10-26T05:58:22.337144+0000","last_became_peered":"2023-10-26T05:58:22.337144+0000","last_unstale":"2023-10-26T05:58:22.339709+0000","last_undegraded":"2023-10-26T05:58:22.339709+0000","last_fullsized":"2023-10-26T05:58:22.339709+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:48.492831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.334236+0000","last_change":"2023-10-26T05:58:22.333954+0000","last_active":"2023-10-26T05:58:22.334236+0000","last_peered":"2023-10-26T05:58:22.334236+0000","last_clean":"2023-10-26T05:58:22.334236+0000","last_became_active":"2023-10-26T05:58:22.333132+0000","last_became_peered":"2023-10-26T05:58:22.333132+0000","last_unstale":"2023-10-26T05:58:22.334236+0000","last_undegraded":"2023-10-26T05:58:22.334236+0000","last_fullsized":"2023-10-26T05:58:22.334236+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:51.788771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.341594+0000","last_change":"2023-10-26T05:58:22.341398+0000","last_active":"2023-10-26T05:58:22.341594+0000","last_peered":"2023-10-26T05:58:22.341594+0000","last_clean":"2023-10-26T05:58:22.341594+0000","last_became_active":"2023-10-26T05:58:22.340692+0000","last_became_peered":"2023-10-26T05:58:22.340692+0000","last_unstale":"2023-10-26T05:58:22.341594+0000","last_undegraded":"2023-10-26T05:58:22.341594+0000","last_fullsized":"2023-10-26T05:58:22.341594+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:40:23.255743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354752+0000","last_change":"2023-10-26T05:58:22.354588+0000","last_active":"2023-10-26T05:58:22.354752+0000","last_peered":"2023-10-26T05:58:22.354752+0000","last_clean":"2023-10-26T05:58:22.354752+0000","last_became_active":"2023-10-26T05:58:22.349082+0000","last_became_peered":"2023-10-26T05:58:22.349082+0000","last_unstale":"2023-10-26T05:58:22.354752+0000","last_undegraded":"2023-10-26T05:58:22.354752+0000","last_fullsized":"2023-10-26T05:58:22.354752+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:56.725878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315350+0000","last_change":"2023-10-26T05:58:22.354487+0000","last_active":"2023-10-26T05:58:24.315350+0000","last_peered":"2023-10-26T05:58:24.315350+0000","last_clean":"2023-10-26T05:58:24.315350+0000","last_became_active":"2023-10-26T05:58:22.352585+0000","last_became_peered":"2023-10-26T05:58:22.352585+0000","last_unstale":"2023-10-26T05:58:24.315350+0000","last_undegraded":"2023-10-26T05:58:24.315350+0000","last_fullsized":"2023-10-26T05:58:24.315350+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:56:49.677051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.328634+0000","last_change":"2023-10-26T05:58:22.328482+0000","last_active":"2023-10-26T05:58:22.328634+0000","last_peered":"2023-10-26T05:58:22.328634+0000","last_clean":"2023-10-26T05:58:22.328634+0000","last_became_active":"2023-10-26T05:58:22.327511+0000","last_became_peered":"2023-10-26T05:58:22.327511+0000","last_unstale":"2023-10-26T05:58:22.328634+0000","last_undegraded":"2023-10-26T05:58:22.328634+0000","last_fullsized":"2023-10-26T05:58:22.328634+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:04:14.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315020+0000","last_change":"2023-10-26T05:58:22.337961+0000","last_active":"2023-10-26T05:58:24.315020+0000","last_peered":"2023-10-26T05:58:24.315020+0000","last_clean":"2023-10-26T05:58:24.315020+0000","last_became_active":"2023-10-26T05:58:22.337347+0000","last_became_peered":"2023-10-26T05:58:22.337347+0000","last_unstale":"2023-10-26T05:58:24.315020+0000","last_undegraded":"2023-10-26T05:58:24.315020+0000","last_fullsized":"2023-10-26T05:58:24.315020+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:17:44.061654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310484+0000","last_change":"2023-10-26T05:58:22.316896+0000","last_active":"2023-10-26T05:58:23.310484+0000","last_peered":"2023-10-26T05:58:23.310484+0000","last_clean":"2023-10-26T05:58:23.310484+0000","last_became_active":"2023-10-26T05:58:22.316290+0000","last_became_peered":"2023-10-26T05:58:22.316290+0000","last_unstale":"2023-10-26T05:58:23.310484+0000","last_undegraded":"2023-10-26T05:58:23.310484+0000","last_fullsized":"2023-10-26T05:58:23.310484+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:31.198902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315462+0000","last_change":"2023-10-26T05:58:22.351075+0000","last_active":"2023-10-26T05:58:24.315462+0000","last_peered":"2023-10-26T05:58:24.315462+0000","last_clean":"2023-10-26T05:58:24.315462+0000","last_became_active":"2023-10-26T05:58:22.346285+0000","last_became_peered":"2023-10-26T05:58:22.346285+0000","last_unstale":"2023-10-26T05:58:24.315462+0000","last_undegraded":"2023-10-26T05:58:24.315462+0000","last_fullsized":"2023-10-26T05:58:24.315462+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:01:12.184966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320456+0000","last_change":"2023-10-26T05:58:22.333395+0000","last_active":"2023-10-26T05:58:23.320456+0000","last_peered":"2023-10-26T05:58:23.320456+0000","last_clean":"2023-10-26T05:58:23.320456+0000","last_became_active":"2023-10-26T05:58:22.332776+0000","last_became_peered":"2023-10-26T05:58:22.332776+0000","last_unstale":"2023-10-26T05:58:23.320456+0000","last_undegraded":"2023-10-26T05:58:23.320456+0000","last_fullsized":"2023-10-26T05:58:23.320456+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:52:56.973845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320341+0000","last_change":"2023-10-26T05:58:22.341366+0000","last_active":"2023-10-26T05:58:23.320341+0000","last_peered":"2023-10-26T05:58:23.320341+0000","last_clean":"2023-10-26T05:58:23.320341+0000","last_became_active":"2023-10-26T05:58:22.339141+0000","last_became_peered":"2023-10-26T05:58:22.339141+0000","last_unstale":"2023-10-26T05:58:23.320341+0000","last_undegraded":"2023-10-26T05:58:23.320341+0000","last_fullsized":"2023-10-26T05:58:23.320341+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:48.591623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313073+0000","last_change":"2023-10-26T05:58:22.344844+0000","last_active":"2023-10-26T05:58:23.313073+0000","last_peered":"2023-10-26T05:58:23.313073+0000","last_clean":"2023-10-26T05:58:23.313073+0000","last_became_active":"2023-10-26T05:58:22.343803+0000","last_became_peered":"2023-10-26T05:58:22.343803+0000","last_unstale":"2023-10-26T05:58:23.313073+0000","last_undegraded":"2023-10-26T05:58:23.313073+0000","last_fullsized":"2023-10-26T05:58:23.313073+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:48.591623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.341626+0000","last_change":"2023-10-26T05:58:22.341431+0000","last_active":"2023-10-26T05:58:22.341626+0000","last_peered":"2023-10-26T05:58:22.341626+0000","last_clean":"2023-10-26T05:58:22.341626+0000","last_became_active":"2023-10-26T05:58:22.340363+0000","last_became_peered":"2023-10-26T05:58:22.340363+0000","last_unstale":"2023-10-26T05:58:22.341626+0000","last_undegraded":"2023-10-26T05:58:22.341626+0000","last_fullsized":"2023-10-26T05:58:22.341626+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:54.644310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"187'4","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.721064+0000","last_change":"2023-10-26T05:58:21.694753+0000","last_active":"2023-10-26T05:58:22.721064+0000","last_peered":"2023-10-26T05:58:22.721064+0000","last_clean":"2023-10-26T05:58:22.721064+0000","last_became_active":"2023-10-26T05:58:21.694236+0000","last_became_peered":"2023-10-26T05:58:21.694236+0000","last_unstale":"2023-10-26T05:58:22.721064+0000","last_undegraded":"2023-10-26T05:58:22.721064+0000","last_fullsized":"2023-10-26T05:58:22.721064+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:06.106664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320310+0000","last_change":"2023-10-26T05:58:22.324197+0000","last_active":"2023-10-26T05:58:23.320310+0000","last_peered":"2023-10-26T05:58:23.320310+0000","last_clean":"2023-10-26T05:58:23.320310+0000","last_became_active":"2023-10-26T05:58:22.323458+0000","last_became_peered":"2023-10-26T05:58:22.323458+0000","last_unstale":"2023-10-26T05:58:23.320310+0000","last_undegraded":"2023-10-26T05:58:23.320310+0000","last_fullsized":"2023-10-26T05:58:23.320310+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:15.325100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.319760+0000","last_change":"2023-10-26T05:58:21.691196+0000","last_active":"2023-10-26T05:58:23.319760+0000","last_peered":"2023-10-26T05:58:23.319760+0000","last_clean":"2023-10-26T05:58:23.319760+0000","last_became_active":"2023-10-26T05:58:21.686744+0000","last_became_peered":"2023-10-26T05:58:21.686744+0000","last_unstale":"2023-10-26T05:58:23.319760+0000","last_undegraded":"2023-10-26T05:58:23.319760+0000","last_fullsized":"2023-10-26T05:58:23.319760+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:49:49.331522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339607+0000","last_change":"2023-10-26T05:58:22.335576+0000","last_active":"2023-10-26T05:58:22.339607+0000","last_peered":"2023-10-26T05:58:22.339607+0000","last_clean":"2023-10-26T05:58:22.339607+0000","last_became_active":"2023-10-26T05:58:22.334072+0000","last_became_peered":"2023-10-26T05:58:22.334072+0000","last_unstale":"2023-10-26T05:58:22.339607+0000","last_undegraded":"2023-10-26T05:58:22.339607+0000","last_fullsized":"2023-10-26T05:58:22.339607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:26:59.854107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311781+0000","last_change":"2023-10-26T05:58:21.673643+0000","last_active":"2023-10-26T05:58:23.311781+0000","last_peered":"2023-10-26T05:58:23.311781+0000","last_clean":"2023-10-26T05:58:23.311781+0000","last_became_active":"2023-10-26T05:58:21.671877+0000","last_became_peered":"2023-10-26T05:58:21.671877+0000","last_unstale":"2023-10-26T05:58:23.311781+0000","last_undegraded":"2023-10-26T05:58:23.311781+0000","last_fullsized":"2023-10-26T05:58:23.311781+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:26.886007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315551+0000","last_change":"2023-10-26T05:58:22.355252+0000","last_active":"2023-10-26T05:58:24.315551+0000","last_peered":"2023-10-26T05:58:24.315551+0000","last_clean":"2023-10-26T05:58:24.315551+0000","last_became_active":"2023-10-26T05:58:22.353472+0000","last_became_peered":"2023-10-26T05:58:22.353472+0000","last_unstale":"2023-10-26T05:58:24.315551+0000","last_undegraded":"2023-10-26T05:58:24.315551+0000","last_fullsized":"2023-10-26T05:58:24.315551+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:25:00.604256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310838+0000","last_change":"2023-10-26T05:58:21.669865+0000","last_active":"2023-10-26T05:58:23.310838+0000","last_peered":"2023-10-26T05:58:23.310838+0000","last_clean":"2023-10-26T05:58:23.310838+0000","last_became_active":"2023-10-26T05:58:21.669180+0000","last_became_peered":"2023-10-26T05:58:21.669180+0000","last_unstale":"2023-10-26T05:58:23.310838+0000","last_undegraded":"2023-10-26T05:58:23.310838+0000","last_fullsized":"2023-10-26T05:58:23.310838+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:58:42.474934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.326754+0000","last_change":"2023-10-26T05:58:22.326376+0000","last_active":"2023-10-26T05:58:22.326754+0000","last_peered":"2023-10-26T05:58:22.326754+0000","last_clean":"2023-10-26T05:58:22.326754+0000","last_became_active":"2023-10-26T05:58:22.324974+0000","last_became_peered":"2023-10-26T05:58:22.324974+0000","last_unstale":"2023-10-26T05:58:22.326754+0000","last_undegraded":"2023-10-26T05:58:22.326754+0000","last_fullsized":"2023-10-26T05:58:22.326754+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:24.705234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315505+0000","last_change":"2023-10-26T05:58:21.695730+0000","last_active":"2023-10-26T05:58:24.315505+0000","last_peered":"2023-10-26T05:58:24.315505+0000","last_clean":"2023-10-26T05:58:24.315505+0000","last_became_active":"2023-10-26T05:58:21.694072+0000","last_became_peered":"2023-10-26T05:58:21.694072+0000","last_unstale":"2023-10-26T05:58:24.315505+0000","last_undegraded":"2023-10-26T05:58:24.315505+0000","last_fullsized":"2023-10-26T05:58:24.315505+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:01:17.591703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311446+0000","last_change":"2023-10-26T05:58:22.326272+0000","last_active":"2023-10-26T05:58:23.311446+0000","last_peered":"2023-10-26T05:58:23.311446+0000","last_clean":"2023-10-26T05:58:23.311446+0000","last_became_active":"2023-10-26T05:58:22.324085+0000","last_became_peered":"2023-10-26T05:58:22.324085+0000","last_unstale":"2023-10-26T05:58:23.311446+0000","last_undegraded":"2023-10-26T05:58:23.311446+0000","last_fullsized":"2023-10-26T05:58:23.311446+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:54:38.006089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310579+0000","last_change":"2023-10-26T05:58:21.658840+0000","last_active":"2023-10-26T05:58:23.310579+0000","last_peered":"2023-10-26T05:58:23.310579+0000","last_clean":"2023-10-26T05:58:23.310579+0000","last_became_active":"2023-10-26T05:58:21.658544+0000","last_became_peered":"2023-10-26T05:58:21.658544+0000","last_unstale":"2023-10-26T05:58:23.310579+0000","last_undegraded":"2023-10-26T05:58:23.310579+0000","last_fullsized":"2023-10-26T05:58:23.310579+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:57.438255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311080+0000","last_change":"2023-10-26T05:58:22.336290+0000","last_active":"2023-10-26T05:58:23.311080+0000","last_peered":"2023-10-26T05:58:23.311080+0000","last_clean":"2023-10-26T05:58:23.311080+0000","last_became_active":"2023-10-26T05:58:22.333625+0000","last_became_peered":"2023-10-26T05:58:22.333625+0000","last_unstale":"2023-10-26T05:58:23.311080+0000","last_undegraded":"2023-10-26T05:58:23.311080+0000","last_fullsized":"2023-10-26T05:58:23.311080+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:19:33.092207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.331026+0000","last_change":"2023-10-26T05:58:21.684041+0000","last_active":"2023-10-26T05:58:22.331026+0000","last_peered":"2023-10-26T05:58:22.331026+0000","last_clean":"2023-10-26T05:58:22.331026+0000","last_became_active":"2023-10-26T05:58:21.682532+0000","last_became_peered":"2023-10-26T05:58:21.682532+0000","last_unstale":"2023-10-26T05:58:22.331026+0000","last_undegraded":"2023-10-26T05:58:22.331026+0000","last_fullsized":"2023-10-26T05:58:22.331026+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:51.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339564+0000","last_change":"2023-10-26T05:58:22.333799+0000","last_active":"2023-10-26T05:58:22.339564+0000","last_peered":"2023-10-26T05:58:22.339564+0000","last_clean":"2023-10-26T05:58:22.339564+0000","last_became_active":"2023-10-26T05:58:22.333170+0000","last_became_peered":"2023-10-26T05:58:22.333170+0000","last_unstale":"2023-10-26T05:58:22.339564+0000","last_undegraded":"2023-10-26T05:58:22.339564+0000","last_fullsized":"2023-10-26T05:58:22.339564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:20:58.280859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.320739+0000","last_change":"2023-10-26T05:58:21.673169+0000","last_active":"2023-10-26T05:58:22.320739+0000","last_peered":"2023-10-26T05:58:22.320739+0000","last_clean":"2023-10-26T05:58:22.320739+0000","last_became_active":"2023-10-26T05:58:21.672853+0000","last_became_peered":"2023-10-26T05:58:21.672853+0000","last_unstale":"2023-10-26T05:58:22.320739+0000","last_undegraded":"2023-10-26T05:58:22.320739+0000","last_fullsized":"2023-10-26T05:58:22.320739+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:08.794339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313269+0000","last_change":"2023-10-26T05:58:22.318650+0000","last_active":"2023-10-26T05:58:23.313269+0000","last_peered":"2023-10-26T05:58:23.313269+0000","last_clean":"2023-10-26T05:58:23.313269+0000","last_became_active":"2023-10-26T05:58:22.318039+0000","last_became_peered":"2023-10-26T05:58:22.318039+0000","last_unstale":"2023-10-26T05:58:23.313269+0000","last_undegraded":"2023-10-26T05:58:23.313269+0000","last_fullsized":"2023-10-26T05:58:23.313269+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:12.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.332984+0000","last_change":"2023-10-26T05:58:21.679705+0000","last_active":"2023-10-26T05:58:22.332984+0000","last_peered":"2023-10-26T05:58:22.332984+0000","last_clean":"2023-10-26T05:58:22.332984+0000","last_became_active":"2023-10-26T05:58:21.679073+0000","last_became_peered":"2023-10-26T05:58:21.679073+0000","last_unstale":"2023-10-26T05:58:22.332984+0000","last_undegraded":"2023-10-26T05:58:22.332984+0000","last_fullsized":"2023-10-26T05:58:22.332984+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:34:38.017686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312601+0000","last_change":"2023-10-26T05:58:22.329056+0000","last_active":"2023-10-26T05:58:23.312601+0000","last_peered":"2023-10-26T05:58:23.312601+0000","last_clean":"2023-10-26T05:58:23.312601+0000","last_became_active":"2023-10-26T05:58:22.328392+0000","last_became_peered":"2023-10-26T05:58:22.328392+0000","last_unstale":"2023-10-26T05:58:23.312601+0000","last_undegraded":"2023-10-26T05:58:23.312601+0000","last_fullsized":"2023-10-26T05:58:23.312601+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:17.086809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"187'3","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.710907+0000","last_change":"2023-10-26T05:58:21.669862+0000","last_active":"2023-10-26T05:58:22.710907+0000","last_peered":"2023-10-26T05:58:22.710907+0000","last_clean":"2023-10-26T05:58:22.710907+0000","last_became_active":"2023-10-26T05:58:21.669554+0000","last_became_peered":"2023-10-26T05:58:21.669554+0000","last_unstale":"2023-10-26T05:58:22.710907+0000","last_undegraded":"2023-10-26T05:58:22.710907+0000","last_fullsized":"2023-10-26T05:58:22.710907+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:21:33.811150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320502+0000","last_change":"2023-10-26T05:58:22.353971+0000","last_active":"2023-10-26T05:58:23.320502+0000","last_peered":"2023-10-26T05:58:23.320502+0000","last_clean":"2023-10-26T05:58:23.320502+0000","last_became_active":"2023-10-26T05:58:22.351409+0000","last_became_peered":"2023-10-26T05:58:22.351409+0000","last_unstale":"2023-10-26T05:58:23.320502+0000","last_undegraded":"2023-10-26T05:58:23.320502+0000","last_fullsized":"2023-10-26T05:58:23.320502+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:12.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"187'6","reported_seq":18,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320001+0000","last_change":"2023-10-26T05:58:21.687392+0000","last_active":"2023-10-26T05:58:23.320001+0000","last_peered":"2023-10-26T05:58:23.320001+0000","last_clean":"2023-10-26T05:58:23.320001+0000","last_became_active":"2023-10-26T05:58:21.685415+0000","last_became_peered":"2023-10-26T05:58:21.685415+0000","last_unstale":"2023-10-26T05:58:23.320001+0000","last_undegraded":"2023-10-26T05:58:23.320001+0000","last_fullsized":"2023-10-26T05:58:23.320001+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:53.631378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313487+0000","last_change":"2023-10-26T05:58:22.337374+0000","last_active":"2023-10-26T05:58:23.313487+0000","last_peered":"2023-10-26T05:58:23.313487+0000","last_clean":"2023-10-26T05:58:23.313487+0000","last_became_active":"2023-10-26T05:58:22.336722+0000","last_became_peered":"2023-10-26T05:58:22.336722+0000","last_unstale":"2023-10-26T05:58:23.313487+0000","last_undegraded":"2023-10-26T05:58:23.313487+0000","last_fullsized":"2023-10-26T05:58:23.313487+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:45:46.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311998+0000","last_change":"2023-10-26T05:58:21.680120+0000","last_active":"2023-10-26T05:58:23.311998+0000","last_peered":"2023-10-26T05:58:23.311998+0000","last_clean":"2023-10-26T05:58:23.311998+0000","last_became_active":"2023-10-26T05:58:21.674533+0000","last_became_peered":"2023-10-26T05:58:21.674533+0000","last_unstale":"2023-10-26T05:58:23.311998+0000","last_undegraded":"2023-10-26T05:58:23.311998+0000","last_fullsized":"2023-10-26T05:58:23.311998+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:37:05.453161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.338271+0000","last_change":"2023-10-26T05:58:22.338088+0000","last_active":"2023-10-26T05:58:22.338271+0000","last_peered":"2023-10-26T05:58:22.338271+0000","last_clean":"2023-10-26T05:58:22.338271+0000","last_became_active":"2023-10-26T05:58:22.337483+0000","last_became_peered":"2023-10-26T05:58:22.337483+0000","last_unstale":"2023-10-26T05:58:22.338271+0000","last_undegraded":"2023-10-26T05:58:22.338271+0000","last_fullsized":"2023-10-26T05:58:22.338271+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:23:02.083024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313710+0000","last_change":"2023-10-26T05:58:21.676152+0000","last_active":"2023-10-26T05:58:23.313710+0000","last_peered":"2023-10-26T05:58:23.313710+0000","last_clean":"2023-10-26T05:58:23.313710+0000","last_became_active":"2023-10-26T05:58:21.675523+0000","last_became_peered":"2023-10-26T05:58:21.675523+0000","last_unstale":"2023-10-26T05:58:23.313710+0000","last_undegraded":"2023-10-26T05:58:23.313710+0000","last_fullsized":"2023-10-26T05:58:23.313710+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:04:13.062337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310353+0000","last_change":"2023-10-26T05:58:22.336105+0000","last_active":"2023-10-26T05:58:23.310353+0000","last_peered":"2023-10-26T05:58:23.310353+0000","last_clean":"2023-10-26T05:58:23.310353+0000","last_became_active":"2023-10-26T05:58:22.335458+0000","last_became_peered":"2023-10-26T05:58:22.335458+0000","last_unstale":"2023-10-26T05:58:23.310353+0000","last_undegraded":"2023-10-26T05:58:23.310353+0000","last_fullsized":"2023-10-26T05:58:23.310353+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:02:59.661655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312109+0000","last_change":"2023-10-26T05:58:21.672239+0000","last_active":"2023-10-26T05:58:23.312109+0000","last_peered":"2023-10-26T05:58:23.312109+0000","last_clean":"2023-10-26T05:58:23.312109+0000","last_became_active":"2023-10-26T05:58:21.671488+0000","last_became_peered":"2023-10-26T05:58:21.671488+0000","last_unstale":"2023-10-26T05:58:23.312109+0000","last_undegraded":"2023-10-26T05:58:23.312109+0000","last_fullsized":"2023-10-26T05:58:23.312109+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:49:44.945084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315221+0000","last_change":"2023-10-26T05:58:22.334849+0000","last_active":"2023-10-26T05:58:24.315221+0000","last_peered":"2023-10-26T05:58:24.315221+0000","last_clean":"2023-10-26T05:58:24.315221+0000","last_became_active":"2023-10-26T05:58:22.333604+0000","last_became_peered":"2023-10-26T05:58:22.333604+0000","last_unstale":"2023-10-26T05:58:24.315221+0000","last_undegraded":"2023-10-26T05:58:24.315221+0000","last_fullsized":"2023-10-26T05:58:24.315221+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:45:42.137159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"187'8","reported_seq":20,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320630+0000","last_change":"2023-10-26T05:58:21.691418+0000","last_active":"2023-10-26T05:58:23.320630+0000","last_peered":"2023-10-26T05:58:23.320630+0000","last_clean":"2023-10-26T05:58:23.320630+0000","last_became_active":"2023-10-26T05:58:21.687603+0000","last_became_peered":"2023-10-26T05:58:21.687603+0000","last_unstale":"2023-10-26T05:58:23.320630+0000","last_undegraded":"2023-10-26T05:58:23.320630+0000","last_fullsized":"2023-10-26T05:58:23.320630+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:08.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311561+0000","last_change":"2023-10-26T05:58:22.334259+0000","last_active":"2023-10-26T05:58:23.311561+0000","last_peered":"2023-10-26T05:58:23.311561+0000","last_clean":"2023-10-26T05:58:23.311561+0000","last_became_active":"2023-10-26T05:58:22.331850+0000","last_became_peered":"2023-10-26T05:58:22.331850+0000","last_unstale":"2023-10-26T05:58:23.311561+0000","last_undegraded":"2023-10-26T05:58:23.311561+0000","last_fullsized":"2023-10-26T05:58:23.311561+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:47.387461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.316025+0000","last_change":"2023-10-26T05:58:21.683255+0000","last_active":"2023-10-26T05:58:24.316025+0000","last_peered":"2023-10-26T05:58:24.316025+0000","last_clean":"2023-10-26T05:58:24.316025+0000","last_became_active":"2023-10-26T05:58:21.682638+0000","last_became_peered":"2023-10-26T05:58:21.682638+0000","last_unstale":"2023-10-26T05:58:24.316025+0000","last_undegraded":"2023-10-26T05:58:24.316025+0000","last_fullsized":"2023-10-26T05:58:24.316025+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:32:15.290323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312977+0000","last_change":"2023-10-26T05:58:22.327777+0000","last_active":"2023-10-26T05:58:23.312977+0000","last_peered":"2023-10-26T05:58:23.312977+0000","last_clean":"2023-10-26T05:58:23.312977+0000","last_became_active":"2023-10-26T05:58:22.315487+0000","last_became_peered":"2023-10-26T05:58:22.315487+0000","last_unstale":"2023-10-26T05:58:23.312977+0000","last_undegraded":"2023-10-26T05:58:23.312977+0000","last_fullsized":"2023-10-26T05:58:23.312977+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:15.325100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315874+0000","last_change":"2023-10-26T05:58:21.696555+0000","last_active":"2023-10-26T05:58:24.315874+0000","last_peered":"2023-10-26T05:58:24.315874+0000","last_clean":"2023-10-26T05:58:24.315874+0000","last_became_active":"2023-10-26T05:58:21.695560+0000","last_became_peered":"2023-10-26T05:58:21.695560+0000","last_unstale":"2023-10-26T05:58:24.315874+0000","last_undegraded":"2023-10-26T05:58:24.315874+0000","last_fullsized":"2023-10-26T05:58:24.315874+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:45.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311922+0000","last_change":"2023-10-26T05:58:22.325785+0000","last_active":"2023-10-26T05:58:23.311922+0000","last_peered":"2023-10-26T05:58:23.311922+0000","last_clean":"2023-10-26T05:58:23.311922+0000","last_became_active":"2023-10-26T05:58:22.323127+0000","last_became_peered":"2023-10-26T05:58:22.323127+0000","last_unstale":"2023-10-26T05:58:23.311922+0000","last_undegraded":"2023-10-26T05:58:23.311922+0000","last_fullsized":"2023-10-26T05:58:23.311922+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:56:36.661796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.720607+0000","last_change":"2023-10-26T05:58:21.692853+0000","last_active":"2023-10-26T05:58:22.720607+0000","last_peered":"2023-10-26T05:58:22.720607+0000","last_clean":"2023-10-26T05:58:22.720607+0000","last_became_active":"2023-10-26T05:58:21.691588+0000","last_became_peered":"2023-10-26T05:58:21.691588+0000","last_unstale":"2023-10-26T05:58:22.720607+0000","last_undegraded":"2023-10-26T05:58:22.720607+0000","last_fullsized":"2023-10-26T05:58:22.720607+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:28.073023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.337400+0000","last_change":"2023-10-26T05:58:22.337237+0000","last_active":"2023-10-26T05:58:22.337400+0000","last_peered":"2023-10-26T05:58:22.337400+0000","last_clean":"2023-10-26T05:58:22.337400+0000","last_became_active":"2023-10-26T05:58:22.336034+0000","last_became_peered":"2023-10-26T05:58:22.336034+0000","last_unstale":"2023-10-26T05:58:22.337400+0000","last_undegraded":"2023-10-26T05:58:22.337400+0000","last_fullsized":"2023-10-26T05:58:22.337400+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:15.325100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.329826+0000","last_change":"2023-10-26T05:58:21.687342+0000","last_active":"2023-10-26T05:58:22.329826+0000","last_peered":"2023-10-26T05:58:22.329826+0000","last_clean":"2023-10-26T05:58:22.329826+0000","last_became_active":"2023-10-26T05:58:21.686710+0000","last_became_peered":"2023-10-26T05:58:21.686710+0000","last_unstale":"2023-10-26T05:58:22.329826+0000","last_undegraded":"2023-10-26T05:58:22.329826+0000","last_fullsized":"2023-10-26T05:58:22.329826+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:10.211119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315794+0000","last_change":"2023-10-26T05:58:22.352535+0000","last_active":"2023-10-26T05:58:24.315794+0000","last_peered":"2023-10-26T05:58:24.315794+0000","last_clean":"2023-10-26T05:58:24.315794+0000","last_became_active":"2023-10-26T05:58:22.350355+0000","last_became_peered":"2023-10-26T05:58:22.350355+0000","last_unstale":"2023-10-26T05:58:24.315794+0000","last_undegraded":"2023-10-26T05:58:24.315794+0000","last_fullsized":"2023-10-26T05:58:24.315794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:37:57.699270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310681+0000","last_change":"2023-10-26T05:58:21.668410+0000","last_active":"2023-10-26T05:58:23.310681+0000","last_peered":"2023-10-26T05:58:23.310681+0000","last_clean":"2023-10-26T05:58:23.310681+0000","last_became_active":"2023-10-26T05:58:21.668076+0000","last_became_peered":"2023-10-26T05:58:21.668076+0000","last_unstale":"2023-10-26T05:58:23.310681+0000","last_undegraded":"2023-10-26T05:58:23.310681+0000","last_fullsized":"2023-10-26T05:58:23.310681+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:19.673528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.327057+0000","last_change":"2023-10-26T05:58:22.326912+0000","last_active":"2023-10-26T05:58:22.327057+0000","last_peered":"2023-10-26T05:58:22.327057+0000","last_clean":"2023-10-26T05:58:22.327057+0000","last_became_active":"2023-10-26T05:58:22.325781+0000","last_became_peered":"2023-10-26T05:58:22.325781+0000","last_unstale":"2023-10-26T05:58:22.327057+0000","last_undegraded":"2023-10-26T05:58:22.327057+0000","last_fullsized":"2023-10-26T05:58:22.327057+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:03:20.402710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.329352+0000","last_change":"2023-10-26T05:58:21.683764+0000","last_active":"2023-10-26T05:58:22.329352+0000","last_peered":"2023-10-26T05:58:22.329352+0000","last_clean":"2023-10-26T05:58:22.329352+0000","last_became_active":"2023-10-26T05:58:21.683122+0000","last_became_peered":"2023-10-26T05:58:21.683122+0000","last_unstale":"2023-10-26T05:58:22.329352+0000","last_undegraded":"2023-10-26T05:58:22.329352+0000","last_fullsized":"2023-10-26T05:58:22.329352+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:50.534395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.351961+0000","last_change":"2023-10-26T05:58:22.351670+0000","last_active":"2023-10-26T05:58:22.351961+0000","last_peered":"2023-10-26T05:58:22.351961+0000","last_clean":"2023-10-26T05:58:22.351961+0000","last_became_active":"2023-10-26T05:58:22.350976+0000","last_became_peered":"2023-10-26T05:58:22.350976+0000","last_unstale":"2023-10-26T05:58:22.351961+0000","last_undegraded":"2023-10-26T05:58:22.351961+0000","last_fullsized":"2023-10-26T05:58:22.351961+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:31:05.893224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312193+0000","last_change":"2023-10-26T05:58:21.690261+0000","last_active":"2023-10-26T05:58:23.312193+0000","last_peered":"2023-10-26T05:58:23.312193+0000","last_clean":"2023-10-26T05:58:23.312193+0000","last_became_active":"2023-10-26T05:58:21.690061+0000","last_became_peered":"2023-10-26T05:58:21.690061+0000","last_unstale":"2023-10-26T05:58:23.312193+0000","last_undegraded":"2023-10-26T05:58:23.312193+0000","last_fullsized":"2023-10-26T05:58:23.312193+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:56.009066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.344376+0000","last_change":"2023-10-26T05:58:22.344231+0000","last_active":"2023-10-26T05:58:22.344376+0000","last_peered":"2023-10-26T05:58:22.344376+0000","last_clean":"2023-10-26T05:58:22.344376+0000","last_became_active":"2023-10-26T05:58:22.342817+0000","last_became_peered":"2023-10-26T05:58:22.342817+0000","last_unstale":"2023-10-26T05:58:22.344376+0000","last_undegraded":"2023-10-26T05:58:22.344376+0000","last_fullsized":"2023-10-26T05:58:22.344376+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:23.736483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.333862+0000","last_change":"2023-10-26T05:58:21.693224+0000","last_active":"2023-10-26T05:58:22.333862+0000","last_peered":"2023-10-26T05:58:22.333862+0000","last_clean":"2023-10-26T05:58:22.333862+0000","last_became_active":"2023-10-26T05:58:21.692310+0000","last_became_peered":"2023-10-26T05:58:21.692310+0000","last_unstale":"2023-10-26T05:58:22.333862+0000","last_undegraded":"2023-10-26T05:58:22.333862+0000","last_fullsized":"2023-10-26T05:58:22.333862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:45.334233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313443+0000","last_change":"2023-10-26T05:58:22.317439+0000","last_active":"2023-10-26T05:58:23.313443+0000","last_peered":"2023-10-26T05:58:23.313443+0000","last_clean":"2023-10-26T05:58:23.313443+0000","last_became_active":"2023-10-26T05:58:22.316804+0000","last_became_peered":"2023-10-26T05:58:22.316804+0000","last_unstale":"2023-10-26T05:58:23.313443+0000","last_undegraded":"2023-10-26T05:58:23.313443+0000","last_fullsized":"2023-10-26T05:58:23.313443+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:06:48.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313997+0000","last_change":"2023-10-26T05:58:21.678448+0000","last_active":"2023-10-26T05:58:23.313997+0000","last_peered":"2023-10-26T05:58:23.313997+0000","last_clean":"2023-10-26T05:58:23.313997+0000","last_became_active":"2023-10-26T05:58:21.677332+0000","last_became_peered":"2023-10-26T05:58:21.677332+0000","last_unstale":"2023-10-26T05:58:23.313997+0000","last_undegraded":"2023-10-26T05:58:23.313997+0000","last_fullsized":"2023-10-26T05:58:23.313997+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:08.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.329978+0000","last_change":"2023-10-26T05:58:22.329814+0000","last_active":"2023-10-26T05:58:22.329978+0000","last_peered":"2023-10-26T05:58:22.329978+0000","last_clean":"2023-10-26T05:58:22.329978+0000","last_became_active":"2023-10-26T05:58:22.328287+0000","last_became_peered":"2023-10-26T05:58:22.328287+0000","last_unstale":"2023-10-26T05:58:22.329978+0000","last_undegraded":"2023-10-26T05:58:22.329978+0000","last_fullsized":"2023-10-26T05:58:22.329978+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:17.086809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.316235+0000","last_change":"2023-10-26T05:58:21.695289+0000","last_active":"2023-10-26T05:58:24.316235+0000","last_peered":"2023-10-26T05:58:24.316235+0000","last_clean":"2023-10-26T05:58:24.316235+0000","last_became_active":"2023-10-26T05:58:21.694747+0000","last_became_peered":"2023-10-26T05:58:21.694747+0000","last_unstale":"2023-10-26T05:58:24.316235+0000","last_undegraded":"2023-10-26T05:58:24.316235+0000","last_fullsized":"2023-10-26T05:58:24.316235+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:52.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339751+0000","last_change":"2023-10-26T05:58:22.339246+0000","last_active":"2023-10-26T05:58:22.339751+0000","last_peered":"2023-10-26T05:58:22.339751+0000","last_clean":"2023-10-26T05:58:22.339751+0000","last_became_active":"2023-10-26T05:58:22.338545+0000","last_became_peered":"2023-10-26T05:58:22.338545+0000","last_unstale":"2023-10-26T05:58:22.339751+0000","last_undegraded":"2023-10-26T05:58:22.339751+0000","last_fullsized":"2023-10-26T05:58:22.339751+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:35:07.143366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.331385+0000","last_change":"2023-10-26T05:58:21.685710+0000","last_active":"2023-10-26T05:58:22.331385+0000","last_peered":"2023-10-26T05:58:22.331385+0000","last_clean":"2023-10-26T05:58:22.331385+0000","last_became_active":"2023-10-26T05:58:21.685071+0000","last_became_peered":"2023-10-26T05:58:21.685071+0000","last_unstale":"2023-10-26T05:58:22.331385+0000","last_undegraded":"2023-10-26T05:58:22.331385+0000","last_fullsized":"2023-10-26T05:58:22.331385+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:49:33.854076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313294+0000","last_change":"2023-10-26T05:58:22.338035+0000","last_active":"2023-10-26T05:58:23.313294+0000","last_peered":"2023-10-26T05:58:23.313294+0000","last_clean":"2023-10-26T05:58:23.313294+0000","last_became_active":"2023-10-26T05:58:22.336886+0000","last_became_peered":"2023-10-26T05:58:22.336886+0000","last_unstale":"2023-10-26T05:58:23.313294+0000","last_undegraded":"2023-10-26T05:58:23.313294+0000","last_fullsized":"2023-10-26T05:58:23.313294+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:15.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.314045+0000","last_change":"2023-10-26T05:58:21.690003+0000","last_active":"2023-10-26T05:58:23.314045+0000","last_peered":"2023-10-26T05:58:23.314045+0000","last_clean":"2023-10-26T05:58:23.314045+0000","last_became_active":"2023-10-26T05:58:21.689688+0000","last_became_peered":"2023-10-26T05:58:21.689688+0000","last_unstale":"2023-10-26T05:58:23.314045+0000","last_undegraded":"2023-10-26T05:58:23.314045+0000","last_fullsized":"2023-10-26T05:58:23.314045+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:03:19.389777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313719+0000","last_change":"2023-10-26T05:58:22.344182+0000","last_active":"2023-10-26T05:58:23.313719+0000","last_peered":"2023-10-26T05:58:23.313719+0000","last_clean":"2023-10-26T05:58:23.313719+0000","last_became_active":"2023-10-26T05:58:22.342504+0000","last_became_peered":"2023-10-26T05:58:22.342504+0000","last_unstale":"2023-10-26T05:58:23.313719+0000","last_undegraded":"2023-10-26T05:58:23.313719+0000","last_fullsized":"2023-10-26T05:58:23.313719+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:28:11.957614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320915+0000","last_change":"2023-10-26T05:58:21.691685+0000","last_active":"2023-10-26T05:58:23.320915+0000","last_peered":"2023-10-26T05:58:23.320915+0000","last_clean":"2023-10-26T05:58:23.320915+0000","last_became_active":"2023-10-26T05:58:21.690821+0000","last_became_peered":"2023-10-26T05:58:21.690821+0000","last_unstale":"2023-10-26T05:58:23.320915+0000","last_undegraded":"2023-10-26T05:58:23.320915+0000","last_fullsized":"2023-10-26T05:58:23.320915+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:16.073876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320524+0000","last_change":"2023-10-26T05:58:22.340779+0000","last_active":"2023-10-26T05:58:23.320524+0000","last_peered":"2023-10-26T05:58:23.320524+0000","last_clean":"2023-10-26T05:58:23.320524+0000","last_became_active":"2023-10-26T05:58:22.334743+0000","last_became_peered":"2023-10-26T05:58:22.334743+0000","last_unstale":"2023-10-26T05:58:23.320524+0000","last_undegraded":"2023-10-26T05:58:23.320524+0000","last_fullsized":"2023-10-26T05:58:23.320524+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:15.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312287+0000","last_change":"2023-10-26T05:58:21.672381+0000","last_active":"2023-10-26T05:58:23.312287+0000","last_peered":"2023-10-26T05:58:23.312287+0000","last_clean":"2023-10-26T05:58:23.312287+0000","last_became_active":"2023-10-26T05:58:21.671920+0000","last_became_peered":"2023-10-26T05:58:21.671920+0000","last_unstale":"2023-10-26T05:58:23.312287+0000","last_undegraded":"2023-10-26T05:58:23.312287+0000","last_fullsized":"2023-10-26T05:58:23.312287+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:26:25.486224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.332227+0000","last_change":"2023-10-26T05:58:22.331143+0000","last_active":"2023-10-26T05:58:22.332227+0000","last_peered":"2023-10-26T05:58:22.332227+0000","last_clean":"2023-10-26T05:58:22.332227+0000","last_became_active":"2023-10-26T05:58:22.330438+0000","last_became_peered":"2023-10-26T05:58:22.330438+0000","last_unstale":"2023-10-26T05:58:22.332227+0000","last_undegraded":"2023-10-26T05:58:22.332227+0000","last_fullsized":"2023-10-26T05:58:22.332227+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:06.721462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.328542+0000","last_change":"2023-10-26T05:58:21.677323+0000","last_active":"2023-10-26T05:58:22.328542+0000","last_peered":"2023-10-26T05:58:22.328542+0000","last_clean":"2023-10-26T05:58:22.328542+0000","last_became_active":"2023-10-26T05:58:21.676385+0000","last_became_peered":"2023-10-26T05:58:21.676385+0000","last_unstale":"2023-10-26T05:58:22.328542+0000","last_undegraded":"2023-10-26T05:58:22.328542+0000","last_fullsized":"2023-10-26T05:58:22.328542+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:47.746497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315625+0000","last_change":"2023-10-26T05:58:22.350309+0000","last_active":"2023-10-26T05:58:24.315625+0000","last_peered":"2023-10-26T05:58:24.315625+0000","last_clean":"2023-10-26T05:58:24.315625+0000","last_became_active":"2023-10-26T05:58:22.349633+0000","last_became_peered":"2023-10-26T05:58:22.349633+0000","last_unstale":"2023-10-26T05:58:24.315625+0000","last_undegraded":"2023-10-26T05:58:24.315625+0000","last_fullsized":"2023-10-26T05:58:24.315625+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:19:26.328828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320238+0000","last_change":"2023-10-26T05:58:21.680981+0000","last_active":"2023-10-26T05:58:23.320238+0000","last_peered":"2023-10-26T05:58:23.320238+0000","last_clean":"2023-10-26T05:58:23.320238+0000","last_became_active":"2023-10-26T05:58:21.680243+0000","last_became_peered":"2023-10-26T05:58:21.680243+0000","last_unstale":"2023-10-26T05:58:23.320238+0000","last_undegraded":"2023-10-26T05:58:23.320238+0000","last_fullsized":"2023-10-26T05:58:23.320238+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:23.692301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313398+0000","last_change":"2023-10-26T05:58:22.319747+0000","last_active":"2023-10-26T05:58:23.313398+0000","last_peered":"2023-10-26T05:58:23.313398+0000","last_clean":"2023-10-26T05:58:23.313398+0000","last_became_active":"2023-10-26T05:58:22.319116+0000","last_became_peered":"2023-10-26T05:58:22.319116+0000","last_unstale":"2023-10-26T05:58:23.313398+0000","last_undegraded":"2023-10-26T05:58:23.313398+0000","last_fullsized":"2023-10-26T05:58:23.313398+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:58.719687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320722+0000","last_change":"2023-10-26T05:58:21.689429+0000","last_active":"2023-10-26T05:58:23.320722+0000","last_peered":"2023-10-26T05:58:23.320722+0000","last_clean":"2023-10-26T05:58:23.320722+0000","last_became_active":"2023-10-26T05:58:21.677549+0000","last_became_peered":"2023-10-26T05:58:21.677549+0000","last_unstale":"2023-10-26T05:58:23.320722+0000","last_undegraded":"2023-10-26T05:58:23.320722+0000","last_fullsized":"2023-10-26T05:58:23.320722+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:08.794339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":319,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-26T05:58:21.665741+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:58:21.665741+0000","last_peered":"2023-10-26T05:58:21.665741+0000","last_clean":"2023-10-26T05:58:21.665741+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:58:21.665741+0000","last_undegraded":"2023-10-26T05:58:21.665741+0000","last_fullsized":"2023-10-26T05:58:21.665741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1608,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193703,"statfs":{"total":200000000,"available":198352768,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.878,"15min":0.86299999999999999},"min":{"1min":0.52700000000000002,"5min":0.501,"15min":0.39600000000000002},"max":{"1min":1.1739999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.86699999999999999,"15min":0.84699999999999998},"min":{"1min":0.67400000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.3100000000000001},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.95599999999999996,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3149999999999999,"5min":1.387,"15min":1.4319999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.0580000000000001,"15min":0.999},"min":{"1min":0.84099999999999997,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.93500000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90000000000000002,"15min":0.84699999999999998},"min":{"1min":0.67500000000000004,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.448,"15min":1.448},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":0.93000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1870000000000001,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.93999999999999995,"15min":0.90500000000000003},"min":{"1min":0.65700000000000003,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.018,"5min":0.96699999999999997,"15min":0.97499999999999998},"min":{"1min":0.624,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.65300000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.236,"5min":1.2589999999999999,"15min":1.639},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.749}]},{"osd":5,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.002,"15min":1.0669999999999999},"min":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.1599999999999999,"5min":1.482,"15min":1.6679999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0289999999999999,"15min":1.0600000000000001},"min":{"1min":0.75700000000000001,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.3360000000000001,"5min":1.504,"15min":1.702},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.89300000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.347,"5min":1.391,"15min":1.7509999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0629999999999999,"15min":1.0549999999999999},"min":{"1min":0.71099999999999997,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.3320000000000001,"5min":1.5,"15min":1.6180000000000001},"last":0.88800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1029,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198945468,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.88400000000000001,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.216,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.872,"15min":0.86599999999999999},"min":{"1min":0.503,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.25}]},{"osd":1,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.80100000000000005},"min":{"1min":0.65000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.077,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55400000000000005,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.1830000000000001,"5min":1.3899999999999999,"15min":1.405},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88900000000000001,"15min":0.84799999999999998},"min":{"1min":0.57299999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1100000000000001,"5min":1.268,"15min":1.8460000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.88300000000000001,"15min":0.85799999999999998},"min":{"1min":0.52600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.137,"5min":1.337,"15min":1.9239999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.998,"15min":0.93600000000000005},"min":{"1min":0.67200000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.29,"5min":1.296,"15min":1.3080000000000001},"last":1.341},{"interface":"front","average":{"1min":0.998,"5min":1.0620000000000001,"15min":1.0049999999999999},"min":{"1min":0.64400000000000002,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.3759999999999999,"15min":1.399},"last":1.397}]},{"osd":4,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":0.95599999999999996},"min":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.502},"max":{"1min":1.1970000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.4299999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0029999999999999,"15min":0.98199999999999998},"min":{"1min":0.629,"5min":0.621,"15min":0.442},"max":{"1min":1.264,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.077,"15min":1.0509999999999999},"min":{"1min":0.77600000000000002,"5min":0.59999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2569999999999999,"5min":1.5509999999999999,"15min":1.554},"last":1.204},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.98099999999999998},"min":{"1min":0.69199999999999995,"5min":0.53300000000000003,"15min":0.36799999999999999},"max":{"1min":1.3029999999999999,"5min":1.512,"15min":1.512},"last":1.159}]},{"osd":7,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.1279999999999999,"15min":1.1100000000000001},"min":{"1min":0.71599999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3260000000000001,"5min":1.498,"15min":1.498},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.113,"15min":1.1100000000000001},"min":{"1min":0.89700000000000002,"5min":0.49399999999999999,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.294}]}]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198944886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94899999999999995,"15min":0.95499999999999996},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.196,"5min":1.381,"15min":1.579},"last":1.264},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.99399999999999999,"15min":0.96799999999999997},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.3400000000000001,"5min":1.591,"15min":1.591},"last":0.92000000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.119,"5min":1.206,"15min":2.1150000000000002},"last":1.214},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88,"15min":0.86299999999999999},"min":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.52400000000000002},"max":{"1min":1.179,"5min":1.272,"15min":1.464},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93400000000000005,"15min":0.91700000000000004},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.204,"5min":1.569,"15min":1.569},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.90500000000000003,"15min":0.92200000000000004},"min":{"1min":0.68200000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":1.117}]},{"osd":3,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.323,"5min":1.448,"15min":1.548},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.99199999999999999,"15min":0.997},"min":{"1min":0.81399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.395,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.335}]},{"osd":4,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.72499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.236,"5min":1.286,"15min":2.1909999999999998},"last":1.373},{"interface":"front","average":{"1min":1.147,"5min":1.0860000000000001,"15min":1.093},"min":{"1min":0.90500000000000003,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.357,"5min":1.536,"15min":1.536},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.008,"15min":1.0089999999999999},"min":{"1min":0.60899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4590000000000001,"5min":1.744,"15min":1.744},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0509999999999999,"15min":1.0349999999999999},"min":{"1min":0.77600000000000002,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.3620000000000001,"5min":1.4199999999999999,"15min":1.4950000000000001},"last":1.008}]},{"osd":7,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.0860000000000001,"15min":1.077},"min":{"1min":0.99299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.452,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.05},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1599999999999999,"15min":1.157},"min":{"1min":0.91400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.444,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.077}]}]},{"osd":4,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1037,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937508,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.90000000000000002,"15min":0.87},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.131,"5min":1.248,"15min":1.347},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91700000000000004,"15min":0.89000000000000001},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.234,"5min":1.2909999999999999,"15min":1.502},"last":0.59799999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89100000000000001,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.1719999999999999,"5min":1.21,"15min":1.375},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97099999999999997,"15min":0.92400000000000004},"min":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.387,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92500000000000004,"15min":0.85699999999999998},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.143,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90700000000000003,"15min":0.88300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.373},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97699999999999998,"15min":0.94999999999999996},"min":{"1min":0.60199999999999998,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.248,"5min":1.397,"15min":2.0190000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.006,"5min":0.997,"15min":0.96699999999999997},"min":{"1min":0.64400000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2210000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.026}]},{"osd":5,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.88800000000000001,"15min":0.89000000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":0.999},"min":{"1min":0.67800000000000005,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.673},"last":1.089}]},{"osd":6,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.034,"15min":1.016},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.349,"5min":1.762,"15min":1.7969999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.67700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2629999999999999,"5min":1.355,"15min":1.4970000000000001},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.1000000000000001,"15min":1.0620000000000001},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.405,"5min":1.659,"15min":1.7509999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.133,"15min":1.123},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2849999999999999,"5min":1.484,"15min":1.728},"last":1.1359999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1035,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194276,"statfs":{"total":200000000,"available":198939156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.385},"last":0.376},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.623,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":1.0600000000000001,"5min":1.115,"15min":1.2689999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.66000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":6.109,"5min":6.109,"15min":6.109},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.36099999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.018,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.442}]},{"osd":2,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.81599999999999995,"15min":0.77400000000000002},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.30399999999999999},"max":{"1min":6.1710000000000003,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":0.877,"15min":0.81100000000000005},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":6.0389999999999997,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.873,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.181,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.90700000000000003,"15min":0.92000000000000004},"min":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.429,"15min":1.4810000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.61699999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.454,"5min":1.454,"15min":1.478},"last":0.752},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.93899999999999995,"15min":0.92200000000000004},"min":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.1970000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0049999999999999,"15min":0.99199999999999999},"min":{"1min":0.63,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.113,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.98099999999999998,"15min":0.94999999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.4159999999999999,"5min":2.016,"15min":2.016},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.27,"5min":1.054,"15min":1.0469999999999999},"min":{"1min":0.65900000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.7300000000000004,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.85299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1029,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198945400,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63600000000000001,"15min":0.57399999999999995},"min":{"1min":0.374,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.99399999999999999,"5min":1.097,"15min":1.097},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.64700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.157,"5min":1.379,"15min":1.5249999999999999},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.70099999999999996},"min":{"1min":0.36199999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.96699999999999997,"5min":1.198,"15min":1.2609999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":1.3600000000000001},"last":0.501}]},{"osd":3,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.77200000000000002},"min":{"1min":0.42499999999999999,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.1240000000000001,"5min":1.24,"15min":2.8809999999999998},"last":0.435},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.64600000000000002},"min":{"1min":0.32500000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.88100000000000001,"5min":1.125,"15min":1.3440000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1439999999999999,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.879,"15min":0.86499999999999999},"min":{"1min":0.68600000000000005,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.099,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98799999999999999,"15min":0.94599999999999995},"min":{"1min":0.66800000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5289999999999999,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97599999999999998,"15min":0.93500000000000005},"min":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.4790000000000001,"15min":1.653},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.99399999999999999,"15min":0.95399999999999996},"min":{"1min":0.64200000000000002,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.073,"15min":1.024},"min":{"1min":0.68300000000000005,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.486,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1739999999999999,"5min":1.581,"15min":1.581},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":1.077},"min":{"1min":0.65600000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.411,"5min":1.823,"15min":15.352},"last":0.78700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1035,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194276,"statfs":{"total":200000000,"available":198939156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.78100000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66900000000000004,"15min":0.64100000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.94599999999999995,"5min":1.089,"15min":1.157},"last":0.72799999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79500000000000004,"15min":0.749},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.8169999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0820000000000001,"5min":1.147,"15min":1.7949999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.38400000000000001,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.004,"5min":1.175,"15min":1.298},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.88500000000000001,"15min":0.877},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.157,"5min":1.238,"15min":1.5780000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95899999999999996,"15min":0.95599999999999996},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":1.1719999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96999999999999997,"15min":0.96499999999999997},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.492},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.353,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.997,"15min":0.97999999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.393,"5min":1.393,"15min":1.837},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.429},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.048,"15min":1.0780000000000001},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4710000000000001,"5min":1.4890000000000001,"15min":5.2430000000000003},"last":0.92800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574985,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1606,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193706,"statfs":{"total":200000000,"available":198355122,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.76200000000000001,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70299999999999996,"15min":0.64100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.90100000000000002,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.042}]},{"osd":2,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.72199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":1.073,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.90600000000000003,"5min":1.1020000000000001,"15min":1.177},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68100000000000005,"15min":0.64300000000000002},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0149999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.80500000000000005,"15min":0.73099999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.048,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91000000000000003,"15min":0.876},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.0880000000000001,"5min":1.395,"15min":1.395},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0029999999999999,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1870000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91600000000000004,"15min":0.92100000000000004},"min":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.073,"5min":1.2430000000000001,"15min":2.7480000000000002},"last":1.006},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.92500000000000004},"min":{"1min":0.63800000000000001,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1579999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.92600000000000005,"15min":0.91800000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.5640000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.121,"15min":1.0209999999999999},"min":{"1min":0.59999999999999998,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.167,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.98899999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.194,"5min":1.4399999999999999,"15min":1.6990000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.23,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":0.86199999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:58:25.717 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:58:25.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- 172.21.15.71:0/2755099747 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc980b2a60 msgr2=0x7efc980caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 --2- 172.21.15.71:0/2755099747 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc980b2a60 0x7efc980caad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efc88009900 tx=0x7efc8801c8c0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- 172.21.15.71:0/2755099747 shutdown_connections 2023-10-26T05:58:25.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 --2- 172.21.15.71:0/2755099747 >> v2:172.21.15.86:3300/0 conn(0x7efc980cb010 0x7efc98159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 --2- 172.21.15.71:0/2755099747 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc980b2a60 0x7efc980caad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- 172.21.15.71:0/2755099747 >> 172.21.15.71:0/2755099747 conn(0x7efc98069770 msgr2=0x7efc98069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:25.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- 172.21.15.71:0/2755099747 shutdown_connections 2023-10-26T05:58:25.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- 172.21.15.71:0/2755099747 wait complete. 2023-10-26T05:58:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 Processor -- start 2023-10-26T05:58:25.883 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- start start 2023-10-26T05:58:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980b2a60 0x7efc981531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 0x7efc98130380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc9813c2f0 0x7efc981308c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc98137de0 con 0x7efc980cb010 2023-10-26T05:58:25.884 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc98131200 con 0x7efc980b2a60 2023-10-26T05:58:25.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc9ed65640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc98137260 con 0x7efc9813c2f0 2023-10-26T05:58:25.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc97fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 0x7efc98130380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc97fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 0x7efc98130380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46986/0 (socket says 172.21.15.71:46986) 2023-10-26T05:58:25.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.871+0000 7efc97fff640 1 -- 172.21.15.71:0/3221609628 learned_addr learned my addr 172.21.15.71:0/3221609628 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:25.885 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9d302640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc9813c2f0 0x7efc981308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9cb01640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980b2a60 0x7efc981531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc97fff640 1 -- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc9813c2f0 msgr2=0x7efc981308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc97fff640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc9813c2f0 0x7efc981308c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc97fff640 1 -- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980b2a60 msgr2=0x7efc981531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:25.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc97fff640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980b2a60 0x7efc981531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:25.886 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc97fff640 1 -- 172.21.15.71:0/3221609628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc90009820 con 0x7efc980cb010 2023-10-26T05:58:25.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc97fff640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 0x7efc98130380 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7efc8801a4f0 tx=0x7efc880a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:25.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88004240 con 0x7efc980cb010 2023-10-26T05:58:25.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc88004490 con 0x7efc980cb010 2023-10-26T05:58:25.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc88009560 con 0x7efc980cb010 2023-10-26T05:58:25.887 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88005280 con 0x7efc980cb010 2023-10-26T05:58:25.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc98137820 con 0x7efc980cb010 2023-10-26T05:58:25.888 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc98066b10 con 0x7efc980cb010 2023-10-26T05:58:25.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc880b4040 con 0x7efc980cb010 2023-10-26T05:58:25.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc957fa640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc84066470 0x7efc84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:25.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc8811d8d0 con 0x7efc980cb010 2023-10-26T05:58:25.889 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9cb01640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc84066470 0x7efc84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:25.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.875+0000 7efc9cb01640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc84066470 0x7efc84068930 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efc8c001770 tx=0x7efc8c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:25.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:25.879+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc880eade0 con 0x7efc980cb010 2023-10-26T05:58:26.037 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.027+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc9812e380 con 0x7efc84066470 2023-10-26T05:58:26.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.051+0000 7efc957fa640 1 -- 172.21.15.71:0/3221609628 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283884 (secure 0 0 0) 0x7efc9812e380 con 0x7efc84066470 2023-10-26T05:58:26.061 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:26.063 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:58:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc84066470 msgr2=0x7efc84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc84066470 0x7efc84068930 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7efc8c001770 tx=0x7efc8c009040 comp rx=0 tx=0).stop 2023-10-26T05:58:26.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 msgr2=0x7efc98130380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 0x7efc98130380 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7efc8801a4f0 tx=0x7efc880a5690 comp rx=0 tx=0).stop 2023-10-26T05:58:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 shutdown_connections 2023-10-26T05:58:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc9813c2f0 0x7efc981308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc84066470 0x7efc84068930 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc980cb010 0x7efc98130380 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 --2- 172.21.15.71:0/3221609628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980b2a60 0x7efc981531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 >> 172.21.15.71:0/3221609628 conn(0x7efc98069770 msgr2=0x7efc98071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 shutdown_connections 2023-10-26T05:58:26.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.055+0000 7efc9ed65640 1 -- 172.21.15.71:0/3221609628 wait complete. 2023-10-26T05:58:26.079 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2023-10-26T05:58:25.635709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":73,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9414,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553085,"statfs":{"total":1600000000,"available":1590359464,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.997241"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.340628+0000","last_change":"2023-10-26T05:58:22.340449+0000","last_active":"2023-10-26T05:58:22.340628+0000","last_peered":"2023-10-26T05:58:22.340628+0000","last_clean":"2023-10-26T05:58:22.340628+0000","last_became_active":"2023-10-26T05:58:22.339601+0000","last_became_peered":"2023-10-26T05:58:22.339601+0000","last_unstale":"2023-10-26T05:58:22.340628+0000","last_undegraded":"2023-10-26T05:58:22.340628+0000","last_fullsized":"2023-10-26T05:58:22.340628+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:48:29.332958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.356232+0000","last_change":"2023-10-26T05:58:22.356083+0000","last_active":"2023-10-26T05:58:22.356232+0000","last_peered":"2023-10-26T05:58:22.356232+0000","last_clean":"2023-10-26T05:58:22.356232+0000","last_became_active":"2023-10-26T05:58:22.355404+0000","last_became_peered":"2023-10-26T05:58:22.355404+0000","last_unstale":"2023-10-26T05:58:22.356232+0000","last_undegraded":"2023-10-26T05:58:22.356232+0000","last_fullsized":"2023-10-26T05:58:22.356232+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:54:00.413413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.316128+0000","last_change":"2023-10-26T05:58:22.344429+0000","last_active":"2023-10-26T05:58:24.316128+0000","last_peered":"2023-10-26T05:58:24.316128+0000","last_clean":"2023-10-26T05:58:24.316128+0000","last_became_active":"2023-10-26T05:58:22.343512+0000","last_became_peered":"2023-10-26T05:58:22.343512+0000","last_unstale":"2023-10-26T05:58:24.316128+0000","last_undegraded":"2023-10-26T05:58:24.316128+0000","last_fullsized":"2023-10-26T05:58:24.316128+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:08:56.697268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311108+0000","last_change":"2023-10-26T05:58:22.327278+0000","last_active":"2023-10-26T05:58:23.311108+0000","last_peered":"2023-10-26T05:58:23.311108+0000","last_clean":"2023-10-26T05:58:23.311108+0000","last_became_active":"2023-10-26T05:58:22.325660+0000","last_became_peered":"2023-10-26T05:58:22.325660+0000","last_unstale":"2023-10-26T05:58:23.311108+0000","last_undegraded":"2023-10-26T05:58:23.311108+0000","last_fullsized":"2023-10-26T05:58:23.311108+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:13:34.729841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354660+0000","last_change":"2023-10-26T05:58:22.349160+0000","last_active":"2023-10-26T05:58:22.354660+0000","last_peered":"2023-10-26T05:58:22.354660+0000","last_clean":"2023-10-26T05:58:22.354660+0000","last_became_active":"2023-10-26T05:58:22.347926+0000","last_became_peered":"2023-10-26T05:58:22.347926+0000","last_unstale":"2023-10-26T05:58:22.354660+0000","last_undegraded":"2023-10-26T05:58:22.354660+0000","last_fullsized":"2023-10-26T05:58:22.354660+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:27.056484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339648+0000","last_change":"2023-10-26T05:58:22.336063+0000","last_active":"2023-10-26T05:58:22.339648+0000","last_peered":"2023-10-26T05:58:22.339648+0000","last_clean":"2023-10-26T05:58:22.339648+0000","last_became_active":"2023-10-26T05:58:22.334975+0000","last_became_peered":"2023-10-26T05:58:22.334975+0000","last_unstale":"2023-10-26T05:58:22.339648+0000","last_undegraded":"2023-10-26T05:58:22.339648+0000","last_fullsized":"2023-10-26T05:58:22.339648+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:13:05.798695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.335035+0000","last_change":"2023-10-26T05:58:22.334855+0000","last_active":"2023-10-26T05:58:22.335035+0000","last_peered":"2023-10-26T05:58:22.335035+0000","last_clean":"2023-10-26T05:58:22.335035+0000","last_became_active":"2023-10-26T05:58:22.334219+0000","last_became_peered":"2023-10-26T05:58:22.334219+0000","last_unstale":"2023-10-26T05:58:22.335035+0000","last_undegraded":"2023-10-26T05:58:22.335035+0000","last_fullsized":"2023-10-26T05:58:22.335035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:05:39.737968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346519+0000","last_change":"2023-10-26T05:58:22.345771+0000","last_active":"2023-10-26T05:58:22.346519+0000","last_peered":"2023-10-26T05:58:22.346519+0000","last_clean":"2023-10-26T05:58:22.346519+0000","last_became_active":"2023-10-26T05:58:22.345135+0000","last_became_peered":"2023-10-26T05:58:22.345135+0000","last_unstale":"2023-10-26T05:58:22.346519+0000","last_undegraded":"2023-10-26T05:58:22.346519+0000","last_fullsized":"2023-10-26T05:58:22.346519+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:42:01.516098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354709+0000","last_change":"2023-10-26T05:58:22.349676+0000","last_active":"2023-10-26T05:58:22.354709+0000","last_peered":"2023-10-26T05:58:22.354709+0000","last_clean":"2023-10-26T05:58:22.354709+0000","last_became_active":"2023-10-26T05:58:22.348049+0000","last_became_peered":"2023-10-26T05:58:22.348049+0000","last_unstale":"2023-10-26T05:58:22.354709+0000","last_undegraded":"2023-10-26T05:58:22.354709+0000","last_fullsized":"2023-10-26T05:58:22.354709+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:42:23.374903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354609+0000","last_change":"2023-10-26T05:58:22.348701+0000","last_active":"2023-10-26T05:58:22.354609+0000","last_peered":"2023-10-26T05:58:22.354609+0000","last_clean":"2023-10-26T05:58:22.354609+0000","last_became_active":"2023-10-26T05:58:22.347122+0000","last_became_peered":"2023-10-26T05:58:22.347122+0000","last_unstale":"2023-10-26T05:58:22.354609+0000","last_undegraded":"2023-10-26T05:58:22.354609+0000","last_fullsized":"2023-10-26T05:58:22.354609+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:48:17.361064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346467+0000","last_change":"2023-10-26T05:58:22.345833+0000","last_active":"2023-10-26T05:58:22.346467+0000","last_peered":"2023-10-26T05:58:22.346467+0000","last_clean":"2023-10-26T05:58:22.346467+0000","last_became_active":"2023-10-26T05:58:22.345068+0000","last_became_peered":"2023-10-26T05:58:22.345068+0000","last_unstale":"2023-10-26T05:58:22.346467+0000","last_undegraded":"2023-10-26T05:58:22.346467+0000","last_fullsized":"2023-10-26T05:58:22.346467+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:28:19.284971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346563+0000","last_change":"2023-10-26T05:58:22.346152+0000","last_active":"2023-10-26T05:58:22.346563+0000","last_peered":"2023-10-26T05:58:22.346563+0000","last_clean":"2023-10-26T05:58:22.346563+0000","last_became_active":"2023-10-26T05:58:22.345448+0000","last_became_peered":"2023-10-26T05:58:22.345448+0000","last_unstale":"2023-10-26T05:58:22.346563+0000","last_undegraded":"2023-10-26T05:58:22.346563+0000","last_fullsized":"2023-10-26T05:58:22.346563+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:44:36.632867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315282+0000","last_change":"2023-10-26T05:58:22.345674+0000","last_active":"2023-10-26T05:58:24.315282+0000","last_peered":"2023-10-26T05:58:24.315282+0000","last_clean":"2023-10-26T05:58:24.315282+0000","last_became_active":"2023-10-26T05:58:22.345021+0000","last_became_peered":"2023-10-26T05:58:22.345021+0000","last_unstale":"2023-10-26T05:58:24.315282+0000","last_undegraded":"2023-10-26T05:58:24.315282+0000","last_fullsized":"2023-10-26T05:58:24.315282+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:17:14.416563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315123+0000","last_change":"2023-10-26T05:58:22.334920+0000","last_active":"2023-10-26T05:58:24.315123+0000","last_peered":"2023-10-26T05:58:24.315123+0000","last_clean":"2023-10-26T05:58:24.315123+0000","last_became_active":"2023-10-26T05:58:22.334242+0000","last_became_peered":"2023-10-26T05:58:22.334242+0000","last_unstale":"2023-10-26T05:58:24.315123+0000","last_undegraded":"2023-10-26T05:58:24.315123+0000","last_fullsized":"2023-10-26T05:58:24.315123+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:46:52.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.346082+0000","last_change":"2023-10-26T05:58:22.345914+0000","last_active":"2023-10-26T05:58:22.346082+0000","last_peered":"2023-10-26T05:58:22.346082+0000","last_clean":"2023-10-26T05:58:22.346082+0000","last_became_active":"2023-10-26T05:58:22.345265+0000","last_became_peered":"2023-10-26T05:58:22.345265+0000","last_unstale":"2023-10-26T05:58:22.346082+0000","last_undegraded":"2023-10-26T05:58:22.346082+0000","last_fullsized":"2023-10-26T05:58:22.346082+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:48:36.639748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.333307+0000","last_change":"2023-10-26T05:58:22.333116+0000","last_active":"2023-10-26T05:58:22.333307+0000","last_peered":"2023-10-26T05:58:22.333307+0000","last_clean":"2023-10-26T05:58:22.333307+0000","last_became_active":"2023-10-26T05:58:22.332079+0000","last_became_peered":"2023-10-26T05:58:22.332079+0000","last_unstale":"2023-10-26T05:58:22.333307+0000","last_undegraded":"2023-10-26T05:58:22.333307+0000","last_fullsized":"2023-10-26T05:58:22.333307+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:09.897677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.341240+0000","last_change":"2023-10-26T05:58:22.340945+0000","last_active":"2023-10-26T05:58:22.341240+0000","last_peered":"2023-10-26T05:58:22.341240+0000","last_clean":"2023-10-26T05:58:22.341240+0000","last_became_active":"2023-10-26T05:58:22.339985+0000","last_became_peered":"2023-10-26T05:58:22.339985+0000","last_unstale":"2023-10-26T05:58:22.341240+0000","last_undegraded":"2023-10-26T05:58:22.341240+0000","last_fullsized":"2023-10-26T05:58:22.341240+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:08:19.030437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313106+0000","last_change":"2023-10-26T05:58:22.340074+0000","last_active":"2023-10-26T05:58:23.313106+0000","last_peered":"2023-10-26T05:58:23.313106+0000","last_clean":"2023-10-26T05:58:23.313106+0000","last_became_active":"2023-10-26T05:58:22.339192+0000","last_became_peered":"2023-10-26T05:58:22.339192+0000","last_unstale":"2023-10-26T05:58:23.313106+0000","last_undegraded":"2023-10-26T05:58:23.313106+0000","last_fullsized":"2023-10-26T05:58:23.313106+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:52:56.973845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339496+0000","last_change":"2023-10-26T05:58:22.332034+0000","last_active":"2023-10-26T05:58:22.339496+0000","last_peered":"2023-10-26T05:58:22.339496+0000","last_clean":"2023-10-26T05:58:22.339496+0000","last_became_active":"2023-10-26T05:58:22.331398+0000","last_became_peered":"2023-10-26T05:58:22.331398+0000","last_unstale":"2023-10-26T05:58:22.339496+0000","last_undegraded":"2023-10-26T05:58:22.339496+0000","last_fullsized":"2023-10-26T05:58:22.339496+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:42:17.722074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315145+0000","last_change":"2023-10-26T05:58:22.350736+0000","last_active":"2023-10-26T05:58:24.315145+0000","last_peered":"2023-10-26T05:58:24.315145+0000","last_clean":"2023-10-26T05:58:24.315145+0000","last_became_active":"2023-10-26T05:58:22.342396+0000","last_became_peered":"2023-10-26T05:58:22.342396+0000","last_unstale":"2023-10-26T05:58:24.315145+0000","last_undegraded":"2023-10-26T05:58:24.315145+0000","last_fullsized":"2023-10-26T05:58:24.315145+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:14:34.235783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339709+0000","last_change":"2023-10-26T05:58:22.337768+0000","last_active":"2023-10-26T05:58:22.339709+0000","last_peered":"2023-10-26T05:58:22.339709+0000","last_clean":"2023-10-26T05:58:22.339709+0000","last_became_active":"2023-10-26T05:58:22.337144+0000","last_became_peered":"2023-10-26T05:58:22.337144+0000","last_unstale":"2023-10-26T05:58:22.339709+0000","last_undegraded":"2023-10-26T05:58:22.339709+0000","last_fullsized":"2023-10-26T05:58:22.339709+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:30:48.492831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.334236+0000","last_change":"2023-10-26T05:58:22.333954+0000","last_active":"2023-10-26T05:58:22.334236+0000","last_peered":"2023-10-26T05:58:22.334236+0000","last_clean":"2023-10-26T05:58:22.334236+0000","last_became_active":"2023-10-26T05:58:22.333132+0000","last_became_peered":"2023-10-26T05:58:22.333132+0000","last_unstale":"2023-10-26T05:58:22.334236+0000","last_undegraded":"2023-10-26T05:58:22.334236+0000","last_fullsized":"2023-10-26T05:58:22.334236+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:02:51.788771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.341594+0000","last_change":"2023-10-26T05:58:22.341398+0000","last_active":"2023-10-26T05:58:22.341594+0000","last_peered":"2023-10-26T05:58:22.341594+0000","last_clean":"2023-10-26T05:58:22.341594+0000","last_became_active":"2023-10-26T05:58:22.340692+0000","last_became_peered":"2023-10-26T05:58:22.340692+0000","last_unstale":"2023-10-26T05:58:22.341594+0000","last_undegraded":"2023-10-26T05:58:22.341594+0000","last_fullsized":"2023-10-26T05:58:22.341594+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:40:23.255743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.354752+0000","last_change":"2023-10-26T05:58:22.354588+0000","last_active":"2023-10-26T05:58:22.354752+0000","last_peered":"2023-10-26T05:58:22.354752+0000","last_clean":"2023-10-26T05:58:22.354752+0000","last_became_active":"2023-10-26T05:58:22.349082+0000","last_became_peered":"2023-10-26T05:58:22.349082+0000","last_unstale":"2023-10-26T05:58:22.354752+0000","last_undegraded":"2023-10-26T05:58:22.354752+0000","last_fullsized":"2023-10-26T05:58:22.354752+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:28:56.725878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315350+0000","last_change":"2023-10-26T05:58:22.354487+0000","last_active":"2023-10-26T05:58:24.315350+0000","last_peered":"2023-10-26T05:58:24.315350+0000","last_clean":"2023-10-26T05:58:24.315350+0000","last_became_active":"2023-10-26T05:58:22.352585+0000","last_became_peered":"2023-10-26T05:58:22.352585+0000","last_unstale":"2023-10-26T05:58:24.315350+0000","last_undegraded":"2023-10-26T05:58:24.315350+0000","last_fullsized":"2023-10-26T05:58:24.315350+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:56:49.677051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.328634+0000","last_change":"2023-10-26T05:58:22.328482+0000","last_active":"2023-10-26T05:58:22.328634+0000","last_peered":"2023-10-26T05:58:22.328634+0000","last_clean":"2023-10-26T05:58:22.328634+0000","last_became_active":"2023-10-26T05:58:22.327511+0000","last_became_peered":"2023-10-26T05:58:22.327511+0000","last_unstale":"2023-10-26T05:58:22.328634+0000","last_undegraded":"2023-10-26T05:58:22.328634+0000","last_fullsized":"2023-10-26T05:58:22.328634+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:04:14.075269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315020+0000","last_change":"2023-10-26T05:58:22.337961+0000","last_active":"2023-10-26T05:58:24.315020+0000","last_peered":"2023-10-26T05:58:24.315020+0000","last_clean":"2023-10-26T05:58:24.315020+0000","last_became_active":"2023-10-26T05:58:22.337347+0000","last_became_peered":"2023-10-26T05:58:22.337347+0000","last_unstale":"2023-10-26T05:58:24.315020+0000","last_undegraded":"2023-10-26T05:58:24.315020+0000","last_fullsized":"2023-10-26T05:58:24.315020+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:17:44.061654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310484+0000","last_change":"2023-10-26T05:58:22.316896+0000","last_active":"2023-10-26T05:58:23.310484+0000","last_peered":"2023-10-26T05:58:23.310484+0000","last_clean":"2023-10-26T05:58:23.310484+0000","last_became_active":"2023-10-26T05:58:22.316290+0000","last_became_peered":"2023-10-26T05:58:22.316290+0000","last_unstale":"2023-10-26T05:58:23.310484+0000","last_undegraded":"2023-10-26T05:58:23.310484+0000","last_fullsized":"2023-10-26T05:58:23.310484+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:47:31.198902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315462+0000","last_change":"2023-10-26T05:58:22.351075+0000","last_active":"2023-10-26T05:58:24.315462+0000","last_peered":"2023-10-26T05:58:24.315462+0000","last_clean":"2023-10-26T05:58:24.315462+0000","last_became_active":"2023-10-26T05:58:22.346285+0000","last_became_peered":"2023-10-26T05:58:22.346285+0000","last_unstale":"2023-10-26T05:58:24.315462+0000","last_undegraded":"2023-10-26T05:58:24.315462+0000","last_fullsized":"2023-10-26T05:58:24.315462+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:01:12.184966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320456+0000","last_change":"2023-10-26T05:58:22.333395+0000","last_active":"2023-10-26T05:58:23.320456+0000","last_peered":"2023-10-26T05:58:23.320456+0000","last_clean":"2023-10-26T05:58:23.320456+0000","last_became_active":"2023-10-26T05:58:22.332776+0000","last_became_peered":"2023-10-26T05:58:22.332776+0000","last_unstale":"2023-10-26T05:58:23.320456+0000","last_undegraded":"2023-10-26T05:58:23.320456+0000","last_fullsized":"2023-10-26T05:58:23.320456+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:52:56.973845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320341+0000","last_change":"2023-10-26T05:58:22.341366+0000","last_active":"2023-10-26T05:58:23.320341+0000","last_peered":"2023-10-26T05:58:23.320341+0000","last_clean":"2023-10-26T05:58:23.320341+0000","last_became_active":"2023-10-26T05:58:22.339141+0000","last_became_peered":"2023-10-26T05:58:22.339141+0000","last_unstale":"2023-10-26T05:58:23.320341+0000","last_undegraded":"2023-10-26T05:58:23.320341+0000","last_fullsized":"2023-10-26T05:58:23.320341+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:48.591623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313073+0000","last_change":"2023-10-26T05:58:22.344844+0000","last_active":"2023-10-26T05:58:23.313073+0000","last_peered":"2023-10-26T05:58:23.313073+0000","last_clean":"2023-10-26T05:58:23.313073+0000","last_became_active":"2023-10-26T05:58:22.343803+0000","last_became_peered":"2023-10-26T05:58:22.343803+0000","last_unstale":"2023-10-26T05:58:23.313073+0000","last_undegraded":"2023-10-26T05:58:23.313073+0000","last_fullsized":"2023-10-26T05:58:23.313073+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:49:48.591623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.341626+0000","last_change":"2023-10-26T05:58:22.341431+0000","last_active":"2023-10-26T05:58:22.341626+0000","last_peered":"2023-10-26T05:58:22.341626+0000","last_clean":"2023-10-26T05:58:22.341626+0000","last_became_active":"2023-10-26T05:58:22.340363+0000","last_became_peered":"2023-10-26T05:58:22.340363+0000","last_unstale":"2023-10-26T05:58:22.341626+0000","last_undegraded":"2023-10-26T05:58:22.341626+0000","last_fullsized":"2023-10-26T05:58:22.341626+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:54.644310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"187'4","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.721064+0000","last_change":"2023-10-26T05:58:21.694753+0000","last_active":"2023-10-26T05:58:22.721064+0000","last_peered":"2023-10-26T05:58:22.721064+0000","last_clean":"2023-10-26T05:58:22.721064+0000","last_became_active":"2023-10-26T05:58:21.694236+0000","last_became_peered":"2023-10-26T05:58:21.694236+0000","last_unstale":"2023-10-26T05:58:22.721064+0000","last_undegraded":"2023-10-26T05:58:22.721064+0000","last_fullsized":"2023-10-26T05:58:22.721064+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:26:06.106664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320310+0000","last_change":"2023-10-26T05:58:22.324197+0000","last_active":"2023-10-26T05:58:23.320310+0000","last_peered":"2023-10-26T05:58:23.320310+0000","last_clean":"2023-10-26T05:58:23.320310+0000","last_became_active":"2023-10-26T05:58:22.323458+0000","last_became_peered":"2023-10-26T05:58:22.323458+0000","last_unstale":"2023-10-26T05:58:23.320310+0000","last_undegraded":"2023-10-26T05:58:23.320310+0000","last_fullsized":"2023-10-26T05:58:23.320310+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:15.325100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.319760+0000","last_change":"2023-10-26T05:58:21.691196+0000","last_active":"2023-10-26T05:58:23.319760+0000","last_peered":"2023-10-26T05:58:23.319760+0000","last_clean":"2023-10-26T05:58:23.319760+0000","last_became_active":"2023-10-26T05:58:21.686744+0000","last_became_peered":"2023-10-26T05:58:21.686744+0000","last_unstale":"2023-10-26T05:58:23.319760+0000","last_undegraded":"2023-10-26T05:58:23.319760+0000","last_fullsized":"2023-10-26T05:58:23.319760+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:49:49.331522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339607+0000","last_change":"2023-10-26T05:58:22.335576+0000","last_active":"2023-10-26T05:58:22.339607+0000","last_peered":"2023-10-26T05:58:22.339607+0000","last_clean":"2023-10-26T05:58:22.339607+0000","last_became_active":"2023-10-26T05:58:22.334072+0000","last_became_peered":"2023-10-26T05:58:22.334072+0000","last_unstale":"2023-10-26T05:58:22.339607+0000","last_undegraded":"2023-10-26T05:58:22.339607+0000","last_fullsized":"2023-10-26T05:58:22.339607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T09:26:59.854107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311781+0000","last_change":"2023-10-26T05:58:21.673643+0000","last_active":"2023-10-26T05:58:23.311781+0000","last_peered":"2023-10-26T05:58:23.311781+0000","last_clean":"2023-10-26T05:58:23.311781+0000","last_became_active":"2023-10-26T05:58:21.671877+0000","last_became_peered":"2023-10-26T05:58:21.671877+0000","last_unstale":"2023-10-26T05:58:23.311781+0000","last_undegraded":"2023-10-26T05:58:23.311781+0000","last_fullsized":"2023-10-26T05:58:23.311781+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:17:26.886007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315551+0000","last_change":"2023-10-26T05:58:22.355252+0000","last_active":"2023-10-26T05:58:24.315551+0000","last_peered":"2023-10-26T05:58:24.315551+0000","last_clean":"2023-10-26T05:58:24.315551+0000","last_became_active":"2023-10-26T05:58:22.353472+0000","last_became_peered":"2023-10-26T05:58:22.353472+0000","last_unstale":"2023-10-26T05:58:24.315551+0000","last_undegraded":"2023-10-26T05:58:24.315551+0000","last_fullsized":"2023-10-26T05:58:24.315551+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:25:00.604256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310838+0000","last_change":"2023-10-26T05:58:21.669865+0000","last_active":"2023-10-26T05:58:23.310838+0000","last_peered":"2023-10-26T05:58:23.310838+0000","last_clean":"2023-10-26T05:58:23.310838+0000","last_became_active":"2023-10-26T05:58:21.669180+0000","last_became_peered":"2023-10-26T05:58:21.669180+0000","last_unstale":"2023-10-26T05:58:23.310838+0000","last_undegraded":"2023-10-26T05:58:23.310838+0000","last_fullsized":"2023-10-26T05:58:23.310838+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:58:42.474934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.326754+0000","last_change":"2023-10-26T05:58:22.326376+0000","last_active":"2023-10-26T05:58:22.326754+0000","last_peered":"2023-10-26T05:58:22.326754+0000","last_clean":"2023-10-26T05:58:22.326754+0000","last_became_active":"2023-10-26T05:58:22.324974+0000","last_became_peered":"2023-10-26T05:58:22.324974+0000","last_unstale":"2023-10-26T05:58:22.326754+0000","last_undegraded":"2023-10-26T05:58:22.326754+0000","last_fullsized":"2023-10-26T05:58:22.326754+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:24.705234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315505+0000","last_change":"2023-10-26T05:58:21.695730+0000","last_active":"2023-10-26T05:58:24.315505+0000","last_peered":"2023-10-26T05:58:24.315505+0000","last_clean":"2023-10-26T05:58:24.315505+0000","last_became_active":"2023-10-26T05:58:21.694072+0000","last_became_peered":"2023-10-26T05:58:21.694072+0000","last_unstale":"2023-10-26T05:58:24.315505+0000","last_undegraded":"2023-10-26T05:58:24.315505+0000","last_fullsized":"2023-10-26T05:58:24.315505+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:01:17.591703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311446+0000","last_change":"2023-10-26T05:58:22.326272+0000","last_active":"2023-10-26T05:58:23.311446+0000","last_peered":"2023-10-26T05:58:23.311446+0000","last_clean":"2023-10-26T05:58:23.311446+0000","last_became_active":"2023-10-26T05:58:22.324085+0000","last_became_peered":"2023-10-26T05:58:22.324085+0000","last_unstale":"2023-10-26T05:58:23.311446+0000","last_undegraded":"2023-10-26T05:58:23.311446+0000","last_fullsized":"2023-10-26T05:58:23.311446+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:54:38.006089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310579+0000","last_change":"2023-10-26T05:58:21.658840+0000","last_active":"2023-10-26T05:58:23.310579+0000","last_peered":"2023-10-26T05:58:23.310579+0000","last_clean":"2023-10-26T05:58:23.310579+0000","last_became_active":"2023-10-26T05:58:21.658544+0000","last_became_peered":"2023-10-26T05:58:21.658544+0000","last_unstale":"2023-10-26T05:58:23.310579+0000","last_undegraded":"2023-10-26T05:58:23.310579+0000","last_fullsized":"2023-10-26T05:58:23.310579+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:27:57.438255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311080+0000","last_change":"2023-10-26T05:58:22.336290+0000","last_active":"2023-10-26T05:58:23.311080+0000","last_peered":"2023-10-26T05:58:23.311080+0000","last_clean":"2023-10-26T05:58:23.311080+0000","last_became_active":"2023-10-26T05:58:22.333625+0000","last_became_peered":"2023-10-26T05:58:22.333625+0000","last_unstale":"2023-10-26T05:58:23.311080+0000","last_undegraded":"2023-10-26T05:58:23.311080+0000","last_fullsized":"2023-10-26T05:58:23.311080+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:19:33.092207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.331026+0000","last_change":"2023-10-26T05:58:21.684041+0000","last_active":"2023-10-26T05:58:22.331026+0000","last_peered":"2023-10-26T05:58:22.331026+0000","last_clean":"2023-10-26T05:58:22.331026+0000","last_became_active":"2023-10-26T05:58:21.682532+0000","last_became_peered":"2023-10-26T05:58:21.682532+0000","last_unstale":"2023-10-26T05:58:22.331026+0000","last_undegraded":"2023-10-26T05:58:22.331026+0000","last_fullsized":"2023-10-26T05:58:22.331026+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:19:51.622783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339564+0000","last_change":"2023-10-26T05:58:22.333799+0000","last_active":"2023-10-26T05:58:22.339564+0000","last_peered":"2023-10-26T05:58:22.339564+0000","last_clean":"2023-10-26T05:58:22.339564+0000","last_became_active":"2023-10-26T05:58:22.333170+0000","last_became_peered":"2023-10-26T05:58:22.333170+0000","last_unstale":"2023-10-26T05:58:22.339564+0000","last_undegraded":"2023-10-26T05:58:22.339564+0000","last_fullsized":"2023-10-26T05:58:22.339564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:20:58.280859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.320739+0000","last_change":"2023-10-26T05:58:21.673169+0000","last_active":"2023-10-26T05:58:22.320739+0000","last_peered":"2023-10-26T05:58:22.320739+0000","last_clean":"2023-10-26T05:58:22.320739+0000","last_became_active":"2023-10-26T05:58:21.672853+0000","last_became_peered":"2023-10-26T05:58:21.672853+0000","last_unstale":"2023-10-26T05:58:22.320739+0000","last_undegraded":"2023-10-26T05:58:22.320739+0000","last_fullsized":"2023-10-26T05:58:22.320739+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:08.794339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313269+0000","last_change":"2023-10-26T05:58:22.318650+0000","last_active":"2023-10-26T05:58:23.313269+0000","last_peered":"2023-10-26T05:58:23.313269+0000","last_clean":"2023-10-26T05:58:23.313269+0000","last_became_active":"2023-10-26T05:58:22.318039+0000","last_became_peered":"2023-10-26T05:58:22.318039+0000","last_unstale":"2023-10-26T05:58:23.313269+0000","last_undegraded":"2023-10-26T05:58:23.313269+0000","last_fullsized":"2023-10-26T05:58:23.313269+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:12.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.332984+0000","last_change":"2023-10-26T05:58:21.679705+0000","last_active":"2023-10-26T05:58:22.332984+0000","last_peered":"2023-10-26T05:58:22.332984+0000","last_clean":"2023-10-26T05:58:22.332984+0000","last_became_active":"2023-10-26T05:58:21.679073+0000","last_became_peered":"2023-10-26T05:58:21.679073+0000","last_unstale":"2023-10-26T05:58:22.332984+0000","last_undegraded":"2023-10-26T05:58:22.332984+0000","last_fullsized":"2023-10-26T05:58:22.332984+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:34:38.017686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312601+0000","last_change":"2023-10-26T05:58:22.329056+0000","last_active":"2023-10-26T05:58:23.312601+0000","last_peered":"2023-10-26T05:58:23.312601+0000","last_clean":"2023-10-26T05:58:23.312601+0000","last_became_active":"2023-10-26T05:58:22.328392+0000","last_became_peered":"2023-10-26T05:58:22.328392+0000","last_unstale":"2023-10-26T05:58:23.312601+0000","last_undegraded":"2023-10-26T05:58:23.312601+0000","last_fullsized":"2023-10-26T05:58:23.312601+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:17.086809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"187'3","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.710907+0000","last_change":"2023-10-26T05:58:21.669862+0000","last_active":"2023-10-26T05:58:22.710907+0000","last_peered":"2023-10-26T05:58:22.710907+0000","last_clean":"2023-10-26T05:58:22.710907+0000","last_became_active":"2023-10-26T05:58:21.669554+0000","last_became_peered":"2023-10-26T05:58:21.669554+0000","last_unstale":"2023-10-26T05:58:22.710907+0000","last_undegraded":"2023-10-26T05:58:22.710907+0000","last_fullsized":"2023-10-26T05:58:22.710907+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:21:33.811150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320502+0000","last_change":"2023-10-26T05:58:22.353971+0000","last_active":"2023-10-26T05:58:23.320502+0000","last_peered":"2023-10-26T05:58:23.320502+0000","last_clean":"2023-10-26T05:58:23.320502+0000","last_became_active":"2023-10-26T05:58:22.351409+0000","last_became_peered":"2023-10-26T05:58:22.351409+0000","last_unstale":"2023-10-26T05:58:23.320502+0000","last_undegraded":"2023-10-26T05:58:23.320502+0000","last_fullsized":"2023-10-26T05:58:23.320502+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:12.980989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"187'6","reported_seq":18,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320001+0000","last_change":"2023-10-26T05:58:21.687392+0000","last_active":"2023-10-26T05:58:23.320001+0000","last_peered":"2023-10-26T05:58:23.320001+0000","last_clean":"2023-10-26T05:58:23.320001+0000","last_became_active":"2023-10-26T05:58:21.685415+0000","last_became_peered":"2023-10-26T05:58:21.685415+0000","last_unstale":"2023-10-26T05:58:23.320001+0000","last_undegraded":"2023-10-26T05:58:23.320001+0000","last_fullsized":"2023-10-26T05:58:23.320001+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:33:53.631378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313487+0000","last_change":"2023-10-26T05:58:22.337374+0000","last_active":"2023-10-26T05:58:23.313487+0000","last_peered":"2023-10-26T05:58:23.313487+0000","last_clean":"2023-10-26T05:58:23.313487+0000","last_became_active":"2023-10-26T05:58:22.336722+0000","last_became_peered":"2023-10-26T05:58:22.336722+0000","last_unstale":"2023-10-26T05:58:23.313487+0000","last_undegraded":"2023-10-26T05:58:23.313487+0000","last_fullsized":"2023-10-26T05:58:23.313487+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:45:46.728645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311998+0000","last_change":"2023-10-26T05:58:21.680120+0000","last_active":"2023-10-26T05:58:23.311998+0000","last_peered":"2023-10-26T05:58:23.311998+0000","last_clean":"2023-10-26T05:58:23.311998+0000","last_became_active":"2023-10-26T05:58:21.674533+0000","last_became_peered":"2023-10-26T05:58:21.674533+0000","last_unstale":"2023-10-26T05:58:23.311998+0000","last_undegraded":"2023-10-26T05:58:23.311998+0000","last_fullsized":"2023-10-26T05:58:23.311998+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:37:05.453161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.338271+0000","last_change":"2023-10-26T05:58:22.338088+0000","last_active":"2023-10-26T05:58:22.338271+0000","last_peered":"2023-10-26T05:58:22.338271+0000","last_clean":"2023-10-26T05:58:22.338271+0000","last_became_active":"2023-10-26T05:58:22.337483+0000","last_became_peered":"2023-10-26T05:58:22.337483+0000","last_unstale":"2023-10-26T05:58:22.338271+0000","last_undegraded":"2023-10-26T05:58:22.338271+0000","last_fullsized":"2023-10-26T05:58:22.338271+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:23:02.083024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313710+0000","last_change":"2023-10-26T05:58:21.676152+0000","last_active":"2023-10-26T05:58:23.313710+0000","last_peered":"2023-10-26T05:58:23.313710+0000","last_clean":"2023-10-26T05:58:23.313710+0000","last_became_active":"2023-10-26T05:58:21.675523+0000","last_became_peered":"2023-10-26T05:58:21.675523+0000","last_unstale":"2023-10-26T05:58:23.313710+0000","last_undegraded":"2023-10-26T05:58:23.313710+0000","last_fullsized":"2023-10-26T05:58:23.313710+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:04:13.062337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310353+0000","last_change":"2023-10-26T05:58:22.336105+0000","last_active":"2023-10-26T05:58:23.310353+0000","last_peered":"2023-10-26T05:58:23.310353+0000","last_clean":"2023-10-26T05:58:23.310353+0000","last_became_active":"2023-10-26T05:58:22.335458+0000","last_became_peered":"2023-10-26T05:58:22.335458+0000","last_unstale":"2023-10-26T05:58:23.310353+0000","last_undegraded":"2023-10-26T05:58:23.310353+0000","last_fullsized":"2023-10-26T05:58:23.310353+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T16:02:59.661655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312109+0000","last_change":"2023-10-26T05:58:21.672239+0000","last_active":"2023-10-26T05:58:23.312109+0000","last_peered":"2023-10-26T05:58:23.312109+0000","last_clean":"2023-10-26T05:58:23.312109+0000","last_became_active":"2023-10-26T05:58:21.671488+0000","last_became_peered":"2023-10-26T05:58:21.671488+0000","last_unstale":"2023-10-26T05:58:23.312109+0000","last_undegraded":"2023-10-26T05:58:23.312109+0000","last_fullsized":"2023-10-26T05:58:23.312109+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:49:44.945084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315221+0000","last_change":"2023-10-26T05:58:22.334849+0000","last_active":"2023-10-26T05:58:24.315221+0000","last_peered":"2023-10-26T05:58:24.315221+0000","last_clean":"2023-10-26T05:58:24.315221+0000","last_became_active":"2023-10-26T05:58:22.333604+0000","last_became_peered":"2023-10-26T05:58:22.333604+0000","last_unstale":"2023-10-26T05:58:24.315221+0000","last_undegraded":"2023-10-26T05:58:24.315221+0000","last_fullsized":"2023-10-26T05:58:24.315221+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:45:42.137159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"187'8","reported_seq":20,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320630+0000","last_change":"2023-10-26T05:58:21.691418+0000","last_active":"2023-10-26T05:58:23.320630+0000","last_peered":"2023-10-26T05:58:23.320630+0000","last_clean":"2023-10-26T05:58:23.320630+0000","last_became_active":"2023-10-26T05:58:21.687603+0000","last_became_peered":"2023-10-26T05:58:21.687603+0000","last_unstale":"2023-10-26T05:58:23.320630+0000","last_undegraded":"2023-10-26T05:58:23.320630+0000","last_fullsized":"2023-10-26T05:58:23.320630+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:08.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311561+0000","last_change":"2023-10-26T05:58:22.334259+0000","last_active":"2023-10-26T05:58:23.311561+0000","last_peered":"2023-10-26T05:58:23.311561+0000","last_clean":"2023-10-26T05:58:23.311561+0000","last_became_active":"2023-10-26T05:58:22.331850+0000","last_became_peered":"2023-10-26T05:58:22.331850+0000","last_unstale":"2023-10-26T05:58:23.311561+0000","last_undegraded":"2023-10-26T05:58:23.311561+0000","last_fullsized":"2023-10-26T05:58:23.311561+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:27:47.387461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"187'1","reported_seq":13,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.316025+0000","last_change":"2023-10-26T05:58:21.683255+0000","last_active":"2023-10-26T05:58:24.316025+0000","last_peered":"2023-10-26T05:58:24.316025+0000","last_clean":"2023-10-26T05:58:24.316025+0000","last_became_active":"2023-10-26T05:58:21.682638+0000","last_became_peered":"2023-10-26T05:58:21.682638+0000","last_unstale":"2023-10-26T05:58:24.316025+0000","last_undegraded":"2023-10-26T05:58:24.316025+0000","last_fullsized":"2023-10-26T05:58:24.316025+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:32:15.290323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312977+0000","last_change":"2023-10-26T05:58:22.327777+0000","last_active":"2023-10-26T05:58:23.312977+0000","last_peered":"2023-10-26T05:58:23.312977+0000","last_clean":"2023-10-26T05:58:23.312977+0000","last_became_active":"2023-10-26T05:58:22.315487+0000","last_became_peered":"2023-10-26T05:58:22.315487+0000","last_unstale":"2023-10-26T05:58:23.312977+0000","last_undegraded":"2023-10-26T05:58:23.312977+0000","last_fullsized":"2023-10-26T05:58:23.312977+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:15.325100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315874+0000","last_change":"2023-10-26T05:58:21.696555+0000","last_active":"2023-10-26T05:58:24.315874+0000","last_peered":"2023-10-26T05:58:24.315874+0000","last_clean":"2023-10-26T05:58:24.315874+0000","last_became_active":"2023-10-26T05:58:21.695560+0000","last_became_peered":"2023-10-26T05:58:21.695560+0000","last_unstale":"2023-10-26T05:58:24.315874+0000","last_undegraded":"2023-10-26T05:58:24.315874+0000","last_fullsized":"2023-10-26T05:58:24.315874+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:29:45.071605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.311922+0000","last_change":"2023-10-26T05:58:22.325785+0000","last_active":"2023-10-26T05:58:23.311922+0000","last_peered":"2023-10-26T05:58:23.311922+0000","last_clean":"2023-10-26T05:58:23.311922+0000","last_became_active":"2023-10-26T05:58:22.323127+0000","last_became_peered":"2023-10-26T05:58:22.323127+0000","last_unstale":"2023-10-26T05:58:23.311922+0000","last_undegraded":"2023-10-26T05:58:23.311922+0000","last_fullsized":"2023-10-26T05:58:23.311922+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:56:36.661796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"187'1","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.720607+0000","last_change":"2023-10-26T05:58:21.692853+0000","last_active":"2023-10-26T05:58:22.720607+0000","last_peered":"2023-10-26T05:58:22.720607+0000","last_clean":"2023-10-26T05:58:22.720607+0000","last_became_active":"2023-10-26T05:58:21.691588+0000","last_became_peered":"2023-10-26T05:58:21.691588+0000","last_unstale":"2023-10-26T05:58:22.720607+0000","last_undegraded":"2023-10-26T05:58:22.720607+0000","last_fullsized":"2023-10-26T05:58:22.720607+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:54:28.073023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.337400+0000","last_change":"2023-10-26T05:58:22.337237+0000","last_active":"2023-10-26T05:58:22.337400+0000","last_peered":"2023-10-26T05:58:22.337400+0000","last_clean":"2023-10-26T05:58:22.337400+0000","last_became_active":"2023-10-26T05:58:22.336034+0000","last_became_peered":"2023-10-26T05:58:22.336034+0000","last_unstale":"2023-10-26T05:58:22.337400+0000","last_undegraded":"2023-10-26T05:58:22.337400+0000","last_fullsized":"2023-10-26T05:58:22.337400+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:06:15.325100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.329826+0000","last_change":"2023-10-26T05:58:21.687342+0000","last_active":"2023-10-26T05:58:22.329826+0000","last_peered":"2023-10-26T05:58:22.329826+0000","last_clean":"2023-10-26T05:58:22.329826+0000","last_became_active":"2023-10-26T05:58:21.686710+0000","last_became_peered":"2023-10-26T05:58:21.686710+0000","last_unstale":"2023-10-26T05:58:22.329826+0000","last_undegraded":"2023-10-26T05:58:22.329826+0000","last_fullsized":"2023-10-26T05:58:22.329826+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:41:10.211119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315794+0000","last_change":"2023-10-26T05:58:22.352535+0000","last_active":"2023-10-26T05:58:24.315794+0000","last_peered":"2023-10-26T05:58:24.315794+0000","last_clean":"2023-10-26T05:58:24.315794+0000","last_became_active":"2023-10-26T05:58:22.350355+0000","last_became_peered":"2023-10-26T05:58:22.350355+0000","last_unstale":"2023-10-26T05:58:24.315794+0000","last_undegraded":"2023-10-26T05:58:24.315794+0000","last_fullsized":"2023-10-26T05:58:24.315794+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:37:57.699270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.310681+0000","last_change":"2023-10-26T05:58:21.668410+0000","last_active":"2023-10-26T05:58:23.310681+0000","last_peered":"2023-10-26T05:58:23.310681+0000","last_clean":"2023-10-26T05:58:23.310681+0000","last_became_active":"2023-10-26T05:58:21.668076+0000","last_became_peered":"2023-10-26T05:58:21.668076+0000","last_unstale":"2023-10-26T05:58:23.310681+0000","last_undegraded":"2023-10-26T05:58:23.310681+0000","last_fullsized":"2023-10-26T05:58:23.310681+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T15:36:19.673528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.327057+0000","last_change":"2023-10-26T05:58:22.326912+0000","last_active":"2023-10-26T05:58:22.327057+0000","last_peered":"2023-10-26T05:58:22.327057+0000","last_clean":"2023-10-26T05:58:22.327057+0000","last_became_active":"2023-10-26T05:58:22.325781+0000","last_became_peered":"2023-10-26T05:58:22.325781+0000","last_unstale":"2023-10-26T05:58:22.327057+0000","last_undegraded":"2023-10-26T05:58:22.327057+0000","last_fullsized":"2023-10-26T05:58:22.327057+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:03:20.402710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.329352+0000","last_change":"2023-10-26T05:58:21.683764+0000","last_active":"2023-10-26T05:58:22.329352+0000","last_peered":"2023-10-26T05:58:22.329352+0000","last_clean":"2023-10-26T05:58:22.329352+0000","last_became_active":"2023-10-26T05:58:21.683122+0000","last_became_peered":"2023-10-26T05:58:21.683122+0000","last_unstale":"2023-10-26T05:58:22.329352+0000","last_undegraded":"2023-10-26T05:58:22.329352+0000","last_fullsized":"2023-10-26T05:58:22.329352+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T07:00:50.534395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.351961+0000","last_change":"2023-10-26T05:58:22.351670+0000","last_active":"2023-10-26T05:58:22.351961+0000","last_peered":"2023-10-26T05:58:22.351961+0000","last_clean":"2023-10-26T05:58:22.351961+0000","last_became_active":"2023-10-26T05:58:22.350976+0000","last_became_peered":"2023-10-26T05:58:22.350976+0000","last_unstale":"2023-10-26T05:58:22.351961+0000","last_undegraded":"2023-10-26T05:58:22.351961+0000","last_fullsized":"2023-10-26T05:58:22.351961+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T11:31:05.893224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312193+0000","last_change":"2023-10-26T05:58:21.690261+0000","last_active":"2023-10-26T05:58:23.312193+0000","last_peered":"2023-10-26T05:58:23.312193+0000","last_clean":"2023-10-26T05:58:23.312193+0000","last_became_active":"2023-10-26T05:58:21.690061+0000","last_became_peered":"2023-10-26T05:58:21.690061+0000","last_unstale":"2023-10-26T05:58:23.312193+0000","last_undegraded":"2023-10-26T05:58:23.312193+0000","last_fullsized":"2023-10-26T05:58:23.312193+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:39:56.009066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.344376+0000","last_change":"2023-10-26T05:58:22.344231+0000","last_active":"2023-10-26T05:58:22.344376+0000","last_peered":"2023-10-26T05:58:22.344376+0000","last_clean":"2023-10-26T05:58:22.344376+0000","last_became_active":"2023-10-26T05:58:22.342817+0000","last_became_peered":"2023-10-26T05:58:22.342817+0000","last_unstale":"2023-10-26T05:58:22.344376+0000","last_undegraded":"2023-10-26T05:58:22.344376+0000","last_fullsized":"2023-10-26T05:58:22.344376+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:44:23.736483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.333862+0000","last_change":"2023-10-26T05:58:21.693224+0000","last_active":"2023-10-26T05:58:22.333862+0000","last_peered":"2023-10-26T05:58:22.333862+0000","last_clean":"2023-10-26T05:58:22.333862+0000","last_became_active":"2023-10-26T05:58:21.692310+0000","last_became_peered":"2023-10-26T05:58:21.692310+0000","last_unstale":"2023-10-26T05:58:22.333862+0000","last_undegraded":"2023-10-26T05:58:22.333862+0000","last_fullsized":"2023-10-26T05:58:22.333862+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:55:45.334233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313443+0000","last_change":"2023-10-26T05:58:22.317439+0000","last_active":"2023-10-26T05:58:23.313443+0000","last_peered":"2023-10-26T05:58:23.313443+0000","last_clean":"2023-10-26T05:58:23.313443+0000","last_became_active":"2023-10-26T05:58:22.316804+0000","last_became_peered":"2023-10-26T05:58:22.316804+0000","last_unstale":"2023-10-26T05:58:23.313443+0000","last_undegraded":"2023-10-26T05:58:23.313443+0000","last_fullsized":"2023-10-26T05:58:23.313443+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:06:48.077805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"187'2","reported_seq":14,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313997+0000","last_change":"2023-10-26T05:58:21.678448+0000","last_active":"2023-10-26T05:58:23.313997+0000","last_peered":"2023-10-26T05:58:23.313997+0000","last_clean":"2023-10-26T05:58:23.313997+0000","last_became_active":"2023-10-26T05:58:21.677332+0000","last_became_peered":"2023-10-26T05:58:21.677332+0000","last_unstale":"2023-10-26T05:58:23.313997+0000","last_undegraded":"2023-10-26T05:58:23.313997+0000","last_fullsized":"2023-10-26T05:58:23.313997+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:10:08.884745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.329978+0000","last_change":"2023-10-26T05:58:22.329814+0000","last_active":"2023-10-26T05:58:22.329978+0000","last_peered":"2023-10-26T05:58:22.329978+0000","last_clean":"2023-10-26T05:58:22.329978+0000","last_became_active":"2023-10-26T05:58:22.328287+0000","last_became_peered":"2023-10-26T05:58:22.328287+0000","last_unstale":"2023-10-26T05:58:22.329978+0000","last_undegraded":"2023-10-26T05:58:22.329978+0000","last_fullsized":"2023-10-26T05:58:22.329978+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:17.086809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.316235+0000","last_change":"2023-10-26T05:58:21.695289+0000","last_active":"2023-10-26T05:58:24.316235+0000","last_peered":"2023-10-26T05:58:24.316235+0000","last_clean":"2023-10-26T05:58:24.316235+0000","last_became_active":"2023-10-26T05:58:21.694747+0000","last_became_peered":"2023-10-26T05:58:21.694747+0000","last_unstale":"2023-10-26T05:58:24.316235+0000","last_undegraded":"2023-10-26T05:58:24.316235+0000","last_fullsized":"2023-10-26T05:58:24.316235+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T08:41:52.348732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.339751+0000","last_change":"2023-10-26T05:58:22.339246+0000","last_active":"2023-10-26T05:58:22.339751+0000","last_peered":"2023-10-26T05:58:22.339751+0000","last_clean":"2023-10-26T05:58:22.339751+0000","last_became_active":"2023-10-26T05:58:22.338545+0000","last_became_peered":"2023-10-26T05:58:22.338545+0000","last_unstale":"2023-10-26T05:58:22.339751+0000","last_undegraded":"2023-10-26T05:58:22.339751+0000","last_fullsized":"2023-10-26T05:58:22.339751+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:35:07.143366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.331385+0000","last_change":"2023-10-26T05:58:21.685710+0000","last_active":"2023-10-26T05:58:22.331385+0000","last_peered":"2023-10-26T05:58:22.331385+0000","last_clean":"2023-10-26T05:58:22.331385+0000","last_became_active":"2023-10-26T05:58:21.685071+0000","last_became_peered":"2023-10-26T05:58:21.685071+0000","last_unstale":"2023-10-26T05:58:22.331385+0000","last_undegraded":"2023-10-26T05:58:22.331385+0000","last_fullsized":"2023-10-26T05:58:22.331385+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:49:33.854076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313294+0000","last_change":"2023-10-26T05:58:22.338035+0000","last_active":"2023-10-26T05:58:23.313294+0000","last_peered":"2023-10-26T05:58:23.313294+0000","last_clean":"2023-10-26T05:58:23.313294+0000","last_became_active":"2023-10-26T05:58:22.336886+0000","last_became_peered":"2023-10-26T05:58:22.336886+0000","last_unstale":"2023-10-26T05:58:23.313294+0000","last_undegraded":"2023-10-26T05:58:23.313294+0000","last_fullsized":"2023-10-26T05:58:23.313294+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:15.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.314045+0000","last_change":"2023-10-26T05:58:21.690003+0000","last_active":"2023-10-26T05:58:23.314045+0000","last_peered":"2023-10-26T05:58:23.314045+0000","last_clean":"2023-10-26T05:58:23.314045+0000","last_became_active":"2023-10-26T05:58:21.689688+0000","last_became_peered":"2023-10-26T05:58:21.689688+0000","last_unstale":"2023-10-26T05:58:23.314045+0000","last_undegraded":"2023-10-26T05:58:23.314045+0000","last_fullsized":"2023-10-26T05:58:23.314045+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T17:03:19.389777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313719+0000","last_change":"2023-10-26T05:58:22.344182+0000","last_active":"2023-10-26T05:58:23.313719+0000","last_peered":"2023-10-26T05:58:23.313719+0000","last_clean":"2023-10-26T05:58:23.313719+0000","last_became_active":"2023-10-26T05:58:22.342504+0000","last_became_peered":"2023-10-26T05:58:22.342504+0000","last_unstale":"2023-10-26T05:58:23.313719+0000","last_undegraded":"2023-10-26T05:58:23.313719+0000","last_fullsized":"2023-10-26T05:58:23.313719+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T06:28:11.957614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320915+0000","last_change":"2023-10-26T05:58:21.691685+0000","last_active":"2023-10-26T05:58:23.320915+0000","last_peered":"2023-10-26T05:58:23.320915+0000","last_clean":"2023-10-26T05:58:23.320915+0000","last_became_active":"2023-10-26T05:58:21.690821+0000","last_became_peered":"2023-10-26T05:58:21.690821+0000","last_unstale":"2023-10-26T05:58:23.320915+0000","last_undegraded":"2023-10-26T05:58:23.320915+0000","last_fullsized":"2023-10-26T05:58:23.320915+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:09:16.073876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320524+0000","last_change":"2023-10-26T05:58:22.340779+0000","last_active":"2023-10-26T05:58:23.320524+0000","last_peered":"2023-10-26T05:58:23.320524+0000","last_clean":"2023-10-26T05:58:23.320524+0000","last_became_active":"2023-10-26T05:58:22.334743+0000","last_became_peered":"2023-10-26T05:58:22.334743+0000","last_unstale":"2023-10-26T05:58:23.320524+0000","last_undegraded":"2023-10-26T05:58:23.320524+0000","last_fullsized":"2023-10-26T05:58:23.320524+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:24:15.721971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.312287+0000","last_change":"2023-10-26T05:58:21.672381+0000","last_active":"2023-10-26T05:58:23.312287+0000","last_peered":"2023-10-26T05:58:23.312287+0000","last_clean":"2023-10-26T05:58:23.312287+0000","last_became_active":"2023-10-26T05:58:21.671920+0000","last_became_peered":"2023-10-26T05:58:21.671920+0000","last_unstale":"2023-10-26T05:58:23.312287+0000","last_undegraded":"2023-10-26T05:58:23.312287+0000","last_fullsized":"2023-10-26T05:58:23.312287+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:26:25.486224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.332227+0000","last_change":"2023-10-26T05:58:22.331143+0000","last_active":"2023-10-26T05:58:22.332227+0000","last_peered":"2023-10-26T05:58:22.332227+0000","last_clean":"2023-10-26T05:58:22.332227+0000","last_became_active":"2023-10-26T05:58:22.330438+0000","last_became_peered":"2023-10-26T05:58:22.330438+0000","last_unstale":"2023-10-26T05:58:22.332227+0000","last_undegraded":"2023-10-26T05:58:22.332227+0000","last_fullsized":"2023-10-26T05:58:22.332227+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:46:06.721462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-26T05:58:22.328542+0000","last_change":"2023-10-26T05:58:21.677323+0000","last_active":"2023-10-26T05:58:22.328542+0000","last_peered":"2023-10-26T05:58:22.328542+0000","last_clean":"2023-10-26T05:58:22.328542+0000","last_became_active":"2023-10-26T05:58:21.676385+0000","last_became_peered":"2023-10-26T05:58:21.676385+0000","last_unstale":"2023-10-26T05:58:22.328542+0000","last_undegraded":"2023-10-26T05:58:22.328542+0000","last_fullsized":"2023-10-26T05:58:22.328542+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:32:47.746497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:24.315625+0000","last_change":"2023-10-26T05:58:22.350309+0000","last_active":"2023-10-26T05:58:24.315625+0000","last_peered":"2023-10-26T05:58:24.315625+0000","last_clean":"2023-10-26T05:58:24.315625+0000","last_became_active":"2023-10-26T05:58:22.349633+0000","last_became_peered":"2023-10-26T05:58:22.349633+0000","last_unstale":"2023-10-26T05:58:24.315625+0000","last_undegraded":"2023-10-26T05:58:24.315625+0000","last_fullsized":"2023-10-26T05:58:24.315625+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T14:19:26.328828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320238+0000","last_change":"2023-10-26T05:58:21.680981+0000","last_active":"2023-10-26T05:58:23.320238+0000","last_peered":"2023-10-26T05:58:23.320238+0000","last_clean":"2023-10-26T05:58:23.320238+0000","last_became_active":"2023-10-26T05:58:21.680243+0000","last_became_peered":"2023-10-26T05:58:21.680243+0000","last_unstale":"2023-10-26T05:58:23.320238+0000","last_undegraded":"2023-10-26T05:58:23.320238+0000","last_fullsized":"2023-10-26T05:58:23.320238+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T13:07:23.692301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":11,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.313398+0000","last_change":"2023-10-26T05:58:22.319747+0000","last_active":"2023-10-26T05:58:23.313398+0000","last_peered":"2023-10-26T05:58:23.313398+0000","last_clean":"2023-10-26T05:58:23.313398+0000","last_became_active":"2023-10-26T05:58:22.319116+0000","last_became_peered":"2023-10-26T05:58:22.319116+0000","last_unstale":"2023-10-26T05:58:23.313398+0000","last_undegraded":"2023-10-26T05:58:23.313398+0000","last_fullsized":"2023-10-26T05:58:23.313398+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":186,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:21.629062+0000","last_clean_scrub_stamp":"2023-10-26T05:58:21.629062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:36:58.719687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":188,"state":"active+clean","last_fresh":"2023-10-26T05:58:23.320722+0000","last_change":"2023-10-26T05:58:21.689429+0000","last_active":"2023-10-26T05:58:23.320722+0000","last_peered":"2023-10-26T05:58:23.320722+0000","last_clean":"2023-10-26T05:58:23.320722+0000","last_became_active":"2023-10-26T05:58:21.677549+0000","last_became_peered":"2023-10-26T05:58:21.677549+0000","last_unstale":"2023-10-26T05:58:23.320722+0000","last_undegraded":"2023-10-26T05:58:23.320722+0000","last_fullsized":"2023-10-26T05:58:23.320722+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:58:20.616129+0000","last_clean_scrub_stamp":"2023-10-26T05:58:20.616129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:54:08.794339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":319,"reported_epoch":186,"state":"active+clean","last_fresh":"2023-10-26T05:58:21.665741+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:58:21.665741+0000","last_peered":"2023-10-26T05:58:21.665741+0000","last_clean":"2023-10-26T05:58:21.665741+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:58:21.665741+0000","last_undegraded":"2023-10-26T05:58:21.665741+0000","last_fullsized":"2023-10-26T05:58:21.665741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1608,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193703,"statfs":{"total":200000000,"available":198352768,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.878,"15min":0.86299999999999999},"min":{"1min":0.52700000000000002,"5min":0.501,"15min":0.39600000000000002},"max":{"1min":1.1739999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.86699999999999999,"15min":0.84699999999999998},"min":{"1min":0.67400000000000004,"5min":0.44600000000000001,"15min":0.36399999999999999},"max":{"1min":1.196,"5min":1.196,"15min":1.3100000000000001},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.95599999999999996,"15min":1.0009999999999999},"min":{"1min":0.72499999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3149999999999999,"5min":1.387,"15min":1.4319999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.0580000000000001,"15min":0.999},"min":{"1min":0.84099999999999997,"5min":0.58499999999999996,"15min":0.496},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.93500000000000005}]},{"osd":2,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.90000000000000002,"15min":0.84699999999999998},"min":{"1min":0.67500000000000004,"5min":0.48799999999999999,"15min":0.40799999999999997},"max":{"1min":1.1479999999999999,"5min":1.448,"15min":1.448},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91700000000000004,"15min":0.93000000000000005},"min":{"1min":0.58399999999999996,"5min":0.51300000000000001,"15min":0.47699999999999998},"max":{"1min":1.1870000000000001,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.93999999999999995,"15min":0.90500000000000003},"min":{"1min":0.65700000000000003,"5min":0.55600000000000005,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":1.2709999999999999,"15min":1.4330000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.018,"5min":0.96699999999999997,"15min":0.97499999999999998},"min":{"1min":0.624,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.0169999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.91300000000000003},"min":{"1min":0.65300000000000002,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.236,"5min":1.2589999999999999,"15min":1.639},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92900000000000005,"15min":0.93899999999999995},"min":{"1min":0.64900000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.206,"5min":1.3149999999999999,"15min":1.4159999999999999},"last":0.749}]},{"osd":5,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":1.002,"15min":1.0669999999999999},"min":{"1min":0.65400000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.1599999999999999,"5min":1.482,"15min":1.6679999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0289999999999999,"15min":1.0600000000000001},"min":{"1min":0.75700000000000001,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.3360000000000001,"5min":1.504,"15min":1.702},"last":0.98999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:57:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.89300000000000002,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.347,"5min":1.391,"15min":1.7509999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0629999999999999,"15min":1.0549999999999999},"min":{"1min":0.71099999999999997,"5min":0.65400000000000003,"15min":0.46200000000000002},"max":{"1min":1.3320000000000001,"5min":1.5,"15min":1.6180000000000001},"last":0.88800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1029,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198945468,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.88400000000000001,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.216,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.872,"15min":0.86599999999999999},"min":{"1min":0.503,"5min":0.376,"15min":0.376},"max":{"1min":1.216,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.25}]},{"osd":1,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.83299999999999996,"15min":0.80100000000000005},"min":{"1min":0.65000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.077,"5min":1.1719999999999999,"15min":1.2609999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.871},"min":{"1min":0.55400000000000005,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.1830000000000001,"5min":1.3899999999999999,"15min":1.405},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.88900000000000001,"15min":0.84799999999999998},"min":{"1min":0.57299999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1100000000000001,"5min":1.268,"15min":1.8460000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.88300000000000001,"15min":0.85799999999999998},"min":{"1min":0.52600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.137,"5min":1.337,"15min":1.9239999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.998,"15min":0.93600000000000005},"min":{"1min":0.67200000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.29,"5min":1.296,"15min":1.3080000000000001},"last":1.341},{"interface":"front","average":{"1min":0.998,"5min":1.0620000000000001,"15min":1.0049999999999999},"min":{"1min":0.64400000000000002,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.2749999999999999,"5min":1.3759999999999999,"15min":1.399},"last":1.397}]},{"osd":4,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.95899999999999996,"15min":0.95599999999999996},"min":{"1min":0.60399999999999998,"5min":0.57599999999999996,"15min":0.502},"max":{"1min":1.1970000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.4299999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0029999999999999,"15min":0.98199999999999998},"min":{"1min":0.629,"5min":0.621,"15min":0.442},"max":{"1min":1.264,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1040000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.077,"15min":1.0509999999999999},"min":{"1min":0.77600000000000002,"5min":0.59999999999999998,"15min":0.45600000000000002},"max":{"1min":1.2569999999999999,"5min":1.5509999999999999,"15min":1.554},"last":1.204},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.98199999999999998,"15min":0.98099999999999998},"min":{"1min":0.69199999999999995,"5min":0.53300000000000003,"15min":0.36799999999999999},"max":{"1min":1.3029999999999999,"5min":1.512,"15min":1.512},"last":1.159}]},{"osd":7,"last update":"Thu Oct 26 05:57:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.1279999999999999,"15min":1.1100000000000001},"min":{"1min":0.71599999999999997,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3260000000000001,"5min":1.498,"15min":1.498},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.113,"15min":1.1100000000000001},"min":{"1min":0.89700000000000002,"5min":0.49399999999999999,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.294}]}]},{"osd":5,"up_from":13,"seq":55834574985,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198944886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94899999999999995,"15min":0.95499999999999996},"min":{"1min":0.59499999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.196,"5min":1.381,"15min":1.579},"last":1.264},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.99399999999999999,"15min":0.96799999999999997},"min":{"1min":0.58799999999999997,"5min":0.499,"15min":0.499},"max":{"1min":1.3400000000000001,"5min":1.591,"15min":1.591},"last":0.92000000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.119,"5min":1.206,"15min":2.1150000000000002},"last":1.214},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88,"15min":0.86299999999999999},"min":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.52400000000000002},"max":{"1min":1.179,"5min":1.272,"15min":1.464},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.93400000000000005,"15min":0.91700000000000004},"min":{"1min":0.61099999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.204,"5min":1.569,"15min":1.569},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.90500000000000003,"15min":0.92200000000000004},"min":{"1min":0.68200000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":1.117}]},{"osd":3,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.92600000000000005,"15min":0.91900000000000004},"min":{"1min":0.60999999999999999,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.323,"5min":1.448,"15min":1.548},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.99199999999999999,"15min":0.997},"min":{"1min":0.81399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.395,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.335}]},{"osd":4,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.72499999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.236,"5min":1.286,"15min":2.1909999999999998},"last":1.373},{"interface":"front","average":{"1min":1.147,"5min":1.0860000000000001,"15min":1.093},"min":{"1min":0.90500000000000003,"5min":0.51700000000000002,"15min":0.40799999999999997},"max":{"1min":1.357,"5min":1.536,"15min":1.536},"last":1.4099999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.008,"15min":1.0089999999999999},"min":{"1min":0.60899999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4590000000000001,"5min":1.744,"15min":1.744},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0509999999999999,"15min":1.0349999999999999},"min":{"1min":0.77600000000000002,"5min":0.53600000000000003,"15min":0.504},"max":{"1min":1.3620000000000001,"5min":1.4199999999999999,"15min":1.4950000000000001},"last":1.008}]},{"osd":7,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.0860000000000001,"15min":1.077},"min":{"1min":0.99299999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.452,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.05},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.1599999999999999,"15min":1.157},"min":{"1min":0.91400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.444,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.077}]}]},{"osd":4,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1037,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198937508,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.90000000000000002,"15min":0.87},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.131,"5min":1.248,"15min":1.347},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91700000000000004,"15min":0.89000000000000001},"min":{"1min":0.63100000000000001,"5min":0.48299999999999998,"15min":0.47199999999999998},"max":{"1min":1.234,"5min":1.2909999999999999,"15min":1.502},"last":0.59799999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89100000000000001,"15min":0.84499999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.1719999999999999,"5min":1.21,"15min":1.375},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97099999999999997,"15min":0.92400000000000004},"min":{"1min":0.66900000000000004,"5min":0.57699999999999996,"15min":0.52300000000000002},"max":{"1min":1.387,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.92500000000000004,"15min":0.85699999999999998},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.143,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90700000000000003,"15min":0.88300000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.373},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.97699999999999998,"15min":0.94999999999999996},"min":{"1min":0.60199999999999998,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.248,"5min":1.397,"15min":2.0190000000000001},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.006,"5min":0.997,"15min":0.96699999999999997},"min":{"1min":0.64400000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2210000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.026}]},{"osd":5,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.88800000000000001,"15min":0.89000000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2390000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.99199999999999999,"15min":0.999},"min":{"1min":0.67800000000000005,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.673},"last":1.089}]},{"osd":6,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.034,"15min":1.016},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.349,"5min":1.762,"15min":1.7969999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.0429999999999999,"15min":1.038},"min":{"1min":0.67700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2629999999999999,"5min":1.355,"15min":1.4970000000000001},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:57:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.1000000000000001,"15min":1.0620000000000001},"min":{"1min":0.57899999999999996,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.405,"5min":1.659,"15min":1.7509999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.133,"15min":1.123},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2849999999999999,"5min":1.484,"15min":1.728},"last":1.1359999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574985,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1035,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194276,"statfs":{"total":200000000,"available":198939156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.371,"5min":0.28699999999999998,"15min":0.249},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.385},"last":0.376},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.623,"15min":0.59999999999999998},"min":{"1min":0.38600000000000001,"5min":0.30399999999999999,"15min":0.26000000000000001},"max":{"1min":1.0600000000000001,"5min":1.115,"15min":1.2689999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.66000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.27700000000000002},"max":{"1min":6.109,"5min":6.109,"15min":6.109},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.66900000000000004,"15min":0.629},"min":{"1min":0.36099999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.018,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.442}]},{"osd":2,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.81599999999999995,"15min":0.77400000000000002},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.30399999999999999},"max":{"1min":6.1710000000000003,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":0.877,"15min":0.81100000000000005},"min":{"1min":0.42699999999999999,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":6.0389999999999997,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.873,"15min":0.84999999999999998},"min":{"1min":0.61299999999999999,"5min":0.48499999999999999,"15min":0.40000000000000002},"max":{"1min":1.181,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.90700000000000003,"15min":0.92000000000000004},"min":{"1min":0.59999999999999998,"5min":0.57499999999999996,"15min":0.52400000000000002},"max":{"1min":1.3720000000000001,"5min":1.429,"15min":1.4810000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93100000000000005,"15min":0.95499999999999996},"min":{"1min":0.61699999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.454,"5min":1.454,"15min":1.478},"last":0.752},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.93899999999999995,"15min":0.92200000000000004},"min":{"1min":0.64500000000000002,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.1970000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.0049999999999999,"15min":0.99199999999999999},"min":{"1min":0.63,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.113,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:57:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.98099999999999998,"15min":0.94999999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.4159999999999999,"5min":2.016,"15min":2.016},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.27,"5min":1.054,"15min":1.0469999999999999},"min":{"1min":0.65900000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":4.7300000000000004,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.85299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1029,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194282,"statfs":{"total":200000000,"available":198945400,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63600000000000001,"15min":0.57399999999999995},"min":{"1min":0.374,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.99399999999999999,"5min":1.097,"15min":1.097},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71099999999999997,"15min":0.64700000000000002},"min":{"1min":0.34300000000000003,"5min":0.29299999999999998,"15min":0.26000000000000001},"max":{"1min":1.157,"5min":1.379,"15min":1.5249999999999999},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.74399999999999999,"15min":0.70099999999999996},"min":{"1min":0.36199999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.96699999999999997,"5min":1.198,"15min":1.2609999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65800000000000003,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.29299999999999998,"15min":0.26700000000000002},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":1.3600000000000001},"last":0.501}]},{"osd":3,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82399999999999995,"15min":0.77200000000000002},"min":{"1min":0.42499999999999999,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.1240000000000001,"5min":1.24,"15min":2.8809999999999998},"last":0.435},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71999999999999997,"15min":0.64600000000000002},"min":{"1min":0.32500000000000001,"5min":0.313,"15min":0.27800000000000002},"max":{"1min":0.88100000000000001,"5min":1.125,"15min":1.3440000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.92400000000000004,"15min":0.86899999999999999},"min":{"1min":0.61799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1439999999999999,"5min":1.5189999999999999,"15min":2.6139999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.879,"15min":0.86499999999999999},"min":{"1min":0.68600000000000005,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.099,"5min":1.4870000000000001,"15min":5.9550000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98799999999999999,"15min":0.94599999999999995},"min":{"1min":0.66800000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.5289999999999999,"5min":1.5600000000000001,"15min":2.8319999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.97599999999999998,"15min":0.93500000000000005},"min":{"1min":0.65400000000000003,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.232,"5min":1.4790000000000001,"15min":1.653},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.99399999999999999,"15min":0.95399999999999996},"min":{"1min":0.64200000000000002,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.272,"5min":1.6000000000000001,"15min":2.5760000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.073,"15min":1.024},"min":{"1min":0.68300000000000005,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.486,"5min":1.7769999999999999,"15min":6.7729999999999997},"last":0.90000000000000002}]},{"osd":7,"last update":"Thu Oct 26 05:57:29 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95799999999999996,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1739999999999999,"5min":1.581,"15min":1.581},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0900000000000001,"15min":1.077},"min":{"1min":0.65600000000000003,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.411,"5min":1.823,"15min":15.352},"last":0.78700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1035,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194276,"statfs":{"total":200000000,"available":198939156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58699999999999997,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.78100000000000003,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66900000000000004,"15min":0.64100000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.94599999999999995,"5min":1.089,"15min":1.157},"last":0.72799999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67000000000000004,"15min":0.65900000000000003},"min":{"1min":0.35799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79500000000000004,"15min":0.749},"min":{"1min":0.441,"5min":0.35599999999999998,"15min":0.27700000000000002},"max":{"1min":1.492,"5min":1.492,"15min":1.8169999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.751,"15min":0.75800000000000001},"min":{"1min":0.35599999999999998,"5min":0.312,"15min":0.29899999999999999},"max":{"1min":1.0820000000000001,"5min":1.147,"15min":1.7949999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.70499999999999996,"15min":0.69299999999999995},"min":{"1min":0.38400000000000001,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":1.004,"5min":1.175,"15min":1.298},"last":0.63700000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.88500000000000001,"15min":0.877},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.157,"5min":1.238,"15min":1.5780000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.95899999999999996,"15min":0.95599999999999996},"min":{"1min":0.57799999999999996,"5min":0.57199999999999995,"15min":0.52100000000000002},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":1.1719999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.96999999999999997,"15min":0.96499999999999997},"min":{"1min":0.56299999999999994,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.492},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":1.0920000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.99099999999999999,"15min":1.02},"min":{"1min":0.58099999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.353,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.997,"15min":0.97999999999999998},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.393,"5min":1.393,"15min":1.837},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:57:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.036,"15min":1.0289999999999999},"min":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.429},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.048,"15min":1.0780000000000001},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.56599999999999995},"max":{"1min":1.4710000000000001,"5min":1.4890000000000001,"15min":5.2430000000000003},"last":0.92800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574985,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1606,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193706,"statfs":{"total":200000000,"available":198355122,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.76200000000000001,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.70299999999999996,"15min":0.64100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.90100000000000002,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":1.042}]},{"osd":2,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81599999999999995,"15min":0.72199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":1.073,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.624,"5min":0.67900000000000005,"15min":0.63700000000000001},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.26300000000000001},"max":{"1min":0.90600000000000003,"5min":1.1020000000000001,"15min":1.177},"last":0.88900000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.68100000000000005,"15min":0.64300000000000002},"min":{"1min":0.33200000000000002,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.0149999999999999,"5min":1.3520000000000001,"15min":1.409},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.80500000000000005,"15min":0.73099999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.048,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.093}]},{"osd":4,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.91000000000000003,"15min":0.876},"min":{"1min":0.53200000000000003,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.0880000000000001,"5min":1.395,"15min":1.395},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0029999999999999,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1870000000000001,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.1950000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.91600000000000004,"15min":0.92100000000000004},"min":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.073,"5min":1.2430000000000001,"15min":2.7480000000000002},"last":1.006},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95799999999999996,"15min":0.92500000000000004},"min":{"1min":0.63800000000000001,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.1579999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.92600000000000005,"15min":0.91800000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.5640000000000001},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.121,"15min":1.0209999999999999},"min":{"1min":0.59999999999999998,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.167,"5min":6.4690000000000003,"15min":6.4690000000000003},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.98899999999999999,"15min":0.99299999999999999},"min":{"1min":0.48999999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.194,"5min":1.4399999999999999,"15min":1.6990000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.022,"15min":1.0009999999999999},"min":{"1min":0.60699999999999998,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.23,"5min":1.4790000000000001,"15min":1.5660000000000001},"last":0.86199999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:58:26.085 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-26T05:58:26.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-26T05:58:26.257 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 -- 172.21.15.71:0/1098294756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1013ef60 msgr2=0x7f1b10069c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 --2- 172.21.15.71:0/1098294756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1013ef60 0x7f1b10069c00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1b1006c780 tx=0x7f1b0801ccf0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 -- 172.21.15.71:0/1098294756 shutdown_connections 2023-10-26T05:58:26.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 --2- 172.21.15.71:0/1098294756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b1013ef60 0x7f1b10069c00 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 --2- 172.21.15.71:0/1098294756 >> v2:172.21.15.86:3300/0 conn(0x7f1b1013c120 0x7f1b100cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 -- 172.21.15.71:0/1098294756 >> 172.21.15.71:0/1098294756 conn(0x7f1b100aec60 msgr2=0x7f1b100af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:26.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 -- 172.21.15.71:0/1098294756 shutdown_connections 2023-10-26T05:58:26.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.247+0000 7f1b16803640 1 -- 172.21.15.71:0/1098294756 wait complete. 2023-10-26T05:58:26.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 Processor -- start 2023-10-26T05:58:26.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- start start 2023-10-26T05:58:26.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 0x7f1b10135610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b1013c120 0x7f1b1013c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1013ef60 0x7f1b1013b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1013aeb0 con 0x7f1b100cbe00 2023-10-26T05:58:26.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10070510 con 0x7f1b1013ef60 2023-10-26T05:58:26.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10135b50 con 0x7f1b1013c120 2023-10-26T05:58:26.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 0x7f1b10135610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 0x7f1b10135610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47002/0 (socket says 172.21.15.71:47002) 2023-10-26T05:58:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 -- 172.21.15.71:0/369469204 learned_addr learned my addr 172.21.15.71:0/369469204 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b14da0640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1013ef60 0x7f1b1013b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b07fff640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b1013c120 0x7f1b1013c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 -- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b1013c120 msgr2=0x7f1b1013c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b1013c120 0x7f1b1013c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 -- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1013ef60 msgr2=0x7f1b1013b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1013ef60 0x7f1b1013b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 -- 172.21.15.71:0/369469204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080098e0 con 0x7f1b100cbe00 2023-10-26T05:58:26.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b14da0640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1013ef60 0x7f1b1013b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:26.265 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0ffff640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 0x7f1b10135610 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c9a0 tx=0x7f1b0000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00007bb0 con 0x7f1b100cbe00 2023-10-26T05:58:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1b00007e20 con 0x7f1b100cbe00 2023-10-26T05:58:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00005280 con 0x7f1b100cbe00 2023-10-26T05:58:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1006f900 con 0x7f1b100cbe00 2023-10-26T05:58:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1006fec0 con 0x7f1b100cbe00 2023-10-26T05:58:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b10066b10 con 0x7f1b100cbe00 2023-10-26T05:58:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b000053e0 con 0x7f1b100cbe00 2023-10-26T05:58:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.251+0000 7f1b0d7fa640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1af0066470 0x7f1af0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.255+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b00084790 con 0x7f1b100cbe00 2023-10-26T05:58:26.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.255+0000 7f1b07fff640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1af0066470 0x7f1af0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.255+0000 7f1b07fff640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1af0066470 0x7f1af0068930 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1b10136150 tx=0x7f1af8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:26.268 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.259+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1b00051c10 con 0x7f1b100cbe00 2023-10-26T05:58:26.474 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.467+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1b100721e0 con 0x7f1b100cbe00 2023-10-26T05:58:26.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b0d7fa640 1 -- 172.21.15.71:0/369469204 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v39) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1b0001d020 con 0x7f1b100cbe00 2023-10-26T05:58:26.479 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2023-10-26T05:58:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBH/TllK9SgAhAAIvj9hRKsSoLN2xYfXJ0Trg== 2023-10-26T05:58:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: caps mds = "allow" 2023-10-26T05:58:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: caps mon = "allow r" 2023-10-26T05:58:26.480 INFO:teuthology.orchestra.run.smithi071.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-26T05:58:26.481 INFO:teuthology.orchestra.run.smithi071.stderr:updated caps for client.0 2023-10-26T05:58:26.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1af0066470 msgr2=0x7f1af0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1af0066470 0x7f1af0068930 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1b10136150 tx=0x7f1af8009040 comp rx=0 tx=0).stop 2023-10-26T05:58:26.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 msgr2=0x7f1b10135610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 0x7f1b10135610 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1b0000c9a0 tx=0x7f1b0000ce60 comp rx=0 tx=0).stop 2023-10-26T05:58:26.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 shutdown_connections 2023-10-26T05:58:26.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b1013ef60 0x7f1b1013b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b1013c120 0x7f1b1013c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1af0066470 0x7f1af0068930 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 --2- 172.21.15.71:0/369469204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b100cbe00 0x7f1b10135610 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 >> 172.21.15.71:0/369469204 conn(0x7f1b100aec60 msgr2=0x7f1b1015ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:26.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 shutdown_connections 2023-10-26T05:58:26.484 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.471+0000 7f1b16803640 1 -- 172.21.15.71:0/369469204 wait complete. 2023-10-26T05:58:26.498 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:26.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/1031107656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff358140cc0 msgr2=0x7ff35815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 --2- 172.21.15.71:0/1031107656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff358140cc0 0x7ff35815c760 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ff348009930 tx=0x7ff34801c850 comp rx=0 tx=0).stop 2023-10-26T05:58:26.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/1031107656 shutdown_connections 2023-10-26T05:58:26.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 --2- 172.21.15.71:0/1031107656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff358140cc0 0x7ff35815c760 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/1031107656 >> 172.21.15.71:0/1031107656 conn(0x7ff358069770 msgr2=0x7ff358069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:26.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/1031107656 shutdown_connections 2023-10-26T05:58:26.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/1031107656 wait complete. 2023-10-26T05:58:26.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 Processor -- start 2023-10-26T05:58:26.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- start start 2023-10-26T05:58:26.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff358070fe0 0x7ff358154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 0x7ff35806fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff358140cc0 0x7ff3580703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358135a30 con 0x7ff3580b2370 2023-10-26T05:58:26.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff358140350 con 0x7ff358070fe0 2023-10-26T05:58:26.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff358135c40 con 0x7ff358140cc0 2023-10-26T05:58:26.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 0x7ff35806fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 0x7ff35806fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47024/0 (socket says 172.21.15.71:47024) 2023-10-26T05:58:26.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 -- 172.21.15.71:0/158038184 learned_addr learned my addr 172.21.15.71:0/158038184 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:26.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff356d9d640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff358070fe0 0x7ff358154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 -- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff358140cc0 msgr2=0x7ff3580703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff358140cc0 0x7ff3580703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 -- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff358070fe0 msgr2=0x7ff358154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff358070fe0 0x7ff358154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 -- 172.21.15.71:0/158038184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348009560 con 0x7ff3580b2370 2023-10-26T05:58:26.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff356d9d640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff358070fe0 0x7ff358154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:26.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35659c640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 0x7ff35806fe80 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c370 tx=0x7ff34c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:26.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c0052d0 con 0x7ff3580b2370 2023-10-26T05:58:26.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff34c007020 con 0x7ff3580b2370 2023-10-26T05:58:26.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff358135f20 con 0x7ff3580b2370 2023-10-26T05:58:26.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c007430 con 0x7ff3580b2370 2023-10-26T05:58:26.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.615+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3581400d0 con 0x7ff3580b2370 2023-10-26T05:58:26.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.619+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358066b10 con 0x7ff3580b2370 2023-10-26T05:58:26.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.619+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff34c0067e0 con 0x7ff3580b2370 2023-10-26T05:58:26.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.619+0000 7ff33f7fe640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff344066470 0x7ff344068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:26.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.619+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff34c083460 con 0x7ff3580b2370 2023-10-26T05:58:26.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.619+0000 7ff356d9d640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff344066470 0x7ff344068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:26.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.619+0000 7ff356d9d640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff344066470 0x7ff344068930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff3480af000 tx=0x7ff34801b090 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:26.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.623+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff34c0508f0 con 0x7ff3580b2370 2023-10-26T05:58:26.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.807+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3580721e0 con 0x7ff3580b2370 2023-10-26T05:58:26.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.807+0000 7ff33f7fe640 1 -- 172.21.15.71:0/158038184 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7ff34c053f10 con 0x7ff3580b2370 2023-10-26T05:58:26.816 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:26.816 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:58:26.817 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:58:26.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff344066470 msgr2=0x7ff344068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff344066470 0x7ff344068930 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff3480af000 tx=0x7ff34801b090 comp rx=0 tx=0).stop 2023-10-26T05:58:26.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 msgr2=0x7ff35806fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:26.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 0x7ff35806fe80 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00c370 tx=0x7ff34c00c830 comp rx=0 tx=0).stop 2023-10-26T05:58:26.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 shutdown_connections 2023-10-26T05:58:26.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff358140cc0 0x7ff3580703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.819 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff344066470 0x7ff344068930 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3580b2370 0x7ff35806fe80 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 --2- 172.21.15.71:0/158038184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff358070fe0 0x7ff358154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:26.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 >> 172.21.15.71:0/158038184 conn(0x7ff358069770 msgr2=0x7ff3580b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:26.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 shutdown_connections 2023-10-26T05:58:26.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:26.811+0000 7ff35d034640 1 -- 172.21.15.71:0/158038184 wait complete. 2023-10-26T05:58:27.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-26T05:58:22.010694+0000', 'modified': '2023-10-26T05:58:24.036008+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9387}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9387': {'gid': 9387, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6835/327274368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 327274368}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 327274368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-26T05:58:27.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-26T05:58:27.237 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-26T05:58:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- 172.21.15.71:0/2759266339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c15b8b0 msgr2=0x7f059c15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 --2- 172.21.15.71:0/2759266339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c15b8b0 0x7f059c15bd40 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f0590009840 tx=0x7f059001c7d0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- 172.21.15.71:0/2759266339 shutdown_connections 2023-10-26T05:58:27.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 --2- 172.21.15.71:0/2759266339 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059c0b1050 0x7f059c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 --2- 172.21.15.71:0/2759266339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c15b8b0 0x7f059c15bd40 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- 172.21.15.71:0/2759266339 >> 172.21.15.71:0/2759266339 conn(0x7f059c070650 msgr2=0x7f059c070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- 172.21.15.71:0/2759266339 shutdown_connections 2023-10-26T05:58:27.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- 172.21.15.71:0/2759266339 wait complete. 2023-10-26T05:58:27.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 Processor -- start 2023-10-26T05:58:27.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- start start 2023-10-26T05:58:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 0x7f059c155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059c158e50 0x7f059c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059c15b8b0 0x7f059c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c14da10 con 0x7f059c0b1050 2023-10-26T05:58:27.376 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c153000 con 0x7f059c158e50 2023-10-26T05:58:27.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f05a118b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c1532e0 con 0x7f059c15b8b0 2023-10-26T05:58:27.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f059ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 0x7f059c155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f059ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 0x7f059c155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47030/0 (socket says 172.21.15.71:47030) 2023-10-26T05:58:27.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.363+0000 7f059ad9d640 1 -- 172.21.15.71:0/1745404963 learned_addr learned my addr 172.21.15.71:0/1745404963 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:27.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059b59e640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059c15b8b0 0x7f059c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059a59c640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059c158e50 0x7f059c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059ad9d640 1 -- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059c15b8b0 msgr2=0x7f059c1529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059ad9d640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059c15b8b0 0x7f059c1529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059ad9d640 1 -- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059c158e50 msgr2=0x7f059c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059ad9d640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059c158e50 0x7f059c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.378 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059ad9d640 1 -- 172.21.15.71:0/1745404963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900094a0 con 0x7f059c0b1050 2023-10-26T05:58:27.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059ad9d640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 0x7f059c155ce0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0594001570 tx=0x7f059400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05837fe640 1 -- 172.21.15.71:0/1745404963 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594003db0 con 0x7f059c0b1050 2023-10-26T05:58:27.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05837fe640 1 -- 172.21.15.71:0/1745404963 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0594005070 con 0x7f059c0b1050 2023-10-26T05:58:27.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c1535c0 con 0x7f059c0b1050 2023-10-26T05:58:27.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05837fe640 1 -- 172.21.15.71:0/1745404963 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594005480 con 0x7f059c0b1050 2023-10-26T05:58:27.380 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c156220 con 0x7f059c0b1050 2023-10-26T05:58:27.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05837fe640 1 -- 172.21.15.71:0/1745404963 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f05940055e0 con 0x7f059c0b1050 2023-10-26T05:58:27.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05837fe640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0588066470 0x7f0588068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05837fe640 1 -- 172.21.15.71:0/1745404963 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0594083a80 con 0x7f059c0b1050 2023-10-26T05:58:27.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059a59c640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0588066470 0x7f0588068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05a42d5000 1 Processor -- start 2023-10-26T05:58:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f05a42d5000 1 -- start start 2023-10-26T05:58:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.367+0000 7f059a59c640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0588066470 0x7f0588068930 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f05900af000 tx=0x7f059001b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.387 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5563123ea2e0 0x55631239fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55631239ff80 0x5563123a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556312194cc0 con 0x5563123e99c0 2023-10-26T05:58:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563121f7920 con 0x55631239ff80 2023-10-26T05:58:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563121f4e50 con 0x5563123ea2e0 2023-10-26T05:58:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.388 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47038/0 (socket says 172.21.15.71:47038) 2023-10-26T05:58:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 -- 172.21.15.71:0/3988997779 learned_addr learned my addr 172.21.15.71:0/3988997779 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059a59c640 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5563123ea2e0 0x55631239fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 -- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5563123ea2e0 msgr2=0x55631239fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55631239ff80 0x5563123a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5563123ea2e0 0x55631239fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.389 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 -- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55631239ff80 msgr2=0x5563123a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55631239ff80 0x5563123a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 -- 172.21.15.71:0/3988997779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556311ee9890 con 0x5563123e99c0 2023-10-26T05:58:27.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e9da0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f059405c990 tx=0x7f05940552c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4e46fbbcf5e4b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f0580ff9640 1 -- 172.21.15.71:0/3988997779 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059406c070 con 0x5563123e99c0 2023-10-26T05:58:27.390 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f0580ff9640 1 -- 172.21.15.71:0/3988997779 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0594052f60 con 0x5563123e99c0 2023-10-26T05:58:27.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f0580ff9640 1 -- 172.21.15.71:0/3988997779 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05940540b0 con 0x5563123e99c0 2023-10-26T05:58:27.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- 172.21.15.71:0/3988997779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 msgr2=0x5563123e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e9da0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f059405c990 tx=0x7f05940552c0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- 172.21.15.71:0/3988997779 shutdown_connections 2023-10-26T05:58:27.391 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55631239ff80 0x5563123a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x5563123ea2e0 0x55631239fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- 172.21.15.71:0/3988997779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e9da0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- 172.21.15.71:0/3988997779 >> 172.21.15.71:0/3988997779 conn(0x5563123972f0 msgr2=0x55631239e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- 172.21.15.71:0/3988997779 shutdown_connections 2023-10-26T05:58:27.392 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- 172.21.15.71:0/3988997779 wait complete. 2023-10-26T05:58:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- start start 2023-10-26T05:58:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 Processor -- start 2023-10-26T05:58:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563123a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.393 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563121f0740 con 0x5563123ea2e0 2023-10-26T05:58:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563121afd10 con 0x5563123e99c0 2023-10-26T05:58:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563121af9b0 con 0x55631239ff80 2023-10-26T05:58:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 10 client.0 _ll_register_callbacks cb 0x55631238b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:58:27.394 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123abbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47048/0 (socket says 172.21.15.71:47048) 2023-10-26T05:58:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 -- 172.21.15.71:0/2129711879 learned_addr learned my addr 172.21.15.71:0/2129711879 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059a59c640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563123a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059ad9d640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 -- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 msgr2=0x5563123a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.395 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123a53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 -- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 msgr2=0x5563123a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563123a58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 -- 172.21.15.71:0/2129711879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c156220 con 0x5563123ea2e0 2023-10-26T05:58:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f059b59e640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123abbe0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f058c00af30 tx=0x7f058c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0241a9fe4135ab6 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.396 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f0579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:58:27.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f0579ffb640 1 -- 172.21.15.71:0/2129711879 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011850 con 0x5563123ea2e0 2023-10-26T05:58:27.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f0579ffb640 1 -- 172.21.15.71:0/2129711879 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f058c015070 con 0x5563123ea2e0 2023-10-26T05:58:27.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.371+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563121a4370 con 0x5563123ea2e0 2023-10-26T05:58:27.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 1 -- 172.21.15.71:0/2129711879 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058c0119b0 con 0x5563123ea2e0 2023-10-26T05:58:27.397 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 10 client.9534 fetch_fsmap learned FSMap version 71 2023-10-26T05:58:27.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x5563121b2460 con 0x5563123ea2e0 2023-10-26T05:58:27.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 1 -- 172.21.15.71:0/2129711879 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5563121a4370 con 0x5563123ea2e0 2023-10-26T05:58:27.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 1 -- 172.21.15.71:0/2129711879 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f058c011b10 con 0x5563123ea2e0 2023-10-26T05:58:27.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 10 client.9534 fetch_fsmap finished waiting for FSMap version 71 2023-10-26T05:58:27.398 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 10 client.9534 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:58:27.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 20 client.9534 populate_metadata read hostname 'smithi071' 2023-10-26T05:58:27.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x5563123ac5f0 0x5563123fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 4 client.9534 mds_command: new command op to 9387 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-26T05:58:27.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 --> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55631218a2d0 con 0x5563123ac5f0 2023-10-26T05:58:27.399 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f059ad9d640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x5563123ac5f0 0x5563123fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 10 client.9534 ms_handle_connect on v2:172.21.15.71:6834/327274368 2023-10-26T05:58:27.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f059ad9d640 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x5563123ac5f0 0x5563123fba70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ccd58cee7cdf808 server_cookie=57a2a0b40d3ff085 in_seq=0 out_seq=0 2023-10-26T05:58:27.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 1 -- 172.21.15.71:0/2129711879 <== mds.0 v2:172.21.15.71:6834/327274368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55631218a2d0 con 0x5563123ac5f0 2023-10-26T05:58:27.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 10 client.9534 handle_command_reply: tid=0 2023-10-26T05:58:27.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 client.9534 shutdown 2023-10-26T05:58:27.400 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 msgr2=0x5563123abbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123abbe0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f058c00af30 tx=0x7f058c00f7f0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 shutdown_connections 2023-10-26T05:58:27.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x5563123ac5f0 0x5563123fba70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123abbe0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.401 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f0579ffb640 0 client.9534 ms_handle_reset on v2:172.21.15.71:6834/327274368 2023-10-26T05:58:27.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563123a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2129711879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 >> 172.21.15.71:0/2129711879 conn(0x5563123972f0 msgr2=0x55631239ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 shutdown_connections 2023-10-26T05:58:27.402 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 1 -- 172.21.15.71:0/2129711879 wait complete. 2023-10-26T05:58:27.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.375+0000 7f05a42d5000 20 client.9534 trim_cache size 0 max 16384 2023-10-26T05:58:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 Processor -- start 2023-10-26T05:58:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- start start 2023-10-26T05:58:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123ea2e0 0x5563123f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556311eaa940 con 0x5563123e99c0 2023-10-26T05:58:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563121ec0b0 con 0x5563123ea2e0 2023-10-26T05:58:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x556311eea1b0 con 0x55631239ff80 2023-10-26T05:58:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47064/0 (socket says 172.21.15.71:47064) 2023-10-26T05:58:27.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 -- 172.21.15.71:0/435350932 learned_addr learned my addr 172.21.15.71:0/435350932 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 -- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 msgr2=0x5563123e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059ad9d640 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123e7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059b59e640 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123ea2e0 0x5563123f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 -- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123ea2e0 msgr2=0x5563123f9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123ea2e0 0x5563123f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 -- 172.21.15.71:0/435350932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556312198760 con 0x5563123e99c0 2023-10-26T05:58:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e80f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f059001ccb0 tx=0x7f05900b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9038712292c650 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05817fa640 1 -- 172.21.15.71:0/435350932 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590005040 con 0x5563123e99c0 2023-10-26T05:58:27.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05817fa640 1 -- 172.21.15.71:0/435350932 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f05900c1040 con 0x5563123e99c0 2023-10-26T05:58:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05817fa640 1 -- 172.21.15.71:0/435350932 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590005510 con 0x5563123e99c0 2023-10-26T05:58:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- 172.21.15.71:0/435350932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 msgr2=0x5563123e80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e80f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f059001ccb0 tx=0x7f05900b7000 comp rx=0 tx=0).stop 2023-10-26T05:58:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- 172.21.15.71:0/435350932 shutdown_connections 2023-10-26T05:58:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123ea2e0 0x5563123f9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123e99c0 0x5563123e80f0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- 172.21.15.71:0/435350932 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x5563123e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- 172.21.15.71:0/435350932 >> 172.21.15.71:0/435350932 conn(0x5563123972f0 msgr2=0x5563123ab420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- 172.21.15.71:0/435350932 shutdown_connections 2023-10-26T05:58:27.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- 172.21.15.71:0/435350932 wait complete. 2023-10-26T05:58:27.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- start start 2023-10-26T05:58:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 Processor -- start 2023-10-26T05:58:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x556312447cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563124481f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563121f4740 con 0x5563123ea2e0 2023-10-26T05:58:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563121f32a0 con 0x5563123e99c0 2023-10-26T05:58:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563121b9720 con 0x55631239ff80 2023-10-26T05:58:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f05a42d5000 10 client.0 _ll_register_callbacks cb 0x5563121eeea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-26T05:58:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059b59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059b59e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123fbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47068/0 (socket says 172.21.15.71:47068) 2023-10-26T05:58:27.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059b59e640 1 -- 172.21.15.71:0/2750751889 learned_addr learned my addr 172.21.15.71:0/2750751889 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:27.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059ad9d640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x556312447cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.391+0000 7f059a59c640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563124481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059b59e640 1 -- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 msgr2=0x556312447cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059b59e640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x556312447cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059b59e640 1 -- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 msgr2=0x5563124481f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-26T05:58:27.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059b59e640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563124481f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059b59e640 1 -- 172.21.15.71:0/2750751889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c15c370 con 0x5563123ea2e0 2023-10-26T05:58:27.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059a59c640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563124481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:27.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059b59e640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123fbd80 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f058c011d50 tx=0x7f058c00a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad3b6f117d1da13 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2023-10-26T05:58:27.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 1 -- 172.21.15.71:0/2750751889 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c005220 con 0x5563123ea2e0 2023-10-26T05:58:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563121886f0 con 0x5563123ea2e0 2023-10-26T05:58:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 1 -- 172.21.15.71:0/2750751889 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f058c02c070 con 0x5563123ea2e0 2023-10-26T05:58:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 1 -- 172.21.15.71:0/2750751889 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058c005cb0 con 0x5563123ea2e0 2023-10-26T05:58:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 10 client.9540 fetch_fsmap learned FSMap version 71 2023-10-26T05:58:27.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x556311ee9cc0 con 0x5563123ea2e0 2023-10-26T05:58:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 1 -- 172.21.15.71:0/2750751889 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x5563121886f0 con 0x5563123ea2e0 2023-10-26T05:58:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 1 -- 172.21.15.71:0/2750751889 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 71) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f058c005380 con 0x5563123ea2e0 2023-10-26T05:58:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 10 client.9540 fetch_fsmap finished waiting for FSMap version 71 2023-10-26T05:58:27.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 10 client.9540 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-26T05:58:27.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 20 client.9540 populate_metadata read hostname 'smithi071' 2023-10-26T05:58:27.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x55631244eb80 0x556312450f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 4 client.9540 mds_command: new command op to 9387 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-26T05:58:27.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 --> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563121e9720 con 0x55631244eb80 2023-10-26T05:58:27.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059ad9d640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x55631244eb80 0x556312450f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 10 client.9540 ms_handle_connect on v2:172.21.15.71:6834/327274368 2023-10-26T05:58:27.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f059ad9d640 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x55631244eb80 0x556312450f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d388f4da3d06f0 server_cookie=b1263ba245942760 in_seq=0 out_seq=0 2023-10-26T05:58:27.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 1 -- 172.21.15.71:0/2750751889 <== mds.0 v2:172.21.15.71:6834/327274368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5563121e9720 con 0x55631244eb80 2023-10-26T05:58:27.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 10 client.9540 handle_command_reply: tid=0 2023-10-26T05:58:27.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 client.9540 shutdown 2023-10-26T05:58:27.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 msgr2=0x5563123fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123fbd80 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f058c011d50 tx=0x7f058c00a820 comp rx=0 tx=0).stop 2023-10-26T05:58:27.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 shutdown_connections 2023-10-26T05:58:27.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:6834/327274368,v1:172.21.15.71:6835/327274368] conn(0x55631244eb80 0x556312450f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563123ea2e0 0x5563123fbd80 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563123e99c0 0x5563124481f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f0579ffb640 0 client.9540 ms_handle_reset on v2:172.21.15.71:6834/327274368 2023-10-26T05:58:27.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 --2- 172.21.15.71:0/2750751889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x55631239ff80 0x556312447cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 >> 172.21.15.71:0/2750751889 conn(0x5563123972f0 msgr2=0x5563123e6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 shutdown_connections 2023-10-26T05:58:27.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 1 -- 172.21.15.71:0/2750751889 wait complete. 2023-10-26T05:58:27.425 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a42d5000 20 client.9540 trim_cache size 0 max 16384 2023-10-26T05:58:27.425 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:27.426 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:27.426 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2023-10-26T05:58:27.426 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 9387, 2023-10-26T05:58:27.426 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2023-10-26T05:58:27.426 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2023-10-26T05:58:27.427 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2023-10-26T05:58:27.427 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 4.7033907709999996, 2023-10-26T05:58:27.427 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 71, 2023-10-26T05:58:27.427 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 188, 2023-10-26T05:58:27.427 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 188, 2023-10-26T05:58:27.427 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 9.5145344699999992 2023-10-26T05:58:27.428 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:27.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0588066470 msgr2=0x7f0588068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0588066470 0x7f0588068930 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f05900af000 tx=0x7f059001b010 comp rx=0 tx=0).stop 2023-10-26T05:58:27.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 msgr2=0x7f059c155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 0x7f059c155ce0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0594001570 tx=0x7f059400cd00 comp rx=0 tx=0).stop 2023-10-26T05:58:27.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 shutdown_connections 2023-10-26T05:58:27.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f059c15b8b0 0x7f059c1529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f059c158e50 0x7f059c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0588066470 0x7f0588068930 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 --2- 172.21.15.71:0/1745404963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f059c0b1050 0x7f059c155ce0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 >> 172.21.15.71:0/1745404963 conn(0x7f059c070650 msgr2=0x7f059c15d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 shutdown_connections 2023-10-26T05:58:27.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.395+0000 7f05a118b640 1 -- 172.21.15.71:0/1745404963 wait complete. 2023-10-26T05:58:27.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:58:27.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/532878803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c280cc050 msgr2=0x7f1c280cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 --2- 172.21.15.71:0/532878803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c280cc050 0x7f1c280cd9f0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f1c2815dd70 tx=0x7f1c2001cc60 comp rx=0 tx=0).stop 2023-10-26T05:58:27.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/532878803 shutdown_connections 2023-10-26T05:58:27.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 --2- 172.21.15.71:0/532878803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c280cc050 0x7f1c280cd9f0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 --2- 172.21.15.71:0/532878803 >> v2:172.21.15.86:3300/0 conn(0x7f1c280af990 0x7f1c280cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/532878803 >> 172.21.15.71:0/532878803 conn(0x7f1c2806f440 msgr2=0x7f1c2806f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/532878803 shutdown_connections 2023-10-26T05:58:27.540 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/532878803 wait complete. 2023-10-26T05:58:27.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 Processor -- start 2023-10-26T05:58:27.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- start start 2023-10-26T05:58:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c280af990 0x7f1c28133850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c280cc050 0x7f1c28133d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 0x7f1c281512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28133470 con 0x7f1c28142b30 2023-10-26T05:58:27.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28138450 con 0x7f1c280af990 2023-10-26T05:58:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c28138730 con 0x7f1c280cc050 2023-10-26T05:58:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 0x7f1c281512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 0x7f1c281512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47078/0 (socket says 172.21.15.71:47078) 2023-10-26T05:58:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 -- 172.21.15.71:0/3932623980 learned_addr learned my addr 172.21.15.71:0/3932623980 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:27.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 -- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c280cc050 msgr2=0x7f1c28133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c280cc050 0x7f1c28133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 -- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c280af990 msgr2=0x7f1c28133850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c280af990 0x7f1c28133850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 -- 172.21.15.71:0/3932623980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200098e0 con 0x7f1c28142b30 2023-10-26T05:58:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c26d9d640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 0x7f1c281512c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f1c2001a890 tx=0x7f1c200a5c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.545 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200bc070 con 0x7f1c28142b30 2023-10-26T05:58:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1c20004250 con 0x7f1c28142b30 2023-10-26T05:58:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c28132eb0 con 0x7f1c28142b30 2023-10-26T05:58:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20004660 con 0x7f1c28142b30 2023-10-26T05:58:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.531+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c28154130 con 0x7f1c28142b30 2023-10-26T05:58:27.546 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.535+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c28066b10 con 0x7f1c28142b30 2023-10-26T05:58:27.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.535+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1c200b4040 con 0x7f1c28142b30 2023-10-26T05:58:27.547 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.535+0000 7f1c16ffd640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c08066470 0x7f1c08068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:27.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.535+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c2011ed00 con 0x7f1c28142b30 2023-10-26T05:58:27.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.535+0000 7f1c2659c640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c08066470 0x7f1c08068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:27.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.535+0000 7f1c2659c640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c08066470 0x7f1c08068930 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009650 tx=0x7f1c18006bb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:27.548 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.539+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1c200ec060 con 0x7f1c28142b30 2023-10-26T05:58:27.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.711+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1c28154320 con 0x7f1c28142b30 2023-10-26T05:58:27.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c16ffd640 1 -- 172.21.15.71:0/3932623980 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c200047c0 con 0x7f1c28142b30 2023-10-26T05:58:27.723 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:58:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c08066470 msgr2=0x7f1c08068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c08066470 0x7f1c08068930 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009650 tx=0x7f1c18006bb0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 msgr2=0x7f1c281512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 0x7f1c281512c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f1c2001a890 tx=0x7f1c200a5c40 comp rx=0 tx=0).stop 2023-10-26T05:58:27.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 shutdown_connections 2023-10-26T05:58:27.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1c08066470 0x7f1c08068930 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c28142b30 0x7f1c281512c0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1c280cc050 0x7f1c28133d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 --2- 172.21.15.71:0/3932623980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c280af990 0x7f1c28133850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:27.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 >> 172.21.15.71:0/3932623980 conn(0x7f1c2806f440 msgr2=0x7f1c280ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:27.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 shutdown_connections 2023-10-26T05:58:27.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:27.715+0000 7f1c2c941640 1 -- 172.21.15.71:0/3932623980 wait complete. 2023-10-26T05:58:27.740 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:58:27.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:58:27.740 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2023-10-26T05:58:27.741 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-26T05:58:27.741 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:27.741 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:58:27.742 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:58:27.742 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:58:27.742 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:58:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:27.743 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2023-10-26T05:58:27.749 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:58:27.749 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:58:27.750 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:58:27.750 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.750 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2023-10-26T05:58:27.750 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.751 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:58:27.751 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:27.751 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2023-10-26T05:58:27.751 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:58:27.751 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:27.752 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2023-10-26T05:58:27.752 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:58:27.752 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:27.752 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:58:27.752 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:27.753 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:58:27.753 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 41869sec preferred_lft 41869sec 2023-10-26T05:58:27.753 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link 2023-10-26T05:58:27.753 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.753 INFO:teuthology.orchestra.run.smithi071.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:58:27.754 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 02:42:c7:4a:cc:40 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:27.754 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:58:27.754 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.754 INFO:teuthology.orchestra.run.smithi071.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:58:27.754 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:27.755 INFO:teuthology.orchestra.run.smithi071.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:58:27.755 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.755 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:58:27.755 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.755 INFO:teuthology.orchestra.run.smithi071.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:58:27.755 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:27.756 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:58:27.756 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.756 INFO:teuthology.orchestra.run.smithi071.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:58:27.756 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:27.756 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:58:27.757 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:27.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:27.757 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:58:27.804 INFO:teuthology.orchestra.run.smithi071.stdout:1 2023-10-26T05:58:27.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:27.806 DEBUG:teuthology.orchestra.run.smithi071:> route 2023-10-26T05:58:27.857 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2023-10-26T05:58:27.857 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:58:27.857 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:58:27.858 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:58:27.858 INFO:teuthology.orchestra.run.smithi071.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:58:27.858 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:58:27.858 INFO:teuthology.orchestra.run.smithi071.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:58:27.858 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:58:27.859 INFO:teuthology.orchestra.run.smithi071.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:58:27.859 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:58:27.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:27.860 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec bash -c ' 2023-10-26T05:58:27.860 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2023-10-26T05:58:27.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:58:27.861 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:58:27.862 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:58:27.862 DEBUG:teuthology.orchestra.run.smithi071:> ') 2023-10-26T05:58:27.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:27.930 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:58:27.982 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:58:27.983 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:58:27.983 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:58:27.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:27.984 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2023-10-26T05:58:28.039 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-26T05:58:28.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:58:28.040 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:28.086 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-26T05:58:28.087 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:58:28.087 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:28.138 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:58:28.187 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:58:28.188 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:58:28.188 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:58:28.188 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:58:28.188 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:58:28.188 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:58:28.189 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:58:28.189 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:58:28.189 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:58:28.189 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:58:28.189 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:58:28.190 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:58:28.190 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:58:28.190 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:58:28.190 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:58:28.190 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:58:28.191 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:58:28.191 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:58:28.191 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:58:28.191 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:58:28.192 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:58:28.192 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:58:28.192 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:58:28.192 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:58:28.192 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:58:28.193 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:58:28.193 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:58:28.193 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:58:28.193 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:58:28.193 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:28.194 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:28.194 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:28.194 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:28.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:28.195 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:58:28.247 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:58:28.247 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-26T05:58:28.292 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:58:28.299 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:58:28.299 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:58:28.299 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:58:28.300 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:58:28.300 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:58:28.300 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:58:28.300 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:58:28.300 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:58:28.300 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:58:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:58:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:58:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:58:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:58:28.301 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:58:28.302 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:58:28.302 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:58:28.302 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:58:28.302 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:58:28.302 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:58:28.303 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:58:28.303 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:58:28.303 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:58:28.303 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:58:28.303 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:58:28.304 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:58:28.304 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:58:28.304 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:58:28.304 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:58:28.304 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:58:28.305 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:28.305 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:28.305 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:28.305 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:28.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:28.306 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:58:28.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:58:28.367+0000 7f6de16ec580 -1 init, newargv = 0x5622d7c06250 newargc=15 2023-10-26T05:58:28.376 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[54812]: starting ceph client 2023-10-26T05:58:28.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[54812]: starting fuse 2023-10-26T05:58:28.621 INFO:tasks.ceph.mds.c.smithi071.stderr:2023-10-26T05:58:28.611+0000 7fa3735cf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:58:29.455 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:58:29.463 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2023-10-26T05:58:29.464 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2023-10-26T05:58:29.464 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2023-10-26T05:58:29.464 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2023-10-26T05:58:29.464 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2023-10-26T05:58:29.465 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2023-10-26T05:58:29.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2023-10-26T05:58:29.465 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2023-10-26T05:58:29.465 INFO:teuthology.orchestra.run.smithi071.stdout:/run/lock 2023-10-26T05:58:29.465 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2023-10-26T05:58:29.466 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2023-10-26T05:58:29.466 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2023-10-26T05:58:29.466 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:58:29.466 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2023-10-26T05:58:29.466 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2023-10-26T05:58:29.467 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2023-10-26T05:58:29.467 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2023-10-26T05:58:29.467 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/fs/nfsd 2023-10-26T05:58:29.467 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2023-10-26T05:58:29.468 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2023-10-26T05:58:29.468 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:58:29.468 INFO:teuthology.orchestra.run.smithi071.stdout:/run/rpc_pipefs 2023-10-26T05:58:29.468 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2023-10-26T05:58:29.468 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2023-10-26T05:58:29.469 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-0 2023-10-26T05:58:29.469 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-1 2023-10-26T05:58:29.469 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-2 2023-10-26T05:58:29.469 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph/osd/ceph-3 2023-10-26T05:58:29.470 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2023-10-26T05:58:29.470 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:29.470 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:58:29.470 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:29.470 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:58:29.471 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:29.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:29.472 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2023-10-26T05:58:29.519 INFO:teuthology.orchestra.run.smithi071.stdout:44 2023-10-26T05:58:29.519 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2023-10-26T05:58:29.520 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:58:29.520 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2023-10-26T05:58:29.520 DEBUG:teuthology.orchestra.run.smithi071:> import re 2023-10-26T05:58:29.521 DEBUG:teuthology.orchestra.run.smithi071:> import os 2023-10-26T05:58:29.521 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2023-10-26T05:58:29.521 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:58:29.522 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2023-10-26T05:58:29.522 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-26T05:58:29.523 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2023-10-26T05:58:29.523 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-26T05:58:29.523 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:58:29.524 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2023-10-26T05:58:29.524 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2023-10-26T05:58:29.525 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2023-10-26T05:58:29.525 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2023-10-26T05:58:29.525 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:58:29.526 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2023-10-26T05:58:29.526 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:58:29.526 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:58:29.527 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:58:29.527 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2023-10-26T05:58:29.528 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2023-10-26T05:58:29.528 DEBUG:teuthology.orchestra.run.smithi071:> return f 2023-10-26T05:58:29.528 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:58:29.529 DEBUG:teuthology.orchestra.run.smithi071:> 2023-10-26T05:58:29.529 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.0")) 2023-10-26T05:58:29.530 DEBUG:teuthology.orchestra.run.smithi071:> ' 2023-10-26T05:58:29.641 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.0.54812.asok 2023-10-26T05:58:29.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:29.652 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54812.asok 2023-10-26T05:58:29.653 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:29.653 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.0.54812.asok status 2023-10-26T05:58:29.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:29.842 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:29.843 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2023-10-26T05:58:29.843 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:58:29.843 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:58:29.843 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "0", 2023-10-26T05:58:29.844 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2023-10-26T05:58:29.844 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-26T05:58:29.844 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "54812", 2023-10-26T05:58:29.844 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2023-10-26T05:58:29.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:29.845 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2023-10-26T05:58:29.845 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2023-10-26T05:58:29.845 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 9552, 2023-10-26T05:58:29.845 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2023-10-26T05:58:29.845 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2023-10-26T05:58:29.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2023-10-26T05:58:29.846 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 9552 2023-10-26T05:58:29.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:29.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:58:29.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:29.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:58:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2029846625 2023-10-26T05:58:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2023-10-26T05:58:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:29.847 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:58:29.848 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2029846625 2023-10-26T05:58:29.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:29.848 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.9552 192.168.0.1:0/2029846625", 2023-10-26T05:58:29.848 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/2029846625", 2023-10-26T05:58:29.848 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2023-10-26T05:58:29.849 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 71, 2023-10-26T05:58:29.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 188, 2023-10-26T05:58:29.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:58:29.849 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2023-10-26T05:58:29.849 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2023-10-26T05:58:29.849 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:29.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:29.864 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:29.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:58:29.864 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:29.907 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:29.908 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2023-10-26T05:58:29.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:29.915 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:29.915 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:29.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-26T05:58:29.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:30.001 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:30.007 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-26T05:58:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 -- 172.21.15.71:0/390668847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1540b2890 msgr2=0x7ff1540caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 --2- 172.21.15.71:0/390668847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1540b2890 0x7ff1540caad0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7ff15406f190 tx=0x7ff14401c7c0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.178 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 -- 172.21.15.71:0/390668847 shutdown_connections 2023-10-26T05:58:30.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 --2- 172.21.15.71:0/390668847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1540b2890 0x7ff1540caad0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 --2- 172.21.15.71:0/390668847 >> v2:172.21.15.86:3300/0 conn(0x7ff15413c2f0 0x7ff1540b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 -- 172.21.15.71:0/390668847 >> 172.21.15.71:0/390668847 conn(0x7ff154069770 msgr2=0x7ff154069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:30.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 -- 172.21.15.71:0/390668847 shutdown_connections 2023-10-26T05:58:30.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.167+0000 7ff15abc5640 1 -- 172.21.15.71:0/390668847 wait complete. 2023-10-26T05:58:30.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 Processor -- start 2023-10-26T05:58:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- start start 2023-10-26T05:58:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1540b2890 0x7ff15406aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1540cb010 0x7ff15414ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 0x7ff15415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:30.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406a0f0 con 0x7ff15413c2f0 2023-10-26T05:58:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1540b2050 con 0x7ff1540b2890 2023-10-26T05:58:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406a3b0 con 0x7ff1540cb010 2023-10-26T05:58:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 0x7ff15415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 0x7ff15415c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47100/0 (socket says 172.21.15.71:47100) 2023-10-26T05:58:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 -- 172.21.15.71:0/4081176914 learned_addr learned my addr 172.21.15.71:0/4081176914 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:30.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff158961640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1540b2890 0x7ff15406aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff153fff640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1540cb010 0x7ff15414ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 -- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1540cb010 msgr2=0x7ff15414ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1540cb010 0x7ff15414ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 -- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1540b2890 msgr2=0x7ff15406aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:30.184 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1540b2890 0x7ff15406aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 -- 172.21.15.71:0/4081176914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440094a0 con 0x7ff15413c2f0 2023-10-26T05:58:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff159162640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 0x7ff15415c220 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c410 tx=0x7ff13c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c011790 con 0x7ff15413c2f0 2023-10-26T05:58:30.185 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7ff13c0119e0 con 0x7ff15413c2f0 2023-10-26T05:58:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15406a690 con 0x7ff15413c2f0 2023-10-26T05:58:30.186 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c011df0 con 0x7ff15413c2f0 2023-10-26T05:58:30.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15414f880 con 0x7ff15413c2f0 2023-10-26T05:58:30.187 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.171+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff154066b10 con 0x7ff15413c2f0 2023-10-26T05:58:30.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.175+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff13c01c460 con 0x7ff15413c2f0 2023-10-26T05:58:30.188 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.175+0000 7ff1517fa640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:30.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.175+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff13c084000 con 0x7ff15413c2f0 2023-10-26T05:58:30.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.175+0000 7ff158961640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:30.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.175+0000 7ff158961640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005c50 tx=0x7ff14c005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:30.189 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.179+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7ff13c054f10 con 0x7ff15413c2f0 2023-10-26T05:58:30.381 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.371+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff15406a8d0 con 0x7ff15413c2f0 2023-10-26T05:58:30.382 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.371+0000 7ff1517fa640 1 -- 172.21.15.71:0/4081176914 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff13c007e60 con 0x7ff15413c2f0 2023-10-26T05:58:30.382 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-26T05:58:30.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 msgr2=0x7ff138068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:30.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff14c005c50 tx=0x7ff14c005be0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 msgr2=0x7ff15415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:30.384 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 0x7ff15415c220 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00c410 tx=0x7ff13c00c8d0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 shutdown_connections 2023-10-26T05:58:30.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7ff138066470 0x7ff138068930 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff15413c2f0 0x7ff15415c220 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1540cb010 0x7ff15414ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.385 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 --2- 172.21.15.71:0/4081176914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1540b2890 0x7ff15406aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:30.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 >> 172.21.15.71:0/4081176914 conn(0x7ff154069770 msgr2=0x7ff1540ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:30.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 shutdown_connections 2023-10-26T05:58:30.386 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:30.375+0000 7ff15abc5640 1 -- 172.21.15.71:0/4081176914 wait complete. 2023-10-26T05:58:30.399 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-26T05:58:30.399 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-26T05:58:30.399 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-26T05:58:30.400 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-26T05:58:30.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:30.400 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-26T05:58:30.401 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-26T05:58:30.401 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-26T05:58:30.401 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:58:30.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.402 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-26T05:58:30.410 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-26T05:58:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-26T05:58:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-26T05:58:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.410 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-26T05:58:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.411 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:58:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:30.411 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-26T05:58:30.411 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:58:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-26T05:58:30.412 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-26T05:58:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:30.412 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-26T05:58:30.412 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-26T05:58:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41860sec preferred_lft 41860sec 2023-10-26T05:58:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-26T05:58:30.413 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.413 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-26T05:58:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:bd:4a:1f:42 brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-26T05:58:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.414 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-26T05:58:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-26T05:58:30.414 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-26T05:58:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-26T05:58:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.415 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:58:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:30.415 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-26T05:58:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.416 INFO:teuthology.orchestra.run.smithi086.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-26T05:58:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-10-26T05:58:30.416 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-26T05:58:30.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.417 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-26T05:58:30.463 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-26T05:58:30.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.465 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-26T05:58:30.514 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-26T05:58:30.514 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-26T05:58:30.514 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-26T05:58:30.515 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-26T05:58:30.515 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-26T05:58:30.515 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-26T05:58:30.515 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:58:30.515 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:58:30.515 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-26T05:58:30.516 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-26T05:58:30.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.516 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-26T05:58:30.517 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-26T05:58:30.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-26T05:58:30.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-26T05:58:30.518 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-26T05:58:30.518 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-26T05:58:30.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.587 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:58:30.635 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:58:30.635 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:58:30.636 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:58:30.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-26T05:58:30.691 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-26T05:58:30.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:58:30.691 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:30.742 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-26T05:58:30.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-26T05:58:30.743 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:30.787 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:58:30.836 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:58:30.836 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:58:30.837 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:58:30.837 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:58:30.837 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:58:30.837 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:58:30.837 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:58:30.838 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:58:30.838 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:58:30.838 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:58:30.838 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:58:30.838 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:58:30.839 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:58:30.839 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:58:30.839 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:58:30.839 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:58:30.839 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:58:30.839 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:58:30.840 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:58:30.840 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:58:30.840 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:58:30.840 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:58:30.840 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:58:30.841 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:58:30.841 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:58:30.841 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:58:30.841 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:58:30.841 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:58:30.842 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:58:30.842 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:30.842 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:30.842 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:30.842 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:30.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.844 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:58:30.891 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-26T05:58:30.892 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-26T05:58:30.936 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:58:30.943 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:58:30.943 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:58:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:58:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:58:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:58:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:58:30.944 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:58:30.945 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:58:30.945 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:58:30.945 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:58:30.945 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:58:30.945 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:58:30.946 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:58:30.946 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:58:30.946 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:58:30.946 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:58:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:58:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:58:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:58:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:58:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:58:30.947 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:58:30.948 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:58:30.948 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:58:30.948 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:58:30.948 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:58:30.948 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:58:30.949 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:58:30.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:58:30.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:30.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:30.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:30.949 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:30.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:30.951 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:58:31.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:58:31.020+0000 7f17bd2ce580 -1 init, newargv = 0x55f6181a71e0 newargc=15 2023-10-26T05:58:31.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35525]: starting ceph client 2023-10-26T05:58:31.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35525]: starting fuse 2023-10-26T05:58:32.103 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-26T05:58:32.112 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-26T05:58:32.112 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-26T05:58:32.112 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-26T05:58:32.113 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-26T05:58:32.113 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-26T05:58:32.113 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-26T05:58:32.113 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-26T05:58:32.113 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-26T05:58:32.114 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-26T05:58:32.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-26T05:58:32.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-26T05:58:32.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-26T05:58:32.114 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-26T05:58:32.115 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-26T05:58:32.115 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-26T05:58:32.115 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-26T05:58:32.115 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-26T05:58:32.115 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-26T05:58:32.116 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-26T05:58:32.116 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-26T05:58:32.116 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-26T05:58:32.116 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-26T05:58:32.116 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-26T05:58:32.117 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-26T05:58:32.117 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-26T05:58:32.117 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-26T05:58:32.117 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-26T05:58:32.117 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-26T05:58:32.118 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-26T05:58:32.118 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:32.118 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:58:32.118 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:32.118 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:58:32.119 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:32.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:32.120 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-26T05:58:32.168 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-26T05:58:32.168 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-26T05:58:32.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-26T05:58:32.169 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-26T05:58:32.169 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-26T05:58:32.169 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-26T05:58:32.170 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-26T05:58:32.170 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:58:32.173 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-26T05:58:32.173 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-26T05:58:32.174 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-26T05:58:32.174 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-26T05:58:32.175 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:58:32.175 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-26T05:58:32.175 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-26T05:58:32.176 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-26T05:58:32.176 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-26T05:58:32.176 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:58:32.177 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-26T05:58:32.177 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-26T05:58:32.178 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-26T05:58:32.178 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-26T05:58:32.178 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-26T05:58:32.179 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-26T05:58:32.179 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-26T05:58:32.179 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-26T05:58:32.180 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-26T05:58:32.180 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-26T05:58:32.181 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-26T05:58:32.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.35525.asok 2023-10-26T05:58:32.294 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:32.301 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35525.asok 2023-10-26T05:58:32.302 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:32.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.1.35525.asok status 2023-10-26T05:58:32.390 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:32.491 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-26T05:58:32.492 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-26T05:58:32.492 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "384993681abceb8cb8affea10538d607052db4b4", 2023-10-26T05:58:32.492 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6863-g38499368 (384993681abceb8cb8affea10538d607052db4b4) reef (dev)", 2023-10-26T05:58:32.492 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-26T05:58:32.492 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-26T05:58:32.493 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-26T05:58:32.493 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "35525", 2023-10-26T05:58:32.493 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-26T05:58:32.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:58:32.493 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-26T05:58:32.494 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-26T05:58:32.494 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7036, 2023-10-26T05:58:32.494 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-26T05:58:32.494 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-26T05:58:32.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-26T05:58:32.495 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 7036 2023-10-26T05:58:32.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:58:32.495 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:58:32.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:58:32.495 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:58:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2170026516 2023-10-26T05:58:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-26T05:58:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:58:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-26T05:58:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-26T05:58:32.497 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-26T05:58:32.497 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2170026516 2023-10-26T05:58:32.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-26T05:58:32.497 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.7036 192.168.0.1:0/2170026516", 2023-10-26T05:58:32.497 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2170026516", 2023-10-26T05:58:32.497 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-26T05:58:32.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 71, 2023-10-26T05:58:32.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 188, 2023-10-26T05:58:32.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-26T05:58:32.498 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-26T05:58:32.499 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-26T05:58:32.499 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-26T05:58:32.503 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:32.511 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:32.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-10-26T05:58:32.511 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:32.603 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-26T05:58:32.603 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:32.610 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:32.610 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:58:32.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-26T05:58:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:58:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:58:32.667 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:32.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/1660127359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 msgr2=0x7f9f60005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:32.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 --2- 172.21.15.71:0/1660127359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f60005190 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009e30 tx=0x7f9f6c01d150 comp rx=0 tx=0).stop 2023-10-26T05:58:32.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a832640 1 -- 172.21.15.71:0/1660127359 reap_dead start 2023-10-26T05:58:32.803 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/1660127359 shutdown_connections 2023-10-26T05:58:32.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/1660127359 >> 172.21.15.71:0/1660127359 conn(0x7f9f740cdd80 msgr2=0x7f9f740ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:32.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/1660127359 shutdown_connections 2023-10-26T05:58:32.804 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/1660127359 wait complete. 2023-10-26T05:58:32.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 Processor -- start 2023-10-26T05:58:32.806 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- start start 2023-10-26T05:58:32.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f7414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:32.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f7415d370 0x7f9f740c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:32.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f741528d0 0x7f9f74152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:32.807 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7406a650 con 0x7f9f60004d10 2023-10-26T05:58:32.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74151500 con 0x7f9f7415d370 2023-10-26T05:58:32.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7415d850 con 0x7f9f741528d0 2023-10-26T05:58:32.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f7414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:32.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f7414f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47142/0 (socket says 172.21.15.71:47142) 2023-10-26T05:58:32.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 -- 172.21.15.71:0/437996443 learned_addr learned my addr 172.21.15.71:0/437996443 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:32.808 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a832640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f741528d0 0x7f9f74152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:32.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f79830640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f7415d370 0x7f9f740c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:32.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 -- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f741528d0 msgr2=0x7f9f74152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:32.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f741528d0 0x7f9f74152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 -- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f7415d370 msgr2=0x7f9f740c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:32.809 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f7415d370 0x7f9f740c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 -- 172.21.15.71:0/437996443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7415db30 con 0x7f9f60004d10 2023-10-26T05:58:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a832640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f741528d0 0x7f9f74152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7a031640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f7414f320 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e590 tx=0x7f9f7000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f700129d0 con 0x7f9f60004d10 2023-10-26T05:58:32.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9f70012c20 con 0x7f9f60004d10 2023-10-26T05:58:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7415de10 con 0x7f9f60004d10 2023-10-26T05:58:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70011430 con 0x7f9f60004d10 2023-10-26T05:58:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.795+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f74154c70 con 0x7f9f60004d10 2023-10-26T05:58:32.811 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.799+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f74066b10 con 0x7f9f60004d10 2023-10-26T05:58:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.799+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9f70011590 con 0x7f9f60004d10 2023-10-26T05:58:32.812 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.799+0000 7f9f6affd640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f58066470 0x7f9f58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.799+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f70087b30 con 0x7f9f60004d10 2023-10-26T05:58:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.799+0000 7f9f79830640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f58066470 0x7f9f58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.799+0000 7f9f79830640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f58066470 0x7f9f58068930 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f64006820 tx=0x7f9f640067b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:32.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.803+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9f70054fb0 con 0x7f9f60004d10 2023-10-26T05:58:32.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f74154e90 con 0x7f9f60004d10 2023-10-26T05:58:32.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f6affd640 1 -- 172.21.15.71:0/437996443 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f9f70021090 con 0x7f9f60004d10 2023-10-26T05:58:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:32.992 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:58:32.992 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:58:32.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f58066470 msgr2=0x7f9f58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:32.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f58066470 0x7f9f58068930 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f64006820 tx=0x7f9f640067b0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 msgr2=0x7f9f7414f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:32.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f7414f320 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e590 tx=0x7f9f7000ea50 comp rx=0 tx=0).stop 2023-10-26T05:58:32.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 shutdown_connections 2023-10-26T05:58:32.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9f741528d0 0x7f9f74152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9f7415d370 0x7f9f740c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9f58066470 0x7f9f58068930 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 --2- 172.21.15.71:0/437996443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f60004d10 0x7f9f7414f320 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 >> 172.21.15.71:0/437996443 conn(0x7f9f740cdd80 msgr2=0x7f9f740b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:32.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 shutdown_connections 2023-10-26T05:58:32.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:32.983+0000 7f9f7c295640 1 -- 172.21.15.71:0/437996443 wait complete. 2023-10-26T05:58:33.008 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:58:33.140 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":9387,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.426580960000001,"mdsmap_epoch":71,"osdmap_epoch":188,"osdmap_epoch_barrier":188,"uptime":15.237734963999999} 2023-10-26T05:58:33.140 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 9387, "mdsmap_epoch": 71, "osdmap_epoch": 188, "osdmap_epoch_barrier": 188, "rank_uptime": 10.42658096, "state": "up:active", "uptime": 15.237734964, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:58:33.141 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-10-26T05:58:33.269 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/600486482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 msgr2=0x7fd0dc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/600486482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0dc003da0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40cbce0 tx=0x7fd0e401cfb0 comp rx=0 tx=0).stop 2023-10-26T05:58:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0f8b63640 1 -- 172.21.15.71:0/600486482 reap_dead start 2023-10-26T05:58:33.270 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/600486482 shutdown_connections 2023-10-26T05:58:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/600486482 >> 172.21.15.71:0/600486482 conn(0x7fd0f40d13f0 msgr2=0x7fd0f40d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/600486482 shutdown_connections 2023-10-26T05:58:33.271 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.259+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/600486482 wait complete. 2023-10-26T05:58:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 Processor -- start 2023-10-26T05:58:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- start start 2023-10-26T05:58:33.274 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0f415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40c5c90 0x7fd0f40c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40c6b40 0x7fd0f4150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4153e90 con 0x7fd0dc003980 2023-10-26T05:58:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40c4a90 con 0x7fd0f40c6b40 2023-10-26T05:58:33.275 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40c7470 con 0x7fd0f40c5c90 2023-10-26T05:58:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0f415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0f415b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47160/0 (socket says 172.21.15.71:47160) 2023-10-26T05:58:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 -- 172.21.15.71:0/4090793414 learned_addr learned my addr 172.21.15.71:0/4090793414 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f8b63640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40c6b40 0x7fd0f4150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f37fe640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40c5c90 0x7fd0f40c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:33.276 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 -- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40c5c90 msgr2=0x7fd0f40c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:33.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40c5c90 0x7fd0f40c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:33.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 -- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40c6b40 msgr2=0x7fd0f4150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:33.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40c6b40 0x7fd0f4150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:33.277 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 -- 172.21.15.71:0/4090793414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f41537d0 con 0x7fd0dc003980 2023-10-26T05:58:33.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f37fe640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40c5c90 0x7fd0f40c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:33.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f3fff640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0f415b8b0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4019d90 tx=0x7fd0e4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:33.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4006350 con 0x7fd0dc003980 2023-10-26T05:58:33.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0e40065a0 con 0x7fd0dc003980 2023-10-26T05:58:33.278 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f4153ab0 con 0x7fd0dc003980 2023-10-26T05:58:33.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4003880 con 0x7fd0dc003980 2023-10-26T05:58:33.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f40c71a0 con 0x7fd0dc003980 2023-10-26T05:58:33.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f4066b10 con 0x7fd0dc003980 2023-10-26T05:58:33.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.263+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0e40039e0 con 0x7fd0dc003980 2023-10-26T05:58:33.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.267+0000 7fd0f0ff9640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0e00664a0 0x7fd0e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:33.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.267+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0e411d640 con 0x7fd0dc003980 2023-10-26T05:58:33.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.267+0000 7fd0f37fe640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0e00664a0 0x7fd0e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:33.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.267+0000 7fd0f37fe640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0e00664a0 0x7fd0e0068960 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005b70 tx=0x7fd0e80091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:33.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.271+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0e40eab50 con 0x7fd0dc003980 2023-10-26T05:58:33.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:33.411+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fd0f4153630 con 0x7fd0dc003980 2023-10-26T05:58:34.340 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.331+0000 7fd0f0ff9640 1 -- 172.21.15.71:0/4090793414 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v189) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd0f4153630 con 0x7fd0dc003980 2023-10-26T05:58:34.341 INFO:teuthology.orchestra.run.smithi071.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-10-26T05:58:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0e00664a0 msgr2=0x7fd0e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:34.344 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0e00664a0 0x7fd0e0068960 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8005b70 tx=0x7fd0e80091e0 comp rx=0 tx=0).stop 2023-10-26T05:58:34.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 msgr2=0x7fd0f415b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:34.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0f415b8b0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4019d90 tx=0x7fd0e4001d80 comp rx=0 tx=0).stop 2023-10-26T05:58:34.345 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 shutdown_connections 2023-10-26T05:58:34.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40c6b40 0x7fd0f4150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:34.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40c5c90 0x7fd0f40c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:34.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0e00664a0 0x7fd0e0068960 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:34.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 --2- 172.21.15.71:0/4090793414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc003980 0x7fd0f415b8b0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:34.346 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 >> 172.21.15.71:0/4090793414 conn(0x7fd0f40d13f0 msgr2=0x7fd0f4155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:34.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 shutdown_connections 2023-10-26T05:58:34.347 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.335+0000 7fd0fa5c6640 1 -- 172.21.15.71:0/4090793414 wait complete. 2023-10-26T05:58:34.363 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-10-26T05:58:34.497 INFO:teuthology.orchestra.run.smithi071.stdout:{"osd_mon_report_interval":"5"} 2023-10-26T05:58:34.498 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-10-26T05:58:34.498 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-10-26T05:58:34.499 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:34.499 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:34.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:34.505 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-10-26T05:58:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:16+0 records in 2023-10-26T05:58:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:16+0 records out 2023-10-26T05:58:34.702 INFO:teuthology.orchestra.run.smithi071.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.144008 s, 117 MB/s 2023-10-26T05:58:34.703 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:34.704 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:34.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:34.711 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-10-26T05:58:34.877 INFO:teuthology.orchestra.run.smithi071.stderr:17+0 records in 2023-10-26T05:58:34.877 INFO:teuthology.orchestra.run.smithi071.stderr:17+0 records out 2023-10-26T05:58:34.877 INFO:teuthology.orchestra.run.smithi071.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.109419 s, 163 MB/s 2023-10-26T05:58:34.878 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-10-26T05:58:34.879 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:35.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/828190110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac06d1b0 msgr2=0x7f18ac06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:35.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 --2- 172.21.15.71:0/828190110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac06d1b0 0x7f18ac06d5e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f18ac06bbc0 tx=0x7f18a401cc90 comp rx=0 tx=0).stop 2023-10-26T05:58:35.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/828190110 shutdown_connections 2023-10-26T05:58:35.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 --2- 172.21.15.71:0/828190110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac06d1b0 0x7f18ac06d5e0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 --2- 172.21.15.71:0/828190110 >> v2:172.21.15.86:3300/0 conn(0x7f18ac06b010 0x7f18ac06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/828190110 >> 172.21.15.71:0/828190110 conn(0x7f18ac0cddc0 msgr2=0x7f18ac06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:35.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/828190110 shutdown_connections 2023-10-26T05:58:35.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/828190110 wait complete. 2023-10-26T05:58:35.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 Processor -- start 2023-10-26T05:58:35.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- start start 2023-10-26T05:58:35.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18ac06b010 0x7f18ac133a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:35.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ac06d1b0 0x7f18ac1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:35.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 0x7f18ac136370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:35.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac14f9d0 con 0x7f18ac159090 2023-10-26T05:58:35.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac1372a0 con 0x7f18ac06d1b0 2023-10-26T05:58:35.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac148c60 con 0x7f18ac06b010 2023-10-26T05:58:35.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 0x7f18ac136370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:35.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 0x7f18ac136370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47176/0 (socket says 172.21.15.71:47176) 2023-10-26T05:58:35.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 -- 172.21.15.71:0/1322963223 learned_addr learned my addr 172.21.15.71:0/1322963223 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:35.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aa59c640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18ac06b010 0x7f18ac133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:35.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18a3fff640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ac06d1b0 0x7f18ac1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:35.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 -- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18ac06b010 msgr2=0x7f18ac133a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:35.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18ac06b010 0x7f18ac133a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 -- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ac06d1b0 msgr2=0x7f18ac1483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:35.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ac06d1b0 0x7f18ac1483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 -- 172.21.15.71:0/1322963223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40098e0 con 0x7f18ac159090 2023-10-26T05:58:35.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18a3fff640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ac06d1b0 0x7f18ac1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:35.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18aad9d640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 0x7f18ac136370 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f18a401a8c0 tx=0x7f18a40a5cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:35.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4004e60 con 0x7f18ac159090 2023-10-26T05:58:35.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f18a40ad6a0 con 0x7f18ac159090 2023-10-26T05:58:35.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ac15a050 con 0x7f18ac159090 2023-10-26T05:58:35.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40ada70 con 0x7f18ac159090 2023-10-26T05:58:35.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:34.999+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac136fc0 con 0x7f18ac159090 2023-10-26T05:58:35.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.003+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ac066d00 con 0x7f18ac159090 2023-10-26T05:58:35.016 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.003+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f18a40b4040 con 0x7f18ac159090 2023-10-26T05:58:35.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.003+0000 7f18a2ffd640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f188c066470 0x7f188c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:35.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.003+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18a411e000 con 0x7f18ac159090 2023-10-26T05:58:35.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.003+0000 7f18aa59c640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f188c066470 0x7f188c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:35.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.003+0000 7f18aa59c640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f188c066470 0x7f188c068930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098e0 tx=0x7f189c009060 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:35.021 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.007+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f18a40eb5c0 con 0x7f18ac159090 2023-10-26T05:58:35.156 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.147+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f18ac15bce0 con 0x7f18ac159090 2023-10-26T05:58:35.159 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18a2ffd640 1 -- 172.21.15.71:0/1322963223 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) v1 ==== 81+0+33930 (secure 0 0 0) 0x7f18a40b2090 con 0x7f18ac159090 2023-10-26T05:58:35.160 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:35.160 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:35.160 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 189, 2023-10-26T05:58:35.160 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:35.160 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:35.161 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:34.321864+0000", 2023-10-26T05:58:35.161 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:35.161 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:35.161 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:35.161 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:35.162 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:35.162 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:35.162 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:35.162 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:35.162 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:35.163 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:35.163 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:35.163 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:35.163 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:35.163 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:35.164 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:35.164 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:35.164 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:35.164 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:35.164 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:35.164 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:35.165 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:35.165 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:35.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.165 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:35.165 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:35.166 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:35.166 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:35.166 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:35.166 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:35.166 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:35.167 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:35.167 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:35.167 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:35.167 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:35.167 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:35.168 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:35.168 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:35.168 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:35.168 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:35.168 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:35.169 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:35.169 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:35.169 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:35.169 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:35.169 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:35.169 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:35.170 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:35.170 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:35.170 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:35.170 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:35.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.171 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:35.171 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:35.171 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:35.171 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:35.171 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:35.172 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:35.172 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:35.172 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:35.172 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:35.172 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:35.173 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:35.173 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:35.173 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:35.173 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:35.173 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:35.174 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:35.174 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:35.174 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:35.174 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:35.175 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:35.175 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:35.175 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:35.175 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:35.175 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:35.176 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:35.176 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:35.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:35.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.176 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:35.177 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:35.177 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:35.177 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:35.177 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:35.177 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:35.178 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:35.178 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:35.178 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:35.178 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:35.178 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:35.178 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:35.179 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:35.179 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:35.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.179 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:35.179 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:35.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.180 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:35.180 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:35.180 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:35.181 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:35.181 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:35.181 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:35.181 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:35.181 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:35.182 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:35.182 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.182 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.182 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:35.183 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:35.183 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:35.183 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:35.183 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:35.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:35.184 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:35.184 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:35.184 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:35.184 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:35.184 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:35.185 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:35.185 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:35.185 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:35.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:35.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:35.185 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:35.186 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:35.186 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:35.186 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:35.186 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:35.186 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:35.187 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:35.187 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:35.187 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:35.187 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:35.187 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:35.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.188 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:35.188 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:35.188 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:35.188 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:35.189 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:35.189 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:35.189 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:35.189 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:35.189 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:35.190 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:35.190 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:35.190 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:35.190 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:35.190 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:35.191 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:35.191 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:35.191 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:35.191 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:35.191 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:35.193 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:35.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:35.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.193 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:35.193 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:35.194 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:35.194 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:35.194 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:35.194 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:35.194 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:35.195 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:35.195 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:35.195 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:35.195 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:35.195 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:35.196 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:35.196 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:35.196 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:35.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.196 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:35.196 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:35.197 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:35.197 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.197 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:35.197 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:35.198 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:35.198 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:35.198 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:35.198 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:35.198 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:35.199 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:35.199 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:35.199 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.199 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.199 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:35.200 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:35.200 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:35.200 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:35.200 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:35.200 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:35.201 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:35.201 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:35.201 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:35.201 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:35.201 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:35.202 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:35.202 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:35.202 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:35.202 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:35.202 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:35.203 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:35.203 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:35.203 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:35.203 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:35.203 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:35.203 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:35.204 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:35.204 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:35.204 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:35.204 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:35.204 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:35.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.205 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "189", 2023-10-26T05:58:35.205 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:35.205 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:35.205 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:35.206 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:35.206 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:35.206 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:35.206 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:35.206 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:35.207 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:35.207 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:35.207 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:35.207 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:35.207 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:35.208 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:35.208 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:35.208 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:35.208 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:35.208 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:35.208 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:35.209 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:35.209 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:35.209 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:35.209 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:35.209 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:35.210 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:35.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:35.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.210 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:35.210 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:35.211 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:35.211 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:35.211 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:35.211 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:35.211 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:35.212 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:35.212 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:35.212 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:35.212 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:35.212 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:35.212 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:35.213 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:35.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.213 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:35.213 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:35.213 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:35.214 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.214 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.214 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:35.214 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:35.214 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:35.215 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:35.215 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:35.215 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:35.215 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:35.215 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:35.216 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:35.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.216 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.216 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:35.216 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:35.216 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.217 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.218 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.218 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.218 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.218 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.218 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.219 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.219 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.219 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.219 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.220 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:35.220 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.220 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.221 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:35.221 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.221 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.221 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.222 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.222 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.222 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:35.222 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.223 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.223 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.223 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:35.223 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.224 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.224 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.224 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.224 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.224 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.225 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:35.225 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.225 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.226 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.226 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:35.226 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.226 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.226 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.227 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.227 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.227 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.227 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:35.228 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.228 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.228 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:35.229 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:35.229 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.229 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.229 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:35.230 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:35.230 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:35.230 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:35.230 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.230 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.230 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.231 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.231 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.231 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:35.231 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.232 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.232 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.232 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.232 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.232 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.233 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.233 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.233 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.233 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.233 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.233 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.234 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.234 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:35.234 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.235 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.235 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:35.235 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.236 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.236 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.236 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:35.237 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.238 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:35.238 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.238 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.238 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.239 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.239 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.239 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:35.240 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:35.241 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.241 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.241 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.242 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.242 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.242 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.242 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:35.243 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.243 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.243 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.243 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:35.243 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:35.244 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.244 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.245 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:35.245 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:35.246 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:35.246 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:35.246 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.246 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.247 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.247 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:35.247 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.247 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.248 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.248 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.248 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.248 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.248 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.249 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.249 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.249 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.249 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.249 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:35.250 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.251 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.251 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:35.251 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.252 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.252 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:35.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:35.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.254 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.255 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:35.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:35.257 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.257 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.257 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.258 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:35.258 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.259 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:35.259 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:35.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.260 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.260 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.260 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:35.260 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:35.260 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:35.261 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:35.261 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.261 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.261 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.261 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.262 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:35.262 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.262 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.262 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.263 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.263 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.263 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.263 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.264 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.264 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.264 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.264 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.264 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:35.265 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:35.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.266 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.267 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.267 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.267 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:35.268 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:35.269 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.269 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:35.270 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.271 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.272 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.272 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.273 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:35.273 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.273 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.274 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.274 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:35.274 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:35.274 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:35.275 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:35.276 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.276 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.276 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.276 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.276 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.277 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:35.277 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.277 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.277 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.277 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.278 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.278 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.278 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.278 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.278 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.278 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.279 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.279 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.279 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.279 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.280 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:35.280 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.280 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.280 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.281 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:35.281 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.281 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.281 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.281 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.282 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.282 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.282 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:35.282 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.283 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.283 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:35.283 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.284 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.284 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.284 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.285 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:35.285 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.286 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:35.286 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.286 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.286 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.287 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:35.288 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.288 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:35.289 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:35.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.289 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.289 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:35.289 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:35.290 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:35.290 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:35.290 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.290 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.290 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.291 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.291 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:35.291 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.292 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.292 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.292 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.292 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.293 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.293 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.293 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.293 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.293 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.293 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.294 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.294 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:35.294 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.295 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:35.295 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.296 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.296 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.296 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.297 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.297 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:35.297 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.298 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:35.298 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.298 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.299 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.299 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.299 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:35.300 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.300 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:35.301 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.301 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.301 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.301 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.302 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.302 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:35.302 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.303 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.303 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.303 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:35.303 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:35.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.304 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.304 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:35.304 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:35.304 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:35.304 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:35.305 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.305 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.305 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.305 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.306 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.306 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:35.306 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.306 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.306 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.306 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.307 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.307 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.307 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.307 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.307 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.308 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.308 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.308 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.308 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.309 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:35.309 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.310 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.310 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:35.310 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.310 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.311 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.311 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.311 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.311 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:35.312 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.312 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:35.313 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.313 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.313 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.313 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.313 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.314 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.314 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:35.314 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.315 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.315 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:35.315 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.316 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.316 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.316 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.316 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.317 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:35.317 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.318 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:35.318 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:35.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.318 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.318 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:35.319 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:35.319 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:35.319 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:35.319 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.319 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.320 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.320 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.320 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:35.321 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:35.321 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:35.321 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:35.321 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:35.321 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:35.321 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:35.322 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:35.322 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:35.322 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:35.322 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:35.322 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:35.323 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:35.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:35.324 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.324 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:35.324 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.325 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.325 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:35.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.326 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:35.326 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.327 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.327 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:35.327 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.328 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.328 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:35.328 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.329 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:35.329 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.330 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:35.330 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.330 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.331 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:35.331 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:35.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.331 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:35.331 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:35.332 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.332 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:35.332 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:35.332 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:35.333 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.333 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.333 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:35.333 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:35.334 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:35.334 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:35.334 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:35.334 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:35.334 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:35.335 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:35.335 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.335 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:35.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:35.335 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.335 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:35.336 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.336 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.336 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.336 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.336 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.337 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:35.337 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.337 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:35.338 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.338 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.338 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.338 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.338 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.339 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:35.339 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.339 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:35.339 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.340 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.340 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.340 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.340 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.340 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:35.341 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.341 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.341 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:35.341 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.342 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.342 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.342 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.342 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.342 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:35.343 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.343 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:35.343 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:35.344 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.345 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:35.345 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.345 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.346 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.346 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.346 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.346 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:35.346 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.347 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.347 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:35.347 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.347 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.348 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.348 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.348 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.348 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:35.348 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:35.349 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:35.349 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:35.349 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:35.349 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:35.350 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:35.350 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:35.350 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:35.350 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:35.350 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.351 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:35.351 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:35.351 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:35.351 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:35.351 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:35.352 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:35.352 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:35.352 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:35.352 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:35.352 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:35.352 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:35.353 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:35.353 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:35.353 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:35.353 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:35.353 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:35.354 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:35.354 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:35.354 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:35.354 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:35.355 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:35.355 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:35.355 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:35.355 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:35.355 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:35.355 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:35.356 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:35.356 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:35.356 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:35.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f188c066470 msgr2=0x7f188c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:35.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f188c066470 0x7f188c068930 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f189c0098e0 tx=0x7f189c009060 comp rx=0 tx=0).stop 2023-10-26T05:58:35.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 msgr2=0x7f18ac136370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:35.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 0x7f18ac136370 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f18a401a8c0 tx=0x7f18a40a5cc0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 shutdown_connections 2023-10-26T05:58:35.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f188c066470 0x7f188c068930 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18ac159090 0x7f18ac136370 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18ac06d1b0 0x7f18ac1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 --2- 172.21.15.71:0/1322963223 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f18ac06b010 0x7f18ac133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:35.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 >> 172.21.15.71:0/1322963223 conn(0x7f18ac0cddc0 msgr2=0x7f18ac0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:35.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 shutdown_connections 2023-10-26T05:58:35.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:35.151+0000 7f18b084b640 1 -- 172.21.15.71:0/1322963223 wait complete. 2023-10-26T05:58:35.361 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:58:40.362 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:40.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- 172.21.15.71:0/3277900730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c0cb1e0 msgr2=0x7f6a9c159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:40.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/3277900730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c0cb1e0 0x7f6a9c159810 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009bc0 tx=0x7f6a9401cb80 comp rx=0 tx=0).stop 2023-10-26T05:58:40.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- 172.21.15.71:0/3277900730 shutdown_connections 2023-10-26T05:58:40.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/3277900730 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c0cb1e0 0x7f6a9c159810 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/3277900730 >> v2:172.21.15.86:3300/0 conn(0x7f6a9c13c2f0 0x7f6a9c0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- 172.21.15.71:0/3277900730 >> 172.21.15.71:0/3277900730 conn(0x7f6a9c069770 msgr2=0x7f6a9c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:40.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- 172.21.15.71:0/3277900730 shutdown_connections 2023-10-26T05:58:40.501 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- 172.21.15.71:0/3277900730 wait complete. 2023-10-26T05:58:40.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 Processor -- start 2023-10-26T05:58:40.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- start start 2023-10-26T05:58:40.503 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c0b2890 0x7f6a9c153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:40.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 0x7f6a9c13f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:40.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c13c2f0 0x7f6a9c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:40.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c13c8c0 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c163e00 con 0x7f6a9c0b2890 2023-10-26T05:58:40.504 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.491+0000 7f6aa3f00640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c13cb60 con 0x7f6a9c13c2f0 2023-10-26T05:58:40.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 0x7f6a9c13f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:40.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 0x7f6a9c13f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47180/0 (socket says 172.21.15.71:47180) 2023-10-26T05:58:40.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 -- 172.21.15.71:0/2796805080 learned_addr learned my addr 172.21.15.71:0/2796805080 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:40.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa249d640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c13c2f0 0x7f6a9c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:40.505 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa1c9c640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c0b2890 0x7f6a9c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:40.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 -- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c13c2f0 msgr2=0x7f6a9c069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:40.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c13c2f0 0x7f6a9c069b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 -- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c0b2890 msgr2=0x7f6a9c153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:40.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c0b2890 0x7f6a9c153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 -- 172.21.15.71:0/2796805080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a94009820 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa149b640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 0x7f6a9c13f1b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c430 tx=0x7f6a8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:40.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c004230 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a8c0044a0 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c13ce40 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c003840 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c13d400 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c066b10 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6a8c0039a0 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6a927fc640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a80066470 0x7f6a80068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:40.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a8c083770 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.495+0000 7f6aa1c9c640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a80066470 0x7f6a80068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:40.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.499+0000 7f6aa1c9c640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a80066470 0x7f6a80068930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009ad0 tx=0x7f6a980091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:40.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.499+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6a8c054600 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.666 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.659+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6a9c0721e0 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.659+0000 7f6a927fc640 1 -- 172.21.15.71:0/2796805080 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) v1 ==== 81+0+33930 (secure 0 0 0) 0x7f6a8c054190 con 0x7f6a9c0cb1e0 2023-10-26T05:58:40.669 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:40.670 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:40.670 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 189, 2023-10-26T05:58:40.670 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:40.670 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:40.671 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:34.321864+0000", 2023-10-26T05:58:40.671 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:40.671 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:40.671 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:40.671 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:40.672 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:40.672 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:40.672 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:40.672 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:40.672 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:40.673 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:40.673 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:40.673 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:40.673 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:40.674 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:40.674 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:40.674 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:40.674 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:40.674 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:40.675 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:40.675 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:40.675 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:40.675 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:40.675 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.675 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:40.676 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:40.676 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:40.676 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:40.676 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:40.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:40.677 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:40.677 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:40.677 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:40.677 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:40.677 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:40.677 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:40.678 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:40.678 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:40.678 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:40.678 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:40.678 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:40.678 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:40.679 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:40.679 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:40.679 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:40.679 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:40.679 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:40.680 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:40.680 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:40.680 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:40.680 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:40.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.680 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:40.681 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:40.681 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:40.681 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:40.681 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:40.681 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:40.682 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:40.682 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:40.682 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:40.682 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:40.682 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:40.682 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:40.683 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:40.683 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:40.683 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:40.683 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:40.683 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:40.684 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:40.684 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:40.684 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:40.684 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:40.684 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:40.685 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:40.685 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:40.685 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:40.685 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:40.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:40.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.686 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:40.686 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:40.686 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:40.686 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:40.687 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:40.687 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:40.687 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:40.687 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:40.687 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:40.688 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:40.688 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:40.688 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:40.688 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:40.688 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:40.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.689 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:40.689 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:40.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.689 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:40.690 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:40.690 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:40.690 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:40.690 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:40.691 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:40.691 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:40.691 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:40.691 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:40.691 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.692 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.692 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.692 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:40.692 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:40.692 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:40.693 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:40.693 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:40.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:40.693 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:40.693 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:40.694 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:40.694 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:40.694 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:40.694 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:40.694 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:40.695 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:40.695 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:40.695 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:40.695 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:40.695 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:40.696 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:40.696 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:40.696 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:40.696 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:40.696 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:40.697 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:40.697 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:40.697 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:40.697 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:40.697 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.698 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:40.698 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:40.698 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:40.698 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:40.698 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:40.699 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:40.699 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:40.699 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:40.699 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:40.699 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:40.700 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:40.700 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:40.700 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:40.700 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:40.700 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:40.701 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:40.702 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:40.702 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:40.702 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:40.702 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:40.702 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:40.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:40.703 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.703 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:40.703 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:40.703 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:40.704 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:40.704 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:40.704 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:40.704 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:40.704 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:40.705 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:40.706 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:40.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.706 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:40.706 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:40.707 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:40.707 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.707 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:40.707 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:40.707 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:40.708 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:40.708 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:40.708 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:40.708 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:40.708 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:40.709 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:40.709 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.709 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.709 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:40.710 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:40.710 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:40.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:40.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:40.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:40.710 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:40.711 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:40.711 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:40.711 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:40.711 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:40.711 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:40.712 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:40.712 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:40.712 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:40.712 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:40.712 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:40.713 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:40.713 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:40.713 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:40.713 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:40.713 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:40.714 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:40.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.715 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "189", 2023-10-26T05:58:40.715 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:40.715 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:40.715 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:40.716 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:40.716 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:40.716 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:40.716 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:40.716 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:40.716 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:40.717 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:40.717 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:40.717 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:40.717 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:40.717 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:40.717 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:40.718 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:40.718 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:40.718 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:40.718 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:40.718 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:40.719 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:40.719 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:40.719 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:40.719 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:40.719 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:40.720 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:40.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.720 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:40.720 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:40.720 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:40.720 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:40.721 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:40.721 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:40.721 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:40.721 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:40.721 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:40.722 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:40.722 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:40.722 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:40.722 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:40.722 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:40.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.723 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:40.723 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:40.723 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:40.723 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.724 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:40.724 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:40.724 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:40.724 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:40.724 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.725 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.726 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:40.726 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:40.726 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.726 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:40.726 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.727 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.727 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.727 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.727 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.727 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.728 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.728 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.728 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.728 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.728 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.728 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.730 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.730 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:40.730 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.730 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.731 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.731 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.731 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.732 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:40.732 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.732 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.733 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.733 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:40.733 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.733 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.734 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.734 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.734 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:40.735 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.735 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.735 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.735 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:40.736 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.736 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.736 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.736 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.737 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.737 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:40.737 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.738 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.738 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:40.738 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:40.738 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.739 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.739 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:40.739 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:40.739 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:40.740 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:40.740 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.740 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.740 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.740 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.741 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.741 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:40.741 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.742 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.742 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.742 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.742 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.742 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.743 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.743 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.743 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.743 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.743 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.743 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.744 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.744 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.744 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:40.744 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.745 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.745 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:40.745 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.745 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.746 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.746 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.746 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.746 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.747 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:40.747 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.747 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.748 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:40.748 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.748 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.748 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.749 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.749 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.749 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.749 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.749 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:40.749 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.750 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.750 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:40.750 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.751 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.751 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.751 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.752 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:40.752 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.752 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.752 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.753 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:40.753 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:40.753 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.753 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.753 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:40.754 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:40.754 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:40.754 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:40.754 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.755 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.755 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.755 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.755 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.756 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:40.756 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.756 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.756 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.756 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.756 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.757 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.757 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.757 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.757 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.758 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.759 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:40.759 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.759 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.759 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:40.760 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.760 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.760 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.760 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.761 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:40.761 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.761 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.762 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.762 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:40.762 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.762 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.762 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.763 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.763 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.763 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.763 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:40.764 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.764 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.764 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:40.764 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.765 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.765 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.765 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.765 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.766 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.766 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.766 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:40.766 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.767 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.767 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:40.767 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:40.767 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.767 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.768 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:40.768 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:40.768 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:40.769 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:40.769 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.769 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.769 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.769 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.770 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.770 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:40.770 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.770 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.770 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.771 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.771 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.771 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.771 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.771 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.771 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.772 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.772 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.772 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.772 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.772 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.773 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.773 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:40.773 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.773 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.773 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.774 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:40.774 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.774 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.775 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.775 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.775 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.775 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:40.776 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.776 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:40.776 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.777 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.777 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.777 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.777 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.777 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.778 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:40.778 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.778 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.779 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:40.779 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.779 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.779 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.780 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.780 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.780 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.780 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:40.780 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.781 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:40.781 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:40.782 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.782 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.782 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:40.782 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:40.783 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:40.783 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:40.783 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.783 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.783 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.783 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.784 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.785 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.785 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.785 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.785 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.785 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.786 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.786 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.786 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.786 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.786 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.787 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.787 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:40.787 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.788 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.788 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:40.788 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.788 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.788 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.789 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.789 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.789 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.789 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:40.790 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.790 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.790 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.790 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:40.791 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.791 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.791 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.791 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.792 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.792 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.792 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.792 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:40.792 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.793 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.793 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.793 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:40.793 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.793 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.794 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.794 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.794 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.794 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.795 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:40.795 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.795 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.795 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.796 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:40.796 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:40.796 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.796 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.797 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:40.797 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:40.797 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:40.797 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:40.797 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.797 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.798 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.798 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.798 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.798 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.798 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:40.799 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.799 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.799 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.799 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.799 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.800 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.800 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.800 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.800 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.800 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.802 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:40.802 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.803 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.803 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:40.803 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.803 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.803 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.804 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.804 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.804 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.804 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:40.805 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.805 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.805 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.805 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:40.805 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.806 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.806 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.806 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.806 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.806 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.807 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.807 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:40.807 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.807 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.808 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:40.808 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.808 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.808 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.809 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.809 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.809 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.809 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:40.809 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.810 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.810 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:40.810 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:40.811 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:40.812 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:40.812 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.812 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.812 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.812 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.813 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.813 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:40.813 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.813 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.813 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.813 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.814 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.814 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.814 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.814 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.814 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.814 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.815 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.815 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.815 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.815 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.815 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.816 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:40.816 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.816 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.816 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.816 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:40.817 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.817 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.817 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.818 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.818 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.818 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:40.818 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.819 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.819 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:40.819 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.819 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.820 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.820 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.820 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.820 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:40.821 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.821 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.821 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.821 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:40.821 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.822 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.822 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.822 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.822 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.823 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:40.823 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.823 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.824 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.824 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:40.824 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:40.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.824 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:40.825 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.826 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.826 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.826 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.827 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.827 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:40.827 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:40.827 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:40.827 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:40.827 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:40.828 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:40.828 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:40.828 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:40.828 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:40.828 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:40.828 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:40.829 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:40.829 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:40.829 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.829 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.829 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:40.830 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.830 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.830 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:40.830 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.831 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.831 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.831 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:40.831 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.831 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.832 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:40.832 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.832 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.833 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:40.833 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.833 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.834 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:40.834 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.834 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.834 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:40.835 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.835 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.835 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:40.835 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.836 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.836 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.836 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:40.836 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:40.836 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:40.837 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:40.837 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.837 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.837 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:40.838 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:40.838 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:40.838 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.838 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.839 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:40.839 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:40.839 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:40.839 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:40.839 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:40.840 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:40.840 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:40.840 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:40.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.840 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:40.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:40.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.841 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:40.841 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.841 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.842 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.842 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.842 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.842 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:40.842 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.843 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:40.843 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.843 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.844 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.844 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.844 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.844 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:40.844 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.845 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.845 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:40.845 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.845 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.845 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.846 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.846 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.846 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:40.846 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.847 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:40.847 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.847 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.847 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.847 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.848 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.848 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:40.848 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.848 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.849 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:40.849 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.849 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.849 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.849 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.851 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:40.851 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.851 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.851 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.851 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.851 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.852 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:40.852 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.852 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.852 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:40.852 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.853 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.853 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.853 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.853 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.853 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:40.854 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.854 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:40.854 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:40.854 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:40.855 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:40.855 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:40.855 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:40.855 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:40.855 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:40.855 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:40.856 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.856 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:40.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:40.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:40.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:40.857 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:40.857 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:40.857 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:40.857 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:40.857 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:40.857 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:40.858 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:40.859 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:40.860 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:40.860 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:40.860 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:40.860 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:40.860 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:40.861 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:40.861 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:40.861 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:40.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a80066470 msgr2=0x7f6a80068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:40.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a80066470 0x7f6a80068930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6a98009ad0 tx=0x7f6a980091e0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 msgr2=0x7f6a9c13f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:40.863 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 0x7f6a9c13f1b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00c430 tx=0x7f6a8c00c8f0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 shutdown_connections 2023-10-26T05:58:40.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a9c13c2f0 0x7f6a9c069b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a80066470 0x7f6a80068930 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a9c0cb1e0 0x7f6a9c13f1b0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 --2- 172.21.15.71:0/2796805080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a9c0b2890 0x7f6a9c153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:40.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 >> 172.21.15.71:0/2796805080 conn(0x7f6a9c069770 msgr2=0x7f6a9c137fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:40.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 shutdown_connections 2023-10-26T05:58:40.865 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:40.663+0000 7f6aa3f00640 1 -- 172.21.15.71:0/2796805080 wait complete. 2023-10-26T05:58:40.866 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:58:45.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:46.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 -- 172.21.15.71:0/301551526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 msgr2=0x7f38f40b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/301551526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 0x7f38f40b17d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009c80 tx=0x7f38ec01cd50 comp rx=0 tx=0).stop 2023-10-26T05:58:46.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 -- 172.21.15.71:0/301551526 shutdown_connections 2023-10-26T05:58:46.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/301551526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 0x7f38f40b17d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/301551526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38f4070fe0 0x7f38f4071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 -- 172.21.15.71:0/301551526 >> 172.21.15.71:0/301551526 conn(0x7f38f4069770 msgr2=0x7f38f4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:46.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 -- 172.21.15.71:0/301551526 shutdown_connections 2023-10-26T05:58:46.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 -- 172.21.15.71:0/301551526 wait complete. 2023-10-26T05:58:46.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 Processor -- start 2023-10-26T05:58:46.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.995+0000 7f38fb8ff640 1 -- start start 2023-10-26T05:58:46.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38f4070fe0 0x7f38f4157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 0x7f38f4133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 0x7f38f4134d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f415b5a0 con 0x7f38f4140cc0 2023-10-26T05:58:46.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4158be0 con 0x7f38f40b1360 2023-10-26T05:58:46.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f4157b70 con 0x7f38f4070fe0 2023-10-26T05:58:46.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 0x7f38f4134d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 0x7f38f4134d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41414/0 (socket says 172.21.15.71:41414) 2023-10-26T05:58:46.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 -- 172.21.15.71:0/442443096 learned_addr learned my addr 172.21.15.71:0/442443096 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:46.010 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f969b640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38f4070fe0 0x7f38f4157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f8e9a640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 0x7f38f4133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 -- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38f4070fe0 msgr2=0x7f38f4157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38f4070fe0 0x7f38f4157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 -- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 msgr2=0x7f38f4133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 0x7f38f4133080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 -- 172.21.15.71:0/442443096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0098e0 con 0x7f38f4140cc0 2023-10-26T05:58:46.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38f9e9c640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 0x7f38f4134d60 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009bd0 tx=0x7f38ec0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:46.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0ae730 con 0x7f38f4140cc0 2023-10-26T05:58:46.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f38ec0ae980 con 0x7f38f4140cc0 2023-10-26T05:58:46.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f4137980 con 0x7f38f4140cc0 2023-10-26T05:58:46.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0aed50 con 0x7f38f4140cc0 2023-10-26T05:58:46.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f4137f40 con 0x7f38f4140cc0 2023-10-26T05:58:46.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f4066b10 con 0x7f38f4140cc0 2023-10-26T05:58:46.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f38ec0b4040 con 0x7f38f4140cc0 2023-10-26T05:58:46.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38d8066470 0x7f38d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.014 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38ec11f2c0 con 0x7f38f4140cc0 2023-10-26T05:58:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:45.999+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f38d806dfa0 con 0x7f38f4140cc0 2023-10-26T05:58:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.003+0000 7f38f969b640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38d8066470 0x7f38d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.003+0000 7f38f969b640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38d8066470 0x7f38d8068930 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f38f4141d30 tx=0x7f38e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:46.015 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.003+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f38ec0ec750 con 0x7f38f4140cc0 2023-10-26T05:58:46.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.159+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f38f40721e0 con 0x7f38f4140cc0 2023-10-26T05:58:46.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.159+0000 7f38e9ffb640 1 -- 172.21.15.71:0/442443096 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) v1 ==== 81+0+33949 (secure 0 0 0) 0x7f38ec0b2030 con 0x7f38f4140cc0 2023-10-26T05:58:46.170 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:46.170 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 190, 2023-10-26T05:58:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:46.171 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:42.308638+0000", 2023-10-26T05:58:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:46.172 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.173 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:46.174 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:46.175 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:46.176 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:46.177 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:46.178 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:46.179 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:46.180 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:46.181 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:46.182 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:46.183 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:46.184 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:46.185 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:46.186 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.187 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:46.188 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:46.189 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.190 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:46.191 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:46.192 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.193 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:46.194 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:46.195 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:46.196 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:46.197 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:46.198 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:46.199 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:46.200 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:46.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:46.202 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:46.203 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:46.204 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:46.205 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:46.206 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:46.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:46.209 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:46.210 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:46.211 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.211 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.211 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:46.211 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:58:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:58:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:46.212 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:46.213 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:46.214 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:46.215 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:46.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "190", 2023-10-26T05:58:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:46.217 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:46.218 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:46.219 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:46.220 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:46.221 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.222 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:46.223 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:46.224 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:46.225 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:46.226 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:46.227 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.228 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.229 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.230 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.231 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.235 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.236 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.237 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.238 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.239 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:46.240 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:46.241 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:46.242 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.243 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.244 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.245 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.246 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.248 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.248 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:46.248 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.248 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.249 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.249 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.249 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.250 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:46.250 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.250 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.251 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:46.251 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.251 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.251 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.252 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.252 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.252 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.253 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:46.253 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.253 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.254 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:46.254 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.254 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.254 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.255 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.255 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.255 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:46.256 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.256 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.256 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:46.257 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.257 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.257 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.257 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:46.258 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:46.258 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:46.258 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:46.258 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.258 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.259 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.259 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.259 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.259 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:46.260 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.260 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.260 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.260 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.260 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.261 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.261 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.261 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.261 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.261 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.262 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.262 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.262 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.262 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.262 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.262 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:46.263 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.263 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.263 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.263 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:46.264 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.264 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.264 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.264 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.264 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.265 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.265 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:46.265 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.266 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.266 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:46.266 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.267 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.267 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.267 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.268 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:46.268 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.269 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:46.269 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.269 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.269 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.270 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.270 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.270 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.270 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.270 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.271 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:46.271 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.271 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.273 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:46.273 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:46.273 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:46.273 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:46.273 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.274 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.274 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.274 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.274 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.275 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:46.275 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.275 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.275 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.275 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.276 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.276 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.276 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.276 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.276 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.277 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.277 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.277 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.277 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.277 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.278 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:46.278 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.278 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.278 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.279 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:46.279 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.279 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.279 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.279 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.280 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.281 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.282 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.282 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:46.282 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.282 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.283 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:46.283 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.283 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.283 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.284 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.284 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.284 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.284 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:46.285 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.285 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.285 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.285 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:46.286 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.286 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.286 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.286 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.287 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.287 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:46.287 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.288 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.288 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.288 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:46.288 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.289 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.289 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.289 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:46.289 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:46.289 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:46.290 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:46.290 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.290 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.290 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.290 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.291 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:46.291 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.291 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.292 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.292 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.292 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.292 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.293 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.293 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.293 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.293 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.293 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.293 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.294 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.294 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:46.294 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.295 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:46.295 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.295 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.295 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.296 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.296 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.296 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.296 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.296 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:46.297 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.297 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.297 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:46.297 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.298 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.298 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.298 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.298 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.299 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.299 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.299 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:46.299 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.300 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.300 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:46.300 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.300 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.300 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.301 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.301 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.301 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.301 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.301 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:46.302 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.302 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.302 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:46.302 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.303 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.303 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.303 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.303 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:46.303 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:46.304 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:46.304 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:46.304 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.304 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.304 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.305 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.305 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.305 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:46.305 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.306 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.306 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.306 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.306 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.306 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.307 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.307 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.307 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.307 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.307 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.308 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.308 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.308 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.308 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.308 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:46.309 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.309 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.309 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:46.310 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.310 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.310 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.310 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.311 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.311 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.311 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:46.311 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.312 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.312 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:46.312 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.313 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.313 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.313 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.313 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.313 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.314 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.314 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:46.314 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.314 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.315 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:46.315 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.315 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.315 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.316 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.316 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.316 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.316 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.316 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:46.317 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.317 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.317 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:46.318 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.318 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.318 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:46.319 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:46.319 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:46.319 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:46.319 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.319 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.319 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.320 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.320 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.320 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:46.320 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.321 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.321 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.321 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.321 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.321 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.321 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.322 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.322 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.322 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.322 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.322 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.323 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.323 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.323 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:46.323 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.324 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.324 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:46.324 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.324 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.324 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.325 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.325 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.325 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.325 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:46.326 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.326 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.326 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:46.327 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.327 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.327 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.327 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.327 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.328 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.328 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:46.328 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.329 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.329 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:46.329 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.329 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.329 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.330 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.330 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.330 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.331 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:46.331 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.331 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.331 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.332 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:46.332 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.332 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.332 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.333 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:46.333 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:46.333 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:46.333 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:46.333 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.333 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.334 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.334 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.334 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.334 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:46.334 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.335 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.335 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.335 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.335 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.335 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.336 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.336 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.336 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.336 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.336 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.336 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.337 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.337 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.337 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.337 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:46.337 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.338 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.338 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.338 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:46.338 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.339 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.339 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.339 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.339 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.339 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.340 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:46.340 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.340 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.341 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:46.341 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.341 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.342 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.342 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.342 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:46.343 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.343 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.343 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.343 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:46.344 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.344 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.344 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.344 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.344 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.345 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:46.345 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.345 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.346 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:46.346 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.346 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.347 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.347 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:46.347 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:46.348 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:46.348 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:46.348 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.348 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.348 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.349 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.349 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.349 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.349 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:46.349 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.350 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:46.350 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.350 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.350 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.350 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.351 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.351 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:46.351 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.351 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.352 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:46.352 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.352 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.352 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.352 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.352 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.353 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:46.353 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.353 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.353 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.354 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.355 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:46.356 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.356 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.356 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.356 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.356 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.357 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:46.357 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.357 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.357 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:46.357 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.358 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.358 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.358 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.358 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.359 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:46.359 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.359 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.359 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:46.360 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.360 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.360 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.360 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.360 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.361 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:46.361 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.361 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.361 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:46.362 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.362 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.362 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.362 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.362 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.363 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:46.363 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.363 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:46.364 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.364 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.364 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.364 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.364 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.365 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:46.365 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.365 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.365 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.365 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:46.366 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:46.366 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:46.366 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:46.366 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:46.366 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:46.366 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:46.367 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:46.367 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:46.367 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:46.367 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:46.367 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:46.367 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:46.368 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:46.368 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.368 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:46.368 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:46.369 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:46.369 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:46.369 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:46.369 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:46.369 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:46.369 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:46.370 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:46.370 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:46.370 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:46.370 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.370 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:46.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38d8066470 msgr2=0x7f38d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38d8066470 0x7f38d8068930 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f38f4141d30 tx=0x7f38e4009040 comp rx=0 tx=0).stop 2023-10-26T05:58:46.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 msgr2=0x7f38f4134d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 0x7f38f4134d60 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009bd0 tx=0x7f38ec0092e0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 shutdown_connections 2023-10-26T05:58:46.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f38d8066470 0x7f38d8068930 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4140cc0 0x7f38f4134d60 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38f40b1360 0x7f38f4133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 --2- 172.21.15.71:0/442443096 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38f4070fe0 0x7f38f4157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 >> 172.21.15.71:0/442443096 conn(0x7f38f4069770 msgr2=0x7f38f413ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:46.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 shutdown_connections 2023-10-26T05:58:46.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.163+0000 7f38fb8ff640 1 -- 172.21.15.71:0/442443096 wait complete. 2023-10-26T05:58:46.375 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T05:58:46.376 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:58:46.376 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:58:46.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:46.383 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-10-26T05:58:46.493 INFO:teuthology.orchestra.run.smithi071.stderr:dd: error writing 'large_file_b': No space left on device 2023-10-26T05:58:46.494 INFO:teuthology.orchestra.run.smithi071.stderr:dd: closing output file 'large_file_b': No space left on device 2023-10-26T05:58:46.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:58:46.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 -- 172.21.15.71:0/3823167041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c140cc0 msgr2=0x7f968c15c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3823167041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c140cc0 0x7f968c15c760 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f968c06fa60 tx=0x7f968801c870 comp rx=0 tx=0).stop 2023-10-26T05:58:46.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 -- 172.21.15.71:0/3823167041 shutdown_connections 2023-10-26T05:58:46.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3823167041 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f968c0b0040 0x7f968c0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3823167041 >> v2:172.21.15.86:3300/0 conn(0x7f968c070fe0 0x7f968c071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3823167041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c140cc0 0x7f968c15c760 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 -- 172.21.15.71:0/3823167041 >> 172.21.15.71:0/3823167041 conn(0x7f968c069770 msgr2=0x7f968c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.607+0000 7f9692b5a640 1 -- 172.21.15.71:0/3823167041 shutdown_connections 2023-10-26T05:58:46.620 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- 172.21.15.71:0/3823167041 wait complete. 2023-10-26T05:58:46.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 Processor -- start 2023-10-26T05:58:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- start start 2023-10-26T05:58:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 0x7f968c124e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c0b0040 0x7f968c126ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f968c140cc0 0x7f968c12a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.623 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c154650 con 0x7f968c070fe0 2023-10-26T05:58:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c127db0 con 0x7f968c0b0040 2023-10-26T05:58:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1277f0 con 0x7f968c140cc0 2023-10-26T05:58:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 0x7f968c124e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 0x7f968c124e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41436/0 (socket says 172.21.15.71:41436) 2023-10-26T05:58:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 -- 172.21.15.71:0/3287796506 learned_addr learned my addr 172.21.15.71:0/3287796506 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9683fff640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c0b0040 0x7f968c126ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96910f7640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f968c140cc0 0x7f968c12a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 -- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f968c140cc0 msgr2=0x7f968c12a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f968c140cc0 0x7f968c12a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 -- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c0b0040 msgr2=0x7f968c126ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c0b0040 0x7f968c126ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 -- 172.21.15.71:0/3287796506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9688009560 con 0x7f968c070fe0 2023-10-26T05:58:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9683fff640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c0b0040 0x7f968c126ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:46.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96908f6640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 0x7f968c124e10 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f968801af20 tx=0x7f96880a58b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96880ba070 con 0x7f968c070fe0 2023-10-26T05:58:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f96880042b0 con 0x7f968c070fe0 2023-10-26T05:58:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688003600 con 0x7f968c070fe0 2023-10-26T05:58:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c127ad0 con 0x7f968c070fe0 2023-10-26T05:58:46.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.611+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c125350 con 0x7f968c070fe0 2023-10-26T05:58:46.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c066b10 con 0x7f968c070fe0 2023-10-26T05:58:46.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f96880b4040 con 0x7f968c070fe0 2023-10-26T05:58:46.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f96817fa640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9678066470 0x7f9678068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:46.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f968811e000 con 0x7f968c070fe0 2023-10-26T05:58:46.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f967806dfa0 con 0x7f968c070fe0 2023-10-26T05:58:46.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f9683fff640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9678066470 0x7f9678068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:46.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.615+0000 7f9683fff640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9678066470 0x7f9678068930 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f967c005950 tx=0x7f967c005c90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:46.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.619+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f96880eb510 con 0x7f968c070fe0 2023-10-26T05:58:46.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.771+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f968c154140 con 0x7f968c070fe0 2023-10-26T05:58:46.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.771+0000 7f96817fa640 1 -- 172.21.15.71:0/3287796506 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) v1 ==== 81+0+33949 (secure 0 0 0) 0x7f968c154140 con 0x7f968c070fe0 2023-10-26T05:58:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:46.781 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:46.781 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 190, 2023-10-26T05:58:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:42.308638+0000", 2023-10-26T05:58:46.782 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:46.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.784 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:46.785 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:46.786 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:46.787 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:46.788 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:46.789 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:46.790 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:46.791 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:46.792 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:46.793 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:46.794 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:46.795 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:46.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:46.797 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:46.798 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.799 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:46.800 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:46.801 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.802 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:46.803 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:46.804 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:46.805 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:46.806 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:46.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.808 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:46.808 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:46.808 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:46.808 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:46.808 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:46.809 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:46.809 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:46.809 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:46.809 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:46.809 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:46.810 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:46.810 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:46.810 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:46.810 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:46.810 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:46.811 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:46.811 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:46.811 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:46.811 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:46.811 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:46.812 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:46.812 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:46.812 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:46.812 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:46.812 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:46.812 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:46.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:46.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.813 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:46.813 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:46.813 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:46.814 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:46.814 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:46.814 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:46.814 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:46.814 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:46.814 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:46.815 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:46.815 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:46.815 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:46.815 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:46.815 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:46.816 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:46.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.816 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:46.816 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:46.816 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:46.817 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.817 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:46.817 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:46.817 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:46.817 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:46.818 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:46.818 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:46.818 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:46.818 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:46.818 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:46.819 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.819 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.819 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:46.819 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:46.820 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:46.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:58:46.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:58:46.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:46.820 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:46.820 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:46.821 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:46.821 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:46.821 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:46.821 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:46.821 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:46.822 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:46.822 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:46.822 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:46.822 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:46.822 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:46.823 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:46.823 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:46.823 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:46.823 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:46.823 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:46.824 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:46.824 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:46.824 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:46.824 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:46.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.825 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "190", 2023-10-26T05:58:46.825 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:46.825 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:46.825 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:46.825 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:46.826 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:46.826 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:46.826 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:46.826 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:46.826 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:46.827 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:46.827 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:46.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:46.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:46.827 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:46.827 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:46.828 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:46.828 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:46.828 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:46.828 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:46.828 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:46.828 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:46.829 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:46.829 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:46.829 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:46.829 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:46.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:46.830 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.830 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:46.830 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:46.830 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:46.830 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:46.830 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:46.831 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:46.831 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:46.831 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:46.831 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:46.831 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:46.831 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:46.832 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:46.832 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:46.832 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:46.832 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.832 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:46.832 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:46.833 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:46.833 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.833 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:46.833 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:46.833 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:46.834 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:46.834 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:46.834 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:46.834 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:46.834 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:46.835 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:46.835 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.835 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.835 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.835 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:46.835 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.836 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:46.836 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.836 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.836 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.836 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.836 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.837 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.837 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.837 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.837 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.837 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.837 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.838 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.838 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.838 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.839 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:46.839 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.839 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.839 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:46.840 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.840 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.840 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.840 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.841 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.841 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:46.841 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.842 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.842 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:46.842 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.842 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.843 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.843 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.843 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.843 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.843 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.843 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:46.844 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.844 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.844 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.844 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:46.845 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.845 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.845 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.845 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.845 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.846 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.846 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:46.846 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.847 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.847 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.847 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:46.847 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:46.847 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.847 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.848 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:46.848 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:46.848 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:46.848 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:46.849 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.849 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.849 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.849 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.850 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.850 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:46.850 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.850 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.850 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.850 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.851 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.851 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.851 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.851 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.851 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.852 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.852 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.852 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.852 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.852 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.854 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:46.854 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.854 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.854 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.855 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.855 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.855 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.855 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.855 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:46.855 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.856 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.856 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:46.857 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.857 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.857 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.857 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.857 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.857 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.858 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.858 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.858 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:46.858 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.859 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.860 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.860 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.860 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.860 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:46.861 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.861 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.861 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.861 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:46.862 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:46.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.862 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.862 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:46.863 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:46.863 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:46.863 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:46.863 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.863 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.863 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.864 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.864 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.864 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.864 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:46.864 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.864 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.865 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.865 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.865 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.865 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.865 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.865 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.866 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.866 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.866 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.866 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.866 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.866 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.867 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:46.867 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.867 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.868 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.868 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:46.868 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.868 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.868 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.869 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.869 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.869 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.869 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.869 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:46.870 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.870 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.870 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:46.871 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.871 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.871 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.871 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.871 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.872 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:46.872 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.873 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:46.873 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.873 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.873 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.874 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.874 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.874 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.874 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.875 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:46.875 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.875 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.876 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:46.876 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:46.876 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.876 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.876 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:46.877 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:46.877 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:46.877 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:46.877 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.877 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.878 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.878 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.878 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:46.879 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.879 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.879 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.879 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.879 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.879 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.880 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.880 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.880 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.880 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.880 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.881 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.881 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.881 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.881 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.881 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:46.882 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.882 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.882 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:46.882 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.883 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.883 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.883 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.883 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.884 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.884 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:46.884 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.885 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.885 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.885 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:46.885 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.885 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.886 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.886 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.886 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.886 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.887 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:46.887 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.887 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.888 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:46.888 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.888 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.888 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.889 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.889 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.889 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:46.890 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.890 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.891 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:46.891 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:46.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.891 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.892 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:46.892 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:46.892 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:46.892 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:46.892 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.893 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.893 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.893 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.893 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.893 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.893 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:46.894 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.894 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.894 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.894 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.894 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.895 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.895 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.895 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.895 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.896 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.896 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.896 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.896 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:46.896 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.901 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.902 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.903 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.904 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:46.905 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.905 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.905 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.906 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.906 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:46.906 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.907 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.907 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.907 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:46.907 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:46.908 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.908 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.908 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.908 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:46.908 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:46.908 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:46.909 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:46.909 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.909 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.909 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.909 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.910 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:46.910 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.910 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.911 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.911 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.911 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.911 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.911 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.912 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.912 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.912 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.912 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.912 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.912 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.913 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:46.913 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.914 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.914 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:46.914 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.915 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.915 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.916 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:46.916 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.916 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:46.917 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.917 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.917 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.918 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.918 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.918 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:46.919 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.919 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.919 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.919 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:46.920 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.920 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.920 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.920 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.920 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.921 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.921 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:46.921 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.922 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.922 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.922 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:46.922 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:46.922 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.923 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.923 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:46.923 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:46.923 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:46.923 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:46.924 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.924 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.924 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.924 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.925 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.925 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:46.925 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.925 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.926 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.926 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.926 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.926 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.926 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.927 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.927 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.927 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.927 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.927 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:46.928 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:46.929 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.930 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.930 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.930 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.931 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:46.931 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.931 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:46.932 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.932 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.932 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.933 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:46.934 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:46.935 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.935 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.935 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.935 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.936 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:46.936 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.937 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.938 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:46.938 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:46.938 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:46.938 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:46.938 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.939 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:46.940 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:46.940 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:46.940 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:46.940 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:46.940 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:46.940 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:46.941 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:46.941 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:46.941 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:46.941 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:46.941 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:46.942 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:46.942 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.942 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:46.942 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.943 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:46.943 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.944 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.944 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:46.944 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.946 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:46.946 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.946 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.947 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:46.947 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.947 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:46.947 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.948 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.948 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:46.948 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.949 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.949 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:46.949 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:46.949 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.950 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:46.950 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:46.950 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.951 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:46.951 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:46.951 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:46.951 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.951 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.952 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:46.952 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:46.952 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:46.952 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:46.952 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:46.953 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:46.953 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:46.953 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:46.953 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.953 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.953 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:46.954 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.954 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:46.954 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.954 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.954 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.955 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.955 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.955 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:46.955 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.956 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:46.956 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.956 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.956 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.957 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.957 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.957 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:46.957 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.958 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:46.958 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.958 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.958 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.959 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.959 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.959 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:46.959 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.960 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:46.960 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.960 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.960 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.960 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.961 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.961 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:46.961 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.962 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:46.962 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.962 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.962 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.962 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.962 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.963 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:46.963 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.963 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:46.964 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.964 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.964 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.964 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.964 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.965 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:46.965 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.965 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:46.965 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.966 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.966 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.966 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.966 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.966 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:46.967 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.967 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:46.967 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:46.967 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:46.967 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:46.968 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:46.968 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:46.968 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:46.968 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:46.968 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:46.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:46.969 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:46.970 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:46.970 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:46.970 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:46.970 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:46.970 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:46.971 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:46.971 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:46.971 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:46.971 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:46.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:46.972 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:46.972 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:46.972 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:46.972 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:46.972 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:46.972 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:46.973 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:46.973 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:46.973 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:46.973 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:46.973 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:46.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:46.974 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9678066470 msgr2=0x7f9678068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9678066470 0x7f9678068930 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f967c005950 tx=0x7f967c005c90 comp rx=0 tx=0).stop 2023-10-26T05:58:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 msgr2=0x7f968c124e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 0x7f968c124e10 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f968801af20 tx=0x7f96880a58b0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 shutdown_connections 2023-10-26T05:58:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f968c140cc0 0x7f968c12a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c0b0040 0x7f968c126ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9678066470 0x7f9678068930 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.977 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 --2- 172.21.15.71:0/3287796506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f968c070fe0 0x7f968c124e10 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:46.978 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 >> 172.21.15.71:0/3287796506 conn(0x7f968c069770 msgr2=0x7f968c157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:46.978 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 shutdown_connections 2023-10-26T05:58:46.978 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:46.775+0000 7f9692b5a640 1 -- 172.21.15.71:0/3287796506 wait complete. 2023-10-26T05:58:46.979 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:47.106 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/1550734420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7418004d10 msgr2=0x7f7418005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 --2- 172.21.15.71:0/1550734420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7418004d10 0x7f7418005190 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f7424009eb0 tx=0x7f742401d590 comp rx=0 tx=0).stop 2023-10-26T05:58:47.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7432420640 1 -- 172.21.15.71:0/1550734420 reap_dead start 2023-10-26T05:58:47.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/1550734420 shutdown_connections 2023-10-26T05:58:47.107 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/1550734420 >> 172.21.15.71:0/1550734420 conn(0x7f742c069770 msgr2=0x7f742c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:47.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/1550734420 shutdown_connections 2023-10-26T05:58:47.108 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/1550734420 wait complete. 2023-10-26T05:58:47.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 Processor -- start 2023-10-26T05:58:47.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- start start 2023-10-26T05:58:47.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418004d10 0x7f742c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 0x7f742c1ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f742c1efde0 0x7f742c0c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1f2c90 con 0x7f742c1576e0 2023-10-26T05:58:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1e2b20 con 0x7f7418004d10 2023-10-26T05:58:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0c3530 con 0x7f742c1efde0 2023-10-26T05:58:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 0x7f742c1ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 0x7f742c1ef8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41454/0 (socket says 172.21.15.71:41454) 2023-10-26T05:58:47.112 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 -- 172.21.15.71:0/3318730912 learned_addr learned my addr 172.21.15.71:0/3318730912 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7431c1f640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418004d10 0x7f742c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7432420640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f742c1efde0 0x7f742c0c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 -- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f742c1efde0 msgr2=0x7f742c0c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f742c1efde0 0x7f742c0c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 -- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418004d10 msgr2=0x7f742c1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418004d10 0x7f742c1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 -- 172.21.15.71:0/3318730912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c0c3810 con 0x7f742c1576e0 2023-10-26T05:58:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7432420640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f742c1efde0 0x7f742c0c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f743141e640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 0x7f742c1ef8a0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f741c001210 tx=0x7f741c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00de00 con 0x7f742c1576e0 2023-10-26T05:58:47.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f741c0052e0 con 0x7f742c1576e0 2023-10-26T05:58:47.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c0c3af0 con 0x7f742c1576e0 2023-10-26T05:58:47.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00b8f0 con 0x7f742c1576e0 2023-10-26T05:58:47.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.099+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c0c2f00 con 0x7f742c1576e0 2023-10-26T05:58:47.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c066b10 con 0x7f742c1576e0 2023-10-26T05:58:47.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f741c00ba50 con 0x7f742c1576e0 2023-10-26T05:58:47.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f74227fc640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7410066470 0x7f7410068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f741c083ac0 con 0x7f742c1576e0 2023-10-26T05:58:47.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f741006dfa0 con 0x7f742c1576e0 2023-10-26T05:58:47.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f7431c1f640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7410066470 0x7f7410068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.103+0000 7f7431c1f640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7410066470 0x7f7410068930 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7428001720 tx=0x7f7428009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:47.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.107+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f741c019020 con 0x7f742c1576e0 2023-10-26T05:58:47.145 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:58:47.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:58:47.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.279+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f742c0721e0 con 0x7f742c1576e0 2023-10-26T05:58:47.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.279+0000 7f74227fc640 1 -- 172.21.15.71:0/3318730912 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f741c050f30 con 0x7f742c1576e0 2023-10-26T05:58:47.290 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:47.290 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:58:47.291 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:58:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7410066470 msgr2=0x7f7410068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.292 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7410066470 0x7f7410068930 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7428001720 tx=0x7f7428009040 comp rx=0 tx=0).stop 2023-10-26T05:58:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 msgr2=0x7f742c1ef8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 0x7f742c1ef8a0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f741c001210 tx=0x7f741c00dc10 comp rx=0 tx=0).stop 2023-10-26T05:58:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 shutdown_connections 2023-10-26T05:58:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f742c1efde0 0x7f742c0c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.293 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f7410066470 0x7f7410068930 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f742c1576e0 0x7f742c1ef8a0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 --2- 172.21.15.71:0/3318730912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418004d10 0x7f742c1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 >> 172.21.15.71:0/3318730912 conn(0x7f742c069770 msgr2=0x7f742c06aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 shutdown_connections 2023-10-26T05:58:47.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.283+0000 7f7433e83640 1 -- 172.21.15.71:0/3318730912 wait complete. 2023-10-26T05:58:47.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:58:47.440 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":9387,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":24.726409563000001,"mdsmap_epoch":71,"osdmap_epoch":190,"osdmap_epoch_barrier":188,"uptime":29.537566167000001} 2023-10-26T05:58:47.441 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 9387, "mdsmap_epoch": 71, "osdmap_epoch": 190, "osdmap_epoch_barrier": 188, "rank_uptime": 24.726409563, "state": "up:active", "uptime": 29.537566167, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:58:47.441 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:58:47.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:47.442 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-10-26T05:58:47.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:58:47.452 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-10-26T05:58:47.511 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:47.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2823463386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f974815b5c0 msgr2=0x7f97480b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 --2- 172.21.15.71:0/2823463386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f974815b5c0 0x7f97480b02a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f974815efa0 tx=0x7f974401c7e0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.682 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2823463386 shutdown_connections 2023-10-26T05:58:47.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 --2- 172.21.15.71:0/2823463386 >> v2:172.21.15.86:3300/0 conn(0x7f97480b6910 0x7f97480b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 --2- 172.21.15.71:0/2823463386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f974815b5c0 0x7f97480b02a0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2823463386 >> 172.21.15.71:0/2823463386 conn(0x7f97480d12d0 msgr2=0x7f97480d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:47.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2823463386 shutdown_connections 2023-10-26T05:58:47.683 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2823463386 wait complete. 2023-10-26T05:58:47.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 Processor -- start 2023-10-26T05:58:47.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- start start 2023-10-26T05:58:47.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9748069e60 0x7f97480c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 0x7f97480b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f974815b5c0 0x7f974814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97481502b0 con 0x7f97480b6910 2023-10-26T05:58:47.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9748150590 con 0x7f9748069e60 2023-10-26T05:58:47.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9748150870 con 0x7f974815b5c0 2023-10-26T05:58:47.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 0x7f97480b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 0x7f97480b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41466/0 (socket says 172.21.15.71:41466) 2023-10-26T05:58:47.687 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 -- 172.21.15.71:0/2614762139 learned_addr learned my addr 172.21.15.71:0/2614762139 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974d136640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f974815b5c0 0x7f974814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974c935640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9748069e60 0x7f97480c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 -- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f974815b5c0 msgr2=0x7f974814f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f974815b5c0 0x7f974814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 -- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9748069e60 msgr2=0x7f97480c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9748069e60 0x7f97480c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 -- 172.21.15.71:0/2614762139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9744009560 con 0x7f97480b6910 2023-10-26T05:58:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973ffff640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 0x7f97480b1ee0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f973800c970 tx=0x7f973800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738007bb0 con 0x7f97480b6910 2023-10-26T05:58:47.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9738007e00 con 0x7f97480b6910 2023-10-26T05:58:47.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97480cdb70 con 0x7f97480b6910 2023-10-26T05:58:47.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738005280 con 0x7f97480b6910 2023-10-26T05:58:47.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9748150e30 con 0x7f97480b6910 2023-10-26T05:58:47.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.675+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9748066b10 con 0x7f97480b6910 2023-10-26T05:58:47.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.679+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f97380053e0 con 0x7f97480b6910 2023-10-26T05:58:47.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.679+0000 7f973d7fa640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9734066470 0x7f9734068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:47.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.679+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97380844e0 con 0x7f97480b6910 2023-10-26T05:58:47.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.679+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f973406dfa0 con 0x7f97480b6910 2023-10-26T05:58:47.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.679+0000 7f974c935640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9734066470 0x7f9734068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:47.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.679+0000 7f974c935640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9734066470 0x7f9734068930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f97440af000 tx=0x7f974401b020 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:47.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.683+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f973801d060 con 0x7f97480b6910 2023-10-26T05:58:47.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.839+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9748151050 con 0x7f97480b6910 2023-10-26T05:58:47.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.839+0000 7f973d7fa640 1 -- 172.21.15.71:0/2614762139 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) v1 ==== 81+0+33949 (secure 0 0 0) 0x7f97380553f0 con 0x7f97480b6910 2023-10-26T05:58:47.850 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:47.850 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:47.851 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 190, 2023-10-26T05:58:47.851 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:47.851 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:47.851 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:42.308638+0000", 2023-10-26T05:58:47.852 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:47.852 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:47.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:47.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:47.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:47.853 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:47.853 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:47.853 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:47.853 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:47.853 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:47.854 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:47.854 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:47.854 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:47.854 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:47.854 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:47.855 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:47.855 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:47.855 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:47.855 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:47.855 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:47.855 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:47.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:47.856 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:47.856 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:47.856 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:47.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:47.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:47.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:47.857 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:47.857 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:47.857 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:47.858 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:47.858 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:47.858 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:47.858 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:47.858 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:47.859 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:47.859 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:47.859 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:47.859 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:47.859 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:47.860 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:47.860 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:47.860 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:47.860 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:47.860 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:47.861 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:47.861 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:47.861 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:47.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.861 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:47.861 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:47.862 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:47.862 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:47.862 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:47.862 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:47.862 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:47.863 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:47.863 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:47.863 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:47.863 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:47.863 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:47.863 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:47.864 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:47.864 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:47.864 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:47.864 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:47.864 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:47.865 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:47.865 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:47.865 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:47.865 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:47.865 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:47.866 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:47.866 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:47.866 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:47.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:47.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.867 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:47.867 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:47.867 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:47.867 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:47.867 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:47.868 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:47.868 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:47.868 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:47.868 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:47.868 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:47.869 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:47.869 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:47.869 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:47.869 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:47.869 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.870 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:47.870 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:47.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.870 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:47.870 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:47.871 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:47.871 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:47.871 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:47.871 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:47.871 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:47.871 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:47.872 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:47.872 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.872 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.872 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.872 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:47.872 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:47.873 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:47.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:47.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:47.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:47.873 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:47.873 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:47.874 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:47.874 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:47.874 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:47.874 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:47.874 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:47.875 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:47.875 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:47.875 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:47.875 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:47.875 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:47.876 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:47.876 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:47.876 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:47.876 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:47.876 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:47.877 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:47.877 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:47.877 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:47.877 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:47.877 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.878 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:47.878 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:47.878 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:47.878 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:47.878 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:47.879 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:47.879 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:47.879 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:47.879 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:47.879 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:47.880 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:47.880 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:47.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:47.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:47.880 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:47.881 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:47.881 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:47.881 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:47.881 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:47.881 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:47.882 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:47.882 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:47.882 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:47.882 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:47.882 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:47.883 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:47.883 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:47.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.883 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:47.883 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:47.883 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:47.884 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:47.884 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:47.884 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:47.884 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:47.884 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:47.885 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:47.885 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:47.885 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:47.885 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:47.885 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:47.885 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:47.886 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:47.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.886 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:47.886 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:47.886 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:47.887 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.887 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:47.887 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:47.887 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:47.887 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:47.888 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:47.888 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:47.888 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:47.888 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:47.888 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:47.889 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.889 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.889 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:47.889 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:47.890 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:47.890 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:58:47.890 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:58:47.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:47.890 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:47.891 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:47.891 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:47.891 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:47.891 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:47.891 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:47.892 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:47.892 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:47.892 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:47.892 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:47.892 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:47.893 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:47.893 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:47.893 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:47.893 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:47.893 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:47.894 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:47.894 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:47.894 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:47.894 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:47.894 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:47.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "190", 2023-10-26T05:58:47.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:47.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:47.895 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:47.896 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:47.896 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:47.896 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:47.896 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:47.896 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:47.897 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:47.897 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:47.897 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:47.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:47.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:47.897 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:47.898 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:47.898 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:47.898 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:47.898 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:47.898 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:47.899 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:47.899 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:47.899 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:47.899 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:47.899 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:47.899 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:47.900 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:47.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.900 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:47.900 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:47.900 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:47.901 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:47.901 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:47.901 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:47.901 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:47.901 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:47.902 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:47.902 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:47.902 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:47.902 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:47.902 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:47.902 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:47.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.903 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:47.903 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:47.903 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:47.903 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.904 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:47.904 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:47.904 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:47.904 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:47.904 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:47.905 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:47.905 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:47.905 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:47.905 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:47.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.906 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.906 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:47.906 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:47.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.906 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:47.907 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:47.907 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:47.907 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:47.907 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:47.907 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:47.908 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:47.908 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:47.908 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:47.908 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:47.908 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:47.909 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:47.909 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:47.909 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.909 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:47.910 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.910 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.910 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:47.911 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.911 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.911 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.911 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:47.912 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.912 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.912 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.912 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:47.912 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.913 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.913 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:47.913 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.914 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.914 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.914 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:47.914 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.915 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.915 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:47.915 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.916 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:47.916 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.916 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.916 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.917 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:47.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.917 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.917 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.917 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:47.918 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.918 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.921 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:47.921 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:47.921 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:47.922 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:47.923 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:47.924 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.924 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:47.924 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:47.924 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:47.925 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:47.925 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:47.925 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:47.925 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:47.925 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:47.926 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:47.926 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:47.926 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:47.926 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:47.926 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:47.927 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.927 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.927 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:47.927 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.928 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.928 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:47.928 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.929 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.929 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:47.929 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.929 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.930 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.930 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:47.930 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.931 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:47.931 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.931 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.931 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.932 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:47.932 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.932 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.933 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:47.933 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.933 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:47.934 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.934 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.934 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.935 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:47.935 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.935 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.935 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:47.935 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.936 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.936 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:47.937 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:47.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.937 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.938 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:47.938 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:47.938 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:47.938 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:47.939 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:47.939 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:47.939 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:47.939 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.940 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:47.940 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:47.940 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:47.941 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:47.941 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:47.941 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:47.941 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:47.942 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:47.942 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:47.942 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:47.942 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:47.942 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:47.943 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:47.943 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.943 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.944 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:47.944 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.945 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:47.945 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.945 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.946 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.946 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:47.946 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.947 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.947 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:47.947 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.948 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.948 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:47.949 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.949 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.949 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.950 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:47.950 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.950 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.950 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.951 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:47.951 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.951 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.951 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.952 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:47.952 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.952 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.952 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.953 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:47.953 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.954 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:47.954 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.955 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.955 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:47.955 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:47.956 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.956 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.956 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:47.956 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:47.957 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:47.957 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:47.957 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:47.957 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:47.958 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:47.958 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.958 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:47.959 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:47.959 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:47.959 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:47.959 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:47.959 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:47.960 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:47.960 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:47.960 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:47.960 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:47.961 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:47.961 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:47.961 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:47.961 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.962 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.962 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:47.962 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.963 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.963 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.963 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:47.963 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.964 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.964 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:47.964 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.965 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.965 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.965 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:47.965 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.966 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.966 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:47.967 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.967 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.967 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:47.968 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.968 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.968 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.968 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:47.969 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.970 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:47.970 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.970 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.970 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.971 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:47.971 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.971 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.971 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.972 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:47.972 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.972 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.973 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:47.973 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:47.973 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.973 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.974 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:47.974 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:47.974 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:47.975 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:47.975 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:47.975 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:47.975 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:47.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.976 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.976 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:47.976 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:47.977 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:47.977 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:47.977 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:47.977 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:47.978 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:47.978 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:47.978 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:47.978 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:47.979 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:47.979 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:47.979 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:47.979 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.980 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.980 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:47.980 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.981 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.981 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:47.982 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.982 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.982 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.983 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:47.983 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.983 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.983 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.983 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:47.984 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.985 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:47.985 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.985 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.985 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.986 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:47.986 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.986 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.987 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.987 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:47.987 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.988 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:47.988 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.989 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.989 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.989 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:47.989 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.990 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:47.990 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.991 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:47.991 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:47.991 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:47.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:47.992 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.992 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:47.993 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:47.993 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:47.993 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:47.993 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:47.993 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:47.994 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:47.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:47.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.994 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.995 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:47.995 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:47.995 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:47.995 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:47.996 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:47.996 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:47.996 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:47.996 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:47.997 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:47.997 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:47.997 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:47.997 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:47.998 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:47.998 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:47.998 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:47.998 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:47.999 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:47.999 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:47.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:47.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.000 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.000 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:48.000 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.000 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.000 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.001 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:48.001 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.002 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:48.002 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.003 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:48.003 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.003 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.004 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.004 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:48.004 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.004 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.005 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:48.005 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.006 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:48.006 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.007 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.007 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:48.007 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.008 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:48.008 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.009 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:48.009 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:48.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.009 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.010 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:48.010 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:48.010 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:48.010 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:48.011 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:48.011 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:48.011 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:48.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.012 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:48.012 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:48.012 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:48.012 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:48.012 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:48.013 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:48.013 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:48.013 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:48.013 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:48.013 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:48.014 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:48.014 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:48.014 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:48.014 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.015 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.015 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:48.015 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.015 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.016 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:48.016 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.016 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.017 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:48.017 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.017 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:48.017 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.018 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:48.018 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.019 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.019 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.019 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:48.019 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.019 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.020 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:48.020 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.021 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.021 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:48.021 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.021 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.021 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.022 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:48.022 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.023 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:48.023 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.023 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.023 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:48.024 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:48.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.024 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.024 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:48.025 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:48.025 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:48.025 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:48.025 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:48.025 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:48.025 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:48.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.026 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:48.026 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:48.027 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:48.027 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:48.027 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:48.027 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:48.027 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:48.028 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:48.028 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:48.028 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:48.028 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:48.028 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:48.029 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:48.029 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.029 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:48.030 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.030 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.030 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:48.031 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.031 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.031 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:48.031 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.032 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:48.032 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.033 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.033 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:48.033 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.033 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.033 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.034 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:48.034 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.034 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.035 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:48.035 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.035 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.036 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:48.036 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.036 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.036 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:48.037 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:48.038 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:48.039 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:48.040 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:48.040 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:48.040 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:48.040 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:48.040 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:48.041 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:48.041 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.041 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:48.041 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:48.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.041 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:48.042 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.042 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.042 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.042 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.042 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.043 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:48.043 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.043 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.043 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:48.044 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.044 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.044 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.044 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.044 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.045 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:48.045 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.045 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:48.045 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.046 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.046 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.046 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.046 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.046 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:48.047 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.047 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:48.047 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.048 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.048 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.048 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.048 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.048 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:48.048 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.049 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.049 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:48.049 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.049 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.050 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.050 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.050 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.050 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:48.050 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.051 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:48.051 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.051 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.052 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.052 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.052 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.052 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:48.052 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.053 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:48.053 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.053 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.053 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.054 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.054 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.054 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:48.054 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.055 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:48.055 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:48.055 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:48.055 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:48.055 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:48.056 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:48.056 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:48.056 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:48.056 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:48.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.056 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:48.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:48.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:48.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:48.057 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:48.057 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:48.058 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:48.058 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:48.058 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:48.058 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:48.058 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:48.059 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:48.059 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:48.059 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:48.059 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:48.059 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:48.060 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:48.060 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:48.060 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:48.060 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:48.061 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:48.061 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:48.061 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:48.061 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:48.061 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:48.061 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:48.062 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:48.062 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:48.062 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9734066470 msgr2=0x7f9734068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9734066470 0x7f9734068930 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f97440af000 tx=0x7f974401b020 comp rx=0 tx=0).stop 2023-10-26T05:58:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 msgr2=0x7f97480b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 0x7f97480b1ee0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f973800c970 tx=0x7f973800ce30 comp rx=0 tx=0).stop 2023-10-26T05:58:48.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 shutdown_connections 2023-10-26T05:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f974815b5c0 0x7f974814f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9734066470 0x7f9734068930 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97480b6910 0x7f97480b1ee0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 --2- 172.21.15.71:0/2614762139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9748069e60 0x7f97480c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:48.065 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 >> 172.21.15.71:0/2614762139 conn(0x7f97480d12d0 msgr2=0x7f97480cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 shutdown_connections 2023-10-26T05:58:48.066 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:47.843+0000 7f974eb99640 1 -- 172.21.15.71:0/2614762139 wait complete. 2023-10-26T05:58:48.066 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:58:53.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:53.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/3529409399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13dc20 msgr2=0x7f2e3c15b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:53.203 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- 172.21.15.71:0/3529409399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13dc20 0x7f2e3c15b9f0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0afd40 tx=0x7f2e3401cd60 comp rx=0 tx=0).stop 2023-10-26T05:58:53.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/3529409399 shutdown_connections 2023-10-26T05:58:53.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- 172.21.15.71:0/3529409399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13dc20 0x7f2e3c15b9f0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- 172.21.15.71:0/3529409399 >> v2:172.21.15.86:3300/0 conn(0x7f2e3c13d800 0x7f2e3c15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- 172.21.15.71:0/3529409399 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3c13d420 0x7f2e3c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.204 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/3529409399 >> 172.21.15.71:0/3529409399 conn(0x7f2e3c069770 msgr2=0x7f2e3c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:53.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/3529409399 shutdown_connections 2023-10-26T05:58:53.205 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/3529409399 wait complete. 2023-10-26T05:58:53.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 Processor -- start 2023-10-26T05:58:53.207 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- start start 2023-10-26T05:58:53.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 0x7f2e3c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:53.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3c13d800 0x7f2e3c1348a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:53.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c13dc20 0x7f2e3c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:53.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0accb0 con 0x7f2e3c13d420 2023-10-26T05:58:53.208 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0acf90 con 0x7f2e3c13dc20 2023-10-26T05:58:53.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c0ad270 con 0x7f2e3c13d800 2023-10-26T05:58:53.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 0x7f2e3c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:53.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 0x7f2e3c0ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45770/0 (socket says 172.21.15.71:45770) 2023-10-26T05:58:53.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 -- 172.21.15.71:0/1595651563 learned_addr learned my addr 172.21.15.71:0/1595651563 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:53.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e42d9b640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c13dc20 0x7f2e3c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:53.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e41d99640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3c13d800 0x7f2e3c1348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:53.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 -- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3c13d800 msgr2=0x7f2e3c1348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:53.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3c13d800 0x7f2e3c1348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 -- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c13dc20 msgr2=0x7f2e3c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:53.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c13dc20 0x7f2e3c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 -- 172.21.15.71:0/1595651563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340098e0 con 0x7f2e3c13d420 2023-10-26T05:58:53.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e4259a640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 0x7f2e3c0ae3a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800c940 tx=0x7f2e3800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:53.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38007bb0 con 0x7f2e3c13d420 2023-10-26T05:58:53.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2e38007e00 con 0x7f2e3c13d420 2023-10-26T05:58:53.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38005280 con 0x7f2e3c13d420 2023-10-26T05:58:53.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c0ad550 con 0x7f2e3c13d420 2023-10-26T05:58:53.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.195+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c0ae8e0 con 0x7f2e3c13d420 2023-10-26T05:58:53.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c066b10 con 0x7f2e3c13d420 2023-10-26T05:58:53.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2e380053e0 con 0x7f2e3c13d420 2023-10-26T05:58:53.212 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e32ffd640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e200664a0 0x7f2e20068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:53.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e380846a0 con 0x7f2e3c13d420 2023-10-26T05:58:53.213 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=191}) v3 -- 0x7f2e2006dfd0 con 0x7f2e3c13d420 2023-10-26T05:58:53.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e41d99640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e200664a0 0x7f2e20068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:53.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.199+0000 7f2e41d99640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e200664a0 0x7f2e20068960 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005fd0 tx=0x7f2e2c005db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:53.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.203+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2e3801d070 con 0x7f2e3c13d420 2023-10-26T05:58:53.366 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2e3c0721e0 con 0x7f2e3c13d420 2023-10-26T05:58:53.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e32ffd640 1 -- 172.21.15.71:0/1595651563 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) v1 ==== 81+0+33949 (secure 0 0 0) 0x7f2e38051bb0 con 0x7f2e3c13d420 2023-10-26T05:58:53.368 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:53.369 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:53.369 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 190, 2023-10-26T05:58:53.369 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:53.369 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:53.369 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:42.308638+0000", 2023-10-26T05:58:53.370 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:53.370 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:53.370 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:53.370 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:53.370 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:53.371 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:53.371 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:53.371 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:53.371 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:53.371 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:53.372 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:53.372 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:53.372 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:53.372 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:53.372 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:53.373 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:53.373 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:53.373 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:53.373 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:53.373 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:53.373 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:53.374 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:53.374 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.374 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:53.374 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:53.374 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:53.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:53.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:53.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:53.375 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:53.375 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:53.375 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:53.376 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:53.376 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:53.376 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:53.376 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:53.376 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:53.376 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:53.377 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:53.377 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:53.377 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:53.377 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:53.377 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:53.378 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:53.378 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:53.378 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:53.378 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:53.378 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:53.378 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:53.379 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:53.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.379 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:53.379 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:53.379 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:53.380 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:53.380 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:53.380 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:53.380 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:53.380 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:53.381 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:53.381 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:53.381 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:53.381 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:53.381 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:53.381 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:53.382 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:53.382 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:53.382 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:53.382 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:53.382 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:53.382 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:53.383 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:53.383 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:53.383 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:53.383 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:53.383 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:53.383 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:53.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:53.384 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.384 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:53.384 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:53.384 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:53.384 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:53.385 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:53.385 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:53.385 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:53.385 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:53.385 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:53.386 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:53.386 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:53.386 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:53.386 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:53.386 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:53.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.387 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:53.387 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:53.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.387 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:53.387 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:53.388 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:53.389 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.389 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:53.389 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:53.390 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:53.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:53.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:53.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:53.390 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:53.391 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:53.391 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:53.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:53.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:53.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:53.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:53.393 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:53.393 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:53.393 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:53.393 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:53.393 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:53.393 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:53.394 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:53.394 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:53.394 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:53.394 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:53.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:53.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:53.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:53.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:53.395 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:53.396 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:53.396 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:53.396 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:53.396 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:53.396 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:53.397 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:53.397 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:53.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:53.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:53.397 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:53.397 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:53.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:53.398 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:53.398 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:53.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:53.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:53.399 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:53.399 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:53.399 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:53.399 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:53.399 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:53.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:53.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.400 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:53.400 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:53.400 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:53.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:53.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:53.401 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:53.401 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:53.401 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:53.402 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:53.402 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:53.402 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:53.402 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:53.402 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:53.403 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:53.403 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:53.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.403 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:53.403 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:53.404 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:53.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.404 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:53.404 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:53.404 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:53.405 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:53.405 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:53.405 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:53.405 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:53.405 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:53.406 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:53.406 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.406 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.406 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:53.407 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:53.407 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:53.407 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1027, 2023-10-26T05:58:53.407 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool,full,full_quota", 2023-10-26T05:58:53.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:53.407 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:53.408 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:53.408 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:53.408 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:53.408 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:53.408 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:53.409 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:53.409 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:53.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:53.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:53.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:53.410 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:53.410 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:53.410 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:53.410 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:53.410 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:53.411 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:53.411 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:53.411 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:53.411 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:53.411 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:53.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.412 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "190", 2023-10-26T05:58:53.412 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:53.412 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:53.412 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:53.412 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:53.412 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:53.413 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:53.413 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:53.413 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:53.413 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:53.413 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:53.414 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:53.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:53.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:53.414 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:53.414 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:53.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:53.415 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:53.415 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:53.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:53.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:53.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:53.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:53.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:53.418 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:53.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.420 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:53.421 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.422 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.423 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.424 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.425 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:53.431 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:53.432 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.433 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.434 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.435 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:53.436 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.437 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.438 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.439 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.441 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.442 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:53.443 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:53.450 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:53.451 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:53.452 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.454 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.455 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.456 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.457 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.458 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:53.459 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:53.460 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.461 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.463 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.465 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.466 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.467 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.468 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.469 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.470 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.471 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.472 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.473 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.478 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:53.478 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.478 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.478 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.479 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.479 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.479 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.480 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:53.481 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.482 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:53.483 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.484 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.485 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:53.486 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:53.487 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.490 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.492 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.493 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:53.494 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:53.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:53.496 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.497 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.498 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.501 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.502 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.503 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.504 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.505 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:53.506 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:53.511 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.513 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.516 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:53.516 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.516 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.517 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:53.517 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.517 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.517 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.517 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.518 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.518 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.518 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:53.519 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.519 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:53.519 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.520 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.520 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.520 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.521 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:53.521 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.522 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:53.522 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.523 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.524 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:53.524 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.525 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:53.525 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:53.525 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.525 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.526 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:53.526 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:53.526 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:53.526 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:53.526 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.528 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:53.528 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:53.528 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:53.528 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:53.528 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:53.529 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:53.530 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:53.530 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:53.530 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.531 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:53.531 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.531 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.531 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.532 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:53.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:53.533 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.534 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:53.534 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.535 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.535 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:53.535 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.535 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.536 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:53.536 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.537 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:53.537 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.537 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.537 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.538 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:53.538 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:53.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:53.538 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:53.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.540 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.540 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:53.540 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:53.541 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:53.541 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:53.541 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:53.541 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:53.541 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:53.542 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:53.542 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.542 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:53.542 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:53.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.542 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:53.543 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.543 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.543 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.543 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.543 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.544 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:53.544 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.544 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.544 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.545 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.546 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:53.546 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.546 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:53.547 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.547 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.547 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.547 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.547 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.548 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:53.548 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.548 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:53.549 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.549 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.549 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.549 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.549 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.550 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:53.550 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.550 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.550 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:53.551 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.551 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.551 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.551 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.551 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.552 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:53.552 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.552 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.552 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.552 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:53.553 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.553 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.553 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.553 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.553 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.553 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:53.554 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.554 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:53.554 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.555 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.555 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.555 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.555 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.555 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:53.556 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:53.556 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:53.556 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:53.557 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:53.557 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:53.557 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:53.557 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:53.557 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:53.558 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:53.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.558 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:53.558 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:53.558 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:53.558 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:53.559 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:53.559 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:53.559 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:53.559 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:53.559 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:53.560 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:53.560 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:53.560 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:53.560 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:53.560 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:53.561 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:53.561 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:53.561 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:53.561 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:53.561 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:53.562 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:53.562 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:53.562 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:53.562 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:53.562 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:53.563 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:53.563 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:53.563 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:53.563 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:53.563 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:53.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e200664a0 msgr2=0x7f2e20068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:53.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e200664a0 0x7f2e20068960 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c005fd0 tx=0x7f2e2c005db0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 msgr2=0x7f2e3c0ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:53.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 0x7f2e3c0ae3a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800c940 tx=0x7f2e3800ce00 comp rx=0 tx=0).stop 2023-10-26T05:58:53.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 shutdown_connections 2023-10-26T05:58:53.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e3c13dc20 0x7f2e3c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2e3c13d800 0x7f2e3c1348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2e200664a0 0x7f2e20068960 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 --2- 172.21.15.71:0/1595651563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e3c13d420 0x7f2e3c0ae3a0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:53.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 >> 172.21.15.71:0/1595651563 conn(0x7f2e3c069770 msgr2=0x7f2e3c159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:53.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 shutdown_connections 2023-10-26T05:58:53.567 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:53.359+0000 7f2e447fe640 1 -- 172.21.15.71:0/1595651563 wait complete. 2023-10-26T05:58:53.568 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-10-26T05:58:58.569 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:58.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86eea7640 1 -- 172.21.15.71:0/82226428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 msgr2=0x7fe850004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:58.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86eea7640 1 --2- 172.21.15.71:0/82226428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe850004c50 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0016b0 tx=0x7fe85c01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:58:58.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86d444640 1 -- 172.21.15.71:0/82226428 reap_dead start 2023-10-26T05:58:58.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86eea7640 1 -- 172.21.15.71:0/82226428 shutdown_connections 2023-10-26T05:58:58.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86eea7640 1 -- 172.21.15.71:0/82226428 >> 172.21.15.71:0/82226428 conn(0x7fe868069770 msgr2=0x7fe868069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:58.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86eea7640 1 -- 172.21.15.71:0/82226428 shutdown_connections 2023-10-26T05:58:58.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.695+0000 7fe86eea7640 1 -- 172.21.15.71:0/82226428 wait complete. 2023-10-26T05:58:58.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 Processor -- start 2023-10-26T05:58:58.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- start start 2023-10-26T05:58:58.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe8680adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:58.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe868160d60 0x7fe86813ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:58.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8681611e0 0x7fe868140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:58.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681409c0 con 0x7fe8500047d0 2023-10-26T05:58:58.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86815fcc0 con 0x7fe868160d60 2023-10-26T05:58:58.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86815ffa0 con 0x7fe8681611e0 2023-10-26T05:58:58.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe8680adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:58.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe8680adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45788/0 (socket says 172.21.15.71:45788) 2023-10-26T05:58:58.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 -- 172.21.15.71:0/237675032 learned_addr learned my addr 172.21.15.71:0/237675032 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:58.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe867fff640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe868160d60 0x7fe86813ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:58.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86d444640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8681611e0 0x7fe868140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:58.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 -- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8681611e0 msgr2=0x7fe868140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:58.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8681611e0 0x7fe868140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:58.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 -- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe868160d60 msgr2=0x7fe86813ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:58.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe868160d60 0x7fe86813ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:58.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 -- 172.21.15.71:0/237675032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86815d680 con 0x7fe8500047d0 2023-10-26T05:58:58.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe867fff640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe868160d60 0x7fe86813ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:58:58.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86cc43640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe8680adff0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fe85800f430 tx=0x7fe85800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:58.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580084e0 con 0x7fe8500047d0 2023-10-26T05:58:58.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe858006720 con 0x7fe8500047d0 2023-10-26T05:58:58.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe868160280 con 0x7fe8500047d0 2023-10-26T05:58:58.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858006b30 con 0x7fe8500047d0 2023-10-26T05:58:58.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe868160840 con 0x7fe8500047d0 2023-10-26T05:58:58.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.699+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe868066b10 con 0x7fe8500047d0 2023-10-26T05:58:58.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.703+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe858006c90 con 0x7fe8500047d0 2023-10-26T05:58:58.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.703+0000 7fe8657fa640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe854066470 0x7fe854068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:58.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.703+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe858086670 con 0x7fe8500047d0 2023-10-26T05:58:58.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.703+0000 7fe867fff640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe854066470 0x7fe854068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:58.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.703+0000 7fe867fff640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe854066470 0x7fe854068930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0a9000 tx=0x7fe85c019540 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:58.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.707+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe858053a50 con 0x7fe8500047d0 2023-10-26T05:58:58.867 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.859+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe8680721e0 con 0x7fe8500047d0 2023-10-26T05:58:58.869 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.859+0000 7fe8657fa640 1 -- 172.21.15.71:0/237675032 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v191) v1 ==== 81+0+33930 (secure 0 0 0) 0x7fe858057070 con 0x7fe8500047d0 2023-10-26T05:58:58.870 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:58.871 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:58.871 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 191, 2023-10-26T05:58:58.871 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:58.871 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:58.872 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:57.314223+0000", 2023-10-26T05:58:58.872 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:58.872 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:58.872 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:58.872 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:58.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:58.873 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:58.873 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:58.873 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:58.873 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:58.874 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:58.874 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:58.874 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:58.874 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:58.874 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:58.875 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:58.875 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:58.875 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:58.875 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:58.875 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:58.876 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:58.876 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:58.876 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:58.876 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.876 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:58.877 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:58.877 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:58.877 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:58.877 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:58.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:58.878 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:58.878 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:58.878 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:58.878 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:58.878 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:58.878 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:58.879 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:58.879 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:58.879 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:58.879 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:58.879 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:58.880 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:58.880 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:58.880 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:58.880 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:58.880 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:58.881 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:58.881 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:58.881 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:58.881 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:58.881 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:58.881 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.882 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:58.882 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:58.882 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:58.882 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:58.882 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:58.883 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:58.883 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:58.883 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:58.883 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:58.883 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:58.884 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:58.884 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:58.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:58.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:58.884 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:58.884 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:58.885 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:58.885 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:58.885 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:58.885 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:58.885 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:58.886 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:58.886 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:58.886 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:58.886 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:58.886 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:58.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:58.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.887 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:58.887 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:58.887 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:58.887 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:58.888 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:58.888 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:58.888 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:58.888 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:58.888 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:58.889 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:58.889 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:58.889 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:58.889 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:58.889 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:58.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.890 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:58.890 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:58.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.890 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:58.890 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:58.891 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:58.891 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:58.891 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:58.891 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:58.891 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:58.892 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:58.892 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:58.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.893 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:58.893 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:58.893 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:58.893 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:58.893 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:58.893 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:58.894 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:58.894 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:58.894 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:58.894 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:58.896 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:58.896 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:58.896 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:58.896 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:58.896 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:58.897 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:58.897 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:58.897 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:58.897 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:58.897 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:58.898 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:58.898 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.898 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:58.898 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:58.898 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:58.898 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:58.899 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:58.899 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:58.899 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:58.899 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:58.899 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:58.900 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:58.900 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:58.900 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:58.900 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:58.900 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:58.901 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:58.901 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:58.901 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:58.901 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:58.901 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:58.901 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:58.902 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:58.902 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:58.902 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:58.902 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:58.902 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:58.903 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:58.903 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:58.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.903 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:58.903 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:58.903 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:58.904 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:58.904 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:58.904 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:58.904 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:58.904 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:58.905 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:58.905 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:58.905 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:58.905 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:58.905 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:58.905 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:58.906 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:58.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.906 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:58.906 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:58.906 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:58.907 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.907 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:58.907 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:58.907 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:58.907 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:58.908 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:58.908 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:58.908 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:58.908 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:58.908 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:58.909 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.909 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.909 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:58.909 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:58.910 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:58.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:58.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:58.910 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:58.911 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:58.911 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:58.911 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:58.911 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:58.911 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:58.912 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:58.912 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:58.912 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:58.912 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:58.913 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:58.913 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:58.913 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:58.913 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:58.914 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:58.914 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:58.914 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:58.914 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:58.915 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:58.915 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:58.915 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:58.915 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:58.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.916 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "191", 2023-10-26T05:58:58.916 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:58.916 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:58.916 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:58.917 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:58.917 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:58.917 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:58.917 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:58.917 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:58.918 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:58.918 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:58.918 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:58.918 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:58.918 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:58.919 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:58.919 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:58.919 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:58.919 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:58.919 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:58.920 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:58.920 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:58.920 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:58.920 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:58.921 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:58.921 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:58.921 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:58.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:58.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.922 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:58.922 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:58.922 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:58.923 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:58.923 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:58.923 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:58.923 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:58.924 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:58.924 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:58.924 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:58.924 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:58.925 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:58.925 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:58.925 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:58.925 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.926 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:58.926 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:58.926 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:58.926 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.927 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:58.927 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:58.927 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:58.928 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:58.928 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:58.928 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:58.928 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:58.929 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:58.929 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:58.929 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.929 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.930 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:58.930 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:58.930 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.931 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:58.931 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:58.931 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:58.931 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:58.932 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:58.932 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:58.932 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:58.932 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:58.932 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:58.933 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:58.933 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:58.933 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:58.933 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:58.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.934 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.934 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:58.935 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.935 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.935 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.936 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:58.936 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.936 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.936 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.936 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.937 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:58.937 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.937 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.937 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:58.938 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.938 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.939 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:58.939 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.939 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.939 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.940 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:58.940 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.940 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.941 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.941 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:58.941 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.942 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.942 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:58.942 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.943 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.943 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.943 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:58.943 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.944 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.944 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.944 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:58.944 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.945 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.945 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:58.946 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:58.946 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.946 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.947 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:58.947 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:58.947 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:58.947 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:58.947 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:58.948 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:58.948 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:58.948 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.948 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.949 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:58.949 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:58.949 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:58.949 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:58.950 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:58.950 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:58.950 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:58.950 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:58.950 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:58.951 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:58.951 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:58.951 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:58.951 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:58.951 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:58.952 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:58.953 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.954 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.955 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.956 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.958 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.960 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.961 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.961 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:58.961 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.961 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.962 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.962 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:58.962 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.962 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.963 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.963 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:58.963 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:58.963 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.963 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.964 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:58.964 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:58.964 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:58.964 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:58.965 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:58.965 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:58.965 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:58.965 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.966 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.966 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:58.966 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:58.966 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:58.967 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:58.967 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:58.967 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:58.967 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:58.967 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:58.968 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:58.968 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:58.968 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:58.968 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:58.968 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:58.969 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.969 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.969 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:58.969 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.970 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.970 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.970 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.970 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:58.971 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.971 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.971 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.971 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:58.972 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.972 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.972 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.972 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:58.972 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.973 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.973 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.973 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.973 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:58.973 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.974 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.974 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:58.974 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.975 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.975 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:58.975 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.976 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.976 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:58.976 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.976 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.976 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.977 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:58.977 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.977 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.977 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.978 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:58.978 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.978 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.978 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.979 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:58.979 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:58.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.979 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.980 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:58.980 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:58.980 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:58.980 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:58.980 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:58.980 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:58.981 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:58.981 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.981 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.981 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:58.982 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:58.982 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:58.982 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:58.982 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:58.982 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:58.983 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:58.983 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:58.983 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:58.983 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:58.983 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:58.984 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:58.984 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:58.984 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.985 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:58.985 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.985 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.985 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.986 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:58.986 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.986 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.986 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.987 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:58.987 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.987 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.987 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.987 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:58.988 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.988 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.988 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.988 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:58.988 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.989 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.989 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.989 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:58.989 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.990 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.990 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.990 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:58.990 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.990 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.991 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.991 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.991 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:58.991 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.992 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.992 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:58.992 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.992 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:58.993 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:58.993 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.993 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.993 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:58.994 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:58.994 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:58.994 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:58.994 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.995 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:58.995 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:58.995 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:58.995 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:58.995 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:58.996 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:58.996 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:58.996 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:58.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:58.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:58.997 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:58.997 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:58.997 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:58.997 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:58.998 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:58.998 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:58.998 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:58.998 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:58.998 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:58.999 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:58.999 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:58.999 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:58.999 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:58.999 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:58.999 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.000 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.000 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:59.000 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.000 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.000 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.001 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.001 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:59.001 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.001 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.002 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.002 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.002 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.002 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.003 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:59.003 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.003 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.003 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:59.004 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.004 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.004 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.004 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.004 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.005 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.005 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.005 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:59.005 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.006 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.006 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:59.006 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.007 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.007 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.007 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.007 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.008 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:59.008 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.008 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.008 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.009 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:59.009 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.009 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.009 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.010 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.010 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:59.010 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:59.010 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:59.010 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.013 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.013 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.013 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.013 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.013 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.014 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.014 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.014 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.014 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.014 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.015 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.015 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:59.015 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.015 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.015 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.016 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:59.016 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.016 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.016 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.016 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.017 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.017 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.017 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.017 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:59.018 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.018 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.018 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:59.019 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.019 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.019 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.019 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.019 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.020 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:59.020 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.021 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.021 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:59.021 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.021 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.022 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.022 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.022 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.022 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.022 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.023 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.023 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:59.023 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.023 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.024 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.024 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:59.024 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.024 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.024 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.025 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:59.025 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:59.025 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:59.025 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:59.025 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.026 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.026 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.026 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.026 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.027 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:59.027 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.027 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.027 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.027 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.027 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.028 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.028 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.028 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.028 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.028 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.029 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.029 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.029 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.029 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.030 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:59.030 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.030 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.030 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.031 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:59.031 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.031 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.031 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.032 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.032 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.032 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.032 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:59.032 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.033 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.033 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.033 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:59.033 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.034 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.034 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.034 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.035 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.035 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.035 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:59.035 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.036 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.036 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:59.036 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.036 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.036 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.037 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.037 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.037 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.038 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:59.038 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.038 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.038 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.039 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:59.039 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.039 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.039 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.040 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:59.040 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:59.040 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:59.040 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:59.040 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.041 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.041 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.041 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.041 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.044 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.044 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.044 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.044 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.044 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.045 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:59.045 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.046 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.047 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:59.048 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.049 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.050 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.050 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.050 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:59.050 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.051 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.051 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.051 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:59.051 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.051 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.052 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.052 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.052 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.052 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.052 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.053 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:59.053 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.053 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.053 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.054 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:59.054 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.054 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.054 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.055 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:59.055 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:59.055 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:59.055 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:59.055 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.056 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.056 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.056 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.056 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:59.057 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:59.057 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.057 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:59.057 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.057 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.058 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.058 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.058 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.058 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:59.058 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.059 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.060 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.060 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.060 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:59.060 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.060 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.061 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.061 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:59.061 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.061 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.061 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.062 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.062 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.062 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:59.062 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.063 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.063 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:59.063 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.063 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.063 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.063 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.064 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.064 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:59.064 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.064 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.064 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.065 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:59.065 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.065 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.065 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.065 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.065 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.066 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:59.066 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.066 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.067 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:59.067 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.067 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.067 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.067 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.068 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.068 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:59.068 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.068 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.069 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:59.069 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.069 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.069 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.069 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.070 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.070 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:59.070 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.070 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.071 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:59.071 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.071 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.071 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.071 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.071 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.072 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:59.072 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.072 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.072 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:59.072 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:59.073 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:59.073 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:59.073 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:59.073 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:59.073 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:59.074 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:59.074 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:59.074 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:59.074 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:59.074 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:59.075 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:59.075 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:59.075 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:59.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.076 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:59.076 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:59.076 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:59.076 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:59.076 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:59.076 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:59.077 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:59.077 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:59.077 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:59.077 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:59.077 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:59.078 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.078 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:59.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe854066470 msgr2=0x7fe854068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe854066470 0x7fe854068930 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0a9000 tx=0x7fe85c019540 comp rx=0 tx=0).stop 2023-10-26T05:58:59.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 msgr2=0x7fe8680adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe8680adff0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fe85800f430 tx=0x7fe85800f8f0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.080 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 shutdown_connections 2023-10-26T05:58:59.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8681611e0 0x7fe868140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe868160d60 0x7fe86813ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe854066470 0x7fe854068930 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 --2- 172.21.15.71:0/237675032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe8500047d0 0x7fe8680adff0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.081 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 >> 172.21.15.71:0/237675032 conn(0x7fe868069770 msgr2=0x7fe868069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:59.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 shutdown_connections 2023-10-26T05:58:59.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:58.863+0000 7fe86eea7640 1 -- 172.21.15.71:0/237675032 wait complete. 2023-10-26T05:58:59.082 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-10-26T05:58:59.083 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-10-26T05:58:59.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/7649009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 msgr2=0x7fccfc15b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 --2- 172.21.15.71:0/7649009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc15b8e0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0c4dd0 tx=0x7fccf401c7b0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/7649009 shutdown_connections 2023-10-26T05:58:59.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 --2- 172.21.15.71:0/7649009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc15b8e0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.215 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 --2- 172.21.15.71:0/7649009 >> v2:172.21.15.86:3300/0 conn(0x7fccfc0d0850 0x7fccfc15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/7649009 >> 172.21.15.71:0/7649009 conn(0x7fccfc0cdd80 msgr2=0x7fccfc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/7649009 shutdown_connections 2023-10-26T05:58:59.216 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/7649009 wait complete. 2023-10-26T05:58:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 Processor -- start 2023-10-26T05:58:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- start start 2023-10-26T05:58:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccfc0d0470 0x7fccfc1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccfc0d0850 0x7fccfc14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc15c8b0 con 0x7fccfc0cd880 2023-10-26T05:58:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc15aaf0 con 0x7fccfc0d0850 2023-10-26T05:58:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc151410 con 0x7fccfc0d0470 2023-10-26T05:58:59.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45814/0 (socket says 172.21.15.71:45814) 2023-10-26T05:58:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 -- 172.21.15.71:0/711768849 learned_addr learned my addr 172.21.15.71:0/711768849 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd00cbf640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccfc0d0470 0x7fccfc1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd01cc1640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccfc0d0850 0x7fccfc14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 -- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccfc0d0470 msgr2=0x7fccfc1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccfc0d0470 0x7fccfc1539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 -- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccfc0d0850 msgr2=0x7fccfc14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccfc0d0850 0x7fccfc14f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 -- 172.21.15.71:0/711768849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4009560 con 0x7fccfc0cd880 2023-10-26T05:58:59.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd014c0640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc151860 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c490 tx=0x7fccec001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec003600 con 0x7fccfc0cd880 2023-10-26T05:58:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fccec003780 con 0x7fccfc0cd880 2023-10-26T05:58:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc159f00 con 0x7fccfc0cd880 2023-10-26T05:58:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec003bd0 con 0x7fccfc0cd880 2023-10-26T05:58:59.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc15a150 con 0x7fccfc0cd880 2023-10-26T05:58:59.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.207+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc066b10 con 0x7fccfc0cd880 2023-10-26T05:58:59.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.211+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fccec003d30 con 0x7fccfc0cd880 2023-10-26T05:58:59.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.211+0000 7fccf9ffb640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcce8066470 0x7fcce8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.211+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccec083610 con 0x7fccfc0cd880 2023-10-26T05:58:59.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.211+0000 7fcd00cbf640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcce8066470 0x7fcce8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.211+0000 7fcd00cbf640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcce8066470 0x7fcce8068930 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0c7870 tx=0x7fccf0005f50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:59.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.215+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fccec050a90 con 0x7fccfc0cd880 2023-10-26T05:58:59.377 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.367+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fccfc0721e0 con 0x7fccfc0cd880 2023-10-26T05:58:59.379 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fccf9ffb640 1 -- 172.21.15.71:0/711768849 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v191) v1 ==== 81+0+33930 (secure 0 0 0) 0x7fccec0540b0 con 0x7fccfc0cd880 2023-10-26T05:58:59.380 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:59.380 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:58:59.380 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 191, 2023-10-26T05:58:59.381 INFO:teuthology.orchestra.run.smithi071.stdout: "fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", 2023-10-26T05:58:59.381 INFO:teuthology.orchestra.run.smithi071.stdout: "created": "2023-10-26T05:47:02.367858+0000", 2023-10-26T05:58:59.381 INFO:teuthology.orchestra.run.smithi071.stdout: "modified": "2023-10-26T05:58:57.314223+0000", 2023-10-26T05:58:59.381 INFO:teuthology.orchestra.run.smithi071.stdout: "last_up_change": "2023-10-26T05:47:09.277950+0000", 2023-10-26T05:58:59.381 INFO:teuthology.orchestra.run.smithi071.stdout: "last_in_change": "2023-10-26T05:47:06.255185+0000", 2023-10-26T05:58:59.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-10-26T05:58:59.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_num": 5799936, 2023-10-26T05:58:59.385 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_set": [ 2023-10-26T05:58:59.385 INFO:teuthology.orchestra.run.smithi071.stdout: "pglog_hardlimit", 2023-10-26T05:58:59.385 INFO:teuthology.orchestra.run.smithi071.stdout: "purged_snapdirs", 2023-10-26T05:58:59.385 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_deletes", 2023-10-26T05:58:59.386 INFO:teuthology.orchestra.run.smithi071.stdout: "sortbitwise" 2023-10-26T05:58:59.386 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:59.386 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_version": 4, 2023-10-26T05:58:59.386 INFO:teuthology.orchestra.run.smithi071.stdout: "full_ratio": 0.69999998807907104, 2023-10-26T05:58:59.387 INFO:teuthology.orchestra.run.smithi071.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-10-26T05:58:59.387 INFO:teuthology.orchestra.run.smithi071.stdout: "nearfull_ratio": 0.60000002384185791, 2023-10-26T05:58:59.387 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_snapshot": "", 2023-10-26T05:58:59.387 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_max": 23, 2023-10-26T05:58:59.387 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd": 8, 2023-10-26T05:58:59.387 INFO:teuthology.orchestra.run.smithi071.stdout: "require_min_compat_client": "luminous", 2023-10-26T05:58:59.388 INFO:teuthology.orchestra.run.smithi071.stdout: "min_compat_client": "jewel", 2023-10-26T05:58:59.388 INFO:teuthology.orchestra.run.smithi071.stdout: "require_osd_release": "reef", 2023-10-26T05:58:59.388 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_crimson": false, 2023-10-26T05:58:59.388 INFO:teuthology.orchestra.run.smithi071.stdout: "pools": [ 2023-10-26T05:58:59.389 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.389 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 1, 2023-10-26T05:58:59.389 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": ".mgr", 2023-10-26T05:58:59.389 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:47:09.468386+0000", 2023-10-26T05:58:59.389 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:59.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:59.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:59.390 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:59.390 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:59.391 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:59.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:59.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:59.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:59.391 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:59.391 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:59.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:59.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 1, 2023-10-26T05:58:59.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 1, 2023-10-26T05:58:59.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 1, 2023-10-26T05:58:59.392 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 1, 2023-10-26T05:58:59.393 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 1, 2023-10-26T05:58:59.393 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:59.393 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:59.393 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:59.393 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:59.394 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:59.394 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:59.394 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:59.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.394 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "112", 2023-10-26T05:58:59.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:59.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:59.395 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:59.395 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:59.395 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:59.396 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:59.396 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:59.396 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:59.396 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:59.396 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:59.397 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:59.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:59.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:59.397 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:59.397 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:59.397 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:59.398 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:59.398 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:59.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:59.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:59.398 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:59.399 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:59.399 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:59.399 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:59.399 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:59.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:59.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.400 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:59.400 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:59.400 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:59.400 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:59.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:59.401 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:59.401 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:59.401 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:59.401 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:59.402 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:59.402 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:59.402 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:59.402 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_max": 32, 2023-10-26T05:58:59.402 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 1 2023-10-26T05:58:59.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.403 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:59.403 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr": {} 2023-10-26T05:58:59.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.403 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:59.403 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 8, 2023-10-26T05:58:59.404 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 8, 2023-10-26T05:58:59.404 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 0.25, 2023-10-26T05:58:59.404 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:59.404 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:59.404 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:59.405 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:59.405 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:59.405 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.405 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.406 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 22, 2023-10-26T05:58:59.406 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_metadata", 2023-10-26T05:58:59.406 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.268236+0000", 2023-10-26T05:58:59.406 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:59.406 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:59.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:59.407 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:59.407 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:59.407 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:59.407 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:59.408 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:59.408 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:59.408 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:59.408 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:59.408 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:59.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 32, 2023-10-26T05:58:59.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 32, 2023-10-26T05:58:59.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 32, 2023-10-26T05:58:59.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 32, 2023-10-26T05:58:59.409 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 32, 2023-10-26T05:58:59.410 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:59.410 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:59.410 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:59.410 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:59.410 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:59.411 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:59.411 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:59.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.411 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "187", 2023-10-26T05:58:59.411 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:59.412 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:59.412 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:59.412 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:59.412 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:59.412 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:59.413 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:59.413 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:59.413 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:59.413 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 0, 2023-10-26T05:58:59.413 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:59.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:59.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:59.414 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:59.414 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:59.414 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:59.414 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:59.415 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:59.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:59.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:59.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:59.415 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:59.416 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:59.416 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:59.416 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:59.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:59.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.417 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:59.417 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:59.417 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:59.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:59.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:59.418 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:59.418 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:59.418 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:59.418 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:59.418 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:59.418 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:59.419 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:59.419 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_bias": 4, 2023-10-26T05:58:59.419 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 16, 2023-10-26T05:58:59.419 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_priority": 5 2023-10-26T05:58:59.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.420 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:59.420 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:59.420 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": "cephfs" 2023-10-26T05:58:59.420 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.421 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:59.421 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 2, 2023-10-26T05:58:59.421 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 2, 2023-10-26T05:58:59.421 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:59.421 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 2, 2023-10-26T05:58:59.422 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 2, 2023-10-26T05:58:59.422 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:59.422 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:59.422 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:59.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.423 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.423 INFO:teuthology.orchestra.run.smithi071.stdout: "pool": 23, 2023-10-26T05:58:59.423 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": "cephfs_data", 2023-10-26T05:58:59.423 INFO:teuthology.orchestra.run.smithi071.stdout: "create_time": "2023-10-26T05:58:20.996636+0000", 2023-10-26T05:58:59.424 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2023-10-26T05:58:59.424 INFO:teuthology.orchestra.run.smithi071.stdout: "flags_names": "hashpspool", 2023-10-26T05:58:59.424 INFO:teuthology.orchestra.run.smithi071.stdout: "type": 1, 2023-10-26T05:58:59.424 INFO:teuthology.orchestra.run.smithi071.stdout: "size": 2, 2023-10-26T05:58:59.424 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": 1, 2023-10-26T05:58:59.424 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_rule": 0, 2023-10-26T05:58:59.425 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_count": 0, 2023-10-26T05:58:59.425 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_target": 0, 2023-10-26T05:58:59.425 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_barrier": 0, 2023-10-26T05:58:59.425 INFO:teuthology.orchestra.run.smithi071.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-10-26T05:58:59.425 INFO:teuthology.orchestra.run.smithi071.stdout: "object_hash": 2, 2023-10-26T05:58:59.426 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscale_mode": "off", 2023-10-26T05:58:59.426 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": 64, 2023-10-26T05:58:59.426 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num": 64, 2023-10-26T05:58:59.426 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_placement_num_target": 64, 2023-10-26T05:58:59.426 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_target": 64, 2023-10-26T05:58:59.427 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_pending": 64, 2023-10-26T05:58:59.427 INFO:teuthology.orchestra.run.smithi071.stdout: "last_pg_merge_meta": { 2023-10-26T05:58:59.427 INFO:teuthology.orchestra.run.smithi071.stdout: "source_pgid": "0.0", 2023-10-26T05:58:59.427 INFO:teuthology.orchestra.run.smithi071.stdout: "ready_epoch": 0, 2023-10-26T05:58:59.427 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_started": 0, 2023-10-26T05:58:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: "last_epoch_clean": 0, 2023-10-26T05:58:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: "source_version": "0'0", 2023-10-26T05:58:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: "target_version": "0'0" 2023-10-26T05:58:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: "last_change": "191", 2023-10-26T05:58:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend": "0", 2023-10-26T05:58:59.429 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_prenautilus": "0", 2023-10-26T05:58:59.429 INFO:teuthology.orchestra.run.smithi071.stdout: "last_force_op_resend_preluminous": "0", 2023-10-26T05:58:59.429 INFO:teuthology.orchestra.run.smithi071.stdout: "auid": 0, 2023-10-26T05:58:59.429 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_mode": "selfmanaged", 2023-10-26T05:58:59.429 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_seq": 0, 2023-10-26T05:58:59.430 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_epoch": 0, 2023-10-26T05:58:59.430 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_snaps": [], 2023-10-26T05:58:59.430 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps": "[]", 2023-10-26T05:58:59.430 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_bytes": 33554432, 2023-10-26T05:58:59.430 INFO:teuthology.orchestra.run.smithi071.stdout: "quota_max_objects": 0, 2023-10-26T05:58:59.431 INFO:teuthology.orchestra.run.smithi071.stdout: "tiers": [], 2023-10-26T05:58:59.431 INFO:teuthology.orchestra.run.smithi071.stdout: "tier_of": -1, 2023-10-26T05:58:59.431 INFO:teuthology.orchestra.run.smithi071.stdout: "read_tier": -1, 2023-10-26T05:58:59.431 INFO:teuthology.orchestra.run.smithi071.stdout: "write_tier": -1, 2023-10-26T05:58:59.431 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_mode": "none", 2023-10-26T05:58:59.432 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_bytes": 0, 2023-10-26T05:58:59.432 INFO:teuthology.orchestra.run.smithi071.stdout: "target_max_objects": 0, 2023-10-26T05:58:59.432 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-10-26T05:58:59.432 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-10-26T05:58:59.432 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_target_full_ratio_micro": 800000, 2023-10-26T05:58:59.433 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_flush_age": 0, 2023-10-26T05:58:59.433 INFO:teuthology.orchestra.run.smithi071.stdout: "cache_min_evict_age": 0, 2023-10-26T05:58:59.433 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profile": "", 2023-10-26T05:58:59.433 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_params": { 2023-10-26T05:58:59.433 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "none" 2023-10-26T05:58:59.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.434 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_period": 0, 2023-10-26T05:58:59.434 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_count": 0, 2023-10-26T05:58:59.434 INFO:teuthology.orchestra.run.smithi071.stdout: "use_gmt_hitset": true, 2023-10-26T05:58:59.434 INFO:teuthology.orchestra.run.smithi071.stdout: "min_read_recency_for_promote": 0, 2023-10-26T05:58:59.434 INFO:teuthology.orchestra.run.smithi071.stdout: "min_write_recency_for_promote": 0, 2023-10-26T05:58:59.435 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_grade_decay_rate": 0, 2023-10-26T05:58:59.435 INFO:teuthology.orchestra.run.smithi071.stdout: "hit_set_search_last_n": 0, 2023-10-26T05:58:59.435 INFO:teuthology.orchestra.run.smithi071.stdout: "grade_table": [], 2023-10-26T05:58:59.435 INFO:teuthology.orchestra.run.smithi071.stdout: "stripe_width": 0, 2023-10-26T05:58:59.435 INFO:teuthology.orchestra.run.smithi071.stdout: "expected_num_objects": 0, 2023-10-26T05:58:59.436 INFO:teuthology.orchestra.run.smithi071.stdout: "fast_read": false, 2023-10-26T05:58:59.436 INFO:teuthology.orchestra.run.smithi071.stdout: "options": { 2023-10-26T05:58:59.436 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num_min": 64, 2023-10-26T05:58:59.436 INFO:teuthology.orchestra.run.smithi071.stdout: "target_size_ratio": 0.89999997615814209 2023-10-26T05:58:59.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.437 INFO:teuthology.orchestra.run.smithi071.stdout: "application_metadata": { 2023-10-26T05:58:59.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cephfs": { 2023-10-26T05:58:59.437 INFO:teuthology.orchestra.run.smithi071.stdout: "data": "cephfs" 2023-10-26T05:58:59.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.437 INFO:teuthology.orchestra.run.smithi071.stdout: "read_balance": { 2023-10-26T05:58:59.438 INFO:teuthology.orchestra.run.smithi071.stdout: "score_acting": 1.3799999952316284, 2023-10-26T05:58:59.438 INFO:teuthology.orchestra.run.smithi071.stdout: "score_stable": 1.3799999952316284, 2023-10-26T05:58:59.438 INFO:teuthology.orchestra.run.smithi071.stdout: "optimal_score": 1, 2023-10-26T05:58:59.438 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_acting": 1.3799999952316284, 2023-10-26T05:58:59.438 INFO:teuthology.orchestra.run.smithi071.stdout: "raw_score_stable": 1.3799999952316284, 2023-10-26T05:58:59.439 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity_weighted": 1, 2023-10-26T05:58:59.439 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity": 1, 2023-10-26T05:58:59.439 INFO:teuthology.orchestra.run.smithi071.stdout: "average_primary_affinity_weighted": 1 2023-10-26T05:58:59.439 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.439 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.440 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:59.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osds": [ 2023-10-26T05:58:59.440 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:59.440 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.441 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.441 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.441 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.441 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.441 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.441 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.442 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.442 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.442 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.442 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.442 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.443 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.443 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.443 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6808", 2023-10-26T05:58:59.443 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.444 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.444 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6809", 2023-10-26T05:58:59.444 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.445 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.445 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.445 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.445 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.445 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.446 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.446 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6810", 2023-10-26T05:58:59.446 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.446 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.447 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6812", 2023-10-26T05:58:59.447 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.447 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.448 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.448 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.448 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.449 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6819", 2023-10-26T05:58:59.449 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.449 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.449 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6823", 2023-10-26T05:58:59.450 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.450 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.451 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.451 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.451 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.451 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6815", 2023-10-26T05:58:59.452 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.452 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.452 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.452 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6817", 2023-10-26T05:58:59.453 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 4161483918 2023-10-26T05:58:59.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.453 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.453 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6809/4161483918", 2023-10-26T05:58:59.453 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6812/4161483918", 2023-10-26T05:58:59.454 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6823/4161483918", 2023-10-26T05:58:59.454 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6817/4161483918", 2023-10-26T05:58:59.454 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.454 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.454 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.455 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.455 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:59.455 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.456 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.456 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.456 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.456 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.456 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.457 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.457 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.457 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.457 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.457 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.458 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.458 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.458 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.458 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2023-10-26T05:58:59.458 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.459 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.459 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.459 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2023-10-26T05:58:59.459 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.460 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.460 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.460 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.460 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.461 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6802", 2023-10-26T05:58:59.461 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.462 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.462 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6803", 2023-10-26T05:58:59.462 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.462 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.463 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.463 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.463 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.463 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.464 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6806", 2023-10-26T05:58:59.464 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.464 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.464 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.464 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.465 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6807", 2023-10-26T05:58:59.465 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.465 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.465 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.465 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.466 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.466 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.466 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.466 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6804", 2023-10-26T05:58:59.466 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.467 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.467 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6805", 2023-10-26T05:58:59.467 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 619557138 2023-10-26T05:58:59.468 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.468 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.468 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6801/619557138", 2023-10-26T05:58:59.468 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6803/619557138", 2023-10-26T05:58:59.469 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6807/619557138", 2023-10-26T05:58:59.469 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6805/619557138", 2023-10-26T05:58:59.469 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.469 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.469 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.469 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.470 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.470 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:59.470 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.470 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.471 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.471 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.471 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.471 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.471 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.472 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.472 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.472 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.472 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.472 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.473 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.473 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.473 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.473 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6824", 2023-10-26T05:58:59.473 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.473 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.474 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6825", 2023-10-26T05:58:59.474 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.475 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.475 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.475 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.475 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.475 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.476 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.476 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6826", 2023-10-26T05:58:59.476 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.477 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6827", 2023-10-26T05:58:59.477 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.477 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.477 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.478 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.478 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.478 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.478 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.479 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6830", 2023-10-26T05:58:59.479 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.479 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.479 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.479 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6831", 2023-10-26T05:58:59.480 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.480 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.480 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.480 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.481 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.481 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.481 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.481 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6828", 2023-10-26T05:58:59.481 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.482 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.482 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.482 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6829", 2023-10-26T05:58:59.482 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2687547983 2023-10-26T05:58:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6825/2687547983", 2023-10-26T05:58:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6827/2687547983", 2023-10-26T05:58:59.483 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6831/2687547983", 2023-10-26T05:58:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6829/2687547983", 2023-10-26T05:58:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.484 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.485 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.485 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:59.485 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.485 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.486 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.486 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.486 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.486 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.486 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.487 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.487 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.487 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.487 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.487 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.488 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.488 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.488 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.488 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6811", 2023-10-26T05:58:59.488 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.489 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6813", 2023-10-26T05:58:59.489 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.490 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.490 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.490 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.491 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6814", 2023-10-26T05:58:59.491 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.491 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.492 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6816", 2023-10-26T05:58:59.492 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.492 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.492 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.493 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.493 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.493 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.494 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6821", 2023-10-26T05:58:59.494 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.494 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.494 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.494 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.495 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6822", 2023-10-26T05:58:59.495 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.495 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.495 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.495 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.496 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.496 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.496 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6818", 2023-10-26T05:58:59.496 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.497 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.497 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6820", 2023-10-26T05:58:59.497 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1313958381 2023-10-26T05:58:59.498 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.498 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.498 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.498 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.71:6813/1313958381", 2023-10-26T05:58:59.498 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.71:6816/1313958381", 2023-10-26T05:58:59.499 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.71:6822/1313958381", 2023-10-26T05:58:59.499 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.71:6820/1313958381", 2023-10-26T05:58:59.499 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.499 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.499 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.499 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.500 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.500 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:59.500 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.500 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.501 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.501 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.501 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.501 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.501 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.502 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.502 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.502 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.502 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.502 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.503 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.503 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.503 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.503 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6800", 2023-10-26T05:58:59.503 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6801", 2023-10-26T05:58:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.504 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.505 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.505 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.505 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.505 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.505 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.506 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6802", 2023-10-26T05:58:59.506 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.506 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.507 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.507 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6803", 2023-10-26T05:58:59.507 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.507 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.507 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.508 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.508 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.508 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.508 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.509 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6806", 2023-10-26T05:58:59.509 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.509 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.510 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6807", 2023-10-26T05:58:59.510 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.510 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.510 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.510 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.511 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.511 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.511 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.511 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6804", 2023-10-26T05:58:59.512 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.512 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.512 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.512 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.512 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6805", 2023-10-26T05:58:59.512 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 893150098 2023-10-26T05:58:59.513 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.513 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.513 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6801/893150098", 2023-10-26T05:58:59.513 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6803/893150098", 2023-10-26T05:58:59.514 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6807/893150098", 2023-10-26T05:58:59.514 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6805/893150098", 2023-10-26T05:58:59.514 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.514 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.514 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.515 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.515 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:59.515 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.516 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.516 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.516 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.516 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.516 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.517 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.517 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.517 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.517 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.517 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.518 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.518 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.518 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.518 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6808", 2023-10-26T05:58:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6809", 2023-10-26T05:58:59.519 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.520 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.520 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.520 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.520 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.520 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.521 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.521 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6810", 2023-10-26T05:58:59.521 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.522 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.522 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6811", 2023-10-26T05:58:59.522 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.522 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.522 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.523 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.523 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.523 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.523 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.524 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6814", 2023-10-26T05:58:59.524 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.524 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.524 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.525 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6815", 2023-10-26T05:58:59.525 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.525 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.525 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.526 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.526 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.526 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.526 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.526 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6812", 2023-10-26T05:58:59.527 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.527 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.527 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6813", 2023-10-26T05:58:59.528 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 88051276 2023-10-26T05:58:59.528 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.528 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.528 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6809/88051276", 2023-10-26T05:58:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6811/88051276", 2023-10-26T05:58:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6815/88051276", 2023-10-26T05:58:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6813/88051276", 2023-10-26T05:58:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.529 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.530 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.530 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.530 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:59.530 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.531 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.531 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.531 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.531 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.532 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.532 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.532 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.532 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.532 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.532 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.533 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.533 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.533 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6824", 2023-10-26T05:58:59.534 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.534 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.534 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6825", 2023-10-26T05:58:59.535 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.535 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.535 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.535 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.536 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.536 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.536 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6826", 2023-10-26T05:58:59.536 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.537 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.537 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.537 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6827", 2023-10-26T05:58:59.537 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.537 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.538 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.538 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.538 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.538 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.539 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.539 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6830", 2023-10-26T05:58:59.539 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.539 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.540 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.540 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6831", 2023-10-26T05:58:59.540 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.540 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.540 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.541 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.541 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.541 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.541 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6828", 2023-10-26T05:58:59.542 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.542 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.542 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.542 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6829", 2023-10-26T05:58:59.543 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1847760170 2023-10-26T05:58:59.543 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.543 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.543 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6825/1847760170", 2023-10-26T05:58:59.544 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6827/1847760170", 2023-10-26T05:58:59.544 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6831/1847760170", 2023-10-26T05:58:59.544 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6829/1847760170", 2023-10-26T05:58:59.544 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.544 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.544 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.545 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.545 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.545 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.545 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:59.545 INFO:teuthology.orchestra.run.smithi071.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-10-26T05:58:59.546 INFO:teuthology.orchestra.run.smithi071.stdout: "up": 1, 2023-10-26T05:58:59.546 INFO:teuthology.orchestra.run.smithi071.stdout: "in": 1, 2023-10-26T05:58:59.546 INFO:teuthology.orchestra.run.smithi071.stdout: "weight": 1, 2023-10-26T05:58:59.546 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_affinity": 1, 2023-10-26T05:58:59.546 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_begin": 0, 2023-10-26T05:58:59.547 INFO:teuthology.orchestra.run.smithi071.stdout: "last_clean_end": 0, 2023-10-26T05:58:59.547 INFO:teuthology.orchestra.run.smithi071.stdout: "up_from": 13, 2023-10-26T05:58:59.547 INFO:teuthology.orchestra.run.smithi071.stdout: "up_thru": 186, 2023-10-26T05:58:59.547 INFO:teuthology.orchestra.run.smithi071.stdout: "down_at": 0, 2023-10-26T05:58:59.547 INFO:teuthology.orchestra.run.smithi071.stdout: "lost_at": 0, 2023-10-26T05:58:59.548 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addrs": { 2023-10-26T05:58:59.548 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.548 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.548 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6816", 2023-10-26T05:58:59.548 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.549 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.549 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.549 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6817", 2023-10-26T05:58:59.549 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.550 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.550 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.550 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addrs": { 2023-10-26T05:58:59.550 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.551 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.551 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6818", 2023-10-26T05:58:59.551 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.551 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.552 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.552 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6819", 2023-10-26T05:58:59.552 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.552 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.553 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addrs": { 2023-10-26T05:58:59.553 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.553 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.554 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6822", 2023-10-26T05:58:59.554 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.555 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6823", 2023-10-26T05:58:59.555 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.555 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.555 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.555 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addrs": { 2023-10-26T05:58:59.556 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2023-10-26T05:58:59.556 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.556 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2023-10-26T05:58:59.556 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6820", 2023-10-26T05:58:59.556 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.557 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2023-10-26T05:58:59.557 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.86:6821", 2023-10-26T05:58:59.557 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2184210851 2023-10-26T05:58:59.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.558 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.558 INFO:teuthology.orchestra.run.smithi071.stdout: "public_addr": "172.21.15.86:6817/2184210851", 2023-10-26T05:58:59.558 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_addr": "172.21.15.86:6819/2184210851", 2023-10-26T05:58:59.559 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_back_addr": "172.21.15.86:6823/2184210851", 2023-10-26T05:58:59.559 INFO:teuthology.orchestra.run.smithi071.stdout: "heartbeat_front_addr": "172.21.15.86:6821/2184210851", 2023-10-26T05:58:59.559 INFO:teuthology.orchestra.run.smithi071.stdout: "state": [ 2023-10-26T05:58:59.559 INFO:teuthology.orchestra.run.smithi071.stdout: "exists", 2023-10-26T05:58:59.559 INFO:teuthology.orchestra.run.smithi071.stdout: "up" 2023-10-26T05:58:59.560 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2023-10-26T05:58:59.560 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.560 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:59.560 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_xinfo": [ 2023-10-26T05:58:59.560 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.561 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 0, 2023-10-26T05:58:59.561 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.561 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.561 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.561 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.562 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.562 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.382801+0000", 2023-10-26T05:58:59.562 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.562 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.563 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 1, 2023-10-26T05:58:59.563 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.563 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.563 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.563 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.564 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.564 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.392178+0000", 2023-10-26T05:58:59.564 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.564 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.564 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.564 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 2, 2023-10-26T05:58:59.565 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.565 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.565 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.565 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.565 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.566 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.360372+0000", 2023-10-26T05:58:59.566 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.566 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 3, 2023-10-26T05:58:59.567 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.567 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.567 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.567 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.567 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.568 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.406606+0000", 2023-10-26T05:58:59.568 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.568 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.568 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 4, 2023-10-26T05:58:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.569 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.390686+0000", 2023-10-26T05:58:59.570 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.570 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.570 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 5, 2023-10-26T05:58:59.570 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.571 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.380624+0000", 2023-10-26T05:58:59.572 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.572 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.572 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.572 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 6, 2023-10-26T05:58:59.572 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.572 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.573 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.573 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.573 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.573 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.358004+0000", 2023-10-26T05:58:59.573 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.574 INFO:teuthology.orchestra.run.smithi071.stdout: { 2023-10-26T05:58:59.574 INFO:teuthology.orchestra.run.smithi071.stdout: "osd": 7, 2023-10-26T05:58:59.574 INFO:teuthology.orchestra.run.smithi071.stdout: "down_stamp": "0.000000", 2023-10-26T05:58:59.574 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_probability": 0, 2023-10-26T05:58:59.575 INFO:teuthology.orchestra.run.smithi071.stdout: "laggy_interval": 0, 2023-10-26T05:58:59.575 INFO:teuthology.orchestra.run.smithi071.stdout: "features": 4540138322906710015, 2023-10-26T05:58:59.575 INFO:teuthology.orchestra.run.smithi071.stdout: "old_weight": 0, 2023-10-26T05:58:59.575 INFO:teuthology.orchestra.run.smithi071.stdout: "last_purged_snaps_scrub": "2023-10-26T05:47:07.410183+0000", 2023-10-26T05:58:59.575 INFO:teuthology.orchestra.run.smithi071.stdout: "dead_epoch": 0 2023-10-26T05:58:59.576 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.576 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2023-10-26T05:58:59.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap": [], 2023-10-26T05:58:59.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_items": [], 2023-10-26T05:58:59.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_upmap_primaries": [], 2023-10-26T05:58:59.576 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_temp": [], 2023-10-26T05:58:59.577 INFO:teuthology.orchestra.run.smithi071.stdout: "primary_temp": [], 2023-10-26T05:58:59.577 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklist": {}, 2023-10-26T05:58:59.577 INFO:teuthology.orchestra.run.smithi071.stdout: "range_blocklist": {}, 2023-10-26T05:58:59.577 INFO:teuthology.orchestra.run.smithi071.stdout: "erasure_code_profiles": { 2023-10-26T05:58:59.577 INFO:teuthology.orchestra.run.smithi071.stdout: "default": { 2023-10-26T05:58:59.578 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-failure-domain": "osd", 2023-10-26T05:58:59.578 INFO:teuthology.orchestra.run.smithi071.stdout: "k": "2", 2023-10-26T05:58:59.578 INFO:teuthology.orchestra.run.smithi071.stdout: "m": "1", 2023-10-26T05:58:59.578 INFO:teuthology.orchestra.run.smithi071.stdout: "plugin": "jerasure", 2023-10-26T05:58:59.578 INFO:teuthology.orchestra.run.smithi071.stdout: "technique": "reed_sol_van" 2023-10-26T05:58:59.579 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2023-10-26T05:58:59.579 INFO:teuthology.orchestra.run.smithi071.stdout: "removed_snaps_queue": [], 2023-10-26T05:58:59.579 INFO:teuthology.orchestra.run.smithi071.stdout: "new_removed_snaps": [], 2023-10-26T05:58:59.579 INFO:teuthology.orchestra.run.smithi071.stdout: "new_purged_snaps": [], 2023-10-26T05:58:59.580 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_node_flags": {}, 2023-10-26T05:58:59.580 INFO:teuthology.orchestra.run.smithi071.stdout: "device_class_flags": {}, 2023-10-26T05:58:59.580 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode": { 2023-10-26T05:58:59.580 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_enabled": false, 2023-10-26T05:58:59.580 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_bucket_count": 0, 2023-10-26T05:58:59.580 INFO:teuthology.orchestra.run.smithi071.stdout: "degraded_stretch_mode": 0, 2023-10-26T05:58:59.581 INFO:teuthology.orchestra.run.smithi071.stdout: "recovering_stretch_mode": 0, 2023-10-26T05:58:59.581 INFO:teuthology.orchestra.run.smithi071.stdout: "stretch_mode_bucket": 0 2023-10-26T05:58:59.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2023-10-26T05:58:59.581 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:58:59.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcce8066470 msgr2=0x7fcce8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcce8066470 0x7fcce8068930 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fccfc0c7870 tx=0x7fccf0005f50 comp rx=0 tx=0).stop 2023-10-26T05:58:59.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 msgr2=0x7fccfc151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc151860 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c490 tx=0x7fccec001e80 comp rx=0 tx=0).stop 2023-10-26T05:58:59.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 shutdown_connections 2023-10-26T05:58:59.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccfc0d0850 0x7fccfc14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccfc0d0470 0x7fccfc1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcce8066470 0x7fcce8068930 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 --2- 172.21.15.71:0/711768849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccfc0cd880 0x7fccfc151860 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 >> 172.21.15.71:0/711768849 conn(0x7fccfc0cdd80 msgr2=0x7fccfc0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:59.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 shutdown_connections 2023-10-26T05:58:59.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.371+0000 7fcd03724640 1 -- 172.21.15.71:0/711768849 wait complete. 2023-10-26T05:58:59.586 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:58:59.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/2633673931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015d390 msgr2=0x7fc0d015aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/2633673931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015d390 0x7fc0d015aa20 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009c80 tx=0x7fc0c801ccc0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/2633673931 shutdown_connections 2023-10-26T05:58:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/2633673931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015d390 0x7fc0d015aa20 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/2633673931 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 0x7fc0d015ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/2633673931 >> 172.21.15.71:0/2633673931 conn(0x7fc0d00b0f20 msgr2=0x7fc0d00b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/2633673931 shutdown_connections 2023-10-26T05:58:59.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/2633673931 wait complete. 2023-10-26T05:58:59.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 Processor -- start 2023-10-26T05:58:59.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- start start 2023-10-26T05:58:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 0x7fc0d0146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 0x7fc0d0150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d015d390 0x7fc0d0152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0147e70 con 0x7fc0d015c720 2023-10-26T05:58:59.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d0150ec0 con 0x7fc0d015d390 2023-10-26T05:58:59.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d7a2d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d00c7860 con 0x7fc0d00cede0 2023-10-26T05:58:59.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 0x7fc0d0150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 0x7fc0d0150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45836/0 (socket says 172.21.15.71:45836) 2023-10-26T05:58:59.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 -- 172.21.15.71:0/1361207190 learned_addr learned my addr 172.21.15.71:0/1361207190 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:58:59.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d57c9640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 0x7fc0d0146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d5fca640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d015d390 0x7fc0d0152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 -- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 msgr2=0x7fc0d0146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 0x7fc0d0146d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 -- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d015d390 msgr2=0x7fc0d0152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d015d390 0x7fc0d0152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d4fc8640 1 -- 172.21.15.71:0/1361207190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80098e0 con 0x7fc0d015c720 2023-10-26T05:58:59.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.707+0000 7fc0d57c9640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 0x7fc0d0146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:58:59.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0d4fc8640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 0x7fc0d0150980 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000d980 tx=0x7fc0c000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:59.723 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0009c30 con 0x7fc0d015c720 2023-10-26T05:58:59.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc0c0009e80 con 0x7fc0d015c720 2023-10-26T05:58:59.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d0147540 con 0x7fc0d015c720 2023-10-26T05:58:59.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00106a0 con 0x7fc0d015c720 2023-10-26T05:58:59.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d0147b00 con 0x7fc0d015c720 2023-10-26T05:58:59.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d0148020 con 0x7fc0d015c720 2023-10-26T05:58:59.724 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc0c0010800 con 0x7fc0d015c720 2023-10-26T05:58:59.725 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0c5ffb640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0b4066470 0x7fc0b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:58:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0c0084340 con 0x7fc0d015c720 2023-10-26T05:58:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0d57c9640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0b4066470 0x7fc0b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:58:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.711+0000 7fc0d57c9640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0b4066470 0x7fc0b4068930 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc001710 tx=0x7fc0cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:58:59.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.715+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc0d0148020 con 0x7fc0d015c720 2023-10-26T05:58:59.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.887+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0d0148020 con 0x7fc0d015c720 2023-10-26T05:58:59.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.887+0000 7fc0c5ffb640 1 -- 172.21.15.71:0/1361207190 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fc0d0148020 con 0x7fc0d015c720 2023-10-26T05:58:59.896 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:58:59.896 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:58:59.897 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:58:59.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.887+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0b4066470 msgr2=0x7fc0b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.887+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0b4066470 0x7fc0b4068930 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc001710 tx=0x7fc0cc009040 comp rx=0 tx=0).stop 2023-10-26T05:58:59.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.887+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 msgr2=0x7fc0d0150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:58:59.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.887+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 0x7fc0d0150980 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000d980 tx=0x7fc0c000de40 comp rx=0 tx=0).stop 2023-10-26T05:58:59.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 shutdown_connections 2023-10-26T05:58:59.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d015d390 0x7fc0d0152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc0b4066470 0x7fc0b4068930 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0d015c720 0x7fc0d0150980 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 --2- 172.21.15.71:0/1361207190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc0d00cede0 0x7fc0d0146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:58:59.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 >> 172.21.15.71:0/1361207190 conn(0x7fc0d00b0f20 msgr2=0x7fc0d00b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:58:59.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 shutdown_connections 2023-10-26T05:58:59.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:58:59.891+0000 7fc0d7a2d640 1 -- 172.21.15.71:0/1361207190 wait complete. 2023-10-26T05:58:59.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-10-26T05:59:00.057 INFO:teuthology.orchestra.run.smithi071.stdout:{"cluster_fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","whoami":0,"id":9387,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":37.341688066000003,"mdsmap_epoch":71,"osdmap_epoch":191,"osdmap_epoch_barrier":190,"uptime":42.152845290999998} 2023-10-26T05:59:00.057 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "e2c462c9-9202-452f-a70c-a4a81693dc19", "fs_name": "cephfs", "id": 9387, "mdsmap_epoch": 71, "osdmap_epoch": 191, "osdmap_epoch_barrier": 190, "rank_uptime": 37.341688066, "state": "up:active", "uptime": 42.152845291, "want_state": "up:active", "whoami": 0 } 2023-10-26T05:59:00.058 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-10-26T05:59:00.058 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:59:00.059 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:00.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:00.065 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-10-26T05:59:00.522 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records in 2023-10-26T05:59:00.522 INFO:teuthology.orchestra.run.smithi071.stderr:50+0 records out 2023-10-26T05:59:00.523 INFO:teuthology.orchestra.run.smithi071.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.404192 s, 130 MB/s 2023-10-26T05:59:00.524 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-26T05:59:00.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:59:00.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-26T05:59:00.648 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-26T05:59:00.649 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:59:00.649 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:00.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:59:00.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 -- 172.21.15.71:0/2344156396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013dc20 msgr2=0x7efc6015b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:00.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 --2- 172.21.15.71:0/2344156396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013dc20 0x7efc6015b9f0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7efc6006db30 tx=0x7efc5801cd60 comp rx=0 tx=0).stop 2023-10-26T05:59:00.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 -- 172.21.15.71:0/2344156396 shutdown_connections 2023-10-26T05:59:00.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 --2- 172.21.15.71:0/2344156396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc6013d800 0x7efc6015b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 --2- 172.21.15.71:0/2344156396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013dc20 0x7efc6015b9f0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 --2- 172.21.15.71:0/2344156396 >> v2:172.21.15.86:3300/0 conn(0x7efc6013d420 0x7efc600ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 -- 172.21.15.71:0/2344156396 >> 172.21.15.71:0/2344156396 conn(0x7efc60069770 msgr2=0x7efc60069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:00.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 -- 172.21.15.71:0/2344156396 shutdown_connections 2023-10-26T05:59:00.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.815+0000 7efc67323640 1 -- 172.21.15.71:0/2344156396 wait complete. 2023-10-26T05:59:00.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 Processor -- start 2023-10-26T05:59:00.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- start start 2023-10-26T05:59:00.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc6013d420 0x7efc6013ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:00.830 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 0x7efc600b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc6013dc20 0x7efc600ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600ac7e0 con 0x7efc6013d800 2023-10-26T05:59:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600acac0 con 0x7efc6013d420 2023-10-26T05:59:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc600acda0 con 0x7efc6013dc20 2023-10-26T05:59:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 0x7efc600b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 0x7efc600b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45852/0 (socket says 172.21.15.71:45852) 2023-10-26T05:59:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 -- 172.21.15.71:0/2066702922 learned_addr learned my addr 172.21.15.71:0/2066702922 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 -- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc6013dc20 msgr2=0x7efc600ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc6013dc20 0x7efc600ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc650bf640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc6013d420 0x7efc6013ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:00.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 -- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc6013d420 msgr2=0x7efc6013ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:00.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc6013d420 0x7efc6013ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 -- 172.21.15.71:0/2066702922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580098e0 con 0x7efc6013d800 2023-10-26T05:59:00.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc648be640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 0x7efc600b1a20 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c970 tx=0x7efc5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:00.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50007bb0 con 0x7efc6013d800 2023-10-26T05:59:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7efc50007e20 con 0x7efc6013d800 2023-10-26T05:59:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc600ad080 con 0x7efc6013d800 2023-10-26T05:59:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50005280 con 0x7efc6013d800 2023-10-26T05:59:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc600b0d80 con 0x7efc6013d800 2023-10-26T05:59:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.819+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60066b10 con 0x7efc6013d800 2023-10-26T05:59:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.823+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc500053e0 con 0x7efc6013d800 2023-10-26T05:59:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.823+0000 7efc55ffb640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc44066470 0x7efc44068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.823+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc500847a0 con 0x7efc6013d800 2023-10-26T05:59:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.823+0000 7efc650bf640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc44066470 0x7efc44068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.823+0000 7efc650bf640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc44066470 0x7efc44068930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efc601ea290 tx=0x7efc5c005bf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.827+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7efc50051c20 con 0x7efc6013d800 2023-10-26T05:59:00.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.983+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc600ac920 con 0x7efc6013d800 2023-10-26T05:59:00.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc55ffb640 1 -- 172.21.15.71:0/2066702922 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16495 (secure 0 0 0) 0x7efc600ac920 con 0x7efc6013d800 2023-10-26T05:59:00.996 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:00.996 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":191,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:57.314223+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:59:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc44066470 msgr2=0x7efc44068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc44066470 0x7efc44068930 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efc601ea290 tx=0x7efc5c005bf0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.998 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 msgr2=0x7efc600b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 0x7efc600b1a20 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7efc5000c970 tx=0x7efc5000ce30 comp rx=0 tx=0).stop 2023-10-26T05:59:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 shutdown_connections 2023-10-26T05:59:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc6013dc20 0x7efc600ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7efc44066470 0x7efc44068930 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc6013d800 0x7efc600b1a20 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:00.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 --2- 172.21.15.71:0/2066702922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc6013d420 0x7efc6013ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.987+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 >> 172.21.15.71:0/2066702922 conn(0x7efc60069770 msgr2=0x7efc6006f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.991+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 shutdown_connections 2023-10-26T05:59:01.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:00.991+0000 7efc67323640 1 -- 172.21.15.71:0/2066702922 wait complete. 2023-10-26T05:59:01.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:58:20.268236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:58:20.996636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:59:01.016 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:59:01.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- 172.21.15.71:0/349493973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 msgr2=0x7f078c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 --2- 172.21.15.71:0/349493973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c06a410 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f078c06da20 tx=0x7f078401cd20 comp rx=0 tx=0).stop 2023-10-26T05:59:01.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- 172.21.15.71:0/349493973 shutdown_connections 2023-10-26T05:59:01.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 --2- 172.21.15.71:0/349493973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c06a410 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 --2- 172.21.15.71:0/349493973 >> v2:172.21.15.86:3300/0 conn(0x7f078c14e7b0 0x7f078c157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.144 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- 172.21.15.71:0/349493973 >> 172.21.15.71:0/349493973 conn(0x7f078c0aec60 msgr2=0x7f078c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- 172.21.15.71:0/349493973 shutdown_connections 2023-10-26T05:59:01.145 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- 172.21.15.71:0/349493973 wait complete. 2023-10-26T05:59:01.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 Processor -- start 2023-10-26T05:59:01.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- start start 2023-10-26T05:59:01.147 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f078c071270 0x7f078c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f078c14e7b0 0x7f078c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1496a0 con 0x7f078c06a000 2023-10-26T05:59:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c148a90 con 0x7f078c14e7b0 2023-10-26T05:59:01.148 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f0792c41640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c148d70 con 0x7f078c071270 2023-10-26T05:59:01.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45874/0 (socket says 172.21.15.71:45874) 2023-10-26T05:59:01.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 -- 172.21.15.71:0/539057170 learned_addr learned my addr 172.21.15.71:0/539057170 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:01.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07911de640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f078c14e7b0 0x7f078c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.149 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f078bfff640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f078c071270 0x7f078c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 -- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f078c071270 msgr2=0x7f078c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f078c071270 0x7f078c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 -- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f078c14e7b0 msgr2=0x7f078c15a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f078c14e7b0 0x7f078c15a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.150 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f07909dd640 1 -- 172.21.15.71:0/539057170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840098e0 con 0x7f078c06a000 2023-10-26T05:59:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.135+0000 7f078bfff640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f078c071270 0x7f078c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07909dd640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c142420 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c9f0 tx=0x7f077c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c007bb0 con 0x7f078c06a000 2023-10-26T05:59:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f077c007e00 con 0x7f078c06a000 2023-10-26T05:59:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c149050 con 0x7f078c06a000 2023-10-26T05:59:01.151 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00f6b0 con 0x7f078c06a000 2023-10-26T05:59:01.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c142960 con 0x7f078c06a000 2023-10-26T05:59:01.152 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c066b10 con 0x7f078c06a000 2023-10-26T05:59:01.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f077c00f810 con 0x7f078c06a000 2023-10-26T05:59:01.153 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07897fa640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0778066470 0x7f0778068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f077c084700 con 0x7f078c06a000 2023-10-26T05:59:01.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f078bfff640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0778066470 0x7f0778068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.139+0000 7f078bfff640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0778066470 0x7f0778068930 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f078c1f5ef0 tx=0x7f0780009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:01.154 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.143+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f077c051b80 con 0x7f078c06a000 2023-10-26T05:59:01.306 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f078c0721e0 con 0x7f078c06a000 2023-10-26T05:59:01.307 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f07897fa640 1 -- 172.21.15.71:0/539057170 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f077c0551a0 con 0x7f078c06a000 2023-10-26T05:59:01.307 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:59:01.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0778066470 msgr2=0x7f0778068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0778066470 0x7f0778068930 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f078c1f5ef0 tx=0x7f0780009040 comp rx=0 tx=0).stop 2023-10-26T05:59:01.309 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 msgr2=0x7f078c142420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c142420 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c9f0 tx=0x7f077c00ceb0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 shutdown_connections 2023-10-26T05:59:01.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f078c14e7b0 0x7f078c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f078c071270 0x7f078c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.310 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0778066470 0x7f0778068930 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 --2- 172.21.15.71:0/539057170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f078c06a000 0x7f078c142420 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 >> 172.21.15.71:0/539057170 conn(0x7f078c0aec60 msgr2=0x7f078c157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 shutdown_connections 2023-10-26T05:59:01.311 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.299+0000 7f0792c41640 1 -- 172.21.15.71:0/539057170 wait complete. 2023-10-26T05:59:01.325 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:59:01.451 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- 172.21.15.71:0/3578660713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340cd880 msgr2=0x7fd03415bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3578660713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340cd880 0x7fd03415bab0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd024009840 tx=0x7fd02401c6b0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- 172.21.15.71:0/3578660713 shutdown_connections 2023-10-26T05:59:01.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3578660713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340cd880 0x7fd03415bab0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.452 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3578660713 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0340d0470 0x7fd03406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- 172.21.15.71:0/3578660713 >> 172.21.15.71:0/3578660713 conn(0x7fd0340cdd80 msgr2=0x7fd0340ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- 172.21.15.71:0/3578660713 shutdown_connections 2023-10-26T05:59:01.453 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- 172.21.15.71:0/3578660713 wait complete. 2023-10-26T05:59:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 Processor -- start 2023-10-26T05:59:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- start start 2023-10-26T05:59:01.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0340cd880 0x7fd03414edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 0x7fd03414f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0340d0850 0x7fd0340b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03415db30 con 0x7fd0340d0470 2023-10-26T05:59:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03415de10 con 0x7fd0340cd880 2023-10-26T05:59:01.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0399f5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03415e0f0 con 0x7fd0340d0850 2023-10-26T05:59:01.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 0x7fd03414f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 0x7fd03414f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45894/0 (socket says 172.21.15.71:45894) 2023-10-26T05:59:01.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 -- 172.21.15.71:0/3734457965 learned_addr learned my addr 172.21.15.71:0/3734457965 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:01.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd032ffd640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0340cd880 0x7fd03414edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0337fe640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0340d0850 0x7fd0340b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 -- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0340d0850 msgr2=0x7fd0340b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0340d0850 0x7fd0340b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 -- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0340cd880 msgr2=0x7fd03414edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0340cd880 0x7fd03414edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.458 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.443+0000 7fd0327fc640 1 -- 172.21.15.71:0/3734457965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0240094a0 con 0x7fd0340d0470 2023-10-26T05:59:01.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd0327fc640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 0x7fd03414f420 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c370 tx=0x7fd02800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:01.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028004230 con 0x7fd0340d0470 2023-10-26T05:59:01.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0280044a0 con 0x7fd0340d0470 2023-10-26T05:59:01.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03415e3d0 con 0x7fd0340d0470 2023-10-26T05:59:01.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028005240 con 0x7fd0340d0470 2023-10-26T05:59:01.459 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03415d5e0 con 0x7fd0340d0470 2023-10-26T05:59:01.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd034066b10 con 0x7fd0340d0470 2023-10-26T05:59:01.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0280053a0 con 0x7fd0340d0470 2023-10-26T05:59:01.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd01bfff640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd020066470 0x7fd020068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd028083440 con 0x7fd0340d0470 2023-10-26T05:59:01.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd032ffd640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd020066470 0x7fd020068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.447+0000 7fd032ffd640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd020066470 0x7fd020068930 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd02c005c50 tx=0x7fd02c005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:01.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.451+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0280508b0 con 0x7fd0340d0470 2023-10-26T05:59:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.603+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0340721e0 con 0x7fd0340d0470 2023-10-26T05:59:01.614 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd01bfff640 1 -- 172.21.15.71:0/3734457965 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v191) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd028053ed0 con 0x7fd0340d0470 2023-10-26T05:59:01.615 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:59:01.616 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd020066470 msgr2=0x7fd020068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd020066470 0x7fd020068930 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd02c005c50 tx=0x7fd02c005be0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 msgr2=0x7fd03414f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 0x7fd03414f420 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c370 tx=0x7fd02800c830 comp rx=0 tx=0).stop 2023-10-26T05:59:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 shutdown_connections 2023-10-26T05:59:01.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0340d0850 0x7fd0340b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd020066470 0x7fd020068930 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0340d0470 0x7fd03414f420 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 --2- 172.21.15.71:0/3734457965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0340cd880 0x7fd03414edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 >> 172.21.15.71:0/3734457965 conn(0x7fd0340cdd80 msgr2=0x7fd034158190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.618 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 shutdown_connections 2023-10-26T05:59:01.619 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.607+0000 7fd0399f5640 1 -- 172.21.15.71:0/3734457965 wait complete. 2023-10-26T05:59:01.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:59:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- 172.21.15.71:0/4204452593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0af990 msgr2=0x7f1bec0cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 --2- 172.21.15.71:0/4204452593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0af990 0x7f1bec0cbb10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009900 tx=0x7f1bdc01c800 comp rx=0 tx=0).stop 2023-10-26T05:59:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- 172.21.15.71:0/4204452593 shutdown_connections 2023-10-26T05:59:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 --2- 172.21.15.71:0/4204452593 >> v2:172.21.15.86:3300/0 conn(0x7f1bec0cc050 0x7f1bec0cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 --2- 172.21.15.71:0/4204452593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0af990 0x7f1bec0cbb10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- 172.21.15.71:0/4204452593 >> 172.21.15.71:0/4204452593 conn(0x7f1bec06f440 msgr2=0x7f1bec06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- 172.21.15.71:0/4204452593 shutdown_connections 2023-10-26T05:59:01.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- 172.21.15.71:0/4204452593 wait complete. 2023-10-26T05:59:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 Processor -- start 2023-10-26T05:59:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- start start 2023-10-26T05:59:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bec0af990 0x7f1bec0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0cc050 0x7f1bec14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 0x7f1bec151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec1373f0 con 0x7f1bec142b30 2023-10-26T05:59:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec137660 con 0x7f1bec0af990 2023-10-26T05:59:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf3597640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec137940 con 0x7f1bec0cc050 2023-10-26T05:59:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 0x7f1bec151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 0x7f1bec151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41904/0 (socket says 172.21.15.71:41904) 2023-10-26T05:59:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 -- 172.21.15.71:0/1880872184 learned_addr learned my addr 172.21.15.71:0/1880872184 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1333640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bec0af990 0x7f1bec0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf0b32640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0cc050 0x7f1bec14fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 -- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0cc050 msgr2=0x7f1bec14fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0cc050 0x7f1bec14fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 -- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bec0af990 msgr2=0x7f1bec0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bec0af990 0x7f1bec0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 -- 172.21.15.71:0/1880872184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc009560 con 0x7f1bec142b30 2023-10-26T05:59:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.759+0000 7f1bf1b34640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 0x7f1bec151c10 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f1be400eb20 tx=0x7f1be400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4004b30 con 0x7f1bec142b30 2023-10-26T05:59:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f1be4004cb0 con 0x7f1bec142b30 2023-10-26T05:59:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec137c20 con 0x7f1bec142b30 2023-10-26T05:59:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4005280 con 0x7f1bec142b30 2023-10-26T05:59:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec136c10 con 0x7f1bec142b30 2023-10-26T05:59:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec066b10 con 0x7f1bec142b30 2023-10-26T05:59:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1be40053e0 con 0x7f1bec142b30 2023-10-26T05:59:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1be1ffb640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1bd0066470 0x7f1bd0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1be4014070 con 0x7f1bec142b30 2023-10-26T05:59:01.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1bf1333640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1bd0066470 0x7f1bd0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:01.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.763+0000 7f1bf1333640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1bd0066470 0x7f1bd0068930 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1be80015b0 tx=0x7f1be8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:01.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.767+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f1be4052100 con 0x7f1bec142b30 2023-10-26T05:59:01.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.915+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1bec138140 con 0x7f1bec142b30 2023-10-26T05:59:01.926 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1be1ffb640 1 -- 172.21.15.71:0/1880872184 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v191) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1be4055720 con 0x7f1bec142b30 2023-10-26T05:59:01.927 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:59:01.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1bd0066470 msgr2=0x7f1bd0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1bd0066470 0x7f1bd0068930 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1be80015b0 tx=0x7f1be8009040 comp rx=0 tx=0).stop 2023-10-26T05:59:01.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 msgr2=0x7f1bec151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:01.929 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 0x7f1bec151c10 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f1be400eb20 tx=0x7f1be400c2d0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 shutdown_connections 2023-10-26T05:59:01.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f1bd0066470 0x7f1bd0068930 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1bec142b30 0x7f1bec151c10 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1bec0cc050 0x7f1bec14fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.930 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 --2- 172.21.15.71:0/1880872184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1bec0af990 0x7f1bec0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:01.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 >> 172.21.15.71:0/1880872184 conn(0x7f1bec06f440 msgr2=0x7f1bec138620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:01.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 shutdown_connections 2023-10-26T05:59:01.931 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:01.919+0000 7f1bf3597640 1 -- 172.21.15.71:0/1880872184 wait complete. 2023-10-26T05:59:01.944 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:59:02.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- 172.21.15.71:0/50365190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f4069d30 msgr2=0x7fd0f406a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.068 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 --2- 172.21.15.71:0/50365190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f4069d30 0x7fd0f406a1c0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009c80 tx=0x7fd0ec01ccb0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- 172.21.15.71:0/50365190 shutdown_connections 2023-10-26T05:59:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 --2- 172.21.15.71:0/50365190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f4069d30 0x7fd0f406a1c0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 --2- 172.21.15.71:0/50365190 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f40ca7f0 0x7fd0f4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- 172.21.15.71:0/50365190 >> 172.21.15.71:0/50365190 conn(0x7fd0f406b8c0 msgr2=0x7fd0f406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:02.069 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- 172.21.15.71:0/50365190 shutdown_connections 2023-10-26T05:59:02.070 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- 172.21.15.71:0/50365190 wait complete. 2023-10-26T05:59:02.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 Processor -- start 2023-10-26T05:59:02.072 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- start start 2023-10-26T05:59:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f4069d30 0x7fd0f4071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40ca7f0 0x7fd0f4157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 0x7fd0f4163010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4071640 con 0x7fd0f40cabd0 2023-10-26T05:59:02.073 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f40aa8e0 con 0x7fd0f40ca7f0 2023-10-26T05:59:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.059+0000 7fd0fc780640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f4158110 con 0x7fd0f4069d30 2023-10-26T05:59:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 0x7fd0f4163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 0x7fd0f4163010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41926/0 (socket says 172.21.15.71:41926) 2023-10-26T05:59:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 -- 172.21.15.71:0/131664039 learned_addr learned my addr 172.21.15.71:0/131664039 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:02.074 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fa51c640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f4069d30 0x7fd0f4071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0f9d1b640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40ca7f0 0x7fd0f4157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 -- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f4069d30 msgr2=0x7fd0f4071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f4069d30 0x7fd0f4071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 -- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40ca7f0 msgr2=0x7fd0f4157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.075 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40ca7f0 0x7fd0f4157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 -- 172.21.15.71:0/131664039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ec0098e0 con 0x7fd0f40cabd0 2023-10-26T05:59:02.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0f9d1b640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40ca7f0 0x7fd0f4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:02.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fad1d640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 0x7fd0f4163010 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009bd0 tx=0x7fd0ec0a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:02.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec004e60 con 0x7fd0f40cabd0 2023-10-26T05:59:02.076 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0ec004000 con 0x7fd0f40cabd0 2023-10-26T05:59:02.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f4158360 con 0x7fd0f40cabd0 2023-10-26T05:59:02.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0043f0 con 0x7fd0f40cabd0 2023-10-26T05:59:02.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f4158920 con 0x7fd0f40cabd0 2023-10-26T05:59:02.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f4066b10 con 0x7fd0f40cabd0 2023-10-26T05:59:02.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0ec0b4040 con 0x7fd0f40cabd0 2023-10-26T05:59:02.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0eaffd640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0d8066470 0x7fd0d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec11e2c0 con 0x7fd0f40cabd0 2023-10-26T05:59:02.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fa51c640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0d8066470 0x7fd0d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.063+0000 7fd0fa51c640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0d8066470 0x7fd0d8068930 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006fd0 tx=0x7fd0f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:02.079 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.067+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0ec0eb740 con 0x7fd0f40cabd0 2023-10-26T05:59:02.243 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.235+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd0f40aaa70 con 0x7fd0f40cabd0 2023-10-26T05:59:02.244 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.235+0000 7fd0eaffd640 1 -- 172.21.15.71:0/131664039 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v191) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd0ec0bc080 con 0x7fd0f40cabd0 2023-10-26T05:59:02.244 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:59:02.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.235+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0d8066470 msgr2=0x7fd0d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.235+0000 7fd0fc780640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0d8066470 0x7fd0d8068930 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006fd0 tx=0x7fd0f0008040 comp rx=0 tx=0).stop 2023-10-26T05:59:02.246 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.235+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 msgr2=0x7fd0f4163010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.235+0000 7fd0fc780640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 0x7fd0f4163010 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009bd0 tx=0x7fd0ec0a5a20 comp rx=0 tx=0).stop 2023-10-26T05:59:02.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 shutdown_connections 2023-10-26T05:59:02.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0d8066470 0x7fd0d8068930 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0f40cabd0 0x7fd0f4163010 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.247 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0f40ca7f0 0x7fd0f4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 --2- 172.21.15.71:0/131664039 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0f4069d30 0x7fd0f4071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 >> 172.21.15.71:0/131664039 conn(0x7fd0f406b8c0 msgr2=0x7fd0f41413c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:02.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 shutdown_connections 2023-10-26T05:59:02.249 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.239+0000 7fd0fc780640 1 -- 172.21.15.71:0/131664039 wait complete. 2023-10-26T05:59:02.262 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2023-10-26T05:59:02.263 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:59:02.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-26T05:59:02.313 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:59:02.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:ceph-fuse[54812]: fuse finished with error 0 and tester_r 0 2023-10-26T05:59:02.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:59:02.326 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:02.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:02.331 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-26T05:59:02.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi071.stderr:2023-10-26T05:59:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:59:02.357 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:59:02.358 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:02.370 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:59:02.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 -- 172.21.15.71:0/1500767913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28140cc0 msgr2=0x7fac2815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/1500767913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28140cc0 0x7fac2815c760 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fac18009840 tx=0x7fac1801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 -- 172.21.15.71:0/1500767913 shutdown_connections 2023-10-26T05:59:02.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/1500767913 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac280b2370 0x7fac28071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/1500767913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac28140cc0 0x7fac2815c760 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 -- 172.21.15.71:0/1500767913 >> 172.21.15.71:0/1500767913 conn(0x7fac28069770 msgr2=0x7fac28069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:02.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 -- 172.21.15.71:0/1500767913 shutdown_connections 2023-10-26T05:59:02.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.503+0000 7fac2f6b1640 1 -- 172.21.15.71:0/1500767913 wait complete. 2023-10-26T05:59:02.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 Processor -- start 2023-10-26T05:59:02.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- start start 2023-10-26T05:59:02.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac28070fe0 0x7fac2815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b2370 0x7fac28138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 0x7fac28136f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2813f3f0 con 0x7fac28140cc0 2023-10-26T05:59:02.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2813f690 con 0x7fac280b2370 2023-10-26T05:59:02.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2813f970 con 0x7fac28070fe0 2023-10-26T05:59:02.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 0x7fac28136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 0x7fac28136f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41942/0 (socket says 172.21.15.71:41942) 2023-10-26T05:59:02.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 -- 172.21.15.71:0/4050754120 learned_addr learned my addr 172.21.15.71:0/4050754120 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:02.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2d44d640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac28070fe0 0x7fac2815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2cc4c640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b2370 0x7fac28138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 -- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac28070fe0 msgr2=0x7fac2815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac28070fe0 0x7fac2815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 -- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b2370 msgr2=0x7fac28138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b2370 0x7fac28138a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 -- 172.21.15.71:0/4050754120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac20009820 con 0x7fac28140cc0 2023-10-26T05:59:02.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2dc4e640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 0x7fac28136f80 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fac2000efd0 tx=0x7fac2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:02.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20009b80 con 0x7fac28140cc0 2023-10-26T05:59:02.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fac20003b80 con 0x7fac28140cc0 2023-10-26T05:59:02.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac180094a0 con 0x7fac28140cc0 2023-10-26T05:59:02.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20004020 con 0x7fac28140cc0 2023-10-26T05:59:02.522 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2813d190 con 0x7fac28140cc0 2023-10-26T05:59:02.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.507+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28066b10 con 0x7fac28140cc0 2023-10-26T05:59:02.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.511+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac20004180 con 0x7fac28140cc0 2023-10-26T05:59:02.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.511+0000 7fac1dffb640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac0c066470 0x7fac0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.511+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac20014070 con 0x7fac28140cc0 2023-10-26T05:59:02.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.511+0000 7fac2d44d640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac0c066470 0x7fac0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.511+0000 7fac2d44d640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac0c066470 0x7fac0c068930 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fac180af000 tx=0x7fac1801b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:02.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.515+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fac20052250 con 0x7fac28140cc0 2023-10-26T05:59:02.665 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.655+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac2806d9f0 con 0x7fac28140cc0 2023-10-26T05:59:02.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.659+0000 7fac1dffb640 1 -- 172.21.15.71:0/4050754120 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16495 (secure 0 0 0) 0x7fac20055870 con 0x7fac28140cc0 2023-10-26T05:59:02.668 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:02.669 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":191,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:57.314223+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:59:02.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac0c066470 msgr2=0x7fac0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac0c066470 0x7fac0c068930 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fac180af000 tx=0x7fac1801b010 comp rx=0 tx=0).stop 2023-10-26T05:59:02.670 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 msgr2=0x7fac28136f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 0x7fac28136f80 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fac2000efd0 tx=0x7fac2000c6a0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 shutdown_connections 2023-10-26T05:59:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fac0c066470 0x7fac0c068930 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac28140cc0 0x7fac28136f80 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac280b2370 0x7fac28138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 --2- 172.21.15.71:0/4050754120 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac28070fe0 0x7fac2815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 >> 172.21.15.71:0/4050754120 conn(0x7fac28069770 msgr2=0x7fac2815c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:02.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 shutdown_connections 2023-10-26T05:59:02.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.663+0000 7fac2f6b1640 1 -- 172.21.15.71:0/4050754120 wait complete. 2023-10-26T05:59:02.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:58:20.268236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:58:20.996636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:59:02.686 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:59:02.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 -- 172.21.15.71:0/900356360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 msgr2=0x7f9224004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 --2- 172.21.15.71:0/900356360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f9224004c50 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f923c06fc50 tx=0x7f923001f700 comp rx=0 tx=0).stop 2023-10-26T05:59:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f923bfff640 1 -- 172.21.15.71:0/900356360 reap_dead start 2023-10-26T05:59:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 -- 172.21.15.71:0/900356360 shutdown_connections 2023-10-26T05:59:02.817 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 -- 172.21.15.71:0/900356360 >> 172.21.15.71:0/900356360 conn(0x7f923c0aec60 msgr2=0x7f923c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:02.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 -- 172.21.15.71:0/900356360 shutdown_connections 2023-10-26T05:59:02.818 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 -- 172.21.15.71:0/900356360 wait complete. 2023-10-26T05:59:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 Processor -- start 2023-10-26T05:59:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.807+0000 7f9241b7f640 1 -- start start 2023-10-26T05:59:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f923c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f923c140fb0 0x7f923c141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c13ec80 0x7f923c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c13f640 con 0x7f92240047d0 2023-10-26T05:59:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0ab2c0 con 0x7f923c13ec80 2023-10-26T05:59:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0ac110 con 0x7f923c140fb0 2023-10-26T05:59:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f923c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.821 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f923c157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41958/0 (socket says 172.21.15.71:41958) 2023-10-26T05:59:02.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 -- 172.21.15.71:0/2049806201 learned_addr learned my addr 172.21.15.71:0/2049806201 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:02.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 -- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f923c140fb0 msgr2=0x7f923c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f923c140fb0 0x7f923c141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.822 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 -- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c13ec80 msgr2=0x7f923c13f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c13ec80 0x7f923c13f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 -- 172.21.15.71:0/2049806201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0ab500 con 0x7f92240047d0 2023-10-26T05:59:02.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923b7fe640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f923c157940 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f922c010a70 tx=0x7f922c010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:02.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c00ed80 con 0x7f92240047d0 2023-10-26T05:59:02.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f922c004030 con 0x7f92240047d0 2023-10-26T05:59:02.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c00eee0 con 0x7f92240047d0 2023-10-26T05:59:02.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c0ab7e0 con 0x7f92240047d0 2023-10-26T05:59:02.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c0abda0 con 0x7f92240047d0 2023-10-26T05:59:02.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c066b10 con 0x7f92240047d0 2023-10-26T05:59:02.825 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f922c003600 con 0x7f92240047d0 2023-10-26T05:59:02.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9223fff640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92280664a0 0x7f9228068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:02.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f922c016070 con 0x7f92240047d0 2023-10-26T05:59:02.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.811+0000 7f923affd640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92280664a0 0x7f9228068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:02.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.815+0000 7f923affd640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92280664a0 0x7f9228068960 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f92300ae000 tx=0x7f9224005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:02.826 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.815+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f922c052190 con 0x7f92240047d0 2023-10-26T05:59:02.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.959+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f923c0721e0 con 0x7f92240047d0 2023-10-26T05:59:02.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.959+0000 7f9223fff640 1 -- 172.21.15.71:0/2049806201 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f922c0557b0 con 0x7f92240047d0 2023-10-26T05:59:02.968 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:59:02.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.959+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92280664a0 msgr2=0x7f9228068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.959+0000 7f9241b7f640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92280664a0 0x7f9228068960 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f92300ae000 tx=0x7f9224005800 comp rx=0 tx=0).stop 2023-10-26T05:59:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 msgr2=0x7f923c157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f923c157940 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f922c010a70 tx=0x7f922c010f30 comp rx=0 tx=0).stop 2023-10-26T05:59:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 shutdown_connections 2023-10-26T05:59:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f923c13ec80 0x7f923c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f923c140fb0 0x7f923c141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f92280664a0 0x7f9228068960 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 --2- 172.21.15.71:0/2049806201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92240047d0 0x7f923c157940 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 >> 172.21.15.71:0/2049806201 conn(0x7f923c0aec60 msgr2=0x7f923c13d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 shutdown_connections 2023-10-26T05:59:02.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:02.963+0000 7f9241b7f640 1 -- 172.21.15.71:0/2049806201 wait complete. 2023-10-26T05:59:02.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:59:03.109 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 -- 172.21.15.71:0/236946895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 msgr2=0x7f37700cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 --2- 172.21.15.71:0/236946895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f37700cbbe0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f376c009c80 tx=0x7f376c01cd40 comp rx=0 tx=0).stop 2023-10-26T05:59:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 -- 172.21.15.71:0/236946895 shutdown_connections 2023-10-26T05:59:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 --2- 172.21.15.71:0/236946895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f377015c8c0 0x7f37700c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 --2- 172.21.15.71:0/236946895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f37700cbbe0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.110 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 -- 172.21.15.71:0/236946895 >> 172.21.15.71:0/236946895 conn(0x7f37700caa90 msgr2=0x7f37700caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:03.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 -- 172.21.15.71:0/236946895 shutdown_connections 2023-10-26T05:59:03.111 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.099+0000 7f3777924640 1 -- 172.21.15.71:0/236946895 wait complete. 2023-10-26T05:59:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 Processor -- start 2023-10-26T05:59:03.113 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- start start 2023-10-26T05:59:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f3770157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f377015bcb0 0x7f37701555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377015c8c0 0x7f3770155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770131fc0 con 0x7f3770152720 2023-10-26T05:59:03.114 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37701310f0 con 0x7f377015c8c0 2023-10-26T05:59:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3770131340 con 0x7f377015bcb0 2023-10-26T05:59:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f3770157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f3770157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41976/0 (socket says 172.21.15.71:41976) 2023-10-26T05:59:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 -- 172.21.15.71:0/2542582819 learned_addr learned my addr 172.21.15.71:0/2542582819 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:03.115 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3775ec1640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377015c8c0 0x7f3770155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 -- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f377015bcb0 msgr2=0x7f37701555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f377015bcb0 0x7f37701555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 -- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377015c8c0 msgr2=0x7f3770155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377015c8c0 0x7f3770155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 -- 172.21.15.71:0/2542582819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0098e0 con 0x7f3770152720 2023-10-26T05:59:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3775ec1640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377015c8c0 0x7f3770155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f37756c0640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f3770157a60 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f376c000f90 tx=0x7f376c003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c0ba070 con 0x7f3770152720 2023-10-26T05:59:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f376c0ad9a0 con 0x7f3770152720 2023-10-26T05:59:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3770131620 con 0x7f3770152720 2023-10-26T05:59:03.117 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c0add70 con 0x7f3770152720 2023-10-26T05:59:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3770131be0 con 0x7f3770152720 2023-10-26T05:59:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3770066d00 con 0x7f3770152720 2023-10-26T05:59:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f376c0b4040 con 0x7f3770152720 2023-10-26T05:59:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3765ffb640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3754066470 0x7f3754068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.118 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f376c11e680 con 0x7f3770152720 2023-10-26T05:59:03.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.103+0000 7f3774ebf640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3754066470 0x7f3754068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.119 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.107+0000 7f3774ebf640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3754066470 0x7f3754068930 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f37700c8170 tx=0x7f3760006c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:03.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.107+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f376c0ebb00 con 0x7f3770152720 2023-10-26T05:59:03.258 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37700cd740 con 0x7f3770152720 2023-10-26T05:59:03.259 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3765ffb640 1 -- 172.21.15.71:0/2542582819 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v191) v1 ==== 104+0+11 (secure 0 0 0) 0x7f376c0ef120 con 0x7f3770152720 2023-10-26T05:59:03.259 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:59:03.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3754066470 msgr2=0x7f3754068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3754066470 0x7f3754068930 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f37700c8170 tx=0x7f3760006c00 comp rx=0 tx=0).stop 2023-10-26T05:59:03.262 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 msgr2=0x7f3770157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f3770157a60 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f376c000f90 tx=0x7f376c003c40 comp rx=0 tx=0).stop 2023-10-26T05:59:03.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 shutdown_connections 2023-10-26T05:59:03.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.251+0000 7f3777924640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f377015c8c0 0x7f3770155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.255+0000 7f3777924640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f377015bcb0 0x7f37701555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.263 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.255+0000 7f3777924640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3754066470 0x7f3754068930 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.255+0000 7f3777924640 1 --2- 172.21.15.71:0/2542582819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3770152720 0x7f3770157a60 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.255+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 >> 172.21.15.71:0/2542582819 conn(0x7f37700caa90 msgr2=0x7f37700adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:03.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.255+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 shutdown_connections 2023-10-26T05:59:03.264 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.255+0000 7f3777924640 1 -- 172.21.15.71:0/2542582819 wait complete. 2023-10-26T05:59:03.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:59:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2733363188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 msgr2=0x7f45300047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.403 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 --2- 172.21.15.71:0/2733363188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45300047d0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c290 tx=0x7f453c01f780 comp rx=0 tx=0).stop 2023-10-26T05:59:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f4546d9d640 1 -- 172.21.15.71:0/2733363188 reap_dead start 2023-10-26T05:59:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2733363188 shutdown_connections 2023-10-26T05:59:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2733363188 >> 172.21.15.71:0/2733363188 conn(0x7f45480aec60 msgr2=0x7f45480af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:03.404 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2733363188 shutdown_connections 2023-10-26T05:59:03.405 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2733363188 wait complete. 2023-10-26T05:59:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 Processor -- start 2023-10-26T05:59:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- start start 2023-10-26T05:59:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45481f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.408 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45480b1c60 0x7f45481351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45480b20e0 0x7f45481356e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481e29d0 con 0x7f4530003fb0 2023-10-26T05:59:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45481f0880 con 0x7f45480b20e0 2023-10-26T05:59:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4548135c20 con 0x7f45480b1c60 2023-10-26T05:59:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45481f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45481f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41988/0 (socket says 172.21.15.71:41988) 2023-10-26T05:59:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 -- 172.21.15.71:0/2170468923 learned_addr learned my addr 172.21.15.71:0/2170468923 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f4545d9b640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45480b1c60 0x7f45481351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 -- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45480b1c60 msgr2=0x7f45481351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f4546d9d640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45480b20e0 0x7f45481356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45480b1c60 0x7f45481351a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 -- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45480b20e0 msgr2=0x7f45481356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45480b20e0 0x7f45481356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 -- 172.21.15.71:0/2170468923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45481e2430 con 0x7f4530003fb0 2023-10-26T05:59:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454659c640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45481f01f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f453800d980 tx=0x7f453800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538009c30 con 0x7f4530003fb0 2023-10-26T05:59:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4538009e80 con 0x7f4530003fb0 2023-10-26T05:59:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4548142a30 con 0x7f4530003fb0 2023-10-26T05:59:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538004290 con 0x7f4530003fb0 2023-10-26T05:59:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.395+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45481e1af0 con 0x7f4530003fb0 2023-10-26T05:59:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.399+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4548066b10 con 0x7f4530003fb0 2023-10-26T05:59:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.399+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4538003600 con 0x7f4530003fb0 2023-10-26T05:59:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.399+0000 7f4536ffd640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4528066470 0x7f4528068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.399+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4538084230 con 0x7f4530003fb0 2023-10-26T05:59:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.399+0000 7f4545d9b640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4528066470 0x7f4528068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.415 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.399+0000 7f4545d9b640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4528066470 0x7f4528068930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f453c0b0000 tx=0x7f453c01c520 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:03.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.403+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f45380516b0 con 0x7f4530003fb0 2023-10-26T05:59:03.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45480721e0 con 0x7f4530003fb0 2023-10-26T05:59:03.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f4536ffd640 1 -- 172.21.15.71:0/2170468923 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v191) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4538054cd0 con 0x7f4530003fb0 2023-10-26T05:59:03.555 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:59:03.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4528066470 msgr2=0x7f4528068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.557 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4528066470 0x7f4528068930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f453c0b0000 tx=0x7f453c01c520 comp rx=0 tx=0).stop 2023-10-26T05:59:03.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 msgr2=0x7f45481f01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45481f01f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f453800d980 tx=0x7f453800de40 comp rx=0 tx=0).stop 2023-10-26T05:59:03.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 shutdown_connections 2023-10-26T05:59:03.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45480b20e0 0x7f45481356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f45480b1c60 0x7f45481351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4528066470 0x7f4528068930 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 --2- 172.21.15.71:0/2170468923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4530003fb0 0x7f45481f01f0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 >> 172.21.15.71:0/2170468923 conn(0x7f45480aec60 msgr2=0x7f4548137c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 shutdown_connections 2023-10-26T05:59:03.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.547+0000 7f454cc25640 1 -- 172.21.15.71:0/2170468923 wait complete. 2023-10-26T05:59:03.573 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-26T05:59:03.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- 172.21.15.71:0/1056636324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62200cd140 msgr2=0x7f6220141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- 172.21.15.71:0/1056636324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62200cd140 0x7f6220141190 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f622006fb90 tx=0x7f621c01c7e0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- 172.21.15.71:0/1056636324 shutdown_connections 2023-10-26T05:59:03.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- 172.21.15.71:0/1056636324 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6220155e30 0x7f62200ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- 172.21.15.71:0/1056636324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62200cd140 0x7f6220141190 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- 172.21.15.71:0/1056636324 >> v2:172.21.15.86:3300/0 conn(0x7f6220155a50 0x7f62201590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- 172.21.15.71:0/1056636324 >> 172.21.15.71:0/1056636324 conn(0x7f6220070650 msgr2=0x7f6220070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:03.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- 172.21.15.71:0/1056636324 shutdown_connections 2023-10-26T05:59:03.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- 172.21.15.71:0/1056636324 wait complete. 2023-10-26T05:59:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 Processor -- start 2023-10-26T05:59:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- start start 2023-10-26T05:59:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62200cd140 0x7f62200b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 0x7f622013b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6220155e30 0x7f622013b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6220157d10 con 0x7f6220155a50 2023-10-26T05:59:03.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62201402c0 con 0x7f62200cd140 2023-10-26T05:59:03.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f62287fb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62201405a0 con 0x7f6220155e30 2023-10-26T05:59:03.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f6225d96640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 0x7f622013b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f6225d96640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 0x7f622013b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42004/0 (socket says 172.21.15.71:42004) 2023-10-26T05:59:03.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.691+0000 7f6225d96640 1 -- 172.21.15.71:0/608575829 learned_addr learned my addr 172.21.15.71:0/608575829 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:03.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6226d98640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6220155e30 0x7f622013b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6226597640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62200cd140 0x7f62200b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6225d96640 1 -- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6220155e30 msgr2=0x7f622013b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6225d96640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6220155e30 0x7f622013b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6225d96640 1 -- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62200cd140 msgr2=0x7f62200b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6225d96640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62200cd140 0x7f62200b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6225d96640 1 -- 172.21.15.71:0/608575829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009560 con 0x7f6220155a50 2023-10-26T05:59:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6225d96640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 0x7f622013b360 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f621000c9f0 tx=0x7f621000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210007bb0 con 0x7f6220155a50 2023-10-26T05:59:03.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6210007e00 con 0x7f6220155a50 2023-10-26T05:59:03.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6220140880 con 0x7f6220155a50 2023-10-26T05:59:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000f6b0 con 0x7f6220155a50 2023-10-26T05:59:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622013bde0 con 0x7f6220155a50 2023-10-26T05:59:03.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6220069770 con 0x7f6220155a50 2023-10-26T05:59:03.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f621000f810 con 0x7f6220155a50 2023-10-26T05:59:03.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6216ffd640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6204066470 0x7f6204068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:03.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6210084730 con 0x7f6220155a50 2023-10-26T05:59:03.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6226597640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6204066470 0x7f6204068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:03.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.695+0000 7f6226597640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6204066470 0x7f6204068930 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6218009dd0 tx=0x7f62180093c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:03.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.699+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6210051bb0 con 0x7f6220155a50 2023-10-26T05:59:03.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.867+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62200adaa0 con 0x7f6220155a50 2023-10-26T05:59:03.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f6216ffd640 1 -- 172.21.15.71:0/608575829 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v191) v1 ==== 80+0+0 (secure 0 0 0) 0x7f62100551d0 con 0x7f6220155a50 2023-10-26T05:59:03.878 INFO:teuthology.orchestra.run.smithi071.stderr:listed 0 entries 2023-10-26T05:59:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6204066470 msgr2=0x7f6204068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6204066470 0x7f6204068930 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6218009dd0 tx=0x7f62180093c0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 msgr2=0x7f622013b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 0x7f622013b360 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f621000c9f0 tx=0x7f621000ceb0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 shutdown_connections 2023-10-26T05:59:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6220155e30 0x7f622013b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6204066470 0x7f6204068930 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6220155a50 0x7f622013b360 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 --2- 172.21.15.71:0/608575829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62200cd140 0x7f62200b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 >> 172.21.15.71:0/608575829 conn(0x7f6220070650 msgr2=0x7f6220070a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 shutdown_connections 2023-10-26T05:59:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:03.871+0000 7f62287fb640 1 -- 172.21.15.71:0/608575829 wait complete. 2023-10-26T05:59:03.897 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-26T05:59:03.897 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-10-26T05:59:03.897 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-26T05:59:03.951 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:59:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-10-26T05:59:03.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[35525]: fuse finished with error 0 and tester_r 0 2023-10-26T05:59:03.952 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-26T05:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:59:03.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-26T05:59:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-26T05:59:03.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:03.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:03.961 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-26T05:59:03.969 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:59:03.969 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:03.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:59:03.976 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:03.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:03.976 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-26T05:59:04.022 INFO:teuthology.orchestra.run.smithi071.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-26T05:59:04.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:59:04.024 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:59:04.024 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:04.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:59:04.030 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:04.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:04.031 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-26T05:59:04.083 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-26T05:59:04.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:59:04.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:59:04.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3066830897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 msgr2=0x7f9e6c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.219 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3066830897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e6c003dd0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000bb10 tx=0x7f9e7001cd80 comp rx=0 tx=0).stop 2023-10-26T05:59:04.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e864af640 1 -- 172.21.15.71:0/3066830897 reap_dead start 2023-10-26T05:59:04.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3066830897 shutdown_connections 2023-10-26T05:59:04.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3066830897 >> 172.21.15.71:0/3066830897 conn(0x7f9e800d12e0 msgr2=0x7f9e800d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:04.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3066830897 shutdown_connections 2023-10-26T05:59:04.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3066830897 wait complete. 2023-10-26T05:59:04.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 Processor -- start 2023-10-26T05:59:04.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- start start 2023-10-26T05:59:04.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e80162c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8014b920 0x7f9e80165110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e800ca720 0x7f9e800caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800cb0e0 con 0x7f9e6c0039b0 2023-10-26T05:59:04.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80163e80 con 0x7f9e8014b920 2023-10-26T05:59:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80163270 con 0x7f9e800ca720 2023-10-26T05:59:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e80162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e80162c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42026/0 (socket says 172.21.15.71:42026) 2023-10-26T05:59:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 -- 172.21.15.71:0/3198278622 learned_addr learned my addr 172.21.15.71:0/3198278622 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e854ad640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8014b920 0x7f9e80165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e864af640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e800ca720 0x7f9e800caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 -- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e800ca720 msgr2=0x7f9e800caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e800ca720 0x7f9e800caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 -- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8014b920 msgr2=0x7f9e80165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8014b920 0x7f9e80165110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 -- 172.21.15.71:0/3198278622 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80163550 con 0x7f9e6c0039b0 2023-10-26T05:59:04.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e854ad640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8014b920 0x7f9e80165110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:04.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e85cae640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e80162c90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001cbb0 tx=0x7f9e700a5f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:04.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70004220 con 0x7f9e6c0039b0 2023-10-26T05:59:04.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f9e70004470 con 0x7f9e6c0039b0 2023-10-26T05:59:04.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e80163830 con 0x7f9e6c0039b0 2023-10-26T05:59:04.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700053e0 con 0x7f9e6c0039b0 2023-10-26T05:59:04.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.211+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8014bd80 con 0x7f9e6c0039b0 2023-10-26T05:59:04.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.215+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e80066b10 con 0x7f9e6c0039b0 2023-10-26T05:59:04.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.215+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9e700046d0 con 0x7f9e6c0039b0 2023-10-26T05:59:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.215+0000 7f9e767fc640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e64066470 0x7f9e64068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.215+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e7011e750 con 0x7f9e6c0039b0 2023-10-26T05:59:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.215+0000 7f9e854ad640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e64066470 0x7f9e64068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.215+0000 7f9e854ad640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e64066470 0x7f9e64068930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c000c10 tx=0x7f9e7c0014d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:04.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.219+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f9e700ebbd0 con 0x7f9e6c0039b0 2023-10-26T05:59:04.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.403+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e800721e0 con 0x7f9e6c0039b0 2023-10-26T05:59:04.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e767fc640 1 -- 172.21.15.71:0/3198278622 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f9e700ab090 con 0x7f9e6c0039b0 2023-10-26T05:59:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:04.415 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:59:04.415 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:59:04.416 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e64066470 msgr2=0x7f9e64068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e64066470 0x7f9e64068930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c000c10 tx=0x7f9e7c0014d0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 msgr2=0x7f9e80162c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e80162c90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001cbb0 tx=0x7f9e700a5f00 comp rx=0 tx=0).stop 2023-10-26T05:59:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 shutdown_connections 2023-10-26T05:59:04.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9e800ca720 0x7f9e800caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8014b920 0x7f9e80165110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9e64066470 0x7f9e64068930 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 --2- 172.21.15.71:0/3198278622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e6c0039b0 0x7f9e80162c90 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 >> 172.21.15.71:0/3198278622 conn(0x7f9e800d12e0 msgr2=0x7f9e800c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 shutdown_connections 2023-10-26T05:59:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.407+0000 7f9e87f12640 1 -- 172.21.15.71:0/3198278622 wait complete. 2023-10-26T05:59:04.432 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:59:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- 172.21.15.71:0/3331270529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c800039b0 msgr2=0x7f6c80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 --2- 172.21.15.71:0/3331270529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c800039b0 0x7f6c80003dd0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00bb10 tx=0x7f6c8c01cd50 comp rx=0 tx=0).stop 2023-10-26T05:59:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 -- 172.21.15.71:0/3331270529 reap_dead start 2023-10-26T05:59:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- 172.21.15.71:0/3331270529 shutdown_connections 2023-10-26T05:59:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- 172.21.15.71:0/3331270529 >> 172.21.15.71:0/3331270529 conn(0x7f6c94069770 msgr2=0x7f6c94069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- 172.21.15.71:0/3331270529 shutdown_connections 2023-10-26T05:59:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- 172.21.15.71:0/3331270529 wait complete. 2023-10-26T05:59:04.558 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 Processor -- start 2023-10-26T05:59:04.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- start start 2023-10-26T05:59:04.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c800039b0 0x7f6c94161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c9413a6c0 0x7f6c94157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.559 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 0x7f6c940afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9415ff90 con 0x7f6c941603b0 2023-10-26T05:59:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9415f0c0 con 0x7f6c9413a6c0 2023-10-26T05:59:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c9b494640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9415f310 con 0x7f6c800039b0 2023-10-26T05:59:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 0x7f6c940afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.560 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 0x7f6c940afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42042/0 (socket says 172.21.15.71:42042) 2023-10-26T05:59:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 -- 172.21.15.71:0/2398841919 learned_addr learned my addr 172.21.15.71:0/2398841919 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99230640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c800039b0 0x7f6c94161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c98a2f640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c9413a6c0 0x7f6c94157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 -- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c800039b0 msgr2=0x7f6c94161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.561 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c800039b0 0x7f6c94161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 -- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c9413a6c0 msgr2=0x7f6c94157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c9413a6c0 0x7f6c94157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 -- 172.21.15.71:0/2398841919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9415f5f0 con 0x7f6c941603b0 2023-10-26T05:59:04.562 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.547+0000 7f6c99a31640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 0x7f6c940afc80 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f6c94158760 tx=0x7f6c900019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:04.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90011870 con 0x7f6c941603b0 2023-10-26T05:59:04.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c90011ac0 con 0x7f6c941603b0 2023-10-26T05:59:04.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9415f8d0 con 0x7f6c941603b0 2023-10-26T05:59:04.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900103c0 con 0x7f6c941603b0 2023-10-26T05:59:04.563 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941620e0 con 0x7f6c941603b0 2023-10-26T05:59:04.564 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94066b10 con 0x7f6c941603b0 2023-10-26T05:59:04.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6c90010520 con 0x7f6c941603b0 2023-10-26T05:59:04.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c89ffb640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c780664a0 0x7f6c78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c90084830 con 0x7f6c941603b0 2023-10-26T05:59:04.565 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c99230640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c780664a0 0x7f6c78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.551+0000 7f6c99230640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c780664a0 0x7f6c78068960 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0b0000 tx=0x7f6c8c0a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:04.566 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.555+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6c90051c10 con 0x7f6c941603b0 2023-10-26T05:59:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.699+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c940721e0 con 0x7f6c941603b0 2023-10-26T05:59:04.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.699+0000 7f6c89ffb640 1 -- 172.21.15.71:0/2398841919 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16495 (secure 0 0 0) 0x7f6c90055230 con 0x7f6c941603b0 2023-10-26T05:59:04.709 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:04.709 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":191,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:57.314223+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:59:04.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c780664a0 msgr2=0x7f6c78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c780664a0 0x7f6c78068960 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0b0000 tx=0x7f6c8c0a7040 comp rx=0 tx=0).stop 2023-10-26T05:59:04.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 msgr2=0x7f6c940afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 0x7f6c940afc80 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f6c94158760 tx=0x7f6c900019c0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 shutdown_connections 2023-10-26T05:59:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c780664a0 0x7f6c78068960 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c941603b0 0x7f6c940afc80 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c9413a6c0 0x7f6c94157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 --2- 172.21.15.71:0/2398841919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c800039b0 0x7f6c94161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 >> 172.21.15.71:0/2398841919 conn(0x7f6c94069770 msgr2=0x7f6c940b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:04.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 shutdown_connections 2023-10-26T05:59:04.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.703+0000 7f6c9b494640 1 -- 172.21.15.71:0/2398841919 wait complete. 2023-10-26T05:59:04.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-26T05:47:09.468386+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-26T05:58:20.268236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-26T05:58:20.996636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-26T05:59:04.727 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-26T05:59:04.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2094479925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0d0850 msgr2=0x7f4b7c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.843 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 --2- 172.21.15.71:0/2094479925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0d0850 0x7f4b7c15b3a0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c08a140 tx=0x7f4b7001c720 comp rx=0 tx=0).stop 2023-10-26T05:59:04.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2094479925 shutdown_connections 2023-10-26T05:59:04.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 --2- 172.21.15.71:0/2094479925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0d0850 0x7f4b7c15b3a0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 --2- 172.21.15.71:0/2094479925 >> v2:172.21.15.86:3300/0 conn(0x7f4b7c0d0470 0x7f4b7c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2094479925 >> 172.21.15.71:0/2094479925 conn(0x7f4b7c0cdd80 msgr2=0x7f4b7c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:04.844 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2094479925 shutdown_connections 2023-10-26T05:59:04.845 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2094479925 wait complete. 2023-10-26T05:59:04.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 Processor -- start 2023-10-26T05:59:04.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- start start 2023-10-26T05:59:04.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 0x7f4b7c15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b7c0d0470 0x7f4b7c0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b7c0d0850 0x7f4b7c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c69c0 con 0x7f4b7c0cd880 2023-10-26T05:59:04.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c5e40 con 0x7f4b7c0d0470 2023-10-26T05:59:04.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c0c6120 con 0x7f4b7c0d0850 2023-10-26T05:59:04.849 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 0x7f4b7c15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 0x7f4b7c15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42052/0 (socket says 172.21.15.71:42052) 2023-10-26T05:59:04.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 -- 172.21.15.71:0/2109214460 learned_addr learned my addr 172.21.15.71:0/2109214460 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:04.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 -- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b7c0d0850 msgr2=0x7f4b7c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b7c0d0850 0x7f4b7c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 -- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b7c0d0470 msgr2=0x7f4b7c0c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:04.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b7c0d0470 0x7f4b7c0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:04.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 -- 172.21.15.71:0/2109214460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b700094a0 con 0x7f4b7c0cd880 2023-10-26T05:59:04.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b812c5640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 0x7f4b7c15a870 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d950 tx=0x7f4b7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:04.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78009c30 con 0x7f4b7c0cd880 2023-10-26T05:59:04.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b78009e80 con 0x7f4b7c0cd880 2023-10-26T05:59:04.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c0c6400 con 0x7f4b7c0cd880 2023-10-26T05:59:04.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780106c0 con 0x7f4b7c0cd880 2023-10-26T05:59:04.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.835+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c15adb0 con 0x7f4b7c0cd880 2023-10-26T05:59:04.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.839+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c066b10 con 0x7f4b7c0cd880 2023-10-26T05:59:04.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.839+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b78010820 con 0x7f4b7c0cd880 2023-10-26T05:59:04.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.839+0000 7f4b6dffb640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b600664a0 0x7f4b60068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:04.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.839+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b780843a0 con 0x7f4b7c0cd880 2023-10-26T05:59:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.839+0000 7f4b80ac4640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b600664a0 0x7f4b60068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.839+0000 7f4b80ac4640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b600664a0 0x7f4b60068960 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4b700af000 tx=0x7f4b700194c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.843+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4b78051780 con 0x7f4b7c0cd880 2023-10-26T05:59:04.999 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.991+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b7c0721e0 con 0x7f4b7c0cd880 2023-10-26T05:59:05.000 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.991+0000 7f4b6dffb640 1 -- 172.21.15.71:0/2109214460 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b78054da0 con 0x7f4b7c0cd880 2023-10-26T05:59:05.000 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2023-10-26T05:59:05.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b600664a0 msgr2=0x7f4b60068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b600664a0 0x7f4b60068960 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4b700af000 tx=0x7f4b700194c0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 msgr2=0x7f4b7c15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 0x7f4b7c15a870 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d950 tx=0x7f4b7800de10 comp rx=0 tx=0).stop 2023-10-26T05:59:05.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 shutdown_connections 2023-10-26T05:59:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b7c0d0850 0x7f4b7c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b7c0d0470 0x7f4b7c0c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b600664a0 0x7f4b60068960 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 --2- 172.21.15.71:0/2109214460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b7c0cd880 0x7f4b7c15a870 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.004 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 >> 172.21.15.71:0/2109214460 conn(0x7f4b7c0cdd80 msgr2=0x7f4b7c0b2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 shutdown_connections 2023-10-26T05:59:05.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:04.995+0000 7f4b83529640 1 -- 172.21.15.71:0/2109214460 wait complete. 2023-10-26T05:59:05.018 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-26T05:59:05.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- 172.21.15.71:0/3140981563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d40003980 msgr2=0x7f4d40003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 --2- 172.21.15.71:0/3140981563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d40003980 0x7f4d40003da0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4d48001da0 tx=0x7f4d4801cad0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- 172.21.15.71:0/3140981563 shutdown_connections 2023-10-26T05:59:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 --2- 172.21.15.71:0/3140981563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d40003980 0x7f4d40003da0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- 172.21.15.71:0/3140981563 >> 172.21.15.71:0/3140981563 conn(0x7f4d580aec60 msgr2=0x7f4d580af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- 172.21.15.71:0/3140981563 shutdown_connections 2023-10-26T05:59:05.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- 172.21.15.71:0/3140981563 wait complete. 2023-10-26T05:59:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 Processor -- start 2023-10-26T05:59:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- start start 2023-10-26T05:59:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d40003980 0x7f4d581f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5813f9b0 0x7f4d580ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 0x7f4d581585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581396b0 con 0x7f4d58158150 2023-10-26T05:59:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58158b20 con 0x7f4d5813f9b0 2023-10-26T05:59:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.123+0000 7f4d5f513640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58070220 con 0x7f4d40003980 2023-10-26T05:59:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 0x7f4d581585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 0x7f4d581585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42070/0 (socket says 172.21.15.71:42070) 2023-10-26T05:59:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 -- 172.21.15.71:0/194473540 learned_addr learned my addr 172.21.15.71:0/194473540 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5caae640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5813f9b0 0x7f4d580ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 -- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d40003980 msgr2=0x7f4d581f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d40003980 0x7f4d581f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 -- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5813f9b0 msgr2=0x7f4d580ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5813f9b0 0x7f4d580ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 -- 172.21.15.71:0/194473540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5806f6c0 con 0x7f4d58158150 2023-10-26T05:59:05.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5caae640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5813f9b0 0x7f4d580ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:05.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5dab0640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 0x7f4d581585e0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f4d580b31e0 tx=0x7f4d5000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:05.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50005480 con 0x7f4d58158150 2023-10-26T05:59:05.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4d50003710 con 0x7f4d58158150 2023-10-26T05:59:05.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5806f980 con 0x7f4d58158150 2023-10-26T05:59:05.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50003b20 con 0x7f4d58158150 2023-10-26T05:59:05.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5806ff40 con 0x7f4d58158150 2023-10-26T05:59:05.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d58066b10 con 0x7f4d58158150 2023-10-26T05:59:05.142 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4d50003c80 con 0x7f4d58158150 2023-10-26T05:59:05.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d55ffb640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d44066470 0x7f4d44068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d50083140 con 0x7f4d58158150 2023-10-26T05:59:05.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5d2af640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d44066470 0x7f4d44068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.127+0000 7f4d5d2af640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d44066470 0x7f4d44068930 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4d480a9000 tx=0x7f4d40005800 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:05.143 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.131+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4d50050650 con 0x7f4d58158150 2023-10-26T05:59:05.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.271+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d581397f0 con 0x7f4d58158150 2023-10-26T05:59:05.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.271+0000 7f4d55ffb640 1 -- 172.21.15.71:0/194473540 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v191) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d581397f0 con 0x7f4d58158150 2023-10-26T05:59:05.281 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2023-10-26T05:59:05.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d44066470 msgr2=0x7f4d44068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d44066470 0x7f4d44068930 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4d480a9000 tx=0x7f4d40005800 comp rx=0 tx=0).stop 2023-10-26T05:59:05.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 msgr2=0x7f4d581585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 0x7f4d581585e0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f4d580b31e0 tx=0x7f4d5000c8e0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 shutdown_connections 2023-10-26T05:59:05.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4d44066470 0x7f4d44068930 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d58158150 0x7f4d581585e0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d5813f9b0 0x7f4d580ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 --2- 172.21.15.71:0/194473540 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4d40003980 0x7f4d581f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 >> 172.21.15.71:0/194473540 conn(0x7f4d580aec60 msgr2=0x7f4d5813de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 shutdown_connections 2023-10-26T05:59:05.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.275+0000 7f4d5f513640 1 -- 172.21.15.71:0/194473540 wait complete. 2023-10-26T05:59:05.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-26T05:59:05.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1578029535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89c0047d0 msgr2=0x7fc89c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1578029535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89c0047d0 0x7fc89c004c50 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800c730 tx=0x7fc8a801c6c0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1578029535 shutdown_connections 2023-10-26T05:59:05.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1578029535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc89c0047d0 0x7fc89c004c50 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1578029535 >> 172.21.15.71:0/1578029535 conn(0x7fc8b40aec60 msgr2=0x7fc8b40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1578029535 shutdown_connections 2023-10-26T05:59:05.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1578029535 wait complete. 2023-10-26T05:59:05.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 Processor -- start 2023-10-26T05:59:05.426 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- start start 2023-10-26T05:59:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89c0047d0 0x7fc8b40afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4138980 0x7fc8b40b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 0x7fc8b413b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4159610 con 0x7fc8b413b480 2023-10-26T05:59:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b4159bd0 con 0x7fc8b4138980 2023-10-26T05:59:05.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b413c220 con 0x7fc89c0047d0 2023-10-26T05:59:05.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 0x7fc8b413b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 0x7fc8b413b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42082/0 (socket says 172.21.15.71:42082) 2023-10-26T05:59:05.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 -- 172.21.15.71:0/1440747298 learned_addr learned my addr 172.21.15.71:0/1440747298 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:05.428 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b259c640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89c0047d0 0x7fc8b40afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b1d9b640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4138980 0x7fc8b40b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 -- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89c0047d0 msgr2=0x7fc8b40afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89c0047d0 0x7fc8b40afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 -- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4138980 msgr2=0x7fc8b40b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4138980 0x7fc8b40b0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 -- 172.21.15.71:0/1440747298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b413c500 con 0x7fc8b413b480 2023-10-26T05:59:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b1d9b640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4138980 0x7fc8b40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b2d9d640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 0x7fc8b413b900 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406c780 tx=0x7fc8ac00fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac018070 con 0x7fc8b413b480 2023-10-26T05:59:05.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc8ac003b90 con 0x7fc8b413b480 2023-10-26T05:59:05.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b4159860 con 0x7fc8b413b480 2023-10-26T05:59:05.431 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac004b30 con 0x7fc8b413b480 2023-10-26T05:59:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.415+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b4140050 con 0x7fc8b413b480 2023-10-26T05:59:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.419+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b413c740 con 0x7fc8b413b480 2023-10-26T05:59:05.432 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.419+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc8ac00d890 con 0x7fc8b413b480 2023-10-26T05:59:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.419+0000 7fc8a2ffd640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc894066470 0x7fc894068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.419+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8ac083e60 con 0x7fc8b413b480 2023-10-26T05:59:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.419+0000 7fc8b259c640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc894066470 0x7fc894068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.419+0000 7fc8b259c640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc894066470 0x7fc894068930 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4000fc0 tx=0x7fc8a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:05.434 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.423+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc8ac0512e0 con 0x7fc8b413b480 2023-10-26T05:59:05.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.571+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8b4066b10 con 0x7fc8b413b480 2023-10-26T05:59:05.581 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.571+0000 7fc8a2ffd640 1 -- 172.21.15.71:0/1440747298 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v191) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8ac054900 con 0x7fc8b413b480 2023-10-26T05:59:05.581 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2023-10-26T05:59:05.583 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc894066470 msgr2=0x7fc894068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc894066470 0x7fc894068930 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4000fc0 tx=0x7fc8a4009040 comp rx=0 tx=0).stop 2023-10-26T05:59:05.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 msgr2=0x7fc8b413b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 0x7fc8b413b900 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406c780 tx=0x7fc8ac00fe10 comp rx=0 tx=0).stop 2023-10-26T05:59:05.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 shutdown_connections 2023-10-26T05:59:05.584 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc894066470 0x7fc894068930 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8b413b480 0x7fc8b413b900 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8b4138980 0x7fc8b40b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 --2- 172.21.15.71:0/1440747298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc89c0047d0 0x7fc8b40afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 >> 172.21.15.71:0/1440747298 conn(0x7fc8b40aec60 msgr2=0x7fc8b413ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.585 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 shutdown_connections 2023-10-26T05:59:05.586 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.575+0000 7fc8b8bff640 1 -- 172.21.15.71:0/1440747298 wait complete. 2023-10-26T05:59:05.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:59:05.712 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 -- 172.21.15.71:0/1533443607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180140cc0 msgr2=0x7fc18015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 --2- 172.21.15.71:0/1533443607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180140cc0 0x7fc18015c760 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fc170009c80 tx=0x7fc17001cd10 comp rx=0 tx=0).stop 2023-10-26T05:59:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 -- 172.21.15.71:0/1533443607 shutdown_connections 2023-10-26T05:59:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 --2- 172.21.15.71:0/1533443607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc180070fe0 0x7fc180071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 --2- 172.21.15.71:0/1533443607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180140cc0 0x7fc18015c760 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 -- 172.21.15.71:0/1533443607 >> 172.21.15.71:0/1533443607 conn(0x7fc180069770 msgr2=0x7fc180069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 -- 172.21.15.71:0/1533443607 shutdown_connections 2023-10-26T05:59:05.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 -- 172.21.15.71:0/1533443607 wait complete. 2023-10-26T05:59:05.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 Processor -- start 2023-10-26T05:59:05.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.703+0000 7fc18704a640 1 -- start start 2023-10-26T05:59:05.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 0x7fc180156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.716 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1800b0040 0x7fc180155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc180140cc0 0x7fc18015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18013a8b0 con 0x7fc180070fe0 2023-10-26T05:59:05.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18013a2f0 con 0x7fc180140cc0 2023-10-26T05:59:05.717 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18013a5d0 con 0x7fc1800b0040 2023-10-26T05:59:05.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 0x7fc180156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 0x7fc180156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42110/0 (socket says 172.21.15.71:42110) 2023-10-26T05:59:05.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 -- 172.21.15.71:0/432754413 learned_addr learned my addr 172.21.15.71:0/432754413 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:05.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 -- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1800b0040 msgr2=0x7fc180155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.718 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1800b0040 0x7fc180155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 -- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc180140cc0 msgr2=0x7fc18015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc180140cc0 0x7fc18015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 -- 172.21.15.71:0/432754413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700098e0 con 0x7fc180070fe0 2023-10-26T05:59:05.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc184de6640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 0x7fc180156620 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc170000f90 tx=0x7fc170003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:05.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1700bc070 con 0x7fc180070fe0 2023-10-26T05:59:05.719 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc170004250 con 0x7fc180070fe0 2023-10-26T05:59:05.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1801599a0 con 0x7fc180070fe0 2023-10-26T05:59:05.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1700045c0 con 0x7fc180070fe0 2023-10-26T05:59:05.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc180139f10 con 0x7fc180070fe0 2023-10-26T05:59:05.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc180066b10 con 0x7fc180070fe0 2023-10-26T05:59:05.720 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc1700b4040 con 0x7fc180070fe0 2023-10-26T05:59:05.721 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17d7fa640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc16c066470 0x7fc16c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc17011ee00 con 0x7fc180070fe0 2023-10-26T05:59:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17ffff640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc16c066470 0x7fc16c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.707+0000 7fc17ffff640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc16c066470 0x7fc16c068930 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc180138260 tx=0x7fc174006ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:05.722 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.711+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc1700ec310 con 0x7fc180070fe0 2023-10-26T05:59:05.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.895+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc180066b10 con 0x7fc180070fe0 2023-10-26T05:59:05.906 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc17d7fa640 1 -- 172.21.15.71:0/432754413 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fc1700b2090 con 0x7fc180070fe0 2023-10-26T05:59:05.907 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:05.907 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:59:05.907 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:59:05.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc16c066470 msgr2=0x7fc16c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc16c066470 0x7fc16c068930 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc180138260 tx=0x7fc174006ca0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 msgr2=0x7fc180156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:05.909 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 0x7fc180156620 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fc170000f90 tx=0x7fc170003cf0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 shutdown_connections 2023-10-26T05:59:05.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc180140cc0 0x7fc18015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc1800b0040 0x7fc180155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc16c066470 0x7fc16c068930 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 --2- 172.21.15.71:0/432754413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc180070fe0 0x7fc180156620 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:05.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 >> 172.21.15.71:0/432754413 conn(0x7fc180069770 msgr2=0x7fc1800b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:05.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 shutdown_connections 2023-10-26T05:59:05.911 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:05.899+0000 7fc18704a640 1 -- 172.21.15.71:0/432754413 wait complete. 2023-10-26T05:59:05.926 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:59:06.038 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1343091070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1c0039b0 msgr2=0x7f6a1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 --2- 172.21.15.71:0/1343091070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1c0039b0 0x7f6a1c003dd0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f6a3008a3b0 tx=0x7f6a2c01d000 comp rx=0 tx=0).stop 2023-10-26T05:59:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 -- 172.21.15.71:0/1343091070 reap_dead start 2023-10-26T05:59:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1343091070 shutdown_connections 2023-10-26T05:59:06.039 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1343091070 >> 172.21.15.71:0/1343091070 conn(0x7f6a300cdd80 msgr2=0x7f6a300ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1343091070 shutdown_connections 2023-10-26T05:59:06.040 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1343091070 wait complete. 2023-10-26T05:59:06.042 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 Processor -- start 2023-10-26T05:59:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- start start 2023-10-26T05:59:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a1c0039b0 0x7f6a3014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a3015f280 0x7f6a30161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 0x7f6a30161c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.043 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a301608e0 con 0x7f6a3015f700 2023-10-26T05:59:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3015fcd0 con 0x7f6a3015f280 2023-10-26T05:59:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3015ffb0 con 0x7f6a1c0039b0 2023-10-26T05:59:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 0x7f6a30161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 0x7f6a30161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42134/0 (socket says 172.21.15.71:42134) 2023-10-26T05:59:06.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 -- 172.21.15.71:0/1914094947 learned_addr learned my addr 172.21.15.71:0/1914094947 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:06.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a35be4640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a3015f280 0x7f6a30161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a363e5640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a1c0039b0 0x7f6a3014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 -- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a1c0039b0 msgr2=0x7f6a3014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a1c0039b0 0x7f6a3014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 -- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a3015f280 msgr2=0x7f6a30161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a3015f280 0x7f6a30161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 -- 172.21.15.71:0/1914094947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30160290 con 0x7f6a3015f700 2023-10-26T05:59:06.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a363e5640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a1c0039b0 0x7f6a3014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:06.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a35be4640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a3015f280 0x7f6a30161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:06.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a36be6640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 0x7f6a30161c70 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f6a300d0c30 tx=0x7f6a2800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.046 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004d30 con 0x7f6a3015f700 2023-10-26T05:59:06.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6a2800daa0 con 0x7f6a3015f700 2023-10-26T05:59:06.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a30160570 con 0x7f6a3015f700 2023-10-26T05:59:06.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800de90 con 0x7f6a3015f700 2023-10-26T05:59:06.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a30159fd0 con 0x7f6a3015f700 2023-10-26T05:59:06.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.031+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a30066b10 con 0x7f6a3015f700 2023-10-26T05:59:06.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.035+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6a28011440 con 0x7f6a3015f700 2023-10-26T05:59:06.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.035+0000 7f6a26ffd640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a140664a0 0x7f6a14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.035+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a28083b70 con 0x7f6a3015f700 2023-10-26T05:59:06.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.035+0000 7f6a363e5640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a140664a0 0x7f6a14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.035+0000 7f6a363e5640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a140664a0 0x7f6a14068960 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0ac000 tx=0x7f6a2c0a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.039+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f6a28050ff0 con 0x7f6a3015f700 2023-10-26T05:59:06.192 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.183+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a30160700 con 0x7f6a3015f700 2023-10-26T05:59:06.193 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.183+0000 7f6a26ffd640 1 -- 172.21.15.71:0/1914094947 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16495 (secure 0 0 0) 0x7f6a28054610 con 0x7f6a3015f700 2023-10-26T05:59:06.194 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:06.194 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":191,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:57.314223+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:59:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a140664a0 msgr2=0x7f6a14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.195 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a140664a0 0x7f6a14068960 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0ac000 tx=0x7f6a2c0a7040 comp rx=0 tx=0).stop 2023-10-26T05:59:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 msgr2=0x7f6a30161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 0x7f6a30161c70 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f6a300d0c30 tx=0x7f6a2800fd90 comp rx=0 tx=0).stop 2023-10-26T05:59:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 shutdown_connections 2023-10-26T05:59:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6a140664a0 0x7f6a14068960 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.196 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a3015f700 0x7f6a30161c70 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a3015f280 0x7f6a30161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 --2- 172.21.15.71:0/1914094947 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6a1c0039b0 0x7f6a3014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 >> 172.21.15.71:0/1914094947 conn(0x7f6a300cdd80 msgr2=0x7f6a3006aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 shutdown_connections 2023-10-26T05:59:06.197 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.187+0000 7f6a38649640 1 -- 172.21.15.71:0/1914094947 wait complete. 2023-10-26T05:59:06.212 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:59:06.212 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:59:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.311+0000 7fa727ce2640 1 -- 172.21.15.71:0/1953721491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa72015bc10 msgr2=0x7fa72015c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.311+0000 7fa727ce2640 1 --2- 172.21.15.71:0/1953721491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa72015bc10 0x7fa72015c0a0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fa710009900 tx=0x7fa71001c7c0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.311+0000 7fa727ce2640 1 -- 172.21.15.71:0/1953721491 shutdown_connections 2023-10-26T05:59:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.311+0000 7fa727ce2640 1 --2- 172.21.15.71:0/1953721491 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa7200b1470 0x7fa7200b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.311+0000 7fa727ce2640 1 --2- 172.21.15.71:0/1953721491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa72015bc10 0x7fa72015c0a0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.311+0000 7fa727ce2640 1 -- 172.21.15.71:0/1953721491 >> 172.21.15.71:0/1953721491 conn(0x7fa7200afeb0 msgr2=0x7fa7200b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- 172.21.15.71:0/1953721491 shutdown_connections 2023-10-26T05:59:06.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- 172.21.15.71:0/1953721491 wait complete. 2023-10-26T05:59:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 Processor -- start 2023-10-26T05:59:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- start start 2023-10-26T05:59:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200b1470 0x7fa720139330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.326 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 0x7fa72013afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa72015bc10 0x7fa72014a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72013a600 con 0x7fa720141a70 2023-10-26T05:59:06.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7201399f0 con 0x7fa7200b1470 2023-10-26T05:59:06.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa720139cd0 con 0x7fa72015bc10 2023-10-26T05:59:06.327 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 0x7fa72013afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 0x7fa72013afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42160/0 (socket says 172.21.15.71:42160) 2023-10-26T05:59:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 -- 172.21.15.71:0/804784890 learned_addr learned my addr 172.21.15.71:0/804784890 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa725a7e640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200b1470 0x7fa720139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 -- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa72015bc10 msgr2=0x7fa72014a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72627f640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa72015bc10 0x7fa72014a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.328 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa72015bc10 0x7fa72014a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 -- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200b1470 msgr2=0x7fa720139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200b1470 0x7fa720139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 -- 172.21.15.71:0/804784890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa710009560 con 0x7fa720141a70 2023-10-26T05:59:06.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa725a7e640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200b1470 0x7fa720139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:06.329 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa72527d640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 0x7fa72013afb0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fa710009850 tx=0x7fa7100a56c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710003800 con 0x7fa720141a70 2023-10-26T05:59:06.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa710003960 con 0x7fa720141a70 2023-10-26T05:59:06.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa720139fb0 con 0x7fa720141a70 2023-10-26T05:59:06.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710003d50 con 0x7fa720141a70 2023-10-26T05:59:06.330 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.315+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa720140ac0 con 0x7fa720141a70 2023-10-26T05:59:06.331 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.319+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa720066d00 con 0x7fa720141a70 2023-10-26T05:59:06.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.319+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa7100b4070 con 0x7fa720141a70 2023-10-26T05:59:06.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.319+0000 7fa7167fc640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa7040664f0 0x7fa7040689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.319+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa71011e330 con 0x7fa720141a70 2023-10-26T05:59:06.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.319+0000 7fa725a7e640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa7040664f0 0x7fa7040689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.332 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.319+0000 7fa725a7e640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa7040664f0 0x7fa7040689b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0016e0 tx=0x7fa71c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.333 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.323+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa7100eb950 con 0x7fa720141a70 2023-10-26T05:59:06.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.499+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa720066d00 con 0x7fa720141a70 2023-10-26T05:59:06.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.499+0000 7fa7167fc640 1 -- 172.21.15.71:0/804784890 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fa7100eef70 con 0x7fa720141a70 2023-10-26T05:59:06.510 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:59:06.511 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:59:06.512 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa7040664f0 msgr2=0x7fa7040689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa7040664f0 0x7fa7040689b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0016e0 tx=0x7fa71c009040 comp rx=0 tx=0).stop 2023-10-26T05:59:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 msgr2=0x7fa72013afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 0x7fa72013afb0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fa710009850 tx=0x7fa7100a56c0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 shutdown_connections 2023-10-26T05:59:06.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa72015bc10 0x7fa72014a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa7040664f0 0x7fa7040689b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa720141a70 0x7fa72013afb0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 --2- 172.21.15.71:0/804784890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200b1470 0x7fa720139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 >> 172.21.15.71:0/804784890 conn(0x7fa7200afeb0 msgr2=0x7fa7200b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 shutdown_connections 2023-10-26T05:59:06.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.503+0000 7fa727ce2640 1 -- 172.21.15.71:0/804784890 wait complete. 2023-10-26T05:59:06.528 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:59:06.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.627+0000 7fbb72a71640 1 -- 172.21.15.71:0/1697494063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c13c2f0 msgr2=0x7fbb6c0b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.627+0000 7fbb72a71640 1 --2- 172.21.15.71:0/1697494063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b2350 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009840 tx=0x7fbb6001c800 comp rx=0 tx=0).stop 2023-10-26T05:59:06.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/1697494063 shutdown_connections 2023-10-26T05:59:06.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 --2- 172.21.15.71:0/1697494063 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb6c0b2890 0x7fbb6c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 --2- 172.21.15.71:0/1697494063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b2350 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/1697494063 >> 172.21.15.71:0/1697494063 conn(0x7fbb6c069770 msgr2=0x7fbb6c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/1697494063 shutdown_connections 2023-10-26T05:59:06.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/1697494063 wait complete. 2023-10-26T05:59:06.642 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 Processor -- start 2023-10-26T05:59:06.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- start start 2023-10-26T05:59:06.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb6c0b2890 0x7fbb6c0abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c0cb010 0x7fbb6c14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0aa8e0 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0aabc0 con 0x7fbb6c0cb010 2023-10-26T05:59:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0aaea0 con 0x7fbb6c0b2890 2023-10-26T05:59:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42164/0 (socket says 172.21.15.71:42164) 2023-10-26T05:59:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 -- 172.21.15.71:0/232000774 learned_addr learned my addr 172.21.15.71:0/232000774 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:06.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb6bfff640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c0cb010 0x7fbb6c14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7080d640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb6c0b2890 0x7fbb6c0abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 -- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb6c0b2890 msgr2=0x7fbb6c0abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb6c0b2890 0x7fbb6c0abc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 -- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c0cb010 msgr2=0x7fbb6c14e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c0cb010 0x7fbb6c14e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 -- 172.21.15.71:0/232000774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600094a0 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb7100e640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b0ce0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400cb40 tx=0x7fbb6400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400aa50 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c0ab180 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fbb6400aca0 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64012680 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c0ab740 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.631+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c066b10 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.635+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbb6401a3b0 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.635+0000 7fbb697fa640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb50066470 0x7fbb50068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.635+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb64084310 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.635+0000 7fbb7080d640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb50066470 0x7fbb50068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.635+0000 7fbb7080d640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb50066470 0x7fbb50068930 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af000 tx=0x7fbb6001b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.639+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fbb64051710 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.823 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.815+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb6c0721e0 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.815+0000 7fbb697fa640 1 -- 172.21.15.71:0/232000774 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fbb64054d30 con 0x7fbb6c13c2f0 2023-10-26T05:59:06.825 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:06.825 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-26T05:58:22.010694+0000","modified":"2023-10-26T05:58:24.036008+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9387},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9387":{"gid":9387,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6835/327274368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":327274368},{"type":"v1","addr":"172.21.15.71:6835","nonce":327274368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-26T05:59:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2023-10-26T05:59:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb50066470 msgr2=0x7fbb50068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb50066470 0x7fbb50068930 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af000 tx=0x7fbb6001b040 comp rx=0 tx=0).stop 2023-10-26T05:59:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 msgr2=0x7fbb6c0b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b0ce0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400cb40 tx=0x7fbb6400cc30 comp rx=0 tx=0).stop 2023-10-26T05:59:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 shutdown_connections 2023-10-26T05:59:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fbb50066470 0x7fbb50068930 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb6c13c2f0 0x7fbb6c0b0ce0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6c0cb010 0x7fbb6c14e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 --2- 172.21.15.71:0/232000774 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb6c0b2890 0x7fbb6c0abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 >> 172.21.15.71:0/232000774 conn(0x7fbb6c069770 msgr2=0x7fbb6c0ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 shutdown_connections 2023-10-26T05:59:06.829 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.819+0000 7fbb72a71640 1 -- 172.21.15.71:0/232000774 wait complete. 2023-10-26T05:59:06.843 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:59:06.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 -- 172.21.15.71:0/3134755155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 msgr2=0x7faef415aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 --2- 172.21.15.71:0/3134755155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef415aff0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7faef406b650 tx=0x7faee801d090 comp rx=0 tx=0).stop 2023-10-26T05:59:06.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 -- 172.21.15.71:0/3134755155 shutdown_connections 2023-10-26T05:59:06.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 --2- 172.21.15.71:0/3134755155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef415b530 0x7faef4157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 --2- 172.21.15.71:0/3134755155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef415aff0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 --2- 172.21.15.71:0/3134755155 >> v2:172.21.15.86:3300/0 conn(0x7faef414e300 0x7faef40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 -- 172.21.15.71:0/3134755155 >> 172.21.15.71:0/3134755155 conn(0x7faef4069770 msgr2=0x7faef4069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:06.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 -- 172.21.15.71:0/3134755155 shutdown_connections 2023-10-26T05:59:06.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.955+0000 7faefaba8640 1 -- 172.21.15.71:0/3134755155 wait complete. 2023-10-26T05:59:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 Processor -- start 2023-10-26T05:59:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- start start 2023-10-26T05:59:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef414e300 0x7faef41f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef415b530 0x7faef41482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef4152c70 con 0x7faef414e6e0 2023-10-26T05:59:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41520d0 con 0x7faef414e300 2023-10-26T05:59:06.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef41f2b40 con 0x7faef415b530 2023-10-26T05:59:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef414ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42176/0 (socket says 172.21.15.71:42176) 2023-10-26T05:59:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 -- 172.21.15.71:0/1636033306 learned_addr learned my addr 172.21.15.71:0/1636033306 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef8944640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef414e300 0x7faef41f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef9145640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef415b530 0x7faef41482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 -- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef415b530 msgr2=0x7faef41482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef415b530 0x7faef41482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 -- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef414e300 msgr2=0x7faef41f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:06.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef414e300 0x7faef41f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:06.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 -- 172.21.15.71:0/1636033306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8009df0 con 0x7faef414e6e0 2023-10-26T05:59:06.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef3fff640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef414ba40 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7faee801cb00 tx=0x7faee80ac5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8005230 con 0x7faef414e6e0 2023-10-26T05:59:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7faee8005480 con 0x7faef414e6e0 2023-10-26T05:59:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80acab0 con 0x7faef414e6e0 2023-10-26T05:59:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef4152340 con 0x7faef414e6e0 2023-10-26T05:59:06.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef4152900 con 0x7faef414e6e0 2023-10-26T05:59:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.959+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef4066b10 con 0x7faef414e6e0 2023-10-26T05:59:06.975 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.963+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faee80acc10 con 0x7faef414e6e0 2023-10-26T05:59:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.963+0000 7faef17fa640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faee0066470 0x7faee0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.963+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faee811eeb0 con 0x7faef414e6e0 2023-10-26T05:59:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.963+0000 7faef8944640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faee0066470 0x7faee0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.963+0000 7faef8944640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faee0066470 0x7faee0068930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7faee4005fd0 tx=0x7faee400a2d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:06.976 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:06.967+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7faee80ec2a0 con 0x7faef414e6e0 2023-10-26T05:59:07.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.123+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faef4152e20 con 0x7faef414e6e0 2023-10-26T05:59:07.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.127+0000 7faef17fa640 1 -- 172.21.15.71:0/1636033306 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16495 (secure 0 0 0) 0x7faef4152e20 con 0x7faef414e6e0 2023-10-26T05:59:07.136 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:07.136 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":191,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:58:57.314223+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-26T05:58:20.268236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-26T05:58:20.996636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:59:07.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faee0066470 msgr2=0x7faee0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faee0066470 0x7faee0068930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7faee4005fd0 tx=0x7faee400a2d0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 msgr2=0x7faef414ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef414ba40 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7faee801cb00 tx=0x7faee80ac5e0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 shutdown_connections 2023-10-26T05:59:07.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faef415b530 0x7faef41482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faee0066470 0x7faee0068930 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faef414e6e0 0x7faef414ba40 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 --2- 172.21.15.71:0/1636033306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faef414e300 0x7faef41f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 >> 172.21.15.71:0/1636033306 conn(0x7faef4069770 msgr2=0x7faef4069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 shutdown_connections 2023-10-26T05:59:07.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.131+0000 7faefaba8640 1 -- 172.21.15.71:0/1636033306 wait complete. 2023-10-26T05:59:07.155 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-26T05:59:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- 172.21.15.71:0/862001429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80ca7f0 msgr2=0x7f0ff8159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/862001429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80ca7f0 0x7f0ff8159700 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f0ff806d820 tx=0x7f0fe801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- 172.21.15.71:0/862001429 shutdown_connections 2023-10-26T05:59:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/862001429 >> v2:172.21.15.86:3300/0 conn(0x7f0ff80cabd0 0x7f0ff80b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/862001429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff80ca7f0 0x7f0ff8159700 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- 172.21.15.71:0/862001429 >> 172.21.15.71:0/862001429 conn(0x7f0ff806b8c0 msgr2=0x7f0ff806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- 172.21.15.71:0/862001429 shutdown_connections 2023-10-26T05:59:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- 172.21.15.71:0/862001429 wait complete. 2023-10-26T05:59:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 Processor -- start 2023-10-26T05:59:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- start start 2023-10-26T05:59:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ff80ca7f0 0x7f0ff81501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff80cabd0 0x7f0ff8134490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 0x7f0ff81581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8150730 con 0x7f0ff815ccf0 2023-10-26T05:59:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8135980 con 0x7f0ff80cabd0 2023-10-26T05:59:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8134e00 con 0x7f0ff80ca7f0 2023-10-26T05:59:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 0x7f0ff81581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 0x7f0ff81581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42194/0 (socket says 172.21.15.71:42194) 2023-10-26T05:59:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 -- 172.21.15.71:0/1437213895 learned_addr learned my addr 172.21.15.71:0/1437213895 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ff7fff640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff80cabd0 0x7f0ff8134490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffc962640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ff80ca7f0 0x7f0ff81501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 -- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ff80ca7f0 msgr2=0x7f0ff81501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ff80ca7f0 0x7f0ff81501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 -- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff80cabd0 msgr2=0x7f0ff8134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff80cabd0 0x7f0ff8134490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 -- 172.21.15.71:0/1437213895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe8009560 con 0x7f0ff815ccf0 2023-10-26T05:59:07.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ff7fff640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff80cabd0 0x7f0ff8134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:07.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffc962640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ff80ca7f0 0x7f0ff81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:07.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffd163640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 0x7f0ff81581d0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c760 tx=0x7f0ff000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:07.287 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0003b80 con 0x7f0ff815ccf0 2023-10-26T05:59:07.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0ff0003ce0 con 0x7f0ff815ccf0 2023-10-26T05:59:07.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.271+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff81350e0 con 0x7f0ff815ccf0 2023-10-26T05:59:07.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000aa50 con 0x7f0ff815ccf0 2023-10-26T05:59:07.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff81356a0 con 0x7f0ff815ccf0 2023-10-26T05:59:07.288 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff8066b10 con 0x7f0ff815ccf0 2023-10-26T05:59:07.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0ff000abb0 con 0x7f0ff815ccf0 2023-10-26T05:59:07.289 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ff57fa640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fe4066470 0x7f0fe4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ff0083d70 con 0x7f0ff815ccf0 2023-10-26T05:59:07.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ffc962640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fe4066470 0x7f0fe4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.275+0000 7f0ffc962640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fe4066470 0x7f0fe4068930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80af000 tx=0x7f0fe801b010 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:07.290 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.279+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f0ff00511f0 con 0x7f0ff815ccf0 2023-10-26T05:59:07.473 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.463+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0ff806c7d0 con 0x7f0ff815ccf0 2023-10-26T05:59:07.474 INFO:tasks.ceph.mon.a.smithi071.stderr:2023-10-26T05:59:07.467+0000 7fc5777c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-26T05:59:07.478 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ff57fa640 1 -- 172.21.15.71:0/1437213895 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0ff000ae60 con 0x7f0ff815ccf0 2023-10-26T05:59:07.478 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-26T05:59:07.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fe4066470 msgr2=0x7f0fe4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fe4066470 0x7f0fe4068930 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80af000 tx=0x7f0fe801b010 comp rx=0 tx=0).stop 2023-10-26T05:59:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 msgr2=0x7f0ff81581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 0x7f0ff81581d0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000c760 tx=0x7f0ff000cc20 comp rx=0 tx=0).stop 2023-10-26T05:59:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 shutdown_connections 2023-10-26T05:59:07.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f0fe4066470 0x7f0fe4068930 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff815ccf0 0x7f0ff81581d0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ff80cabd0 0x7f0ff8134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 --2- 172.21.15.71:0/1437213895 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ff80ca7f0 0x7f0ff81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.482 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 >> 172.21.15.71:0/1437213895 conn(0x7f0ff806b8c0 msgr2=0x7f0ff8137a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:07.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 shutdown_connections 2023-10-26T05:59:07.483 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.471+0000 7f0ffebc6640 1 -- 172.21.15.71:0/1437213895 wait complete. 2023-10-26T05:59:07.489 INFO:tasks.ceph.mds.c.smithi071.stderr: -7873> 2023-10-26T05:58:28.611+0000 7fa3735cf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-26T05:59:07.502 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-26T05:59:07.521 INFO:tasks.ceph.mds.c.smithi071.stdout:starting mds.c at 2023-10-26T05:59:07.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 -- 172.21.15.71:0/407754870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390003980 msgr2=0x7f0390003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 --2- 172.21.15.71:0/407754870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0390003980 0x7f0390003da0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f039800bbe0 tx=0x7f039801ce30 comp rx=0 tx=0).stop 2023-10-26T05:59:07.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03acdc5640 1 -- 172.21.15.71:0/407754870 reap_dead start 2023-10-26T05:59:07.621 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 -- 172.21.15.71:0/407754870 shutdown_connections 2023-10-26T05:59:07.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 -- 172.21.15.71:0/407754870 >> 172.21.15.71:0/407754870 conn(0x7f03a80aec60 msgr2=0x7f03a80af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:07.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 -- 172.21.15.71:0/407754870 shutdown_connections 2023-10-26T05:59:07.622 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 -- 172.21.15.71:0/407754870 wait complete. 2023-10-26T05:59:07.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 Processor -- start 2023-10-26T05:59:07.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.611+0000 7f03ae828640 1 -- start start 2023-10-26T05:59:07.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03a8135490 0x7f03a813c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 0x7f03a81367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03a815c4a0 0x7f03a80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81363d0 con 0x7f03a8135870 2023-10-26T05:59:07.625 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a813e6a0 con 0x7f03a815c4a0 2023-10-26T05:59:07.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a81379e0 con 0x7f03a8135490 2023-10-26T05:59:07.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 0x7f03a81367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 0x7f03a81367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42222/0 (socket says 172.21.15.71:42222) 2023-10-26T05:59:07.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 -- 172.21.15.71:0/2410541782 learned_addr learned my addr 172.21.15.71:0/2410541782 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:07.626 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03acdc5640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03a815c4a0 0x7f03a80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a7fff640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03a8135490 0x7f03a813c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 -- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03a8135490 msgr2=0x7f03a813c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03a8135490 0x7f03a813c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.627 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 -- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03a815c4a0 msgr2=0x7f03a80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:07.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03a815c4a0 0x7f03a80b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:07.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 -- 172.21.15.71:0/2410541782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a813e0e0 con 0x7f03a8135870 2023-10-26T05:59:07.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03acdc5640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03a815c4a0 0x7f03a80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:07.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a77fe640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 0x7f03a81367f0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f039c011a50 tx=0x7f039c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:07.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00fd90 con 0x7f03a8135870 2023-10-26T05:59:07.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f039c012450 con 0x7f03a8135870 2023-10-26T05:59:07.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a813e3c0 con 0x7f03a8135870 2023-10-26T05:59:07.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c012860 con 0x7f03a8135870 2023-10-26T05:59:07.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a8136d30 con 0x7f03a8135870 2023-10-26T05:59:07.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a8066b10 con 0x7f03a8135870 2023-10-26T05:59:07.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f039c0129c0 con 0x7f03a8135870 2023-10-26T05:59:07.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03940664a0 0x7f0394068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:07.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f039c084d70 con 0x7f03a8135870 2023-10-26T05:59:07.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=194}) v3 -- 0x7f039406e0e0 con 0x7f03a8135870 2023-10-26T05:59:07.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.615+0000 7f03a7fff640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03940664a0 0x7f0394068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:07.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.619+0000 7f03a7fff640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03940664a0 0x7f0394068960 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f039800bbe0 tx=0x7f03980a7040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:07.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.619+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f039c01f020 con 0x7f03a8135870 2023-10-26T05:59:07.820 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:07.811+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f03a8137010 con 0x7f03a8135870 2023-10-26T05:59:08.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.499+0000 7f03a4ff9640 1 -- 172.21.15.71:0/2410541782 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f039c0522d0 con 0x7f03a8135870 2023-10-26T05:59:08.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.503+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03940664a0 msgr2=0x7f0394068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:08.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.503+0000 7f03ae828640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03940664a0 0x7f0394068960 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f039800bbe0 tx=0x7f03980a7040 comp rx=0 tx=0).stop 2023-10-26T05:59:08.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.503+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 msgr2=0x7f03a81367f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:08.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.503+0000 7f03ae828640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 0x7f03a81367f0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f039c011a50 tx=0x7f039c011f10 comp rx=0 tx=0).stop 2023-10-26T05:59:08.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 shutdown_connections 2023-10-26T05:59:08.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f03a815c4a0 0x7f03a80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.515 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f03940664a0 0x7f0394068960 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03a8135870 0x7f03a81367f0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 --2- 172.21.15.71:0/2410541782 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f03a8135490 0x7f03a813c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 >> 172.21.15.71:0/2410541782 conn(0x7f03a80aec60 msgr2=0x7f03a80acc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:08.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 shutdown_connections 2023-10-26T05:59:08.516 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.507+0000 7f03ae828640 1 -- 172.21.15.71:0/2410541782 wait complete. 2023-10-26T05:59:08.539 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-26T05:59:08.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 -- 172.21.15.71:0/961325157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f013f2e0 msgr2=0x7f85f006a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:08.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/961325157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f013f2e0 0x7f85f006a010 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009c80 tx=0x7f85e801cd10 comp rx=0 tx=0).stop 2023-10-26T05:59:08.726 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 -- 172.21.15.71:0/961325157 shutdown_connections 2023-10-26T05:59:08.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/961325157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f013f2e0 0x7f85f006a010 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/961325157 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85f01422e0 0x7f85f013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 -- 172.21.15.71:0/961325157 >> 172.21.15.71:0/961325157 conn(0x7f85f00aec60 msgr2=0x7f85f00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:08.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 -- 172.21.15.71:0/961325157 shutdown_connections 2023-10-26T05:59:08.727 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.715+0000 7f85f4ed5640 1 -- 172.21.15.71:0/961325157 wait complete. 2023-10-26T05:59:08.729 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 Processor -- start 2023-10-26T05:59:08.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- start start 2023-10-26T05:59:08.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f013e9c0 0x7f85f01f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:08.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85f013f2e0 0x7f85f0157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:08.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 0x7f85f006f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:08.730 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0070cd0 con 0x7f85f01422e0 2023-10-26T05:59:08.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0070050 con 0x7f85f013e9c0 2023-10-26T05:59:08.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0070330 con 0x7f85f013f2e0 2023-10-26T05:59:08.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 0x7f85f006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:08.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 0x7f85f006f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42242/0 (socket says 172.21.15.71:42242) 2023-10-26T05:59:08.731 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 -- 172.21.15.71:0/930123874 learned_addr learned my addr 172.21.15.71:0/930123874 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:08.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85edd9b640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85f013f2e0 0x7f85f0157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:08.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85ee59c640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f013e9c0 0x7f85f01f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:08.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 -- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85f013f2e0 msgr2=0x7f85f0157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:08.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85f013f2e0 0x7f85f0157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 -- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f013e9c0 msgr2=0x7f85f01f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:08.732 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f013e9c0 0x7f85f01f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:08.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 -- 172.21.15.71:0/930123874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e80098e0 con 0x7f85f01422e0 2023-10-26T05:59:08.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85eed9d640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 0x7f85f006f750 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f85e80b0040 tx=0x7f85e8003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:08.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80ad9c0 con 0x7f85f01422e0 2023-10-26T05:59:08.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f85e80adc10 con 0x7f85f01422e0 2023-10-26T05:59:08.733 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8003f00 con 0x7f85f01422e0 2023-10-26T05:59:08.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f0070610 con 0x7f85f01422e0 2023-10-26T05:59:08.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f006fc90 con 0x7f85f01422e0 2023-10-26T05:59:08.734 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.719+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f0066b10 con 0x7f85f01422e0 2023-10-26T05:59:08.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.723+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f85e80b4040 con 0x7f85f01422e0 2023-10-26T05:59:08.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.723+0000 7f85deffd640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85d0066470 0x7f85d0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:08.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.723+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f85e811f0e0 con 0x7f85f01422e0 2023-10-26T05:59:08.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.723+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=194}) v3 -- 0x7f85d006e0b0 con 0x7f85f01422e0 2023-10-26T05:59:08.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.723+0000 7f85ee59c640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85d0066470 0x7f85d0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:08.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.723+0000 7f85ee59c640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85d0066470 0x7f85d0068930 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f85e0004e30 tx=0x7f85e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:08.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.727+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f85e80b2060 con 0x7f85f01422e0 2023-10-26T05:59:08.892 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:08.883+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f85f0066b10 con 0x7f85f01422e0 2023-10-26T05:59:09.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.515+0000 7f85deffd640 1 -- 172.21.15.71:0/930123874 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v194) v1 ==== 181+0+0 (secure 0 0 0) 0x7f85e80f0020 con 0x7f85f01422e0 2023-10-26T05:59:09.524 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2023-10-26T05:59:09.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85d0066470 msgr2=0x7f85d0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:09.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85d0066470 0x7f85d0068930 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f85e0004e30 tx=0x7f85e0009040 comp rx=0 tx=0).stop 2023-10-26T05:59:09.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 msgr2=0x7f85f006f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:09.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 0x7f85f006f750 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f85e80b0040 tx=0x7f85e8003d10 comp rx=0 tx=0).stop 2023-10-26T05:59:09.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 shutdown_connections 2023-10-26T05:59:09.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f85d0066470 0x7f85d0068930 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85f01422e0 0x7f85f006f750 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f85f013f2e0 0x7f85f0157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 --2- 172.21.15.71:0/930123874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f013e9c0 0x7f85f01f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.523+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 >> 172.21.15.71:0/930123874 conn(0x7f85f00aec60 msgr2=0x7f85f00ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:09.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.527+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 shutdown_connections 2023-10-26T05:59:09.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.527+0000 7f85f4ed5640 1 -- 172.21.15.71:0/930123874 wait complete. 2023-10-26T05:59:09.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-26T05:59:09.684 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 -- 172.21.15.71:0/944499097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 msgr2=0x7fabc40b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:09.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 --2- 172.21.15.71:0/944499097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40b1c60 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fabc406e940 tx=0x7fabac01f090 comp rx=0 tx=0).stop 2023-10-26T05:59:09.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 -- 172.21.15.71:0/944499097 shutdown_connections 2023-10-26T05:59:09.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 --2- 172.21.15.71:0/944499097 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabc40b21a0 0x7fabc40aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.685 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 --2- 172.21.15.71:0/944499097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40b1c60 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 --2- 172.21.15.71:0/944499097 >> v2:172.21.15.86:3300/0 conn(0x7fabc4141a40 0x7fabc40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 -- 172.21.15.71:0/944499097 >> 172.21.15.71:0/944499097 conn(0x7fabc40aec60 msgr2=0x7fabc40af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:09.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 -- 172.21.15.71:0/944499097 shutdown_connections 2023-10-26T05:59:09.686 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 -- 172.21.15.71:0/944499097 wait complete. 2023-10-26T05:59:09.688 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 Processor -- start 2023-10-26T05:59:09.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.675+0000 7fabca24b640 1 -- start start 2023-10-26T05:59:09.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:09.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabc40b21a0 0x7fabc413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:09.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc4141a40 0x7fabc4140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:09.689 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc413a380 con 0x7fabc40b1230 2023-10-26T05:59:09.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40ac470 con 0x7fabc4141a40 2023-10-26T05:59:09.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc41496a0 con 0x7fabc40b21a0 2023-10-26T05:59:09.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:09.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42266/0 (socket says 172.21.15.71:42266) 2023-10-26T05:59:09.690 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 -- 172.21.15.71:0/1829813507 learned_addr learned my addr 172.21.15.71:0/1829813507 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:09.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc3fff640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc4141a40 0x7fabc4140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:09.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc2ffd640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabc40b21a0 0x7fabc413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:09.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 -- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabc40b21a0 msgr2=0x7fabc413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:09.691 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabc40b21a0 0x7fabc413fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 -- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc4141a40 msgr2=0x7fabc4140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc4141a40 0x7fabc4140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac009560 con 0x7fabc40b1230 2023-10-26T05:59:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc3fff640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc4141a40 0x7fabc4140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:09.692 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabc37fe640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40abf30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fabb400eaa0 tx=0x7fabb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:09.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400ce10 con 0x7fabc40b1230 2023-10-26T05:59:09.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fabb4011470 con 0x7fabc40b1230 2023-10-26T05:59:09.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4011860 con 0x7fabc40b1230 2023-10-26T05:59:09.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc4149980 con 0x7fabc40b1230 2023-10-26T05:59:09.693 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc4149f40 con 0x7fabc40b1230 2023-10-26T05:59:09.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4066b10 con 0x7fabc40b1230 2023-10-26T05:59:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fabb40119c0 con 0x7fabc40b1230 2023-10-26T05:59:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabbbfff640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faba4066470 0x7faba4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.679+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fabb40849b0 con 0x7fabc40b1230 2023-10-26T05:59:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.683+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=195}) v3 -- 0x7faba406d240 con 0x7fabc40b1230 2023-10-26T05:59:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.683+0000 7fabc2ffd640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faba4066470 0x7faba4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.683+0000 7fabc2ffd640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faba4066470 0x7faba4068930 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fabac0b2000 tx=0x7fabac01d8d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:09.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.687+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fabb401f070 con 0x7fabc40b1230 2023-10-26T05:59:09.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:09.839+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fabc4071070 con 0x7fabc40b1230 2023-10-26T05:59:10.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.527+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v195) v1 ==== 169+0+0 (secure 0 0 0) 0x7fabb4055a20 con 0x7fabc40b1230 2023-10-26T05:59:10.536 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2023-10-26T05:59:10.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.527+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 <== mon.0 v2:172.21.15.71:3300/0 8 ==== osd_map(195..195 src has 1..195) v4 ==== 296+0+0 (secure 0 0 0) 0x7fabb4056350 con 0x7fabc40b1230 2023-10-26T05:59:10.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.527+0000 7fabbbfff640 1 -- 172.21.15.71:0/1829813507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=196}) v3 -- 0x7faba406bfd0 con 0x7fabc40b1230 2023-10-26T05:59:10.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.539+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faba4066470 msgr2=0x7faba4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.539+0000 7fabca24b640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faba4066470 0x7faba4068930 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fabac0b2000 tx=0x7fabac01d8d0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.539+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 msgr2=0x7fabc40abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.539+0000 7fabca24b640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40abf30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fabb400eaa0 tx=0x7fabb400ef60 comp rx=0 tx=0).stop 2023-10-26T05:59:10.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 shutdown_connections 2023-10-26T05:59:10.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabc4141a40 0x7fabc4140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fabc40b21a0 0x7fabc413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7faba4066470 0x7faba4068930 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 --2- 172.21.15.71:0/1829813507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fabc40b1230 0x7fabc40abf30 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 >> 172.21.15.71:0/1829813507 conn(0x7fabc40aec60 msgr2=0x7fabc4157990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:10.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 shutdown_connections 2023-10-26T05:59:10.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.543+0000 7fabca24b640 1 -- 172.21.15.71:0/1829813507 wait complete. 2023-10-26T05:59:10.575 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 18 2023-10-26T05:59:10.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 -- 172.21.15.71:0/2772028520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 msgr2=0x7fb9fc0aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 --2- 172.21.15.71:0/2772028520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc0aefc0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4009c80 tx=0x7fb9f401cd80 comp rx=0 tx=0).stop 2023-10-26T05:59:10.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 -- 172.21.15.71:0/2772028520 shutdown_connections 2023-10-26T05:59:10.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 --2- 172.21.15.71:0/2772028520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc0aefc0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 --2- 172.21.15.71:0/2772028520 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9fc159060 0x7fb9fc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 -- 172.21.15.71:0/2772028520 >> 172.21.15.71:0/2772028520 conn(0x7fb9fc0b0e40 msgr2=0x7fb9fc0b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:10.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 -- 172.21.15.71:0/2772028520 shutdown_connections 2023-10-26T05:59:10.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 -- 172.21.15.71:0/2772028520 wait complete. 2023-10-26T05:59:10.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 Processor -- start 2023-10-26T05:59:10.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.691+0000 7fba00816640 1 -- start start 2023-10-26T05:59:10.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:10.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9fc159060 0x7fb9fc155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:10.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9fc159440 0x7fb9fc06b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:10.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc149090 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc15b1f0 con 0x7fb9fc159440 2023-10-26T05:59:10.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc15b470 con 0x7fb9fc159060 2023-10-26T05:59:10.705 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:10.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc15c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42276/0 (socket says 172.21.15.71:42276) 2023-10-26T05:59:10.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 -- 172.21.15.71:0/4173803114 learned_addr learned my addr 172.21.15.71:0/4173803114 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:10.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fad9d640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9fc159440 0x7fb9fc06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:10.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9f9d9b640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9fc159060 0x7fb9fc155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:10.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 -- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9fc159060 msgr2=0x7fb9fc155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9fc159060 0x7fb9fc155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 -- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9fc159440 msgr2=0x7fb9fc06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9fc159440 0x7fb9fc06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 -- 172.21.15.71:0/4173803114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f40098e0 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9fa59c640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc15c230 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c970 tx=0x7fb9ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:10.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec007c70 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fb9ec00f450 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9fc15b750 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00f860 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9fc15bd10 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9fc06a7b0 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.709 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb9ec00f9c0 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9eaffd640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb9dc066470 0x7fb9dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:10.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb9ec0843a0 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.695+0000 7fb9f9d9b640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb9dc066470 0x7fb9dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:10.710 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.699+0000 7fb9f9d9b640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb9dc066470 0x7fb9dc068930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c5bf0 tx=0x7fb9f0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:10.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.699+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fb9ec051bb0 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.864 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.855+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config reset", "num": 18} v 0) v1 -- 0x7fb9fc04db60 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.873 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.863+0000 7fb9eaffd640 1 -- 172.21.15.71:0/4173803114 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 18}]=0 v38) v1 ==== 71+0+0 (secure 0 0 0) 0x7fb9ec0551d0 con 0x7fb9fc0aeb90 2023-10-26T05:59:10.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb9dc066470 msgr2=0x7fb9dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.876 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb9dc066470 0x7fb9dc068930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0c5bf0 tx=0x7fb9f0009040 comp rx=0 tx=0).stop 2023-10-26T05:59:10.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 msgr2=0x7fb9fc15c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:10.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc15c230 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00c970 tx=0x7fb9ec00ce30 comp rx=0 tx=0).stop 2023-10-26T05:59:10.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 shutdown_connections 2023-10-26T05:59:10.877 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9fc159440 0x7fb9fc06b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb9fc159060 0x7fb9fc155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fb9dc066470 0x7fb9dc068930 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 --2- 172.21.15.71:0/4173803114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc0aeb90 0x7fb9fc15c230 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:10.878 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 >> 172.21.15.71:0/4173803114 conn(0x7fb9fc0b0e40 msgr2=0x7fb9fc06d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:10.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 shutdown_connections 2023-10-26T05:59:10.879 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:10.867+0000 7fba00816640 1 -- 172.21.15.71:0/4173803114 wait complete. 2023-10-26T05:59:10.894 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-10-26T05:59:11.018 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/719635575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e00b2a60 msgr2=0x7fd0e00caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/719635575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e00b2a60 0x7fd0e00caad0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009900 tx=0x7fd0d001c8c0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/719635575 shutdown_connections 2023-10-26T05:59:11.019 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/719635575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e00b2a60 0x7fd0e00caad0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/719635575 >> 172.21.15.71:0/719635575 conn(0x7fd0e0069770 msgr2=0x7fd0e0069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/719635575 shutdown_connections 2023-10-26T05:59:11.020 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/719635575 wait complete. 2023-10-26T05:59:11.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 Processor -- start 2023-10-26T05:59:11.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- start start 2023-10-26T05:59:11.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0e00b2a60 0x7fd0e00adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:11.024 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e00cb010 0x7fd0e0142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:11.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 0x7fd0e013b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:11.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e015cc00 con 0x7fd0e013c2f0 2023-10-26T05:59:11.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e013b600 con 0x7fd0e00cb010 2023-10-26T05:59:11.025 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e015c660 con 0x7fd0e00b2a60 2023-10-26T05:59:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 0x7fd0e013b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 0x7fd0e013b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42294/0 (socket says 172.21.15.71:42294) 2023-10-26T05:59:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 -- 172.21.15.71:0/2623464267 learned_addr learned my addr 172.21.15.71:0/2623464267 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:11.026 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0de59c640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0e00b2a60 0x7fd0e00adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ddd9b640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e00cb010 0x7fd0e0142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 -- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0e00b2a60 msgr2=0x7fd0e00adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0e00b2a60 0x7fd0e00adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 -- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e00cb010 msgr2=0x7fd0e0142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:11.027 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e00cb010 0x7fd0e0142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 -- 172.21.15.71:0/2623464267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0009560 con 0x7fd0e013c2f0 2023-10-26T05:59:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ddd9b640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e00cb010 0x7fd0e0142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0ded9d640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 0x7fd0e013b0c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c770 tx=0x7fd0d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8003b80 con 0x7fd0e013c2f0 2023-10-26T05:59:11.028 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd0d8003ce0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e015c0a0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800aab0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.011+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e013a8c0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.015+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e0066b10 con 0x7fd0e013c2f0 2023-10-26T05:59:11.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.015+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd0d800ac10 con 0x7fd0e013c2f0 2023-10-26T05:59:11.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.015+0000 7fd0d6ffd640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0c00664a0 0x7fd0c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:11.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.015+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd0d8083b40 con 0x7fd0e013c2f0 2023-10-26T05:59:11.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.015+0000 7fd0de59c640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0c00664a0 0x7fd0c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:11.030 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.015+0000 7fd0de59c640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0c00664a0 0x7fd0c0068960 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000f70 tx=0x7fd0c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:11.031 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.019+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fd0d80512c0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.239 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.231+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7fd0e00721e0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.539 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0d6ffd640 1 -- 172.21.15.71:0/2623464267 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v569) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd0d80548e0 con 0x7fd0e013c2f0 2023-10-26T05:59:11.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0c00664a0 msgr2=0x7fd0c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:11.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0c00664a0 0x7fd0c0068960 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000f70 tx=0x7fd0c8009040 comp rx=0 tx=0).stop 2023-10-26T05:59:11.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 msgr2=0x7fd0e013b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:11.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 0x7fd0e013b0c0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c770 tx=0x7fd0d800cc30 comp rx=0 tx=0).stop 2023-10-26T05:59:11.542 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 shutdown_connections 2023-10-26T05:59:11.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fd0c00664a0 0x7fd0c0068960 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0e013c2f0 0x7fd0e013b0c0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e00cb010 0x7fd0e0142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 --2- 172.21.15.71:0/2623464267 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd0e00b2a60 0x7fd0e00adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:11.543 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 >> 172.21.15.71:0/2623464267 conn(0x7fd0e0069770 msgr2=0x7fd0e0158160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:11.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 shutdown_connections 2023-10-26T05:59:11.544 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:11.531+0000 7fd0e48a6640 1 -- 172.21.15.71:0/2623464267 wait complete. 2023-10-26T05:59:11.560 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-10-26T05:59:11.560 INFO:tasks.cephfs_test_runner: 2023-10-26T05:59:11.561 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-26T05:59:11.561 INFO:tasks.cephfs_test_runner:Ran 10 tests in 677.760s 2023-10-26T05:59:11.561 INFO:tasks.cephfs_test_runner: 2023-10-26T05:59:11.562 INFO:tasks.cephfs_test_runner:OK 2023-10-26T05:59:11.562 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-26T05:59:11.571 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-26T05:59:11.615 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-26T05:59:11.616 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:59:11.616 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:11.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:59:11.623 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2023-10-26T05:59:11.623 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:11.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336528k,nr_inodes=4084132,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16950 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-26T05:59:11.676 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:59:11.676 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:11.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:59:11.683 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-26T05:59:11.683 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-26T05:59:11.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336568k,nr_inodes=4084142,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16166 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274692k,nr_inodes=818673,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274696k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-26T05:59:11.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:11.736 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2023-10-26T05:59:11.743 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-26T05:59:11.743 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-26T05:59:11.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:11.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-26T05:59:11.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:11.800 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-26T05:59:11.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:11.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2023-10-26T05:59:12.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:12.032 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-26T05:59:12.038 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-26T05:59:12.039 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-26T05:59:12.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:12.039 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-26T05:59:12.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:12.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-26T05:59:12.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-26T05:59:12.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2023-10-26T05:59:12.308 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-26T05:59:12.318 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-26T05:59:12.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:59:12.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/2432271099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 msgr2=0x7fe380004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.455 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 --2- 172.21.15.71:0/2432271099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe380004c50 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006fb0 tx=0x7fe38c01f6b0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/2432271099 shutdown_connections 2023-10-26T05:59:12.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 --2- 172.21.15.71:0/2432271099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe380004c50 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/2432271099 >> 172.21.15.71:0/2432271099 conn(0x7fe398069770 msgr2=0x7fe398069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:12.456 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/2432271099 shutdown_connections 2023-10-26T05:59:12.457 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/2432271099 wait complete. 2023-10-26T05:59:12.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 Processor -- start 2023-10-26T05:59:12.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- start start 2023-10-26T05:59:12.460 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe398158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe398136e10 0x7fe3981f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe39813eca0 0x7fe39813f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398137290 con 0x7fe3800047d0 2023-10-26T05:59:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39813f660 con 0x7fe39813eca0 2023-10-26T05:59:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe398142360 con 0x7fe398136e10 2023-10-26T05:59:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe398158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe398158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38862/0 (socket says 172.21.15.71:38862) 2023-10-26T05:59:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 -- 172.21.15.71:0/1819107739 learned_addr learned my addr 172.21.15.71:0/1819107739 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe395d9b640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe398136e10 0x7fe3981f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.462 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe396d9d640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe39813eca0 0x7fe39813f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 -- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe398136e10 msgr2=0x7fe3981f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe398136e10 0x7fe3981f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 -- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe39813eca0 msgr2=0x7fe39813f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe39813eca0 0x7fe39813f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.463 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 -- 172.21.15.71:0/1819107739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398141ee0 con 0x7fe3800047d0 2023-10-26T05:59:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39659c640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe398158e00 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c3c0 tx=0x7fe38800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880052d0 con 0x7fe3800047d0 2023-10-26T05:59:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe388003600 con 0x7fe3800047d0 2023-10-26T05:59:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3981425b0 con 0x7fe3800047d0 2023-10-26T05:59:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388003a30 con 0x7fe3800047d0 2023-10-26T05:59:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.447+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3981532e0 con 0x7fe3800047d0 2023-10-26T05:59:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.451+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398066b10 con 0x7fe3800047d0 2023-10-26T05:59:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.451+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe388003b90 con 0x7fe3800047d0 2023-10-26T05:59:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.451+0000 7fe386ffd640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3780664a0 0x7fe378068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.451+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe3880831f0 con 0x7fe3800047d0 2023-10-26T05:59:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.451+0000 7fe395d9b640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3780664a0 0x7fe378068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.451+0000 7fe395d9b640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3780664a0 0x7fe378068960 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0b0000 tx=0x7fe38c01c450 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:12.466 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.455+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe388050ad0 con 0x7fe3800047d0 2023-10-26T05:59:12.617 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.607+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3980721e0 con 0x7fe3780664a0 2023-10-26T05:59:12.624 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.615+0000 7fe386ffd640 1 -- 172.21.15.71:0/1819107739 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48652 (secure 0 0 0) 0x7fe3980721e0 con 0x7fe3780664a0 2023-10-26T05:59:12.626 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:12.628 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:59:12.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.615+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3780664a0 msgr2=0x7fe378068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.615+0000 7fe39c937640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3780664a0 0x7fe378068960 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0b0000 tx=0x7fe38c01c450 comp rx=0 tx=0).stop 2023-10-26T05:59:12.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.615+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 msgr2=0x7fe398158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.615+0000 7fe39c937640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe398158e00 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c3c0 tx=0x7fe38800c880 comp rx=0 tx=0).stop 2023-10-26T05:59:12.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 shutdown_connections 2023-10-26T05:59:12.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe39813eca0 0x7fe39813f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe398136e10 0x7fe3981f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe3780664a0 0x7fe378068960 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.630 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 --2- 172.21.15.71:0/1819107739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3800047d0 0x7fe398158e00 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 >> 172.21.15.71:0/1819107739 conn(0x7fe398069770 msgr2=0x7fe3980ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:12.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 shutdown_connections 2023-10-26T05:59:12.631 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.619+0000 7fe39c937640 1 -- 172.21.15.71:0/1819107739 wait complete. 2023-10-26T05:59:12.642 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2023-10-26T05:59:11.652045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450300,"statfs":{"total":1600000000,"available":1485108092,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52487222,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-121,"num_write_kb":-85082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.321452"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":325,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-26T05:59:07.333954+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:59:07.333954+0000","last_peered":"2023-10-26T05:59:07.333954+0000","last_clean":"2023-10-26T05:59:07.333954+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:59:07.333954+0000","last_undegraded":"2023-10-26T05:59:07.333954+0000","last_fullsized":"2023-10-26T05:59:07.333954+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185420,"statfs":{"total":200000000,"available":189870769,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.39600000000000002},"max":{"1min":1.2290000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.36399999999999999},"max":{"1min":1.125,"5min":1.196,"15min":1.3100000000000001},"last":0.62}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.98599999999999999,"15min":0.997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":1.387,"15min":1.4319999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0609999999999999,"15min":1.0009999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.621,"15min":1.621},"last":1.0580000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":0.85299999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.40799999999999997},"max":{"1min":1.3,"5min":1.448,"15min":1.448},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":1.2190000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.51500000000000001},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.4330000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":0.97599999999999998},"min":{"1min":0.76200000000000001,"5min":0.61599999999999999,"15min":0.52100000000000002},"max":{"1min":1.333,"5min":1.712,"15min":1.712},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.59599999999999997,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.2190000000000001,"5min":1.2589999999999999,"15min":1.639},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.70599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.4159999999999999},"last":1.258}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.999,"15min":1.0589999999999999},"min":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.335,"5min":1.482,"15min":1.6679999999999999},"last":1.145},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.03,"15min":1.0529999999999999},"min":{"1min":0.66700000000000004,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.383,"5min":1.504,"15min":1.702},"last":1.1870000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.087,"15min":1.099},"min":{"1min":0.86199999999999999,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.302,"5min":1.3640000000000001,"15min":1.7509999999999999},"last":1.298},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.069,"15min":1.0549999999999999},"min":{"1min":0.77100000000000002,"5min":0.66700000000000004,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.5,"15min":1.6180000000000001},"last":1.375}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19498,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175813,"statfs":{"total":200000000,"available":180033317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.84099999999999997},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90000000000000002,"15min":0.872},"min":{"1min":0.503,"5min":0.49299999999999999,"15min":0.376},"max":{"1min":1.286,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.85199999999999998,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.39100000000000001},"max":{"1min":1.071,"5min":1.1120000000000001,"15min":1.2609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.91700000000000004,"15min":0.874},"min":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.41999999999999998},"max":{"1min":1.157,"5min":1.3899999999999999,"15min":1.405},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89800000000000002,"15min":0.85699999999999998},"min":{"1min":0.66300000000000003,"5min":0.54800000000000004,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.268,"15min":1.8460000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.86499999999999999},"min":{"1min":0.65100000000000002,"5min":0.502,"15min":0.435},"max":{"1min":1.169,"5min":1.3,"15min":1.9239999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.028,"15min":0.94999999999999996},"min":{"1min":0.73799999999999999,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.073,"5min":1.0609999999999999,"15min":1.0149999999999999},"min":{"1min":0.79800000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.171}]},{"osd":4,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.502},"max":{"1min":1.4299999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98499999999999999},"min":{"1min":0.68200000000000005,"5min":0.621,"15min":0.442},"max":{"1min":1.375,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.075,"15min":1.0509999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.45600000000000002},"max":{"1min":1.4099999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.022,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.36799999999999999},"max":{"1min":1.3260000000000001,"5min":1.452,"15min":1.512},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.147,"15min":1.1180000000000001},"min":{"1min":1.0029999999999999,"5min":0.71599999999999997,"15min":0.41099999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.4550000000000001,"15min":1.5920000000000001},"last":1.099}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173765,"statfs":{"total":200000000,"available":177935785,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96499999999999997,"15min":0.95499999999999996},"min":{"1min":0.48699999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.264,"5min":1.381,"15min":1.579},"last":1.212},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.012,"15min":0.96899999999999997},"min":{"1min":0.68100000000000005,"5min":0.58799999999999997,"15min":0.499},"max":{"1min":1.1499999999999999,"5min":1.591,"15min":1.591},"last":1.1200000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.872},"min":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.214,"5min":1.214,"15min":2.1150000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.86499999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.0740000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93600000000000005,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.21,"5min":1.569,"15min":1.569},"last":1.248},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89900000000000002,"15min":0.92000000000000004},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2569999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":0.85499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.363,"15min":1.548},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.335,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93100000000000005,"15min":0.96599999999999997},"min":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.46999999999999997},"max":{"1min":1.373,"5min":1.373,"15min":2.1909999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1120000000000001,"15min":1.0940000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40799999999999997},"max":{"1min":1.4099999999999999,"5min":1.4950000000000001,"15min":1.536},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.02,"15min":1.0049999999999999},"min":{"1min":0.60199999999999998,"5min":0.50700000000000001,"15min":0.41899999999999998},"max":{"1min":1.2869999999999999,"5min":1.744,"15min":1.744},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.056,"15min":1.032},"min":{"1min":0.71299999999999997,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.3160000000000001,"5min":1.4199999999999999,"15min":1.4950000000000001},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0980000000000001,"15min":1.075},"min":{"1min":0.754,"5min":0.53600000000000003,"15min":0.442},"max":{"1min":1.4810000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.288},{"interface":"front","average":{"1min":1.153,"5min":1.1799999999999999,"15min":1.155},"min":{"1min":0.86099999999999999,"5min":0.61899999999999999,"15min":0.49299999999999999},"max":{"1min":1.3480000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.3300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188045,"statfs":{"total":200000000,"available":192558961,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.2390000000000001,"5min":1.248,"15min":1.347},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.47199999999999998},"max":{"1min":1.1299999999999999,"5min":1.234,"15min":1.502},"last":0.90800000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.85099999999999998},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.375},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.95299999999999996,"15min":0.92400000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.1839999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93200000000000005,"15min":0.86499999999999999},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.65300000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.149,"5min":1.222,"15min":1.373},"last":0.94799999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.96999999999999997,"15min":0.94799999999999995},"min":{"1min":0.63300000000000001,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.2649999999999999,"5min":1.397,"15min":2.0190000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.02,"15min":0.97299999999999998},"min":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.58899999999999997},"max":{"1min":1.2030000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.0549999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2450000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.022,"15min":1.0049999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.673},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.59099999999999997,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.3759999999999999,"5min":1.762,"15min":1.7969999999999999},"last":1.014},{"interface":"front","average":{"1min":1.069,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.70699999999999996,"5min":0.67100000000000004,"15min":0.51600000000000001},"max":{"1min":1.3120000000000001,"5min":1.355,"15min":1.4970000000000001},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.117,"15min":1.0669999999999999},"min":{"1min":0.81299999999999994,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.4610000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1499999999999999,"15min":1.1279999999999999},"min":{"1min":0.76000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.484,"15min":1.728},"last":1.2310000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9258,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186053,"statfs":{"total":200000000,"available":190519065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.999,"5min":0.999,"15min":1.385},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.2689999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68400000000000005,"15min":0.65800000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.93700000000000006,"5min":6.109,"15min":6.109},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.63500000000000001},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83799999999999997,"5min":1.02,"15min":1.1399999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.77500000000000002},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88900000000000001,"15min":0.81699999999999995},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.125,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.40000000000000002},"max":{"1min":1.026,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.52400000000000002},"max":{"1min":1.1830000000000001,"5min":1.429,"15min":1.4810000000000001},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.95499999999999996},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.2130000000000001,"5min":1.454,"15min":1.478},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94599999999999995,"15min":0.92800000000000005},"min":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.1619999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.876}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":0.98899999999999999},"min":{"1min":0.65800000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.002,"15min":1.0209999999999999},"min":{"1min":0.57599999999999996,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":1.286,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.95599999999999996},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.2410000000000001,"5min":2.016,"15min":2.016},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0680000000000001,"15min":1.046},"min":{"1min":0.70799999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3120000000000001,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.98099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173765,"statfs":{"total":200000000,"available":177936299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.628,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":1.097},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26000000000000001},"max":{"1min":1.006,"5min":1.1619999999999999,"15min":1.5249999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.70699999999999996},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.0669999999999999,"5min":1.198,"15min":1.2609999999999999},"last":0.436},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.30199999999999999,"15min":0.26700000000000002},"max":{"1min":0.96599999999999997,"5min":1.157,"15min":1.3600000000000001},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81200000000000006,"15min":0.77600000000000002},"min":{"1min":0.373,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.008,"5min":1.24,"15min":2.8809999999999998},"last":0.308},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.3440000000000001},"last":0.37}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90500000000000003,"15min":0.876},"min":{"1min":0.625,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.149,"5min":1.177,"15min":2.6139999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.73799999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1459999999999999,"15min":5.9550000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2090000000000001,"5min":1.5289999999999999,"15min":2.8319999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.96099999999999997,"15min":0.94099999999999995},"min":{"1min":0.64200000000000002,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.2989999999999999,"5min":1.329,"15min":1.653},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.98099999999999998,"15min":0.95999999999999996},"min":{"1min":0.68999999999999995,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":2.5760000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.0489999999999999,"15min":1.0309999999999999},"min":{"1min":0.65400000000000003,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.274,"5min":1.486,"15min":6.7729999999999997},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.93200000000000005},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.581},"last":0.62},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0700000000000001,"15min":1.083},"min":{"1min":0.64900000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.319,"5min":1.5760000000000001,"15min":15.352},"last":0.80800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13354,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181957,"statfs":{"total":200000000,"available":186324963,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.88100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.439},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.996,"5min":1.089,"15min":1.157},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.752},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.27700000000000002},"max":{"1min":1.1579999999999999,"5min":1.492,"15min":1.8169999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.42399999999999999,"5min":0.32100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.7949999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.298},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.238,"15min":1.5780000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.67000000000000004,"5min":0.57799999999999996,"15min":0.52100000000000002},"max":{"1min":1.292,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96099999999999997,"15min":0.96199999999999997},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.52700000000000002},"max":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.492},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63800000000000001,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.1919999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":0.748}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97899999999999998,"15min":1.012},"min":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.56100000000000005},"max":{"1min":1.202,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0109999999999999,"15min":0.98099999999999998},"min":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.393,"15min":1.837},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.046,"15min":1.0289999999999999},"min":{"1min":0.69699999999999995,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.429,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.097},{"interface":"front","average":{"1min":1.028,"5min":1.046,"15min":1.0740000000000001},"min":{"1min":0.62,"5min":0.60899999999999999,"15min":0.56599999999999995},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":5.2430000000000003},"last":1.024}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9835,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185477,"statfs":{"total":200000000,"available":189928933,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.92400000000000004,"5min":0.96599999999999997,"15min":1.0349999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.091,"5min":1.2270000000000001,"15min":2.8969999999999998},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.73399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.28399999999999997},"max":{"1min":1.1699999999999999,"5min":1.2010000000000001,"15min":2.7530000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.26300000000000001},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.409},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.73799999999999999},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.125,"5min":1.2629999999999999,"15min":2.6509999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90500000000000003,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.2749999999999999,"5min":1.395,"15min":1.395},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.94299999999999995},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1950000000000001,"5min":1.2769999999999999,"15min":2.8290000000000002},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93400000000000005,"15min":0.92800000000000005},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.7480000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.95999999999999996,"15min":0.93200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.6819999999999999},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.014,"15min":1.0269999999999999},"min":{"1min":0.69599999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":1.353,"15min":6.4690000000000003},"last":1.004}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99199999999999999,"15min":0.996},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":1.417,"15min":1.5660000000000001},"last":0.89700000000000002}]}]}],"pool_statfs":[]}} 2023-10-26T05:59:12.644 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:59:12.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 -- 172.21.15.71:0/1163403344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 msgr2=0x7fae3400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.757 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 --2- 172.21.15.71:0/1163403344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae3400b990 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fae44006fd0 tx=0x7fae4401ca00 comp rx=0 tx=0).stop 2023-10-26T05:59:12.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 -- 172.21.15.71:0/1163403344 shutdown_connections 2023-10-26T05:59:12.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 --2- 172.21.15.71:0/1163403344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae3400b990 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 -- 172.21.15.71:0/1163403344 >> 172.21.15.71:0/1163403344 conn(0x7fae4c069770 msgr2=0x7fae4c069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:12.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 -- 172.21.15.71:0/1163403344 shutdown_connections 2023-10-26T05:59:12.758 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.747+0000 7fae50b58640 1 -- 172.21.15.71:0/1163403344 wait complete. 2023-10-26T05:59:12.760 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 Processor -- start 2023-10-26T05:59:12.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- start start 2023-10-26T05:59:12.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae4c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae4c138c10 0x7fae4c139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c136270 0x7fae4c136720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.761 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c136c60 con 0x7fae34005190 2023-10-26T05:59:12.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c13a7b0 con 0x7fae4c136270 2023-10-26T05:59:12.762 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c139710 con 0x7fae4c138c10 2023-10-26T05:59:12.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae4c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae4c0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38882/0 (socket says 172.21.15.71:38882) 2023-10-26T05:59:12.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 -- 172.21.15.71:0/2259726485 learned_addr learned my addr 172.21.15.71:0/2259726485 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:12.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 -- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae4c138c10 msgr2=0x7fae4c139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.763 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae4c138c10 0x7fae4c139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4ad9d640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c136270 0x7fae4c136720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 -- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c136270 msgr2=0x7fae4c136720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c136270 0x7fae4c136720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 -- 172.21.15.71:0/2259726485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c1399f0 con 0x7fae34005190 2023-10-26T05:59:12.764 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae4a59c640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae4c0adf00 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00d780 tx=0x7fae3c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:12.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00de90 con 0x7fae34005190 2023-10-26T05:59:12.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fae3c0052e0 con 0x7fae34005190 2023-10-26T05:59:12.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c139cd0 con 0x7fae34005190 2023-10-26T05:59:12.765 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00b8c0 con 0x7fae34005190 2023-10-26T05:59:12.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c13a290 con 0x7fae34005190 2023-10-26T05:59:12.766 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c066b10 con 0x7fae34005190 2023-10-26T05:59:12.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fae3c00ba20 con 0x7fae34005190 2023-10-26T05:59:12.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae3affd640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae2c066470 0x7fae2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:12.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae3c083830 con 0x7fae34005190 2023-10-26T05:59:12.767 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.751+0000 7fae49d9b640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae2c066470 0x7fae2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:12.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.755+0000 7fae49d9b640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae2c066470 0x7fae2c068930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fae40005fd0 tx=0x7fae40005ca0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:12.768 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.755+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fae3c050fb0 con 0x7fae34005190 2023-10-26T05:59:12.904 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.895+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae4c1412b0 con 0x7fae2c066470 2023-10-26T05:59:12.910 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae3affd640 1 -- 172.21.15.71:0/2259726485 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48652 (secure 0 0 0) 0x7fae40009550 con 0x7fae2c066470 2023-10-26T05:59:12.911 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:12.911 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:59:12.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae2c066470 msgr2=0x7fae2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.912 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae2c066470 0x7fae2c068930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fae40005fd0 tx=0x7fae40005ca0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 msgr2=0x7fae4c0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:12.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae4c0adf00 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00d780 tx=0x7fae3c00dc40 comp rx=0 tx=0).stop 2023-10-26T05:59:12.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 shutdown_connections 2023-10-26T05:59:12.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae4c136270 0x7fae4c136720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.913 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fae4c138c10 0x7fae4c139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fae2c066470 0x7fae2c068930 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 --2- 172.21.15.71:0/2259726485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae34005190 0x7fae4c0adf00 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:12.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 >> 172.21.15.71:0/2259726485 conn(0x7fae4c069770 msgr2=0x7fae4c0ad4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:12.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 shutdown_connections 2023-10-26T05:59:12.914 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:12.903+0000 7fae50b58640 1 -- 172.21.15.71:0/2259726485 wait complete. 2023-10-26T05:59:12.928 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2023-10-26T05:59:11.652045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450300,"statfs":{"total":1600000000,"available":1485108092,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52487222,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-121,"num_write_kb":-85082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.321452"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":325,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-26T05:59:07.333954+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:59:07.333954+0000","last_peered":"2023-10-26T05:59:07.333954+0000","last_clean":"2023-10-26T05:59:07.333954+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:59:07.333954+0000","last_undegraded":"2023-10-26T05:59:07.333954+0000","last_fullsized":"2023-10-26T05:59:07.333954+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185420,"statfs":{"total":200000000,"available":189870769,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.39600000000000002},"max":{"1min":1.2290000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.36399999999999999},"max":{"1min":1.125,"5min":1.196,"15min":1.3100000000000001},"last":0.62}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.98599999999999999,"15min":0.997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":1.387,"15min":1.4319999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0609999999999999,"15min":1.0009999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.621,"15min":1.621},"last":1.0580000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":0.85299999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.40799999999999997},"max":{"1min":1.3,"5min":1.448,"15min":1.448},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":1.2190000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.51500000000000001},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.4330000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":0.97599999999999998},"min":{"1min":0.76200000000000001,"5min":0.61599999999999999,"15min":0.52100000000000002},"max":{"1min":1.333,"5min":1.712,"15min":1.712},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.59599999999999997,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.2190000000000001,"5min":1.2589999999999999,"15min":1.639},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.70599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.4159999999999999},"last":1.258}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.999,"15min":1.0589999999999999},"min":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.335,"5min":1.482,"15min":1.6679999999999999},"last":1.145},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.03,"15min":1.0529999999999999},"min":{"1min":0.66700000000000004,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.383,"5min":1.504,"15min":1.702},"last":1.1870000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.087,"15min":1.099},"min":{"1min":0.86199999999999999,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.302,"5min":1.3640000000000001,"15min":1.7509999999999999},"last":1.298},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.069,"15min":1.0549999999999999},"min":{"1min":0.77100000000000002,"5min":0.66700000000000004,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.5,"15min":1.6180000000000001},"last":1.375}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19498,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175813,"statfs":{"total":200000000,"available":180033317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.84099999999999997},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90000000000000002,"15min":0.872},"min":{"1min":0.503,"5min":0.49299999999999999,"15min":0.376},"max":{"1min":1.286,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.85199999999999998,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.39100000000000001},"max":{"1min":1.071,"5min":1.1120000000000001,"15min":1.2609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.91700000000000004,"15min":0.874},"min":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.41999999999999998},"max":{"1min":1.157,"5min":1.3899999999999999,"15min":1.405},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89800000000000002,"15min":0.85699999999999998},"min":{"1min":0.66300000000000003,"5min":0.54800000000000004,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.268,"15min":1.8460000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.86499999999999999},"min":{"1min":0.65100000000000002,"5min":0.502,"15min":0.435},"max":{"1min":1.169,"5min":1.3,"15min":1.9239999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.028,"15min":0.94999999999999996},"min":{"1min":0.73799999999999999,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.073,"5min":1.0609999999999999,"15min":1.0149999999999999},"min":{"1min":0.79800000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.171}]},{"osd":4,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.502},"max":{"1min":1.4299999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98499999999999999},"min":{"1min":0.68200000000000005,"5min":0.621,"15min":0.442},"max":{"1min":1.375,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.075,"15min":1.0509999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.45600000000000002},"max":{"1min":1.4099999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.022,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.36799999999999999},"max":{"1min":1.3260000000000001,"5min":1.452,"15min":1.512},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.147,"15min":1.1180000000000001},"min":{"1min":1.0029999999999999,"5min":0.71599999999999997,"15min":0.41099999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.4550000000000001,"15min":1.5920000000000001},"last":1.099}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173765,"statfs":{"total":200000000,"available":177935785,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96499999999999997,"15min":0.95499999999999996},"min":{"1min":0.48699999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.264,"5min":1.381,"15min":1.579},"last":1.212},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.012,"15min":0.96899999999999997},"min":{"1min":0.68100000000000005,"5min":0.58799999999999997,"15min":0.499},"max":{"1min":1.1499999999999999,"5min":1.591,"15min":1.591},"last":1.1200000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.872},"min":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.214,"5min":1.214,"15min":2.1150000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.86499999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.0740000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93600000000000005,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.21,"5min":1.569,"15min":1.569},"last":1.248},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89900000000000002,"15min":0.92000000000000004},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2569999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":0.85499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.363,"15min":1.548},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.335,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93100000000000005,"15min":0.96599999999999997},"min":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.46999999999999997},"max":{"1min":1.373,"5min":1.373,"15min":2.1909999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1120000000000001,"15min":1.0940000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40799999999999997},"max":{"1min":1.4099999999999999,"5min":1.4950000000000001,"15min":1.536},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.02,"15min":1.0049999999999999},"min":{"1min":0.60199999999999998,"5min":0.50700000000000001,"15min":0.41899999999999998},"max":{"1min":1.2869999999999999,"5min":1.744,"15min":1.744},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.056,"15min":1.032},"min":{"1min":0.71299999999999997,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.3160000000000001,"5min":1.4199999999999999,"15min":1.4950000000000001},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0980000000000001,"15min":1.075},"min":{"1min":0.754,"5min":0.53600000000000003,"15min":0.442},"max":{"1min":1.4810000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.288},{"interface":"front","average":{"1min":1.153,"5min":1.1799999999999999,"15min":1.155},"min":{"1min":0.86099999999999999,"5min":0.61899999999999999,"15min":0.49299999999999999},"max":{"1min":1.3480000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.3300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188045,"statfs":{"total":200000000,"available":192558961,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.2390000000000001,"5min":1.248,"15min":1.347},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.47199999999999998},"max":{"1min":1.1299999999999999,"5min":1.234,"15min":1.502},"last":0.90800000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.85099999999999998},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.375},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.95299999999999996,"15min":0.92400000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.1839999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93200000000000005,"15min":0.86499999999999999},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.65300000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.149,"5min":1.222,"15min":1.373},"last":0.94799999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.96999999999999997,"15min":0.94799999999999995},"min":{"1min":0.63300000000000001,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.2649999999999999,"5min":1.397,"15min":2.0190000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.02,"15min":0.97299999999999998},"min":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.58899999999999997},"max":{"1min":1.2030000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.0549999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2450000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.022,"15min":1.0049999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.673},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.59099999999999997,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.3759999999999999,"5min":1.762,"15min":1.7969999999999999},"last":1.014},{"interface":"front","average":{"1min":1.069,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.70699999999999996,"5min":0.67100000000000004,"15min":0.51600000000000001},"max":{"1min":1.3120000000000001,"5min":1.355,"15min":1.4970000000000001},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.117,"15min":1.0669999999999999},"min":{"1min":0.81299999999999994,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.4610000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1499999999999999,"15min":1.1279999999999999},"min":{"1min":0.76000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.484,"15min":1.728},"last":1.2310000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9258,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186053,"statfs":{"total":200000000,"available":190519065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.999,"5min":0.999,"15min":1.385},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.2689999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68400000000000005,"15min":0.65800000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.93700000000000006,"5min":6.109,"15min":6.109},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.63500000000000001},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83799999999999997,"5min":1.02,"15min":1.1399999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.77500000000000002},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88900000000000001,"15min":0.81699999999999995},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.125,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.40000000000000002},"max":{"1min":1.026,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.52400000000000002},"max":{"1min":1.1830000000000001,"5min":1.429,"15min":1.4810000000000001},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.95499999999999996},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.2130000000000001,"5min":1.454,"15min":1.478},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94599999999999995,"15min":0.92800000000000005},"min":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.1619999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.876}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":0.98899999999999999},"min":{"1min":0.65800000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.002,"15min":1.0209999999999999},"min":{"1min":0.57599999999999996,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":1.286,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.95599999999999996},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.2410000000000001,"5min":2.016,"15min":2.016},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0680000000000001,"15min":1.046},"min":{"1min":0.70799999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3120000000000001,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.98099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173765,"statfs":{"total":200000000,"available":177936299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.628,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":1.097},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26000000000000001},"max":{"1min":1.006,"5min":1.1619999999999999,"15min":1.5249999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.70699999999999996},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.0669999999999999,"5min":1.198,"15min":1.2609999999999999},"last":0.436},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.30199999999999999,"15min":0.26700000000000002},"max":{"1min":0.96599999999999997,"5min":1.157,"15min":1.3600000000000001},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81200000000000006,"15min":0.77600000000000002},"min":{"1min":0.373,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.008,"5min":1.24,"15min":2.8809999999999998},"last":0.308},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.3440000000000001},"last":0.37}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90500000000000003,"15min":0.876},"min":{"1min":0.625,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.149,"5min":1.177,"15min":2.6139999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.73799999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1459999999999999,"15min":5.9550000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2090000000000001,"5min":1.5289999999999999,"15min":2.8319999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.96099999999999997,"15min":0.94099999999999995},"min":{"1min":0.64200000000000002,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.2989999999999999,"5min":1.329,"15min":1.653},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.98099999999999998,"15min":0.95999999999999996},"min":{"1min":0.68999999999999995,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":2.5760000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.0489999999999999,"15min":1.0309999999999999},"min":{"1min":0.65400000000000003,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.274,"5min":1.486,"15min":6.7729999999999997},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.93200000000000005},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.581},"last":0.62},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0700000000000001,"15min":1.083},"min":{"1min":0.64900000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.319,"5min":1.5760000000000001,"15min":15.352},"last":0.80800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13354,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181957,"statfs":{"total":200000000,"available":186324963,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.88100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.439},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.996,"5min":1.089,"15min":1.157},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.752},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.27700000000000002},"max":{"1min":1.1579999999999999,"5min":1.492,"15min":1.8169999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.42399999999999999,"5min":0.32100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.7949999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.298},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.238,"15min":1.5780000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.67000000000000004,"5min":0.57799999999999996,"15min":0.52100000000000002},"max":{"1min":1.292,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96099999999999997,"15min":0.96199999999999997},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.52700000000000002},"max":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.492},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63800000000000001,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.1919999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":0.748}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97899999999999998,"15min":1.012},"min":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.56100000000000005},"max":{"1min":1.202,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0109999999999999,"15min":0.98099999999999998},"min":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.393,"15min":1.837},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.046,"15min":1.0289999999999999},"min":{"1min":0.69699999999999995,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.429,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.097},{"interface":"front","average":{"1min":1.028,"5min":1.046,"15min":1.0740000000000001},"min":{"1min":0.62,"5min":0.60899999999999999,"15min":0.56599999999999995},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":5.2430000000000003},"last":1.024}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9835,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185477,"statfs":{"total":200000000,"available":189928933,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.92400000000000004,"5min":0.96599999999999997,"15min":1.0349999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.091,"5min":1.2270000000000001,"15min":2.8969999999999998},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.73399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.28399999999999997},"max":{"1min":1.1699999999999999,"5min":1.2010000000000001,"15min":2.7530000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.26300000000000001},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.409},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.73799999999999999},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.125,"5min":1.2629999999999999,"15min":2.6509999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90500000000000003,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.2749999999999999,"5min":1.395,"15min":1.395},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.94299999999999995},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1950000000000001,"5min":1.2769999999999999,"15min":2.8290000000000002},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93400000000000005,"15min":0.92800000000000005},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.7480000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.95999999999999996,"15min":0.93200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.6819999999999999},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.014,"15min":1.0269999999999999},"min":{"1min":0.69599999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":1.353,"15min":6.4690000000000003},"last":1.004}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99199999999999999,"15min":0.996},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":1.417,"15min":1.5660000000000001},"last":0.89700000000000002}]}]}],"pool_statfs":[]}} 2023-10-26T05:59:12.929 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-26T05:59:12.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:59:13.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 -- 172.21.15.71:0/2239368845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0d0470 msgr2=0x7f88fc06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 --2- 172.21.15.71:0/2239368845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0d0470 0x7f88fc06aa70 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0c4dd0 tx=0x7f88ec01c810 comp rx=0 tx=0).stop 2023-10-26T05:59:13.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 -- 172.21.15.71:0/2239368845 shutdown_connections 2023-10-26T05:59:13.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 --2- 172.21.15.71:0/2239368845 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88fc0cd880 0x7f88fc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 --2- 172.21.15.71:0/2239368845 >> v2:172.21.15.86:3300/0 conn(0x7f88fc0d0850 0x7f88fc15b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 --2- 172.21.15.71:0/2239368845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0d0470 0x7f88fc06aa70 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 -- 172.21.15.71:0/2239368845 >> 172.21.15.71:0/2239368845 conn(0x7f88fc0cdd80 msgr2=0x7f88fc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:13.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 -- 172.21.15.71:0/2239368845 shutdown_connections 2023-10-26T05:59:13.055 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.043+0000 7f890243f640 1 -- 172.21.15.71:0/2239368845 wait complete. 2023-10-26T05:59:13.057 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 Processor -- start 2023-10-26T05:59:13.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- start start 2023-10-26T05:59:13.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 0x7f88fc0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88fc0d0470 0x7f88fc0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88fc0d0850 0x7f88fc0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.058 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc1546d0 con 0x7f88fc0cd880 2023-10-26T05:59:13.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0cadc0 con 0x7f88fc0d0470 2023-10-26T05:59:13.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0cb0a0 con 0x7f88fc0d0850 2023-10-26T05:59:13.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 0x7f88fc0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 0x7f88fc0c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38900/0 (socket says 172.21.15.71:38900) 2023-10-26T05:59:13.059 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 -- 172.21.15.71:0/2656631768 learned_addr learned my addr 172.21.15.71:0/2656631768 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:13.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 -- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88fc0d0850 msgr2=0x7f88fc0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88fc0d0850 0x7f88fc0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 -- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88fc0d0470 msgr2=0x7f88fc0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fb7fe640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88fc0d0470 0x7f88fc0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88fc0d0470 0x7f88fc0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.060 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 -- 172.21.15.71:0/2656631768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec009560 con 0x7f88fc0cd880 2023-10-26T05:59:13.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88fbfff640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 0x7f88fc0c68d0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0b0040 tx=0x7f88ec0a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:13.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec005440 con 0x7f88fc0cd880 2023-10-26T05:59:13.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f88ec0046d0 con 0x7f88fc0cd880 2023-10-26T05:59:13.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc0cb380 con 0x7f88fc0cd880 2023-10-26T05:59:13.061 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec004ae0 con 0x7f88fc0cd880 2023-10-26T05:59:13.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc0c6e10 con 0x7f88fc0cd880 2023-10-26T05:59:13.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc066b10 con 0x7f88fc0cd880 2023-10-26T05:59:13.062 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.047+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f88ec0b4070 con 0x7f88fc0cd880 2023-10-26T05:59:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.051+0000 7f88f8ff9640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88e8066470 0x7f88e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.051+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f88ec11e1f0 con 0x7f88fc0cd880 2023-10-26T05:59:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.051+0000 7f88fb7fe640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88e8066470 0x7f88e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.063 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.051+0000 7f88fb7fe640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88e8066470 0x7f88e8068930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f88fc15a5a0 tx=0x7f88f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:13.064 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.055+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f88ec0eb980 con 0x7f88fc0cd880 2023-10-26T05:59:13.214 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.203+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88fc0cb950 con 0x7f88e8066470 2023-10-26T05:59:13.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f88f8ff9640 1 -- 172.21.15.71:0/2656631768 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48652 (secure 0 0 0) 0x7f88f000a430 con 0x7f88e8066470 2023-10-26T05:59:13.221 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:13.221 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:59:13.222 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88e8066470 msgr2=0x7f88e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88e8066470 0x7f88e8068930 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f88fc15a5a0 tx=0x7f88f0008040 comp rx=0 tx=0).stop 2023-10-26T05:59:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 msgr2=0x7f88fc0c68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 0x7f88fc0c68d0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0b0040 tx=0x7f88ec0a5870 comp rx=0 tx=0).stop 2023-10-26T05:59:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 shutdown_connections 2023-10-26T05:59:13.223 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f88fc0d0850 0x7f88fc0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88fc0d0470 0x7f88fc0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f88e8066470 0x7f88e8068930 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 --2- 172.21.15.71:0/2656631768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88fc0cd880 0x7f88fc0c68d0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 >> 172.21.15.71:0/2656631768 conn(0x7f88fc0cdd80 msgr2=0x7f88fc0b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:13.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 shutdown_connections 2023-10-26T05:59:13.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.215+0000 7f890243f640 1 -- 172.21.15.71:0/2656631768 wait complete. 2023-10-26T05:59:13.240 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2023-10-26T05:59:11.652045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112199,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450300,"statfs":{"total":1600000000,"available":1485108092,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52487222,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-121,"num_write_kb":-85082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.321452"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":325,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-26T05:59:07.333954+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:59:07.333954+0000","last_peered":"2023-10-26T05:59:07.333954+0000","last_clean":"2023-10-26T05:59:07.333954+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:59:07.333954+0000","last_undegraded":"2023-10-26T05:59:07.333954+0000","last_fullsized":"2023-10-26T05:59:07.333954+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9891,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185420,"statfs":{"total":200000000,"available":189870769,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.39600000000000002},"max":{"1min":1.2290000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.36399999999999999},"max":{"1min":1.125,"5min":1.196,"15min":1.3100000000000001},"last":0.62}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.98599999999999999,"15min":0.997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":1.387,"15min":1.4319999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.002,"5min":1.0609999999999999,"15min":1.0009999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.621,"15min":1.621},"last":1.0580000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":0.85299999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.40799999999999997},"max":{"1min":1.3,"5min":1.448,"15min":1.448},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":1.2190000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.51500000000000001},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.4330000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":0.97599999999999998},"min":{"1min":0.76200000000000001,"5min":0.61599999999999999,"15min":0.52100000000000002},"max":{"1min":1.333,"5min":1.712,"15min":1.712},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.59599999999999997,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.2190000000000001,"5min":1.2589999999999999,"15min":1.639},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.70599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.4159999999999999},"last":1.258}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.999,"15min":1.0589999999999999},"min":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.335,"5min":1.482,"15min":1.6679999999999999},"last":1.145},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.03,"15min":1.0529999999999999},"min":{"1min":0.66700000000000004,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.383,"5min":1.504,"15min":1.702},"last":1.1870000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.087,"15min":1.099},"min":{"1min":0.86199999999999999,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.302,"5min":1.3640000000000001,"15min":1.7509999999999999},"last":1.298},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.069,"15min":1.0549999999999999},"min":{"1min":0.77100000000000002,"5min":0.66700000000000004,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.5,"15min":1.6180000000000001},"last":1.375}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19498,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175813,"statfs":{"total":200000000,"available":180033317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.84099999999999997},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90000000000000002,"15min":0.872},"min":{"1min":0.503,"5min":0.49299999999999999,"15min":0.376},"max":{"1min":1.286,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.85199999999999998,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.39100000000000001},"max":{"1min":1.071,"5min":1.1120000000000001,"15min":1.2609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.91700000000000004,"15min":0.874},"min":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.41999999999999998},"max":{"1min":1.157,"5min":1.3899999999999999,"15min":1.405},"last":1.0209999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89800000000000002,"15min":0.85699999999999998},"min":{"1min":0.66300000000000003,"5min":0.54800000000000004,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.268,"15min":1.8460000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.86499999999999999},"min":{"1min":0.65100000000000002,"5min":0.502,"15min":0.435},"max":{"1min":1.169,"5min":1.3,"15min":1.9239999999999999},"last":0.97899999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.028,"15min":0.94999999999999996},"min":{"1min":0.73799999999999999,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.073,"5min":1.0609999999999999,"15min":1.0149999999999999},"min":{"1min":0.79800000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.171}]},{"osd":4,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.502},"max":{"1min":1.4299999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98499999999999999},"min":{"1min":0.68200000000000005,"5min":0.621,"15min":0.442},"max":{"1min":1.375,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.94199999999999995}]},{"osd":5,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.075,"15min":1.0509999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.45600000000000002},"max":{"1min":1.4099999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.022,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.36799999999999999},"max":{"1min":1.3260000000000001,"5min":1.452,"15min":1.512},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.147,"15min":1.1180000000000001},"min":{"1min":1.0029999999999999,"5min":0.71599999999999997,"15min":0.41099999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.4550000000000001,"15min":1.5920000000000001},"last":1.099}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21547,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173765,"statfs":{"total":200000000,"available":177935785,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96499999999999997,"15min":0.95499999999999996},"min":{"1min":0.48699999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.264,"5min":1.381,"15min":1.579},"last":1.212},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.012,"15min":0.96899999999999997},"min":{"1min":0.68100000000000005,"5min":0.58799999999999997,"15min":0.499},"max":{"1min":1.1499999999999999,"5min":1.591,"15min":1.591},"last":1.1200000000000001}]},{"osd":1,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.872},"min":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.214,"5min":1.214,"15min":2.1150000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.86499999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.0740000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93600000000000005,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.21,"5min":1.569,"15min":1.569},"last":1.248},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89900000000000002,"15min":0.92000000000000004},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2569999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":0.85499999999999998}]},{"osd":3,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.363,"15min":1.548},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.335,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93100000000000005,"15min":0.96599999999999997},"min":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.46999999999999997},"max":{"1min":1.373,"5min":1.373,"15min":2.1909999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1120000000000001,"15min":1.0940000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40799999999999997},"max":{"1min":1.4099999999999999,"5min":1.4950000000000001,"15min":1.536},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.02,"15min":1.0049999999999999},"min":{"1min":0.60199999999999998,"5min":0.50700000000000001,"15min":0.41899999999999998},"max":{"1min":1.2869999999999999,"5min":1.744,"15min":1.744},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.056,"15min":1.032},"min":{"1min":0.71299999999999997,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.3160000000000001,"5min":1.4199999999999999,"15min":1.4950000000000001},"last":1.028}]},{"osd":7,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0980000000000001,"15min":1.075},"min":{"1min":0.754,"5min":0.53600000000000003,"15min":0.442},"max":{"1min":1.4810000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.288},{"interface":"front","average":{"1min":1.153,"5min":1.1799999999999999,"15min":1.155},"min":{"1min":0.86099999999999999,"5min":0.61899999999999999,"15min":0.49299999999999999},"max":{"1min":1.3480000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.3300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7266,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188045,"statfs":{"total":200000000,"available":192558961,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.2390000000000001,"5min":1.248,"15min":1.347},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.47199999999999998},"max":{"1min":1.1299999999999999,"5min":1.234,"15min":1.502},"last":0.90800000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.85099999999999998},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.375},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.95299999999999996,"15min":0.92400000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.1839999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93200000000000005,"15min":0.86499999999999999},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.65300000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.149,"5min":1.222,"15min":1.373},"last":0.94799999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.96999999999999997,"15min":0.94799999999999995},"min":{"1min":0.63300000000000001,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.2649999999999999,"5min":1.397,"15min":2.0190000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.02,"15min":0.97299999999999998},"min":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.58899999999999997},"max":{"1min":1.2030000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.0549999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2450000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.091,"5min":1.022,"15min":1.0049999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.673},"last":1.1699999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.59099999999999997,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.3759999999999999,"5min":1.762,"15min":1.7969999999999999},"last":1.014},{"interface":"front","average":{"1min":1.069,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.70699999999999996,"5min":0.67100000000000004,"15min":0.51600000000000001},"max":{"1min":1.3120000000000001,"5min":1.355,"15min":1.4970000000000001},"last":1.1459999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.117,"15min":1.0669999999999999},"min":{"1min":0.81299999999999994,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.4610000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1499999999999999,"15min":1.1279999999999999},"min":{"1min":0.76000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.484,"15min":1.728},"last":1.2310000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9258,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186053,"statfs":{"total":200000000,"available":190519065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.999,"5min":0.999,"15min":1.385},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.2689999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68400000000000005,"15min":0.65800000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.93700000000000006,"5min":6.109,"15min":6.109},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.63500000000000001},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83799999999999997,"5min":1.02,"15min":1.1399999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.77500000000000002},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88900000000000001,"15min":0.81699999999999995},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.125,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.40000000000000002},"max":{"1min":1.026,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.52400000000000002},"max":{"1min":1.1830000000000001,"5min":1.429,"15min":1.4810000000000001},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.95499999999999996},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.2130000000000001,"5min":1.454,"15min":1.478},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94599999999999995,"15min":0.92800000000000005},"min":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.1619999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.876}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":0.98899999999999999},"min":{"1min":0.65800000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.024,"5min":1.002,"15min":1.0209999999999999},"min":{"1min":0.57599999999999996,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":1.286,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.95599999999999996},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.2410000000000001,"5min":2.016,"15min":2.016},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0680000000000001,"15min":1.046},"min":{"1min":0.70799999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3120000000000001,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.98099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21546,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173765,"statfs":{"total":200000000,"available":177936299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.628,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":1.097},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26000000000000001},"max":{"1min":1.006,"5min":1.1619999999999999,"15min":1.5249999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.70699999999999996},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.0669999999999999,"5min":1.198,"15min":1.2609999999999999},"last":0.436},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.30199999999999999,"15min":0.26700000000000002},"max":{"1min":0.96599999999999997,"5min":1.157,"15min":1.3600000000000001},"last":0.23899999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81200000000000006,"15min":0.77600000000000002},"min":{"1min":0.373,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.008,"5min":1.24,"15min":2.8809999999999998},"last":0.308},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.3440000000000001},"last":0.37}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90500000000000003,"15min":0.876},"min":{"1min":0.625,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.149,"5min":1.177,"15min":2.6139999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.73799999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1459999999999999,"15min":5.9550000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2090000000000001,"5min":1.5289999999999999,"15min":2.8319999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.96099999999999997,"15min":0.94099999999999995},"min":{"1min":0.64200000000000002,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.2989999999999999,"5min":1.329,"15min":1.653},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.98099999999999998,"15min":0.95999999999999996},"min":{"1min":0.68999999999999995,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":2.5760000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.0489999999999999,"15min":1.0309999999999999},"min":{"1min":0.65400000000000003,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.274,"5min":1.486,"15min":6.7729999999999997},"last":0.83399999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.93200000000000005},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.581},"last":0.62},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0700000000000001,"15min":1.083},"min":{"1min":0.64900000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.319,"5min":1.5760000000000001,"15min":15.352},"last":0.80800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13354,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181957,"statfs":{"total":200000000,"available":186324963,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.88100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.439},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.996,"5min":1.089,"15min":1.157},"last":0.47099999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.752},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.27700000000000002},"max":{"1min":1.1579999999999999,"5min":1.492,"15min":1.8169999999999999},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.42399999999999999,"5min":0.32100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.7949999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.298},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.238,"15min":1.5780000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.67000000000000004,"5min":0.57799999999999996,"15min":0.52100000000000002},"max":{"1min":1.292,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":0.76800000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96099999999999997,"15min":0.96199999999999997},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.52700000000000002},"max":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.492},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63800000000000001,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.1919999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":0.748}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97899999999999998,"15min":1.012},"min":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.56100000000000005},"max":{"1min":1.202,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0109999999999999,"15min":0.98099999999999998},"min":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.393,"15min":1.837},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.046,"15min":1.0289999999999999},"min":{"1min":0.69699999999999995,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.429,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.097},{"interface":"front","average":{"1min":1.028,"5min":1.046,"15min":1.0740000000000001},"min":{"1min":0.62,"5min":0.60899999999999999,"15min":0.56599999999999995},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":5.2430000000000003},"last":1.024}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9835,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185477,"statfs":{"total":200000000,"available":189928933,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.92400000000000004,"5min":0.96599999999999997,"15min":1.0349999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.091,"5min":1.2270000000000001,"15min":2.8969999999999998},"last":0.71899999999999997}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.73399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.28399999999999997},"max":{"1min":1.1699999999999999,"5min":1.2010000000000001,"15min":2.7530000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.26300000000000001},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.76300000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.409},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.73799999999999999},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.125,"5min":1.2629999999999999,"15min":2.6509999999999998},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90500000000000003,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.2749999999999999,"5min":1.395,"15min":1.395},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.94299999999999995},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1950000000000001,"5min":1.2769999999999999,"15min":2.8290000000000002},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93400000000000005,"15min":0.92800000000000005},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.7480000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.95999999999999996,"15min":0.93200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.6819999999999999},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.014,"15min":1.0269999999999999},"min":{"1min":0.69599999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":1.353,"15min":6.4690000000000003},"last":1.004}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99199999999999999,"15min":0.996},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":1.417,"15min":1.5660000000000001},"last":0.89700000000000002}]}]}],"pool_statfs":[]}} 2023-10-26T05:59:13.242 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-26T05:59:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- 172.21.15.71:0/1558348860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e8004d10 msgr2=0x7f17e8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 --2- 172.21.15.71:0/1558348860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17e8004d10 0x7f17e8005190 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c610 tx=0x7f17f401d360 comp rx=0 tx=0).stop 2023-10-26T05:59:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f1800c73640 1 -- 172.21.15.71:0/1558348860 reap_dead start 2023-10-26T05:59:13.364 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- 172.21.15.71:0/1558348860 shutdown_connections 2023-10-26T05:59:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- 172.21.15.71:0/1558348860 >> 172.21.15.71:0/1558348860 conn(0x7f17fc069770 msgr2=0x7f17fc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- 172.21.15.71:0/1558348860 shutdown_connections 2023-10-26T05:59:13.365 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- 172.21.15.71:0/1558348860 wait complete. 2023-10-26T05:59:13.367 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 Processor -- start 2023-10-26T05:59:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- start start 2023-10-26T05:59:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17e8004d10 0x7f17fc135980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 0x7f17fc0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17fc142670 0x7f17fc0ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.368 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc139300 con 0x7f17fc1421f0 2023-10-26T05:59:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc1395e0 con 0x7f17fc142670 2023-10-26T05:59:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f18026d6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc1398c0 con 0x7f17e8004d10 2023-10-26T05:59:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f17f37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 0x7f17fc0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f17f37fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 0x7f17fc0acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38912/0 (socket says 172.21.15.71:38912) 2023-10-26T05:59:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f17f37fe640 1 -- 172.21.15.71:0/2450073078 learned_addr learned my addr 172.21.15.71:0/2450073078 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.355+0000 7f1800c73640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17fc142670 0x7f17fc0ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17fbfff640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17e8004d10 0x7f17fc135980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f37fe640 1 -- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17e8004d10 msgr2=0x7f17fc135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f37fe640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17e8004d10 0x7f17fc135980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f37fe640 1 -- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17fc142670 msgr2=0x7f17fc0ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f37fe640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17fc142670 0x7f17fc0ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f37fe640 1 -- 172.21.15.71:0/2450073078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc139ba0 con 0x7f17fc1421f0 2023-10-26T05:59:13.371 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f37fe640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 0x7f17fc0acb70 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f17e4011a50 tx=0x7f17e4011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e400fd90 con 0x7f17fc1421f0 2023-10-26T05:59:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f17e4012450 con 0x7f17fc1421f0 2023-10-26T05:59:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4012800 con 0x7f17fc1421f0 2023-10-26T05:59:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc139e80 con 0x7f17fc1421f0 2023-10-26T05:59:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc13a3a0 con 0x7f17fc1421f0 2023-10-26T05:59:13.372 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc066b10 con 0x7f17fc1421f0 2023-10-26T05:59:13.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f17e4012960 con 0x7f17fc1421f0 2023-10-26T05:59:13.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f97fa640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17dc066470 0x7f17dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:13.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f17e4084b20 con 0x7f17fc1421f0 2023-10-26T05:59:13.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17fbfff640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17dc066470 0x7f17dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:13.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.359+0000 7f17fbfff640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17dc066470 0x7f17dc068930 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0058d0 tx=0x7f17ec005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:13.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.363+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f17e4055de0 con 0x7f17fc1421f0 2023-10-26T05:59:13.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.523+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17fc0721e0 con 0x7f17fc1421f0 2023-10-26T05:59:13.533 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.523+0000 7f17f97fa640 1 -- 172.21.15.71:0/2450073078 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+12911 (secure 0 0 0) 0x7f17e401f020 con 0x7f17fc1421f0 2023-10-26T05:59:13.533 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:13.534 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":195,"fsid":"e2c462c9-9202-452f-a70c-a4a81693dc19","created":"2023-10-26T05:47:02.367858+0000","modified":"2023-10-26T05:59:10.520177+0000","last_up_change":"2023-10-26T05:47:09.277950+0000","last_in_change":"2023-10-26T05:47:06.255185+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-26T05:47:09.468386+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6809","nonce":4161483918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6812","nonce":4161483918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6819","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6823","nonce":4161483918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6815","nonce":4161483918},{"type":"v1","addr":"172.21.15.71:6817","nonce":4161483918}]},"public_addr":"172.21.15.71:6809/4161483918","cluster_addr":"172.21.15.71:6812/4161483918","heartbeat_back_addr":"172.21.15.71:6823/4161483918","heartbeat_front_addr":"172.21.15.71:6817/4161483918","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6801","nonce":619557138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6803","nonce":619557138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6807","nonce":619557138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":619557138},{"type":"v1","addr":"172.21.15.71:6805","nonce":619557138}]},"public_addr":"172.21.15.71:6801/619557138","cluster_addr":"172.21.15.71:6803/619557138","heartbeat_back_addr":"172.21.15.71:6807/619557138","heartbeat_front_addr":"172.21.15.71:6805/619557138","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6825","nonce":2687547983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6827","nonce":2687547983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6831","nonce":2687547983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":2687547983},{"type":"v1","addr":"172.21.15.71:6829","nonce":2687547983}]},"public_addr":"172.21.15.71:6825/2687547983","cluster_addr":"172.21.15.71:6827/2687547983","heartbeat_back_addr":"172.21.15.71:6831/2687547983","heartbeat_front_addr":"172.21.15.71:6829/2687547983","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6811","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6813","nonce":1313958381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6816","nonce":1313958381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6821","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6822","nonce":1313958381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1313958381},{"type":"v1","addr":"172.21.15.71:6820","nonce":1313958381}]},"public_addr":"172.21.15.71:6813/1313958381","cluster_addr":"172.21.15.71:6816/1313958381","heartbeat_back_addr":"172.21.15.71:6822/1313958381","heartbeat_front_addr":"172.21.15.71:6820/1313958381","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6801","nonce":893150098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6803","nonce":893150098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6807","nonce":893150098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":893150098},{"type":"v1","addr":"172.21.15.86:6805","nonce":893150098}]},"public_addr":"172.21.15.86:6801/893150098","cluster_addr":"172.21.15.86:6803/893150098","heartbeat_back_addr":"172.21.15.86:6807/893150098","heartbeat_front_addr":"172.21.15.86:6805/893150098","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6809","nonce":88051276}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6811","nonce":88051276}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6815","nonce":88051276}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":88051276},{"type":"v1","addr":"172.21.15.86:6813","nonce":88051276}]},"public_addr":"172.21.15.86:6809/88051276","cluster_addr":"172.21.15.86:6811/88051276","heartbeat_back_addr":"172.21.15.86:6815/88051276","heartbeat_front_addr":"172.21.15.86:6813/88051276","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6825","nonce":1847760170}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6827","nonce":1847760170}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6831","nonce":1847760170}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1847760170},{"type":"v1","addr":"172.21.15.86:6829","nonce":1847760170}]},"public_addr":"172.21.15.86:6825/1847760170","cluster_addr":"172.21.15.86:6827/1847760170","heartbeat_back_addr":"172.21.15.86:6831/1847760170","heartbeat_front_addr":"172.21.15.86:6829/1847760170","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6817","nonce":2184210851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6819","nonce":2184210851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6823","nonce":2184210851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2184210851},{"type":"v1","addr":"172.21.15.86:6821","nonce":2184210851}]},"public_addr":"172.21.15.86:6817/2184210851","cluster_addr":"172.21.15.86:6819/2184210851","heartbeat_back_addr":"172.21.15.86:6823/2184210851","heartbeat_front_addr":"172.21.15.86:6821/2184210851","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.382801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.392178+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.360372+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.406606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.390686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.380624+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.358004+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-26T05:47:07.410183+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:6835/327274368":"2023-10-27T05:59:07.470507+0000","172.21.15.71:6834/327274368":"2023-10-27T05:59:07.470507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-26T05:59:13.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17dc066470 msgr2=0x7f17dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17dc066470 0x7f17dc068930 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0058d0 tx=0x7f17ec005860 comp rx=0 tx=0).stop 2023-10-26T05:59:13.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 msgr2=0x7f17fc0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:13.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 0x7f17fc0acb70 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f17e4011a50 tx=0x7f17e4011f10 comp rx=0 tx=0).stop 2023-10-26T05:59:13.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 shutdown_connections 2023-10-26T05:59:13.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17fc142670 0x7f17fc0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f17dc066470 0x7f17dc068930 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17fc1421f0 0x7f17fc0acb70 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 --2- 172.21.15.71:0/2450073078 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f17e8004d10 0x7f17fc135980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:13.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 >> 172.21.15.71:0/2450073078 conn(0x7f17fc069770 msgr2=0x7f17fc136850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:13.537 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 shutdown_connections 2023-10-26T05:59:13.538 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:13.527+0000 7f18026d6640 1 -- 172.21.15.71:0/2450073078 wait complete. 2023-10-26T05:59:14.553 INFO:tasks.ceph:Scrubbing osd.0 2023-10-26T05:59:14.554 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:14.694 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.683+0000 7fba3204a640 1 -- 172.21.15.71:0/2731218268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0b04e0 msgr2=0x7fba2c06aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.683+0000 7fba3204a640 1 --2- 172.21.15.71:0/2731218268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0b04e0 0x7fba2c06aa50 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c01c890 comp rx=0 tx=0).stop 2023-10-26T05:59:14.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/2731218268 shutdown_connections 2023-10-26T05:59:14.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 --2- 172.21.15.71:0/2731218268 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fba2c0cce80 0x7fba2c0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 --2- 172.21.15.71:0/2731218268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0b04e0 0x7fba2c06aa50 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.695 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/2731218268 >> 172.21.15.71:0/2731218268 conn(0x7fba2c0cdd80 msgr2=0x7fba2c0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:14.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/2731218268 shutdown_connections 2023-10-26T05:59:14.696 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/2731218268 wait complete. 2023-10-26T05:59:14.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 Processor -- start 2023-10-26T05:59:14.698 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- start start 2023-10-26T05:59:14.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fba2c0b04e0 0x7fba2c0c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c0cce80 0x7fba2c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0c7460 0x7fba2c0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.699 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c153df0 con 0x7fba2c0c7460 2023-10-26T05:59:14.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0c69d0 con 0x7fba2c0cce80 2023-10-26T05:59:14.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c0c9880 con 0x7fba2c0b04e0 2023-10-26T05:59:14.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0c7460 0x7fba2c0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.700 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0c7460 0x7fba2c0ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38940/0 (socket says 172.21.15.71:38940) 2023-10-26T05:59:14.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 -- 172.21.15.71:0/236834471 learned_addr learned my addr 172.21.15.71:0/236834471 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:14.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2b7fe640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fba2c0b04e0 0x7fba2c0c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba23fff640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c0cce80 0x7fba2c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 -- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fba2c0b04e0 msgr2=0x7fba2c0c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.701 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fba2c0b04e0 0x7fba2c0c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 -- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c0cce80 msgr2=0x7fba2c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba2c0cce80 0x7fba2c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 -- 172.21.15.71:0/236834471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c009560 con 0x7fba2c0c7460 2023-10-26T05:59:14.702 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2b7fe640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fba2c0b04e0 0x7fba2c0c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-26T05:59:14.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba2bfff640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0c7460 0x7fba2c0ca340 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fba2400ca60 tx=0x7fba2400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:14.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24007d90 con 0x7fba2c0c7460 2023-10-26T05:59:14.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fba24004b30 con 0x7fba2c0c7460 2023-10-26T05:59:14.703 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c0c6c70 con 0x7fba2c0c7460 2023-10-26T05:59:14.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24005020 con 0x7fba2c0c7460 2023-10-26T05:59:14.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c0c9b40 con 0x7fba2c0c7460 2023-10-26T05:59:14.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.687+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba2c066d00 con 0x7fba2c0c7460 2023-10-26T05:59:14.704 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fba24005180 con 0x7fba2c0c7460 2023-10-26T05:59:14.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fba0c0664a0 0x7fba0c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fba240847f0 con 0x7fba2c0c7460 2023-10-26T05:59:14.706 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fba0c06c300 0x7fba0c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 --> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba0c06edd0 con 0x7fba0c06c300 2023-10-26T05:59:14.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba24084c40 con 0x7fba2c0c7460 2023-10-26T05:59:14.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba2b7fe640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fba0c0664a0 0x7fba0c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.707 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba23fff640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fba0c06c300 0x7fba0c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba23fff640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fba0c06c300 0x7fba0c06e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:14.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba2b7fe640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fba0c0664a0 0x7fba0c068960 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0af000 tx=0x7fba1c01b0d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:14.708 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.691+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== osd.0 v2:172.21.15.71:6808/4161483918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7fba0c06edd0 con 0x7fba0c06c300 2023-10-26T05:59:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 --> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fba2c0c7190 con 0x7fba0c06c300 2023-10-26T05:59:14.711 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba28ff9640 1 -- 172.21.15.71:0/236834471 <== osd.0 v2:172.21.15.71:6808/4161483918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7fba2c0c7190 con 0x7fba0c06c300 2023-10-26T05:59:14.712 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:14.712 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:14.712 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:14.713 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fba0c06c300 msgr2=0x7fba0c06e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:6808/4161483918,v1:172.21.15.71:6809/4161483918] conn(0x7fba0c06c300 0x7fba0c06e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fba0c0664a0 msgr2=0x7fba0c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fba0c0664a0 0x7fba0c068960 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0af000 tx=0x7fba1c01b0d0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0c7460 msgr2=0x7fba2c0ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.714 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 --2- 172.21.15.71:0/236834471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba2c0c7460 0x7fba2c0ca340 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fba2400ca60 tx=0x7fba2400cf20 comp rx=0 tx=0).stop 2023-10-26T05:59:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba2bfff640 1 -- 172.21.15.71:0/236834471 reap_dead start 2023-10-26T05:59:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 shutdown_connections 2023-10-26T05:59:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 >> 172.21.15.71:0/236834471 conn(0x7fba2c0cdd80 msgr2=0x7fba2c1570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 shutdown_connections 2023-10-26T05:59:14.715 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.703+0000 7fba3204a640 1 -- 172.21.15.71:0/236834471 wait complete. 2023-10-26T05:59:14.729 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-26T05:59:14.846 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1946879069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734069d30 msgr2=0x7fc73406a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 --2- 172.21.15.71:0/1946879069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734069d30 0x7fc73406a1c0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009c80 tx=0x7fc72c01cc70 comp rx=0 tx=0).stop 2023-10-26T05:59:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1946879069 shutdown_connections 2023-10-26T05:59:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 --2- 172.21.15.71:0/1946879069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734069d30 0x7fc73406a1c0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.847 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 --2- 172.21.15.71:0/1946879069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc734159700 0x7fc7340b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1946879069 >> 172.21.15.71:0/1946879069 conn(0x7fc73406b8c0 msgr2=0x7fc73406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1946879069 shutdown_connections 2023-10-26T05:59:14.848 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1946879069 wait complete. 2023-10-26T05:59:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 Processor -- start 2023-10-26T05:59:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- start start 2023-10-26T05:59:14.850 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc734069d30 0x7fc7341f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7340ca7f0 0x7fc73413ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 0x7fc73413d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340a6f10 con 0x7fc734159700 2023-10-26T05:59:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7341f60f0 con 0x7fc7340ca7f0 2023-10-26T05:59:14.851 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7340a6570 con 0x7fc734069d30 2023-10-26T05:59:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 0x7fc73413d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 0x7fc73413d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38950/0 (socket says 172.21.15.71:38950) 2023-10-26T05:59:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 -- 172.21.15.71:0/1969831069 learned_addr learned my addr 172.21.15.71:0/1969831069 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc72bfff640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7340ca7f0 0x7fc73413ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.852 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 -- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc734069d30 msgr2=0x7fc7341f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc734069d30 0x7fc7341f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 -- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7340ca7f0 msgr2=0x7fc73413ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7340ca7f0 0x7fc73413ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 -- 172.21.15.71:0/1969831069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0098e0 con 0x7fc734159700 2023-10-26T05:59:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73359e640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 0x7fc73413d3f0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009bd0 tx=0x7fc72c0a5b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:14.853 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c004e60 con 0x7fc734159700 2023-10-26T05:59:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fc72c004000 con 0x7fc734159700 2023-10-26T05:59:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7340a6850 con 0x7fc734159700 2023-10-26T05:59:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c004410 con 0x7fc734159700 2023-10-26T05:59:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.839+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734163b10 con 0x7fc734159700 2023-10-26T05:59:14.854 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.843+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734066b10 con 0x7fc734159700 2023-10-26T05:59:14.855 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.843+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc72c0b4040 con 0x7fc734159700 2023-10-26T05:59:14.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.843+0000 7fc72b7fe640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc714066470 0x7fc714068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:14.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.843+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc72c11e050 con 0x7fc734159700 2023-10-26T05:59:14.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.843+0000 7fc732d9d640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc714066470 0x7fc714068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:14.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.843+0000 7fc732d9d640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc714066470 0x7fc714068930 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc734153830 tx=0x7fc7240079c0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:14.856 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.847+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fc72c0eb7d0 con 0x7fc734159700 2023-10-26T05:59:15.008 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.999+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7340a7050 con 0x7fc714066470 2023-10-26T05:59:15.009 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:14.999+0000 7fc72b7fe640 1 -- 172.21.15.71:0/1969831069 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc7340a7050 con 0x7fc714066470 2023-10-26T05:59:15.009 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 0 to deep-scrub 2023-10-26T05:59:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc714066470 msgr2=0x7fc714068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.011 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc714066470 0x7fc714068930 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc734153830 tx=0x7fc7240079c0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 msgr2=0x7fc73413d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 0x7fc73413d3f0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009bd0 tx=0x7fc72c0a5b00 comp rx=0 tx=0).stop 2023-10-26T05:59:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 shutdown_connections 2023-10-26T05:59:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fc714066470 0x7fc714068930 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.012 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc734159700 0x7fc73413d3f0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7340ca7f0 0x7fc73413ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 --2- 172.21.15.71:0/1969831069 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc734069d30 0x7fc7341f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 >> 172.21.15.71:0/1969831069 conn(0x7fc73406b8c0 msgr2=0x7fc7340acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 shutdown_connections 2023-10-26T05:59:15.013 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.003+0000 7fc73919b640 1 -- 172.21.15.71:0/1969831069 wait complete. 2023-10-26T05:59:15.028 INFO:tasks.ceph:Scrubbing osd.1 2023-10-26T05:59:15.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:15.162 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/3565098673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b0c004d10 msgr2=0x7f4b0c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 --2- 172.21.15.71:0/3565098673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b0c004d10 0x7f4b0c005190 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009e60 tx=0x7f4b1c01d170 comp rx=0 tx=0).stop 2023-10-26T05:59:15.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b22d9d640 1 -- 172.21.15.71:0/3565098673 reap_dead start 2023-10-26T05:59:15.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/3565098673 shutdown_connections 2023-10-26T05:59:15.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/3565098673 >> 172.21.15.71:0/3565098673 conn(0x7f4b240ca2c0 msgr2=0x7f4b240ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.163 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/3565098673 shutdown_connections 2023-10-26T05:59:15.164 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/3565098673 wait complete. 2023-10-26T05:59:15.165 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 Processor -- start 2023-10-26T05:59:15.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- start start 2023-10-26T05:59:15.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b2414d7a0 0x7f4b24150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2414db80 0x7f4b2414e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.166 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b240aba40 0x7f4b240abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b240ac430 con 0x7f4b2414db80 2023-10-26T05:59:15.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b241569c0 con 0x7f4b2414d7a0 2023-10-26T05:59:15.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2414e970 con 0x7f4b240aba40 2023-10-26T05:59:15.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2414db80 0x7f4b2414e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.167 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2414db80 0x7f4b2414e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38960/0 (socket says 172.21.15.71:38960) 2023-10-26T05:59:15.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 -- 172.21.15.71:0/1961076031 learned_addr learned my addr 172.21.15.71:0/1961076031 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:15.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b2259c640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b2414d7a0 0x7f4b24150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b22d9d640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b240aba40 0x7f4b240abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 -- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b240aba40 msgr2=0x7f4b240abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.168 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4b240aba40 0x7f4b240abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 -- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b2414d7a0 msgr2=0x7f4b24150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b2414d7a0 0x7f4b24150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 -- 172.21.15.71:0/1961076031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2414eb90 con 0x7f4b2414db80 2023-10-26T05:59:15.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b21d9b640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2414db80 0x7f4b2414e430 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800d520 tx=0x7f4b1800d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.169 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18011960 con 0x7f4b2414db80 2023-10-26T05:59:15.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4b18011bb0 con 0x7f4b2414db80 2023-10-26T05:59:15.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2414ee70 con 0x7f4b2414db80 2023-10-26T05:59:15.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18012470 con 0x7f4b2414db80 2023-10-26T05:59:15.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2414f430 con 0x7f4b2414db80 2023-10-26T05:59:15.170 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.155+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b240af2a0 con 0x7f4b2414db80 2023-10-26T05:59:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4b1801b400 con 0x7f4b2414db80 2023-10-26T05:59:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b040664a0 0x7f4b04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4b18005e40 con 0x7f4b2414db80 2023-10-26T05:59:15.172 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7f4b0406c300 0x7f4b0406e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 --> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b0406edd0 con 0x7f4b0406c300 2023-10-26T05:59:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b18087ec0 con 0x7f4b2414db80 2023-10-26T05:59:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b22d9d640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7f4b0406c300 0x7f4b0406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b2259c640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b040664a0 0x7f4b04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.173 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b22d9d640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7f4b0406c300 0x7f4b0406e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b2259c640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b040664a0 0x7f4b04068960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ac10 tx=0x7f4b1400a2f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.174 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.159+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== osd.1 v2:172.21.15.71:6800/619557138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f4b0406edd0 con 0x7f4b0406c300 2023-10-26T05:59:15.179 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 --> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4b24066d00 con 0x7f4b0406c300 2023-10-26T05:59:15.180 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b12ffd640 1 -- 172.21.15.71:0/1961076031 <== osd.1 v2:172.21.15.71:6800/619557138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f4b24066d00 con 0x7f4b0406c300 2023-10-26T05:59:15.180 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:15.180 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:15.180 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:15.181 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7f4b0406c300 msgr2=0x7f4b0406e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:6800/619557138,v1:172.21.15.71:6801/619557138] conn(0x7f4b0406c300 0x7f4b0406e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b040664a0 msgr2=0x7f4b04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f4b040664a0 0x7f4b04068960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400ac10 tx=0x7f4b1400a2f0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2414db80 msgr2=0x7f4b2414e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.182 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 --2- 172.21.15.71:0/1961076031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b2414db80 0x7f4b2414e430 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800d520 tx=0x7f4b1800d9e0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b22d9d640 1 -- 172.21.15.71:0/1961076031 reap_dead start 2023-10-26T05:59:15.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 shutdown_connections 2023-10-26T05:59:15.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 >> 172.21.15.71:0/1961076031 conn(0x7f4b240ca2c0 msgr2=0x7f4b240cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 shutdown_connections 2023-10-26T05:59:15.183 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.171+0000 7f4b28c04640 1 -- 172.21.15.71:0/1961076031 wait complete. 2023-10-26T05:59:15.197 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-26T05:59:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/289991807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f437006e580 msgr2=0x7f43700acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 --2- 172.21.15.71:0/289991807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f437006e580 0x7f43700acfc0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f4368009c80 tx=0x7f436801cd10 comp rx=0 tx=0).stop 2023-10-26T05:59:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/289991807 shutdown_connections 2023-10-26T05:59:15.315 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 --2- 172.21.15.71:0/289991807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f437006e580 0x7f43700acfc0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 --2- 172.21.15.71:0/289991807 >> v2:172.21.15.86:3300/0 conn(0x7f4370155930 0x7f437015f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/289991807 >> 172.21.15.71:0/289991807 conn(0x7f43700cacb0 msgr2=0x7f43700cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/289991807 shutdown_connections 2023-10-26T05:59:15.316 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/289991807 wait complete. 2023-10-26T05:59:15.318 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 Processor -- start 2023-10-26T05:59:15.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- start start 2023-10-26T05:59:15.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437006e580 0x7f4370149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 0x7f4370150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.319 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437015f670 0x7f43700af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43701f9040 con 0x7f4370155930 2023-10-26T05:59:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437014fe80 con 0x7f437015f670 2023-10-26T05:59:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4370150160 con 0x7f437006e580 2023-10-26T05:59:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 0x7f4370150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.320 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 0x7f4370150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38984/0 (socket says 172.21.15.71:38984) 2023-10-26T05:59:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 -- 172.21.15.71:0/621162330 learned_addr learned my addr 172.21.15.71:0/621162330 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4375028640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437006e580 0x7f4370149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4375829640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437015f670 0x7f43700af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 -- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437006e580 msgr2=0x7f4370149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.321 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437006e580 0x7f4370149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 -- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437015f670 msgr2=0x7f43700af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437015f670 0x7f43700af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 -- 172.21.15.71:0/621162330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680098e0 con 0x7f4370155930 2023-10-26T05:59:15.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4374827640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 0x7f4370150c40 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f436c001ec0 tx=0x7f436c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.322 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0054e0 con 0x7f4370155930 2023-10-26T05:59:15.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f436c003600 con 0x7f4370155930 2023-10-26T05:59:15.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4370150440 con 0x7f4370155930 2023-10-26T05:59:15.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c003a10 con 0x7f4370155930 2023-10-26T05:59:15.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.307+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4370149db0 con 0x7f4370155930 2023-10-26T05:59:15.323 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.311+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4370150960 con 0x7f4370155930 2023-10-26T05:59:15.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.311+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f436c003b70 con 0x7f4370155930 2023-10-26T05:59:15.324 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.311+0000 7f4365ffb640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43540664a0 0x7f4354068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.311+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f436c083270 con 0x7f4370155930 2023-10-26T05:59:15.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.311+0000 7f4375028640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43540664a0 0x7f4354068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.311+0000 7f4375028640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43540664a0 0x7f4354068960 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4360005fd0 tx=0x7f4360005db0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.325 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.315+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f4370150960 con 0x7f4370155930 2023-10-26T05:59:15.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.467+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4370150960 con 0x7f43540664a0 2023-10-26T05:59:15.477 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 1 to deep-scrub 2023-10-26T05:59:15.477 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.467+0000 7f4365ffb640 1 -- 172.21.15.71:0/621162330 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4370150960 con 0x7f43540664a0 2023-10-26T05:59:15.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43540664a0 msgr2=0x7f4354068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43540664a0 0x7f4354068960 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4360005fd0 tx=0x7f4360005db0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.479 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 msgr2=0x7f4370150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 0x7f4370150c40 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f436c001ec0 tx=0x7f436c00c8e0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 shutdown_connections 2023-10-26T05:59:15.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f437015f670 0x7f43700af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f43540664a0 0x7f4354068960 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.480 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4370155930 0x7f4370150c40 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 --2- 172.21.15.71:0/621162330 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f437006e580 0x7f4370149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 >> 172.21.15.71:0/621162330 conn(0x7f43700cacb0 msgr2=0x7f43700aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 shutdown_connections 2023-10-26T05:59:15.481 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.471+0000 7f437728c640 1 -- 172.21.15.71:0/621162330 wait complete. 2023-10-26T05:59:15.494 INFO:tasks.ceph:Scrubbing osd.2 2023-10-26T05:59:15.495 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:15.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- 172.21.15.71:0/1417035591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 msgr2=0x7f8df00b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 --2- 172.21.15.71:0/1417035591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df00b0fa0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009840 tx=0x7f8de001c650 comp rx=0 tx=0).stop 2023-10-26T05:59:15.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- 172.21.15.71:0/1417035591 shutdown_connections 2023-10-26T05:59:15.628 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 --2- 172.21.15.71:0/1417035591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df00b0fa0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- 172.21.15.71:0/1417035591 >> 172.21.15.71:0/1417035591 conn(0x7f8df00aec60 msgr2=0x7f8df00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- 172.21.15.71:0/1417035591 shutdown_connections 2023-10-26T05:59:15.629 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- 172.21.15.71:0/1417035591 wait complete. 2023-10-26T05:59:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 Processor -- start 2023-10-26T05:59:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- start start 2023-10-26T05:59:15.632 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8df0070dc0 0x7f8df013b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df01539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df015bf70 0x7f8df0156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df013f730 con 0x7f8df015bb90 2023-10-26T05:59:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df013fa10 con 0x7f8df015bf70 2023-10-26T05:59:15.633 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df7206640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df013fcf0 con 0x7f8df0070dc0 2023-10-26T05:59:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df01539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df01539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38996/0 (socket says 172.21.15.71:38996) 2023-10-26T05:59:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 -- 172.21.15.71:0/1644440752 learned_addr learned my addr 172.21.15.71:0/1644440752 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df4fa2640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8df0070dc0 0x7f8df013b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.634 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df57a3640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df015bf70 0x7f8df0156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 -- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8df0070dc0 msgr2=0x7f8df013b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8df0070dc0 0x7f8df013b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.635 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 -- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df015bf70 msgr2=0x7f8df0156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df015bf70 0x7f8df0156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 -- 172.21.15.71:0/1644440752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00094a0 con 0x7f8df015bb90 2023-10-26T05:59:15.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8df57a3640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8df015bf70 0x7f8df0156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:15.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.619+0000 7f8deffff640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df01539c0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f8de400faf0 tx=0x7f8de400d060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.636 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de4008480 con 0x7f8df015bb90 2023-10-26T05:59:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8de400d830 con 0x7f8df015bb90 2023-10-26T05:59:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df013ffd0 con 0x7f8df015bb90 2023-10-26T05:59:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de400dc60 con 0x7f8df015bb90 2023-10-26T05:59:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df0153f00 con 0x7f8df015bb90 2023-10-26T05:59:15.637 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8df013fb80 con 0x7f8df015bb90 2023-10-26T05:59:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8de401e3b0 con 0x7f8df015bb90 2023-10-26T05:59:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ddc0664a0 0x7f8ddc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8de4086ea0 con 0x7f8df015bb90 2023-10-26T05:59:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f8ddc06c300 0x7f8ddc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.638 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 --> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ddc06edd0 con 0x7f8ddc06c300 2023-10-26T05:59:15.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8de40872f0 con 0x7f8df015bb90 2023-10-26T05:59:15.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df57a3640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f8ddc06c300 0x7f8ddc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df4fa2640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ddc0664a0 0x7f8ddc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.639 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df57a3640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f8ddc06c300 0x7f8ddc06e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8df4fa2640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ddc0664a0 0x7f8ddc068960 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8de00af000 tx=0x7f8de001ae90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.640 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.623+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== osd.2 v2:172.21.15.71:6824/2687547983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f8ddc06edd0 con 0x7f8ddc06c300 2023-10-26T05:59:15.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 --> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8df00cade0 con 0x7f8ddc06c300 2023-10-26T05:59:15.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8ded7fa640 1 -- 172.21.15.71:0/1644440752 <== osd.2 v2:172.21.15.71:6824/2687547983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f8de8007aa0 con 0x7f8ddc06c300 2023-10-26T05:59:15.644 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:15.644 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:15.644 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:15.645 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f8ddc06c300 msgr2=0x7f8ddc06e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:6824/2687547983,v1:172.21.15.71:6825/2687547983] conn(0x7f8ddc06c300 0x7f8ddc06e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ddc0664a0 msgr2=0x7f8ddc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8ddc0664a0 0x7f8ddc068960 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f8de00af000 tx=0x7f8de001ae90 comp rx=0 tx=0).stop 2023-10-26T05:59:15.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 msgr2=0x7f8df01539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 --2- 172.21.15.71:0/1644440752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8df015bb90 0x7f8df01539c0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f8de400faf0 tx=0x7f8de400d060 comp rx=0 tx=0).stop 2023-10-26T05:59:15.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df57a3640 1 -- 172.21.15.71:0/1644440752 reap_dead start 2023-10-26T05:59:15.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 shutdown_connections 2023-10-26T05:59:15.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 >> 172.21.15.71:0/1644440752 conn(0x7f8df00aec60 msgr2=0x7f8df015aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 shutdown_connections 2023-10-26T05:59:15.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.635+0000 7f8df7206640 1 -- 172.21.15.71:0/1644440752 wait complete. 2023-10-26T05:59:15.658 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-26T05:59:15.775 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- 172.21.15.71:0/512435249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf980cc7c0 msgr2=0x7fcf980ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 --2- 172.21.15.71:0/512435249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf980cc7c0 0x7fcf980ccbf0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fcf98072e20 tx=0x7fcf9001f350 comp rx=0 tx=0).stop 2023-10-26T05:59:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- 172.21.15.71:0/512435249 shutdown_connections 2023-10-26T05:59:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 --2- 172.21.15.71:0/512435249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf980cc7c0 0x7fcf980ccbf0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 --2- 172.21.15.71:0/512435249 >> v2:172.21.15.86:3300/0 conn(0x7fcf9806a390 0x7fcf9806a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.776 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- 172.21.15.71:0/512435249 >> 172.21.15.71:0/512435249 conn(0x7fcf980d12d0 msgr2=0x7fcf980d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- 172.21.15.71:0/512435249 shutdown_connections 2023-10-26T05:59:15.777 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- 172.21.15.71:0/512435249 wait complete. 2023-10-26T05:59:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 Processor -- start 2023-10-26T05:59:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- start start 2023-10-26T05:59:15.779 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf9806a390 0x7fcf980a8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf980cc7c0 0x7fcf9814aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 0x7fcf980b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980a9bd0 con 0x7fcf9815b3b0 2023-10-26T05:59:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9814c540 con 0x7fcf9806a390 2023-10-26T05:59:15.780 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9e853640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf980a8c70 con 0x7fcf980cc7c0 2023-10-26T05:59:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9cdf0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 0x7fcf980b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9cdf0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 0x7fcf980b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39002/0 (socket says 172.21.15.71:39002) 2023-10-26T05:59:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.767+0000 7fcf9cdf0640 1 -- 172.21.15.71:0/4232343438 learned_addr learned my addr 172.21.15.71:0/4232343438 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf97fff640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf9806a390 0x7fcf980a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.781 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf977fe640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf980cc7c0 0x7fcf9814aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9cdf0640 1 -- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf980cc7c0 msgr2=0x7fcf9814aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9cdf0640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf980cc7c0 0x7fcf9814aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9cdf0640 1 -- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf9806a390 msgr2=0x7fcf980a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9cdf0640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf9806a390 0x7fcf980a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.782 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9cdf0640 1 -- 172.21.15.71:0/4232343438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90009820 con 0x7fcf9815b3b0 2023-10-26T05:59:15.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9cdf0640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 0x7fcf980b5c30 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fcf9001cf80 tx=0x7fcf900af4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004d10 con 0x7fcf9815b3b0 2023-10-26T05:59:15.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fcf900af850 con 0x7fcf9815b3b0 2023-10-26T05:59:15.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9814af50 con 0x7fcf9815b3b0 2023-10-26T05:59:15.783 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf900afc40 con 0x7fcf9815b3b0 2023-10-26T05:59:15.784 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9814bc10 con 0x7fcf9815b3b0 2023-10-26T05:59:15.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf98066b10 con 0x7fcf9815b3b0 2023-10-26T05:59:15.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcf900b6070 con 0x7fcf9815b3b0 2023-10-26T05:59:15.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf94ff9640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf84066470 0x7fcf84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:15.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcf90122170 con 0x7fcf9815b3b0 2023-10-26T05:59:15.785 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf97fff640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf84066470 0x7fcf84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:15.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.771+0000 7fcf97fff640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf84066470 0x7fcf84068930 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fcf880046c0 tx=0x7fcf88009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:15.786 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.775+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fcf900ef8f0 con 0x7fcf9815b3b0 2023-10-26T05:59:15.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf9814c6b0 con 0x7fcf84066470 2023-10-26T05:59:15.939 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf94ff9640 1 -- 172.21.15.71:0/4232343438 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf9814c6b0 con 0x7fcf84066470 2023-10-26T05:59:15.940 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 2 to deep-scrub 2023-10-26T05:59:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf84066470 msgr2=0x7fcf84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf84066470 0x7fcf84068930 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fcf880046c0 tx=0x7fcf88009040 comp rx=0 tx=0).stop 2023-10-26T05:59:15.942 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 msgr2=0x7fcf980b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 0x7fcf980b5c30 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fcf9001cf80 tx=0x7fcf900af4f0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 shutdown_connections 2023-10-26T05:59:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fcf84066470 0x7fcf84068930 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf9815b3b0 0x7fcf980b5c30 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcf980cc7c0 0x7fcf9814aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.943 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 --2- 172.21.15.71:0/4232343438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf9806a390 0x7fcf980a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.931+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 >> 172.21.15.71:0/4232343438 conn(0x7fcf980d12d0 msgr2=0x7fcf980ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.935+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 shutdown_connections 2023-10-26T05:59:15.944 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:15.935+0000 7fcf9e853640 1 -- 172.21.15.71:0/4232343438 wait complete. 2023-10-26T05:59:15.958 INFO:tasks.ceph:Scrubbing osd.3 2023-10-26T05:59:15.959 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:16.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 -- 172.21.15.71:0/1122657617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 msgr2=0x7f6c98005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 --2- 172.21.15.71:0/1122657617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 0x7f6c98005190 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400c610 tx=0x7f6ca401faa0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.077 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cab59e640 1 -- 172.21.15.71:0/1122657617 reap_dead start 2023-10-26T05:59:16.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 -- 172.21.15.71:0/1122657617 shutdown_connections 2023-10-26T05:59:16.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 -- 172.21.15.71:0/1122657617 >> 172.21.15.71:0/1122657617 conn(0x7f6cac0aec60 msgr2=0x7f6cac0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 -- 172.21.15.71:0/1122657617 shutdown_connections 2023-10-26T05:59:16.078 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 -- 172.21.15.71:0/1122657617 wait complete. 2023-10-26T05:59:16.082 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 Processor -- start 2023-10-26T05:59:16.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.067+0000 7f6cb1205640 1 -- start start 2023-10-26T05:59:16.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 0x7f6cac1341c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6cac0a68c0 0x7f6cac0a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.083 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cac15bc70 0x7f6cac15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15c660 con 0x7f6c98004d10 2023-10-26T05:59:16.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac134700 con 0x7f6cac15bc70 2023-10-26T05:59:16.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac1f2bd0 con 0x7f6cac0a68c0 2023-10-26T05:59:16.084 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 0x7f6cac1341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 0x7f6cac1341c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39026/0 (socket says 172.21.15.71:39026) 2023-10-26T05:59:16.085 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 -- 172.21.15.71:0/1829527207 learned_addr learned my addr 172.21.15.71:0/1829527207 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:16.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caa59c640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6cac0a68c0 0x7f6cac0a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.086 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 -- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6cac0a68c0 msgr2=0x7f6cac0a6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6cac0a68c0 0x7f6cac0a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 -- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cac15bc70 msgr2=0x7f6cac15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cab59e640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cac15bc70 0x7f6cac15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.087 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cac15bc70 0x7f6cac15c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 -- 172.21.15.71:0/1829527207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac136bf0 con 0x7f6c98004d10 2023-10-26T05:59:16.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caad9d640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 0x7f6cac1341c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c010a10 tx=0x7f6c9c010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00eda0 con 0x7f6c98004d10 2023-10-26T05:59:16.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6c9c004030 con 0x7f6c98004d10 2023-10-26T05:59:16.088 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac1e2b30 con 0x7f6c98004d10 2023-10-26T05:59:16.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c004420 con 0x7f6c98004d10 2023-10-26T05:59:16.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac136630 con 0x7f6c98004d10 2023-10-26T05:59:16.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6cac0721e0 con 0x7f6c98004d10 2023-10-26T05:59:16.089 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6c9c003600 con 0x7f6c98004d10 2023-10-26T05:59:16.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c8c066470 0x7f6c8c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c9c016070 con 0x7f6c98004d10 2023-10-26T05:59:16.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f6c8c06c2d0 0x7f6c8c06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.090 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 --> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6c8c06eda0 con 0x7f6c8c06c2d0 2023-10-26T05:59:16.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c9c085290 con 0x7f6c98004d10 2023-10-26T05:59:16.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cab59e640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f6c8c06c2d0 0x7f6c8c06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6caa59c640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c8c066470 0x7f6c8c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.071+0000 7f6cab59e640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f6c8c06c2d0 0x7f6c8c06e6f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.091 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.075+0000 7f6caa59c640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c8c066470 0x7f6c8c068930 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6c940058d0 tx=0x7f6c94005860 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.075+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== osd.3 v2:172.21.15.71:6811/1313958381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f6c8c06eda0 con 0x7f6c8c06c2d0 2023-10-26T05:59:16.092 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 --> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6cac1e2cc0 con 0x7f6c8c06c2d0 2023-10-26T05:59:16.093 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6ca37fe640 1 -- 172.21.15.71:0/1829527207 <== osd.3 v2:172.21.15.71:6811/1313958381 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f6ca4002050 con 0x7f6c8c06c2d0 2023-10-26T05:59:16.093 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:16.093 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:16.094 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:16.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f6c8c06c2d0 msgr2=0x7f6c8c06e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:6811/1313958381,v1:172.21.15.71:6813/1313958381] conn(0x7f6c8c06c2d0 0x7f6c8c06e6f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c8c066470 msgr2=0x7f6c8c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.095 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6c8c066470 0x7f6c8c068930 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6c940058d0 tx=0x7f6c94005860 comp rx=0 tx=0).stop 2023-10-26T05:59:16.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 msgr2=0x7f6cac1341c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 --2- 172.21.15.71:0/1829527207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c98004d10 0x7f6cac1341c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c010a10 tx=0x7f6c9c010ed0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cab59e640 1 -- 172.21.15.71:0/1829527207 reap_dead start 2023-10-26T05:59:16.096 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 shutdown_connections 2023-10-26T05:59:16.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 >> 172.21.15.71:0/1829527207 conn(0x7f6cac0aec60 msgr2=0x7f6cac0b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 shutdown_connections 2023-10-26T05:59:16.097 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.083+0000 7f6cb1205640 1 -- 172.21.15.71:0/1829527207 wait complete. 2023-10-26T05:59:16.107 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-26T05:59:16.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- 172.21.15.71:0/2779401486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33580cced0 msgr2=0x7f335806a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 --2- 172.21.15.71:0/2779401486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33580cced0 0x7f335806a7b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3350009bc0 tx=0x7f335001f390 comp rx=0 tx=0).stop 2023-10-26T05:59:16.220 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- 172.21.15.71:0/2779401486 shutdown_connections 2023-10-26T05:59:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 --2- 172.21.15.71:0/2779401486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33580cced0 0x7f335806a7b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 --2- 172.21.15.71:0/2779401486 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f335815b3b0 0x7f33580b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- 172.21.15.71:0/2779401486 >> 172.21.15.71:0/2779401486 conn(0x7f33580d12d0 msgr2=0x7f33580d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- 172.21.15.71:0/2779401486 shutdown_connections 2023-10-26T05:59:16.221 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- 172.21.15.71:0/2779401486 wait complete. 2023-10-26T05:59:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 Processor -- start 2023-10-26T05:59:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- start start 2023-10-26T05:59:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335806a390 0x7f33580afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.224 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 0x7f3358159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f335815b3b0 0x7f3358159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580c77a0 con 0x7f33580cced0 2023-10-26T05:59:16.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580ca6c0 con 0x7f335806a390 2023-10-26T05:59:16.225 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f335ea33640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33580ca3e0 con 0x7f335815b3b0 2023-10-26T05:59:16.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f33577fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 0x7f3358159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.226 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f33577fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 0x7f3358159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39038/0 (socket says 172.21.15.71:39038) 2023-10-26T05:59:16.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.211+0000 7f33577fe640 1 -- 172.21.15.71:0/1785035927 learned_addr learned my addr 172.21.15.71:0/1785035927 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:16.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3357fff640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335806a390 0x7f33580afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f335cfd0640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f335815b3b0 0x7f3358159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f33577fe640 1 -- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f335815b3b0 msgr2=0x7f3358159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.227 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f33577fe640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f335815b3b0 0x7f3358159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f33577fe640 1 -- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335806a390 msgr2=0x7f33580afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f33577fe640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335806a390 0x7f33580afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f33577fe640 1 -- 172.21.15.71:0/1785035927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3350009820 con 0x7f33580cced0 2023-10-26T05:59:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f33577fe640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 0x7f3358159100 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c480 tx=0x7f334c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.228 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c012830 con 0x7f33580cced0 2023-10-26T05:59:16.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f334c012a80 con 0x7f33580cced0 2023-10-26T05:59:16.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33581474f0 con 0x7f33580cced0 2023-10-26T05:59:16.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c011380 con 0x7f33580cced0 2023-10-26T05:59:16.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3358147ab0 con 0x7f33580cced0 2023-10-26T05:59:16.229 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3358066b10 con 0x7f33580cced0 2023-10-26T05:59:16.230 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f334c0114e0 con 0x7f33580cced0 2023-10-26T05:59:16.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3354ff9640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3344066470 0x7f3344068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f334c084020 con 0x7f33580cced0 2023-10-26T05:59:16.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3357fff640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3344066470 0x7f3344068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.215+0000 7f3357fff640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3344066470 0x7f3344068930 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3348009ad0 tx=0x7f33480091e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.231 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.219+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f334c0b5090 con 0x7f33580cced0 2023-10-26T05:59:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.359+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f335806c350 con 0x7f3344066470 2023-10-26T05:59:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f3354ff9640 1 -- 172.21.15.71:0/1785035927 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f335806c350 con 0x7f3344066470 2023-10-26T05:59:16.370 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 3 to deep-scrub 2023-10-26T05:59:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3344066470 msgr2=0x7f3344068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3344066470 0x7f3344068930 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3348009ad0 tx=0x7f33480091e0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 msgr2=0x7f3358159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.373 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 0x7f3358159100 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c480 tx=0x7f334c00c940 comp rx=0 tx=0).stop 2023-10-26T05:59:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 shutdown_connections 2023-10-26T05:59:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f335815b3b0 0x7f3358159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3344066470 0x7f3344068930 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33580cced0 0x7f3358159100 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.374 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 --2- 172.21.15.71:0/1785035927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f335806a390 0x7f33580afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 >> 172.21.15.71:0/1785035927 conn(0x7f33580d12d0 msgr2=0x7f33580d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 shutdown_connections 2023-10-26T05:59:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.363+0000 7f335ea33640 1 -- 172.21.15.71:0/1785035927 wait complete. 2023-10-26T05:59:16.388 INFO:tasks.ceph:Scrubbing osd.4 2023-10-26T05:59:16.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:16.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 -- 172.21.15.71:0/4015978773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d20003980 msgr2=0x7f3d20003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.513 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 --2- 172.21.15.71:0/4015978773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d20003980 0x7f3d20003da0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009ee0 tx=0x7f3d2801d5f0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 -- 172.21.15.71:0/4015978773 shutdown_connections 2023-10-26T05:59:16.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 --2- 172.21.15.71:0/4015978773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d20003980 0x7f3d20003da0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 -- 172.21.15.71:0/4015978773 >> 172.21.15.71:0/4015978773 conn(0x7f3d380aec60 msgr2=0x7f3d380af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 -- 172.21.15.71:0/4015978773 shutdown_connections 2023-10-26T05:59:16.514 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.503+0000 7f3d40634640 1 -- 172.21.15.71:0/4015978773 wait complete. 2023-10-26T05:59:16.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 Processor -- start 2023-10-26T05:59:16.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- start start 2023-10-26T05:59:16.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d20003980 0x7f3d381f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d380adbd0 0x7f3d381563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.517 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d38156be0 0x7f3d3813b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380ac720 con 0x7f3d38156be0 2023-10-26T05:59:16.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d381f1b50 con 0x7f3d20003980 2023-10-26T05:59:16.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380ae050 con 0x7f3d380adbd0 2023-10-26T05:59:16.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d38156be0 0x7f3d3813b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.518 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d38156be0 0x7f3d3813b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39052/0 (socket says 172.21.15.71:39052) 2023-10-26T05:59:16.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 -- 172.21.15.71:0/8409962 learned_addr learned my addr 172.21.15.71:0/8409962 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:16.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3e3d0640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d20003980 0x7f3d381f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3dbcf640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d380adbd0 0x7f3d381563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 -- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d380adbd0 msgr2=0x7f3d381563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.519 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3d380adbd0 0x7f3d381563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 -- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d20003980 msgr2=0x7f3d381f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d20003980 0x7f3d381f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 -- 172.21.15.71:0/8409962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d381f5850 con 0x7f3d38156be0 2023-10-26T05:59:16.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3e3d0640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d20003980 0x7f3d381f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:16.520 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d3ebd1640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d38156be0 0x7f3d3813b2d0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f3d3806c780 tx=0x7f3d30001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d300107d0 con 0x7f3d38156be0 2023-10-26T05:59:16.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3d30010a20 con 0x7f3d38156be0 2023-10-26T05:59:16.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30010e30 con 0x7f3d38156be0 2023-10-26T05:59:16.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d381f5b30 con 0x7f3d38156be0 2023-10-26T05:59:16.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d381f60f0 con 0x7f3d38156be0 2023-10-26T05:59:16.521 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d380721e0 con 0x7f3d38156be0 2023-10-26T05:59:16.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.507+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3d30011460 con 0x7f3d38156be0 2023-10-26T05:59:16.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d36ffd640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d240664a0 0x7f3d24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.523 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3d30084d50 con 0x7f3d38156be0 2023-10-26T05:59:16.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d36ffd640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f3d2406c300 0x7f3d2406e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 --> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d2406edd0 con 0x7f3d2406c300 2023-10-26T05:59:16.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d300851a0 con 0x7f3d38156be0 2023-10-26T05:59:16.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d3e3d0640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d240664a0 0x7f3d24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.524 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d3dbcf640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f3d2406c300 0x7f3d2406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d3e3d0640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d240664a0 0x7f3d24068960 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3d280af000 tx=0x7f3d2801a390 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d3dbcf640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f3d2406c300 0x7f3d2406e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.525 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.511+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== osd.4 v2:172.21.15.86:6800/893150098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f3d2406edd0 con 0x7f3d2406c300 2023-10-26T05:59:16.531 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 --> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3d38066b10 con 0x7f3d2406c300 2023-10-26T05:59:16.532 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d36ffd640 1 -- 172.21.15.71:0/8409962 <== osd.4 v2:172.21.15.86:6800/893150098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f3d38066b10 con 0x7f3d2406c300 2023-10-26T05:59:16.532 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:16.533 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:16.533 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:16.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f3d2406c300 msgr2=0x7f3d2406e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6800/893150098,v1:172.21.15.86:6801/893150098] conn(0x7f3d2406c300 0x7f3d2406e720 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.534 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d240664a0 msgr2=0x7f3d24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3d240664a0 0x7f3d24068960 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3d280af000 tx=0x7f3d2801a390 comp rx=0 tx=0).stop 2023-10-26T05:59:16.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d38156be0 msgr2=0x7f3d3813b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 --2- 172.21.15.71:0/8409962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d38156be0 0x7f3d3813b2d0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f3d3806c780 tx=0x7f3d30001640 comp rx=0 tx=0).stop 2023-10-26T05:59:16.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d3ebd1640 1 -- 172.21.15.71:0/8409962 reap_dead start 2023-10-26T05:59:16.535 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 shutdown_connections 2023-10-26T05:59:16.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 >> 172.21.15.71:0/8409962 conn(0x7f3d380aec60 msgr2=0x7f3d380abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 shutdown_connections 2023-10-26T05:59:16.536 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.523+0000 7f3d40634640 1 -- 172.21.15.71:0/8409962 wait complete. 2023-10-26T05:59:16.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-26T05:59:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- 172.21.15.71:0/987833892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81780ace90 msgr2=0x7f817815a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 --2- 172.21.15.71:0/987833892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81780ace90 0x7f817815a2d0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f81780cd720 tx=0x7f817001c7a0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- 172.21.15.71:0/987833892 shutdown_connections 2023-10-26T05:59:16.668 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 --2- 172.21.15.71:0/987833892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81780ace90 0x7f817815a2d0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 --2- 172.21.15.71:0/987833892 >> v2:172.21.15.86:3300/0 conn(0x7f81780ca960 0x7f81780b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- 172.21.15.71:0/987833892 >> 172.21.15.71:0/987833892 conn(0x7f8178070bc0 msgr2=0x7f81780af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- 172.21.15.71:0/987833892 shutdown_connections 2023-10-26T05:59:16.669 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- 172.21.15.71:0/987833892 wait complete. 2023-10-26T05:59:16.671 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 Processor -- start 2023-10-26T05:59:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- start start 2023-10-26T05:59:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81780ace90 0x7f817814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81780ca960 0x7f817815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 0x7f81780ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.672 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817814f780 con 0x7f8178153960 2023-10-26T05:59:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178160010 con 0x7f81780ace90 2023-10-26T05:59:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.659+0000 7f818070e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81781602f0 con 0x7f81780ca960 2023-10-26T05:59:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 0x7f81780ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.673 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 0x7f81780ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39070/0 (socket says 172.21.15.71:39070) 2023-10-26T05:59:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 -- 172.21.15.71:0/4092575178 learned_addr learned my addr 172.21.15.71:0/4092575178 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817e4aa640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81780ace90 0x7f817814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817dca9640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81780ca960 0x7f817815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 -- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81780ca960 msgr2=0x7f817815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.674 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81780ca960 0x7f817815f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 -- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81780ace90 msgr2=0x7f817814ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81780ace90 0x7f817814ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 -- 172.21.15.71:0/4092575178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170009560 con 0x7f8178153960 2023-10-26T05:59:16.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817dca9640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81780ca960 0x7f817815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:16.675 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817ecab640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 0x7f81780ad550 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f8170009850 tx=0x7f81700a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700042c0 con 0x7f8178153960 2023-10-26T05:59:16.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8170004510 con 0x7f8178153960 2023-10-26T05:59:16.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81781605d0 con 0x7f8178153960 2023-10-26T05:59:16.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170003860 con 0x7f8178153960 2023-10-26T05:59:16.676 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8178160b90 con 0x7f8178153960 2023-10-26T05:59:16.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178071580 con 0x7f8178153960 2023-10-26T05:59:16.677 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81700b4070 con 0x7f8178153960 2023-10-26T05:59:16.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f8176ffd640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8164066470 0x7f8164068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f817011e460 con 0x7f8178153960 2023-10-26T05:59:16.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817e4aa640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8164066470 0x7f8164068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.678 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.663+0000 7f817e4aa640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8164066470 0x7f8164068930 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8178072ff0 tx=0x7f81680094d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.679 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.667+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f81700ebc70 con 0x7f8178153960 2023-10-26T05:59:16.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.823+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8178071580 con 0x7f8164066470 2023-10-26T05:59:16.833 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.823+0000 7f8176ffd640 1 -- 172.21.15.71:0/4092575178 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8178071580 con 0x7f8164066470 2023-10-26T05:59:16.834 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 4 to deep-scrub 2023-10-26T05:59:16.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8164066470 msgr2=0x7f8164068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8164066470 0x7f8164068930 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8178072ff0 tx=0x7f81680094d0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 msgr2=0x7f81780ad550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.836 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 0x7f81780ad550 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f8170009850 tx=0x7f81700a56f0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 shutdown_connections 2023-10-26T05:59:16.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8164066470 0x7f8164068930 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8178153960 0x7f81780ad550 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f81780ca960 0x7f817815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.837 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 --2- 172.21.15.71:0/4092575178 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81780ace90 0x7f817814ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 >> 172.21.15.71:0/4092575178 conn(0x7f8178070bc0 msgr2=0x7f817815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 shutdown_connections 2023-10-26T05:59:16.838 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.827+0000 7f818070e640 1 -- 172.21.15.71:0/4092575178 wait complete. 2023-10-26T05:59:16.852 INFO:tasks.ceph:Scrubbing osd.5 2023-10-26T05:59:16.853 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 -- 172.21.15.71:0/2893414791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 msgr2=0x7f6bf4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 --2- 172.21.15.71:0/2893414791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 0x7f6bf4003da0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00c260 tx=0x7f6bfc01cf70 comp rx=0 tx=0).stop 2023-10-26T05:59:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c110e0640 1 -- 172.21.15.71:0/2893414791 reap_dead start 2023-10-26T05:59:16.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 -- 172.21.15.71:0/2893414791 shutdown_connections 2023-10-26T05:59:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 -- 172.21.15.71:0/2893414791 >> 172.21.15.71:0/2893414791 conn(0x7f6c0c0aec60 msgr2=0x7f6c0c0af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 -- 172.21.15.71:0/2893414791 shutdown_connections 2023-10-26T05:59:16.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 -- 172.21.15.71:0/2893414791 wait complete. 2023-10-26T05:59:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 Processor -- start 2023-10-26T05:59:16.988 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.975+0000 7f6c12b43640 1 -- start start 2023-10-26T05:59:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 0x7f6c0c14cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c0c15e2b0 0x7f6c0c148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c15c9c0 0x7f6c0c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c1491e0 con 0x7f6bf4003980 2023-10-26T05:59:16.989 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c15d210 con 0x7f6c0c15c9c0 2023-10-26T05:59:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c15d4f0 con 0x7f6c0c15e2b0 2023-10-26T05:59:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 0x7f6c0c14cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 0x7f6c0c14cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39088/0 (socket says 172.21.15.71:39088) 2023-10-26T05:59:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 -- 172.21.15.71:0/4002475297 learned_addr learned my addr 172.21.15.71:0/4002475297 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:16.990 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c0bfff640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c0c15e2b0 0x7f6c0c148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c110e0640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c15c9c0 0x7f6c0c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 -- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c0c15e2b0 msgr2=0x7f6c0c148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6c0c15e2b0 0x7f6c0c148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 -- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c15c9c0 msgr2=0x7f6c0c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:16.991 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c0c15c9c0 0x7f6c0c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:16.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 -- 172.21.15.71:0/4002475297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0ac610 con 0x7f6bf4003980 2023-10-26T05:59:16.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c108df640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 0x7f6c0c14cd30 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc01b7f0 tx=0x7f6bfc0066c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc0b1070 con 0x7f6bf4003980 2023-10-26T05:59:16.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f6bfc001cb0 con 0x7f6bf4003980 2023-10-26T05:59:16.992 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c15d7d0 con 0x7f6bf4003980 2023-10-26T05:59:16.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc003860 con 0x7f6bf4003980 2023-10-26T05:59:16.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c15dd90 con 0x7f6bf4003980 2023-10-26T05:59:16.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c0c0b20b0 con 0x7f6bf4003980 2023-10-26T05:59:16.993 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6bfc0b8440 con 0x7f6bf4003980 2023-10-26T05:59:16.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bf80664a0 0x7f6bf8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.994 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6bfc0b4040 con 0x7f6bf4003980 2023-10-26T05:59:16.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f6bf806c300 0x7f6bf806e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:16.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 --> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bf806edd0 con 0x7f6bf806c300 2023-10-26T05:59:16.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bfc120460 con 0x7f6bf4003980 2023-10-26T05:59:16.995 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c0bfff640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bf80664a0 0x7f6bf8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.979+0000 7f6c110e0640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f6bf806c300 0x7f6bf806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:16.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.983+0000 7f6c110e0640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f6bf806c300 0x7f6bf806e720 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.983+0000 7f6c0bfff640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bf80664a0 0x7f6bf8068960 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005b70 tx=0x7f6c00005ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:16.996 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.983+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== osd.5 v2:172.21.15.86:6808/88051276 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f6bf806edd0 con 0x7f6bf806c300 2023-10-26T05:59:17.002 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 --> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6c0c166070 con 0x7f6bf806c300 2023-10-26T05:59:17.003 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c097fa640 1 -- 172.21.15.71:0/4002475297 <== osd.5 v2:172.21.15.86:6808/88051276 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f6c04009c10 con 0x7f6bf806c300 2023-10-26T05:59:17.003 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:17.004 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:17.004 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f6bf806c300 msgr2=0x7f6bf806e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6808/88051276,v1:172.21.15.86:6809/88051276] conn(0x7f6bf806c300 0x7f6bf806e720 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bf80664a0 msgr2=0x7f6bf8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.005 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6bf80664a0 0x7f6bf8068960 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6c00005b70 tx=0x7f6c00005ac0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 msgr2=0x7f6c0c14cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 --2- 172.21.15.71:0/4002475297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bf4003980 0x7f6c0c14cd30 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc01b7f0 tx=0x7f6bfc0066c0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c110e0640 1 -- 172.21.15.71:0/4002475297 reap_dead start 2023-10-26T05:59:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 shutdown_connections 2023-10-26T05:59:17.006 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 >> 172.21.15.71:0/4002475297 conn(0x7f6c0c0aec60 msgr2=0x7f6c0c0cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 shutdown_connections 2023-10-26T05:59:17.007 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:16.995+0000 7f6c12b43640 1 -- 172.21.15.71:0/4002475297 wait complete. 2023-10-26T05:59:17.019 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-26T05:59:17.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/1422614334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a20158360 msgr2=0x7f2a2006de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 --2- 172.21.15.71:0/1422614334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a20158360 0x7f2a2006de70 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009900 tx=0x7f2a1c01c8c0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/1422614334 shutdown_connections 2023-10-26T05:59:17.131 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 --2- 172.21.15.71:0/1422614334 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a200b2710 0x7f2a2006e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 --2- 172.21.15.71:0/1422614334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a20158360 0x7f2a2006de70 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/1422614334 >> 172.21.15.71:0/1422614334 conn(0x7f2a200cacb0 msgr2=0x7f2a200cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/1422614334 shutdown_connections 2023-10-26T05:59:17.132 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/1422614334 wait complete. 2023-10-26T05:59:17.134 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 Processor -- start 2023-10-26T05:59:17.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- start start 2023-10-26T05:59:17.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a200b2710 0x7f2a20147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 0x7f2a20162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a20158360 0x7f2a20158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.135 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20161410 con 0x7f2a200cd800 2023-10-26T05:59:17.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20159310 con 0x7f2a20158360 2023-10-26T05:59:17.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20161660 con 0x7f2a200b2710 2023-10-26T05:59:17.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 0x7f2a20162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 0x7f2a20162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39120/0 (socket says 172.21.15.71:39120) 2023-10-26T05:59:17.136 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 -- 172.21.15.71:0/3104697195 learned_addr learned my addr 172.21.15.71:0/3104697195 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:17.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a2521f640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a200b2710 0x7f2a20147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a25a20640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a20158360 0x7f2a20158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 -- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a200b2710 msgr2=0x7f2a20147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a200b2710 0x7f2a20147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.137 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 -- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a20158360 msgr2=0x7f2a20158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a20158360 0x7f2a20158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 -- 172.21.15.71:0/3104697195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c009560 con 0x7f2a200cd800 2023-10-26T05:59:17.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a25a20640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a20158360 0x7f2a20158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:17.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a24a1e640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 0x7f2a20162140 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b550 tx=0x7f2a1000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.138 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10010870 con 0x7f2a200cd800 2023-10-26T05:59:17.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2a10010ae0 con 0x7f2a200cd800 2023-10-26T05:59:17.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1000f3c0 con 0x7f2a200cd800 2023-10-26T05:59:17.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a20161940 con 0x7f2a200cd800 2023-10-26T05:59:17.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.123+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a20158f50 con 0x7f2a200cd800 2023-10-26T05:59:17.139 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.127+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2004db60 con 0x7f2a200cd800 2023-10-26T05:59:17.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.127+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2a1000f520 con 0x7f2a200cd800 2023-10-26T05:59:17.140 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.127+0000 7f2a15ffb640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a04066470 0x7f2a04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.127+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a10083de0 con 0x7f2a200cd800 2023-10-26T05:59:17.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.127+0000 7f2a2521f640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a04066470 0x7f2a04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.127+0000 7f2a2521f640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a04066470 0x7f2a04068930 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0af000 tx=0x7f2a1c01b100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.141 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.131+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f2a10051560 con 0x7f2a200cd800 2023-10-26T05:59:17.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:59:17.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:59:17.294 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.283+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a20162ff0 con 0x7f2a04066470 2023-10-26T05:59:17.295 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a15ffb640 1 -- 172.21.15.71:0/3104697195 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2a20162ff0 con 0x7f2a04066470 2023-10-26T05:59:17.295 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 5 to deep-scrub 2023-10-26T05:59:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a04066470 msgr2=0x7f2a04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a04066470 0x7f2a04068930 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0af000 tx=0x7f2a1c01b100 comp rx=0 tx=0).stop 2023-10-26T05:59:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 msgr2=0x7f2a20162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 0x7f2a20162140 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000b550 tx=0x7f2a1000ba10 comp rx=0 tx=0).stop 2023-10-26T05:59:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 shutdown_connections 2023-10-26T05:59:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a20158360 0x7f2a20158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2a04066470 0x7f2a04068930 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a200cd800 0x7f2a20162140 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 --2- 172.21.15.71:0/3104697195 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2a200b2710 0x7f2a20147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 >> 172.21.15.71:0/3104697195 conn(0x7f2a200cacb0 msgr2=0x7f2a2006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 shutdown_connections 2023-10-26T05:59:17.299 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.287+0000 7f2a27483640 1 -- 172.21.15.71:0/3104697195 wait complete. 2023-10-26T05:59:17.313 INFO:tasks.ceph:Scrubbing osd.6 2023-10-26T05:59:17.314 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- 172.21.15.71:0/3151114580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c06c520 msgr2=0x7f692c144590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 --2- 172.21.15.71:0/3151114580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c06c520 0x7f692c144590 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6924009bc0 tx=0x7f692401cb10 comp rx=0 tx=0).stop 2023-10-26T05:59:17.488 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- 172.21.15.71:0/3151114580 shutdown_connections 2023-10-26T05:59:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 --2- 172.21.15.71:0/3151114580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c06c520 0x7f692c144590 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- 172.21.15.71:0/3151114580 >> 172.21.15.71:0/3151114580 conn(0x7f692c06f440 msgr2=0x7f692c06f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- 172.21.15.71:0/3151114580 shutdown_connections 2023-10-26T05:59:17.489 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- 172.21.15.71:0/3151114580 wait complete. 2023-10-26T05:59:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 Processor -- start 2023-10-26T05:59:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- start start 2023-10-26T05:59:17.492 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692c06c520 0x7f692c160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c0cf6f0 0x7f692c0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f692c1441b0 0x7f692c1396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c140800 con 0x7f692c0cf6f0 2023-10-26T05:59:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c140ac0 con 0x7f692c06c520 2023-10-26T05:59:17.493 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f6931ebf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c140da0 con 0x7f692c1441b0 2023-10-26T05:59:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f692affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c0cf6f0 0x7f692c0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f692affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c0cf6f0 0x7f692c0b1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39128/0 (socket says 172.21.15.71:39128) 2023-10-26T05:59:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.479+0000 7f692affd640 1 -- 172.21.15.71:0/2063162435 learned_addr learned my addr 172.21.15.71:0/2063162435 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692affd640 1 -- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f692c1441b0 msgr2=0x7f692c1396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.494 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692bfff640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f692c1441b0 0x7f692c1396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692b7fe640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692c06c520 0x7f692c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692affd640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f692c1441b0 0x7f692c1396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692affd640 1 -- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692c06c520 msgr2=0x7f692c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692affd640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692c06c520 0x7f692c160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.495 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692affd640 1 -- 172.21.15.71:0/2063162435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009820 con 0x7f692c0cf6f0 2023-10-26T05:59:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692b7fe640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f692c06c520 0x7f692c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692affd640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c0cf6f0 0x7f692c0b1710 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c410 tx=0x7f691c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c011790 con 0x7f692c0cf6f0 2023-10-26T05:59:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f691c0119e0 con 0x7f692c0cf6f0 2023-10-26T05:59:17.496 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c141080 con 0x7f692c0cf6f0 2023-10-26T05:59:17.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c011e30 con 0x7f692c0cf6f0 2023-10-26T05:59:17.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c1605b0 con 0x7f692c0cf6f0 2023-10-26T05:59:17.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f692c0721e0 con 0x7f692c0cf6f0 2023-10-26T05:59:17.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f691c01c460 con 0x7f692c0cf6f0 2023-10-26T05:59:17.497 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6918066470 0x7f6918068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.498 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f691c083b90 con 0x7f692c0cf6f0 2023-10-26T05:59:17.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f691806c2d0 0x7f691806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 --> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f691806eda0 con 0x7f691806c2d0 2023-10-26T05:59:17.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f691c0882c0 con 0x7f692c0cf6f0 2023-10-26T05:59:17.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692b7fe640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6918066470 0x7f6918068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.499 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692bfff640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f691806c2d0 0x7f691806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692bfff640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f691806c2d0 0x7f691806e6f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.483+0000 7f692b7fe640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6918066470 0x7f6918068930 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f69200046c0 tx=0x7f6920009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.500 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.487+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== osd.6 v2:172.21.15.86:6824/1847760170 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f691806eda0 con 0x7f691806c2d0 2023-10-26T05:59:17.506 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 --> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f692c066b10 con 0x7f691806c2d0 2023-10-26T05:59:17.507 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6913fff640 1 -- 172.21.15.71:0/2063162435 <== osd.6 v2:172.21.15.86:6824/1847760170 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f692c066b10 con 0x7f691806c2d0 2023-10-26T05:59:17.507 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:17.507 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:17.508 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:17.508 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f691806c2d0 msgr2=0x7f691806e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6824/1847760170,v1:172.21.15.86:6825/1847760170] conn(0x7f691806c2d0 0x7f691806e6f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6918066470 msgr2=0x7f6918068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f6918066470 0x7f6918068930 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f69200046c0 tx=0x7f6920009040 comp rx=0 tx=0).stop 2023-10-26T05:59:17.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c0cf6f0 msgr2=0x7f692c0b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.509 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 --2- 172.21.15.71:0/2063162435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692c0cf6f0 0x7f692c0b1710 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c410 tx=0x7f691c00c8d0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f692bfff640 1 -- 172.21.15.71:0/2063162435 reap_dead start 2023-10-26T05:59:17.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 shutdown_connections 2023-10-26T05:59:17.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 >> 172.21.15.71:0/2063162435 conn(0x7f692c06f440 msgr2=0x7f692c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.510 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 shutdown_connections 2023-10-26T05:59:17.511 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.499+0000 7f6931ebf640 1 -- 172.21.15.71:0/2063162435 wait complete. 2023-10-26T05:59:17.524 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-26T05:59:17.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/3018780091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 msgr2=0x7fa02406f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 --2- 172.21.15.71:0/3018780091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa02406f6f0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fa02406b1f0 tx=0x7fa01c01cc60 comp rx=0 tx=0).stop 2023-10-26T05:59:17.643 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/3018780091 shutdown_connections 2023-10-26T05:59:17.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 --2- 172.21.15.71:0/3018780091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa02406f6f0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 --2- 172.21.15.71:0/3018780091 >> v2:172.21.15.86:3300/0 conn(0x7fa02413ae90 0x7fa0240cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/3018780091 >> 172.21.15.71:0/3018780091 conn(0x7fa02406b8c0 msgr2=0x7fa02406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/3018780091 shutdown_connections 2023-10-26T05:59:17.644 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/3018780091 wait complete. 2023-10-26T05:59:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 Processor -- start 2023-10-26T05:59:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- start start 2023-10-26T05:59:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa02413ae90 0x7fa024156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa024157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa024159060 0x7fa024134ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024158110 con 0x7fa02413de80 2023-10-26T05:59:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02414e9e0 con 0x7fa024159060 2023-10-26T05:59:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02414ecc0 con 0x7fa02413ae90 2023-10-26T05:59:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa024157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa024157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39146/0 (socket says 172.21.15.71:39146) 2023-10-26T05:59:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 -- 172.21.15.71:0/2663574177 learned_addr learned my addr 172.21.15.71:0/2663574177 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa023fff640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa024159060 0x7fa024134ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 -- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa02413ae90 msgr2=0x7fa024156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa02413ae90 0x7fa024156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 -- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa024159060 msgr2=0x7fa024134ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa024159060 0x7fa024134ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 -- 172.21.15.71:0/2663574177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0098e0 con 0x7fa02413de80 2023-10-26T05:59:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa023fff640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa024159060 0x7fa024134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa01affd640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa024157460 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00c450 tx=0x7fa00c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c012830 con 0x7fa02413de80 2023-10-26T05:59:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa00c012aa0 con 0x7fa02413de80 2023-10-26T05:59:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02414efa0 con 0x7fa02413de80 2023-10-26T05:59:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c011380 con 0x7fa02413de80 2023-10-26T05:59:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.635+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02414f560 con 0x7fa02413de80 2023-10-26T05:59:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.639+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa024066b10 con 0x7fa02413de80 2023-10-26T05:59:17.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.639+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa00c007020 con 0x7fa02413de80 2023-10-26T05:59:17.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.639+0000 7fa020ff9640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0040664a0 0x7fa004068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.652 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.639+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa00c084210 con 0x7fa02413de80 2023-10-26T05:59:17.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.639+0000 7fa0237fe640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0040664a0 0x7fa004068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.639+0000 7fa0237fe640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0040664a0 0x7fa004068960 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa01400a910 tx=0x7fa014005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.653 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.643+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fa00c051b00 con 0x7fa02413de80 2023-10-26T05:59:17.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.799+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa024070980 con 0x7fa0040664a0 2023-10-26T05:59:17.810 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa020ff9640 1 -- 172.21.15.71:0/2663574177 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa01400a500 con 0x7fa0040664a0 2023-10-26T05:59:17.811 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 6 to deep-scrub 2023-10-26T05:59:17.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0040664a0 msgr2=0x7fa004068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0040664a0 0x7fa004068960 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa01400a910 tx=0x7fa014005be0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.813 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 msgr2=0x7fa024157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa024157460 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00c450 tx=0x7fa00c00c910 comp rx=0 tx=0).stop 2023-10-26T05:59:17.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 shutdown_connections 2023-10-26T05:59:17.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa024159060 0x7fa024134ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.814 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fa0040664a0 0x7fa004068960 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa02413de80 0x7fa024157460 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 --2- 172.21.15.71:0/2663574177 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa02413ae90 0x7fa024156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 >> 172.21.15.71:0/2663574177 conn(0x7fa02406b8c0 msgr2=0x7fa0241520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.815 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 shutdown_connections 2023-10-26T05:59:17.816 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.803+0000 7fa029d37640 1 -- 172.21.15.71:0/2663574177 wait complete. 2023-10-26T05:59:17.830 INFO:tasks.ceph:Scrubbing osd.7 2023-10-26T05:59:17.831 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-26T05:59:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 -- 172.21.15.71:0/992347993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 msgr2=0x7f2cf806ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 --2- 172.21.15.71:0/992347993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 0x7f2cf806ac60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009840 tx=0x7f2ce801c7d0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 -- 172.21.15.71:0/992347993 shutdown_connections 2023-10-26T05:59:17.961 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 --2- 172.21.15.71:0/992347993 >> v2:172.21.15.86:3300/0 conn(0x7f2cf80cf2f0 0x7f2cf80cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 --2- 172.21.15.71:0/992347993 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 0x7f2cf806ac60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 -- 172.21.15.71:0/992347993 >> 172.21.15.71:0/992347993 conn(0x7f2cf80d13f0 msgr2=0x7f2cf80d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 -- 172.21.15.71:0/992347993 shutdown_connections 2023-10-26T05:59:17.962 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.951+0000 7f2cff424640 1 -- 172.21.15.71:0/992347993 wait complete. 2023-10-26T05:59:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 Processor -- start 2023-10-26T05:59:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- start start 2023-10-26T05:59:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 0x7f2cf8156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf80cf2f0 0x7f2cf80c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.965 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf815b4d0 0x7f2cf81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80c5370 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80c5650 con 0x7f2cf815b4d0 2023-10-26T05:59:17.966 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf8155c50 con 0x7f2cf806a7e0 2023-10-26T05:59:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf80cf2f0 0x7f2cf80c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf80cf2f0 0x7f2cf80c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39152/0 (socket says 172.21.15.71:39152) 2023-10-26T05:59:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 -- 172.21.15.71:0/1387875189 learned_addr learned my addr 172.21.15.71:0/1387875189 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfd9c1640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf815b4d0 0x7f2cf81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.967 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfd1c0640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 0x7f2cf8156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 -- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 msgr2=0x7f2cf8156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 0x7f2cf8156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 -- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf815b4d0 msgr2=0x7f2cf81543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cf815b4d0 0x7f2cf81543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.968 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 -- 172.21.15.71:0/1387875189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00098e0 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfd1c0640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2cf806a7e0 0x7f2cf8156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:17.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cfc9bf640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf80cf2f0 0x7f2cf80c87d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009790 tx=0x7f2ce80a5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80043c0 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f2ce8005020 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce80094a0 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.969 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce8005410 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf8156210 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.970 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cf815b8b0 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.971 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.955+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2ce80b4040 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cedffb640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cdc066470 0x7f2cdc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2ce811e190 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cedffb640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f2cdc06c2d0 0x7f2cdc06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:17.972 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 --> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2cdc06eda0 con 0x7f2cdc06c2d0 2023-10-26T05:59:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=195) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ce811e5e0 con 0x7f2cf80cf2f0 2023-10-26T05:59:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cfd1c0640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cdc066470 0x7f2cdc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cfd9c1640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f2cdc06c2d0 0x7f2cdc06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cfd1c0640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cdc066470 0x7f2cdc068930 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4000f70 tx=0x7f2cf4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.973 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cfd9c1640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f2cdc06c2d0 0x7f2cdc06e6f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:17.974 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.959+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== osd.7 v2:172.21.15.86:6816/2184210851 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23595 (crc 0 0 0) 0x7f2cdc06eda0 con 0x7f2cdc06c2d0 2023-10-26T05:59:17.979 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 --> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2cf8066b10 con 0x7f2cdc06c2d0 2023-10-26T05:59:17.980 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cedffb640 1 -- 172.21.15.71:0/1387875189 <== osd.7 v2:172.21.15.86:6816/2184210851 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+632 (crc 0 0 0) 0x7f2cf8066b10 con 0x7f2cdc06c2d0 2023-10-26T05:59:17.981 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2023-10-26T05:59:17.981 INFO:teuthology.orchestra.run.smithi071.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2023-10-26T05:59:17.981 INFO:teuthology.orchestra.run.smithi071.stdout:} 2023-10-26T05:59:17.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f2cdc06c2d0 msgr2=0x7f2cdc06e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6816/2184210851,v1:172.21.15.86:6817/2184210851] conn(0x7f2cdc06c2d0 0x7f2cdc06e6f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:17.983 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cdc066470 msgr2=0x7f2cdc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f2cdc066470 0x7f2cdc068930 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4000f70 tx=0x7f2cf4009040 comp rx=0 tx=0).stop 2023-10-26T05:59:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf80cf2f0 msgr2=0x7f2cf80c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:17.984 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 --2- 172.21.15.71:0/1387875189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cf80cf2f0 0x7f2cf80c87d0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009790 tx=0x7f2ce80a5920 comp rx=0 tx=0).stop 2023-10-26T05:59:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cfd9c1640 1 -- 172.21.15.71:0/1387875189 reap_dead start 2023-10-26T05:59:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 shutdown_connections 2023-10-26T05:59:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 >> 172.21.15.71:0/1387875189 conn(0x7f2cf80d13f0 msgr2=0x7f2cf815fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:17.985 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 shutdown_connections 2023-10-26T05:59:17.986 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:17.971+0000 7f2cff424640 1 -- 172.21.15.71:0/1387875189 wait complete. 2023-10-26T05:59:17.998 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-26T05:59:18.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- 172.21.15.71:0/3008348631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 msgr2=0x7fe3340cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.120 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3008348631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe3340cb8c0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fe324009840 tx=0x7fe32401c740 comp rx=0 tx=0).stop 2023-10-26T05:59:18.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- 172.21.15.71:0/3008348631 shutdown_connections 2023-10-26T05:59:18.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3008348631 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe33413de80 0x7fe33406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3008348631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe3340cb8c0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- 172.21.15.71:0/3008348631 >> 172.21.15.71:0/3008348631 conn(0x7fe33406b8c0 msgr2=0x7fe33406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:18.121 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- 172.21.15.71:0/3008348631 shutdown_connections 2023-10-26T05:59:18.122 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- 172.21.15.71:0/3008348631 wait complete. 2023-10-26T05:59:18.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 Processor -- start 2023-10-26T05:59:18.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- start start 2023-10-26T05:59:18.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe33406f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.124 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe33413de80 0x7fe334134c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe334159060 0x7fe33414f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.111+0000 7fe338a8c640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe334136030 con 0x7fe33413ae90 2023-10-26T05:59:18.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe338a8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33414ae90 con 0x7fe334159060 2023-10-26T05:59:18.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe338a8c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341314c0 con 0x7fe33413de80 2023-10-26T05:59:18.125 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe33406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe33406f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39164/0 (socket says 172.21.15.71:39164) 2023-10-26T05:59:18.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 -- 172.21.15.71:0/3247117629 learned_addr learned my addr 172.21.15.71:0/3247117629 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:18.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe332d9d640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe334159060 0x7fe33414f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.126 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 -- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe33413de80 msgr2=0x7fe334134c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe33413de80 0x7fe334134c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 -- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe334159060 msgr2=0x7fe33414f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe334159060 0x7fe33414f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 -- 172.21.15.71:0/3247117629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3240094a0 con 0x7fe33413ae90 2023-10-26T05:59:18.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe33259c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe33406f680 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fe324009790 tx=0x7fe3240a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:18.127 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240ba070 con 0x7fe33413ae90 2023-10-26T05:59:18.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe324004230 con 0x7fe33413ae90 2023-10-26T05:59:18.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe334131740 con 0x7fe33413ae90 2023-10-26T05:59:18.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324003600 con 0x7fe33413ae90 2023-10-26T05:59:18.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe334136520 con 0x7fe33413ae90 2023-10-26T05:59:18.128 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe334066b10 con 0x7fe33413ae90 2023-10-26T05:59:18.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe3240b4040 con 0x7fe33413ae90 2023-10-26T05:59:18.129 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe322ffd640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe314066470 0x7fe314068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe32411dba0 con 0x7fe33413ae90 2023-10-26T05:59:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe331d9b640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe314066470 0x7fe314068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.115+0000 7fe331d9b640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe314066470 0x7fe314068930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe328000f70 tx=0x7fe328007a40 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:18.130 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.119+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7fe3240eb330 con 0x7fe33413ae90 2023-10-26T05:59:18.279 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.271+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe334070a10 con 0x7fe314066470 2023-10-26T05:59:18.280 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.271+0000 7fe322ffd640 1 -- 172.21.15.71:0/3247117629 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe328014400 con 0x7fe314066470 2023-10-26T05:59:18.280 INFO:teuthology.orchestra.run.smithi071.stderr:instructed osd(s) 7 to deep-scrub 2023-10-26T05:59:18.281 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.271+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe314066470 msgr2=0x7fe314068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.271+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe314066470 0x7fe314068930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe328000f70 tx=0x7fe328007a40 comp rx=0 tx=0).stop 2023-10-26T05:59:18.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.271+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 msgr2=0x7fe33406f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.282 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.271+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe33406f680 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fe324009790 tx=0x7fe3240a5820 comp rx=0 tx=0).stop 2023-10-26T05:59:18.284 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 shutdown_connections 2023-10-26T05:59:18.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe334159060 0x7fe33414f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe33413de80 0x7fe334134c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7fe314066470 0x7fe314068930 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.285 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 --2- 172.21.15.71:0/3247117629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe33413ae90 0x7fe33406f680 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 >> 172.21.15.71:0/3247117629 conn(0x7fe33406b8c0 msgr2=0x7fe334152150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:18.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 shutdown_connections 2023-10-26T05:59:18.286 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.275+0000 7fe338a8c640 1 -- 172.21.15.71:0/3247117629 wait complete. 2023-10-26T05:59:18.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:59:18.412 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 -- 172.21.15.71:0/71485179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc15bc50 msgr2=0x7f3edc15c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/71485179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc15bc50 0x7f3edc15c030 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0719c0 tx=0x7f3ed801c8a0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 -- 172.21.15.71:0/71485179 shutdown_connections 2023-10-26T05:59:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/71485179 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc0b1f10 0x7f3edc1416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.413 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/71485179 >> v2:172.21.15.86:3300/0 conn(0x7f3edc0b1a60 0x7f3edc141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/71485179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc15bc50 0x7f3edc15c030 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 -- 172.21.15.71:0/71485179 >> 172.21.15.71:0/71485179 conn(0x7f3edc0caa90 msgr2=0x7f3edc0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 -- 172.21.15.71:0/71485179 shutdown_connections 2023-10-26T05:59:18.414 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 -- 172.21.15.71:0/71485179 wait complete. 2023-10-26T05:59:18.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 Processor -- start 2023-10-26T05:59:18.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.403+0000 7f3ee46ff640 1 -- start start 2023-10-26T05:59:18.417 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3edc0b1a60 0x7f3edc13a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 0x7f3edc150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc15bc50 0x7f3edc151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1398d0 con 0x7f3edc0b1f10 2023-10-26T05:59:18.418 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc139bb0 con 0x7f3edc0b1a60 2023-10-26T05:59:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc139e90 con 0x7f3edc15bc50 2023-10-26T05:59:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 0x7f3edc150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 0x7f3edc150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39182/0 (socket says 172.21.15.71:39182) 2023-10-26T05:59:18.419 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 -- 172.21.15.71:0/2050544749 learned_addr learned my addr 172.21.15.71:0/2050544749 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:18.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee249b640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3edc0b1a60 0x7f3edc13a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee2c9c640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc15bc50 0x7f3edc151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 -- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc15bc50 msgr2=0x7f3edc151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc15bc50 0x7f3edc151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.420 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 -- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3edc0b1a60 msgr2=0x7f3edc13a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3edc0b1a60 0x7f3edc13a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 -- 172.21.15.71:0/2050544749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8009560 con 0x7f3edc0b1f10 2023-10-26T05:59:18.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee249b640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3edc0b1a60 0x7f3edc13a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:18.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee2c9c640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc15bc50 0x7f3edc151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:18.421 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee1c9a640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 0x7f3edc150f50 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b410 tx=0x7f3ecc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:18.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc011830 con 0x7f3edc0b1f10 2023-10-26T05:59:18.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f3ecc011a80 con 0x7f3edc0b1f10 2023-10-26T05:59:18.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc011e90 con 0x7f3edc0b1f10 2023-10-26T05:59:18.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc13a170 con 0x7f3edc0b1f10 2023-10-26T05:59:18.422 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc13ac50 con 0x7f3edc0b1f10 2023-10-26T05:59:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc066d00 con 0x7f3edc0b1f10 2023-10-26T05:59:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3ecc01a400 con 0x7f3edc0b1f10 2023-10-26T05:59:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ed2ffd640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ec00664a0 0x7f3ec0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ecc084060 con 0x7f3edc0b1f10 2023-10-26T05:59:18.423 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.407+0000 7f3ee249b640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ec00664a0 0x7f3ec0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.411+0000 7f3ee249b640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ec00664a0 0x7f3ec0068960 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed801b0e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:18.424 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.411+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f3ecc051730 con 0x7f3edc0b1f10 2023-10-26T05:59:18.568 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.559+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3edc1586d0 con 0x7f3ec00664a0 2023-10-26T05:59:18.575 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ed2ffd640 1 -- 172.21.15.71:0/2050544749 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48657 (secure 0 0 0) 0x7f3ed80a5640 con 0x7f3ec00664a0 2023-10-26T05:59:18.576 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:18.576 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:59:18.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ec00664a0 msgr2=0x7f3ec0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.577 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ec00664a0 0x7f3ec0068960 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed801b0e0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 msgr2=0x7f3edc150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:18.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 0x7f3edc150f50 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00b410 tx=0x7f3ecc00b8d0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 shutdown_connections 2023-10-26T05:59:18.578 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3edc15bc50 0x7f3edc151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f3ec00664a0 0x7f3ec0068960 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3edc0b1f10 0x7f3edc150f50 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 --2- 172.21.15.71:0/2050544749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3edc0b1a60 0x7f3edc13a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:18.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 >> 172.21.15.71:0/2050544749 conn(0x7f3edc0caa90 msgr2=0x7f3edc163d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:18.579 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 shutdown_connections 2023-10-26T05:59:18.580 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:18.567+0000 7f3ee46ff640 1 -- 172.21.15.71:0/2050544749 wait complete. 2023-10-26T05:59:18.593 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":551,"stamp":"2023-10-26T05:59:17.653562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9775,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552724,"statfs":{"total":1600000000,"available":1589989992,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52428800,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-84992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.123440"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":328,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-26T05:59:10.547786+0000","last_change":"2023-10-26T05:47:11.346856+0000","last_active":"2023-10-26T05:59:10.547786+0000","last_peered":"2023-10-26T05:59:10.547786+0000","last_clean":"2023-10-26T05:59:10.547786+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:59:10.547786+0000","last_undegraded":"2023-10-26T05:59:10.547786+0000","last_fullsized":"2023-10-26T05:59:10.547786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-26T05:47:10.303938+0000","last_clean_scrub_stamp":"2023-10-26T05:47:10.303938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T12:50:24.729596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193658,"statfs":{"total":200000000,"available":198305973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.39600000000000002},"max":{"1min":1.2290000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.36399999999999999},"max":{"1min":1.125,"5min":1.196,"15min":1.3100000000000001},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.98599999999999999,"15min":0.997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":1.387,"15min":1.4319999999999999},"last":0.751},{"interface":"front","average":{"1min":1.002,"5min":1.0609999999999999,"15min":1.0009999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.621,"15min":1.621},"last":0.97699999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":0.85299999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.40799999999999997},"max":{"1min":1.3,"5min":1.448,"15min":1.448},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.51500000000000001},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.4330000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":0.97599999999999998},"min":{"1min":0.76200000000000001,"5min":0.61599999999999999,"15min":0.52100000000000002},"max":{"1min":1.333,"5min":1.712,"15min":1.712},"last":0.54500000000000004}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.59599999999999997,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.2190000000000001,"5min":1.2589999999999999,"15min":1.639},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.70599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.4159999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.999,"15min":1.0589999999999999},"min":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.335,"5min":1.482,"15min":1.6679999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.03,"15min":1.0529999999999999},"min":{"1min":0.66700000000000004,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.383,"5min":1.504,"15min":1.702},"last":1.1519999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.087,"15min":1.099},"min":{"1min":0.86199999999999999,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.302,"5min":1.3640000000000001,"15min":1.7509999999999999},"last":1.103},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.069,"15min":1.0549999999999999},"min":{"1min":0.77100000000000002,"5min":0.66700000000000004,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.5,"15min":1.6180000000000001},"last":1.0269999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.84099999999999997},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90000000000000002,"15min":0.872},"min":{"1min":0.503,"5min":0.49299999999999999,"15min":0.376},"max":{"1min":1.286,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.83799999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.85199999999999998,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.39100000000000001},"max":{"1min":1.071,"5min":1.1120000000000001,"15min":1.2609999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.91700000000000004,"15min":0.874},"min":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.41999999999999998},"max":{"1min":1.157,"5min":1.3899999999999999,"15min":1.405},"last":0.63200000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89800000000000002,"15min":0.85699999999999998},"min":{"1min":0.66300000000000003,"5min":0.54800000000000004,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.268,"15min":1.8460000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.86499999999999999},"min":{"1min":0.65100000000000002,"5min":0.502,"15min":0.435},"max":{"1min":1.169,"5min":1.3,"15min":1.9239999999999999},"last":1.141}]},{"osd":3,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.028,"15min":0.94999999999999996},"min":{"1min":0.73799999999999999,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0609999999999999,"15min":1.0149999999999999},"min":{"1min":0.79800000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.103}]},{"osd":4,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.502},"max":{"1min":1.4299999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98499999999999999},"min":{"1min":0.68200000000000005,"5min":0.621,"15min":0.442},"max":{"1min":1.375,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.1830000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.075,"15min":1.0509999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.45600000000000002},"max":{"1min":1.4099999999999999,"5min":1.5509999999999999,"15min":1.554},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.36799999999999999},"max":{"1min":1.3260000000000001,"5min":1.452,"15min":1.512},"last":1.0569999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.147,"15min":1.1180000000000001},"min":{"1min":1.0029999999999999,"5min":0.71599999999999997,"15min":0.41099999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.4550000000000001,"15min":1.5920000000000001},"last":1.2190000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96499999999999997,"15min":0.95499999999999996},"min":{"1min":0.48699999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.264,"5min":1.381,"15min":1.579},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.012,"15min":0.96899999999999997},"min":{"1min":0.68100000000000005,"5min":0.58799999999999997,"15min":0.499},"max":{"1min":1.1499999999999999,"5min":1.591,"15min":1.591},"last":0.79800000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85399999999999998,"15min":0.872},"min":{"1min":0.52000000000000002,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.214,"5min":1.214,"15min":2.1150000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88600000000000001,"15min":0.86499999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.93600000000000005,"15min":0.91100000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.21,"5min":1.569,"15min":1.569},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89900000000000002,"15min":0.92000000000000004},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2569999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":0.755}]},{"osd":3,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92500000000000004,"15min":0.91700000000000004},"min":{"1min":0.57699999999999996,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.1100000000000001,"5min":1.363,"15min":1.548},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.71099999999999997,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.335,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0880000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.93100000000000005,"15min":0.96599999999999997},"min":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.46999999999999997},"max":{"1min":1.373,"5min":1.373,"15min":2.1909999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1120000000000001,"15min":1.0940000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40799999999999997},"max":{"1min":1.4099999999999999,"5min":1.4950000000000001,"15min":1.536},"last":1.2270000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.02,"15min":1.0049999999999999},"min":{"1min":0.60199999999999998,"5min":0.50700000000000001,"15min":0.41899999999999998},"max":{"1min":1.2869999999999999,"5min":1.744,"15min":1.744},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.056,"15min":1.032},"min":{"1min":0.71299999999999997,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.3160000000000001,"5min":1.4199999999999999,"15min":1.4950000000000001},"last":1.127}]},{"osd":7,"last update":"Thu Oct 26 05:58:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0980000000000001,"15min":1.075},"min":{"1min":0.754,"5min":0.53600000000000003,"15min":0.442},"max":{"1min":1.4810000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.3},{"interface":"front","average":{"1min":1.153,"5min":1.1799999999999999,"15min":1.155},"min":{"1min":0.86099999999999999,"5min":0.61899999999999999,"15min":0.49299999999999999},"max":{"1min":1.3480000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.268}]}]},{"osd":4,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.2390000000000001,"5min":1.248,"15min":1.347},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.47199999999999998},"max":{"1min":1.1299999999999999,"5min":1.234,"15min":1.502},"last":1.05}]},{"osd":1,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.85099999999999998},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.375},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.95299999999999996,"15min":0.92400000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.1839999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.872}]},{"osd":2,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93200000000000005,"15min":0.86499999999999999},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.65300000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.149,"5min":1.222,"15min":1.373},"last":1.0920000000000001}]},{"osd":3,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.96999999999999997,"15min":0.94799999999999995},"min":{"1min":0.63300000000000001,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.2649999999999999,"5min":1.397,"15min":2.0190000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.02,"15min":0.97299999999999998},"min":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.58899999999999997},"max":{"1min":1.2030000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2450000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.091,"5min":1.022,"15min":1.0049999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.673},"last":1.1599999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.59099999999999997,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.3759999999999999,"5min":1.762,"15min":1.7969999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.069,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.70699999999999996,"5min":0.67100000000000004,"15min":0.51600000000000001},"max":{"1min":1.3120000000000001,"5min":1.355,"15min":1.4970000000000001},"last":0.97099999999999997}]},{"osd":7,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.117,"15min":1.0669999999999999},"min":{"1min":0.81299999999999994,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.4610000000000001,"5min":1.659,"15min":1.7509999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1499999999999999,"15min":1.1279999999999999},"min":{"1min":0.76000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.484,"15min":1.728},"last":1.2110000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.999,"5min":0.999,"15min":1.385},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.60399999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.88,"5min":1.0600000000000001,"15min":1.2689999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.68400000000000005,"15min":0.65800000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.93700000000000006,"5min":6.109,"15min":6.109},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.63500000000000001},"min":{"1min":0.35599999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.83799999999999997,"5min":1.02,"15min":1.1399999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.77500000000000002},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88900000000000001,"15min":0.81699999999999995},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.125,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.52900000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.86199999999999999,"15min":0.85099999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.40000000000000002},"max":{"1min":1.026,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.91700000000000004,"15min":0.91900000000000004},"min":{"1min":0.64700000000000002,"5min":0.57499999999999996,"15min":0.52400000000000002},"max":{"1min":1.1830000000000001,"5min":1.429,"15min":1.4810000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.93899999999999995,"15min":0.95499999999999996},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.2130000000000001,"5min":1.454,"15min":1.478},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.94599999999999995,"15min":0.92800000000000005},"min":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.1619999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0149999999999999,"15min":0.98899999999999999},"min":{"1min":0.65800000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.2629999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.024,"5min":1.002,"15min":1.0209999999999999},"min":{"1min":0.57599999999999996,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":1.286,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98099999999999998,"15min":0.95599999999999996},"min":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.2410000000000001,"5min":2.016,"15min":2.016},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0680000000000001,"15min":1.046},"min":{"1min":0.70799999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3120000000000001,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.88600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.628,"15min":0.57999999999999996},"min":{"1min":0.35399999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":1.097},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.70099999999999996,"15min":0.65500000000000003},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.26000000000000001},"max":{"1min":1.006,"5min":1.1619999999999999,"15min":1.5249999999999999},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.70699999999999996},"min":{"1min":0.35399999999999998,"5min":0.248,"15min":0.248},"max":{"1min":1.0669999999999999,"5min":1.198,"15min":1.2609999999999999},"last":0.498},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.42099999999999999,"5min":0.30199999999999999,"15min":0.26700000000000002},"max":{"1min":0.96599999999999997,"5min":1.157,"15min":1.3600000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.81200000000000006,"15min":0.77600000000000002},"min":{"1min":0.373,"5min":0.28399999999999997,"15min":0.28299999999999997},"max":{"1min":1.008,"5min":1.24,"15min":2.8809999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72599999999999998,"15min":0.65800000000000003},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.3440000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90500000000000003,"15min":0.876},"min":{"1min":0.625,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.149,"5min":1.177,"15min":2.6139999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.88300000000000001,"15min":0.871},"min":{"1min":0.73799999999999999,"5min":0.54400000000000004,"15min":0.51800000000000002},"max":{"1min":1.1259999999999999,"5min":1.1459999999999999,"15min":5.9550000000000001},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.97899999999999998,"15min":0.94999999999999996},"min":{"1min":0.628,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.2090000000000001,"5min":1.5289999999999999,"15min":2.8319999999999999},"last":1.01},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.96099999999999997,"15min":0.94099999999999995},"min":{"1min":0.64200000000000002,"5min":0.57799999999999996,"15min":0.51700000000000002},"max":{"1min":1.2989999999999999,"5min":1.329,"15min":1.653},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.98099999999999998,"15min":0.95999999999999996},"min":{"1min":0.68999999999999995,"5min":0.54100000000000004,"15min":0.54000000000000004},"max":{"1min":1.385,"5min":1.385,"15min":2.5760000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0489999999999999,"15min":1.0309999999999999},"min":{"1min":0.65400000000000003,"5min":0.57199999999999995,"15min":0.5},"max":{"1min":1.274,"5min":1.486,"15min":6.7729999999999997},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.93200000000000005},"min":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.581},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0700000000000001,"15min":1.083},"min":{"1min":0.64900000000000002,"5min":0.52000000000000002,"15min":0.51800000000000002},"max":{"1min":1.319,"5min":1.5760000000000001,"15min":15.352},"last":1.071}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.88100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.996,"5min":1.089,"15min":1.157},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.752},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.27700000000000002},"max":{"1min":1.1579999999999999,"5min":1.492,"15min":1.8169999999999999},"last":0.94599999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.42399999999999999,"5min":0.32100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.7949999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.298},"last":0.68899999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.238,"15min":1.5780000000000001},"last":0.875},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.67000000000000004,"5min":0.57799999999999996,"15min":0.52100000000000002},"max":{"1min":1.292,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96099999999999997,"15min":0.96199999999999997},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.52700000000000002},"max":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.492},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63800000000000001,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.1919999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":1.002}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97899999999999998,"15min":1.012},"min":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.56100000000000005},"max":{"1min":1.202,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0109999999999999,"15min":0.98099999999999998},"min":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.393,"15min":1.837},"last":1.115}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.046,"15min":1.0289999999999999},"min":{"1min":0.69699999999999995,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.429,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.147},{"interface":"front","average":{"1min":1.028,"5min":1.046,"15min":1.0740000000000001},"min":{"1min":0.62,"5min":0.60899999999999999,"15min":0.56599999999999995},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":5.2430000000000003},"last":1.0509999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193658,"statfs":{"total":200000000,"available":198305973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.92400000000000004,"5min":0.96599999999999997,"15min":1.0349999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.091,"5min":1.2270000000000001,"15min":2.8969999999999998},"last":0.79800000000000004}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.73399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.28399999999999997},"max":{"1min":1.1699999999999999,"5min":1.2010000000000001,"15min":2.7530000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.26300000000000001},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.749}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.409},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.73799999999999999},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.125,"5min":1.2629999999999999,"15min":2.6509999999999998},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90500000000000003,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.2749999999999999,"5min":1.395,"15min":1.395},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.94299999999999995},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1950000000000001,"5min":1.2769999999999999,"15min":2.8290000000000002},"last":0.86799999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93400000000000005,"15min":0.92800000000000005},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.7480000000000002},"last":1.02},{"interface":"front","average":{"1min":1.002,"5min":0.95999999999999996,"15min":0.93200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.6819999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.129},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.014,"15min":1.0269999999999999},"min":{"1min":0.69599999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":1.353,"15min":6.4690000000000003},"last":1.0029999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99199999999999999,"15min":0.996},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":1.417,"15min":1.5660000000000001},"last":0.93300000000000005}]}]}],"pool_statfs":[]}} 2023-10-26T05:59:18.597 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-26T05:47:10.303938+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=26, tm_hour=5, tm_min=47, tm_sec=10, tm_wday=3, tm_yday=299, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=26, tm_hour=5, tm_min=59, tm_sec=13, tm_wday=3, tm_yday=299, tm_isdst=0) 2023-10-26T05:59:18.598 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-26T05:59:38.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-26T05:59:38.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2106938539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0850 msgr2=0x7f929415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:38.735 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 --2- 172.21.15.71:0/2106938539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0850 0x7f929415b3a0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f927c009900 tx=0x7f927c01c730 comp rx=0 tx=0).stop 2023-10-26T05:59:38.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2106938539 shutdown_connections 2023-10-26T05:59:38.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 --2- 172.21.15.71:0/2106938539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0850 0x7f929415b3a0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 --2- 172.21.15.71:0/2106938539 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92940d0470 0x7f929406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2106938539 >> 172.21.15.71:0/2106938539 conn(0x7f92940cdd80 msgr2=0x7f92940ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:38.736 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2106938539 shutdown_connections 2023-10-26T05:59:38.737 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2106938539 wait complete. 2023-10-26T05:59:38.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 Processor -- start 2023-10-26T05:59:38.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- start start 2023-10-26T05:59:38.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92940cd880 0x7f9294159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:38.739 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 0x7f9294148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:38.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92940d0850 0x7f9294149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:38.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929415cf60 con 0x7f92940d0470 2023-10-26T05:59:38.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929415d240 con 0x7f92940cd880 2023-10-26T05:59:38.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929415d520 con 0x7f92940d0850 2023-10-26T05:59:38.740 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 0x7f9294148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 0x7f9294148a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57240/0 (socket says 172.21.15.71:57240) 2023-10-26T05:59:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 -- 172.21.15.71:0/2255232273 learned_addr learned my addr 172.21.15.71:0/2255232273 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92937fe640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92940d0850 0x7f9294149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f9292ffd640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92940cd880 0x7f9294159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:38.741 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 -- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92940d0850 msgr2=0x7f9294149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92940d0850 0x7f9294149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 -- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92940cd880 msgr2=0x7f9294159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92940cd880 0x7f9294159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 -- 172.21.15.71:0/2255232273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009560 con 0x7f92940d0470 2023-10-26T05:59:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f92927fc640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 0x7f9294148a90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f927c009850 tx=0x7f927c0a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c004240 con 0x7f92940d0470 2023-10-26T05:59:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f927c004490 con 0x7f92940d0470 2023-10-26T05:59:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929415d800 con 0x7f92940d0470 2023-10-26T05:59:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c005280 con 0x7f92940d0470 2023-10-26T05:59:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.726+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92941598d0 con 0x7f92940d0470 2023-10-26T05:59:38.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.730+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9294066b10 con 0x7f92940d0470 2023-10-26T05:59:38.744 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.730+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f927c0b4040 con 0x7f92940d0470 2023-10-26T05:59:38.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.730+0000 7f928b7fe640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9274066470 0x7f9274068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:38.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.730+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f927c11dbb0 con 0x7f92940d0470 2023-10-26T05:59:38.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.730+0000 7f9292ffd640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9274066470 0x7f9274068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:38.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.730+0000 7f9292ffd640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9274066470 0x7f9274068930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f928c005c50 tx=0x7f928c005be0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:38.745 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.734+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f927c0eb3c0 con 0x7f92940d0470 2023-10-26T05:59:38.890 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.878+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 --> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f929415d3b0 con 0x7f9274066470 2023-10-26T05:59:38.895 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f928b7fe640 1 -- 172.21.15.71:0/2255232273 <== mgr.4112 v2:172.21.15.86:6832/1120505646 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48625 (secure 0 0 0) 0x7f929415d3b0 con 0x7f9274066470 2023-10-26T05:59:38.896 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:38.897 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2023-10-26T05:59:38.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9274066470 msgr2=0x7f9274068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:38.898 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9274066470 0x7f9274068930 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f928c005c50 tx=0x7f928c005be0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 msgr2=0x7f9294148a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 0x7f9294148a90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f927c009850 tx=0x7f927c0a56a0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 shutdown_connections 2023-10-26T05:59:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92940d0850 0x7f9294149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.899 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f9274066470 0x7f9274068930 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92940d0470 0x7f9294148a90 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 --2- 172.21.15.71:0/2255232273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92940cd880 0x7f9294159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:38.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 >> 172.21.15.71:0/2255232273 conn(0x7f92940cdd80 msgr2=0x7f92941582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:38.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 shutdown_connections 2023-10-26T05:59:38.900 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:38.890+0000 7f929943b640 1 -- 172.21.15.71:0/2255232273 wait complete. 2023-10-26T05:59:38.913 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":561,"stamp":"2023-10-26T05:59:37.659116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9775,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552724,"statfs":{"total":1600000000,"available":1589989992,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003321"},"pg_stats":[{"pgid":"1.0","version":"195'119","reported_seq":339,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-10-26T05:59:20.568027+0000","last_change":"2023-10-26T05:59:20.568206+0000","last_active":"2023-10-26T05:59:20.568027+0000","last_peered":"2023-10-26T05:59:20.568027+0000","last_clean":"2023-10-26T05:59:20.568027+0000","last_became_active":"2023-10-26T05:47:11.346290+0000","last_became_peered":"2023-10-26T05:47:11.346290+0000","last_unstale":"2023-10-26T05:59:20.568027+0000","last_undegraded":"2023-10-26T05:59:20.568027+0000","last_fullsized":"2023-10-26T05:59:20.568027+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2023-10-26T05:59:20.567881+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2023-10-26T05:59:20.567881+0000","last_clean_scrub_stamp":"2023-10-26T05:59:20.567881+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-27T10:24:44.675802+0000","scrub_duration":0.0057618280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193658,"statfs":{"total":200000000,"available":198305973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89600000000000002,"15min":0.86099999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.39600000000000002},"max":{"1min":1.2290000000000001,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.36399999999999999},"max":{"1min":1.125,"5min":1.196,"15min":1.3100000000000001},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.98599999999999999,"15min":0.997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":1.387,"15min":1.4319999999999999},"last":1.137},{"interface":"front","average":{"1min":1.002,"5min":1.0609999999999999,"15min":1.0009999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.621,"15min":1.621},"last":1.113}]},{"osd":2,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90600000000000003,"15min":0.85299999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.40799999999999997},"max":{"1min":1.3,"5min":1.448,"15min":1.448},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92200000000000004,"15min":0.92600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.2909999999999999,"15min":1.4590000000000001},"last":1.2509999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.95299999999999996,"15min":0.90700000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.51500000000000001},"max":{"1min":1.2729999999999999,"5min":1.2729999999999999,"15min":1.4330000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0029999999999999,"15min":0.97599999999999998},"min":{"1min":0.76200000000000001,"5min":0.61599999999999999,"15min":0.52100000000000002},"max":{"1min":1.333,"5min":1.712,"15min":1.712},"last":0.86799999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89800000000000002,"15min":0.91100000000000003},"min":{"1min":0.59599999999999997,"5min":0.46100000000000002,"15min":0.433},"max":{"1min":1.2190000000000001,"5min":1.2589999999999999,"15min":1.639},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.93400000000000005,"15min":0.94199999999999995},"min":{"1min":0.70599999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.4159999999999999},"last":0.96199999999999997}]},{"osd":5,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":0.999,"15min":1.0589999999999999},"min":{"1min":0.65500000000000003,"5min":0.60899999999999999,"15min":0.58699999999999997},"max":{"1min":1.335,"5min":1.482,"15min":1.6679999999999999},"last":1.032},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.03,"15min":1.0529999999999999},"min":{"1min":0.66700000000000004,"5min":0.55300000000000005,"15min":0.53800000000000003},"max":{"1min":1.383,"5min":1.504,"15min":1.702},"last":1.2809999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:31 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.087,"15min":1.099},"min":{"1min":0.86199999999999999,"5min":0.68999999999999995,"15min":0.59599999999999997},"max":{"1min":1.302,"5min":1.3640000000000001,"15min":1.7509999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.069,"15min":1.0549999999999999},"min":{"1min":0.77100000000000002,"5min":0.66700000000000004,"15min":0.46200000000000002},"max":{"1min":1.3819999999999999,"5min":1.5,"15min":1.6180000000000001},"last":1.208}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88300000000000001,"15min":0.84099999999999997},"min":{"1min":0.52500000000000002,"5min":0.47699999999999998,"15min":0.46000000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.90000000000000002,"15min":0.872},"min":{"1min":0.503,"5min":0.49299999999999999,"15min":0.376},"max":{"1min":1.286,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.038}]},{"osd":1,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.85199999999999998,"15min":0.81299999999999994},"min":{"1min":0.54800000000000004,"5min":0.51700000000000002,"15min":0.39100000000000001},"max":{"1min":1.071,"5min":1.1120000000000001,"15min":1.2609999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.91700000000000004,"15min":0.874},"min":{"1min":0.63300000000000001,"5min":0.55400000000000005,"15min":0.41999999999999998},"max":{"1min":1.157,"5min":1.3899999999999999,"15min":1.405},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.89800000000000002,"15min":0.85699999999999998},"min":{"1min":0.66300000000000003,"5min":0.54800000000000004,"15min":0.32800000000000001},"max":{"1min":1.1970000000000001,"5min":1.268,"15min":1.8460000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88100000000000001,"15min":0.86499999999999999},"min":{"1min":0.65100000000000002,"5min":0.502,"15min":0.435},"max":{"1min":1.169,"5min":1.3,"15min":1.9239999999999999},"last":1.0629999999999999}]},{"osd":3,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.028,"15min":0.94999999999999996},"min":{"1min":0.73799999999999999,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.879},{"interface":"front","average":{"1min":1.073,"5min":1.0609999999999999,"15min":1.0149999999999999},"min":{"1min":0.79800000000000004,"5min":0.54000000000000004,"15min":0.502},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.246}]},{"osd":4,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.96699999999999997,"15min":0.95599999999999996},"min":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.502},"max":{"1min":1.4299999999999999,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.004,"5min":0.99399999999999999,"15min":0.98499999999999999},"min":{"1min":0.68200000000000005,"5min":0.621,"15min":0.442},"max":{"1min":1.375,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.075,"15min":1.0509999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.45600000000000002},"max":{"1min":1.4099999999999999,"5min":1.5509999999999999,"15min":1.554},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.36799999999999999},"max":{"1min":1.3260000000000001,"5min":1.452,"15min":1.512},"last":0.95399999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:58:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.147,"15min":1.1180000000000001},"min":{"1min":1.0029999999999999,"5min":0.71599999999999997,"15min":0.41099999999999998},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.41699999999999998},"max":{"1min":1.3740000000000001,"5min":1.4550000000000001,"15min":1.5920000000000001},"last":1.113}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.96999999999999997,"15min":0.95199999999999996},"min":{"1min":0.5,"5min":0.48699999999999999,"15min":0.47599999999999998},"max":{"1min":1.238,"5min":1.3640000000000001,"15min":1.579},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.0169999999999999,"15min":0.97099999999999997},"min":{"1min":0.69099999999999995,"5min":0.58799999999999997,"15min":0.499},"max":{"1min":1.179,"5min":1.5089999999999999,"15min":1.591},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.876,"15min":0.872},"min":{"1min":0.60299999999999998,"5min":0.51800000000000002,"15min":0.441},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":2.1150000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.871,"15min":0.86199999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1120000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.65400000000000003}]},{"osd":2,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.92500000000000004,"15min":0.90900000000000003},"min":{"1min":0.57399999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.248,"5min":1.569,"15min":1.569},"last":1.129},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92400000000000004,"15min":0.92200000000000004},"min":{"1min":0.755,"5min":0.53400000000000003,"15min":0.51900000000000002},"max":{"1min":1.2829999999999999,"5min":1.2849999999999999,"15min":1.3160000000000001},"last":0.92500000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.91200000000000003,"15min":0.91400000000000003},"min":{"1min":0.59299999999999997,"5min":0.52300000000000002,"15min":0.502},"max":{"1min":1.1830000000000001,"5min":1.363,"15min":1.548},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.998,"15min":0.98799999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.55600000000000005},"max":{"1min":1.232,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.84199999999999997}]},{"osd":4,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.94099999999999995,"15min":0.96399999999999997},"min":{"1min":0.76000000000000001,"5min":0.59299999999999997,"15min":0.46999999999999997},"max":{"1min":1.194,"5min":1.373,"15min":2.1909999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.111,"15min":1.0920000000000001},"min":{"1min":0.747,"5min":0.50700000000000001,"15min":0.40799999999999997},"max":{"1min":1.2889999999999999,"5min":1.47,"15min":1.536},"last":1.0169999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.0009999999999999},"min":{"1min":0.63800000000000001,"5min":0.50700000000000001,"15min":0.41899999999999998},"max":{"1min":1.319,"5min":1.4590000000000001,"15min":1.744},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.05,"15min":1.0329999999999999},"min":{"1min":0.77400000000000002,"5min":0.56699999999999995,"15min":0.504},"max":{"1min":1.2809999999999999,"5min":1.3620000000000001,"15min":1.4950000000000001},"last":1.052}]},{"osd":7,"last update":"Thu Oct 26 05:59:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.091,"15min":1.073},"min":{"1min":0.72299999999999998,"5min":0.53600000000000003,"15min":0.442},"max":{"1min":1.3,"5min":1.4810000000000001,"15min":1.7789999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1699999999999999,"15min":1.153},"min":{"1min":0.83599999999999997,"5min":0.61899999999999999,"15min":0.49299999999999999},"max":{"1min":1.3799999999999999,"5min":1.6299999999999999,"15min":1.8939999999999999},"last":1.214}]}]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.46400000000000002},"max":{"1min":1.2390000000000001,"5min":1.248,"15min":1.347},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.59799999999999998,"5min":0.53100000000000003,"15min":0.47199999999999998},"max":{"1min":1.1299999999999999,"5min":1.234,"15min":1.502},"last":0.99299999999999999}]},{"osd":1,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.85099999999999998},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.46899999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.375},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.874,"5min":0.95299999999999996,"15min":0.92400000000000004},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.52300000000000002},"max":{"1min":1.1839999999999999,"5min":1.3959999999999999,"15min":1.4390000000000001},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.93200000000000005,"15min":0.86499999999999999},"min":{"1min":0.64100000000000001,"5min":0.54600000000000004,"15min":0.501},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.91200000000000003,"15min":0.88800000000000001},"min":{"1min":0.65300000000000002,"5min":0.52000000000000002,"15min":0.49199999999999999},"max":{"1min":1.149,"5min":1.222,"15min":1.373},"last":0.94899999999999995}]},{"osd":3,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.96999999999999997,"15min":0.94799999999999995},"min":{"1min":0.63300000000000001,"5min":0.60199999999999998,"15min":0.54100000000000004},"max":{"1min":1.2649999999999999,"5min":1.397,"15min":2.0190000000000001},"last":1.03},{"interface":"front","average":{"1min":1.018,"5min":1.02,"15min":0.97299999999999998},"min":{"1min":0.68999999999999995,"5min":0.64300000000000002,"15min":0.58899999999999997},"max":{"1min":1.2030000000000001,"5min":1.3380000000000001,"15min":1.4359999999999999},"last":1.1559999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.48799999999999999,"15min":0.44800000000000001},"max":{"1min":1.2450000000000001,"5min":1.3340000000000001,"15min":1.3859999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.091,"5min":1.022,"15min":1.0049999999999999},"min":{"1min":0.77400000000000002,"5min":0.59799999999999998,"15min":0.47799999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.673},"last":1.077}]},{"osd":6,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0449999999999999,"15min":1.0169999999999999},"min":{"1min":0.59099999999999997,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":1.3759999999999999,"5min":1.762,"15min":1.7969999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.70699999999999996,"5min":0.67100000000000004,"15min":0.51600000000000001},"max":{"1min":1.3120000000000001,"5min":1.355,"15min":1.4970000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Oct 26 05:58:33 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.117,"15min":1.0669999999999999},"min":{"1min":0.81299999999999994,"5min":0.51800000000000002,"15min":0.46700000000000003},"max":{"1min":1.4610000000000001,"5min":1.659,"15min":1.7509999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1499999999999999,"15min":1.1279999999999999},"min":{"1min":0.76000000000000001,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.4510000000000001,"5min":1.484,"15min":1.728},"last":1.234}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.249},"max":{"1min":0.85499999999999998,"5min":0.999,"15min":1.385},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63500000000000001,"15min":0.60599999999999998},"min":{"1min":0.32600000000000001,"5min":0.28899999999999998,"15min":0.26000000000000001},"max":{"1min":0.85899999999999999,"5min":1.0600000000000001,"15min":1.2689999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.69799999999999995,"15min":0.65400000000000003},"min":{"1min":0.29999999999999999,"5min":0.246,"15min":0.246},"max":{"1min":1.0269999999999999,"5min":6.109,"15min":6.109},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67700000000000005,"15min":0.64100000000000001},"min":{"1min":0.41799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.93000000000000005,"5min":1.02,"15min":1.1399999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.84499999999999997,"15min":0.77700000000000002},"min":{"1min":0.41099999999999998,"5min":0.36599999999999999,"15min":0.30399999999999999},"max":{"1min":1.1859999999999999,"5min":6.1710000000000003,"15min":6.1710000000000003},"last":1.099},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.90200000000000002,"15min":0.81899999999999995},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.29799999999999999},"max":{"1min":1.1479999999999999,"5min":6.0389999999999997,"15min":6.0389999999999997},"last":0.94599999999999995}]},{"osd":4,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.56799999999999995,"5min":0.47599999999999998,"15min":0.40000000000000002},"max":{"1min":1.052,"5min":1.181,"15min":1.9059999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.92600000000000005,"15min":0.91600000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.52400000000000002},"max":{"1min":1.113,"5min":1.429,"15min":1.4810000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94799999999999995,"15min":0.94999999999999996},"min":{"1min":0.58099999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.165,"5min":1.454,"15min":1.478},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":0.92900000000000005},"min":{"1min":0.73299999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"max":{"1min":1.153,"5min":1.54,"15min":2.0910000000000002},"last":0.97199999999999998}]},{"osd":6,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.0209999999999999,"15min":0.97999999999999998},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.149,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.95499999999999996,"5min":1.0089999999999999,"15min":1.0169999999999999},"min":{"1min":0.68600000000000005,"5min":0.51000000000000001,"15min":0.49299999999999999},"max":{"1min":1.216,"5min":2.9220000000000002,"15min":5.6859999999999999},"last":0.997}]},{"osd":7,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.98299999999999998,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.54100000000000004,"15min":0.53400000000000003},"max":{"1min":1.1240000000000001,"5min":1.472,"15min":2.016},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.0780000000000001,"15min":1.0409999999999999},"min":{"1min":0.63200000000000001,"5min":0.60099999999999998,"15min":0.57999999999999996},"max":{"1min":1.256,"5min":4.7300000000000004,"15min":4.7300000000000004},"last":0.90800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.58799999999999997},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.20100000000000001},"max":{"1min":1.0780000000000001,"5min":1.097,"15min":1.097},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70899999999999996,"15min":0.66400000000000003},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.26000000000000001},"max":{"1min":1.1140000000000001,"5min":1.1619999999999999,"15min":1.5249999999999999},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.749,"15min":0.71299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.248},"max":{"1min":1.0429999999999999,"5min":1.1539999999999999,"15min":1.2609999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0149999999999999,"5min":1.0289999999999999,"15min":1.3600000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81100000000000005,"15min":0.78200000000000003},"min":{"1min":0.308,"5min":0.308,"15min":0.28299999999999997},"max":{"1min":1.1080000000000001,"5min":1.24,"15min":2.8809999999999998},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73899999999999999,"15min":0.67300000000000004},"min":{"1min":0.33300000000000002,"5min":0.32500000000000001,"15min":0.27800000000000002},"max":{"1min":1.0469999999999999,"5min":1.1830000000000001,"15min":1.3440000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.91900000000000004,"15min":0.88300000000000001},"min":{"1min":0.63200000000000001,"5min":0.61799999999999999,"15min":0.46200000000000002},"max":{"1min":1.151,"5min":1.177,"15min":2.6139999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.89800000000000002,"15min":0.877},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0980000000000001,"5min":1.1259999999999999,"15min":5.9550000000000001},"last":1.0640000000000001}]},{"osd":5,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.99399999999999999,"15min":0.95999999999999996},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.39100000000000001},"max":{"1min":1.3120000000000001,"5min":1.5289999999999999,"15min":2.8319999999999999},"last":1.036},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.97499999999999998,"15min":0.94799999999999995},"min":{"1min":0.67300000000000004,"5min":0.64200000000000002,"15min":0.51700000000000002},"max":{"1min":1.2869999999999999,"5min":1.329,"15min":1.653},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.998,"15min":0.96699999999999997},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.54000000000000004},"max":{"1min":1.234,"5min":1.385,"15min":2.5760000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.5},"max":{"1min":1.423,"5min":1.486,"15min":6.7729999999999997},"last":1.1639999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95699999999999996,"15min":0.93799999999999994},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.51500000000000001},"max":{"1min":1.2130000000000001,"5min":1.224,"15min":1.581},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.089,"15min":1.095},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.51800000000000002},"max":{"1min":1.3759999999999999,"5min":1.5760000000000001,"15min":15.352},"last":1.2230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194234,"statfs":{"total":200000000,"available":198896341,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59399999999999997,"15min":0.57599999999999996},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.254},"max":{"1min":0.88100000000000001,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67900000000000005,"15min":0.64300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.996,"5min":1.089,"15min":1.157},"last":0.83199999999999996}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67800000000000005,"15min":0.66100000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.29899999999999999},"max":{"1min":0.94999999999999996,"5min":1.2769999999999999,"15min":1.3500000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.79900000000000004,"15min":0.752},"min":{"1min":0.43099999999999999,"5min":0.36099999999999999,"15min":0.27700000000000002},"max":{"1min":1.1579999999999999,"5min":1.492,"15min":1.8169999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.42399999999999999,"5min":0.32100000000000001,"15min":0.29899999999999999},"max":{"1min":1.0720000000000001,"5min":1.147,"15min":1.7949999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70399999999999996,"15min":0.69299999999999995},"min":{"1min":0.35399999999999998,"5min":0.30499999999999999,"15min":0.27700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0760000000000001,"15min":1.298},"last":0.66900000000000004}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.53800000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.0820000000000001,"5min":1.238,"15min":1.5780000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.96699999999999997,"15min":0.95799999999999996},"min":{"1min":0.67000000000000004,"5min":0.57799999999999996,"15min":0.52100000000000002},"max":{"1min":1.292,"5min":1.3999999999999999,"15min":3.7440000000000002},"last":1.1639999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.96099999999999997,"15min":0.96199999999999997},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.52700000000000002},"max":{"1min":1.2729999999999999,"5min":1.4370000000000001,"15min":1.492},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.93899999999999995,"15min":0.93500000000000005},"min":{"1min":0.63800000000000001,"5min":0.54600000000000004,"15min":0.51700000000000002},"max":{"1min":1.1919999999999999,"5min":1.2529999999999999,"15min":1.5349999999999999},"last":1.0289999999999999}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.97899999999999998,"15min":1.012},"min":{"1min":0.59299999999999997,"5min":0.58099999999999996,"15min":0.56100000000000005},"max":{"1min":1.202,"5min":1.4259999999999999,"15min":2.7759999999999998},"last":1.069},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0109999999999999,"15min":0.98099999999999998},"min":{"1min":0.70499999999999996,"5min":0.60099999999999998,"15min":0.56899999999999995},"max":{"1min":1.282,"5min":1.393,"15min":1.837},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.046,"15min":1.0289999999999999},"min":{"1min":0.69699999999999995,"5min":0.58799999999999997,"15min":0.57099999999999995},"max":{"1min":1.429,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.028,"5min":1.046,"15min":1.0740000000000001},"min":{"1min":0.62,"5min":0.60899999999999999,"15min":0.56599999999999995},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":5.2430000000000003},"last":0.995}]}]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193658,"statfs":{"total":200000000,"available":198305973,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57199999999999995,"15min":0.55900000000000005},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.92400000000000004,"5min":0.96599999999999997,"15min":1.0349999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.66800000000000004,"15min":0.65000000000000002},"min":{"1min":0.32800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.091,"5min":1.2270000000000001,"15min":2.8969999999999998},"last":0.74199999999999999}]},{"osd":2,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78600000000000003,"15min":0.73399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.28399999999999997},"max":{"1min":1.1699999999999999,"5min":1.2010000000000001,"15min":2.7530000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.26300000000000001},"max":{"1min":2.3820000000000001,"5min":2.3820000000000001,"15min":2.3820000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68300000000000005,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.32000000000000001,"15min":0.26800000000000002},"max":{"1min":1.22,"5min":1.22,"15min":1.409},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.73799999999999999},"min":{"1min":0.29999999999999999,"5min":0.28899999999999998,"15min":0.247},"max":{"1min":1.125,"5min":1.2629999999999999,"15min":2.6509999999999998},"last":1.0149999999999999}]},{"osd":4,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90500000000000003,"15min":0.88200000000000001},"min":{"1min":0.64100000000000001,"5min":0.51300000000000001,"15min":0.46100000000000002},"max":{"1min":1.2749999999999999,"5min":1.395,"15min":1.395},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95999999999999996,"15min":0.94299999999999995},"min":{"1min":0.51600000000000001,"5min":0.46800000000000003,"15min":0.46000000000000002},"max":{"1min":1.1950000000000001,"5min":1.2769999999999999,"15min":2.8290000000000002},"last":1.0629999999999999}]},{"osd":5,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.93400000000000005,"15min":0.92800000000000005},"min":{"1min":0.53800000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.431,"5min":2.431,"15min":2.7480000000000002},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.002,"5min":0.95999999999999996,"15min":0.93200000000000005},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.52400000000000002},"max":{"1min":1.431,"5min":1.431,"15min":1.6819999999999999},"last":1.085}]},{"osd":6,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.94099999999999995,"15min":0.92500000000000004},"min":{"1min":0.56899999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.036},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.014,"15min":1.0269999999999999},"min":{"1min":0.69599999999999995,"5min":0.58699999999999997,"15min":0.54800000000000004},"max":{"1min":1.353,"5min":1.353,"15min":6.4690000000000003},"last":1.1719999999999999}]},{"osd":7,"last update":"Thu Oct 26 05:58:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.99199999999999999,"15min":0.996},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.115},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.67900000000000005,"5min":0.59699999999999998,"15min":0.56399999999999995},"max":{"1min":1.282,"5min":1.417,"15min":1.5660000000000001},"last":1.1499999999999999}]}]}],"pool_statfs":[]}} 2023-10-26T05:59:38.915 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-26T05:59:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- 172.21.15.71:0/3531671165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eb4005190 msgr2=0x7f8eb400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/3531671165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eb4005190 0x7f8eb400b990 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc0016b0 tx=0x7f8ebc01c7a0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- 172.21.15.71:0/3531671165 shutdown_connections 2023-10-26T05:59:39.044 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/3531671165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eb4005190 0x7f8eb400b990 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- 172.21.15.71:0/3531671165 >> 172.21.15.71:0/3531671165 conn(0x7f8ecc0cdd80 msgr2=0x7f8ecc0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- 172.21.15.71:0/3531671165 shutdown_connections 2023-10-26T05:59:39.045 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- 172.21.15.71:0/3531671165 wait complete. 2023-10-26T05:59:39.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 Processor -- start 2023-10-26T05:59:39.047 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- start start 2023-10-26T05:59:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8eb4005190 0x7f8ecc1473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 0x7f8ecc15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ecc15bef0 0x7f8ecc1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc06a650 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1478e0 con 0x7f8ecc15bef0 2023-10-26T05:59:39.048 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.034+0000 7f8ed2a33640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc1536a0 con 0x7f8eb4005190 2023-10-26T05:59:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 0x7f8ecc15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 0x7f8ecc15b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57258/0 (socket says 172.21.15.71:57258) 2023-10-26T05:59:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 -- 172.21.15.71:0/2159343592 learned_addr learned my addr 172.21.15.71:0/2159343592 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ed0fd0640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ecc15bef0 0x7f8ecc1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.049 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecbfff640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8eb4005190 0x7f8ecc1473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 -- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8eb4005190 msgr2=0x7f8ecc1473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8eb4005190 0x7f8ecc1473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 -- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ecc15bef0 msgr2=0x7f8ecc1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ecc15bef0 0x7f8ecc1f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.050 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 -- 172.21.15.71:0/2159343592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc15b520 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecbfff640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8eb4005190 0x7f8ecc1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-26T05:59:39.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecb7fe640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 0x7f8ecc15b9b0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00046e0 tx=0x7f8ec000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:39.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00118a0 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(5 keys) v1 ==== 141+0+0 (secure 0 0 0) 0x7f8ec0011af0 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.051 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0011e60 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc152b80 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.052 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc1530e0 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc066b10 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.053 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8ec0012470 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ec8ff9640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8eb8066470 0x7f8eb8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ec0016070 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.038+0000 7f8ecbfff640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8eb8066470 0x7f8eb8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.042+0000 7f8ecbfff640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8eb8066470 0x7f8eb8068930 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009f80 tx=0x7f8ec4009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:39.054 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.042+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f8ec00584e0 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.198 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.190+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8ecc15b6b0 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.206 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ec8ff9640 1 -- 172.21.15.71:0/2159343592 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v39)=0 v39) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8ec0058070 con 0x7f8ecc15cdd0 2023-10-26T05:59:39.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8eb8066470 msgr2=0x7f8eb8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8eb8066470 0x7f8eb8068930 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009f80 tx=0x7f8ec4009500 comp rx=0 tx=0).stop 2023-10-26T05:59:39.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 msgr2=0x7f8ecc15b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 0x7f8ecc15b9b0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00046e0 tx=0x7f8ec000fa30 comp rx=0 tx=0).stop 2023-10-26T05:59:39.209 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 shutdown_connections 2023-10-26T05:59:39.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ecc15bef0 0x7f8ecc1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f8eb8066470 0x7f8eb8068930 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ecc15cdd0 0x7f8ecc15b9b0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 --2- 172.21.15.71:0/2159343592 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8eb4005190 0x7f8ecc1473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.210 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 >> 172.21.15.71:0/2159343592 conn(0x7f8ecc0cdd80 msgr2=0x7f8ecc0b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:39.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 shutdown_connections 2023-10-26T05:59:39.211 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.198+0000 7f8ed2a33640 1 -- 172.21.15.71:0/2159343592 wait complete. 2023-10-26T05:59:39.225 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-26T05:59:39.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-26T05:59:39.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/434773676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80cd880 msgr2=0x7f56a815b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.351 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 --2- 172.21.15.71:0/434773676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80cd880 0x7f56a815b8e0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f56a80c4dd0 tx=0x7f56a001cab0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/434773676 shutdown_connections 2023-10-26T05:59:39.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 --2- 172.21.15.71:0/434773676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80cd880 0x7f56a815b8e0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 --2- 172.21.15.71:0/434773676 >> v2:172.21.15.86:3300/0 conn(0x7f56a80d0850 0x7f56a815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.352 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/434773676 >> 172.21.15.71:0/434773676 conn(0x7f56a80cdd80 msgr2=0x7f56a80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:39.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/434773676 shutdown_connections 2023-10-26T05:59:39.353 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/434773676 wait complete. 2023-10-26T05:59:39.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 Processor -- start 2023-10-26T05:59:39.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- start start 2023-10-26T05:59:39.355 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a80cd880 0x7f56a8150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a80d0470 0x7f56a815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 0x7f56a8159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a815aa60 con 0x7f56a80d0850 2023-10-26T05:59:39.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81588b0 con 0x7f56a80d0470 2023-10-26T05:59:39.356 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a8158b90 con 0x7f56a80cd880 2023-10-26T05:59:39.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 0x7f56a8159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 0x7f56a8159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57268/0 (socket says 172.21.15.71:57268) 2023-10-26T05:59:39.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 -- 172.21.15.71:0/1143927773 learned_addr learned my addr 172.21.15.71:0/1143927773 (peer_addr_for_me v2:172.21.15.71:0/0) 2023-10-26T05:59:39.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f569effd640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a80d0470 0x7f56a815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.357 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a77fe640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a80cd880 0x7f56a8150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 -- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a80cd880 msgr2=0x7f56a8150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a80cd880 0x7f56a8150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 -- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a80d0470 msgr2=0x7f56a815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a80d0470 0x7f56a815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.358 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 -- 172.21.15.71:0/1143927773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a0009820 con 0x7f56a80d0850 2023-10-26T05:59:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a7fff640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 0x7f56a8159950 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f56a00196c0 tx=0x7f56a00a58f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005450 con 0x7f56a80d0850 2023-10-26T05:59:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(6 keys) v1 ==== 172+0+0 (secure 0 0 0) 0x7f56a00056c0 con 0x7f56a80d0850 2023-10-26T05:59:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a8158e70 con 0x7f56a80d0850 2023-10-26T05:59:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00b7440 con 0x7f56a80d0850 2023-10-26T05:59:39.359 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.342+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a8159430 con 0x7f56a80d0850 2023-10-26T05:59:39.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.346+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a8066b10 con 0x7f56a80d0850 2023-10-26T05:59:39.360 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.346+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56a00a6d50 con 0x7f56a80d0850 2023-10-26T05:59:39.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.346+0000 7f56a4ff9640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5688066470 0x7f5688068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-26T05:59:39.361 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.346+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f56a011e6c0 con 0x7f56a80d0850 2023-10-26T05:59:39.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.346+0000 7f56a77fe640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5688066470 0x7f5688068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-26T05:59:39.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.346+0000 7f56a77fe640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5688066470 0x7f5688068930 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5698007fd0 tx=0x7f5698009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-26T05:59:39.362 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.350+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172159 (secure 0 0 0) 0x7f56a00ebe20 con 0x7f56a80d0850 2023-10-26T05:59:39.549 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.538+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56a8159620 con 0x7f56a80d0850 2023-10-26T05:59:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56a4ff9640 1 -- 172.21.15.71:0/1143927773 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f56a00ef440 con 0x7f56a80d0850 2023-10-26T05:59:39.550 INFO:teuthology.orchestra.run.smithi071.stdout: 2023-10-26T05:59:39.551 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6338,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/3763251424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3763251424},{"type":"v1","addr":"172.21.15.86:6837","nonce":3763251424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":6964,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2251486257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2251486257},{"type":"v1","addr":"172.21.15.86:6836","nonce":2251486257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/847310624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":847310624},{"type":"v1","addr":"172.21.15.71:6833","nonce":847310624}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/597219837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":597219837},{"type":"v1","addr":"172.21.15.71:6835","nonce":597219837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[]} 2023-10-26T05:59:39.551 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 73 2023-10-26T05:59:39.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5688066470 msgr2=0x7f5688068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.552 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5688066470 0x7f5688068930 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5698007fd0 tx=0x7f5698009040 comp rx=0 tx=0).stop 2023-10-26T05:59:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 msgr2=0x7f56a8159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-26T05:59:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 0x7f56a8159950 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f56a00196c0 tx=0x7f56a00a58f0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 shutdown_connections 2023-10-26T05:59:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:6832/1120505646,v1:172.21.15.86:6833/1120505646] conn(0x7f5688066470 0x7f5688068930 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56a80d0850 0x7f56a8159950 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a80d0470 0x7f56a815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 --2- 172.21.15.71:0/1143927773 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a80cd880 0x7f56a8150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-26T05:59:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 >> 172.21.15.71:0/1143927773 conn(0x7f56a80cdd80 msgr2=0x7f56a80b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-26T05:59:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 shutdown_connections 2023-10-26T05:59:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2023-10-26T05:59:39.542+0000 7f56adfed640 1 -- 172.21.15.71:0/1143927773 wait complete. 2023-10-26T05:59:39.569 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-26T05:59:39.569 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-26T05:59:39.570 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-26T05:59:39.570 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.580 INFO:tasks.ceph.mds.a:Stopped 2023-10-26T05:59:39.580 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-26T05:59:39.580 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.634 INFO:tasks.ceph.mds.c:Stopped 2023-10-26T05:59:39.635 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-26T05:59:39.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.650 INFO:tasks.ceph.mds.b:Stopped 2023-10-26T05:59:39.650 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-26T05:59:39.650 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.713 INFO:tasks.ceph.mds.d:Stopped 2023-10-26T05:59:39.713 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-26T05:59:39.713 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-26T05:59:39.714 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.748 INFO:tasks.ceph.osd.0:Stopped 2023-10-26T05:59:39.748 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-26T05:59:39.749 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.821 INFO:tasks.ceph.osd.1:Stopped 2023-10-26T05:59:39.822 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-26T05:59:39.822 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.889 INFO:tasks.ceph.osd.2:Stopped 2023-10-26T05:59:39.890 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-26T05:59:39.890 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:39.974 INFO:tasks.ceph.osd.3:Stopped 2023-10-26T05:59:39.975 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-26T05:59:39.975 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.024 INFO:tasks.ceph.osd.4:Stopped 2023-10-26T05:59:40.024 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-26T05:59:40.024 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.061 INFO:tasks.ceph.osd.5:Stopped 2023-10-26T05:59:40.061 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-26T05:59:40.061 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.141 INFO:tasks.ceph.osd.6:Stopped 2023-10-26T05:59:40.141 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-26T05:59:40.142 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.226 INFO:tasks.ceph.osd.7:Stopped 2023-10-26T05:59:40.227 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-26T05:59:40.228 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-26T05:59:40.228 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.257 INFO:tasks.ceph.mgr.y:Stopped 2023-10-26T05:59:40.258 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-26T05:59:40.259 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.287 INFO:tasks.ceph.mgr.x:Stopped 2023-10-26T05:59:40.288 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-26T05:59:40.288 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-26T05:59:40.288 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.321 INFO:tasks.ceph.mon.a:Stopped 2023-10-26T05:59:40.322 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-26T05:59:40.322 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.344 INFO:tasks.ceph.mon.b:Stopped 2023-10-26T05:59:40.345 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-26T05:59:40.345 INFO:teuthology.orchestra.run:waiting for 300 2023-10-26T05:59:40.408 INFO:tasks.ceph.mon.c:Stopped 2023-10-26T05:59:40.408 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-26T05:59:40.409 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T05:59:40.426 INFO:teuthology.orchestra.run.smithi071.stdout:2023-10-26T05:47:08.297699+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 164877.815646 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-26T05:59:40.426 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-26T05:59:40.427 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T05:59:40.486 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T05:59:40.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-10-26T05:59:40.601 INFO:teuthology.orchestra.run.smithi071.stdout:2023-10-26T05:47:08.297699+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 164877.815646 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-10-26T05:59:40.602 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:40.602 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-26T05:59:40.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:40.911 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-26T05:59:41.154 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:41.154 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-26T05:59:41.336 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:41.336 DEBUG:teuthology.orchestra.run.smithi071:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-26T05:59:41.527 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:41.528 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-26T05:59:41.767 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:41.768 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-26T05:59:41.910 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:41.911 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-26T05:59:42.076 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:42.077 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-26T05:59:42.235 INFO:tasks.ceph:Archiving mon data... 2023-10-26T05:59:42.237 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/data/mon.a.tgz 2023-10-26T05:59:42.237 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2023-10-26T05:59:42.243 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.rKVfp7SZwB 2023-10-26T05:59:42.244 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rKVfp7SZwB 2023-10-26T05:59:42.383 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.rKVfp7SZwB 2023-10-26T05:59:42.446 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.rKVfp7SZwB is 431KB 2023-10-26T05:59:42.508 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.rKVfp7SZwB 2023-10-26T05:59:42.514 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/data/mon.b.tgz 2023-10-26T05:59:42.514 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2023-10-26T05:59:42.520 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.jNFtoHwsMj 2023-10-26T05:59:42.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.jNFtoHwsMj 2023-10-26T05:59:42.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.jNFtoHwsMj 2023-10-26T05:59:42.725 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.jNFtoHwsMj is 439KB 2023-10-26T05:59:42.788 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.jNFtoHwsMj 2023-10-26T05:59:42.793 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/data/mon.c.tgz 2023-10-26T05:59:42.794 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2023-10-26T05:59:42.842 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.nhxg8pkGAk 2023-10-26T05:59:42.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.nhxg8pkGAk 2023-10-26T05:59:42.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.nhxg8pkGAk 2023-10-26T05:59:43.050 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.nhxg8pkGAk is 439KB 2023-10-26T05:59:43.108 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.nhxg8pkGAk 2023-10-26T05:59:43.113 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-26T05:59:43.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-26T05:59:43.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-26T05:59:43.167 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-26T05:59:43.167 DEBUG:teuthology.orchestra.run.smithi071:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-26T05:59:43.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-26T05:59:43.193 INFO:teuthology.orchestra.run.smithi071.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-26T05:59:43.232 INFO:teuthology.orchestra.run.smithi086.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-26T05:59:43.236 INFO:tasks.ceph:Archiving crash dumps... 2023-10-26T05:59:43.238 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/remote/smithi071/crash 2023-10-26T05:59:43.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-26T05:59:43.259 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/remote/smithi086/crash 2023-10-26T05:59:43.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-26T05:59:43.288 INFO:tasks.ceph:Shutting down logrotate 2023-10-26T05:59:43.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:59:43.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-26T05:59:43.384 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-26T05:59:43.396 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-26T05:59:43.408 INFO:tasks.ceph:Compressing logs... 2023-10-26T05:59:43.409 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T05:59:43.439 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-26T05:59:43.465 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46171.log 2023-10-26T05:59:43.466 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54249.log 2023-10-26T05:59:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49515.log 2023-10-26T05:59:43.471 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46171.log.gz 2023-10-26T05:59:43.471 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52514.log 2023-10-26T05:59:43.471 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54249.log.gz 2023-10-26T05:59:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2023-10-26T05:59:43.472 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49515.log.gz 2023-10-26T05:59:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50962.log 2023-10-26T05:59:43.472 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52514.log.gz 2023-10-26T05:59:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44622.log 2023-10-26T05:59:43.472 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41296.log: /var/log/ceph/ceph-client.admin.50962.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2023-10-26T05:59:43.473 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53549.log 2023-10-26T05:59:43.473 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2023-10-26T05:59:43.473 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2023-10-26T05:59:43.473 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44622.log.gz 2023-10-26T05:59:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53048.log 2023-10-26T05:59:43.474 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53549.log.gz 2023-10-26T05:59:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34128.log 2023-10-26T05:59:43.474 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2023-10-26T05:59:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55431.log 2023-10-26T05:59:43.474 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48242.log 2023-10-26T05:59:43.475 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53048.log.gz 2023-10-26T05:59:43.475 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55431.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34128.log.gz 2023-10-26T05:59:43.475 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49708.log 2023-10-26T05:59:43.475 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55431.log.gz 2023-10-26T05:59:43.475 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56439.log 2023-10-26T05:59:43.500 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33193.log 2023-10-26T05:59:43.559 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35369.log 2023-10-26T05:59:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33193.log: gzip 86.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.33193.log.gz --verbose 2023-10-26T05:59:43.560 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.1.35050.log 2023-10-26T05:59:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34894.log 2023-10-26T05:59:43.560 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.35369.log: /var/log/ceph/ceph-client.1.35050.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-26T05:59:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34894.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-26T05:59:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34740.log 2023-10-26T05:59:43.561 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.1.35050.log.gz 2023-10-26T05:59:43.561 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.7.log: 88.8% -- replaced with /var/log/ceph/ceph-client.1.34894.log.gz 2023-10-26T05:59:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33767.log 2023-10-26T05:59:43.562 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.1.35369.log.gz 2023-10-26T05:59:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2023-10-26T05:59:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34587.log 2023-10-26T05:59:43.562 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.33767.log: /var/log/ceph/ceph-client.admin.33169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34055.log 2023-10-26T05:59:43.563 INFO:teuthology.orchestra.run.smithi086.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.1.34740.log.gz 2023-10-26T05:59:43.563 INFO:teuthology.orchestra.run.smithi086.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2023-10-26T05:59:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34414.log 2023-10-26T05:59:43.563 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34206.log 2023-10-26T05:59:43.563 INFO:teuthology.orchestra.run.smithi086.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.1.33767.log.gz 2023-10-26T05:59:43.564 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33121.log 2023-10-26T05:59:43.564 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.1.34055.log.gz 2023-10-26T05:59:43.564 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.34206.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-26T05:59:43.564 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33121.log: 91.5% -- replaced with /var/log/ceph/ceph-client.1.34587.log.gz 2023-10-26T05:59:43.564 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.33217.log 2023-10-26T05:59:43.565 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.5.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33121.log.gz 2023-10-26T05:59:43.565 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-26T05:59:43.565 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.1.34414.log.gz 2023-10-26T05:59:43.565 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.1.34206.log.gz 2023-10-26T05:59:43.565 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33879.log 2023-10-26T05:59:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.x.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.33217.log.gz 2023-10-26T05:59:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-26T05:59:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.3.33879.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-26T05:59:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-26T05:59:43.566 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33090.log 2023-10-26T05:59:43.567 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.3.33879.log.gz 2023-10-26T05:59:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-26T05:59:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33090.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33090.log.gz 2023-10-26T05:59:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2023-10-26T05:59:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.d.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-26T05:59:43.567 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33145.log 2023-10-26T05:59:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33066.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2023-10-26T05:59:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35205.log 2023-10-26T05:59:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33145.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.33145.log.gz 2023-10-26T05:59:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33241.log 2023-10-26T05:59:43.568 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.35205.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-26T05:59:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-26T05:59:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33241.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.33241.log.gz 2023-10-26T05:59:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-26T05:59:43.569 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.b.log: 95.0% -- replaced with /var/log/ceph/ceph-client.1.35205.log.gz 2023-10-26T05:59:43.570 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-26T05:59:43.570 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.b.log: 92.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-26T05:59:43.570 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35525.log 2023-10-26T05:59:43.572 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.1.35525.log: 91.8% -- replaced with /var/log/ceph/ceph-client.1.35525.log.gz 2023-10-26T05:59:43.573 INFO:teuthology.orchestra.run.smithi086.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-26T05:59:43.658 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-26T05:59:43.742 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49708.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48242.log.gz 2023-10-26T05:59:43.743 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35299.log 2023-10-26T05:59:43.743 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49708.log.gz 2023-10-26T05:59:43.743 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35984.log 2023-10-26T05:59:43.743 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56439.log: /var/log/ceph/ceph-client.admin.35299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49963.log 2023-10-26T05:59:43.744 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46460.log 2023-10-26T05:59:43.744 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35299.log.gz 2023-10-26T05:59:43.744 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56439.log.gz 2023-10-26T05:59:43.744 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35984.log.gz 2023-10-26T05:59:43.745 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43161.log 2023-10-26T05:59:43.745 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49963.log: /var/log/ceph/ceph-client.admin.46460.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49963.log.gz 2023-10-26T05:59:43.745 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45775.log 2023-10-26T05:59:43.745 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46460.log.gz 2023-10-26T05:59:43.745 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42671.log 2023-10-26T05:59:43.746 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43161.log.gz 2023-10-26T05:59:43.746 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34180.log 2023-10-26T05:59:43.746 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45775.log: /var/log/ceph/ceph-client.admin.42671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48443.log 2023-10-26T05:59:43.746 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42671.log.gz 2023-10-26T05:59:43.747 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45775.log.gz 2023-10-26T05:59:43.747 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2023-10-26T05:59:43.747 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34180.log.gz 2023-10-26T05:59:43.747 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52592.log 2023-10-26T05:59:43.747 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47868.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48443.log.gz 2023-10-26T05:59:43.747 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51172.log 2023-10-26T05:59:43.748 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz 2023-10-26T05:59:43.748 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52592.log: -- /var/log/ceph/ceph-client.admin.33772.log 2023-10-26T05:59:43.748 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2023-10-26T05:59:43.748 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52592.log.gz 2023-10-26T05:59:43.748 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51172.log: /var/log/ceph/ceph-client.admin.33772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46434.log 2023-10-26T05:59:43.749 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51172.log.gz 2023-10-26T05:59:43.749 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33772.log.gz 2023-10-26T05:59:43.749 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51842.log 2023-10-26T05:59:43.749 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2023-10-26T05:59:43.749 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56229.log 2023-10-26T05:59:43.750 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38035.log 2023-10-26T05:59:43.750 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46434.log.gz 2023-10-26T05:59:43.750 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51842.log: /var/log/ceph/ceph-client.admin.56229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47142.log 2023-10-26T05:59:43.750 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51842.log.gz 2023-10-26T05:59:43.750 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56229.log.gz 2023-10-26T05:59:43.751 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37957.log 2023-10-26T05:59:43.751 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38035.log.gz 2023-10-26T05:59:43.751 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42269.log 2023-10-26T05:59:43.751 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47142.log.gz 2023-10-26T05:59:43.751 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39459.log 2023-10-26T05:59:43.752 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37957.log: /var/log/ceph/ceph-client.admin.42269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2023-10-26T05:59:43.752 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37957.log.gz 2023-10-26T05:59:43.752 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42269.log.gz 2023-10-26T05:59:43.752 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-26T05:59:43.752 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39459.log: /var/log/ceph/ceph-client.admin.42778.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39459.log.gz 2023-10-26T05:59:43.753 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53205.log 2023-10-26T05:59:43.753 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mgr.y.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2023-10-26T05:59:43.753 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50990.log 2023-10-26T05:59:43.753 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48215.log 2023-10-26T05:59:43.753 INFO:teuthology.orchestra.run.smithi071.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53205.log.gz -5 2023-10-26T05:59:43.754 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41086.log 2023-10-26T05:59:43.754 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50990.log: /var/log/ceph/ceph-client.admin.48215.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50990.log.gz 2023-10-26T05:59:43.754 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50584.log 2023-10-26T05:59:43.754 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48215.log.gz 2023-10-26T05:59:43.754 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41419.log 2023-10-26T05:59:43.755 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41086.log.gz 2023-10-26T05:59:43.755 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49630.log 2023-10-26T05:59:43.755 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50584.log: /var/log/ceph/ceph-client.admin.41419.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50584.log.gz 2023-10-26T05:59:43.755 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51224.log 2023-10-26T05:59:43.756 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41419.log.gz 2023-10-26T05:59:43.756 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52104.log 2023-10-26T05:59:43.756 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49630.log.gz 2023-10-26T05:59:43.756 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53361.log 2023-10-26T05:59:43.756 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51224.log: /var/log/ceph/ceph-client.admin.52104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2023-10-26T05:59:43.757 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51224.log.gz 2023-10-26T05:59:43.757 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52104.log.gz 2023-10-26T05:59:43.757 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35039.log 2023-10-26T05:59:43.757 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53361.log.gz 2023-10-26T05:59:43.758 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35247.log 2023-10-26T05:59:43.758 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42532.log: /var/log/ceph/ceph-client.admin.35039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54706.log 2023-10-26T05:59:43.758 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2023-10-26T05:59:43.758 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35039.log.gz 2023-10-26T05:59:43.759 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37879.log 2023-10-26T05:59:43.759 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35247.log.gz 2023-10-26T05:59:43.759 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51599.log 2023-10-26T05:59:43.759 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54706.log: /var/log/ceph/ceph-client.admin.37879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2023-10-26T05:59:43.759 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37879.log.gz 2023-10-26T05:59:43.760 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51599.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54706.log.gz 2023-10-26T05:59:43.760 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36385.log 2023-10-26T05:59:43.760 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52390.log 2023-10-26T05:59:43.760 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51599.log.gz 2023-10-26T05:59:43.761 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37415.log: /var/log/ceph/ceph-client.admin.36385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34310.log 2023-10-26T05:59:43.761 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2023-10-26T05:59:43.761 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36385.log.gz 2023-10-26T05:59:43.761 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52390.log: gzip -5 --verbose -- 94.6% /var/log/ceph/ceph-client.admin.52488.log 2023-10-26T05:59:43.762 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-26T05:59:43.762 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52390.log.gz 2023-10-26T05:59:43.762 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50252.log 2023-10-26T05:59:43.762 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34310.log: /var/log/ceph/ceph-client.admin.52488.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34310.log.gz 2023-10-26T05:59:43.762 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41322.log 2023-10-26T05:59:43.763 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52488.log.gz 2023-10-26T05:59:43.763 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37363.log 2023-10-26T05:59:43.763 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41322.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50252.log.gz 2023-10-26T05:59:43.763 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2023-10-26T05:59:43.764 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41322.log.gz 2023-10-26T05:59:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40119.log 2023-10-26T05:59:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37363.log: /var/log/ceph/ceph-client.admin.36516.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37363.log.gzgzip 2023-10-26T05:59:43.764 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48417.log 2023-10-26T05:59:43.764 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2023-10-26T05:59:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2023-10-26T05:59:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48417.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.40119.log.gz --verbose -- 2023-10-26T05:59:43.765 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.32997.log 2023-10-26T05:59:43.765 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48417.log.gz 2023-10-26T05:59:43.765 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50876.log 2023-10-26T05:59:43.766 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48294.log: /var/log/ceph/ceph-client.admin.32997.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2023-10-26T05:59:43.766 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44879.log 2023-10-26T05:59:43.766 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32997.log.gz 2023-10-26T05:59:43.766 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56281.log 2023-10-26T05:59:43.767 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50876.log.gz 2023-10-26T05:59:43.767 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42697.log 2023-10-26T05:59:43.767 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44879.log: gzip -5/var/log/ceph/ceph-client.admin.56281.log: --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2023-10-26T05:59:43.767 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44879.log.gz 2023-10-26T05:59:43.767 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56281.log.gz 2023-10-26T05:59:43.768 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34206.log 2023-10-26T05:59:43.768 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42697.log.gz 2023-10-26T05:59:43.768 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2023-10-26T05:59:43.768 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42645.log 2023-10-26T05:59:43.768 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5%/var/log/ceph/ceph-client.admin.34206.log: -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2023-10-26T05:59:43.768 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46087.log 2023-10-26T05:59:43.769 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34206.log.gz 2023-10-26T05:59:43.769 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2023-10-26T05:59:43.769 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41471.log 2023-10-26T05:59:43.769 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42645.log.gz 2023-10-26T05:59:43.770 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39092.log 2023-10-26T05:59:43.770 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46087.log.gz 2023-10-26T05:59:43.770 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34987.log 2023-10-26T05:59:43.770 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41471.log: /var/log/ceph/ceph-client.admin.39092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2023-10-26T05:59:43.770 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41471.log.gz 2023-10-26T05:59:43.771 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39092.log.gz 2023-10-26T05:59:43.771 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-26T05:59:43.771 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34987.log: /var/log/ceph/ceph-client.admin.55347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39977.log 2023-10-26T05:59:43.771 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34987.log.gz 2023-10-26T05:59:43.771 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.0.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2023-10-26T05:59:43.772 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50148.log 2023-10-26T05:59:43.772 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40173.log 2023-10-26T05:59:43.772 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2023-10-26T05:59:43.772 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39977.log.gz 2023-10-26T05:59:43.773 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50148.log: /var/log/ceph/ceph-client.admin.40173.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50148.log.gz 2023-10-26T05:59:43.773 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55126.log 2023-10-26T05:59:43.773 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40173.log.gz 2023-10-26T05:59:43.773 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48469.log 2023-10-26T05:59:43.773 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2023-10-26T05:59:43.774 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34712.log 2023-10-26T05:59:43.774 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55126.log: /var/log/ceph/ceph-client.admin.48469.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55126.log.gz 2023-10-26T05:59:43.774 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55269.log 2023-10-26T05:59:43.774 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48469.log.gz 2023-10-26T05:59:43.774 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33737.log 2023-10-26T05:59:43.775 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.34712.log.gz 2023-10-26T05:59:43.775 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39355.log 2023-10-26T05:59:43.775 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55269.log: /var/log/ceph/ceph-client.admin.33737.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55269.log.gz 2023-10-26T05:59:43.775 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54275.log 2023-10-26T05:59:43.775 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33737.log.gz 2023-10-26T05:59:43.776 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45311.log 2023-10-26T05:59:43.776 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39355.log.gz 2023-10-26T05:59:43.776 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39008.log 2023-10-26T05:59:43.776 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54275.log: /var/log/ceph/ceph-client.admin.45311.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54275.log.gz 2023-10-26T05:59:43.776 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40093.log 2023-10-26T05:59:43.777 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45311.log.gz 2023-10-26T05:59:43.777 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55405.log 2023-10-26T05:59:43.777 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39008.log.gz 2023-10-26T05:59:43.777 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42960.log 2023-10-26T05:59:43.777 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40093.log: /var/log/ceph/ceph-client.admin.55405.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40093.log.gz 2023-10-26T05:59:43.777 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46922.log 2023-10-26T05:59:43.778 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55405.log.gz 2023-10-26T05:59:43.778 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53309.log 2023-10-26T05:59:43.778 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42960.log.gz 2023-10-26T05:59:43.778 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41005.log 2023-10-26T05:59:43.778 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46922.log: /var/log/ceph/ceph-client.admin.53309.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46922.log.gz 2023-10-26T05:59:43.779 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44197.log 2023-10-26T05:59:43.779 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53309.log.gz 2023-10-26T05:59:43.779 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33141.log 2023-10-26T05:59:43.779 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52052.log 2023-10-26T05:59:43.779 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41005.log.gz 2023-10-26T05:59:43.780 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44197.log: /var/log/ceph/ceph-client.admin.33141.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44197.log.gz 2023-10-26T05:59:43.780 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42591.log 2023-10-26T05:59:43.780 INFO:teuthology.orchestra.run.smithi071.stderr: 75.5% -- replaced with /var/log/ceph/ceph-client.admin.33141.log.gz 2023-10-26T05:59:43.780 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54550.log 2023-10-26T05:59:43.780 INFO:teuthology.orchestra.run.smithi071.stderr:gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.52052.log.gz -- 2023-10-26T05:59:43.781 INFO:teuthology.orchestra.run.smithi071.stderr: /var/log/ceph/ceph-client.admin.45652.log 2023-10-26T05:59:43.781 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42591.log: /var/log/ceph/ceph-client.admin.54550.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42591.log.gz 2023-10-26T05:59:43.781 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49479.log 2023-10-26T05:59:43.781 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54550.log.gz 2023-10-26T05:59:43.781 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36626.log 2023-10-26T05:59:43.781 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55694.log 2023-10-26T05:59:43.782 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49479.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45652.log.gz 2023-10-26T05:59:43.782 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36626.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49479.log.gz 2023-10-26T05:59:43.782 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37571.log 2023-10-26T05:59:43.782 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36626.log.gz 2023-10-26T05:59:43.783 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39329.log 2023-10-26T05:59:43.783 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2023-10-26T05:59:43.783 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37571.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55694.log.gz 2023-10-26T05:59:43.783 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39329.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2023-10-26T05:59:43.783 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46009.log 2023-10-26T05:59:43.784 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39329.log.gz 2023-10-26T05:59:43.784 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39582.log 2023-10-26T05:59:43.784 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40225.log 87.7% 2023-10-26T05:59:43.784 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2023-10-26T05:59:43.785 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46009.log: /var/log/ceph/ceph-client.admin.39582.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46009.log.gz 2023-10-26T05:59:43.785 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47352.log 2023-10-26T05:59:43.785 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39582.log.gz 2023-10-26T05:59:43.785 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39277.log 2023-10-26T05:59:43.786 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40225.log.gz 2023-10-26T05:59:43.786 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36062.log 2023-10-26T05:59:43.786 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47352.log: /var/log/ceph/ceph-client.admin.39277.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.47352.log.gz 2023-10-26T05:59:43.786 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51491.log 2023-10-26T05:59:43.787 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39277.log.gz 2023-10-26T05:59:43.787 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43269.log 2023-10-26T05:59:43.787 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36062.log.gz 2023-10-26T05:59:43.787 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41841.log 2023-10-26T05:59:43.787 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51491.log: /var/log/ceph/ceph-client.admin.43269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41957.log 2023-10-26T05:59:43.788 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51491.log.gz 2023-10-26T05:59:43.788 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43269.log.gz 2023-10-26T05:59:43.788 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39634.log 2023-10-26T05:59:43.788 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41841.log.gz 2023-10-26T05:59:43.789 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2023-10-26T05:59:43.789 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41957.log: /var/log/ceph/ceph-client.admin.39634.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41957.log.gz 2023-10-26T05:59:43.789 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48111.log 2023-10-26T05:59:43.789 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39634.log.gz 2023-10-26T05:59:43.789 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37983.log 2023-10-26T05:59:43.789 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2023-10-26T05:59:43.790 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2023-10-26T05:59:43.790 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2023-10-26T05:59:43.790 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37983.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48111.log.gz 2023-10-26T05:59:43.790 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37649.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37983.log.gzgzip 2023-10-26T05:59:43.791 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47429.log 2023-10-26T05:59:43.791 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2023-10-26T05:59:43.791 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52331.log 2023-10-26T05:59:43.791 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41867.log 2023-10-26T05:59:43.791 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47429.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2023-10-26T05:59:43.792 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45283.log 2023-10-26T05:59:43.792 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47429.log.gz 2023-10-26T05:59:43.792 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52331.log: /var/log/ceph/ceph-client.admin.41867.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52331.log.gz 2023-10-26T05:59:43.792 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50936.log 2023-10-26T05:59:43.792 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41867.log.gz 2023-10-26T05:59:43.793 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45141.log 2023-10-26T05:59:43.793 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45283.log.gz 2023-10-26T05:59:43.793 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2023-10-26T05:59:43.793 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50936.log: /var/log/ceph/ceph-client.admin.45141.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.50936.log.gz -5 2023-10-26T05:59:43.794 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.33046.log 2023-10-26T05:59:43.794 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45141.log.gz 2023-10-26T05:59:43.794 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45495.log 2023-10-26T05:59:43.794 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44229.log 2023-10-26T05:59:43.795 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2023-10-26T05:59:43.795 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49004.log 2023-10-26T05:59:43.795 INFO:teuthology.orchestra.run.smithi071.stderr: 75.4% -- replaced with /var/log/ceph/ceph-client.admin.33046.log.gz 2023-10-26T05:59:43.795 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45495.log: /var/log/ceph/ceph-client.admin.44229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54301.log 2023-10-26T05:59:43.796 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44229.log.gz 2023-10-26T05:59:43.796 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41348.log 2023-10-26T05:59:43.796 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45495.log.gz 2023-10-26T05:59:43.796 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49004.log.gz 2023-10-26T05:59:43.796 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40610.log 2023-10-26T05:59:43.797 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54301.log: /var/log/ceph/ceph-client.admin.41348.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54301.log.gz 2023-10-26T05:59:43.797 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54013.log 2023-10-26T05:59:43.797 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41348.log.gz 2023-10-26T05:59:43.797 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.40610.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52279.log 2023-10-26T05:59:43.798 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37931.log 2023-10-26T05:59:43.798 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40610.log.gz 2023-10-26T05:59:43.798 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52279.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54013.log.gz 2023-10-26T05:59:43.798 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37597.log 2023-10-26T05:59:43.799 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52279.log.gz 2023-10-26T05:59:43.799 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38440.log 2023-10-26T05:59:43.799 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56465.log 2023-10-26T05:59:43.799 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37931.log.gz 2023-10-26T05:59:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38440.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz 2023-10-26T05:59:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44171.log 2023-10-26T05:59:43.800 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.38440.log.gz 2023-10-26T05:59:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44015.log 2023-10-26T05:59:43.800 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52814.log 2023-10-26T05:59:43.801 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56465.log.gz 2023-10-26T05:59:43.801 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44015.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44171.log.gz 2023-10-26T05:59:43.801 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56151.log 2023-10-26T05:59:43.801 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44015.log.gz 2023-10-26T05:59:43.802 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53838.log 2023-10-26T05:59:43.802 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56151.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52814.log.gz 2023-10-26T05:59:43.802 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2023-10-26T05:59:43.802 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56151.log.gz 2023-10-26T05:59:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42141.log 2023-10-26T05:59:43.803 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53838.log.gz 2023-10-26T05:59:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56177.log 2023-10-26T05:59:43.803 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44414.log 2023-10-26T05:59:43.803 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2023-10-26T05:59:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56177.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42141.log.gz 2023-10-26T05:59:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33916.log 2023-10-26T05:59:43.804 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56177.log.gz 2023-10-26T05:59:43.804 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.44414.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2023-10-26T05:59:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36411.log 2023-10-26T05:59:43.805 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44414.log.gz 2023-10-26T05:59:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43806.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33916.log.gz 2023-10-26T05:59:43.805 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2023-10-26T05:59:43.805 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2023-10-26T05:59:43.806 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54628.log 2023-10-26T05:59:43.806 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36411.log.gz 2023-10-26T05:59:43.806 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53864.log 2023-10-26T05:59:43.806 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2023-10-26T05:59:43.806 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40817.log 2023-10-26T05:59:43.807 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54628.log.gz 2023-10-26T05:59:43.807 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-26T05:59:43.807 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53864.log: /var/log/ceph/ceph-client.admin.40817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55178.log 2023-10-26T05:59:43.807 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53864.log.gz 2023-10-26T05:59:43.808 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40817.log.gz 2023-10-26T05:59:43.808 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38414.log 2023-10-26T05:59:43.808 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38143.log 2023-10-26T05:59:43.808 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55178.log: /var/log/ceph/ceph-client.admin.38414.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55178.log.gz 2023-10-26T05:59:43.808 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52253.log 2023-10-26T05:59:43.809 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38414.log.gz 2023-10-26T05:59:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36166.log 2023-10-26T05:59:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44849.log 2023-10-26T05:59:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52253.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38143.log.gz 2023-10-26T05:59:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33034.log 2023-10-26T05:59:43.810 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52253.log.gz 2023-10-26T05:59:43.810 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36166.log.gz 2023-10-26T05:59:43.810 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41605.log 2023-10-26T05:59:43.810 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44849.log.gz 2023-10-26T05:59:43.811 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33034.log.gz 2023-10-26T05:59:43.811 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53261.log 2023-10-26T05:59:43.811 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46119.log 2023-10-26T05:59:43.811 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39899.log 2023-10-26T05:59:43.811 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53261.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41605.log.gz 2023-10-26T05:59:43.812 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46119.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53261.log.gz 2023-10-26T05:59:43.812 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47000.log 2023-10-26T05:59:43.812 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46119.log.gz 2023-10-26T05:59:43.812 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41697.log 2023-10-26T05:59:43.812 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39899.log.gz 2023-10-26T05:59:43.813 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35354.log 2023-10-26T05:59:43.813 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47000.log.gz 2023-10-26T05:59:43.813 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34441.log 2023-10-26T05:59:43.813 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38335.log 2023-10-26T05:59:43.814 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35354.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41697.log.gz 2023-10-26T05:59:43.814 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34441.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35354.log.gz 2023-10-26T05:59:43.814 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36966.log 2023-10-26T05:59:43.814 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34441.log.gz 2023-10-26T05:59:43.814 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55064.log 2023-10-26T05:59:43.815 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38335.log.gz 2023-10-26T05:59:43.815 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2023-10-26T05:59:43.815 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36966.log.gz 2023-10-26T05:59:43.815 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49214.log 2023-10-26T05:59:43.815 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50741.log 2023-10-26T05:59:43.816 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35958.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55064.log.gz 2023-10-26T05:59:43.816 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49214.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2023-10-26T05:59:43.816 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37181.log 2023-10-26T05:59:43.816 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49214.log.gz 2023-10-26T05:59:43.816 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41031.log 2023-10-26T05:59:43.817 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37181.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50741.log.gz 2023-10-26T05:59:43.817 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39660.log 2023-10-26T05:59:43.817 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37181.log.gz 2023-10-26T05:59:43.817 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44823.log 2023-10-26T05:59:43.817 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41031.log.gz 2023-10-26T05:59:43.818 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39066.log 2023-10-26T05:59:43.818 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39660.log.gz 2023-10-26T05:59:43.818 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2023-10-26T05:59:43.818 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44823.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.39066.log: -- /var/log/ceph/ceph-client.admin.55204.log 2023-10-26T05:59:43.818 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44823.log.gz 2023-10-26T05:59:43.819 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53413.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39066.log.gz 2023-10-26T05:59:43.819 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55798.log 2023-10-26T05:59:43.819 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2023-10-26T05:59:43.819 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35613.log 2023-10-26T05:59:43.819 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55204.log.gz 2023-10-26T05:59:43.819 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34509.log 2023-10-26T05:59:43.820 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55798.log.gz 2023-10-26T05:59:43.820 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.2.35613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41923.log 2023-10-26T05:59:43.820 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55152.log 2023-10-26T05:59:43.820 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34509.log: /var/log/ceph/ceph-client.admin.41923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54524.log 2023-10-26T05:59:43.820 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34509.log.gz 2023-10-26T05:59:43.821 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41923.log.gz 2023-10-26T05:59:43.821 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49775.log 2023-10-26T05:59:43.821 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55152.log: gzip 91.5% -5 -- replaced with /var/log/ceph/ceph-client.2.35613.log.gz --verbose 2023-10-26T05:59:43.821 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/ceph-client.admin.48391.log 2023-10-26T05:59:43.821 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54524.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55152.log.gz 2023-10-26T05:59:43.822 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49775.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54524.log.gz 2023-10-26T05:59:43.822 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37493.log 2023-10-26T05:59:43.822 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49775.log.gz 2023-10-26T05:59:43.822 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54654.log 2023-10-26T05:59:43.822 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48391.log.gz 2023-10-26T05:59:43.823 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37493.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-26T05:59:43.823 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37493.log.gz 2023-10-26T05:59:43.823 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56542.log 2023-10-26T05:59:43.823 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54654.log.gz 2023-10-26T05:59:43.823 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45337.log 2023-10-26T05:59:43.823 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55969.log 2023-10-26T05:59:43.824 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56542.log.gz 2023-10-26T05:59:43.824 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49570.log 2023-10-26T05:59:43.824 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45337.log.gz 2023-10-26T05:59:43.824 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36940.log 2023-10-26T05:59:43.824 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55969.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55969.log.gz/var/log/ceph/ceph-client.admin.49570.log: 2023-10-26T05:59:43.825 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47484.log 2023-10-26T05:59:43.825 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6%/var/log/ceph/ceph-client.admin.36940.log: -- replaced with /var/log/ceph/ceph-client.admin.49570.log.gz 2023-10-26T05:59:43.825 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34961.log 2023-10-26T05:59:43.825 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36940.log.gz 2023-10-26T05:59:43.825 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.47484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34467.log 2023-10-26T05:59:43.826 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38757.log 2023-10-26T05:59:43.826 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34961.log.gz 2023-10-26T05:59:43.826 INFO:teuthology.orchestra.run.smithi071.stderr:gzip/var/log/ceph/ceph-client.admin.34467.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47196.log 2023-10-26T05:59:43.826 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38757.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34467.log.gz 2023-10-26T05:59:43.827 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38757.log.gz 2023-10-26T05:59:43.827 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55564.log 2023-10-26T05:59:43.827 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34102.log 2023-10-26T05:59:43.827 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55564.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.38928.log 2023-10-26T05:59:43.827 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47196.log.gz 2023-10-26T05:59:43.828 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34102.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55564.log.gz 2023-10-26T05:59:43.828 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42376.log 2023-10-26T05:59:43.828 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34102.log.gz 2023-10-26T05:59:43.828 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43832.log 2023-10-26T05:59:43.829 INFO:teuthology.orchestra.run.smithi071.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.47484.log.gz 2023-10-26T05:59:43.829 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38928.log.gz 2023-10-26T05:59:43.829 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37467.log 2023-10-26T05:59:43.829 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42376.log.gz 2023-10-26T05:59:43.830 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51625.log 2023-10-26T05:59:43.830 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37467.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43832.log.gz 2023-10-26T05:59:43.830 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46486.log 2023-10-26T05:59:43.830 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37467.log.gz 2023-10-26T05:59:43.830 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54812.log 2023-10-26T05:59:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48268.log 2023-10-26T05:59:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46486.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51625.log.gz 2023-10-26T05:59:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.54812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43754.log 2023-10-26T05:59:43.831 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46486.log.gz 2023-10-26T05:59:43.831 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56387.log 2023-10-26T05:59:43.832 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43754.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48268.log.gz 2023-10-26T05:59:43.832 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34232.log 2023-10-26T05:59:43.832 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43754.log.gz 2023-10-26T05:59:43.832 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51761.log 2023-10-26T05:59:43.832 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56387.log.gz 2023-10-26T05:59:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51276.log 2023-10-26T05:59:43.833 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.34232.log.gz 2023-10-26T05:59:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49682.log 2023-10-26T05:59:43.833 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51761.log.gz 2023-10-26T05:59:43.833 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2023-10-26T05:59:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37827.log 2023-10-26T05:59:43.834 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49682.log.gz 2023-10-26T05:59:43.834 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2023-10-26T05:59:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2023-10-26T05:59:43.834 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45678.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2023-10-26T05:59:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37827.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37827.log.gz 2023-10-26T05:59:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53231.log 2023-10-26T05:59:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55590.log 2023-10-26T05:59:43.835 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2023-10-26T05:59:43.835 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49030.log 2023-10-26T05:59:43.836 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53231.log.gz 2023-10-26T05:59:43.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36652.log 2023-10-26T05:59:43.836 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55590.log.gz 2023-10-26T05:59:43.836 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33105.log 2023-10-26T05:59:43.836 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49030.log.gz 2023-10-26T05:59:43.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55772.log 2023-10-26T05:59:43.837 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36652.log.gz 2023-10-26T05:59:43.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46687.log 2023-10-26T05:59:43.837 INFO:teuthology.orchestra.run.smithi071.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33105.log.gz 2023-10-26T05:59:43.837 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34726.log 2023-10-26T05:59:43.837 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55772.log: /var/log/ceph/ceph-client.admin.46687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2023-10-26T05:59:43.838 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55772.log.gz 2023-10-26T05:59:43.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2023-10-26T05:59:43.838 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46687.log.gz 2023-10-26T05:59:43.838 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34726.log.gz 2023-10-26T05:59:43.838 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44388.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2023-10-26T05:59:43.839 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56099.log 2023-10-26T05:59:43.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41190.log 2023-10-26T05:59:43.839 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2023-10-26T05:59:43.839 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48808.log 2023-10-26T05:59:43.839 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56099.log.gz 2023-10-26T05:59:43.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51198.log 2023-10-26T05:59:43.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33132.log 2023-10-26T05:59:43.840 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41190.log.gz 2023-10-26T05:59:43.840 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52918.log 2023-10-26T05:59:43.841 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48808.log.gz 2023-10-26T05:59:43.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54382.log 2023-10-26T05:59:43.841 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51198.log.gz 2023-10-26T05:59:43.841 INFO:teuthology.orchestra.run.smithi071.stderr: 75.6% -- replaced with /var/log/ceph/ceph-client.admin.33132.log.gz 2023-10-26T05:59:43.841 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43908.log 2023-10-26T05:59:43.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54382.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52918.log.gz 2023-10-26T05:59:43.842 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39686.log 2023-10-26T05:59:43.842 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54382.log.gz 2023-10-26T05:59:43.842 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49827.log 2023-10-26T05:59:43.842 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43908.log.gz 2023-10-26T05:59:43.843 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39034.log 2023-10-26T05:59:43.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2023-10-26T05:59:43.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49827.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39686.log.gz 2023-10-26T05:59:43.843 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39034.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49827.log.gz 2023-10-26T05:59:43.843 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44971.log 2023-10-26T05:59:43.843 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39034.log.gz 2023-10-26T05:59:43.844 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55668.log 2023-10-26T05:59:43.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2023-10-26T05:59:43.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44971.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz 2023-10-26T05:59:43.844 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51042.log 2023-10-26T05:59:43.844 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44971.log.gz 2023-10-26T05:59:43.845 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55668.log.gz 2023-10-26T05:59:43.845 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43648.log 2023-10-26T05:59:43.845 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46512.log: /var/log/ceph/ceph-client.admin.51042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2023-10-26T05:59:43.845 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2023-10-26T05:59:43.845 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51042.log.gz 2023-10-26T05:59:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2023-10-26T05:59:43.846 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43648.log.gz 2023-10-26T05:59:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2023-10-26T05:59:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34728.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2023-10-26T05:59:43.846 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35195.log 2023-10-26T05:59:43.847 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2023-10-26T05:59:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52201.log 2023-10-26T05:59:43.847 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2023-10-26T05:59:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2023-10-26T05:59:43.847 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52201.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36088.log 2023-10-26T05:59:43.848 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35195.log.gz 2023-10-26T05:59:43.848 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52201.log.gz 2023-10-26T05:59:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55843.log 2023-10-26T05:59:43.848 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2023-10-26T05:59:43.848 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46538.log 2023-10-26T05:59:43.848 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36088.log.gz 2023-10-26T05:59:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34521.log 2023-10-26T05:59:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33080.log 2023-10-26T05:59:43.849 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55843.log.gz 2023-10-26T05:59:43.849 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53152.log 2023-10-26T05:59:43.849 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46538.log.gz 2023-10-26T05:59:43.850 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34521.log.gz 2023-10-26T05:59:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53909.log 2023-10-26T05:59:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33080.log.gz 2023-10-26T05:59:43.850 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2023-10-26T05:59:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.53152.log: /var/log/ceph/ceph-client.admin.53909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40033.log 2023-10-26T05:59:43.851 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5%/var/log/ceph/ceph-client.admin.36464.log: -- replaced with /var/log/ceph/ceph-client.admin.53909.log.gz 2023-10-26T05:59:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35522.log 2023-10-26T05:59:43.851 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2023-10-26T05:59:43.851 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40033.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2023-10-26T05:59:43.852 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40033.log.gz 2023-10-26T05:59:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43676.log 2023-10-26T05:59:43.852 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.35522.log.gz --verbose 2023-10-26T05:59:43.852 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/ceph-client.admin.48033.log 2023-10-26T05:59:43.852 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8%/var/log/ceph/ceph-client.admin.43676.log: -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2023-10-26T05:59:43.853 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43676.log.gz 2023-10-26T05:59:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44440.log 2023-10-26T05:59:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39873.log 2023-10-26T05:59:43.853 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4%/var/log/ceph/ceph-client.admin.44440.log: -- replaced with /var/log/ceph/ceph-client.admin.48033.log.gz 2023-10-26T05:59:43.853 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51250.log 2023-10-26T05:59:43.854 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44440.log.gz 2023-10-26T05:59:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39873.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39873.log.gz 2023-10-26T05:59:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40752.log 2023-10-26T05:59:43.854 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40379.log 2023-10-26T05:59:43.854 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2%/var/log/ceph/ceph-client.admin.40752.log: -- replaced with /var/log/ceph/ceph-client.admin.51250.log.gz 2023-10-26T05:59:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2023-10-26T05:59:43.855 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40752.log.gz 2023-10-26T05:59:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40379.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40379.log.gz 2023-10-26T05:59:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53335.log 2023-10-26T05:59:43.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2023-10-26T05:59:43.856 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8%/var/log/ceph/ceph-client.admin.53335.log: -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz 2023-10-26T05:59:43.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50453.log 2023-10-26T05:59:43.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip 87.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.45037.log 2023-10-26T05:59:43.856 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53335.log.gz 2023-10-26T05:59:43.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46661.log: /var/log/ceph/ceph-client.admin.50453.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2023-10-26T05:59:43.857 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2023-10-26T05:59:43.857 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56518.log 2023-10-26T05:59:43.857 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48638.log 2023-10-26T05:59:43.857 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6%/var/log/ceph/ceph-client.admin.56518.log: -- replaced with /var/log/ceph/ceph-client.admin.45037.log.gz 2023-10-26T05:59:43.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53935.log 2023-10-26T05:59:43.858 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56518.log.gz 2023-10-26T05:59:43.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48638.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48638.log.gz 2023-10-26T05:59:43.858 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37045.log 2023-10-26T05:59:43.858 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51894.log 2023-10-26T05:59:43.859 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53935.log.gz/var/log/ceph/ceph-client.admin.37045.log: 2023-10-26T05:59:43.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36437.log 2023-10-26T05:59:43.859 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37045.log.gz 2023-10-26T05:59:43.859 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51894.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51894.log.gz 2023-10-26T05:59:43.859 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38701.log 2023-10-26T05:59:43.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44492.log 2023-10-26T05:59:43.860 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6%/var/log/ceph/ceph-client.admin.38701.log: -- replaced with /var/log/ceph/ceph-client.admin.36437.log.gz 2023-10-26T05:59:43.860 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log 2023-10-26T05:59:43.860 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38701.log.gz 2023-10-26T05:59:43.860 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54065.log 2023-10-26T05:59:43.861 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36756.log 2023-10-26T05:59:43.861 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44492.log.gz 2023-10-26T05:59:43.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45745.log: /var/log/ceph/ceph-client.admin.54065.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2023-10-26T05:59:43.861 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54065.log.gz 2023-10-26T05:59:43.861 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45089.log 2023-10-26T05:59:43.861 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2023-10-26T05:59:43.862 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%/var/log/ceph/ceph-client.admin.45089.log: -- replaced with /var/log/ceph/ceph-client.admin.36756.log.gz 2023-10-26T05:59:43.862 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2023-10-26T05:59:43.862 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48950.log 2023-10-26T05:59:43.862 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45089.log.gz 2023-10-26T05:59:43.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39775.log: /var/log/ceph/ceph-client.admin.47594.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2023-10-26T05:59:43.863 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2023-10-26T05:59:43.863 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34284.log 2023-10-26T05:59:43.863 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36782.log 2023-10-26T05:59:43.863 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48950.log.gzgzip 2023-10-26T05:59:43.864 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40147.log 2023-10-26T05:59:43.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34284.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34284.log.gz 2023-10-26T05:59:43.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36782.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36782.log.gz 2023-10-26T05:59:43.864 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45469.log 2023-10-26T05:59:43.864 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40555.log 2023-10-26T05:59:43.865 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4%/var/log/ceph/ceph-client.admin.45469.log: -- replaced with /var/log/ceph/ceph-client.admin.40147.log.gz 2023-10-26T05:59:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44719.log 2023-10-26T05:59:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:gzip 87.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.50532.log 2023-10-26T05:59:43.865 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45469.log.gz 2023-10-26T05:59:43.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40555.log: /var/log/ceph/ceph-client.admin.44719.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40555.log.gz 2023-10-26T05:59:43.865 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44719.log.gz 2023-10-26T05:59:43.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2023-10-26T05:59:43.866 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2023-10-26T05:59:43.866 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5%/var/log/ceph/ceph-client.admin.43596.log: -- replaced with /var/log/ceph/ceph-client.admin.50532.log.gz 2023-10-26T05:59:43.866 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50122.log 2023-10-26T05:59:43.867 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2023-10-26T05:59:43.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47654.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2023-10-26T05:59:43.867 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38257.log 2023-10-26T05:59:43.867 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35802.log 2023-10-26T05:59:43.867 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6%/var/log/ceph/ceph-client.admin.38257.log: -- replaced with /var/log/ceph/ceph-client.admin.50122.log.gz 2023-10-26T05:59:43.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51709.log 2023-10-26T05:59:43.868 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38257.log.gz 2023-10-26T05:59:43.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35802.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35802.log.gz 2023-10-26T05:59:43.868 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35477.log 2023-10-26T05:59:43.868 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49056.log 2023-10-26T05:59:43.868 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%/var/log/ceph/ceph-client.0.35477.log: -- replaced with /var/log/ceph/ceph-client.admin.51709.log.gz 2023-10-26T05:59:43.869 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36678.log 2023-10-26T05:59:43.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49056.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49056.log.gz 2023-10-26T05:59:43.869 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2023-10-26T05:59:43.869 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.35477.log.gz 2023-10-26T05:59:43.869 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51380.log 2023-10-26T05:59:43.870 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35169.log 2023-10-26T05:59:43.870 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36678.log.gz 2023-10-26T05:59:43.870 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46583.log: /var/log/ceph/ceph-client.0.51380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2023-10-26T05:59:43.870 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34050.log 2023-10-26T05:59:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-26T05:59:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35169.log: /var/log/ceph/ceph-client.admin.34050.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35169.log.gz 2023-10-26T05:59:43.871 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34050.log.gz 2023-10-26T05:59:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51461.log 2023-10-26T05:59:43.871 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50650.log 2023-10-26T05:59:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.51461.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51461.log.gz 2023-10-26T05:59:43.872 INFO:teuthology.orchestra.run.smithi071.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.53152.log.gz 2023-10-26T05:59:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46356.log 2023-10-26T05:59:43.872 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38843.log 2023-10-26T05:59:43.872 INFO:teuthology.orchestra.run.smithi071.stderr: 95.1%/var/log/ceph/ceph-client.admin.50650.log: -- replaced with /var/log/ceph/ceph-client.0.54812.log.gz 2023-10-26T05:59:43.873 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%/var/log/ceph/ceph-client.admin.46356.log: -- replaced with /var/log/ceph/ceph-client.admin.50650.log.gz 2023-10-26T05:59:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2023-10-26T05:59:43.873 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46356.log.gz 2023-10-26T05:59:43.873 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40869.log 2023-10-26T05:59:43.873 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38843.log.gz 2023-10-26T05:59:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45257.log 87.2% 2023-10-26T05:59:43.874 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2023-10-26T05:59:43.874 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2023-10-26T05:59:43.874 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8%/var/log/ceph/ceph-client.admin.45257.log: -- replaced with /var/log/ceph/ceph-client.admin.40869.log.gzgzip 2023-10-26T05:59:43.874 INFO:teuthology.orchestra.run.smithi071.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54444.log 2023-10-26T05:59:43.875 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45257.log.gz 2023-10-26T05:59:43.875 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40303.log 2023-10-26T05:59:43.875 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47536.log: /var/log/ceph/ceph-client.admin.54444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43083.log 2023-10-26T05:59:43.875 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2023-10-26T05:59:43.875 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40303.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54444.log.gz 2023-10-26T05:59:43.876 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47926.log 2023-10-26T05:59:43.876 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40303.log.gz 2023-10-26T05:59:43.876 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44093.log 2023-10-26T05:59:43.876 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43083.log: /var/log/ceph/ceph-client.admin.47926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39303.log 2023-10-26T05:59:43.876 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43083.log.gz 2023-10-26T05:59:43.877 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44093.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47926.log.gz 2023-10-26T05:59:43.877 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2023-10-26T05:59:43.877 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44093.log.gz 2023-10-26T05:59:43.877 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37129.log 2023-10-26T05:59:43.877 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39303.log: /var/log/ceph/ceph-client.admin.36274.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39303.log.gz -5 2023-10-26T05:59:43.878 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55746.log 2023-10-26T05:59:43.878 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz/var/log/ceph/ceph-client.admin.37129.log: 2023-10-26T05:59:43.878 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37129.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2023-10-26T05:59:43.878 INFO:teuthology.orchestra.run.smithi071.stderr: 2023-10-26T05:59:43.878 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40843.log 2023-10-26T05:59:43.879 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7%/var/log/ceph/ceph-client.admin.54091.log: -- replaced with /var/log/ceph/ceph-client.admin.55746.log.gz 2023-10-26T05:59:43.879 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2023-10-26T05:59:43.879 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2023-10-26T05:59:43.879 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52620.log 2023-10-26T05:59:43.879 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40843.log: /var/log/ceph/ceph-client.admin.48059.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40843.log.gz 2023-10-26T05:59:43.880 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2023-10-26T05:59:43.880 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2023-10-26T05:59:43.880 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44255.log 2023-10-26T05:59:43.880 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52620.log: /var/log/ceph/ceph-client.admin.43534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40329.log 2023-10-26T05:59:43.880 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44255.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52620.log.gz 2023-10-26T05:59:43.881 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2023-10-26T05:59:43.881 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2023-10-26T05:59:43.881 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44255.log.gz 2023-10-26T05:59:43.881 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47300.log 2023-10-26T05:59:43.881 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45944.log 2023-10-26T05:59:43.881 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40329.log.gz 2023-10-26T05:59:43.882 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34729.log: /var/log/ceph/ceph-client.admin.47300.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2023-10-26T05:59:43.882 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47300.log.gz 2023-10-26T05:59:43.882 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2023-10-26T05:59:43.882 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50848.log 2023-10-26T05:59:43.883 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5%/var/log/ceph/ceph-client.admin.46223.log: -- replaced with /var/log/ceph/ceph-client.admin.45944.log.gz 2023-10-26T05:59:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48189.log 2023-10-26T05:59:43.883 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2023-10-26T05:59:43.883 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50848.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.32598.log 2023-10-26T05:59:43.883 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39381.log 2023-10-26T05:59:43.884 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50848.log.gz 2023-10-26T05:59:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48189.log: /var/log/ceph/ceph.tmp-client.admin.32598.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.32598.log.gz 2023-10-26T05:59:43.884 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48189.log.gz 2023-10-26T05:59:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35932.log 2023-10-26T05:59:43.884 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35143.log 2023-10-26T05:59:43.885 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5%/var/log/ceph/ceph-client.admin.35932.log: -- replaced with /var/log/ceph/ceph-client.admin.39381.log.gz 2023-10-26T05:59:43.885 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45197.log 2023-10-26T05:59:43.885 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35932.log.gz 2023-10-26T05:59:43.885 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53445.log 2023-10-26T05:59:43.885 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35143.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37701.log 2023-10-26T05:59:43.885 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35143.log.gz 2023-10-26T05:59:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45197.log: /var/log/ceph/ceph-client.admin.53445.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45197.log.gz 2023-10-26T05:59:43.886 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53445.log.gz 2023-10-26T05:59:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52970.log 2023-10-26T05:59:43.886 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50044.log 2023-10-26T05:59:43.886 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%/var/log/ceph/ceph-client.admin.52970.log: -- replaced with /var/log/ceph/ceph-client.admin.37701.log.gz 2023-10-26T05:59:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37905.log 2023-10-26T05:59:43.887 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52970.log.gz 2023-10-26T05:59:43.887 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50044.log: /var/log/ceph/ceph-client.admin.53471.log 2023-10-26T05:59:43.887 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2023-10-26T05:59:43.887 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50044.log.gz 2023-10-26T05:59:43.888 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37905.log: /var/log/ceph/ceph-client.admin.53471.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37905.log.gz 2023-10-26T05:59:43.888 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53471.log.gz 2023-10-26T05:59:43.888 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46145.log 2023-10-26T05:59:43.888 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42908.log 2023-10-26T05:59:43.888 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3%/var/log/ceph/ceph-client.admin.46145.log: -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 2023-10-26T05:59:43.888 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2023-10-26T05:59:43.889 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46145.log.gz 2023-10-26T05:59:43.889 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48320.log 2023-10-26T05:59:43.889 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42908.log: /var/log/ceph/ceph-client.admin.53022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36600.log 2023-10-26T05:59:43.889 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48320.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42908.log.gz 2023-10-26T05:59:43.889 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2023-10-26T05:59:43.890 INFO:teuthology.orchestra.run.smithi071.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.51380.log.gz 2023-10-26T05:59:43.890 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39170.log 2023-10-26T05:59:43.890 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48320.log.gz 2023-10-26T05:59:43.890 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2023-10-26T05:59:43.890 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36600.log: /var/log/ceph/ceph-client.admin.39170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50767.log 2023-10-26T05:59:43.891 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% 87.8%/var/log/ceph/ceph-client.admin.46278.log: -- replaced with /var/log/ceph/ceph-client.admin.39170.log.gz 2023-10-26T05:59:43.891 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55512.log 2023-10-26T05:59:43.891 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36600.log.gz 2023-10-26T05:59:43.891 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4%/var/log/ceph/ceph-client.admin.50767.log: -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2023-10-26T05:59:43.891 INFO:teuthology.orchestra.run.smithi071.stderr: 2023-10-26T05:59:43.892 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50767.log.gz 2023-10-26T05:59:43.892 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2023-10-26T05:59:43.892 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46330.log 2023-10-26T05:59:43.892 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55512.log.gz 2023-10-26T05:59:43.892 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41665.log 2023-10-26T05:59:43.892 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41814.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2023-10-26T05:59:43.893 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46330.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36730.log 2023-10-26T05:59:43.893 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2023-10-26T05:59:43.893 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46330.log.gz 2023-10-26T05:59:43.893 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41665.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-26T05:59:43.893 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36730.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2023-10-26T05:59:43.894 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41665.log.gz 2023-10-26T05:59:43.894 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36730.log.gz 2023-10-26T05:59:43.894 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34721.log 2023-10-26T05:59:43.894 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53523.log 2023-10-26T05:59:43.894 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36140.log 2023-10-26T05:59:43.895 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2023-10-26T05:59:43.895 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34721.log: /var/log/ceph/ceph-client.admin.53523.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53523.log.gz 87.6% 2023-10-26T05:59:43.895 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34721.log.gz 2023-10-26T05:59:43.895 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37441.log 2023-10-26T05:59:43.895 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52648.log 2023-10-26T05:59:43.896 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36140.log.gz/var/log/ceph/ceph-client.admin.37441.log: 2023-10-26T05:59:43.896 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51094.log 2023-10-26T05:59:43.896 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37441.log.gz 2023-10-26T05:59:43.896 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50618.log 2023-10-26T05:59:43.896 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52648.log.gz 2023-10-26T05:59:43.897 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35273.log 2023-10-26T05:59:43.897 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51094.log: /var/log/ceph/ceph-client.admin.50618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2023-10-26T05:59:43.897 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51094.log.gz 2023-10-26T05:59:43.897 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50618.log.gz 2023-10-26T05:59:43.897 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35273.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35273.log.gz 2023-10-26T05:59:43.898 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38727.log 2023-10-26T05:59:43.898 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51972.log 2023-10-26T05:59:43.898 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37545.log: /var/log/ceph/ceph-client.admin.38727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55895.log 2023-10-26T05:59:43.898 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51972.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38727.log.gz 2023-10-26T05:59:43.898 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2023-10-26T05:59:43.899 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51972.log.gz 2023-10-26T05:59:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37285.log 2023-10-26T05:59:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41112.log 2023-10-26T05:59:43.899 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55895.log: /var/log/ceph/ceph-client.admin.37285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34076.log 2023-10-26T05:59:43.900 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41112.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55895.log.gz 2023-10-26T05:59:43.900 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37285.log.gz 2023-10-26T05:59:43.900 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.48755.log 2023-10-26T05:59:43.900 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41112.log.gz 2023-10-26T05:59:43.900 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51816.log 2023-10-26T05:59:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34076.log: /var/log/ceph/ceph-client.admin.48755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55616.log 2023-10-26T05:59:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51816.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34076.log.gz 2023-10-26T05:59:43.901 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48755.log.gz 2023-10-26T05:59:43.901 INFO:teuthology.orchestra.run.smithi071.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51816.log.gz -5 2023-10-26T05:59:43.901 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44544.log 2023-10-26T05:59:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45169.log 2023-10-26T05:59:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52996.log 2023-10-26T05:59:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44544.log: /var/log/ceph/ceph-client.admin.45169.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44544.log.gz 2023-10-26T05:59:43.902 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55616.log.gz 2023-10-26T05:59:43.902 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54117.log 2023-10-26T05:59:43.903 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45169.log.gz 2023-10-26T05:59:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52996.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52996.log.gz 2023-10-26T05:59:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37071.log 2023-10-26T05:59:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32974.log 2023-10-26T05:59:43.903 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54117.log: /var/log/ceph/ceph-client.admin.37071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40199.log 2023-10-26T05:59:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32974.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54117.log.gz 2023-10-26T05:59:43.904 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37071.log.gz 2023-10-26T05:59:43.904 INFO:teuthology.orchestra.run.smithi071.stderr: 75.4% -- replaced with /var/log/ceph/ceph-client.admin.32974.log.gz 2023-10-26T05:59:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55243.log 2023-10-26T05:59:43.904 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45599.log 2023-10-26T05:59:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40199.log: /var/log/ceph/ceph-client.admin.55243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50715.log 2023-10-26T05:59:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.45599.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40199.log.gz 2023-10-26T05:59:43.905 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55243.log.gz 2023-10-26T05:59:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34336.log 2023-10-26T05:59:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45115.log 2023-10-26T05:59:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50715.log: /var/log/ceph/ceph-client.admin.34336.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50715.log.gz 2023-10-26T05:59:43.906 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34336.log.gz 2023-10-26T05:59:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44145.log 2023-10-26T05:59:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2023-10-26T05:59:43.906 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44145.log: /var/log/ceph/ceph-client.admin.45115.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45115.log.gz 2023-10-26T05:59:43.906 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44145.log.gz 2023-10-26T05:59:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47168.log 2023-10-26T05:59:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44745.log 2023-10-26T05:59:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48163.log: /var/log/ceph/ceph-client.admin.47168.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz 2023-10-26T05:59:43.907 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47168.log.gz 2023-10-26T05:59:43.907 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33994.log 2023-10-26T05:59:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38817.log 2023-10-26T05:59:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44745.log: /var/log/ceph/ceph-client.admin.33994.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44745.log.gz 2023-10-26T05:59:43.908 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33994.log.gz 2023-10-26T05:59:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43425.log 2023-10-26T05:59:43.908 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34154.log 2023-10-26T05:59:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38817.log: /var/log/ceph/ceph-client.admin.43425.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38817.log.gz 2023-10-26T05:59:43.909 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43425.log.gz 2023-10-26T05:59:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41216.log 2023-10-26T05:59:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56491.log 2023-10-26T05:59:43.909 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34154.log: /var/log/ceph/ceph-client.admin.41216.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34154.log.gz 2023-10-26T05:59:43.909 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41216.log.gz 2023-10-26T05:59:43.910 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50820.log 2023-10-26T05:59:43.910 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51068.log 2023-10-26T05:59:43.910 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56491.log: /var/log/ceph/ceph-client.admin.50820.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50820.log.gz 2023-10-26T05:59:43.910 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56491.log.gz 2023-10-26T05:59:43.910 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2023-10-26T05:59:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42804.log 2023-10-26T05:59:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51068.log: /var/log/ceph/ceph-client.admin.44939.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51068.log.gz 2023-10-26T05:59:43.911 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2023-10-26T05:59:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45415.log 2023-10-26T05:59:43.911 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36993.log 2023-10-26T05:59:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42804.log: /var/log/ceph/ceph-client.admin.45415.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42804.log.gz 2023-10-26T05:59:43.912 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45415.log.gz 2023-10-26T05:59:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51573.log 2023-10-26T05:59:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36036.log 2023-10-26T05:59:43.912 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36993.log: /var/log/ceph/ceph-client.admin.51573.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36993.log.gz 2023-10-26T05:59:43.913 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51573.log.gz 2023-10-26T05:59:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54920.log 2023-10-26T05:59:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43399.log 2023-10-26T05:59:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54920.log: /var/log/ceph/ceph-client.admin.36036.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36036.log.gz 2023-10-26T05:59:43.913 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54920.log.gz 2023-10-26T05:59:43.913 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38791.log 2023-10-26T05:59:43.914 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38309.log 2023-10-26T05:59:43.914 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43399.log: /var/log/ceph/ceph-client.admin.38791.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43399.log.gz 2023-10-26T05:59:43.914 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38791.log.gz 2023-10-26T05:59:43.914 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48495.log 2023-10-26T05:59:43.915 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56255.log 2023-10-26T05:59:43.915 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38309.log: /var/log/ceph/ceph-client.admin.48495.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38309.log.gz 2023-10-26T05:59:43.915 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48495.log.gz 2023-10-26T05:59:43.915 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2023-10-26T05:59:43.915 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39951.log 2023-10-26T05:59:43.916 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56255.log: /var/log/ceph/ceph-client.admin.39225.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56255.log.gz 2023-10-26T05:59:43.916 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2023-10-26T05:59:43.916 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2023-10-26T05:59:43.916 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53578.log 2023-10-26T05:59:43.916 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39951.log: /var/log/ceph/ceph-client.admin.53760.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39951.log.gz 2023-10-26T05:59:43.917 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2023-10-26T05:59:43.917 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43012.log 2023-10-26T05:59:43.917 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2023-10-26T05:59:43.917 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53578.log: /var/log/ceph/ceph-client.admin.43012.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53578.log.gz 2023-10-26T05:59:43.917 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43012.log.gz 2023-10-26T05:59:43.918 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47564.log 2023-10-26T05:59:43.918 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46948.log 2023-10-26T05:59:43.918 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41762.log: /var/log/ceph/ceph-client.admin.47564.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2023-10-26T05:59:43.918 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47564.log.gz 2023-10-26T05:59:43.918 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38982.log 2023-10-26T05:59:43.919 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52422.log 2023-10-26T05:59:43.919 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46948.log: /var/log/ceph/ceph-client.admin.38982.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46948.log.gz 2023-10-26T05:59:43.919 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38982.log.gz 2023-10-26T05:59:43.919 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44119.log 2023-10-26T05:59:43.919 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45892.log 2023-10-26T05:59:43.919 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52422.log: /var/log/ceph/ceph-client.admin.44119.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52422.log.gz 2023-10-26T05:59:43.920 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44119.log.gz 2023-10-26T05:59:43.920 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39199.log 2023-10-26T05:59:43.920 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36300.log 2023-10-26T05:59:43.920 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45892.log: /var/log/ceph/ceph-client.admin.39199.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2023-10-26T05:59:43.920 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39199.log.gz 2023-10-26T05:59:43.921 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44307.log 2023-10-26T05:59:43.921 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37853.log 2023-10-26T05:59:43.921 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36300.log: /var/log/ceph/ceph-client.admin.44307.log: 87.6% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36300.log.gz 2023-10-26T05:59:43.921 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44307.log.gz 2023-10-26T05:59:43.921 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54890.log 2023-10-26T05:59:43.922 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54576.log 2023-10-26T05:59:43.922 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37853.log: /var/log/ceph/ceph-client.admin.54890.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37853.log.gz 2023-10-26T05:59:43.922 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54890.log.gz 2023-10-26T05:59:43.922 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-26T05:59:43.923 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51868.log 2023-10-26T05:59:43.923 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54576.log: /var/log/ceph/ceph.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54576.log.gz 2023-10-26T05:59:43.923 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38594.log 2023-10-26T05:59:43.923 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51868.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51868.log.gz 2023-10-26T05:59:43.923 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49188.log 2023-10-26T05:59:43.924 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38594.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38594.log.gz 2023-10-26T05:59:43.924 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52357.log 2023-10-26T05:59:43.924 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49188.log: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-26T05:59:43.924 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49188.log.gz 2023-10-26T05:59:43.924 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33968.log 2023-10-26T05:59:43.925 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48898.log 2023-10-26T05:59:43.925 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52357.log: /var/log/ceph/ceph-client.admin.33968.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52357.log.gz 2023-10-26T05:59:43.925 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33968.log.gz 2023-10-26T05:59:43.925 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47274.log 2023-10-26T05:59:43.925 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55457.log 2023-10-26T05:59:43.926 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48898.log: /var/log/ceph/ceph-client.admin.47274.log: 86.8% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48898.log.gz 2023-10-26T05:59:43.926 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47274.log.gz 2023-10-26T05:59:43.926 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2023-10-26T05:59:43.926 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2023-10-26T05:59:43.926 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55457.log: /var/log/ceph/ceph-client.admin.42167.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55457.log.gz 2023-10-26T05:59:43.926 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2023-10-26T05:59:43.927 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2023-10-26T05:59:43.927 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56333.log 2023-10-26T05:59:43.927 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50794.log: /var/log/ceph/ceph-client.admin.41374.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2023-10-26T05:59:43.927 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2023-10-26T05:59:43.928 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52762.log 2023-10-26T05:59:43.928 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47248.log 2023-10-26T05:59:43.928 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56333.log: /var/log/ceph/ceph-client.admin.52762.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56333.log.gz 2023-10-26T05:59:43.928 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52762.log.gz 2023-10-26T05:59:43.928 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47790.log 2023-10-26T05:59:43.928 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53961.log 2023-10-26T05:59:43.929 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47248.log: /var/log/ceph/ceph-client.admin.47790.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47248.log.gz 2023-10-26T05:59:43.929 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47790.log.gz 2023-10-26T05:59:43.929 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33154.log 2023-10-26T05:59:43.929 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40895.log 2023-10-26T05:59:43.929 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53961.log: /var/log/ceph/ceph-client.admin.33154.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53961.log.gz 2023-10-26T05:59:43.930 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33154.log.gz 2023-10-26T05:59:43.930 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55321.log 2023-10-26T05:59:43.930 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53708.log 2023-10-26T05:59:43.930 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40895.log: /var/log/ceph/ceph-client.admin.55321.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40895.log.gz 2023-10-26T05:59:43.930 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55321.log.gz 2023-10-26T05:59:43.931 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35117.log 2023-10-26T05:59:43.931 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46974.log 2023-10-26T05:59:43.931 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53708.log: /var/log/ceph/ceph-client.admin.35117.log: 87.6% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53708.log.gz 2023-10-26T05:59:43.931 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35117.log.gz 2023-10-26T05:59:43.931 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43780.log 2023-10-26T05:59:43.932 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52866.log 2023-10-26T05:59:43.932 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46974.log: /var/log/ceph/ceph-client.admin.43780.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46974.log.gz 2023-10-26T05:59:43.932 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43780.log.gz 2023-10-26T05:59:43.932 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2023-10-26T05:59:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49108.log 2023-10-26T05:59:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52866.log: /var/log/ceph/ceph-client.admin.48365.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52866.log.gz 2023-10-26T05:59:43.933 INFO:teuthology.orchestra.run.smithi071.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2023-10-26T05:59:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51547.log 2023-10-26T05:59:43.933 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2023-10-26T05:59:43.934 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49108.log: /var/log/ceph/ceph-client.admin.51547.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49108.log.gz 2023-10-26T05:59:43.934 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51547.log.gz 2023-10-26T05:59:43.934 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52305.log 2023-10-26T05:59:43.934 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53987.log 2023-10-26T05:59:43.934 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42402.log: /var/log/ceph/ceph-client.admin.52305.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2023-10-26T05:59:43.935 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52305.log.gz 2023-10-26T05:59:43.935 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51326.log 2023-10-26T05:59:43.935 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36244.log 2023-10-26T05:59:43.935 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53987.log: /var/log/ceph/ceph-client.admin.51326.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53987.log.gz 2023-10-26T05:59:43.935 INFO:teuthology.orchestra.run.smithi071.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51326.log.gz 2023-10-26T05:59:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48864.log 2023-10-26T05:59:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38061.log 2023-10-26T05:59:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36244.log: /var/log/ceph/ceph-client.admin.48864.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36244.log.gz 2023-10-26T05:59:43.936 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz 2023-10-26T05:59:43.936 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44905.log 2023-10-26T05:59:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42558.log 2023-10-26T05:59:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38061.log: /var/log/ceph/ceph-client.admin.44905.log: 87.5% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38061.log.gz 2023-10-26T05:59:43.937 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44905.log.gz 2023-10-26T05:59:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39742.log 2023-10-26T05:59:43.937 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46635.log 2023-10-26T05:59:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42558.log: /var/log/ceph/ceph-client.admin.39742.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42558.log.gz 2023-10-26T05:59:43.938 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39742.log.gz 2023-10-26T05:59:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2023-10-26T05:59:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55869.log 2023-10-26T05:59:43.938 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46635.log: /var/log/ceph/ceph-client.admin.51146.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46635.log.gz 2023-10-26T05:59:43.939 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2023-10-26T05:59:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2023-10-26T05:59:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55869.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55869.log.gz 2023-10-26T05:59:43.939 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45983.log 2023-10-26T05:59:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36808.log 2023-10-26T05:59:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43243.log: /var/log/ceph/ceph-client.admin.45983.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2023-10-26T05:59:43.940 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45983.log.gz 2023-10-26T05:59:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56073.log 2023-10-26T05:59:43.940 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49541.log 2023-10-26T05:59:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36808.log: /var/log/ceph/ceph-client.admin.56073.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36808.log.gz 2023-10-26T05:59:43.941 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56073.log.gz 2023-10-26T05:59:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54864.log 2023-10-26T05:59:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48978.log 2023-10-26T05:59:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49541.log: /var/log/ceph/ceph-client.admin.54864.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49541.log.gz 2023-10-26T05:59:43.942 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54864.log.gz 2023-10-26T05:59:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2023-10-26T05:59:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48978.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48978.log.gz 2023-10-26T05:59:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2023-10-26T05:59:43.943 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2023-10-26T05:59:43.943 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54756.log: /var/log/ceph/ceph-client.admin.42219.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2023-10-26T05:59:43.943 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2023-10-26T05:59:43.943 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz/var/log/ceph/ceph-client.admin.56307.log: 2023-10-26T05:59:43.943 INFO:teuthology.orchestra.run.smithi071.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2023-10-26T05:59:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50330.log 2023-10-26T05:59:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41575.log 2023-10-26T05:59:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42115.log: /var/log/ceph/ceph-client.admin.50330.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2023-10-26T05:59:43.944 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50330.log.gz 2023-10-26T05:59:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53786.log 2023-10-26T05:59:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53098.log 2023-10-26T05:59:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41575.log: /var/log/ceph/ceph-client.admin.53786.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41575.log.gz 2023-10-26T05:59:43.945 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53786.log.gz 2023-10-26T05:59:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56413.log 2023-10-26T05:59:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47900.log 2023-10-26T05:59:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53098.log: /var/log/ceph/ceph-client.admin.56413.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53098.log.gz 2023-10-26T05:59:43.946 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56413.log.gz 2023-10-26T05:59:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37675.log 2023-10-26T05:59:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47900.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47900.log.gz 2023-10-26T05:59:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39925.log 2023-10-26T05:59:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2023-10-26T05:59:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37675.log: /var/log/ceph/ceph-client.admin.39925.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37675.log.gz 2023-10-26T05:59:43.947 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39925.log.gz 2023-10-26T05:59:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55995.log 2023-10-26T05:59:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50558.log 2023-10-26T05:59:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50096.log: /var/log/ceph/ceph-client.admin.55995.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2023-10-26T05:59:43.948 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.55995.log.gz 2023-10-26T05:59:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52682.log 2023-10-26T05:59:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50480.log 2023-10-26T05:59:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50558.log: /var/log/ceph/ceph-client.admin.52682.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50558.log.gz 2023-10-26T05:59:43.949 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52682.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36218.log 2023-10-26T05:59:43.949 INFO:teuthology.orchestra.run.smithi071.stderr: 2023-10-26T05:59:43.949 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50480.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50480.log.gz 2023-10-26T05:59:43.949 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33862.log 2023-10-26T05:59:43.949 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33836.log 2023-10-26T05:59:43.949 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36218.log: /var/log/ceph/ceph-client.admin.33862.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36218.log.gz 2023-10-26T05:59:43.950 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33862.log.gz 2023-10-26T05:59:43.950 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2023-10-26T05:59:43.950 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33836.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33836.log.gz 2023-10-26T05:59:43.950 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49596.log 2023-10-26T05:59:43.950 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37155.log 2023-10-26T05:59:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48703.log: /var/log/ceph/ceph-client.admin.49596.log: 87.9% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz 2023-10-26T05:59:43.951 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49596.log.gz 2023-10-26T05:59:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55642.log 2023-10-26T05:59:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54410.log 2023-10-26T05:59:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37155.log: /var/log/ceph/ceph-client.admin.55642.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37155.log.gz 2023-10-26T05:59:43.952 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55642.log.gz 2023-10-26T05:59:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2023-10-26T05:59:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35776.log 2023-10-26T05:59:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54410.log: /var/log/ceph/ceph-client.admin.52788.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2023-10-26T05:59:43.952 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54410.log.gz 2023-10-26T05:59:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45801.log 2023-10-26T05:59:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2023-10-26T05:59:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35776.log: /var/log/ceph/ceph-client.admin.45801.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35776.log.gz 2023-10-26T05:59:43.953 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45801.log.gz 2023-10-26T05:59:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49316.log 2023-10-26T05:59:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2023-10-26T05:59:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41268.log: /var/log/ceph/ceph-client.admin.49316.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2023-10-26T05:59:43.954 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49316.log.gz 2023-10-26T05:59:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42986.log 2023-10-26T05:59:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42856.log 2023-10-26T05:59:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46304.log: /var/log/ceph/ceph-client.admin.42986.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2023-10-26T05:59:43.955 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42986.log.gz 2023-10-26T05:59:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34515.log 2023-10-26T05:59:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36359.log 2023-10-26T05:59:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42856.log: /var/log/ceph/ceph-client.admin.34515.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42856.log.gz 2023-10-26T05:59:43.956 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34515.log.gz 2023-10-26T05:59:43.956 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49423.log 2023-10-26T05:59:43.956 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56047.log 2023-10-26T05:59:43.956 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36359.log: /var/log/ceph/ceph-client.admin.49423.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36359.log.gz 2023-10-26T05:59:43.956 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49423.log.gz 2023-10-26T05:59:43.956 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49371.log 2023-10-26T05:59:43.957 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47952.log 2023-10-26T05:59:43.957 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56047.log: /var/log/ceph/ceph-client.0.49371.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56047.log.gz 2023-10-26T05:59:43.957 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34519.log 2023-10-26T05:59:43.957 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47952.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47952.log.gz 2023-10-26T05:59:43.958 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38877.log 2023-10-26T05:59:43.958 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34519.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34519.log.gz 2023-10-26T05:59:43.958 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51946.log 2023-10-26T05:59:43.958 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38877.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38877.log.gz 2023-10-26T05:59:43.958 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46252.log 2023-10-26T05:59:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51946.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51946.log.gz 2023-10-26T05:59:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2023-10-26T05:59:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46252.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46252.log.gz 2023-10-26T05:59:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54328.log 2023-10-26T05:59:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48924.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2023-10-26T05:59:43.960 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38087.log 2023-10-26T05:59:43.960 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54328.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54328.log.gz 2023-10-26T05:59:43.960 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42323.log 2023-10-26T05:59:43.960 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38087.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38087.log.gz 2023-10-26T05:59:43.960 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53387.log 2023-10-26T05:59:43.962 INFO:teuthology.orchestra.run.smithi086.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-26T05:59:43.962 INFO:teuthology.orchestra.run.smithi086.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-26T05:59:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.42323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39433.log 2023-10-26T05:59:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53387.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53387.log.gz 2023-10-26T05:59:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43702.log 2023-10-26T05:59:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39433.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39433.log.gz 2023-10-26T05:59:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40512.log 2023-10-26T05:59:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43702.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43702.log.gz 2023-10-26T05:59:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46382.log 2023-10-26T05:59:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40512.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40512.log.gz 2023-10-26T05:59:43.980 INFO:teuthology.orchestra.run.smithi071.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.49371.log.gz 2023-10-26T05:59:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34520.log 2023-10-26T05:59:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46382.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46382.log.gz 2023-10-26T05:59:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55295.log 2023-10-26T05:59:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34520.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34520.log.gz 2023-10-26T05:59:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39144.log 2023-10-26T05:59:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55295.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55295.log.gz 2023-10-26T05:59:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32933.log 2023-10-26T05:59:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39144.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39144.log.gz 2023-10-26T05:59:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49853.log 2023-10-26T05:59:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32933.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.32933.log.gz 2023-10-26T05:59:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51787.log 2023-10-26T05:59:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49853.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49853.log.gz 2023-10-26T05:59:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48085.log 2023-10-26T05:59:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51787.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51787.log.gz 2023-10-26T05:59:43.982 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2023-10-26T05:59:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48085.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48085.log.gz 2023-10-26T05:59:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44362.log 2023-10-26T05:59:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49266.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2023-10-26T05:59:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56568.log 2023-10-26T05:59:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44362.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44362.log.gz 2023-10-26T05:59:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43478.log 2023-10-26T05:59:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56568.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56568.log.gz 2023-10-26T05:59:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2023-10-26T05:59:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43478.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43478.log.gz 2023-10-26T05:59:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36114.log 2023-10-26T05:59:43.985 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52175.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2023-10-26T05:59:43.985 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32984.log 2023-10-26T05:59:43.985 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36114.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36114.log.gz 2023-10-26T05:59:43.985 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45231.log 2023-10-26T05:59:43.985 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.32984.log: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.32984.log.gz 2023-10-26T05:59:43.986 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54680.log 2023-10-26T05:59:43.986 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45231.log.gz 2023-10-26T05:59:43.986 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35750.log 2023-10-26T05:59:43.986 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54680.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54680.log.gz 2023-10-26T05:59:43.987 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41523.log 2023-10-26T05:59:43.987 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35750.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2023-10-26T05:59:43.987 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34022.log 2023-10-26T05:59:43.987 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41523.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41523.log.gz 2023-10-26T05:59:43.987 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45719.log 2023-10-26T05:59:43.987 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34022.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34022.log.gz 2023-10-26T05:59:43.988 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2023-10-26T05:59:43.988 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45719.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45719.log.gz 2023-10-26T05:59:43.988 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35906.log 2023-10-26T05:59:43.988 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43858.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2023-10-26T05:59:43.988 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47816.log 2023-10-26T05:59:43.989 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35906.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35906.log.gz 2023-10-26T05:59:43.989 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49911.log 2023-10-26T05:59:43.989 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47816.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47816.log.gz 2023-10-26T05:59:43.989 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53682.log 2023-10-26T05:59:43.989 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49911.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49911.log.gz 2023-10-26T05:59:43.990 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40433.log 2023-10-26T05:59:43.990 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53682.log.gz 2023-10-26T05:59:43.990 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2023-10-26T05:59:43.990 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.40433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55100.log 2023-10-26T05:59:43.990 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49989.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2023-10-26T05:59:43.991 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35091.log 2023-10-26T05:59:43.991 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55100.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55100.log.gz 2023-10-26T05:59:43.991 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41164.log 2023-10-26T05:59:43.991 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35091.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35091.log.gz 2023-10-26T05:59:43.992 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-26T05:59:43.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41164.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41164.log.gz 2023-10-26T05:59:43.992 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39495.log 2023-10-26T05:59:43.992 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph.audit.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-26T05:59:43.992 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34258.log 2023-10-26T05:59:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39495.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39495.log.gz 2023-10-26T05:59:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-26T05:59:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34258.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34258.log.gz 2023-10-26T05:59:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34389.log 2023-10-26T05:59:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37207.log 2023-10-26T05:59:43.993 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34389.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34389.log.gz 2023-10-26T05:59:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47764.log 2023-10-26T05:59:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37207.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37207.log.gz 2023-10-26T05:59:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40277.log 2023-10-26T05:59:43.994 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47764.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47764.log.gz 2023-10-26T05:59:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56125.log 2023-10-26T05:59:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40277.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40277.log.gz 2023-10-26T05:59:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2023-10-26T05:59:43.995 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56125.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56125.log.gz 2023-10-26T05:59:43.997 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35828.log 2023-10-26T05:59:43.997 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52892.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz 2023-10-26T05:59:43.997 INFO:teuthology.orchestra.run.smithi071.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.0.45599.log.gz 2023-10-26T05:59:43.998 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-26T05:59:43.998 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35828.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35828.log.gz 2023-10-26T05:59:43.998 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49082.log 2023-10-26T05:59:43.998 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2023-10-26T05:59:43.998 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49082.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.49082.log.gz 2023-10-26T05:59:44.010 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56021.log 2023-10-26T05:59:44.011 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50070.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2023-10-26T05:59:44.011 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48573.log 2023-10-26T05:59:44.012 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.56021.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56021.log.gz 2023-10-26T05:59:44.030 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47082.log 2023-10-26T05:59:44.031 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48573.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48573.log.gz 2023-10-26T05:59:44.038 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37019.log 2023-10-26T05:59:44.039 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47082.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47082.log.gz 2023-10-26T05:59:44.042 INFO:teuthology.orchestra.run.smithi071.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.42323.log.gz 2023-10-26T05:59:44.046 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42009.log 2023-10-26T05:59:44.047 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37019.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37019.log.gz 2023-10-26T05:59:44.047 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38283.log 2023-10-26T05:59:44.048 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42009.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42009.log.gz 2023-10-26T05:59:44.049 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36192.log 2023-10-26T05:59:44.050 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38283.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38283.log.gz 2023-10-26T05:59:44.050 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52566.log 2023-10-26T05:59:44.051 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36192.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36192.log.gz 2023-10-26T05:59:44.051 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2023-10-26T05:59:44.053 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52566.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52566.log.gz 2023-10-26T05:59:44.053 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45389.log 2023-10-26T05:59:44.053 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37389.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2023-10-26T05:59:44.054 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43962.log 2023-10-26T05:59:44.054 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45389.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45389.log.gz 2023-10-26T05:59:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33890.log 2023-10-26T05:59:44.070 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.43962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48729.log 2023-10-26T05:59:44.071 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33890.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33890.log.gz 2023-10-26T05:59:44.094 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50174.log 2023-10-26T05:59:44.095 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48729.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48729.log.gz 2023-10-26T05:59:44.106 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41057.log 2023-10-26T05:59:44.107 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50174.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50174.log.gz 2023-10-26T05:59:44.114 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52540.log 2023-10-26T05:59:44.115 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41057.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41057.log.gz 2023-10-26T05:59:44.122 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34909.log 2023-10-26T05:59:44.123 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52540.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52540.log.gz 2023-10-26T05:59:44.130 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2023-10-26T05:59:44.131 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34909.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34909.log.gz 2023-10-26T05:59:44.138 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37259.log 2023-10-26T05:59:44.139 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49240.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2023-10-26T05:59:44.146 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54184.log 2023-10-26T05:59:44.147 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37259.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37259.log.gz 2023-10-26T05:59:44.154 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47714.log 2023-10-26T05:59:44.155 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54184.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54184.log.gz 2023-10-26T05:59:44.162 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37761.log 2023-10-26T05:59:44.163 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47714.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47714.log.gz 2023-10-26T05:59:44.170 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33457.log 2023-10-26T05:59:44.171 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37761.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37761.log.gz 2023-10-26T05:59:44.178 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2023-10-26T05:59:44.179 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33457.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.33457.log.gz 2023-10-26T05:59:44.185 INFO:teuthology.orchestra.run.smithi071.stderr: 92.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46197.log 2023-10-26T05:59:44.185 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49449.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2023-10-26T05:59:44.186 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-26T05:59:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2023-10-26T05:59:44.203 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46197.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2023-10-26T05:59:44.203 INFO:teuthology.orchestra.run.smithi086.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-26T05:59:44.214 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41983.log 2023-10-26T05:59:44.215 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48836.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2023-10-26T05:59:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2023-10-26T05:59:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41983.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41983.log.gz 2023-10-26T05:59:44.254 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44693.log 2023-10-26T05:59:44.255 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34724.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2023-10-26T05:59:44.256 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-26T05:59:44.270 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2023-10-26T05:59:44.271 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44693.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44693.log.gz 2023-10-26T05:59:44.278 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35686.log 2023-10-26T05:59:44.279 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37623.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2023-10-26T05:59:44.290 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33058.log 2023-10-26T05:59:44.291 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35686.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35686.log.gz 2023-10-26T05:59:44.302 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48521.log 2023-10-26T05:59:44.303 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33058.log.gz 2023-10-26T05:59:44.305 INFO:teuthology.orchestra.run.smithi071.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.40433.log.gz 2023-10-26T05:59:44.306 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46765.log 2023-10-26T05:59:44.306 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48521.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48521.log.gz 2023-10-26T05:59:44.307 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2023-10-26T05:59:44.307 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46765.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46765.log.gz 2023-10-26T05:59:44.308 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43506.log 2023-10-26T05:59:44.308 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43728.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2023-10-26T05:59:44.309 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39251.log 2023-10-26T05:59:44.309 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43506.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43506.log.gz 2023-10-26T05:59:44.310 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40636.log 2023-10-26T05:59:44.311 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39251.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39251.log.gz 2023-10-26T05:59:44.312 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40726.log 2023-10-26T05:59:44.312 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40636.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40636.log.gz 2023-10-26T05:59:44.312 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46791.log 2023-10-26T05:59:44.313 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40726.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40726.log.gz 2023-10-26T05:59:44.314 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38956.log 2023-10-26T05:59:44.314 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46791.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46791.log.gz 2023-10-26T05:59:44.315 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54496.log 2023-10-26T05:59:44.315 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38956.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38956.log.gz 2023-10-26T05:59:44.316 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44044.log 2023-10-26T05:59:44.316 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54496.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54496.log.gz 2023-10-26T05:59:44.317 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48606.log 2023-10-26T05:59:44.317 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44044.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44044.log.gz 2023-10-26T05:59:44.318 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47680.log 2023-10-26T05:59:44.319 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48606.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48606.log.gz 2023-10-26T05:59:44.319 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2023-10-26T05:59:44.320 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47680.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47680.log.gz 2023-10-26T05:59:44.320 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40067.log 2023-10-26T05:59:44.321 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35221.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gz 2023-10-26T05:59:44.321 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42619.log 2023-10-26T05:59:44.322 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40067.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40067.log.gz 2023-10-26T05:59:44.322 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33118.log 2023-10-26T05:59:44.323 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42619.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42619.log.gz 2023-10-26T05:59:44.323 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50375.log 2023-10-26T05:59:44.324 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33118.log.gz 2023-10-26T05:59:44.324 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37519.log 2023-10-26T05:59:44.325 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50375.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50375.log.gz 2023-10-26T05:59:44.326 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47026.log 2023-10-26T05:59:44.326 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37519.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37519.log.gz 2023-10-26T05:59:44.327 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49736.log 2023-10-26T05:59:44.330 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47026.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47026.log.gz 2023-10-26T05:59:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36888.log 2023-10-26T05:59:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49736.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49736.log.gz 2023-10-26T05:59:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34514.log 2023-10-26T05:59:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36888.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36888.log.gz 2023-10-26T05:59:44.331 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2023-10-26T05:59:44.332 INFO:teuthology.orchestra.run.smithi086.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-26T05:59:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34514.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34514.log.gz 2023-10-26T05:59:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46035.log 2023-10-26T05:59:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33404.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2023-10-26T05:59:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47628.log 2023-10-26T05:59:44.334 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46035.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46035.log.gz 2023-10-26T05:59:44.335 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55921.log 2023-10-26T05:59:44.336 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47628.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47628.log.gz 2023-10-26T05:59:44.336 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52944.log 2023-10-26T05:59:44.336 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55921.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55921.log.gz 2023-10-26T05:59:44.337 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41138.log 2023-10-26T05:59:44.337 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52944.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52944.log.gz 2023-10-26T05:59:44.338 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2023-10-26T05:59:44.339 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41138.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41138.log.gz 2023-10-26T05:59:44.339 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39608.log 2023-10-26T05:59:44.340 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38229.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2023-10-26T05:59:44.340 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2023-10-26T05:59:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39608.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39608.log.gz 2023-10-26T05:59:44.341 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53734.log 2023-10-26T05:59:44.342 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35724.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2023-10-26T05:59:44.342 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50401.log 2023-10-26T05:59:44.343 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53734.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53734.log.gz 2023-10-26T05:59:44.343 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41895.log 2023-10-26T05:59:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50401.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50401.log.gz 2023-10-26T05:59:44.344 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55538.log 2023-10-26T05:59:44.345 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41895.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41895.log.gz 2023-10-26T05:59:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40005.log 2023-10-26T05:59:44.346 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55538.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55538.log.gz 2023-10-26T05:59:44.347 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53604.log 2023-10-26T05:59:44.347 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40005.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40005.log.gz 2023-10-26T05:59:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33942.log 2023-10-26T05:59:44.348 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53604.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53604.log.gz 2023-10-26T05:59:44.349 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43622.log 2023-10-26T05:59:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33942.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33942.log.gz 2023-10-26T05:59:44.350 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41631.log 2023-10-26T05:59:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43622.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43622.log.gz 2023-10-26T05:59:44.351 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42830.log 2023-10-26T05:59:44.352 INFO:teuthology.orchestra.run.smithi086.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-26T05:59:44.353 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41631.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41631.log.gz 2023-10-26T05:59:44.353 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35013.log 2023-10-26T05:59:44.353 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42830.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42830.log.gz 2023-10-26T05:59:44.355 INFO:teuthology.orchestra.run.smithi086.stderr: 2023-10-26T05:59:44.355 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m0.900s 2023-10-26T05:59:44.355 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m4.077s 2023-10-26T05:59:44.356 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.186s 2023-10-26T05:59:44.357 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43135.log 2023-10-26T05:59:44.357 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35013.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35013.log.gz 2023-10-26T05:59:44.357 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46061.log 2023-10-26T05:59:44.357 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43135.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43135.log.gz 2023-10-26T05:59:44.358 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40581.log 2023-10-26T05:59:44.358 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46061.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46061.log.gz 2023-10-26T05:59:44.358 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54983.log 2023-10-26T05:59:44.359 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40581.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40581.log.gz 2023-10-26T05:59:44.360 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38115.log 2023-10-26T05:59:44.360 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54983.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54983.log.gz 2023-10-26T05:59:44.360 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37729.log 2023-10-26T05:59:44.361 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38115.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38115.log.gz 2023-10-26T05:59:44.361 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33092.log 2023-10-26T05:59:44.362 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37729.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37729.log.gz 2023-10-26T05:59:44.362 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47326.log 2023-10-26T05:59:44.363 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33092.log.gz 2023-10-26T05:59:44.363 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2023-10-26T05:59:44.364 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47326.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47326.log.gz 2023-10-26T05:59:44.364 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50910.log 2023-10-26T05:59:44.365 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47222.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2023-10-26T05:59:44.365 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2023-10-26T05:59:44.366 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50910.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50910.log.gz 2023-10-26T05:59:44.366 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33069.log 2023-10-26T05:59:44.367 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50226.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gz 2023-10-26T05:59:44.367 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55720.log 2023-10-26T05:59:44.368 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33069.log.gz 2023-10-26T05:59:44.369 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48547.log 2023-10-26T05:59:44.369 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55720.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55720.log.gz 2023-10-26T05:59:44.370 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44333.log 2023-10-26T05:59:44.370 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48547.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48547.log.gz 2023-10-26T05:59:44.371 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38361.log 2023-10-26T05:59:44.372 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44333.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44333.log.gz 2023-10-26T05:59:44.372 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40673.log 2023-10-26T05:59:44.373 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38361.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38361.log.gz 2023-10-26T05:59:44.373 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54957.log 2023-10-26T05:59:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40673.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40673.log.gz 2023-10-26T05:59:44.374 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39556.log 2023-10-26T05:59:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54957.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54957.log.gz 2023-10-26T05:59:44.375 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38203.log 2023-10-26T05:59:44.376 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39556.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39556.log.gz 2023-10-26T05:59:44.377 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38177.log 2023-10-26T05:59:44.377 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38203.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38203.log.gz 2023-10-26T05:59:44.378 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50427.log 2023-10-26T05:59:44.378 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38177.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38177.log.gz 2023-10-26T05:59:44.379 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41445.log 2023-10-26T05:59:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50427.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50427.log.gz 2023-10-26T05:59:44.380 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51016.log 2023-10-26T05:59:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41445.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41445.log.gz 2023-10-26T05:59:44.381 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2023-10-26T05:59:44.382 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51016.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51016.log.gz 2023-10-26T05:59:44.382 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50304.log 2023-10-26T05:59:44.383 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39807.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2023-10-26T05:59:44.384 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46856.log 2023-10-26T05:59:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50304.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50304.log.gz 2023-10-26T05:59:44.385 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2023-10-26T05:59:44.386 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46856.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46856.log.gz 2023-10-26T05:59:44.386 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2023-10-26T05:59:44.387 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34517.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2023-10-26T05:59:44.387 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33705.log 2023-10-26T05:59:44.388 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38388.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2023-10-26T05:59:44.388 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43302.log 2023-10-26T05:59:44.389 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33705.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.33705.log.gz 2023-10-26T05:59:44.389 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40251.log 2023-10-26T05:59:44.390 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43302.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43302.log.gz 2023-10-26T05:59:44.391 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34727.log 2023-10-26T05:59:44.391 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40251.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40251.log.gz 2023-10-26T05:59:44.392 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34415.log 2023-10-26T05:59:44.392 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34727.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34727.log.gz 2023-10-26T05:59:44.393 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46824.log 2023-10-26T05:59:44.393 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34415.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34415.log.gz 2023-10-26T05:59:44.394 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44648.log 2023-10-26T05:59:44.395 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46824.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46824.log.gz 2023-10-26T05:59:44.395 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45545.log 2023-10-26T05:59:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44648.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44648.log.gz 2023-10-26T05:59:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47978.log 2023-10-26T05:59:44.397 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45545.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45545.log.gz 2023-10-26T05:59:44.397 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43109.log 2023-10-26T05:59:44.398 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47978.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47978.log.gz 2023-10-26T05:59:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38492.log 2023-10-26T05:59:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43109.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43109.log.gz 2023-10-26T05:59:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40700.log 2023-10-26T05:59:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38492.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38492.log.gz 2023-10-26T05:59:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49160.log 2023-10-26T05:59:44.402 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40700.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40700.log.gz 2023-10-26T05:59:44.402 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36834.log 2023-10-26T05:59:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49160.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.49160.log.gz 2023-10-26T05:59:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39118.log 2023-10-26T05:59:44.404 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36834.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36834.log.gz 2023-10-26T05:59:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48004.log 2023-10-26T05:59:44.405 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39118.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39118.log.gz 2023-10-26T05:59:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38648.log 2023-10-26T05:59:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48004.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48004.log.gz 2023-10-26T05:59:44.407 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54152.log 2023-10-26T05:59:44.410 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.0.38648.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.38648.log.gz 2023-10-26T05:59:44.410 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36333.log 2023-10-26T05:59:44.411 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54152.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54152.log.gz 2023-10-26T05:59:44.412 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36010.log 2023-10-26T05:59:44.412 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36333.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36333.log.gz 2023-10-26T05:59:44.413 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41242.log 2023-10-26T05:59:44.414 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36010.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36010.log.gz 2023-10-26T05:59:44.414 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37097.log 2023-10-26T05:59:44.415 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41242.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41242.log.gz 2023-10-26T05:59:44.415 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2023-10-26T05:59:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37097.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37097.log.gz 2023-10-26T05:59:44.416 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2023-10-26T05:59:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36704.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2023-10-26T05:59:44.417 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40921.log 2023-10-26T05:59:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44281.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2023-10-26T05:59:44.418 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51683.log 2023-10-26T05:59:44.419 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40921.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40921.log.gz 2023-10-26T05:59:44.420 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55038.log 2023-10-26T05:59:44.420 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51683.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51683.log.gz 2023-10-26T05:59:44.421 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2023-10-26T05:59:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55038.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55038.log.gz 2023-10-26T05:59:44.422 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55009.log 2023-10-26T05:59:44.423 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49656.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2023-10-26T05:59:44.423 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39530.log 2023-10-26T05:59:44.424 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55009.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55009.log.gz 2023-10-26T05:59:44.424 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50506.log 2023-10-26T05:59:44.425 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39530.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39530.log.gz 2023-10-26T05:59:44.426 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36914.log 2023-10-26T05:59:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50506.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50506.log.gz 2023-10-26T05:59:44.427 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40778.log 2023-10-26T05:59:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36914.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36914.log.gz 2023-10-26T05:59:44.428 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48782.log 2023-10-26T05:59:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40778.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40778.log.gz 2023-10-26T05:59:44.429 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2023-10-26T05:59:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48782.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48782.log.gz 2023-10-26T05:59:44.430 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39716.log 2023-10-26T05:59:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37233.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2023-10-26T05:59:44.431 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47842.log 2023-10-26T05:59:44.432 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39716.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39716.log.gz 2023-10-26T05:59:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40953.log 2023-10-26T05:59:44.433 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47842.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47842.log.gz 2023-10-26T05:59:44.434 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40979.log 2023-10-26T05:59:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40953.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40953.log.gz 2023-10-26T05:59:44.435 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2023-10-26T05:59:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40979.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40979.log.gz 2023-10-26T05:59:44.436 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52078.log 2023-10-26T05:59:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43334.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2023-10-26T05:59:44.437 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2023-10-26T05:59:44.438 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52078.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52078.log.gz 2023-10-26T05:59:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52024.log 2023-10-26T05:59:44.439 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43562.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2023-10-26T05:59:44.440 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2023-10-26T05:59:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52024.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52024.log.gz 2023-10-26T05:59:44.441 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41788.log 2023-10-26T05:59:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54354.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2023-10-26T05:59:44.442 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42934.log 2023-10-26T05:59:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.41788.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41788.log.gz 2023-10-26T05:59:44.443 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44466.log 2023-10-26T05:59:44.444 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42934.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42934.log.gz 2023-10-26T05:59:44.444 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53812.log 2023-10-26T05:59:44.445 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44466.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44466.log.gz 2023-10-26T05:59:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33010.log 2023-10-26T05:59:44.446 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.53812.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53812.log.gz 2023-10-26T05:59:44.447 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52708.log 2023-10-26T05:59:44.448 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.33010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33010.log.gz 2023-10-26T05:59:44.448 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37311.log 2023-10-26T05:59:44.449 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52708.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52708.log.gz 2023-10-26T05:59:44.449 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38518.log 2023-10-26T05:59:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37311.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37311.log.gz 2023-10-26T05:59:44.450 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54602.log 2023-10-26T05:59:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38518.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38518.log.gz 2023-10-26T05:59:44.451 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37337.log 2023-10-26T05:59:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54602.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54602.log.gz 2023-10-26T05:59:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2023-10-26T05:59:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.37337.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37337.log.gz 2023-10-26T05:59:44.453 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35854.log 2023-10-26T05:59:44.454 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50278.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2023-10-26T05:59:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35658.log 2023-10-26T05:59:44.455 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35854.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35854.log.gz 2023-10-26T05:59:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51998.log 2023-10-26T05:59:44.456 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35658.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35658.log.gz 2023-10-26T05:59:44.457 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38009.log 2023-10-26T05:59:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51998.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51998.log.gz 2023-10-26T05:59:44.458 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39407.log 2023-10-26T05:59:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38009.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38009.log.gz 2023-10-26T05:59:44.459 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45918.log 2023-10-26T05:59:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.39407.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39407.log.gz 2023-10-26T05:59:44.460 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52149.log 2023-10-26T05:59:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45918.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45918.log.gz 2023-10-26T05:59:44.462 INFO:teuthology.orchestra.run.smithi071.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.43962.log.gz 2023-10-26T05:59:44.462 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55379.log 2023-10-26T05:59:44.463 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2023-10-26T05:59:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55379.log: /var/log/ceph/ceph-client.admin.52149.log: 87.5% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52149.log.gz 2023-10-26T05:59:44.464 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55379.log.gz 2023-10-26T05:59:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2023-10-26T05:59:44.464 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52734.log 2023-10-26T05:59:44.466 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.47379.log: /var/log/ceph/ceph-client.admin.53656.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2023-10-26T05:59:44.466 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2023-10-26T05:59:44.466 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2023-10-26T05:59:44.466 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2023-10-26T05:59:44.467 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52734.log: /var/log/ceph/ceph-client.admin.43451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52734.log.gz 2023-10-26T05:59:44.467 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2023-10-26T05:59:44.467 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log 2023-10-26T05:59:44.467 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44570.log 2023-10-26T05:59:44.470 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.36542.log: /var/log/ceph/ceph-client.admin.45363.log: 87.2% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2023-10-26T05:59:44.471 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2023-10-26T05:59:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35065.log 2023-10-26T05:59:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34710.log 2023-10-26T05:59:44.471 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44570.log: /var/log/ceph/ceph-client.admin.35065.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44570.log.gz 2023-10-26T05:59:44.471 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35065.log.gz 2023-10-26T05:59:44.472 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51920.log 2023-10-26T05:59:44.472 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46739.log 2023-10-26T05:59:44.472 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34710.log: /var/log/ceph/ceph-client.admin.51920.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34710.log.gz 2023-10-26T05:59:44.472 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51920.log.gz 2023-10-26T05:59:44.472 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45063.log 2023-10-26T05:59:44.473 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52227.log 2023-10-26T05:59:44.473 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46739.log: /var/log/ceph/ceph-client.admin.45063.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46739.log.gz 2023-10-26T05:59:44.473 INFO:teuthology.orchestra.run.smithi071.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45063.log.gz 2023-10-26T05:59:44.473 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34935.log 2023-10-26T05:59:44.478 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46408.log 2023-10-26T05:59:44.478 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.52227.log: /var/log/ceph/ceph-client.admin.34935.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52227.log.gz 2023-10-26T05:59:44.478 INFO:teuthology.orchestra.run.smithi071.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34935.log.gz 2023-10-26T05:59:44.478 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41497.log 2023-10-26T05:59:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40486.log 2023-10-26T05:59:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.46408.log: /var/log/ceph/ceph-client.admin.41497.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46408.log.gz 2023-10-26T05:59:44.479 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41497.log.gz 2023-10-26T05:59:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42506.log 2023-10-26T05:59:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45443.log 2023-10-26T05:59:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.40486.log: /var/log/ceph/ceph-client.admin.42506.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40486.log.gz 2023-10-26T05:59:44.480 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42506.log.gz 2023-10-26T05:59:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2023-10-26T05:59:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51735.log 2023-10-26T05:59:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.45443.log: /var/log/ceph/ceph-client.admin.49879.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45443.log.gz 2023-10-26T05:59:44.481 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2023-10-26T05:59:44.481 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42726.log 2023-10-26T05:59:44.481 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42480.log 2023-10-26T05:59:44.481 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.51735.log: /var/log/ceph/ceph-client.admin.42726.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51735.log.gz 2023-10-26T05:59:44.481 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42726.log.gz 2023-10-26T05:59:44.482 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51435.log 2023-10-26T05:59:44.482 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43187.log 2023-10-26T05:59:44.482 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42480.log: /var/log/ceph/ceph-client.admin.51435.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42480.log.gz 2023-10-26T05:59:44.482 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51435.log.gz 2023-10-26T05:59:44.482 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48137.log 2023-10-26T05:59:44.483 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50015.log 2023-10-26T05:59:44.489 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.48137.log: /var/log/ceph/ceph-client.admin.43187.log: 87.1% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43187.log.gz 2023-10-26T05:59:44.490 INFO:teuthology.orchestra.run.smithi071.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48137.log.gz 2023-10-26T05:59:44.490 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2023-10-26T05:59:44.490 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54039.log 2023-10-26T05:59:44.490 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.50015.log: /var/log/ceph/ceph-client.admin.45838.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50015.log.gz 2023-10-26T05:59:44.490 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2023-10-26T05:59:44.491 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53630.log 2023-10-26T05:59:44.491 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38544.log 2023-10-26T05:59:44.491 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.54039.log: /var/log/ceph/ceph-client.admin.53630.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54039.log.gz 2023-10-26T05:59:44.491 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53630.log.gz 2023-10-26T05:59:44.491 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54470.log 2023-10-26T05:59:44.492 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55483.log 2023-10-26T05:59:44.492 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.38544.log: /var/log/ceph/ceph-client.admin.54470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54470.log.gz 2023-10-26T05:59:44.492 INFO:teuthology.orchestra.run.smithi071.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38544.log.gz 2023-10-26T05:59:44.492 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51651.log 2023-10-26T05:59:44.492 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34363.log 2023-10-26T05:59:44.493 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.55483.log: /var/log/ceph/ceph-client.admin.51651.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55483.log.gz 2023-10-26T05:59:44.493 INFO:teuthology.orchestra.run.smithi071.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51651.log.gz 2023-10-26T05:59:44.493 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33021.log 2023-10-26T05:59:44.493 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2023-10-26T05:59:44.493 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34363.log: /var/log/ceph/ceph-client.admin.33021.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33021.log.gz 2023-10-26T05:59:44.494 INFO:teuthology.orchestra.run.smithi071.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34363.log.gz 2023-10-26T05:59:44.494 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44518.log 2023-10-26T05:59:44.494 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2023-10-26T05:59:44.494 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.42193.log: /var/log/ceph/ceph-client.admin.44518.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2023-10-26T05:59:44.494 INFO:teuthology.orchestra.run.smithi071.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44518.log.gz 2023-10-26T05:59:44.495 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51120.log 2023-10-26T05:59:44.495 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34501.log 2023-10-26T05:59:44.495 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.49134.log: /var/log/ceph/ceph-client.admin.51120.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2023-10-26T05:59:44.495 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51120.log.gz 2023-10-26T05:59:44.495 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56203.log 2023-10-26T05:59:44.496 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44797.log 2023-10-26T05:59:44.496 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.34501.log: /var/log/ceph/ceph-client.admin.56203.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34501.log.gz 2023-10-26T05:59:44.496 INFO:teuthology.orchestra.run.smithi071.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56203.log.gz 2023-10-26T05:59:44.496 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2023-10-26T05:59:44.496 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43213.log 2023-10-26T05:59:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.44797.log: /var/log/ceph/ceph-client.admin.52840.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44797.log.gz 2023-10-26T05:59:44.498 INFO:teuthology.orchestra.run.smithi071.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2023-10-26T05:59:44.498 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47116.log 2023-10-26T05:59:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35880.log 2023-10-26T05:59:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.43213.log: /var/log/ceph/ceph-client.admin.47116.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43213.log.gz 2023-10-26T05:59:44.499 INFO:teuthology.orchestra.run.smithi071.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47116.log.gz 2023-10-26T05:59:44.499 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.35880.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35880.log.gz 2023-10-26T05:59:44.530 INFO:teuthology.orchestra.run.smithi071.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-26T05:59:44.548 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-26T05:59:44.552 INFO:teuthology.orchestra.run.smithi071.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-26T05:59:44.627 INFO:teuthology.orchestra.run.smithi071.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-26T05:59:44.677 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-26T05:59:44.705 INFO:teuthology.orchestra.run.smithi071.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-26T05:59:44.706 INFO:teuthology.orchestra.run.smithi071.stderr: 2023-10-26T05:59:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m1.265s 2023-10-26T05:59:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m8.003s 2023-10-26T05:59:44.707 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.428s 2023-10-26T05:59:44.707 INFO:tasks.ceph:Archiving logs... 2023-10-26T05:59:44.708 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/remote/smithi071/log 2023-10-26T05:59:44.709 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T05:59:45.999 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/remote/smithi086/log 2023-10-26T05:59:46.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-26T05:59:46.497 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-26T05:59:46.506 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-26T05:59:46.507 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T05:59:46.510 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-26T05:59:46.585 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-26T05:59:46.593 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-26T05:59:46.594 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T05:59:46.596 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-26T05:59:46.608 INFO:teuthology.orchestra.run.smithi071.stdout: remote refid st t when poll reach delay offset jitter 2023-10-26T05:59:46.608 INFO:teuthology.orchestra.run.smithi071.stdout:============================================================================== 2023-10-26T05:59:46.609 INFO:teuthology.orchestra.run.smithi071.stdout:*hv01.front.sepi 44.4.53.2 2 u 24 64 377 0.100 +2.340 0.777 2023-10-26T05:59:46.609 INFO:teuthology.orchestra.run.smithi071.stdout:+hv02.front.sepi 63.231.80.2 3 u 15 64 377 0.092 +3.828 0.719 2023-10-26T05:59:46.609 INFO:teuthology.orchestra.run.smithi071.stdout:+hv03.front.sepi 74.6.168.72 3 u 18 64 377 0.104 +2.735 0.604 2023-10-26T05:59:46.609 INFO:teuthology.orchestra.run.smithi071.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2023-10-26T05:59:46.611 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2023-10-26T05:59:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2023-10-26T05:59:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:*hv01.front.sepi 44.4.53.2 2 u 14 64 377 0.095 -2.064 2.175 2023-10-26T05:59:46.611 INFO:teuthology.orchestra.run.smithi086.stdout:+hv02.front.sepi 63.231.80.2 3 u 17 64 377 0.109 -2.897 1.614 2023-10-26T05:59:46.612 INFO:teuthology.orchestra.run.smithi086.stdout:+hv03.front.sepi 74.6.168.72 3 u 23 64 377 0.084 -4.705 2.374 2023-10-26T05:59:46.612 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2023-10-26T05:59:46.612 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-26T05:59:46.621 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-26T05:59:46.621 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-26T05:59:46.653 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-26T05:59:46.688 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-26T05:59:46.720 INFO:teuthology.task.internal:Duration was 1400.899462 seconds 2023-10-26T05:59:46.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-26T05:59:46.784 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-26T05:59:46.785 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T05:59:46.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-26T05:59:46.861 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-26T05:59:46.862 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2023-10-26T05:59:46.862 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T05:59:46.872 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2023-10-26T05:59:46.873 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-26T05:59:46.882 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-26T05:59:46.883 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T05:59:46.915 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-26T05:59:46.947 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-26T05:59:46.947 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T05:59:46.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-26T05:59:47.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-26T05:59:47.066 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-26T05:59:47.067 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T05:59:47.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-26T05:59:47.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-26T05:59:47.101 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T05:59:47.123 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-26T05:59:47.131 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2023-10-26T05:59:47.144 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2023-10-26T05:59:47.156 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T05:59:47.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:59:47.190 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-26T05:59:47.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-26T05:59:47.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-26T05:59:47.216 INFO:teuthology.task.internal:Transferring archived files... 2023-10-26T05:59:47.217 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/remote/smithi071 2023-10-26T05:59:47.217 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T05:59:47.251 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445/remote/smithi086 2023-10-26T05:59:47.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-26T05:59:47.271 INFO:teuthology.task.internal:Removing archive directory... 2023-10-26T05:59:47.271 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T05:59:47.291 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-26T05:59:47.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-26T05:59:47.324 INFO:teuthology.task.internal:Not uploading archives. 2023-10-26T05:59:47.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-26T05:59:47.346 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-26T05:59:47.346 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T05:59:47.349 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-26T05:59:47.353 INFO:teuthology.orchestra.run.smithi071.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 05:59 /home/ubuntu/cephtest 2023-10-26T05:59:47.363 INFO:teuthology.orchestra.run.smithi086.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 26 05:59 /home/ubuntu/cephtest 2023-10-26T05:59:47.364 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-26T05:59:47.380 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-26T05:59:47.471 INFO:teuthology.nuke:Checking targets against current locks 2023-10-26T05:59:47.496 DEBUG:teuthology.nuke:shortname: smithi071 2023-10-26T05:59:47.496 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T05:59:47.519 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 05:27:47.988476', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLtb4fp23k+VlYx1uYJYTCE44p699TI4MG/A7y/cOvNDnxwF8mkBhrRrS9zGYanOJNq36Q0kdVZ9kdcE65mBQbo='} 2023-10-26T05:59:47.523 DEBUG:teuthology.nuke:shortname: smithi086 2023-10-26T05:59:47.523 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-26T05:59:47.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-10-26_05:21:22-fs-wip-batrick-testing-20231024.144545-distro-default-smithi/7438445', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-26 05:27:47.990281', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBISjMIFjx/GgWg+LUaxrD5urfkmsGBl6ukeyfx7q03DIKhzwI112qkd5pvKzNU+5gyaKcd+pSCE5dcZ++k7lSQQ='} 2023-10-26T05:59:47.567 INFO:teuthology.orchestra.console.smithi071:Power off 2023-10-26T05:59:47.568 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T05:59:47.595 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2023-10-26T05:59:47.596 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:47.611 INFO:teuthology.orchestra.console.smithi086:Power off 2023-10-26T05:59:47.611 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-26T05:59:47.620 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-10-26T05:59:47.634 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2023-10-26T05:59:47.634 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:47.656 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-26T05:59:51.622 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:51.658 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:51.732 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-10-26T05:59:51.767 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-26T05:59:55.734 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:55.768 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:55.845 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2023-10-26T05:59:55.878 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-10-26T05:59:55.979 INFO:teuthology.orchestra.console.smithi086:Power off completed 2023-10-26T05:59:59.849 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-26T05:59:59.960 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2023-10-26T06:00:00.061 INFO:teuthology.orchestra.console.smithi071:Power off completed 2023-10-26T06:00:00.166 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1400.8994619846344 failure_reason: '"2023-10-26T05:47:08.297699+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 164877.815646 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2023-10-26T06:00:00.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-26T06:00:00.245 INFO:teuthology.run:FAIL